LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] can somebody explain how gro (generic receive offload) b

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] can somebody explain how gro (generic receive offload) bothers lvs? (kernel 2.6.32)
Cc: Ryan O'Hara <rohara@xxxxxxxxxx>, Julian Anastasov <ja@xxxxxx>, Zhongkang Lao <laozhongkang@xxxxxxxxx>, Thomas Bätzler <t.baetzler@xxxxxxxxxx>, Florian Crouzat <gentoo@xxxxxxxxxxxxxxxxxx>
From: Jesper Dangaard Brouer <brouer@xxxxxxxxxx>
Date: Fri, 14 Dec 2012 17:38:46 +0100
On Wed, 2012-12-12 at 10:05 +0100, Florian Crouzat wrote:
> Le 11/12/2012 22:00, Thomas Bätzler a écrit :
[...]
> > 
> > I don't have any answers, but I can confirm the problem (and the
> > workaround) on a box running Debian "Squeeze" with 2.6.32 directing
> > traffic from a VIP bound to lo via NAT to an active/backup bonding
> > interface on the backend. With "gro on" for the bonded interfaces to the
> > backend real servers, throughput was ~64K/s. With "gro off" it was
> > ~50MB/s. It's not a general networking problem since a direct connection
> > to the real server via a DNAT port forward was not impacted.
> >
> >>From what I gleaned from a cursory look with tcpdump it seems that the
> > problem is caused by incorrectly calculated tcp checksums.
> >
> > Anybody interested in pursuing this further?

As Julian notes, this have already been fixed in upstream kernels.
(http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=8f1b03a4c18e8f3f0801447b62330faa8ed3bb37)

And I have recently (Aug 2012) also fixed this for IPv6.
(http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=590e3f79a21edd2e9857ac3ced25ba6b2a491ef8)


> Yes, I can relate.
> I had the same conclusions, incorrectly calculated TCP checksums leading 
> to packet drops by the next firewall (in my architecture).
> Also fixed by disabling gro.

Its not related to incorrect checksums, but (as you noticed) the GRO
(Generic Receive Offload) feature.  (Adjusted via ethtool -K ethX)


> CentOS 6.2, ipvsadm-1.25-9.el6.x86_64, kernel 2.6.32-220.23.1.el6.x86_64

FYI:  I have already backported support (for GRO) for the upcoming
RHEL6.4 and RHEL5.9 kernels. (Its also in zstream for 5.8 in
kernel-2.6.18-308.19.1.el5)

To be exact, this fixed in kernel releases (and beyond):
 RHEL6  kernel-2.6.32-328.el6
 RHEL5  kernel-2.6.18-340.el5

If someone needs this before its officially part of a RHEL release, I
think I'm allowed to give you the kernel with this fix as a test kernel.
Just let me now directly.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>