Re: [lvs-users] can somebody explain how gro (generic receive offload) b

To: " users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] can somebody explain how gro (generic receive offload) bothers lvs? (kernel 2.6.32)
Cc: Ryan O'Hara <rohara@xxxxxxxxxx>, Julian Anastasov <ja@xxxxxx>, Zhongkang Lao <laozhongkang@xxxxxxxxx>, Thomas Bätzler <t.baetzler@xxxxxxxxxx>, Florian Crouzat <gentoo@xxxxxxxxxxxxxxxxxx>
From: Jesper Dangaard Brouer <brouer@xxxxxxxxxx>
Date: Fri, 14 Dec 2012 17:38:46 +0100
On Wed, 2012-12-12 at 10:05 +0100, Florian Crouzat wrote:
> Le 11/12/2012 22:00, Thomas Bätzler a écrit :
> > 
> > I don't have any answers, but I can confirm the problem (and the
> > workaround) on a box running Debian "Squeeze" with 2.6.32 directing
> > traffic from a VIP bound to lo via NAT to an active/backup bonding
> > interface on the backend. With "gro on" for the bonded interfaces to the
> > backend real servers, throughput was ~64K/s. With "gro off" it was
> > ~50MB/s. It's not a general networking problem since a direct connection
> > to the real server via a DNAT port forward was not impacted.
> >
> >>From what I gleaned from a cursory look with tcpdump it seems that the
> > problem is caused by incorrectly calculated tcp checksums.
> >
> > Anybody interested in pursuing this further?

As Julian notes, this have already been fixed in upstream kernels.

And I have recently (Aug 2012) also fixed this for IPv6.

> Yes, I can relate.
> I had the same conclusions, incorrectly calculated TCP checksums leading 
> to packet drops by the next firewall (in my architecture).
> Also fixed by disabling gro.

Its not related to incorrect checksums, but (as you noticed) the GRO
(Generic Receive Offload) feature.  (Adjusted via ethtool -K ethX)

> CentOS 6.2, ipvsadm-1.25-9.el6.x86_64, kernel 2.6.32-220.23.1.el6.x86_64

FYI:  I have already backported support (for GRO) for the upcoming
RHEL6.4 and RHEL5.9 kernels. (Its also in zstream for 5.8 in

To be exact, this fixed in kernel releases (and beyond):
 RHEL6  kernel-2.6.32-328.el6
 RHEL5  kernel-2.6.18-340.el5

If someone needs this before its officially part of a RHEL release, I
think I'm allowed to give you the kernel with this fix as a test kernel.
Just let me now directly.

Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of

Please read the documentation before posting - it's available at: mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to
<Prev in Thread] Current Thread [Next in Thread>