LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] LVS and Broadcom bug

To: Chris Chen <chchen@xxxxxxx>
Subject: Re: [lvs-users] LVS and Broadcom bug
Cc: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>, Jean-Sébastien Frerot <jean-sebastien.frerot@xxxxxxxxxxxx>
From: Simon Horman <horms@xxxxxxxxxxxx>
Date: Wed, 27 Oct 2010 06:33:54 +0900
On Tue, Oct 26, 2010 at 09:52:00AM -0700, Chris Chen wrote:
> Quoting Jean-Sébastien Frerot <jean-sebastien.frerot@xxxxxxxxxxxx>:
> 
> >>         Hello,
> >>
> >> On Mon, 30 Nov 2009, Simon Horman wrote:
> >>
> >> >/ >     The problem exists because IPVS does not/
> >> >/ > disable LRO, it must be done under RTNL and IPVS never runs/
> >> >/ > in this context. And LRO is not supported for forwarding:/
> >> >/ > /
> >> >/ > http://marc.info/?l=linux-netdev&m=121389887114416&w=2  
> >> <http://marc.info/?l=linux-netdev&m=121389887114416&w=2>/
> >> >/ > /
> >> >/ >     IPVS does not call ip_forward for DR method, that/
> >> >/ > is why you do not need forwarding and the LRO warning/
> >> >/ > does not occur before hitting the GSO code. ip_forward/
> >> >/ > just drops LRO packets:/
> >> >/ > /
> >> >/ >         if (skb_warn_if_lro(skb))/
> >> >/ >                 goto drop;/
> >> >/ /
> >> >/ Hi Julian,/
> >> >/ /
> >> >/ do you have any thoughts on how the code might be improved/
> >> >/ to handle this case a bit better?/
> >> >/ /
> >> >/ Perhaps something along the lines of the/
> >> >/ code for LRO in ip_forward?/
> >>
> >>         If you want to disable LRO in IPVS
> >> net/ipv4/devinet.c:inet_forward_change() is an example what
> >> should be done in process context if you want to disable
> >> LRO for all existing devices. Then call skb_warn_if_lro
> >> near or in IP_VS_XMIT and also before calling ip_local_out().
> >> May be LRO can be disabled when the first virtual or
> >> may be real service is added to allow LRO to work if IPVS
> >> is just compiled.
> >>
> >> Regards
> >>
> >> --
> >> Julian Anastasov <ja@xxxxxx>
> >>
> > Hi,
> >   Do you guys know if there is any plan to fix this in ipvs soon ? We
> > have this exact problem when using ipvs and 2 different network cards
> > (intel and broadcom).

Sorry, no I don't believe that there is a fix available.
I will try and rectify that situation. In the mean time
I believe that a work-around is to disable LRO on any
interface that receives packets for fowarding (or LVS)
using ethtool.

> Could this be breaking SSL over LVS-DR? I've been seeing a problem  
> where SSL handshakes fail intermittently with certain clients (Windows  
> 7, particularly), and using LVS-NAT seems to fix it.

The most recent manifestation of this problem that I have observed is that
the linux-director will request fragmentation of packets that are greater
than MTU size which LRO has actually assembled from packets which are MTU
length or less.

That manifestation can be observed as ICMP need to frag messages on the wire.
The kernel in question is 2.5.35. I believe the behaviour should
be the same in newer kernels. I am less sure about older kernels.

I'm not sure if there are other manifestations of this problem.


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>