LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] LVS and Broadcom bug

To: Chris Chen <chchen@xxxxxxx>, "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>, Jean-Sébastien Frerot <jean-sebastien.frerot@xxxxxxxxxxxx>
Subject: Re: [lvs-users] LVS and Broadcom bug
From: Simon Horman <horms@xxxxxxxxxxxx>
Date: Wed, 27 Oct 2010 06:40:59 +0900
On Wed, Oct 27, 2010 at 06:33:54AM +0900, Simon Horman wrote:
> On Tue, Oct 26, 2010 at 09:52:00AM -0700, Chris Chen wrote:
> > Quoting Jean-Sébastien Frerot <jean-sebastien.frerot@xxxxxxxxxxxx>:
> > 
> > >>         Hello,
> > >>
> > >> On Mon, 30 Nov 2009, Simon Horman wrote:
> > >>
> > >> >/ >     The problem exists because IPVS does not/
> > >> >/ > disable LRO, it must be done under RTNL and IPVS never runs/
> > >> >/ > in this context. And LRO is not supported for forwarding:/
> > >> >/ > /
> > >> >/ > http://marc.info/?l=linux-netdev&m=121389887114416&w=2  
> > >> <http://marc.info/?l=linux-netdev&m=121389887114416&w=2>/
> > >> >/ > /
> > >> >/ >     IPVS does not call ip_forward for DR method, that/
> > >> >/ > is why you do not need forwarding and the LRO warning/
> > >> >/ > does not occur before hitting the GSO code. ip_forward/
> > >> >/ > just drops LRO packets:/
> > >> >/ > /
> > >> >/ >         if (skb_warn_if_lro(skb))/
> > >> >/ >                 goto drop;/
> > >> >/ /
> > >> >/ Hi Julian,/
> > >> >/ /
> > >> >/ do you have any thoughts on how the code might be improved/
> > >> >/ to handle this case a bit better?/
> > >> >/ /
> > >> >/ Perhaps something along the lines of the/
> > >> >/ code for LRO in ip_forward?/
> > >>
> > >>         If you want to disable LRO in IPVS
> > >> net/ipv4/devinet.c:inet_forward_change() is an example what
> > >> should be done in process context if you want to disable
> > >> LRO for all existing devices. Then call skb_warn_if_lro
> > >> near or in IP_VS_XMIT and also before calling ip_local_out().
> > >> May be LRO can be disabled when the first virtual or
> > >> may be real service is added to allow LRO to work if IPVS
> > >> is just compiled.
> > >>
> > >> Regards
> > >>
> > >> --
> > >> Julian Anastasov <ja@xxxxxx>
> > >>
> > > Hi,
> > >   Do you guys know if there is any plan to fix this in ipvs soon ? We
> > > have this exact problem when using ipvs and 2 different network cards
> > > (intel and broadcom).
> 
> Sorry, no I don't believe that there is a fix available.
> I will try and rectify that situation. In the mean time
> I believe that a work-around is to disable LRO on any
> interface that receives packets for fowarding (or LVS)
> using ethtool.

Correction, the work-around for the problem that I have seen
is to disable GRO.

> > Could this be breaking SSL over LVS-DR? I've been seeing a problem  
> > where SSL handshakes fail intermittently with certain clients (Windows  
> > 7, particularly), and using LVS-NAT seems to fix it.
> 
> The most recent manifestation of this problem that I have observed is that
> the linux-director will request fragmentation of packets that are greater
> than MTU size which LRO has actually assembled from packets which are MTU
> length or less.
> 
> That manifestation can be observed as ICMP need to frag messages on the wire.
> The kernel in question is 2.5.35. I believe the behaviour should
> be the same in newer kernels. I am less sure about older kernels.
> 
> I'm not sure if there are other manifestations of this problem.
> 

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>