LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Performance issues with LVS-NAT

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Performance issues with LVS-NAT
From: Horms <horms@xxxxxxxxxxxx>
Date: Tue, 22 Feb 2005 18:54:48 +0900
On Fri, Feb 11, 2005 at 05:33:16PM +0000, Graeme Fowler wrote:
> Hi
> 
> On Fri, 11 Feb 2005, Joseph Mack wrote:
> > I think the answer is no. It's nothing to do with LVS; 
> > packet marking and the loss of the mark are all part
> > of the routing code. All LVS is doing is stepping in
> > and changing the routing rules a little bit
> 
> After a long day spent tracing packets through the LVS and netfilter trail
> whilst trying to do cleverness with policy routing using the iproute2 package,
> I can condense quite a lot of reading (and trial, mainly followed by error!)
> down as follows:
> 
> 1. FastNAT, as provided by the iproute2 package, is incompatible with the 
> netfilter conntrack module. As most LVS-NAT systems are also doing 
> masquerading or SNAT for outbound connections from the realservers, the 
> conntrack module is loaded automagically - thus FastNAT via policy routing 
> simply won't work.
> 
> 2. Try as you might to do SNAT, it has to be done in the 'nat POSTROUTING' 
> chain - and the packets being processed via LVS don't traverse this chain, 
> because they're hooked right out of the nat POSTROUTING table and are 
> processed by ip_vs_port_routing instead, which then plonks them back on the 
> wire magically without further processing. So SNAT won't work either.
> 
> 3. Using fwmarks seems inconclusive, because ultimately (in my case at least) 
> I want to SNAT the packets in some way, and point (2) above precludes that.
> 
> 
> Saying all this, I now have a far more detailed understanding of how a packet 
> traverses LVS and what happens to it in the meantime. I'm not sure I wanted 
> this understanding when I got up this morning, but I do now so I'll just have 
> to live with it :)
> 
> Conclusions: mixing policy routing and LVS sounds like a great idea, and 
> probably is if you're using LVS-DR or LVS-TUN. Just with LVS-NAT, it's a 
> no-go 
> (for me at the moment, anyway).
> 
> Back to the drawing board on this one...

Time to start hacking by the looks of it.

With regards to SNAT and LVS-NAT, unless I am completely
off my rocker, check_for_ip_vs_out() is called for outgoing
packets to fource them to skip any SNAT that might take place.
You could try disabling that, but you probably have to look
at making sure the SNAT process is reversed as well.

As for the fwmark being cleared, I am pretty sure that shouldn't happen,
but if it is, it is probably happening as a side effect of
something in ip_vs_nat_xmit().

-- 
Horms

<Prev in Thread] Current Thread [Next in Thread>