|
On Fri, 7 Jan 2005, Horms wrote:
> On Thu, Jan 06, 2005 at 03:17:11PM +0200, Johan van den Berg wrote:
<snip>
> > Upon setting /proc/sys/net/ipv4/vs/debug_level to 7, I noticed that
> > every now and again, I get a "lookup/out" entry that states that a
> > connection from the virtual ip on port 80 to a client port on the client
> > IP was "not hit". This seems to confirm that the original SYN from the
> > client to port 80 on the virtual ip is not stored in the IPVS connection
> > table, and therefore the reply to the client IP is not handled by IPVS,
> > but rather iptables, which causes havoc.
>
> Yes, that does seem to confirm that the entry is not in the IPVS
> connection table, and thys falls back to iptables.
Bingo. I sent a few emails in October (LVS-NAT: intermittent problem with
responses from realservers) about this exact problem but it appears that Johan
has explained the problem in slightly clearer terms than I did :)
> For clafiication, this is most likely occuring in ip_vs_out()
> which is a netfilter hook that reverses IPVS-NATed packets,
> and thus is specific to NAT, though I don't think your problem
> neccessarily is.
I haven't used TUN or DR in this scenario, am using NAT for a webserver farm.
What I saw, in very simple terms, was that every now and again traffic leaving
the cluster would miss the LVS and be NATted to the "default" address of the
load balancer - this address being different from the VIP. I worked around it
by using SNAT rules to force all packets from specific sets of hosts to be
coming from their VIP. If any packets fall through LVS and give a "not hit",
they hit the netfilter rule instead so the connection doesn't break. Whilst
clearly not that desirable, it does work so I left well alone and stopped
thinking about it - until Johan posted the same problem.
Unfortunately I'm not really able to do a huge amount of debugging as the
system is in production, but I should be able to crank up
/proc/sys/net/ipv4/vs/debug_level to see what we get if necessary.
Graeme
|
