|
On Tue, Nov 28, 2006 at 08:37:28AM -0500, RU Admin wrote:
[snip]
> >>When running "ipvsadm -lcn", I can
> >>see connections with the CLOSE state going from 00:59 to 00:01, and
> >>then magically going back to 00:59 again for no reason. The same
> >>holds true for ESTABLISHED connections, I see them go from 29:59 to
> >>00:01 and then back to 29:59, and I know for a fact that the
> >>connection from the client has ended.
> >
> >I seem to recall a bug relating to connection entries having
> >the behaviour you describe above due to a race in reference counting.
> >Which version of the kernel do you have? Is there any chance of updating
> >it to something like 2.6.18?
>
> I'm using a stock Debian Sarge kernel (2.6.8-2-686-smp), I can
> definitely build the latest kernel, and if you feel that it will help
> then I'll do that. It's always risky making a major kernel change on
> a production machine, which is why I wanted to hold off from making
> that change until someone else familiar with IPVS, felt that it might
> help.
I think that it would be worth trying. Can you reproduce the problem
on a non-production machine?
[snip]
> >I am wondering if the problem is that for some reason the
> >linux-directors are not seeing the part of the close sequence
> >that is sent by the end-user (it won't see the portion sent by
> >the real-servers). Supposing for a minute that this is the case,
> >it would explain the strange numbers, and those strange numbers
> >will be effecting how wlc allocates connections.
>
> But shouldn't IPVS timeout? I thought that was the purpose of the
> timeouts...
> So that when the director doesn't see a close event after a specified period
> of
> time, it simply times out.
I actually think my close theory is wrong and that as you point out the
problem is timeouts. I think that you are correct in thinking that they
should time out. So that seems to leave us with two main possiblilities
1) there is a bug (which may have already been fixed) or 2) we are
reading the data wrong.
[snip]
> >How exactly did you deal with ARP, there are several methods.
>
> On the real servers, I'm first bringing up the dummy0 interface with the VIP,
> then I use "sysctl" and set the following:
> net.ipv4.conf.dummy0.rp_filter=0
> net.ipv4.conf.dummy0.arp_ignore=1
> net.ipv4.conf.dummy0.arp_announce=2
> Then I bring up eth0 with the real server's regular IP address, and with
> "sysctl", I set the following (includes a repeat of the above options):
> net.ipv4.conf.default.rp_filter=0
> net.ipv4.conf.all.rp_filter=0
> net.ipv4.conf.lo.rp_filter=0
> net.ipv4.conf.dummy0.rp_filter=0
> net.ipv4.conf.eth0.rp_filter=0
>
> net.ipv4.conf.default.arp_ignore=1
> net.ipv4.conf.all.arp_ignore=1
> net.ipv4.conf.lo.arp_ignore=1
> net.ipv4.conf.dummy0.arp_ignore=1
> net.ipv4.conf.eth0.arp_ignore=1
>
> net.ipv4.conf.default.arp_announce=2
> net.ipv4.conf.all.arp_announce=2
> net.ipv4.conf.lo.arp_announce=2
> net.ipv4.conf.dummy0.arp_announce=2
> net.ipv4.conf.eth0.arp_announce=2
>
> The ARP problem was the one thing that kept me from moving to LVS-DR
> for a long time. I finally started playing with all of the
> net.ipv4.conf options and bringing up the interfaces in a specific
> order, and finally stumbled across a method that actually worked. I'm
> sure some of the above options don't need to be set, but it finally
> works, and I'm a little afraid to touch it.
What you have above is the prefered method these days.
You shouldn't need to bother with lo and dummy0 as these are non-arping
interfaces (right?). Though setting them is harmless.
In any case, I agree with your analysis that ARP does not seem to be
a problem in your setup, as the connections are being forwarded by
the linux-director.
> I'm going to try and build the latest 2.6.18 now, and hopefully
> sometime later this week I can install the new kernel and reboot our
> director. Unfortunately I've never been able to get keepalived to
> handle a MASTER/SLAVE director properly, so I only have one director
> in front of the real servers, so if I make a mistake, our main
> university email server will be down.
ew. Good luck :)
--
Horms
H: http://www.vergenet.net/~horms/
W: http://www.valinux.co.jp/en/
|
