On Thu, 17 May 2001, Mark Miller wrote:
> Okay,
>
> After lots of time spent trying to narrow this down to a root cause I've hit
> a wall. I've included these previous posts for reference. Here's the
> phenomenon I'm seeing:
>
> I have two LD's in HA configuration using heartbeatd. On either one of them
> my ipvs configuration is working fine to load balance two webservers behind
> them using LVS-NAT. My problem is that when I try to cause a failover by
> either killing heartbeatd on one or shutting one down (simulating failure) I
> get very strange results. After the failover happens and heartbeatd brings
> up the shared resources I see some http connections going through fine while
> some show up in ipvsadm as:
>
> IPVS connection entries
> pro expire state source virtual destination
> TCP 00:25.52 SYN_RECV 10.10.9.63:2897 10.10.21.68:80 10.200.200.1:80
> TCP 00:45.92 SYN_RECV 10.10.9.63:2898 10.10.21.68:80 10.200.200.1:80
> TCP 00:42.92 SYN_RECV 10.10.9.63:2900 10.10.21.68:80 10.200.200.1:80
> TCP 00:51.92 SYN_RECV 10.10.9.63:2902 10.10.21.68:80 10.200.200.1:80
> TCP 00:08.92 SYN_RECV 10.10.9.63:2891 10.10.21.68:80 10.200.200.1:80
> TCP 00:52.72 SYN_RECV 10.10.9.63:2895 10.10.21.68:80 10.200.200.1:80
> TCP 00:30.72 SYN_RECV 10.10.9.63:2894 10.10.21.68:80 10.200.200.1:80
Looks like it's heartbeat to blame. It's arhitecture (more complicated than
necessary) doesn't imspire too much confidence to me.
Did you make sure that both the VIP and the IP used for connections with the
real servers are taken by heartbeat ? Are you sure that after they are taken,
the ARP advertisment is sent on both interfaces ? If I remember well heartbeat
only works on one interface (I might be wrong here....).
>
> Now, I've verified that after failover the client is getting the correct arp
> and IP address for the new VIP. After the failover I'd say 1 in 50 http
> attempts works...the rest end up in this SYN_RECV state.
Generally ,it's not the client that gets the MAC of the director, it's usually
a router, and not always your router.
> So my questions are:
> What does a SYN_RECV mean?
> What might it indicated is the problem?
no response from the realserver is seen on the director.
> Does this sound like anything anyone else out there has run into?
> I've pretty much ruled out arp caching on the client and RS side as the
> problem by manually checking them all...could it be arp problems at some
> other level?
>
> Needless to say, I'm beginning to lose faith in ipvs as a solution at this
> point. I've read all the docs, I've followed advice from this list and
> searched the archives for similar issues and still am experiencing things
> that I have seen no reference to anywhere. I'd REALLY like to make this
> configuration work.
IPVS works as it should. Additional software like heartbeat break its
functionality.
Radu-Adrian Feurdean
mailto: raf@xxxxxxxx
----------------------------------------------------------------------------
The light at the end of the tunnel is the headlight of an approaching train.
|