LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

[lvs-users] Connections are "clumped" together on real_server failure

To: "keepalived-users@xxxxxxxxxxxxxxxxxxxxx" <keepalived-users@xxxxxxxxxxxxxxxxxxxxx>, "lvs-users@xxxxxxxxxxxxxxxxxxxxxx" <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: [lvs-users] Connections are "clumped" together on real_server failure
From: Jay Faulkner <jay.faulkner@xxxxxxxxxxxxx>
Date: Thu, 25 Jun 2009 09:23:16 -0500
Hey guys,

We've been noticing some really strange behavior with keepalived/LVS when a 
real_server fails. The behavior we see is that when a r_s fails, all the 
persistent connections on that server fall over to a single server.

For example:

Server 1 - 25000 active conn. - 1500 inactive conn.
Server 2 - 35000 active conn. - 1000 inactive conn.
Server 3 - 20000 active conn. - 500 inactive conn.

Server 2 fails. Instead of seeing all the traffic split between the remaining 
real_servers, we see it all clump over to the same server

Server 1 - 25000 active conn. - 1500 inactive conn.
Server 2 - failed
Server 3 - 55000 active conn. - 1500 inactive conn.

Now, server 3 falls over and dies due to having too many connections on it and 
becoming overloaded.

We've seen this behavior consistently in all of our keepalived installations 
(20 servers or so? Median of about 20 virtual_servers each), and I was curious 
if this is configurable behavior, or even expected.

Thanks!

Jason Faulkner
Linux Engineer, Rackspace Email & Apps
jason.faulkner@xxxxxxxxxxxxx<mailto:jason.faulkner@xxxxxxxxxxxxx>
o: (540) 443-2101 (ex. 505-2101)

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>