Re: [lvs-users] Large clusters and slow realserver checking

To: users mailing list. <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Large clusters and slow realserver checking
From: Jay Faulkner <jay.faulkner@xxxxxxxxxxxxx>
Date: Thu, 24 Sep 2009 10:08:22 -0500
> -----Original Message-----
> From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:lvs-users-
> bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Anthony Sturchio
> Sent: Thursday, September 24, 2009 9:21 AM
> To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Subject: [lvs-users] Large clusters and slow realserver checking
> My company has a large number of secure web applications that we have
> running on an LVS cluster.  There are about 200 real IP's (for 200
> different
> domains / SSL certificates) and we have 5 different realservers in the
> mix.
> Since we are serving http and https (80 and 443) this works out to be
> 2,000
> realserver entries that ldirectord has to go through.  Obviously this
> takes
> some time.  We have seen it take up to 15 minutes to expire a downed
> node,
> or to reinstate a realserver once we bring it back up, of course
> depending
> on how far along the list ldirectord is.  Using the forking option is
> not
> possible since spawning a high number of processes simultaneously
> brings the
> load balancer to its knees.


We have a similar use case, and for us, we don't use ldirectord. We run 
keepalived with the threaded healthcheckers patch. Basically, this allows us to 
define checks outside of virtual_server definitions -- so we say "server1 
healthcheck http://blah"; "server2 healthcheck http://blah2"; and so on -- then 
we point each V_S to the already defined healthcheck.

Using this setup, we're transferring 1Gbit/s peak with about 2000 V_S and about 
5 backend servers hosting all the IP addresses (10k healthchecks if done 

If you're interested in this, look in the keepalived list archives and pull the 
threaded healthcheck patch I posted. There are some good docs inside that, and 
I'm willing to help if you need a hand figuring out the config.

Jason Faulkner
Linux Engineer, Rackspace Email & Apps

Please read the documentation before posting - it's available at: mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to

<Prev in Thread] Current Thread [Next in Thread>