LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] Kernel 2.6.35 and 100% S.I. CPU Time

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Kernel 2.6.35 and 100% S.I. CPU Time
Cc: jslvs@xxxxxxxxxxxxxxxxxxxx, Julian Anastasov <ja@xxxxxx>
From: JL <lvs@xxxxxxxx>
Date: Mon, 27 Sep 2010 16:37:18 +0100
I'm going to clarify a few points, as John's summary sometimes
reference things that are specific to our configuration, which may
confuse people.

All these points are actually irrelevant to the problem, but might
confuse those not familiar with our setup.

On 27 September 2010 16:21, John Sullivan <jslvs@xxxxxxxxxxxxxxxxxxxx> wrote:
> Julian Anastasov <ja@xxxxxx> wrote:
>> Appended is a patch that drops traffic
>> to local addresses. Let me know if it changes something in
>> backup server.
>
> I backed out my patch and tried this instead.
>
> It does appear to fix the 100% SI problem, but creates a new
> one. Under my test load (fire 100 simultaneous HTTPS requests using
> wget at the load-balanced address), all the wgets hang, and netstat
> on the backup server shows all 100 connections in this state:
>
> Proto Recv-Q Send-Q Local Address Foreign Address State       PID/Program name
> tcp      113      0 lb-addr:443   ext-host:49914  ESTABLISHED -
>
>
> It looks to me like packets on those connections are now just being
> discarded (even when they shouldn't be).
>
> I also noticed that ipvsadm starts reporting the weight of the backup
> as 0 so it now looks like (on both servers):
They go to zero because the monitor running now fails, and sets the
weight to zero.

>
>  -> localhost:0                  Local   1      0          0
>  -> master:0                     Route   0      0          0
>  -> backup:0                     Local   0      0          0
>
> (The localhost entry seems to appear whenever there are no available
> hosts - I assume this is normal.)
We add a localhost entry when all realservers have failed, as we might
as well do *something*. This is just a last ditch attempt to not drop
packets. It is tied to our monitoring; not a ipvs specific thing.


> Removing the administrative block on the master allows its weight to
> return to 250 (as reported by ipvsadm on both hosts, and the localhost
> entries disappear from ipvsadm, and requests can be successfully
> served by the master), but the backup never recovers and remains at 0
> weight.
"Removing the administrative block" here means to allow our checking
system to change the weight up to something like 250.

>
> John
> --
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>



-- 
Jarrod Lowe

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>