LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Again: LVS/Apache cluster freezes from time to time

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Again: LVS/Apache cluster freezes from time to time
From: Joseph Mack <mack.joseph@xxxxxxx>
Date: Tue, 04 Nov 2003 13:22:37 -0500
Jan Abraham wrote:
> 

> So I did some more diagnostics and tried to look at the netstat output
> during the freeze. I was a bit confused to see the number of ESTABLISHED
> connections exploding from about 10 (normal operation) to 200 and more
> during "freeze". So the server DOES accept connections, but Apache is
> not able to process it for some reason.

how do you know Apache isn't handling it? (I can't tell whether it is or isn't,
I'm just asking). If apache had handled the request, you wouldn't be able
to tell from ESTABLISHED. The ESTABLISHED state will stay there till the client
does a close or apache closes it, but apache has a delayed close in case
the client makes another request. So the connection could be in ESTABLISHED
with the request fulfilled a while ago (about 2mins I think). 

Has php/apache been configured to allow enough memory, processes.... 

> Next result: I always asumed that something stopped me from starting new
> processes during the freeze. This is wrong. Something stopps tasks that
> accesses the /proc filesystem for process information.
> I can run "netstat -n", but I can't "netstat -n --program" when freezed.
> 
> I can do a "ls /proc/", but I can't do "ls /proc/<id>/" (<id> is the pid
> of an apache thread)

I have no idea. 

Do you have any idea why this doesn't happen if the machine is outside
LVS?

how about trying different schedulers (lc, rr) or a different forwarder
(LVS-NAT, LVS-DR). (I'm just shooting in the dark here, trying to collect
more data).

Joe

-- 
Joseph Mack PhD, High Performance Computing & Scientific Visualization
SAIC, Supporting the EPA Research Triangle Park, NC 919-541-0007
Federal Contact - John B. Smith 919-541-1087 - smith.johnb@xxxxxxx
<Prev in Thread] Current Thread [Next in Thread>