|
On Mon, 27 Nov 2006, Olle Östlund wrote:
Remote logon via ssh does not work (hangs or will get a "connection closed by
remote server" type of message). Logging in via the console does not work
(hangs after password has been typed in), using su will produce the same
result.
I get this when I don't have enough entries in
/etc/securetty
so it sounds like a resource problem but it's hard to think
of a resource that would be short in a realserver but not an
otherwise identical standalone server.
nothing strange with `ulimit -a` ?
Nothing monotonically
increasing as far as we have seen, no.
:-(
We are running a "weighted round robin" load balancer algoritm, and the
director will set weight 0 (= no traffic) on a backend once it has frozen
(not responding to the directors status requests). Then the number of active
connections sloooowly drops.
They shouldn't slowly drop.
All connections should be gone in the time the clients are
connected + FIN_WAIT. How long does a client hold a tomcat
connection? seconds, minutes, hours?
I haven't setup a tomcat server, but I assume the connection
time is comparable to connecting to a regular html/http
server, ie get your hit(s) and you're gone (seconds).
Where are you measuring the number of connections? with
ipvsadm on the director or with netstat on the realserver?
Netstat typically reports a normal number of connections (~ 100) in various
states.
"Runnig out of ports" sounds interesting. It sure looks like we are running
out of something. How would one go about detecting if we are running out of
ports?
Remeber, if looks like we are running out of something which causes
applications to wait instead of bailing out with an error message. Would
running out of ports cause applications to wait?
actually you aren't going to run out of ports, you've got
the server end, which is going to have multiple processes
listening on port 80. The client can (in principle) run out
of ports.
do the realservers talk to each other (eg have a common disk
system)?
Well, no disk- or file-sharing. The Tomcats are communicating HTTP
session-replication data between backends. Our HTTP-sessions contain very
little data, so the traffic should not be that big.
it's the only thing different to operating as a standalone
server that you've mentioned.
Anyone out there having a good idea where to look for clues to what may
be wrong?
no, still just going through obvious checklists here
Joe
--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!
|
