LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Director resets existing tcp connection

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Director resets existing tcp connection
From: Dominik Klein <dk@xxxxxxxxxxxxxxxx>
Date: Mon, 15 May 2006 11:08:20 +0200
G'mornin.

Well, I found one. But it is not an LVS bug. Instead I think this happens on purpose (at least in my case).

So let's just say we have a simple setup:
client -> switch -> director -> switch -> realserver

The client establishes a connection, sends some data, whatever. Then it does nothing for say 10 minutes. After that it tries to reuse the still established connection and it works just fine. Then it does nothing for say 16 minutes and tries again to use the still established connection. In the meantime, the default timeout for the connection table (default 15 minutes) runs out and so this connection is not valid on the director. So the director replies with RST on the PSH packet from the client and the connection breaks for the client.

The realserver does not know anything about the reset on the director, so it still considers the connection established.

That's pretty much it. The client can simply open a new connection, but the old one will still be considered established on the realserver. That's what made my MySQL server hit the max_connection limit and reject any new clients. I will try to set the timeout higher, as it can easily happen that my clients do nothing for a few hours at night, which will - sooner or later - hit the max_connection limit again.

One could also think about tweaking tcp settings on the realserver. But I am not too familiar with that - yet (still learning :) ).

Hope that helps.

Regards
Dominik

Joseph Mack NA3T schrieb:
On Fri, 12 May 2006, Dominik Klein wrote:

I do not use persistence or FTP. Simple NAT setup with as realserver MySQL.

The client sends a PSH packet which is immediately RST by the director. So the realserver still "thinks" the connection is ESTABLISHED. Therefore the max_connection_limit in MySQL is hit within a few hours so that no new clients can connect.

can you tell us what to do to reproduce this?

So can anyone confirm that 2.6.15 fixes this problem?

I don't think it has been fixed, so this is unlikely.

Here's a link to the thread I am referring to:
http://archive.linuxvirtualserver.org/html/lvs-users/2005-12/msg00213.html

this is a long outstanding bug. The problem is that noone has figured out a way of reproducing the problem on demand to fix it.

Joe



<Prev in Thread] Current Thread [Next in Thread>