LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Our LVS/DR backends freezes

To: Olle Ö?stlund <olle@xxxxxxxxxxx>
Subject: Re: Our LVS/DR backends freezes
Cc: Horms <horms@xxxxxxxxxxxx>
Cc: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
From: Joseph Mack NA3T <jmack@xxxxxxxx>
Date: Thu, 30 Nov 2006 14:50:29 -0800 (PST)
On Thu, 30 Nov 2006, Olle Ö~Vstlund wrote:

this is not right. It should be similar to the drop
off for the realserver you show below.

Explain why it should be the same?

the director is using the same (or close) timeouts as the native tcpip stack. It only sees the packets from the client and has to infer that the realserver has closed the connection. So the director's view of the realservers connection table will lag. However it won't be more than two minutes behind the realserver.

Why do you have ESTABLISHED connections after 23 mins?
Shouldn't they all have reconnected to another realserver?
So both the ESTABLISHED and FIN_WAIT counters are wrong
on the director?

We are running the heartbeat/ldirectord LVS shipped with SLES9. In this
setup you don't setup/use ipvsadm directly, but via ldirectord. We have
the ldirectord "quiescent"-attribute is set to true (it's default).

lets get rid of the heartbeat layer, which could be confounding the problem. In my other e-mail I suggested a single director.

According to the text above it will produce "server affinity" if "the
real server has persistent connections". If this is the same
"persistense" as in the ldirectord-config (and ipvsadm-config) then we
do not have it set. Nothing is said about it's effects when you do not
have "persistence" set. Is this what we are seeing in our system?

you look like you have persistence of at least 23 minutes.

Ok. But if we have "quiescent" set to true and ipvs-persistence set,
then the director would allow new connections from existing clients be
sent to the closed realserver, wouldn't it?

yes

each new connection request will come from the next port up
in the list, ie if the last connection was from FromIP:3001,
the next will be from FromIP:3001.

Ok. So if we have a system as ours (quiescent = true, no ipvs
persistense) should these connections be alloved to be sent to a closed
(weight 0) realserver? Your final comment suggests it shouln't, right?

correct, it should not.


Instead I tried using
FromIP as a sole identifier of a connection. I ran the script again and
it still suggested that "new connections" were popping after the
realserver had been closed.

not right.

Is this "not right" as in "malfunctioning system"?

sorry, ambiguous statement

"malfunctioning"

We are using 2.6.5-7.244-smp at the director and 2.6.5-7.97-bigsmp at
the realserver. We are trying to use the latest kernel which
HP-recommends for the Proliant (we want the HP-management drivers to
function properly). The realserver is lagging behind somewhat as you can
see.

Our option is to go on the SLES10-train. It has not been out for very
long though. Are you familiar with Suse Enterprise and can recommend
something from the LVS point-of-view?

I don't use any of the distributions myself. I just get the software from the source. I know you aren't allowed to do this in business where you need some certified distribution. However you have a kernel that's two years old. If there'd been anything wrong with the ipvs code there, we'd know.

Can you put on a new kernel from ftp.kernel.org for a test? The hotplugging and usb etc are quite different, but you probably aren't using those for a director. You should be able to use your standard init scripts enough to get a director working.

Joe

--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!




<Prev in Thread] Current Thread [Next in Thread>