LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

LVS stops balancing after a while

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: LVS stops balancing after a while
From: Jan Jensen <jan.jensen@xxxxxxxxxxxxxx>
Date: Thu, 02 Feb 2006 00:31:52 +0100
Hello,

even though I have red a lot mails from the list, I haven't found a case like our.
We have some trouble with our LVS-Cluster. Since a while we
evaluate a 6 Node-Cluster (3 Realserver/1Devel/2Directors) as a Mail Gateway.

The Setup is the following:

The 2 Director (failover) running direct routing.
Setup is done by  ldirectord(1.77.2.41) which is
invoked by heartbeat.
Addionally there are two drbd-devices for some
configfiles and data (no ldirectord data/config in there).

On the realserver is some MTA(postfix,amavisd-new etc.) running.

When we starting the setup everything is running fine, the heartbeat comes up and starts the ldirectord with the following config:

-------------------------------------------------
# Global Directives
checktimeout=10
checkinterval=2
autoreload=yes
quiescent=yes

#Sample configuration for an smtp virtual service.
#Fallback setting overides global
virtual=10.10.x.60:25
        real=10.10.x.63:25 gate 100
        real=10.10.x.64:25 gate 100
        real=10.10.x.65:25 gate 100
        service=smtp
        scheduler=lc
        checkport=25
        checktype=connect
        protocol=tcp
-------------------------------------------------

so ipvsadm -Ln gives us the following output:

-------------------------------------------------
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.10.x.60:25 lc
  -> 10.10.x.63:25               Route   100    0          0
  -> 10.10.x.64:25               Route   100    0          0
  -> 10.10.x.65:25               Route   100    0          0
-------------------------------------------------

Everything works fine and the Director balances the incoming connections to the 3 realservers. But after a while (sorry dont know a specific time) the LB stops sceduling and the incoming connections will send to the last choosen realserver. Really strange is that after this happens, you can unload the modules (ip_vs/ip_vs_lc) and after the ip_vs module got reloaded the traffic continues send to the last realserver. For testing i've writen the output of ipvsadm -Ln( and c) with timestamp to a file. The last entry was this:

IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
  -> RemoteAddress:Port
TCP  10.10.x.60:25             17617  1872891        0    2541M        0
  -> 10.10.x.63:25              5927   614901        0  832431K        0
  -> 10.10.x.64:25              5964   663196        0  902503K        0
  -> 10.10.x.65:25              5726   594794        0  806841K        0

(No outgoing Pakets, because the mails are dropping in a sink)

Restarting the ldirectord has the same effect, everthing continues doing wrong.

I cant get a conclusion out of the symptoms.
Every help is appreciated and thanks for your time reading this.

Jan



<Prev in Thread] Current Thread [Next in Thread>