LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: All connections to one realserver

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: All connections to one realserver
From: "Karl Kopper" <karl@xxxxxxxxxxxxxxx>
Date: Thu, 12 Feb 2004 08:58:59 -0800
> This question seems to be asked every couple of months, but
> there has never been a clear solution posted back to the list.
>
> I have 1 virtual server with two realservers using LVS-DR. All
> traffic seems to be going to through the load balancer, but it
> is all being sent to a single realserver.
>
> $ ipvsadm -Ln
>                                      Thu Feb 12 09:36:53 2004
>
>
> IP Virtual Server version 1.0.9 (size=65536)
> Prot LocalAddress:Port Scheduler Flags
>    -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
> TCP  134.174.249.126:80 rr persistent 10
>    -> 192.168.1.146:80             Route   8      0          0
>    -> 192.168.1.147:80             Route   8      0          0
>
> $ ipvsadm -Ln --stats
>                                      Thu Feb 12 09:33:01 2004
>
>
> IP Virtual Server version 1.0.9 (size=65536)
> Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes
> -> RemoteAddress:Port
> TCP  134.174.249.126:80              14108   186564        0 12115380
>         -> 192.168.1.146:80                14107   186555        0
> 12114859          -> 192.168.1.147:80                    1        9
>     0      521
>
> If I take down httpd on that realserver, I get connection
> refused in a browser.
>
> I feel like this means that I am actually *not* connecting through
> the load balancer, but I don't think  this is true because of the
> stats above, and because ssh <VIP> connects to the load balancer.
>
> Any suggestions?
>

I started to type this before your dialog started so here goes anyway...

If all your connections are from a single client computer you may always get
sent back to the same real server due to the various timeouts (UDP, TCP,
connection tracking records, persistent connection template entries). Look
at the "--set" option on the ipvsadm man page for TCP, FIN, and UDP timeouts
for example.Also check out the various "timeout_" files in
/proc/sys/net/ipv4/vs/ (documented at
http://www.linuxvirtualserver.org/docs/sysctl.html). Better yet, test from
two client computers.

You can use ldirectord to remove real servers from the cluster when they no
longer offer services and set:

quiescent=no

in your ldirectord configuration file. If you do this the connection
tracking records and the persistent connection template entries will be
removed (expired) from the Director.

You might also want to set "/proc/sys/net/ipv4/vs/expire_nodest_conn" to 1.

Make sure the VIP is hidden on the real servers, or you never know who is
going to respond to your ARP broadcast (from the client computer) for the
VIP--sometimes the packet might go through the Director and sometimes not.

--Karl

<Prev in Thread] Current Thread [Next in Thread>