[lvs-users] TIME_WAIT connection on DSR mode.

[taqu <taqumd@xxxxxxxxx>]

Hi

I have a question about TCP TIME_WAIT connection for ipvs DSR mode.
In DSR topology, 'ipvsadm -L -c' show multiple TIME_WAIT entry in
connection table.

[root@lvs1 ~]#  ipvsadm -L -c
IPVS connection entries
pro expire state       source             virtual            destination
TCP 01:30  TIME_WAIT   10.0.0.1:45373     172.16.1.200:http  172.16.2.2:http
TCP 01:30  TIME_WAIT   10.0.0.1:45385     172.16.1.200:http  172.16.2.2:http
TCP 01:30  TIME_WAIT   10.0.0.1:45363     172.16.1.200:http  172.16.2.2:http
TCP 01:30  TIME_WAIT   10.0.0.1:45398     172.16.1.200:http  172.16.2.2:http
TCP 01:30  TIME_WAIT   10.0.0.1:45390     172.16.1.200:http  172.16.2.2:http
TCP 01:30  TIME_WAIT   10.0.0.1:45369     172.16.1.200:http  172.16.2.2:http

I think this is normal behavior because LVS can see only one-direction
packets, from client
 to VIP of LVS.
My question is,
 1. Is my understanding is correct?(which means a lot of TIME_WAIT on
DSR topology is normal)
 2. How do I know the amount of resources consumed by TIME_WAIT connection.
    Is there any guidline to tweak this?
       example)One TIME_WAIT conn consume N byte of memory so "CPS x
tcpfin timeout x N" bytes
               of memory is always consumed, and can make it small by
"ipvsadm --set" command.


-----------------
# Topology
-----------------
.n is the 4th octet of the IP address.


           ( Client )
            .1|
              |
      --- 10.0.0.0/24----
              |
          .254|
           ( L3 Cisco3750 )
          .254|
              |
      --- 172.16.1.0/24---
         |             |
       .1|           .2|
     ( LVS DR )     ( Web )
 VIP:172.16.1.200:80

-----------------
# Routing Table
-----------------
 Client: default via 10.0.0.254
 LVS DR: default via 172.16.1.254
 Web   : default via 172.16.1.254
 L3    : 172.16.1.200 via 172.16.1.1

-----------------
# LVS setting
-----------------
* keepalived.conf
virtual_server 172.16.1.200 80 {
  delay_loop   3
  lb_algo wrr
  lb_kind DR
  protocol     TCP

  virtualhost  example.org
  real_server  172.16.1.2 80 {
    weight 5
    TCP_CHECK {
      connect_timeout 10
    }
  }
}

* by adding ip to lo, vip address is injected local routing table
ip addr add  172.16.1.200/32 dev lo scope host

-----------------
# Web setting
-----------------
* sysctl.conf to avoid respoinding against arp req to vip
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2

* dding ip to lo so that web can accept connection distined to vip
ip addr add  172.16.1.200/32 dev lo scope host
------------------------------------------------------------

Taku.