LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

LVS/NAT and SYN/ACK issue

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: LVS/NAT and SYN/ACK issue
From: dmitri@xxxxxxxxxxxxxx
Date: Wed, 21 Feb 2007 13:04:54 +0000
Hi,

I should probably say in the beginning that the issue I'm going to 
describe
is not directly related to the problem discussed on this list
a while ago (http syn/ack not translated when ftp loadbalancing
also enabled).
We have several LVS/NAT installations which are managed by Keepalived.
All of them are pretty much identical and exhibit the same issue.
The setup is looking like this (a backup load balancer and a backup
router are omitted) and is LVS/NAT standard:


        !-----------------!
        !                 !
        !     Internet    !
        !                 !
        !-----------------!
                 !
                 !
        !-----------------!
        !                 !
        !     Router      !
        !                 !
        !-----------------!
                 !
                 !
        !-----------------!
        !      eth0       !
        !                 !
        !  LoadBalancer   !
        !                 !
        !      eth1       !
        !-----------------!
                 !
                 !192.168.1.0/24
    ------------------------
    !       !       !      !
  !---!                  !---!
  !RS1!     .........    !RSN!
  !---!                  !---!

This setup is working fine most of the time except when a client sends a 
TCP SYN
packet and then forgets about this connection. In this case a RealServer 
starts to
send SYN/ACK packets until this connection on the server times out and it 
sends RST/ACK.
The issue is that two last packets don't get translated because ipvs on 
the
LoadBalancer already timed out this connection. Below is a tcpdump on 
LoadBalancer/eth0:

10:58:20.655059 IP 213.248.204.8.2113 > 213.248.224.116.43: S 
1402601529:1402601529(0) win 512
10:58:20.655335 IP 213.248.224.116.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
10:58:24.031708 IP 213.248.224.116.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
10:58:30.792336 IP 213.248.224.116.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
10:58:44.303557 IP 213.248.224.116.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
10:59:11.316010 IP 213.248.224.116.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
11:00:05.330972 IP 213.248.224.116.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
11:01:05.346329 IP 192.168.1.32.43 > 213.248.204.8.2113: S 
443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
11:02:05.362233 IP 192.168.1.32.43 > 213.248.204.8.2113: R 1:1(0) ack 1 
win 49312

In this example I simulated the situation with sending SYN packet from my 
PC
to the server and dropping all further packets.
While the SYN/ACK packets were still being translated, ipvsadm -lnc was 
showing this:

TCP 28:12  NONE        213.248.204.8:0    213.248.224.116:43 
192.168.1.32:43
TCP 00:57  SYN_RECV    213.248.204.8:2113 213.248.224.116:43 
192.168.1.32:43

But once I see only this:

TCP 27:02  NONE        213.248.204.8:0    213.248.224.116:43 
192.168.1.32:43

packets from RealServer belonging to this connection (from RealServer 
point of view) stop
getting translated.

This is not a real problem but rather a nuisance for me. I just don't want 
packets with private IP's
leaving LoadBalancer. I can't block this packets with iptables since I 
believe ipvs does SNATing
somewhere in POSTROUTING chain and there is no way to put any other rules 
beyond this chain.
I also can't modify SYN_RECV timeout since there is no 
tcp_timeout_syn_recv entry
in /proc/sys/net/ipv4/vs/ (this is a stock CentOS 4.3 kernel)

My question is: Is it possible to block not translated packets from 
leaving the LoadBalancer without touching
RealServers and the Router?

If it can help, here is additional info:

# uname -a
Linux lb1 2.6.9-34.ELsmp #1 SMP Thu Mar 9 06:23:23 GMT 2006 x86_64 x86_64 
x86_64 GNU/Linux

# ipvsadm --help
ipvsadm v1.24 2003/06/07 (compiled with getopt_long and IPVS v1.2.0)


Thank you
Dmitri Skachkov

<Prev in Thread] Current Thread [Next in Thread>