LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] IPVS with SNAT support on the kernel 2.6.36 + iptables v

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] IPVS with SNAT support on the kernel 2.6.36 + iptables v1.4.10
From: Ivan Havlicek <ivan@xxxxxxxxxxx>
Date: Fri, 4 Mar 2011 15:34:51 +0100
Hi,

Since my last post, I've made some changes... without success.

For now, I've two servers (same kernel, same iptables, same sysctl...)
On the first, as described in my first post, ipvs doesn't work.
The second which was created from the first one by copy,  works fine !
I give there some infos if somebody wants to debug it...
(if more information needed, send me request by mail)

SRV1 (KO) :
-----------------
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.1.2.254:389 wlc
  -> 10.1.11.11:389               Masq    100    0          0
  -> 10.1.12.11:389               Masq    100    0          0

echo 8 > /proc/sys/net/ipv4/vs/debug_level
iptables -t nat -I POSTROUTING -m ipvs --vaddr 10.1.2.254 -j LOG
--log-prefix "ipvs/POSTROUTING : "
iptables -t nat -I POSTROUTING -p tcp --dport 389 -j LOG --log-prefix
"nat/POSTROUTING : "

/var/log/kernel.log :
Mar  4 12:55:16 srv1 kernel: IPVS: ip_vs_wlc_schedule(): Scheduling...
Mar  4 12:55:16 srv1 kernel: IPVS: WLC: server 10.1.12.11:389
activeconns 0 refcnt 1 weight 100 overhead 0
Mar  4 12:55:16 srv1 kernel: IPVS: Bind-dest TCP c:10.1.2.31:54772
v:10.1.2.254:389 d:10.1.12.11:389 fwd:M s:0 conn->flags:100
conn->refcnt:1 dest->refcnt:2
Mar  4 12:55:16 srv1 kernel: IPVS: Schedule fwd:M c:10.1.2.31:54772
v:10.1.2.254:389 d:10.1.12.11:389 conn->flags:140 conn->refcnt:2
Mar  4 12:55:16 srv1 kernel: IPVS: TCP input  [S...]
10.1.12.11:389->10.1.2.31:54772 state: NONE->SYN_RECV conn->refcnt:2
Mar  4 12:55:18 srv1 kernel: IPVS: ip_vs_wlc_schedule(): Scheduling...
Mar  4 12:55:18 srv1 kernel: IPVS: WLC: server 10.1.11.11:389
activeconns 0 refcnt 1 weight 100 overhead 0
Mar  4 12:55:18 srv1 kernel: IPVS: Bind-dest TCP c:10.1.2.31:54774
v:10.1.2.254:389 d:10.1.11.11:389 fwd:M s:0 conn->flags:100
conn->refcnt:1 dest->refcnt:2
Mar  4 12:55:18 srv1 kernel: IPVS: Schedule fwd:M c:10.1.2.31:54774
v:10.1.2.254:389 d:10.1.11.11:389 conn->flags:140 conn->refcnt:2
Mar  4 12:55:18 srv1 kernel: IPVS: TCP input  [S...]
10.1.11.11:389->10.1.2.31:54774 state: NONE->SYN_RECV conn->refcnt:2

==> POSTROUTING never reached...
 pkts bytes target     prot opt in     out     source
destination
   15   900 LOG        tcp  --  *      *       0.0.0.0/0
0.0.0.0/0           tcp dpt:389 LOG flags 0 level 4 prefix
`nat/POSTROUTING : '
    0     0 LOG        all  --  *      *       0.0.0.0/0
0.0.0.0/0           vaddr 10.1.2.254 LOG flags 0 level 4 prefix
`ipvs/POSTROUTING : '
    0     0 SNAT       all  --  *      *       0.0.0.0/0
10.1.12.0/24        vaddr 10.1.2.254 to:192.168.12.12
    0     0 SNAT       all  --  *      *       0.0.0.0/0
10.1.11.0/24        vaddr 10.1.2.254 to:192.168.11.12

SRV2 (OK) :
-----------------
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.1.12.254:389 wlc
  -> 10.1.11.11:389               Masq    90     0          0
  -> 10.1.12.11:389               Masq    100    0          0

echo 8 > /proc/sys/net/ipv4/vs/debug_level
iptables -t nat -I POSTROUTING -m ipvs --vaddr 10.1.1.254 -j LOG
--log-prefix "ipvs/POSTROUTING : "
iptables -t nat -I POSTROUTING -p tcp --dport 389 -j LOG --log-prefix
"nat/POSTROUTING : "

/var/log/kernel.log :
Mar  4 12:02:58 srv2 kernel: IPVS: ip_vs_wlc_schedule(): Scheduling...
Mar  4 12:02:58 srv2 kernel: IPVS: WLC: server 10.1.12.11:389
activeconns 0 refcnt 1 weight 100 overhead 0
Mar  4 12:02:58 srv2 kernel: IPVS: Bind-dest TCP c:10.1.1.31:58424
v:10.1.1.254:389 d:10.1.12.11:389 fwd:M s:0 conn->flags:100
conn->refcnt:1 dest->refcnt:2
Mar  4 12:02:58 srv2 kernel: IPVS: Schedule fwd:M c:10.1.1.31:58424
v:10.1.1.254:389 d:10.1.12.11:389 conn->flags:140 conn->refcnt:2
Mar  4 12:02:58 srv2 kernel: IPVS: TCP input  [S...]
10.1.12.11:389->10.1.1.31:58424 state: NONE->SYN_RECV conn->refcnt:2
Mar  4 12:02:58 srv2 kernel: nat/POSTROUTING : IN= OUT=tun12
SRC=10.1.1.31 DST=10.1.12.11 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=8243
DF PROTO=TCP SPT=58424 DPT=389 WINDOW=5840 RES=0x00 SYN URGP=0
Mar  4 12:02:58 srv2 kernel: ipvs/POSTROUTING : IN= OUT=tun12
SRC=10.1.1.31 DST=10.1.12.11 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=8243
DF PROTO=TCP SPT=58424 DPT=389 WINDOW=5840 RES=0x00 SYN URGP=0
Mar  4 12:02:58 srv2 kernel: IPVS: TCP input  [..A.]
10.1.12.11:389->10.1.1.31:58424 state: SYN_RECV->ESTABLISHED
conn->refcnt:2
Mar  4 12:02:58 srv2 kernel: IPVS: TCP input  [.FA.]
10.1.12.11:389->10.1.1.31:58424 state: ESTABLISHED->CLOSE_WAIT
conn->refcnt:2
Mar  4 12:02:58 srv2 kernel: IPVS: TCP output  [.FA.]
10.1.12.11:389->10.1.1.31:58424 state: CLOSE_WAIT->TIME_WAIT
conn->refcnt:2

Mar  4 12:02:59 srv2 kernel: IPVS: ip_vs_wlc_schedule(): Scheduling...
Mar  4 12:02:59 srv2 kernel: IPVS: WLC: server 10.1.11.11:389
activeconns 0 refcnt 1 weight 100 overhead 0
Mar  4 12:02:59 srv2 kernel: IPVS: Bind-dest TCP c:10.1.1.31:58427
v:10.1.1.254:389 d:10.1.11.11:389 fwd:M s:0 conn->flags:100
conn->refcnt:1 dest->refcnt:2
Mar  4 12:02:59 srv2 kernel: IPVS: Schedule fwd:M c:10.1.1.31:58427
v:10.1.1.254:389 d:10.1.11.11:389 conn->flags:140 conn->refcnt:2
Mar  4 12:02:59 srv2 kernel: IPVS: TCP input  [S...]
10.1.11.11:389->10.1.1.31:58427 state: NONE->SYN_RECV conn->refcnt:2
Mar  4 12:02:59 srv2 kernel: nat/POSTROUTING : IN= OUT=tun11
SRC=10.1.1.31 DST=10.1.11.11 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=1062
DF PROTO=TCP SPT=58427 DPT=389 WINDOW=5840 RES=0x00 SYN URGP=0
Mar  4 12:02:59 srv2 kernel: ipvs/POSTROUTING : IN= OUT=tun11
SRC=10.1.1.31 DST=10.1.11.11 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=1062
DF PROTO=TCP SPT=58427 DPT=389 WINDOW=5840 RES=0x00 SYN URGP=0
Mar  4 12:02:59 srv2 kernel: IPVS: TCP input  [..A.]
10.1.11.11:389->10.1.1.31:58427 state: SYN_RECV->ESTABLISHED
conn->refcnt:2
Mar  4 12:02:59 srv2 kernel: IPVS: TCP input  [.FA.]
10.1.11.11:389->10.1.1.31:58427 state: ESTABLISHED->CLOSE_WAIT
conn->refcnt:2
Mar  4 12:02:59 srv2 kernel: IPVS: TCP output  [.FA.]
10.1.11.11:389->10.1.1.31:58427 state: CLOSE_WAIT->TIME_WAIT
conn->refcnt:2

==> POSTROUTING ok, it work's :
 pkts bytes target     prot opt in     out     source
destination
 5683  341K LOG        tcp  --  *      *       0.0.0.0/0
0.0.0.0/0           tcp dpt:389 LOG flags 0 level 4 prefix
`nat/POSTROUTING : '
   33  1980 LOG        all  --  *      *       0.0.0.0/0
0.0.0.0/0           vaddr 10.1.1.254 LOG flags 0 level 4 prefix
`ipvs/POSTROUTING : '
   22  1320 SNAT       all  --  *      *       0.0.0.0/0
10.1.12.0/24        vaddr 10.1.1.254 to:192.168.12.11
   13   780 SNAT       all  --  *      *       0.0.0.0/0
10.1.11.0/24        vaddr 10.1.1.254 to:192.168.11.11

It's really amazing...
Next week, I'm going to try with 2.6.37 kernel before I become crazy,
with this problem.
I hope that the use of /proc/sys/net/ipv4/vs/conntrack is going to
help me to solve it !
Best regards.
--
                                                               Ivan

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>