LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] IPVS with SNAT support on the kernel 2.6.36 + iptables v

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] IPVS with SNAT support on the kernel 2.6.36 + iptables v1.4.10
From: Ivan Havlicek <ivan@xxxxxxxxxxx>
Date: Fri, 4 Mar 2011 17:26:33 +0100
2011/3/4 Julian Anastasov <ja@xxxxxx>:
>
>        Difference in eth cards? Can you check if there are any
> errors on the tunnel interface:
> cat /proc/net/dev

Any errors in the devices used by ipvs :

virbr2    Lien encap:Ethernet  HWaddr fe:54:10:01:02:01
          inet adr:10.1.2.254  Bcast:10.1.2.255  Masque:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1660942 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1586017 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:0
          RX bytes:268022647 (255.6 MiB)  TX bytes:233688866 (222.8 MiB)
tun11     Lien encap:UNSPEC  HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet adr:192.168.11.12  P-t-P:192.168.11.1  Masque:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:76781 errors:0 dropped:0 overruns:0 frame:0
          TX packets:77686 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:100
          RX bytes:3992828 (3.8 MiB)  TX bytes:38270211 (36.4 MiB)

tun12     Lien encap:UNSPEC  HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet adr:192.168.12.12  P-t-P:192.168.12.1  Masque:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:21 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:100
          RX bytes:1676 (1.6 KiB)  TX bytes:4038 (3.9 KiB)

>        Can you tell us what kind of device is the incoming
> interface, can you disable any kind of hardware offloading
> there, for example, hardware checksums, etc.

The incoming interface is a bridge, and output is OpenVPN tunnels,
that why I need to SNAT...
But the behaviour is the same when I use ethernet interface (eth0).
I've also ipvs on this interfaces on port 80 with the same problem !

>        I don't have test setup to play with xt_ipvs but
> I can send you debug patch for xt_ipvs if it is the
> most suspected part.
>        Also, try to increase the IPVS debug level at least to
> 10, so that we can see such messages:
> IP_VS_DBG_PKT(10, AF_INET, pp, skb, 0, "After DNAT");

I'm afraid that this part of code is never reached !

echo 99 > /proc/sys/net/ipv4/vs/debug_level
(need to eliminate some lines... it's very verbose !)

cat /var/log/kern.log | grep -v 'not hit$'
...
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: lookup service: fwm 0 TCP 10.1.2.254:389 hit
Mar  4 16:52:58 srv1 kernel: IPVS: ip_vs_wlc_schedule(): Scheduling...
Mar  4 16:52:58 srv1 kernel: IPVS: WLC: server 10.1.12.11:389
activeconns 0 refcnt 1 weight 100 overhead 0
Mar  4 16:52:58 srv1 kernel: IPVS: Bind-dest TCP c:192.168.2.111:45792
v:10.1.2.254:389 d:10.1.12.11:389 fwd:M s:0 conn->flags:100
conn->refcnt:1 dest->refcnt:2
Mar  4 16:52:58 srv1 kernel: IPVS: Schedule fwd:M
c:192.168.2.111:45792 v:10.1.2.254:389 d:10.1.12.11:389
conn->flags:140 conn->refcnt:2
Mar  4 16:52:58 srv1 kernel: IPVS: TCP input  [S...]
10.1.12.11:389->192.168.2.111:45792 state: NONE->SYN_RECV
conn->refcnt:2
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_nat_xmit,
net/netfilter/ipvs/ip_vs_xmit.c line 394
Mar  4 16:52:58 srv1 kernel: IPVS: Leave: ip_vs_nat_xmit,
net/netfilter/ipvs/ip_vs_xmit.c line 448
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
Mar  4 16:52:58 srv1 kernel: IPVS: Enter: ip_vs_out,
net/netfilter/ipvs/ip_vs_core.c line 955
...

>        What about these counters in server 1? Are they
> increasing?:

The iptables counter for "LOG : ipvs/POSTROUTING" stay always at 0
The counter for ""LOG : nat/POSTROUTING" increase only when I try
a LDAP request from the server him-self :

Mar  4 12:39:14 srv1 kernel: nat/POSTROUTING : IN= OUT=lo
SRC=10.1.2.254 DST=10.1.2.254 LEN=60 TOS=0x00 PREC=0x00 TTL=64
ID=40419 DF PROTO=TCP SPT=49316 DPT=389 WINDOW=32792 RES=0x00 SYN
URGP=0

In this case, it's seems to be normal that it doesn't work...
(it doesn't on srv2 too)

>        As the server 2 is working, do you have any iptables
> rules in OUTPUT hook on server 1?

No, I'v any other iptables rules... (policy ACCEPT by default)

Thx  for your interest.
-- 
                                                               Ivan

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>