LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] ipvs local node broken since kernel 2.6.28

To: Deniss <dg@xxxxxx>
Subject: Re: [lvs-users] ipvs local node broken since kernel 2.6.28
Cc: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
From: Simon Horman <horms@xxxxxxxxxxxx>
Date: Mon, 29 Mar 2010 23:19:15 +1100
On Wed, Mar 24, 2010 at 03:49:24PM +0200, Deniss wrote:
> 
> ipvs local node setup is broken since kernel 2.6.28

Unfortunately I'm not having any luck reproducing this on 2.6.28.

> 
> test setup is as following:
> 
> MASTER/RS1 (IP 10.0.0.20/24, VIP 10.0.0.1):
> eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
> UNKNOWN qlen 2000
>      link/ether 00:e0:4c:b5:c7:a2 brd ff:ff:ff:ff:ff:ff
>      inet 10.0.0.20/24 brd 10.0.1.255 scope global eth0
>      inet 10.0.0.1/32 scope global eth0
> FWM  5 wlc
>    -> 10.0.0.20:3306               Local   1      0          0
>    -> 10.0.0.30:3306               Route   1      0          0
> 
> RS1 ~ # sysctl -a | egrep "\.(rp_fi|ip_forw)"
> net.ipv4.conf.all.rp_filter = 0
> net.ipv4.conf.all.arp_filter = 0
> net.ipv4.conf.all.arp_announce = 2
> net.ipv4.conf.all.arp_ignore = 1
> net.ipv4.conf.all.arp_accept = 0
> net.ipv4.conf.default.rp_filter = 0
> net.ipv4.conf.default.arp_filter = 0
> net.ipv4.conf.default.arp_announce = 2
> net.ipv4.conf.default.arp_ignore = 1
> net.ipv4.conf.default.arp_accept = 0
> net.ipv4.conf.lo.rp_filter = 1
> net.ipv4.conf.lo.arp_filter = 0
> net.ipv4.conf.lo.arp_announce = 2
> net.ipv4.conf.lo.arp_ignore = 1
> net.ipv4.conf.lo.arp_accept = 0
> net.ipv4.conf.eth0.rp_filter = 1
> net.ipv4.conf.eth0.arp_filter = 0
> net.ipv4.conf.eth0.arp_announce = 2
> net.ipv4.conf.eth0.arp_ignore = 1
> net.ipv4.conf.eth0.arp_accept = 0
> net.ipv4.ip_forward = 0
> 
> RS1 ~ # iptables -L -nvt mangle
> Chain PREROUTING (policy ACCEPT 34M packets, 3959M bytes)
>   pkts bytes target     prot opt in     out     source
> destination
>     0  0 MARK       tcp  --  *      *       0.0.0.0/0
> 10.0.0.1           tcp dpt:3306 MARK xset 0x5/0xffffffff
> 
> RS2 (IP 10.0.0.30/24):
> lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
>      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>      inet 10.0.0.1/32 scope global lo:1
> eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
> UNKNOWN qlen 2000
>      link/ether 00:17:31:8d:58:3d brd ff:ff:ff:ff:ff:ff
>      inet 10.0.0.30/24 brd 10.0.0.255 scope global eth0
> 
> from MASTER trying to connect to VIP (local process tries to connect
> to VIP on local nic):
> telnet -b 10.0.0.20 10.0.0.1 3306  (no connect)
> 
> and tcpdump show (tcpdump -i eth0 -n -e -s0 -v host 10.0.0.1):

Are you sure these packets are showing up on eth0?
I'm seeing this traffic on lo.

> 19:22:05.809708 00:e0:4c:b5:c7:a2 > 00:17:31:8d:58:3d, ethertype IPv4
> (0x0800), length 74: (tos 0x10, ttl 64, id 57858, offset 0, flags [DF],
> proto TCP (6), length 60) 10.0.0.20.42779 > 10.0.0.1.3306: S, cksum
> 0xc2bd (correct), 4115412076:4115412076(0) win 32792 <mss
> 16396,sackOK,timestamp 1955438 0,nop,wscale 8>
> 19:22:05.809811 00:17:31:8d:58:3d > 00:e0:4c:b5:c7:a2, ethertype IPv4
> (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
> TCP (6), length 60) 10.0.0.1.3306 > 10.0.0.20.42779: S, cksum 0xbe1e
> (correct), 920820514:920820514(0) ack 4115412077 win 5792 <mss
> 1460,sackOK,timestamp 1008245313 1955438,nop,wscale 8>
> 19:22:09.647727 00:17:31:8d:58:3d > 00:e0:4c:b5:c7:a2, ethertype IPv4
> (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
> TCP (6), length 60) 10.0.0.1.3306 > 10.0.0.20.42779: S, cksum 0xba5e
> (correct), 920820514:920820514(0) ack 4115412077 win 5792 <mss
> 1460,sackOK,timestamp 1008246273 1955438,nop,wscale 8>
> 
> as can be seen, packets goes through ipvs code and forwarded to RS2 and
> REPLIED by RS2 but then dropped somewhere in iptables/ipvs code !

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>