LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Director doesn't forward packets to realserver

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Director doesn't forward packets to realserver
From: Davy De Winter <davy.dewinter@xxxxxxxxx>
Date: Fri, 15 Jul 2005 12:40:10 +0200
Hi,

I've tried to setup a small LVS-DR testbed but I didn't manage to get things to
work yet. The problem is that my director doesn't forward packets to the
realserver and I don't know where to look further. Here's a graphical
representation of my (simple) testnetwork:

(VIP: 11.5)15.2                       15.1                 7.2          7.1
DIRECTOR------------------------ROUTER--------------CLIENT
    | 10.4 (DIP)                                   | 3.1
    |                                                     |
    |                                                     |
    | 10.1 (RIP)                                   |
REALSERVER-----------------------
(VIP: 11.5)3.2

To keep the drawing clean i've only listed the last 2 bytes of the ip-address
(they all start with 192.168). The client sends http-requests with source ip in
the range 192.168.7.5-192.168.7.254 to the routers which forwards them to the
director. The destination IP of the request is 192.168.11.5. This is the VIP of
the director. The router forwards the first packet (the SYN-packet of the
TCP-stream) to the director. But the director doesn't forward the packet further
to the realserver. When I look with tcpdump at the incoming interface of the
director (192.168.15.2) the SYN-packets arrive just fine:

15:17:12.975822 IP 192.168.7.38.25920 > 192.168.11.5.www: S
292817339:292817339(0) win 32768 <mss 1460>
15:17:12.975926 IP 192.168.7.193.29768 > 192.168.11.5.www: S
284814485:284814485(0) win 32768 <mss 1460>
15:17:13.309114 IP 192.168.7.47.49871 > 192.168.11.5.www: S
294150796:294150796(0) win 32768 <mss 1460>
15:17:13.309158 IP 192.168.7.139.31938 > 192.168.11.5.www: S
270141166:270141166(0) win 32768 <mss 1460>

More information about my director-configuration:

1) The interfaces are configured as follows:

auto eth0
iface eth0 inet static
        address 10.10.2.94
        network 10.0.0.0
        netmask 255.0.0.0
        broadcast 10.255.255.255
        gateway 10.10.10.12

auto eth0:11
iface eth0:11 inet static
        address 192.168.11.5
        network 192.168.11.0
        netmask 255.255.255.0
        broadcast 192.168.11.255

auto eth1
iface eth1 inet static
        address 192.168.15.5
        network 192.168.15.0
        netmask 255.255.255.0
        broadcast 192.168.15.255

auto eth2
iface eth2 inet static
        address 192.168.10.4
        network 192.168.10.0
        netmask 255.255.255.0
        broadcast 192.168.10.255

There's one alias for interface eth0.

2) The virtual server is configured by hand:

cat     /proc/sys/net/ipv4/ip_forward
echo "0" > /proc/sys/net/ipv4/ip_forward
echo 'setting icmp redirects (1 on, 0 off)'
echo "1" > /proc/sys/net/ipv4/conf/all/send_redirects
cat        /proc/sys/net/ipv4/conf/all/send_redirects
echo "1" > /proc/sys/net/ipv4/conf/default/send_redirects
cat        /proc/sys/net/ipv4/conf/default/send_redirects
echo "1" > /proc/sys/net/ipv4/conf/eth1/send_redirects
cat        /proc/sys/net/ipv4/conf/eth1/send_redirects

ipvsadm -C

echo "adding http ipvsadm round robin cheduling..."
ipvsadm -A -t 192.168.11.5:www -s rr

echo "forward to the realservers using direct routing with weight 1"
ipvsadm -a -t 192.168.11.5:www -r 192.168.10.1 -g
ping -c 1 192.168.10.1
#ipvsadm -a -t 192.168.11.5:http -r 192.168.10.2 -g -w 1
#ping -c 1 192.168.10.2
#ipvsadm -a -t 192.168.11.5:http -r 192.168.10.3 -g -w 1
#ping -c 1 192.168.10.3
/sbin/ipvsadm

3)  All the modules necessary are loaded (I use kernel 2.6.11 - Linux Debian 
Sarge).

Module                  Size  Used by
ip_vs_rr                2688  1
iptable_filter          3008  0
ipt_LOG                 7360  1
iptable_nat            26652  1
ip_conntrack           53472  1 iptable_nat
ip_tables              25280  3 iptable_filter,ipt_LOG,iptable_nat
ip_vs_lc                2432  0
ip_vs                 103552  5 ip_vs_rr,ip_vs_lc
uhci_hcd               32592  0
ohci_hcd               18692  0
ehci_hcd               33288  0
usbcore               122748  3 uhci_hcd,ohci_hcd,ehci_hcd
i2c_sis96x              5380  0
i2c_core               22544  1 i2c_sis96x
shpchp                100836  0
pci_hotplug            30772  1 shpchp

4) The route-table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.15.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.11.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth2
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 eth0
0.0.0.0         10.10.10.12     0.0.0.0         UG    0      0        0 eth0

5) ipvsadm -l and ipvsadm -l --stats gave me the following

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.11.5:www rr
  -> 192.168.10.1:www             Route   1      0          0


epaperdirector94:/home/dgdwinte# ipvsadm -l --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  192.168.11.5:www                    0        0        0        0        0
  -> 192.168.10.1:www                    0        0        0        0        0

Everything seems to work fine, it only looks like the module "eats" al the
packets and then they're disappeared. (the initial SYN-packets get never out of
the director to the real server). Can someone give some tips where to look?

Thanks in advance,
davy.







--- Dit bericht werd verzonden met de Webmail van Hogeschool Gent

<Prev in Thread] Current Thread [Next in Thread>