RE: First-time LVS user, having problem with LVS-DR

["David Fix" <3akqef102@xxxxxxxxxxxxxx>]

BTW, I *am* able to use LVS-NAT to do this, but I'd much rather use
LVS-DR...  :)  For some reason, LVS-NAT works, while LVS-DR does not...  :(

        Dave

-----Original Message-----

Hey guys...  :)  As the subject says, I'm a first-time LVS user, and I'm
having problems with DR...

Here's my network setup:

                        ________
                       |        |
                       | client | (Me, at home)
                       |________| IP = x.99.97.254
                           |
                        ________
                       |        |
                       |Internet|
                       |________|
                           |
                    VIP = x.2.119.227 (eth0)
                        ________
                       |        |
                       |Director|
                       |________|
                    DIP = 192.168.0.1 (eth 1)
                           |
                           |
          --------------------------------------
          |                                    |
          |                                    |
 RIP = x.2.119.9 (eth0)               RIP = x.2.119.15
 RIP = 192.168.0.2 (eth1)             RIP = 192.168.0.3 (eth1)
 VIP = x.2.119.227 (lo:0)             VIP = x.2.119.227 (lo:0)
     ______________                       ______________
    |              |                     |              |
    | realserver 1 |                     | realserver 2 |
    |______________|                     |______________|


I've only tried to set it up with realserver1 to start...

Alright...  I've put in the patches for hiding the arps on realservers 2 and
3.
They don't arp on interface lo:
realserver1# cat /proc/sys/net/ipv4/conf/all/hidden
1
realserver1# cat /proc/sys/net/ipv4/conf/lo/hidden 
1

I've added the VIP to the director:
director# ipvsadm -A -t x.2.119.227:80 -s rr

Then to redirect it to realserver1:
director# ipvsadm -a -t x.2.119.227:80 -r 192.168.0.2 -g

Verified:
director# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  x.2.119.227:http rr
  -> 192.168.0.2:http             Route   1      0          0  

The httpd is up and running on realserver1, and listening to the VIP.
The routing is set up on realserver1:
realserver1# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
x.2.119.227     *               255.255.255.255 UH    0      0        0 lo
x.2.119.0       *               255.255.255.0   U     0      0        0 eth0
192.168.0.0     *               255.255.255.0   U     0      0        0 eth1
loopback        *               255.0.0.0       U     0      0        0 lo
default         x.2.119.1       0.0.0.0         UG    0      0        0 eth0

So everything looks like it's set up.  :)

I've done a tcpdump that shows that packets are being received on both
machines:
director # tcpdump -i any port 80
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
22:19:49.939268 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>
22:19:49.948680 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>
22:19:52.910957 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>
22:19:52.910978 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>

So we see things coming in on the director...

Now, on the realserver:
realserver1# tcpdump -i any port 80
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
10:20:04.274125 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>
10:20:07.242179 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>
10:20:13.177336 IP x.99.97.254.2035 > x.2.119.227.http: S
2588101201:2588101201(0) win 65535 <mss 1452,nop,nop,sackOK>

So I'm seeing stuff coming in from my client (me at home), but there are no
packets going out!

I've verified that I can connect to the realserver's "x.2.119.9" httpd from
my client, and there's no problem there, however, if I try to connect
through the LVS, nothing!

I'm running kernel 2.6.18.1 on all machines, and still can't figure out what
the heck's going on!  I've tried it with both a telnet daemon and an httpd,
and no dice!

Arp output:
director# arp -n
Address                  HWtype  HWaddress           Flags Mask
Iface
192.168.0.2              ether   00:04:23:B3:17:3D   C
eth1
x.2.119.1                ether   00:E0:52:AC:A0:00   C
eth0

realserver1# arp -n
Address                  HWtype  HWaddress           Flags Mask
Iface
192.168.0.1              ether   00:0E:0C:5B:AB:59   C
eth1
x.2.119.1                ether   00:E0:52:AC:A0:00   C
eth0

Can anyone help me out here?  :)  If there's anything you need to know, I'll
be happy to tell you!

Thanks in advance!

David Fix