[lvs-users] LVS-DR Director doesn't rewrite MAC nor send to RIP when CIP is not local to subnet

[Jessie <lists@xxxxxxxx>]

Greetings,

We are seeing a case where our directors are not rewriting the MAC  
address when the client IP is not within the same subnet as the VIP.
I've searched the archives and found some similar posts:

http://marc.info/?t=117130643100005&r=1&w=3
http://marc.info/?t=110856612700005&r=1&w=3

I believe our problem to be slightly different.

After doing packet captures, we see the director receiving the  
traffic (SYNs) and never see a packet destined to the RIP.
Normally, we'd see the packet hit the Director, then the very same  
packet is sent to the RIP with a SRC MAC of the Director
and a DST MAC of the RIP (source CIP and dest. VIP are preserved).



Our Setup:

CIP: ...Any IP not local to 172.16.10.0/24

Director:  172.16.10.61/24

VIP: 172.16.10.20

There is a Public IP which a PIX firewall static maps 1-1 to this.
For sake of this setup, lets call the "Public VIP"

Public VIP: 64.65.66.67

RIP: 172.16.10.11/32

DGW: 172.16.10.254/32



Here are the configurations & logs:

PIX(DGW: 172.16.10.254):
access-list acl-in extended permit tcp any host 64.65.66.67 eq www
static (inside,outside) 64.65.66.67 172.16.10.20 netmask 255.255.255.255

The PIX handles the 1 to 1 mapping of a public IP to the private IP.
Only port 80 is permitted through the PIX firewall.


Director:
kernel: 2.6.19-gentoo-r5 running ipvsadm 1.24

RIP:
Windows 2003: MS Loopback Adapter enabled with 172.16.20.10/24 Metric  
254 no DGW for loopback.


Packet Captures:
Below are two example packets. The capture was doing using the switch  
with monitor-only port (ie: port mirror) from the load balancer.
The packets show two attempts from and outside CIP of 4.3.2.1. to  
Public VIP 64.65.66.67.
The PIX does the rewrite to the VIP 172.16.10.20 first.

I'm only including two attempts because they are all the same.

The PIX MAC is 00:18:ba:c6:97:dc
The Director MAC is 00:09:6b:00:8a:79

======================================================================== 
=================================
No.     Time            Source                Destination            
Protocol Info
     104 16:40:03.957859 4.3.2.1         172.16.10.20           
TCP      49385 > http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1380 WS=0  
TSV=265153516 TSER=0

Frame 104 (78 bytes on wire, 78 bytes captured)
Ethernet II, Src: 00:18:ba:c6:97:dc (00:18:ba:c6:97:dc), Dst:  
Ibm_00:8a:79 (00:09:6b:00:8a:79)
     Destination: Ibm_00:8a:79 (00:09:6b:00:8a:79)
     Source: 00:18:ba:c6:97:dc (00:18:ba:c6:97:dc)
     Type: IP (0x0800)
Internet Protocol, Src: 4.3.2.1 (4.3.2.1), Dst: 172.16.10.20  
(172.16.10.20)
Transmission Control Protocol, Src Port: 49385 (49385), Dst Port:  
http (80), Seq: 0, Ack: 0, Len: 0
======================================================================== 
=================================



======================================================================== 
=================================
No.     Time            Source                Destination            
Protocol Info
     117 16:40:06.790703 4.3.2.1         172.16.10.20           
TCP      49385 > http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1380 WS=0  
TSV=265153521 TSER=0

Frame 117 (78 bytes on wire, 78 bytes captured)
Ethernet II, Src: 00:18:ba:c6:97:dc (00:18:ba:c6:97:dc), Dst:  
Ibm_00:8a:79 (00:09:6b:00:8a:79)
     Destination: Ibm_00:8a:79 (00:09:6b:00:8a:79)
     Source: 00:18:ba:c6:97:dc (00:18:ba:c6:97:dc)
     Type: IP (0x0800)
Internet Protocol, Src: 4.3.2.1 (4.3.2.1), Dst: 172.16.10.20  
(172.16.10.20)
Transmission Control Protocol, Src Port: 49385 (49385), Dst Port:  
http (80), Seq: 0, Ack: 0, Len: 0
======================================================================== 
=================================


The Director has both it's native IP address, and the VIP:

~director:# ip address show eth0

1: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast  
qlen 1000
     link/ether 00:09:6b:00:8a:79 brd ff:ff:ff:ff:ff:ff
     inet 172.16.10.61/24 brd 172.16.10.255 scope global eth0
     inet 172.16.10.20/32 brd 172.16.10.255 scope global eth0
     inet6 fe80::209:6bff:fe00:8a79/64 scope link
        valid_lft forever preferred_lft forever





When we tested with a CIP on the localnet, and tested the VIP, the  
director worked as should, we saw the packet hitting the director
and then being rewritten and sent to the RIP.

If anyone has any suggestions that'd be great. We are scratching our  
heads at the moment.


-Jessie