LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Multiple external interfaces on the director

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Multiple external interfaces on the director
From: Ken Brownfield <krb@xxxxxxxxxxx>
Date: Mon, 13 Mar 2006 15:36:27 -0800
First, I'd like to say the LVS has been functioning flawlessly for me for the last two years. Mon needed a heavily reworked alert/ipvs module, but otherwise it's been great.

However, I need to support VIPs on the director that live on two separate external subnets:

      |        |
 eth0 |   eth1 |          eth0 = ISP1_IP on ISP1_SUBNET
----------------------    eth1 = ISP2_IP on ISP2_SUBNET
|      Director      |
----------------------
  internal |
           |

The default gateway is on ISP1_SUBNET/eth0, and I have source routes set up as follows for eth1:

# cat /etc/SuSE-release
SuSE Linux 9.0 (i586)
VERSION = 9.0
# uname -a
Linux lvs0 2.4.21-303-smp4G #1 SMP Tue Dec 6 12:33:10 UTC 2005 i686 i686 i386 GNU/Linux
# ip -V
ip utility, iproute2-ss020116
# ip rule list
0:      from all lookup local
32765:  from ISP2_SUBNET lookup 136
32766:  from all lookup main
32767:  from all lookup default
# ip route show table 136
ISP2_SUBNET dev eth1  scope link  src ISP2_IP
default via ISP2_GW dev eth1

If I perform an mtr/traceroute on the director bind()ed to the ISP2_IP interface, outgoing traceroutes traverse the proper ISP2_GW, and the same for the ISP1_IP interface. I'm pretty sure the source- route behavior is correct, since I can revert from the proper behavior by dropping table 136.

For a single web service, I'm defining identical VIPs but for each of the ISPs:

 -A -t ISP1_VIP:80 -s wlc
 -a -t ISP1_VIP:80 -r 10.10.10.10:80 -m -w 1000
 -a -t ISP1_VIP:80 -r 10.10.10.11:80 -m -w 800
 -A -t ISP2_VIP:80 -s wlc
 -a -t ISP2_VIP:80 -r 10.10.10.10:80 -m -w 1000
 -a -t ISP2_VIP:80 -r 10.10.10.11:80 -m -w 800

Incoming packets come in via the proper gateway, but LVS always emits response packets through the default gateway, seemingly ignoring the source-route rules. I'm not sure if this is a somewhat subtle issue with my source routes and rules, or if LVS needs some special magic.

I can come up with some random guesses (Route scopes? Need to load the source routes before ip_vs is loaded? etc) but I'm hoping someone knows LVS internals well enough to point me in the right direction without having to cause unnecessary trial-and-error down-time.

Thanks for your time, and let me know if any additional info would help!

Ken.
krb@xxxxxxxxxxx


<Prev in Thread] Current Thread [Next in Thread>