|
Andy
On Tue, 2008-02-26 at 13:34 +0000, Andy Ashley wrote:
> I need a bit of help for an LVS setup:
Let's see what we can do...
> I have the following setup: (hope this ASCII comes out OK..)
More or less, but I'm not going to include it quoted! A request,
however: please don't use signature delimiters as breaks in your message
- my mail client refuses to quote anything below the "--" mark, where
you describe your problem!
Let me get something straight - the realservers (web servers) are using
the L3 switch as their default gateway. Correct?
> I have the following set up using ipvsadm:
>
> # ipvsadm -A -t 10.91.243.10:80 -s rr
> # ipvsadm -a -t 10.91.243.10:80 -r 10.19.60.230:80 -m
> # ipvsadm -a -t 10.91.243.10:80 -r 10.19.60.231:80 -m
OK, so you're using LVS-DR. This means the traffic should ideally only
touch the director on the way in.
> I also have the following SNAT rule to change the source address of
> packets leaving the load balancer,
> destined for the web servers, to 10.91.243.4, which is the load balancer
> IP that it chooses to route packets from (as its lowest, I think..)
>
> # iptables -A POSTROUTING -t nat -o eth0 -s 0/0 -d 10.19.60.224/29 -j
> SNAT --to-source 10.91.243.4
This is probably the source of the problem. Netfilter's NAT and IPVS
don't play well together, because in some cases they're going to stomp
on each other's packets. A side effect of this is that packets are
processed differently by the kernel when IPVS is in use, which means
some of the netfilter hooks are bypassed - this probably explains why
your SNAT rule, well, doesn't SNAT.
What you need is an additional address (or more than one) on your web
servers, bound to the loopback adapter, so that they also "have" the VIP
assigned to them.
Put simply:
/sbin/ip address add 10.91.243.10/32 dev lo
You'll then have to ensure that your web servers don't send ARP
responses out for the VIP on interface lo using something like the
following:
# Start the hiding interface functionality
echo 1 > /proc/sys/net/ipv4/conf/all/hidden
# Hide all addresses for this interface
echo 1 > /proc/sys/net/ipv4/conf/lo/hidden
You can put this in rc.local or in /etc/sysctl.conf, depending on distro
(the format differs between the two).
Once done, the director will then send packets through to the
realservers with the client IP as source and the VIP as the destination;
the realservers will then respond directly to the client from the VIP.
The three-way handshake completes, traffic flows, everyone is happy.
HTH
Graeme
|
