LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

LVS and REDIRECT and Policy Routing [Was: Re: LVS-DR and fwmarks questio

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: LVS and REDIRECT and Policy Routing [Was: Re: LVS-DR and fwmarks question]
Cc: Bradley McLean <bradlist@xxxxxxxxx>
From: Horms <horms@xxxxxxxxxxxx>
Date: Fri, 3 Oct 2003 18:50:28 +0900
On Thu, Oct 02, 2003 at 01:25:05PM +0700, Faruk Ahmed wrote:
> > This is all off the top of my head but it should work.  Step two might 
> > be an issue because the iptable docs say the nat table only picks up 
> > socket creation packtes (aka SYN packets).
> 
> If it works this way, then Transparent proxy + LVS with Director
> should work in 2.4.x (ip table based). Have anyone tested it? 
> 
> Horm, what's your opinion ?

I have... now.

Here is my take on the problem.

First, the main hook for LVS, ip_vs_in() sits in the LOCAL_IN
chain. Thus rules in the PREROUTING chain will get to packets
before they hit LVS.

http://www.vergenet.net/~horms/tmp/nf-lvs.png 
http://www.vergenet.net/~horms/tmp/nf-lvs.dia (corrections welcome)

So if you put a REDIRECT rule in PREROUTING then it will take effect.

However as I understand, as confirmed by my testing, this will not
work. REDIRECT works by changing the destination IP address
to a local address so that it ends up in the LOCAL_IN chain.
This address is typically the primary IP addresss of the interface
that the packet arrived on.

The first thing to note here is that if you are using IP/Port
virtual services (i.e. not fwmark virtual services) then the
VIP needs to be the local IP address that REDIRECT has
changed the destination address to. 

The mapping that REDIRECT effects is somewhat similar to the way that 
NAT works. And as with NAT the mapping has to be reversed for
return packets, otherwise they will end up back at the end-user
with the wrong source IP address and the connection won't work.
REDIRECT, like NAT, relies on nefilter's connection tracking do
make the unmapping. Unfortunately LVS bypasses netfilter's connection
tracking code. Thus the unmapping doesn't occur and the connection will
break.

N.B: Bradley McLean reports that he has got REDIRECT to work.
     I suspect his setup happens to be a corner case that
     happens to work, where as the general case does not
     for the reason above.

     http://marc.theaimsgroup.com/?l=linux-virtual-server&m=103612116901768&w=2

The antifacto patch may allow this to work but you still have
the confusion of the destination IP address being mapped to
a local address on the Linux Director.


I think that a much better solution is to use policy routing. 
Which allows packets to be delivered to the LOCAL_IN chain on 
the Linux Director without any of the problems I described 
that REDIRECT has.

Mathew Crocker outlined how to do this a little while ago
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=106020384024935&w=2

In a nutshell. You need a kernel that has advanced routing and
policy routing compiled in.

Then you can add policy routing rules.

e.g. 1: Any packet sent to 172.17.60.201 (the VIP) is handled by table 100

ip rule add prio 100 to 172.17.60.201 table 100

e.g. 2: Any packet with firewall mark 1 is handled by table 1
        In this case an iptables rule is needed to mark packets
        with fwmark 1. Presumably a fwmark virtual service is
        also configured for LVS.

ip rule add prio 100 fwmark 1 table 100

Now have anything in table 100 routed to the loopback interface,
i.e. local.

ip route add local 0/0 dev lo table 100

See http://linux-ip.net/html/tools-ip-rule.html for some
more informatino on how policy routing works.

-- 
Horms
<Prev in Thread] Current Thread [Next in Thread>