LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Issues with braindead network topology and LVS-NAT

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Issues with braindead network topology and LVS-NAT
From: Pascal Bleser <pascal.bleser@xxxxxxxxxxxxxx>
Date: Tue, 18 Oct 2005 13:53:39 +0200
Joseph Mack NA3T wrote:
> On Tue, 18 Oct 2005, Pascal Bleser wrote:
>> Joseph Mack NA3T wrote:
>>> On Tue, 4 Oct 2005, Pascal Bleser wrote:
>>>> Now I use a simple TCP forwarder, rinetd.
>>> hadn't heard of this. Had a look on Boutell's page. For the HOWTO, how
>>> is this better than some iptables rules?
>> Hi Joe, sorry for the very late reply.
> no problem. computers just sit there till a problem is fixed. It's not
> like biology where the object of interest
> is dead if you don't do something right away. I used to
> be a biochemist and you could never leave an experiment.

Heh, no, indeed, the cluster is still alive ;)

>> I do have some experience with iptables and have set up some custom
>> firewall scripts with it, but
>> I'm not aware of how I could actually rewrite both the source (SNAT)
>> and destination (DNAT) addresses.
> 
> yes a problem.

It really seems to be a problem, yes.
Couldn't find any possibility to do that, unless having 2 NAT boxes, but I 
can't do that here (would
be overkill anyway, as it works with the userspace TCP forwarder)

Normally the best solution (AFAICT) would be IP-IP tunneling, but the 
realservers are HP-PITA-UX...
so... let's forget about that :\

With IP-IP tunneling, I could route the LVS-redirected packets anywhere on the 
network (and actually
have them come back to the LVS cluster), never mind how awkward the network 
architecture is, and
still keep the original client IP information.
I don't think I can have the customer replace the HPUX boxes with SLES 9 at the 
moment,
unfortunately. So.. that option is pretty much a dead end.

>> I could try, though, but the only way I can see as of now is to have
>> both an SNAT and a DNAT rule,
>> the first in POSTROUTING, the latter in PREROUTING or OUTPUT.
> 
> sounds mathematically impossible :-)

Not necessarely.

netfilter *could* process the packets as follows:
+===============+===============+===============+
| rule          | source        | destination   |
+===============+===============+===============+
| INPUT         | 1.1.1.1       | cluster VIP   |
|  `-> ACCEPT   | 1.1.1.1       | cluster VIP   |
+---------------+---------------+---------------+
| PREROUTING    | 1.1.1.1       | cluster VIP   |
|  `-> DNAT ==> | 1.1.1.1       | realserver IP |
+---------------+---------------+---------------+
| ROUTING       | 1.1.1.1       | realserver IP |
|  `-> ACCEPT   | 1.1.1.1       | realserver IP |
+---------------+---------------+---------------+
| POSTROUTING   | 1.1.1.1       | realserver IP |
|  `-> SNAT ==> | cluster VIP   | realserver IP |
+---------------+---------------+---------------+
(1.1.1.1 being an Internet IP address, from the client)

That would be a chain for "incoming" packets, selected with:
  iptables --append INPUT -p tcp --dport 80 -j INCOMING

The issue is rather when the reply packets come back from the realserver (to 
the cluster VIP).
Can netfilter possibly keep the NAT state information about the SNAT'ing it has 
done for the
"incoming" packets ?
I suppose not (well, if it could, it would be some hell of a voodoo trick).

> can you make the machine in the middle into a bridge for packets on the
> LVS network (I don't know if you can do that - proxy arp perhaps?).

Hmm... I don't exactly see what you mean.
The only machine in the middle is the firewall.
It runs Checkpoint and, hence, is on Windows (AFAIK). So I'm afraid there's not 
much packet
rewriting or low-level networking magic to expect there.

        internet
           |
     +---[fw]---+
     |          |  +--Realserver1
     |          |  |
  +-[S]-+      [S]-+
  |     |          |
 LVS1--LVS2        +--Realserver2

[S] = switch
[fw] = firewall

> Since you already have a working solution, I can't see any reason to put
> much more effort into it.

Yep. Well, I would assume that using netfilter has a much lower overhead than a 
userspace TCP
forwarder and, hence, should perform a lot better. I also guess that netfilter 
is more "solid" than
rinetd (though unless the rinetd author is a PR kind of guy, it sure looks 
well-tested and fast).

But I also have to de-SSL the incoming HTTPS packets to HTTP before forwarding 
them to the active
realserver. I use stunnel for that, it does its job really well.
Now if I mix that with iptables, it could get really ugly. For "incoming" 
packets on destination
port 143 (https), I wouldn't redirect them in netfilter, but instead keep them 
going into userspace,
where stunnel is listening. Then, stunnel would redirect the decrypted HTTP 
packets to 127.0.0.1:80,
where netfilter would have to pick them up again, and DNAT them to the 
realserver.

Hm. The more I think/write about it, the more the userspace solution looks 
easier to maintain ;))

Anyway, thanks for the help.

cheers
-- 
  -o) Pascal Bleser               ATOS Worldline/Aachen(DE)
  /\\   System Architect              WLP Business Platform
 _\_v "Really, I'm not out to destroy Microsoft.  That will
just be a completely unintentional side effect."-L.Torvalds

<Prev in Thread] Current Thread [Next in Thread>