|
Julian,
If you need for the response from a real server to go through a
specific director (as seems to be the case in the original diagram provided
on this question) then you need to SNAT all packets from the director to the
real server (and unSNAT them when they flow in the other direction). This
is also needed in the scenario where LVS is providing high availability
within a cluster (i.e. the "clients" are in the same subnet as the real
servers). If these LVS packets aren't SNAT'd then the real server will send
the responses directly to the clients and TCP will break).
Given the current implementation, it is impossible to SNAT a packet
that LVS has modified. The ip_vs_post_routing() hook steals all LVS-handled
packets. Thus SNAT and conntrack never get to see these packets.
It's pretty clear in the code. I've also verified this empirically
by creating my own NF_IP_POST_ROUTING hook at the "location" where SNAT
would have been (NF_IP_PRI_NAT_SRC). In these cases, my hook never sees
anything that LVS has modified (but it sees everything else). If I remove
the LVS NF_IP_POST_ROUTING hook then my hook sees the packets and the
SNATing can occur.
> They are already SNAT-ed from LVS.
LVS only changes the source address for packets it forwards back to the
client -- it doesn't SNAT in the director-to-real_server direction -- which
is what is needed here.
> LVS does not touch the netfilter's conntrack packets,
> they reach unchanged post_routing where they can be SNAT-ed
> if such manipulation is scheduled from netfilter.
I don't think this is true. All NF_IP_INPUT packets that LVS modifies are
stolen by the ip_vs_post_routing before SNAT and conntrack get to see them.
Rodger Erickson
>
> Hello,
>
> On Tue, 29 Jan 2002, Rodger Erickson wrote:
>
> > After looking through your situation below, I believe that this
> > would work properly if you removed the post_routing hook
> from ip_vs_conn.c
> > (or did some #ifdef'ing to replace the contents of the
> existing hook with
> > "return NF_ACCEPT").
> >
> > This way your packets could continue through the
> netfilter mechanism
> > and be SNAT'd properly.
>
> They are already SNAT-ed from LVS.
>
> > I don't think that routing is an issue with SNAT if all
> you want to
> > do is masquerade the source address to force responses to
> come back through
> > a particular (and thus the correct route is the one you
> would have used
> > anyway).
>
> The NAT fails with multipath routes on route cache flush
>
> > As a caveat, I'm not 100% certain why the ip_vs_post_routing()
> > routine was put there (and what things would break by
> effectively removing
> > it). It seems to me that all it's doing is preventing
> conntrack entries
> > from being created for packets leaving the box. While this
> is a good thing
> > under most circumstances, it is certainly undesireable for
> your situation.
>
> LVS does not touch the netfilter's conntrack packets,
> they reach unchanged post_routing where they can be SNAT-ed
> if such manipulation is scheduled from netfilter.
>
> As for the LVS in->out NAT traffic, it is SNAT-ed at
> FORWARD and don't reach ip_nat_out. This is the goal we achieve
> by using ip_vs_post_routing.
>
> > Can one of the core LVS guys tell me if I'm missing
> something here?
> > I've done something similar to what I've described above
> and it seems to
> > work great for me in my environment.
> >
> >
> > Rodger Erickson
>
> Regards
>
> --
> Julian Anastasov <ja@xxxxxx>
>
|
