|
Didn't include lvs-devel in CC of the original message, adding it now...
On Tue, Sep 23, 2008 at 3:54 PM, Julius Volz <juliusv@xxxxxxxxxx> wrote:
> On Tue, Sep 23, 2008 at 3:21 PM, Jan Engelhardt <jengelh@xxxxxxxxxx> wrote:
>>
>> On Tuesday 2008-09-23 09:13, Julius Volz wrote:
>>>
>>>Ok, the SYN/ACK from the backend is logged as --cstate INVALID in
>>>PREROUTING and INPUT. This means that Netfilter thinks it doesn't
>>>belong to any connection, although it just SNATed the SYN to the
>>>backend correctly? Hmm... how can this be?
>>
>> That probably means skb->nfct is lost (set to NULL, which is what INVALID
>> indicates) after SNAT (PREROUTING), when IPVS kicks in.
>
> But at PREROUTING, the skb hasn't seen any part of IPVS yet. And the
> prior SNAT for the SYN packet happened in POSTROUTING after all of
> IPVS was finished. Basically, there should be no IPVS involved from
> the time that the SYN is correctly SNATed and the time that the
> SYN/ACK is not. Please correct me if I'm missing something here...
Ok, after lots of debugging, I've found the problem and got SNAT
working with IPVS (in a very hacky way).
The problem was that Netfilter already assigns a conntrack entry (in
skb->nfct) to the CIP->VIP SYN packet in PREROUTING. This contains a
tuple which contains source and destination addresses as seen by
Netfilter during PREROUTING (before IPVS). IPVS then modifies the
destination address without updating the associated skb->nfct entry.
Later, Netfilter SNAT in POSTROUTING finds the already assigned tuple
and updates it with the new source address, but the destination
address changed by IPVS is never updated in this tuple. This is why
this wrong conntrack entry does not match the SYN/ACK packet returning
from the backend server.
So as a proof of concept for IPVS SNAT, it is enough to remove the
ip_vs_post_routing() function (to let Netfilter handle anything at all
in POSTROUTING) and put a simple "skb->nfct = NULL;" into the IPVS NAT
transmitter function, ip_vs_nat_xmit(), to let the conntrack tuple be
regenerated in by Netfilter POSTROUTING. Of course, this is a memory
leak, the conntrack hash entries aren't removed, etc.
I guess to do this correctly, we should probably update the skb->nfct
entry in IPVS while doing our manual DNAT?
I realize there's a big patch integrating IPVS with conntrack
(http://www.ssi.bg/~ja/nfct/) in some way, but it seems like that does
much more than would be needed for SNAT.
Julius
--
Julius Volz - Corporate Operations - SysOps
Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1
--
To unsubscribe from this list: send the line "unsubscribe lvs-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
|
