LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Netfilter connection tracking support for IPVS

To: "Nicklas Bondesson" <nicklas.bondesson@xxxxxxxxxxxx>
Subject: Re: Netfilter connection tracking support for IPVS
Cc: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
From: Janusz Krzysztofik <jkrzyszt@xxxxxxxxxxxx>
Date: Fri, 23 Feb 2007 18:13:28 +0100
Dnia piątek, 23 lutego 2007 01:29, Nicklas Bondesson napisał:
> ... I am however still unable to SNAT
> traffic leaving the box. I'm runnng the director and firewall on the same
> box.
> ...
> And this is how I do SNAT: iptables -t nat -A POSTROUTING -o eth0 -j SNAT
> --to-source 11.22.33.44
>
> Any suggestions?

Hello everybody,

Niklas, if you mean masquerading of LVS-DR client IPs on their way to real 
servers, you can try my approach described below.

When I tried Julian's patch several month ago (I am not sure if this have 
changed), I found it not suitable for use on a director that would also do 
masquerading (SNAT) of client IPs. I have learned that when Julian 
says "SNAT" he means processing of packets coming from an LVS-NAT driven real 
server (OUT direction) while forwarding them to clients. IN direction packets 
never pass through the netfilter nat POSTROUTING hook (and conntrack 
POSTROUTING as well), the are sent out directly by ip_vs_out() with optional 
ip_vs_conntrack_confirm().

Some time ago I have set up an LVS-DR based internet gateway that not only 
accepts connections from the internet to a VIP and redirects them to several 
RIPs (typical IPVS usage), but also rediects connections from intranet 
clients to the internet through serveral DSL/FrameRelay links, or their 
respective routers acting as real servers (similiar to LVS driven transparent 
cache cluster case). As I have no control over these routers (they are 
managed by their respective providers), I have to do masquerading (or SNAT) 
on the director itself to avoid putting several more boxes in between. In 
order to achieve this functionality, I have started with a "hardware" method 
of sending IN packets back to the director via several vlans set up over 2 
additional network interfaces connected with a crossover cable. Then I have 
created a small patch that affects processing of LVS-DR packets only (and 
bypass as well), so they are not caught by ip_vs_out() and just travel 
through all netfilter POSTROUTNG hooks, including nat and conntrack. This 
solution works for me as expected.

In my opinion, Julian's patch is particularily suitable for LVS-NAT, where any 
other approach would probably not work at all. Furthermore, it looks for me 
that Julian's way (or maybe any way) of connection tracking could be not 
applicable to LVS-TUN, where packets leaving the director are encapsulated 
before they reach ip_vs_out(). But for LVS-DR there are probably two good 
ways at least: Julian's, but without masquerading, and my own, that I have 
successfully used for several months now.

My patch applies cleanly against debian linux-source-2.6.18-3 version 2.6.18-7 
and is also available at 
http://www.icnet.pl/download/ip_vs_dr-conntrack.patch

Cheers,
Janusz

Signed-off-by: Janusz Krzysztofik <jkrzyszt@xxxxxxxxxxxx>
================================================
--- linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_core.c.orig    
2006-06-18 03:49:35.000000000 +0200
+++ linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_core.c 
2006-10-21 21:38:20.000000000 +0200
@@ -672,6 +672,9 @@ static int ip_vs_out_icmp(struct sk_buff
        if (!cp)
                return NF_ACCEPT;

+       if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_DROUTE)
+               return NF_ACCEPT;
+
        verdict = NF_DROP;

        if (IP_VS_FWD_METHOD(cp) != 0) {
@@ -801,6 +804,9 @@ ip_vs_out(unsigned int hooknum, struct s
                return NF_ACCEPT;
        }

+       if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_DROUTE)
+               return NF_ACCEPT;
+
        IP_VS_DBG_PKT(11, pp, skb, 0, "Outgoing packet");

        if (!ip_vs_make_skb_writable(pskb, ihl))
--- linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_xmit.c.orig    
2006-06-18 03:49:35.000000000 +0200
+++ linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_xmit.c 
2006-10-21 21:22:56.000000000 +0200
@@ -127,7 +127,6 @@ ip_vs_dst_reset(struct ip_vs_dest *dest)

 #define IP_VS_XMIT(skb, rt)                            \
 do {                                                   \
-       (skb)->ipvs_property = 1;                       \
        (skb)->ip_summed = CHECKSUM_NONE;               \
        NF_HOOK(PF_INET, NF_IP_LOCAL_OUT, (skb), NULL,  \
                (rt)->u.dst.dev, dst_output);           \
@@ -278,6 +277,7 @@ ip_vs_nat_xmit(struct sk_buff *skb, stru
        /* Another hack: avoid icmp_send in ip_fragment */
        skb->local_df = 1;

+       skb->ipvs_property = 1;
        IP_VS_XMIT(skb, rt);

        LeaveFunction(10);
@@ -411,6 +411,7 @@ ip_vs_tunnel_xmit(struct sk_buff *skb, s
        /* Another hack: avoid icmp_send in ip_fragment */
        skb->local_df = 1;

+       skb->ipvs_property = 1;
        IP_VS_XMIT(skb, rt);

        LeaveFunction(10);
@@ -542,6 +543,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, str
        /* Another hack: avoid icmp_send in ip_fragment */
        skb->local_df = 1;

+       skb->ipvs_property = 1;
        IP_VS_XMIT(skb, rt);

        rc = NF_STOLEN;

<Prev in Thread] Current Thread [Next in Thread>