LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Multiple external interfaces on the director

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Multiple external interfaces on the director
From: Ken Brownfield <krb@xxxxxxxxxxx>
Date: Wed, 15 Mar 2006 02:52:29 -0800
On Mar 14, 2006, at 7:00 PM, Ken Brownfield wrote:
On Mar 14, 2006, at 2:49 PM, Joseph Mack NA3T wrote:
Julian's nfct code is not used much so we don't hear a lot about it. It came after the -SH scheduler. Maybe the -SH scheduler shouldn't be neede if the netfilter problems really have been cleaned up.

Yes, the route_me_harder() function in the nfct code seems promising. I fear I'm going to have to grab the source and track down the routing behavior specifically.

Scanning the nfct patch and looking at the icmp handling, I'm pretty sure the problem is that ip_vs_out() is sending out the packet with a route calculated from the real server's IP. Since ip_vs_out() is reputedly only called for masq return traffic, I think this is just plain incorrect behavior.

I pulled out the route_me_harder() mod and created the attached patch. My only concern would be performance, but it seems netfilter's NAT uses this.

I'll try to set up a tiny test environment tomorrow. Assuming I'm not all wet!
--
Ken.



--- ip_vs_core.c        2006/03/15 08:10:01     1.1
+++ ip_vs_core.c        2006/03/15 08:41:48
@@ -625,6 +625,42 @@
        return NF_ACCEPT;
}
+/* This code stolen from ip_nat_standalone.c, as is the
+ * following comment:
+ *
+ * FIXME: change in oif may mean change in hh_len.  Check and realloc
+ * --RR
+ */
+static inline int
+ip_vs_route_me_harder(struct sk_buff *skb)
+{
+       struct iphdr *iph = skb->nh.iph;
+       struct rtable *rt;
+       struct rt_key key = { dst:iph->daddr,
+                             src:iph->saddr,
+                             oif:skb->sk ? skb->sk->bound_dev_if : 0,
+                             tos:RT_TOS(iph->tos)|RTO_CONN,
+#ifdef CONFIG_IP_ROUTE_FWMARK
+                             fwmark:skb->nfmark,
+#endif
+                           };
+
+       /* Note that ip_route_output_key() makes routing
+        * decisions assuming that the packet has originated
+        * from this machine itself.  This is the correct
+        * behaviour for outgoing VS/NAT traffic.
+        */
+       if (ip_route_output_key(&rt, &key) != 0) {
+               printk("ip_vs_route_me_harder(): No more route.\n");
+               return -EINVAL;
+       }
+
+       /* Drop old route. */
+       dst_release(skb->dst);
+       skb->dst = &rt->u.dst;
+
+       return 0;
+}
/*
  *     It is hooked at the NF_IP_FORWARD chain, used only for VS/NAT.
@@ -643,6 +679,7 @@
        struct ip_vs_conn *cp;
        int size;
        int ihl;
+       int retval;
        EnterFunction(11);
@@ -812,8 +849,20 @@
        skb->nfcache |= NFC_IPVS_PROPERTY;
+       /* For policy routing, packets originating from this
+        * machine itself may be routed differently to packets
+        * passing through.  We want this packet to be routed as
+        * if it came from this machine itself.  So re-compute
+        * the routing information.
+        */
+       if (ip_vs_route_me_harder(skb) == 0)
+               retval = NF_ACCEPT;
+       else
+               /* No route available; what can we do? */
+               retval = NF_DROP;
+
        LeaveFunction(11);
-       return NF_ACCEPT;
+       return retval;
}


<Prev in Thread] Current Thread [Next in Thread>