Re: Network namespace, ipvlan and IPVS source NAT

To: Julian Anastasov <ja@xxxxxx>
Subject: Re: Network namespace, ipvlan and IPVS source NAT
Cc: Damien Claisse <d.claisse@xxxxxxxxxx>, "lvs-devel@xxxxxxxxxxxxxxx" <lvs-devel@xxxxxxxxxxxxxxx>, "netfilter-devel@xxxxxxxxxxxxxxx" <netfilter-devel@xxxxxxxxxxxxxxx>
From: Mahesh Bandewar (महेश बंडेवार) <maheshb@xxxxxxxxxx>
Date: Tue, 20 Oct 2020 21:41:34 -0700
On Tue, Oct 20, 2020 at 11:01 AM Julian Anastasov <ja@xxxxxx> wrote:
>         Hello,
> On Tue, 20 Oct 2020, Damien Claisse wrote:
> > I'm trying to understand a limitation in ipvlan/netfilter that prevents 
> > doing routed IPVS with source NAT inside a namespace.
> >
> > Setup is the following: there is an "lvs" namespace, with an ipvlan 
> > interface (in l3 mode) moved to this namespace, linked to physical 
> > interface. Goal is to isolate load balanced traffic in a separate namespace.
> > This setup is working flawlessly with l3 routed IPIP encapsulation, but I 
> > also have a use case for applications that don't support encapsulation, 
> > hence the need to do l3 routed load balancing with source NAT.
> > Issue is that if I put iptables NAT rule in namespace using ipvs module, 
> > nothing happens, packet is forwarded but source IP is not translated. It 
> > seems like netfilter is blind to ipvlan l3 traffic. I also tried using 
> > "l3s" mode that should to go through netfilter, but in that case, packets 
> > for virtual IPs are rejected with a TCP reset. Virtual IPs in namespace 
> > seem not visible to this mode.
> >
> > I'm wondering what would be the best way to make it happen:
> > - patch ipvlan to lookup for VIPs in namespaces
> > - patch netfilter ipvs NAT module to translate in root namespace
> > - any other better idea is welcome
> >
> > Please find below commands to reproduce the issue. In this example physical 
> > load balancer interface is enp4s0, virtual IP is (to be 
> > exported by a routing protocol, or route manually added to a client in same 
> > subnet as load balancer for testing), load balancer IP is, 
> > and real server IP is
> >
> > - In root namespace:
> > ip netns add lvs
> > ip link add ipvlan0 link enp4s0 type ipvlan mode l3s
> > ip link set ipvlan0 up
> > ip link set ipvlan0 netns lvs
> >
> > - In lvs namespace (ip netns exec lvs bash):
> > ip addr add dev ipvlan0
> > ip route add default via dev ipvlan0 onlink
> > ipvsadm -A -t -s rr
> > ipvsadm -a -t -r -m
> > iptables -t nat -A POSTROUTING -m ipvs --vaddr --vport 80 
> > -j SNAT --to-source
>         Make sure /proc/sys/net/ipv4/vs/conntrack is set to 1
> to allow IPVS to keep the netfilter conntracks for its
> packets. This is supported with the CONFIG_IP_VS_NFCT=y
> kernel option.
> > What I'd expect: a packet outgoing from enp4s0 with source IP 
> > and destination IP
> > What I see from a test client:
> > - in l3 mode: a packet outgoing from enp4s0 with source client IP address 
> > and destination (hence missing source NAT). I also don't see 
> > any conntrack event when doing conntrack -E
> > - in l3s mode: connection reset sent to client. While reading l3s 
> > implementation, I wonder where route lookup is done in ipvlan_l3_rcv, it 
> > seems that namespaces' virtual IP addresses are not visible during this 
> > lookup, hence the TCP RST.
Unfortunately iptables and IPvlan don't do well together, especially
the L3* modes. L3s mode is a little better but please keep in mind
iptables functionality that involves "forwarding" path, is simply not
compatible with IPvlan L3* modes. Your best bet is to use L2 mode and
perform all IPtables operations inside the namespace if possible.


> Regards
> --
> Julian Anastasov <ja@xxxxxx>

<Prev in Thread] Current Thread [Next in Thread>