|
On 18.06.2014, Stephen Carville wrote:
> I set up a CentOS 6.5 box to test ipvsadm. So far I have been unable to
> get it to forward connections. When I try to connect, it doesn't write
> anything in /var/log/messages to tell me what is happening. Netstat
> doesn't see anything listening on the interface IP (I read elsewhere
> that is normal) and tshark sees the incoming SYN but there is either a
> timeout or a RST.
>
> Rules right now:
>
> $ ipvsadm -L
>
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 10.212.160.40:4172 lc persistent 360
> -> 10.212.170.162:4172 Route 1 0 0
> -> 10.212.170.163:4172 Route 1 0 0
>
> IP forwarding is turned on:
>
> $ sysctl net.ipv4.ip_forward
> net.ipv4.ip_forward = 1
Short answer: switch to kernel 3.6 or newer, turn off rp_filter for the
interface receiving the reply packet, and replace rp_filter functionality by
more accurate and flexible iptables rules in the FORWARD chain.
Long:
When you're using direct routing/gatewaying or tunneling, your replies will
have the VIP address as the source address of the packet. They're routed via
your balancer, who also knows the same IP address to be a local address.
Depending on /proc/sys/net/ipv4/conf/*/rp_filter or kernel version, you need to
either change a little bit of configuration or apply special kernel patches
(forward_shared).
If you're running a linux kernel <3.6, your kernel will always drop those
packets as being spoofed. If you're turning on net/ipv4/conf/*/log_martians,
you'll also see kernel log messages on this.
>From the stone age of IPVS, there is a kernel patch introducing a sysctl
>switch (forward_shared) to turn off this behaviour for selected interfaces. If
>misconfigured (e.g. "turn it on for all interfaces"), this patch will
>introduce an option for third parties to spoof packets onto your network
>(which is something you don't want to do).
http://www.ssi.bg/~ja/ does offer recent patches for about any major linux
kernel source from 2.4.19 to 3.15, just in case you're interested.
Linux Kernel 3.6 did somehow obsolete the forward_shared-patch by changing the
behaviour of rp_filter for the interface receiving the spoofed packet:
-if rp_filter is turned on and ip_forward=1, the kernel will most likely drop
your reply packets, as they're most likely received by the 'wrong' interface:
an interface without a matching entry in the routing table for this IP address.
Given the point many linux distros do enable rp_filter per default, you most
likely hit that wall.
-if rp_filter is turned off (=0) and ip_forward=1, the kernel will gladly
forward your "spoofed" reply packets.
The exact use of rp_filter on a dedicated load balancer is fairly limited, if
at all. It's much better to replace its functionality by dedicated iptables
rules and restricting the FORWARD chain.
For your configuration, this results in the following configuration changes:
$ iptables -P FORWARD DROP
$ iptables -A FORWARD -i eth1 -o eth0 --source 10.212.160.40 -j ACCEPT
$ sysctl -w proc/sys/net/ipv4/conf/eth1/rp_filter=0
If you're keen on extra-tight security, you may also copy the permissive line
and add "-m --mac-source $realserver-mac" for every realserver. This ensures
only your real servers may send "spoofed" traffic via your balancer.
Best,
Anders
--
1&1 Internet AG Expert Systems Architect (IT Operations)
Brauerstrasse 50 v://49.721.91374.0
D-76135 Karlsruhe f://49.721.91374.225
Amtsgericht Montabaur HRB 6484
Vorstand: Ralph Dommermuth, Frank Einhellinger, Robert Hoffmann,
Andreas Hofmann, Markus Huhn, Hans-Henning Kettler, Uwe Lamnek,
Jan Oetjen, Christian Würst
Aufsichtsratsvorsitzender: Michael Scheeren
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
