|
More info. I now realize that these dropped packets are FIN and RST ACKs
being blocked, probably because my rules to the VIP include: -m state
--state NEW -j ACCEPT. Can these dropped packets affect the TCP
connections, resulting in client connection issues?
After further testing it appears that most of the connection issues are
those that go over the ldirector. I've personally seen a case where a
client is 3 hops away experiencing catastrophic connection issues
(sometimes it works, but the site is unusable). Whereas most are not
experiencing these problems. If I bypass the ldirector it appears that
things work a lot better (some http connection issues) and if I bypass the
lvs/director altogether (different hardware, low traffic), things work fine.
Seems a bit like:
http://archive.linuxvirtualserver.org/html/lvs-users/2006-07/msg00184.html
Any comments to my iptable rules (I'm no expert). Any high-traffic,
Iptables LVS-NAT rules / best practices out there anyone can share?
Thanks, Brent
At 01:45 PM 8/2/2010 -0700, you wrote:
>I'm running LVS-NAT / Ldirector with the firewall running on the same
>boxes. I've been running a similar iptables script on a 2.4.x kernel
>(heartbeat 1.2.3) for years and have moved to a 2.4.18x kernel (Centos 5)
>with heartbeat 3.0.3 & ldirector v1.186-ha. However, outbound real-server
>traffic failed until I added a "fix" below on the firewall rules (see below).
>
>(1) Ultimate problem: I'm getting a small % of clients with sporadic
>(sometimes high at certain times) dropped http connections, the majority of
>users seem to be across the pond; however, I have not been able to rule out
>an problems on my side. I personally have never seen any connection
>problems. I'm not necessary asking help for this, but first need to address
>the following:
>
>
>(2) I see some incoming dropped packets that should seem to be legitimate
>(this also existed on the 2.4 version). Here's a sample:
>
>CATCHALL -- DENY IN=eth0 OUT= MAC=xxx SRC=<outside_ip1> DST=<vip> LEN=52
>TOS=0x00 PREC=0x00 TTL=54 ID=52546 DF PROTO=TCP SPT=32852 DPT=80
>WINDOW=33120 RES=0x00 ACK FIN URGP=0
>CATCHALL -- DENY IN=eth0 OUT= MAC=xxx SRC=<outside_ip2> DST=<vip> LEN=40
>TOS=0x00 PREC=0x20 TTL=55 ID=37057 DF PROTO=TCP SPT=33069 DPT=80 WINDOW=27
>RES=0x00 ACK FIN URGP=0
>CATCHALL -- DENY IN=eth0 OUT= MAC=xxx SRC=<outside_ip3> DST=<vip> LEN=52
>TOS=0x00 PREC=0x00 TTL=54 ID=48415 DF PROTO=TCP SPT=56014 DPT=80
>WINDOW=1002 RES=0x00 ACK FIN URGP=0
>CATCHALL -- DENY IN=eth0 OUT= MAC=xxx SRC=<outside_ip4> DST=<vip> LEN=40
>TOS=0x00 PREC=0x20 TTL=51 ID=10009 DF PROTO=TCP SPT=8177 DPT=80
>WINDOW=65535 RES=0x00 ACK FIN URGP=0
>
>The outside IPs are legitimate going to a legitimate VIP. The drops are
>sporadic and there doesn't appear to be any connections problems for the
>majority of users, even if I see my own IPs getting dropped. Is this
>normal? And why? I never see any drops on non-ldirector VIPs (except for
>unwanted traffic).
>
>
>(3) What are the best iptables practices for allowing the VIP traffic? I
>haven't seen many LVS-NAT firewall scripts so I don't know if my chains are
>good (especially per the "Fix" below, which was required to allow the
>Ldirector VIP connections). I've seen a lot of stuff over the years but am
>not sure with the newer kernels (okay, RHEL5 kernels are rather old).
>
>Here is part of my condensed iptables script:
>
>...
>
>#
># Allow connections already established
>#
>$IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>$IPTABLES -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>$IPTABLES -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
>
>#
># allow everything on loopback
>#
>$IPTABLES -A INPUT -i lo -j ACCEPT
>$IPTABLES -A OUTPUT -o lo -j ACCEPT
>
>#
># HTTP VIP example
>#
>$IPTABLES -A INPUT -p tcp -d <vip> --destination-port 80 -m state --state
>NEW -j ACCEPT
>
>...
>More stuff here (e.g. chains to other non-RIPs)
>...
>
>#
># NAT Rule
>#
>$IPTABLES -t nat -A POSTROUTING -j MASQUERADE -s $INET
>
>
>#
># The Fix. Works to allow outgoing VIP traffic. Not sure if this is the
>best. Required for 2.6.18x. Didn't need it on 2.4x
># eth0 is the outside interface, eth1 is the inside interface.
>#
>$IPTABLES -A OUTPUT -o eth1 -d $INET -j ACCEPT
>
>
>#
># 'masquerading' rule
>#
>$IPTABLES -A INPUT -s $INET -j ACCEPT
>$IPTABLES -A OUTPUT -s $INET -j ACCEPT
>$IPTABLES -A FORWARD -s $INET -j ACCEPT
>
>
>#
># Allow stuff originating from the firewall
>#
>$IPTABLES -A OUTPUT -s $OIP -m state --state NEW -j ACCEPT
>
>
>#
># 'catch all' rule
>#
>$IPTABLES -N CATCHALL
>$IPTABLES -A OUTPUT -j CATCHALL
>$IPTABLES -A INPUT -j CATCHALL
>$IPTABLES -A FORWARD -j CATCHALL
>$IPTABLES -A CATCHALL -j LOG --log-level debug --log-prefix "CATCHALL --
>DENY "
>$IPTABLES -A CATCHALL -j DROP
>
>
>Thanks!
>
>Brent
>
>
>_______________________________________________
>Please read the documentation before posting - it's available at:
>http://www.linuxvirtualserver.org/
>
>LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
>Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
>or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
