LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

[lvs-users] LVS / RHEL 5.3 / iptables problem

To: <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: [lvs-users] LVS / RHEL 5.3 / iptables problem
From: devin <devin@xxxxxxxxxxxxxxxx>
Date: Mon, 29 Nov 2010 14:54:32 -0700
 We use LVS/Keepalived using DR (Direct Routing) running stock IPVS with 
 the size=4096, we have about
 35 servers in a single cluster using LVS to answer for both port 80 and 
 port 443 traffic. We are pushing
 around 100 Mbps out, and 80 Mbps in currently through our firewalls 
 which is all serving this cluster.

 I know with using the DR (Direct Routing) method of balancing that the 
 packet comes into the load balancer
 initially but doesn't actually see the packet going back out, as it 
 comes directly from the web server.

 However sometimes we have clients who send us crazy amounts of traffic 
 and we end up having to block the
 traffic at the load balancer/keepalived server using IPTABLES. We 
 basically configure IPTABLES to send
 a TCP RESET packet back to the user's browser so that the browser will 
 give up trying to connect to our
 cluster.

 This is the command we run on Linux to do the block.

 /sbin/iptables -A INPUT -p tcp -m tcp --dport 80 -m string --string 
 "sid=3225" --algo bm --to 65535 -j REJECT --reject-with tcp-reset

 So I know sometimes this messes with the ActiveConn count, and I am not 
 sure if this is affecting LVS in any
 way as we have to sometimes block a user for an entire day. So the 
 problem I notice with this is that the “Activeconn” count
 just keeps building up as we are rejecting the traffic for a while but 
 does seem to cap off at 32,000 connections per server right now.

 I wasn’t sure if there is a way to better let LVS know to consider the 
 connection InActiveConn so it doesn’t use up memory.
 Because if I am correct in assuming that LVS is consuming some memory 
 for the connection that it see’s as an incoming connection,
 But because we are doing a TCP RST, LVS doesn’t clear out that 
 connection.

 Please correct me if I am wrong in my assumption and shouldn’t need to 
 worry??

 IP Virtual Server version 1.2.1 (size=4096)
 Prot LocalAddress:Port Scheduler Flags
   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
 TCP  pixel.host.com:http wrr
   -> box70.domain:http Route   2      31787      9127
   -> box71.domain:http Route   2      32233      9681
   -> box48.domain:http Route   2      32266      9658
   -> box73.domain:http Route   2      32062      9658
   -> box83.domain:http Route   2      32430      9672
 .... total of 35 servers listed here.

 Any ideas would be greatly appreciated here. Would there be any 
 improvement to change the size of the 4096 to something
 higher? The load balancer server we are using has 4GB of Ram, and Quad 
 Core 2.5Ghz processor, we are not using the box
 for anything else but to run the "keepalived" process and IPVS.

 Thanks so much.

 Devin Acosta

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>