|
We use LVS/Keepalived using DR (Direct Routing) running stock IPVS with
the size=4096, we have about
35 servers in a single cluster using LVS to answer for both port 80 and
port 443 traffic. We are pushing
around 100 Mbps out, and 80 Mbps in currently through our firewalls
which is all serving this cluster.
I know with using the DR (Direct Routing) method of balancing that the
packet comes into the load balancer
initially but doesn't actually see the packet going back out, as it
comes directly from the web server.
However sometimes we have clients who send us crazy amounts of traffic
and we end up having to block the
traffic at the load balancer/keepalived server using IPTABLES. We
basically configure IPTABLES to send
a TCP RESET packet back to the user's browser so that the browser will
give up trying to connect to our
cluster.
This is the command we run on Linux to do the block.
/sbin/iptables -A INPUT -p tcp -m tcp --dport 80 -m string --string
"sid=3225" --algo bm --to 65535 -j REJECT --reject-with tcp-reset
So I know sometimes this messes with the ActiveConn count, and I am not
sure if this is affecting LVS in any
way as we have to sometimes block a user for an entire day. So the
problem I notice with this is that the “Activeconn” count
just keeps building up as we are rejecting the traffic for a while but
does seem to cap off at 32,000 connections per server right now.
I wasn’t sure if there is a way to better let LVS know to consider the
connection InActiveConn so it doesn’t use up memory.
Because if I am correct in assuming that LVS is consuming some memory
for the connection that it see’s as an incoming connection,
But because we are doing a TCP RST, LVS doesn’t clear out that
connection.
Please correct me if I am wrong in my assumption and shouldn’t need to
worry??
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP pixel.host.com:http wrr
-> box70.domain:http Route 2 31787 9127
-> box71.domain:http Route 2 32233 9681
-> box48.domain:http Route 2 32266 9658
-> box73.domain:http Route 2 32062 9658
-> box83.domain:http Route 2 32430 9672
.... total of 35 servers listed here.
Any ideas would be greatly appreciated here. Would there be any
improvement to change the size of the 4096 to something
higher? The load balancer server we are using has 4GB of Ram, and Quad
Core 2.5Ghz processor, we are not using the box
for anything else but to run the "keepalived" process and IPVS.
Thanks so much.
Devin Acosta
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
