I would like to use IPVS with the lblc scheduler and limit the number of
ACTIVE connections to a given server to a relatively small number (25)
using the upper threshold option to ipvsadm.
However, based on both my testing and reading of the code, the upper
threshold is checked against the total connections (Inactive+Active):
if (dest->u_threshold != 0 &&
ip_vs_dest_totalconns(dest) >= dest->u_threshold)
dest->flags |= IP_VS_DEST_F_OVERLOAD;
Is this a bug or there a good reason why it is checked against total
connections instead of active connections? I can't come up with a
reason why Inactive connections would be counted against the total
connections to a server in this situation but smarter folks then I wrote
IPVS so I figured it would make sense to ask :)
I am trying to limit connections to realservers running an Apache-based
application with a rather large memory footprint per child. We have the
children limited to 25 in Apache and I would like to use upper
thresholds to make IPVS aware of this as well so that with the lblc
scheduler it can know to grab a new realserver.
What I have is 10 servers, each of which can support 25 connections at a
time. I need some affinity to the server for application reasons but I
would like to be able to grow to use the whole set.
Right now I have 5 IP-based pools, each containing 2 servers, and using
a mechanism to somewhat manually distribute traffic to each of the 5
IPs. If one of the IPs gets hit with a lot of traffic then those two
servers max out even if there are spare resources on the other servers.
What I had envisioned doing is to create one FWMark-based service and
use iptables to associate that one mark with the 5 IP address. Then use
lblc with upper limit thresholds to keep server affinity under normal
load conditions, but allow for up to the whole pool to be included if
the maximum number of connections on a given real server (25) is
reached. Other then the code including inactive connections in the
count, it seems to do exactly what I had hoped in limited testing.
Is there anyway to do this without modifying the IPVS code? If I change
the above threshold checks in the code to only consider active
connections will I be making a terrible mistake?
Any advise is appreciated.