|
Hello,
On Mon, 29 Jan 2001, ratz wrote:
> Hi guys,
>
> This patch on top of ipvs-1.0.3-2.2.18 adds support for threshhold
> settings per realserver for all scheduler that have the -w option.
>
> Description/Purpose:
> --------------------
> I was always thinking of how a kernel based implementation of
> connection limitation per real server would work and how it could
> be implemented so while waiting in the hospital for the x-ray I
> had enough time to write up some little dirty hack to show a
> proof of concept. It works like follows. I added three new entries
> to the ip_vs_dest() struct, u_thresh and l_thresh in ip_vs.* and
> I modified the ipvsadm to add the two new options x and y.
> A typical setup would be:
>
> ipvsadm -A -t 192.168.100.100:80 -s wlc
> ipvsadm -a -t 192.168.100.100:80 -r 192.168.100.3:80 -w 3 -x 1145 -y 923
> ipvsadm -a -t 192.168.100.100:80 -r 192.168.100.3:80 -w 2 -x 982 -y 677
> ipvsadm -a -t 192.168.100.100:80 -r 127.0.0.1:80 -w 1 -x 100 -y 50
>
> So, this means, as soon as (dest->inactconns + dest->activeconns)
> exceed the x value the weight of this server is set to zero. As
> soon as the connections drop below the lower threshhold (y) the
> weight is set back to the initial value.
> What is it good for? Yeah well, I don't know exactly, imagine yourself,
> but first of all this is proposal and I wanted to ask for a discussion
> about a possible inclusion of such a feature or even a derived one into
> the main code (of course after fixing the race conditions and bugs and
> cleaning up the code) and second, I found out with tons of talks with
> customers that such a feature is needed, because also commercial lb
> have this and managers always like to have a nice comparision of all
> features to decide which product they take. Doing all this in user-
> space is unfortunately just not atomic enough.
Yes, greping for the connection counters from the /proc fs
is not good :)
> Anyway, if anybody else thinks that such a feature might be vital
> for inclusion we can talk about it. If you look at the code, it
> wouldn't break anything and just add two lousy CPU cycles for checking
> if u_thresh is < 0. This feature can easily be disabled by just
> setting u_thresh to zero or not even initialize it.
>
> Well, I'm open for discussion and flames. I have it running in
> production :) but with a special SLA. I implemented the last
How we will defend against DDoS? Using the number of active
or inactive connections to assign a new weight is _very_ dangerous.
In theory, the number of connections is related to the load but
this is true when the world is ideal. The inactive counter can
be set with very high values when we are under attack. Even the WLC
method loads proporcionally the real servers but they are never
excluded from operation.
I have some thoughts about limiting the traffic per
connection but this idea must be analyzed. The other alternatives
are to use the Netfilter's "limit" target or QoS to limit the
traffic to the real servers.
> server of resort which works like this: If all RS of a service
> are down (healthcheck took it out or treshhold check set weight
> to zero), my userspace tool automagically invokes the last
> server of resort, a tiny httpd with a static page saying that
> the service is currently unavailable. This is also useful if you
> want to do maintainance of the realservers.
Yes, user space solution :)
> I already implemented a dozen of such setups and they work all
> pretty well.
Let's analyze the problem. If we move new connections from
"overloaded" real server and redirect them to the other real servers we
will overload them too. IMO, the problem is that there are
more connection requests than the cluster can handle. The solutions
to try to move the traffic between the real servers can only cause
more problems. If at the same time we set the weights to 0 this
leads to more delay in the processing. May be more useful is to
start to reduce the weights first but this again returns us to
the theory for the smart cluster software.
So, we can't exit from this situation without dropping
requests. There is more traffic that can't be served from the cluster.
If there is no cluster software to keep the real servers equally
loaded, some of them can go offline too early.
The cluster software can take the role to monitor the load
instead of relying on the connection counters. I agree, changing the
weights and deciding how much traffic to drop can be explained
with a complex formula. But I can see it only as a complete solution:
to balance the load and to drop the exceeding requests, serve as many
requests as possible. Even the drop_packet strategy can help here,
we can explicitly enable it specifying the proper drop rate. We don't
need to use it only to defend the LVS box but to drop the exceeding
traffic. But someone have to control the drop rate :) If there is no
exceeding traffic what problems we can expect? Only from the bad load
balancing :)
> Best regards,
> Roberto Nibali, ratz
>
> --
> mailto: `echo NrOatSz@xxxxxxxxx | sed 's/[NOSPAM]//g'`
Regards
--
Julian Anastasov <ja@xxxxxx>
|
