|
Hello,
Sorry, I missed to answer the first posting in this thread!
On Sun, 14 May 2000, jamal wrote:
>
>
> On Sun, 14 May 2000, jamal wrote:
>
> > If we talk about limiting system resources then LVS is contributing to
> > their consumption in both memory and CPU. Making a blanket statement that
> > it doesnt is kind of ridiculous.
> >
>
> OTOH, i think i might have misunderstood you (and you might have
> misunderstood the original poster). I apologize.
> I think you are saying that LVS in this case uses Linux masquareding (and
> therefore in itself doesnt add any extra code).
> I think though that the question was what kind of performance can you get
> out of the described PC (relative not to Linux but rather to some
> commercial load balancer).
If the packets are masqueraded the situations is worse
compared to the other forwarding methods. But we still can add
more NICs on the masq box and to increase the outgoing traffic.
If the problem is not the CPU. We can use many directors too
but this is not handled from LVS. It depends on the fact how the
requests come to the LVS Directors. May be every cluster software can
do it.
I think, nobody have compared LVS and other commercial
balancer. At least I don't know for such benchmarks. But the facts
are these:
- we use large hash table for connections, configured by default to 4096
rows, two pointers for row
- each entry allocates ~128 bytes => 512MB are 3,000,000 - 4,000,000
entries. You know, the TCP states have different timeouts which can be
tuned in the latest LVS versions.
If we must say what will be the connections/sec rate, lets
estimate it to 3,000,000/60 (60=FIN_WAIT) => 50000 requests/sec.
So, with 512MB RAM we have near 50000 req/sec. We didn't considered
the CPU overhead, DoS, how many seconds one requests is served.
We just estimated the max request rate based on the RAM and
the minimum entry life, i.e. if the connections are just created
and terminated. So, in normal cases the rate of 50,000 reqs/sec
must be considered as upper limit based on the 512MB RAM.
So, if you don't include the DoS attack in your calculations
512MB is enough. But! We don't know how many packets are
transferred for one connection. So, my answer is: it depends. The
connection rate is not enough, to answer the question "Is
512MB enough" we must ask for the connection life time.
In my tests with a web server configured with keepalive=off
the established and non-established connections in the LVS
table are 1:10 to 1:15. With default keepalive=15 seconds they
are ~ 3:5. This is one normal web server where the established
time is not 0 seconds. The actual number of the entries in
the LVS table are in /proc/sys/net/ipv4/ip_always_defrag.
When checking the RAM limits this value is very useful!
If the counter reaches 3,000,000 we are in dangerous area.
Also the memory state can be checked with "free".
About the CPU overhead. Yep, in 2.2 we can't be faster
on SMP. Some people try with "top" to show how free is the
CPU but that is not correct. By default we don't try to reach
the CPU limits :) We can use many directors to handle the
traffic. There are always solutions.
In 2.4 we will try to solve the problem by a new locking
strategy in the LVS code and the LVS will work better on SMP. The main
critical regions in LVS are the access to the connection table and the
schedulers. By this way we can better support multihomed SMP directors
and directors which are used as real servers too.
Some time ago I found in the mail list examples for the LVS
usage in very busy sites. I can't give examples now but everyone can try
to search the web site and the mailing list. In fact, not everyone
shows what traffic handles its site powered by LVS. We can only
estimate. May be there are many happy users for which we don't know :)
I have never found complains in the mailing list
about the fact that LVS code (the CPU overhead) is a bottleneck.
Not on 100Mbps at least :) The problem is that with small number
of clients the tests are not accurate. There must be one powerful
guy to test and later install the LVS in its production cluster and to
show the results :) And this is not difficult. The LVS code can be
downloaded from the web site. Soon for 2.4 too. Until then we
think LVS code is faster enough. May be there are other opinions
though, based on some tests.
Regards
--
Julian Anastasov <uli@xxxxxxxxxxxxxxxxxxxxxx>
|
