|
Hello,
On Thu, 25 Jul 2002, Stefan Peter wrote:
> I tested my cluster using testlvs code and I found something interesting:
>
> 1. I have two distributor a master and a backup one. When I was spoofing
> the cluster with large number of sources and large number of packets the
> two distributors were alternating their functions at aprox. 5-10 seconds.
> First the master distributor produced large kernel load (even my shell on
> that machine has gone away), then the backup took its role, it got staffed
> as did the master in a couple of seconds, but by this time the master
> had come back again. Can I accept this as normal behavior?
Normal/Expected, yes. Some loads don't allow the processes
to work, you need a kernel space redundancy solution or faster
hardware :)
> 2. The measurement values were also interesting: on the distributor
> machine I measured 26000 packets/sec, as long as on the RSs I measured
26,000KP/s is typical performance result with LVS
when PCI is 33MHz. With 66MHz PCI you can achieve far more.
> 3000 packets/sec. I have 4 RS, and I cannot figure out where the packets
> have been lost? 26000!=4*3000. On top of all, on the client I measured
They were dropped. The kernel simply can not keep the
in and out packets with same rate, the incoming rate is higher.
You have to see for any counters in the receiving path, why not
to patch the kernel just to see where they are lost :) I simply
didn't tried it :)
> even larger speed. I looked into the switches, but I found no packet loss
> or error.
> (I used TCP spoof with srcnum 500000 and packets 10^8.)
500K is too high value, it creates high load for the
routing cache which is usually tuned for ... max 16K entries,
there are some parameters to tune the cache management but
I'm not sure it can help for such load. Try with small number
of spoofed client IPs, 10, 1000, 10000.
> (Machines are IBM330s, 512MBs of RAM, 1GHz Pentiums inside, RedHat 7.3
> with 2.4.19-prexx kernel, the network is 100Mbit swiched network.)
>
> This may give explanation to the packet loss, at least in my
> interpretation. Question again: is it normal operation? (Though spoofing
> is abnormal.) How bottlenecks can be determined?
Latest 2.4 kernels maintain counters for the places where
the packets are dropped. May be you can look at
/proc/net/softnet_stat, write a program that shows rates
by reading these counters and may be that can help. Of
course, there can be other counters, I remember for such
kernel changes but never looked at them.
> 3. An off-topic question:
>
> How many VIPs can be configured. On what property it depends?
How much you need? There are no explicit limits. May be
only in the hash table rows (there are 2 tables for virtual
services):
#define IP_VS_SVC_TAB_BITS 8
> Regards,
> Peter
Regards
--
Julian Anastasov <ja@xxxxxx>
|
