|
On Fri, Sep 09, 2022 at 01:21:05AM +0300, Julian Anastasov wrote:
> It is interesting to know what value for
> IPVS_EST_TICK_CHAINS to use, it is used for the
> IPVS_EST_MAX_COUNT calculation. We should determine
> it from tests once the loops are in final form.
> Now the limit increased a little bit to 38400.
> Tomorrow I'll check again the patches for possible
> problems.
I couldn't wait so I have run tests on various machines and used the
sched_switch tracepoint to measure the time needed to process one chain. The
table contains a median time for processing one chain, the maximum time
measured, the median divided by the number of CPUs and the time needed to
process one chain if there were 1024 CPUs of that type in a machine:
> NR CPU Time(ms) Max(ms) Time/CPU(ms) 1024
> CPUs(ms)
> 48 Intel Xeon CPU E5-2670 v3, 2 nodes 1.220 1.343 0.025
> 26.027
> 64 Intel Xeon Gold 6326, 2 nodes 0.920 1.494 0.014
> 14.720
> 192 Intel Xeon Gold 6330H, 4 nodes 3.957 4.153 0.021
> 21.104
> 256 AMD EPYC 7713, 2 NUMA nodes 3.927 5.464 0.015
> 15.708
> 80 ARM Neoverse-N1, 1 NUMA node 1.833 2.502 0.023
> 23.462
> 128 ARM Kunpeng 920, 4 NUMA nodes 3.822 4.635 0.030
> 30.576
I have to admit I was hoping the current IPVS_EST_CHAIN_DEPTH would work on
machines with more than 1024 CPUs. If the max time values are used the time
needed to process one chain on a 1024 CPU machine gets even closer to 40 ms,
which it must not reach lest the estimates become inaccurate. I also have
profiling data so I intend to look at the disassembly of
ip_vs_estimation_kthread() to see which instructions take the most time. I will
take a look at the v2 of the code on Monday.
--
Jiri Wiesner
SUSE Labs
|
