|
Hello,
This patchset implements stats estimation in
kthread context. Simple tests do not show any problem.
As it is late here, I'm sending v2 for
review. It is interesting to know what value for
IPVS_EST_TICK_CHAINS to use, it is used for the
IPVS_EST_MAX_COUNT calculation. We should determine
it from tests once the loops are in final form.
Now the limit increased a little bit to 38400.
Tomorrow I'll check again the patches for possible
problems.
Overview of the basic concepts. More in the
commit messages...
RCU Locking:
- As stats are now RCU-locked, tot_stats, svc and dest which
hold estimator structures are now always freed from RCU
callback. This ensures RCU grace period after the
ip_vs_stop_estimator() call.
Kthread data:
- every kthread works over its own data structure and all
such structures are attached to array
- even while there can be a kthread structure, its task
may not be running, eg. before first service is added or
while the sysctl var is set to an empty cpulist or
when run_estimation is 0.
- a task and its structure may be released if all
estimators are unlinked from its chains, leaving the
slot in the array empty
- every kthread data structure allows limited number
of estimators
- to add new estimators we use the last added kthread
context (est_add_ktid). The new estimators are linked to
the chains just before the estimated one, based on add_row.
This ensures their estimation will start after 2 seconds.
If estimators are added in bursts, common case if all
services and dests are initially configured, we may
spread the estimators to more chains. This will reduce
the chain imbalance.
Not done yet:
* limit for kthreads (sysctl var)
Changes in v2:
Patch 2:
* kd->mutex is gone, cond_resched rate determined by
IPVS_EST_CHAIN_DEPTH
* IPVS_EST_MAX_COUNT is a hard limit now
* kthread data is now 1-50 allocated tick structures,
each containing heads for limited chains. Bitmaps
should allow faster access. We avoid large
allocations for structs.
* as the td->present bitmap is shared, use atomic bitops
* ip_vs_start_estimator now returns error code
* _bh locking removed from stats->lock
* bump arg is gone from ip_vs_est_reload_start
* prepare for upcoming changes that remove _irq
from u64_stats_fetch_begin_irq/u64_stats_fetch_retry_irq
* est_add_ktid is now always valid
Patch 3:
* use .. in est_nice docs
Julian Anastasov (4):
ipvs: add rcu protection to stats
ipvs: use kthreads for stats estimation
ipvs: add est_cpulist and est_nice sysctl vars
ipvs: run_estimation should control the kthread tasks
Documentation/networking/ipvs-sysctl.rst | 24 +-
include/net/ip_vs.h | 122 +++++-
net/netfilter/ipvs/ip_vs_core.c | 10 +-
net/netfilter/ipvs/ip_vs_ctl.c | 367 +++++++++++++++---
net/netfilter/ipvs/ip_vs_est.c | 450 +++++++++++++++++++----
5 files changed, 843 insertions(+), 130 deletions(-)
--
2.37.3
|
