LVS
lvs-devel
Google
 
Web LinuxVirtualServer.org

Long delay on estimation_timer causes packet latency

To: horms@xxxxxxxxxxxx, ja@xxxxxx
Subject: Long delay on estimation_timer causes packet latency
Cc: netdev@xxxxxxxxxxxxxxx, lvs-devel@xxxxxxxxxxxxxxx, Yunhong Jiang <yunhjiang@xxxxxxxx>
From: yunhong-cgl jiang <xintian1976@xxxxxxxxx>
Date: Thu, 16 Apr 2020 15:00:53 -0700
Hi, Simon & Julian,
        We noticed that on our kubernetes node utilizing IPVS, the 
estimation_timer() takes very long (>200sm as shown below). Such long delay on 
timer softirq causes long packet latency.  

          <idle>-0     [007] dNH. 25652945.670814: softirq_raise: vec=1 
[action=TIMER]
.....
          <idle>-0     [007] .Ns. 25652945.992273: softirq_exit: vec=1 
[action=TIMER]

        The long latency is caused by the big service number (>50k) and large 
CPU number (>80 CPUs),

        We tried to move the timer function into a kernel thread so that it 
will not block the system and seems solves our problem. Is this the right 
direction? If yes, we will do more testing and send out the RFC patch. If not, 
can you give us some suggestion?

Thanks
—yunhong 
<Prev in Thread] Current Thread [Next in Thread>