LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Ipvs 0.9.3 : panic on heavy load.

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Ipvs 0.9.3 : panic on heavy load.
From: Lionel Bringuier <lb@xxxxxxxxxxxxxxxxx>
Date: Fri, 30 Nov 2001 12:25:48 +0100
Hello.

I am using ipvs 0.9.3 over a kernel 2.4.5 (on Intel Red Hat 7.1). The
following issues are experienced on a VoIP H323 load-balancing system, under
heavy load (40+ non-RAS H323 calls/sec, 50 simultaneous, for those who can
understand this jargon ;).

I have notices two problems, the second of which I cannot solve.

1. On a single CPU machine, with a kernel compiled with SMP support, I get a
kernel freeze in mod_sltimer (ip_vs_timer.c). I get locked on a concurrent
write_lock/write_unlock(&__ip_vs_sltimerlist_lock) acces in mod_sltimer.
That problem disappears if I disable CONFIG_SMP (on a single CPU machine).
Notice that I did not reproduce that with a bi-CPU machine.

2. I have some systematic kernel panics after 10000 to 80000 successful
calls. After some investigation, it appeared that the crash occurs due to
something in the sltimer_handler (ip_vs_timer.c).  I can say that the crash
is appearing in the call for "fn(data)" in run_sltimer_list, which seems to
be standing for a call for 'ip_vs_conn_expire'.  The fix I used is to
comment all the inside of sltimer_handler function "void
sltimer_handler(unsigned long data) { }". I can get millions of calls
succeeding this way. However, I have no garbage collecting of lost ip_vs
connection, which is 'a little' embarassing ;) 

Has anyone any clue about that... I'm working on that for weeks now, and I
feel desperatly lost.

(After a coarse ksyms/System.map analysis, my 'fine' debuging was made with
traces on screen directly using the video RAM [b8000] to have them as
quickly as possible). 

Regards,

-- 
=========================================================
       Lionel Bringuier - lb_at_fr.netcentrex.net
      Team Leader - Linux Applications Development
Phone : +33 (0)2 31 46 35 70 - http://www.netcentrex.net


<Prev in Thread] Current Thread [Next in Thread>