|
We have multiple LVS systems set up in master/backup configuration,
running 2.4.27. The backup server is a recently setup pair oopses and
then panics once every 0.5-2 weeks.
The only non-build patch to the stock 2.4.27 kernel is a 2.4 version of
http://sourceforge.net/mailarchive/message.php?msg_id=9252540 which I
don't think is causing the problem here.
The one that fails is the busiest of the various pairs that we have set
up, both in terms of number of connections, and number of real servers
set up - its handling mail for a large ISP.
Its also running an smp kernel, with a single CPU with hyperthreading,
while the other LVS systems we have are UP. That may be related..
Theres one VIP, with multiple services on different ports, all set up
for LVS-DR and managed with keepalived.
This is not hardware related; we swapped out the entire chassis last
week and its failed again.
Happy to run some debugging patches on this kernel if required, after
the long weekend.
Thanks,
Bradley
Oops (decoded after reboot) is:
ksymoops 2.4.4 on i686 2.4.27-0OIE6smp. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.27-0OIE6smp/ (default)
-m /boot/System.map-2.4.27-0OIE6smp (default)
Warning: You did not tell me where to find symbol information. I will
assume that the log matches the kernel and modules that are running
right now and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc. ksymoops -h explains the options.
Error (expand_objects): cannot stat(/lib/aic79xx.o) for aic79xx
ksymoops: No such file or directory
Error (expand_objects): cannot stat(/lib/sd_mod.o) for sd_mod
ksymoops: No such file or directory
Error (expand_objects): cannot stat(/lib/scsi_mod.o) for scsi_mod
ksymoops: No such file or directory
Unable to handle kernel NULL pointer dereference at virtual address 00000004
f89ec1d6
*pde = 00000000
Oops: 0002
CPU: 0
EIP: 0010:[<f89ec1d6>] Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010202
eax: 00000000 ebx: f710a2e0 ecx: 0000e459 edx: 00000000
esi: f89ed620 edi: 00000000 ebp: 00000000 esp: c02aff08
ds: 0018 es: 0018 ss: 0018
Process swapper (pid: 0, stackpage=c02af000)
Stack: f710a2e0 f89ed64e f710a2e0 f710a300 c0125797 f710a2e0 c02aff28 00000086
c02aff28 c02aff28 00000000 00000001 00000000 00000000 c012177b c02e68a0
c0121621 00000000 00000001 c02d1580 fffffffe 00000000 c01213ab c02d1580
Call Trace: [<f89ed64e>] [<c0125797>] [<c012177b>] [<c0121621>] [<c01213ab>]
[<c010a74e>] [<c0106e40>] [<c0106e40>] [<c0106e40>] [<c0106e40>] [<c0106e6c>]
[<c0106ef2>] [<c0105000>]
Code: 89 50 04 89 02 0f b7 43 40 c7 03 00 00 00 00 c7 43 04 00 00
>>EIP; f89ec1d6 <[ip_vs]__kstrtab_register_ip_vs_app+16/40> <=====
Trace; f89ed64e <[ip_vs]ip_vs_conn_expire+2e/150>
Trace; c0125797 <timer_bh+2b7/3f0>
Trace; c012177b <bh_action+4b/80>
Trace; c0121621 <tasklet_hi_action+61/a0>
Trace; c01213ab <do_softirq+6b/d0>
Trace; c010a74e <do_IRQ+fe/110>
Trace; c0106e40 <default_idle+0/40>
Trace; c0106e40 <default_idle+0/40>
Trace; c0106e40 <default_idle+0/40>
Trace; c0106e40 <default_idle+0/40>
Trace; c0106e6c <default_idle+2c/40>
Trace; c0106ef2 <cpu_idle+52/70>
Trace; c0105000 <_stext+0/0>
Code; f89ec1d6 <[ip_vs]__kstrtab_register_ip_vs_app+16/40>
00000000 <_EIP>:
Code; f89ec1d6 <[ip_vs]__kstrtab_register_ip_vs_app+16/40> <=====
0: 89 50 04 mov %edx,0x4(%eax) <=====
Code; f89ec1d9 <[ip_vs]__kstrtab_register_ip_vs_app+19/40>
3: 89 02 mov %eax,(%edx)
Code; f89ec1db <[ip_vs]__kstrtab_register_ip_vs_app+1b/40>
5: 0f b7 43 40 movzwl 0x40(%ebx),%eax
Code; f89ec1df <[ip_vs]__kstrtab_register_ip_vs_app+1f/40>
9: c7 03 00 00 00 00 movl $0x0,(%ebx)
Code; f89ec1e5 <[ip_vs]__kstrtab_register_ip_vs_app+25/40>
f: c7 43 04 00 00 00 00 movl $0x0,0x4(%ebx)
<0>Kernel panic: Aiee, killing interrupt handler!
1 warning and 3 errors issued. Results may not be reliable.
|
