LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] LVS hangs on 2.6.37

To: Simon Kirby <sim@xxxxxxxxxx>
Subject: Re: [lvs-users] LVS hangs on 2.6.37
Cc: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
From: Julian Anastasov <ja@xxxxxx>
Date: Sat, 19 Feb 2011 23:26:53 +0200 (EET)
        Hello,

On Sat, 19 Feb 2011, Simon Kirby wrote:

> Hi! :)
>
> We've been having a large number of LVS boxes fall over (hang) since
> upgrading from 2.6.35 or earlier to 2.6.37. They seem to hard lock with
> nothing on the console. The ones I've enabled nmi_watchdog on haven't
> died yet, but I did enable remote syslog and managed to catch the kernel
> log below.
>
> It seems to be spinning on &dest->dst_lock. On this pair that this was
> recorded on, we have keepalived running with a MISC_CHECK script that
> actually fiddles with the weights with an ipvsadm command, hence the
> "ipvsadm" command running, but we have also seen hangs on other master
> and backup nodes without such scripts. I'm not sure if this is a useful
> trace or not, since it has all '?'. There seem to be not many changes
> except quite a few locking changes around the new features introduced in
> 2.6.37, and then not much since. Any ideas?

        Ops, sorry, it was introduced in 2.6.37 by me.
Patch appended. Please, confirm that it fixes the lockups.

> [14315.510017] BUG: soft lockup - CPU#0 stuck for 67s! [ipvsadm:8382]
> [14315.510104] Modules linked in: ip_vs_rr ip_vs_lblc ip_vs nf_conntrack 
> xt_hashlimit ts_kmp xt_string xt_recent ipmi_devintf ipmi_si ipmi_msghandler 
> bnx2
> [14315.510111] CPU 0
> [14315.510112] Modules linked in: ip_vs_rr ip_vs_lblc ip_vs nf_conntrack 
> xt_hashlimit ts_kmp xt_string xt_recent ipmi_devintf ipmi_si ipmi_msghandler 
> bnx2
> [14315.510118]
> [14315.510121] Pid: 8382, comm: ipvsadm Not tainted 2.6.37-hw #8 
> 01V648/PowerEdge R410
> [14315.510122] RIP: 0010:[<ffffffff816b3f66>]  [<ffffffff816b3f66>] 
> _raw_spin_lock+0x16/0x20
> [14315.510128] RSP: 0018:ffff8800cee037d0  EFLAGS: 00000293
> [14315.510130] RAX: 000000000000fffd RBX: ffff8800cee037d0 RCX: 
> 0000000000000000
> [14315.510131] RDX: 000000001a204740 RSI: ffff88011fe83200 RDI: 
> ffff88011fe832dc
> [14315.510133] RBP: ffffffff8100b853 R08: 0000000000000003 R09: 
> ffff8800cee038b0
> [14315.510135] R10: ffff880111b3cf98 R11: 0000000000000000 R12: 
> ffff8800cee03750
> [14315.510136] R13: ffff88011fe83200 R14: ffff88012a2b0c54 R15: 
> ffffffff8102a5a7
> [14315.510138] FS:  00007eff00e526e0(0000) GS:ffff8800cee00000(0000) 
> knlGS:0000000000000000
> [14315.510140] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [14315.510142] CR2: 00007eff00805b40 CR3: 0000000110a90000 CR4: 
> 00000000000006f0
> [14315.510144] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> [14315.510146] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> [14315.510147] Process ipvsadm (pid: 8382, threadinfo ffff8801106fe000, task 
> ffff880110f11640)
> [14315.510149] Stack:
> [14315.510209]  ffff8800cee038b0 ffffffffa008370d ffff88012a2b0c68 
> 1a20474000000003
> [14315.510212]  000000081fcab800 0000000000000246 ffff8800cee03850 
> ffffffff815e02e1
> [14315.510215]  ffff88011fcab800 0000000000000000 ffff8800cee03890 
> 0000000000000000
> [14315.510218] Call Trace:
> [14315.510279]  <IRQ>
> [14315.510388]  [<ffffffffa008370d>] ? __ip_vs_get_out_rt+0x4d/0x380 [ip_vs]
> [14315.510391]  [<ffffffff815e02e1>] ? dev_queue_xmit+0x171/0x460
> [14315.510395]  [<ffffffff810647a6>] ? lock_timer_base+0x36/0x70
> [14315.510400]  [<ffffffffa0084b60>] ? ip_vs_dr_xmit+0x50/0x420 [ip_vs]
> [14315.510404]  [<ffffffffa007cdfa>] ? ip_vs_in+0x15a/0x5a0 [ip_vs]
> [14315.510407]  [<ffffffffa007c3e5>] ? ip_vs_out+0x675/0x9b0 [ip_vs]
> [14315.510411]  [<ffffffffa007d28e>] ? ip_vs_remote_request4+0xe/0x10 [ip_vs]
> [14315.510414]  [<ffffffff815fe8f7>] ? nf_iterate+0x67/0xa0
> [14315.510417]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.510420]  [<ffffffff815fe9db>] ? nf_hook_slow+0xab/0x100
> [14315.510422]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.510425]  [<ffffffff81608460>] ? ip_rcv_finish+0x0/0x310
> [14315.510427]  [<ffffffff81608c75>] ? ip_local_deliver+0x55/0xa0
> [14315.510430]  [<ffffffff816085b2>] ? ip_rcv_finish+0x152/0x310
> [14315.510433]  [<ffffffff816089de>] ? ip_rcv+0x26e/0x2e0
> [14315.510436]  [<ffffffff815dd833>] ? __netif_receive_skb+0x433/0x520
> [14315.510439]  [<ffffffff815dd885>] ? __netif_receive_skb+0x485/0x520
> [14315.510444]  [<ffffffff8107f5d4>] ? getnstimeofday+0x64/0xf0
> [14315.510446]  [<ffffffff815de848>] ? netif_receive_skb+0x78/0x80
> [14315.510448]  [<ffffffff815de960>] ? napi_skb_finish+0x50/0x70
> [14315.510450]  [<ffffffff815deee5>] ? napi_gro_receive+0xc5/0xd0
> [14315.510454]  [<ffffffffa000a9d8>] ? bnx2_poll_work+0x608/0x1540 [bnx2]
> [14315.510458]  [<ffffffff810cfe79>] ? free_one_page+0x1e9/0x460
> [14315.510463]  [<ffffffff81109ed1>] ? add_partial+0x21/0x80
> [14315.510465]  [<ffffffff8110b923>] ? __slab_free+0x123/0x130
> [14315.510468]  [<ffffffff81109ed1>] ? add_partial+0x21/0x80
> [14315.510471]  [<ffffffffa000b94e>] ? bnx2_poll_msix+0x3e/0xd0 [bnx2]
> [14315.510474]  [<ffffffff815df0db>] ? net_rx_action+0x11b/0x2e0
> [14315.510478]  [<ffffffff8105c4b0>] ? __do_softirq+0xf0/0x1f0
> [14315.510481]  [<ffffffff8100bd9c>] ? call_softirq+0x1c/0x30
> [14315.510483]  [<ffffffff8100d7a5>] ? do_softirq+0x65/0xa0
> [14315.510485]  [<ffffffff8105c315>] ? irq_exit+0x75/0xa0
> [14315.510487]  [<ffffffff8100cd0f>] ? do_IRQ+0x6f/0xf0
> [14315.510490]  [<ffffffff816b4653>] ? ret_from_intr+0x0/0x11
> [14315.510492]  <EOI>
> [14315.510597]  [<ffffffff8103f2ee>] ? mutex_spin_on_owner+0x8e/0xb0
> [14315.510601]  [<ffffffff816b2915>] ? __mutex_lock_slowpath+0x65/0x180
> [14315.510604]  [<ffffffff815d668a>] ? __alloc_skb+0x7a/0x170
> [14315.510606]  [<ffffffff816b27de>] ? mutex_lock+0x1e/0x40
> [14315.510609]  [<ffffffff815fa2f9>] ? netlink_dump+0x49/0x200
> [14315.510612]  [<ffffffff815d539d>] ? consume_skb+0x5d/0x80
> [14315.510614]  [<ffffffff815fa998>] ? netlink_recvmsg+0x268/0x3a0
> [14315.510617]  [<ffffffff815fc1fb>] ? netlink_sendmsg+0x10b/0x370
> [14315.510620]  [<ffffffff815ccc77>] ? sock_recvmsg+0xf7/0x130
> [14315.510623]  [<ffffffff810caa7b>] ? find_get_page+0x1b/0x90
> [14315.510625]  [<ffffffff810cb461>] ? filemap_fault+0x81/0x400
> [14315.510628]  [<ffffffff810cad85>] ? unlock_page+0x25/0x30
> [14315.510630]  [<ffffffff815cc8ae>] ? __sys_recvmsg+0x13e/0x2e0
> [14315.510634]  [<ffffffff816b7d74>] ? do_page_fault+0x254/0x4a0
> [14315.510636]  [<ffffffff815fc99f>] ? netlink_bind+0x15f/0x230
> [14315.510638]  [<ffffffff815cb775>] ? move_addr_to_user+0x65/0x70
> [14315.510641]  [<ffffffff815cd879>] ? sys_getsockname+0x99/0xa0
> [14315.510644]  [<ffffffff815cd294>] ? sys_recvmsg+0x44/0x70
> [14315.510647]  [<ffffffff8100af12>] ? system_call_fastpath+0x16/0x1b
> [14315.510648] Code: c2 85 c0 c9 0f 95 c0 0f b6 c0 c3 66 2e 0f 1f 84 00 00 00 
> 00 00 55 b8 00 01 00 00 48 89 e5 f0 66 0f c1 07 38 e0 74 06 f3 90 8a 07 <eb> 
> f6 c9 c3 66 0f 1f 44 00 00 55 48 89 e5 9c 58 0f 1f 44 00 00
> [14315.513532] Call Trace:
> [14315.513533]  <IRQ>  [<ffffffffa008370d>] ? __ip_vs_get_out_rt+0x4d/0x380 
> [ip_vs]
> [14315.513540]  [<ffffffff815e02e1>] ? dev_queue_xmit+0x171/0x460
> [14315.513542]  [<ffffffff810647a6>] ? lock_timer_base+0x36/0x70
> [14315.513546]  [<ffffffffa0084b60>] ? ip_vs_dr_xmit+0x50/0x420 [ip_vs]
> [14315.513550]  [<ffffffffa007cdfa>] ? ip_vs_in+0x15a/0x5a0 [ip_vs]
> [14315.513554]  [<ffffffffa007c3e5>] ? ip_vs_out+0x675/0x9b0 [ip_vs]
> [14315.513558]  [<ffffffffa007d28e>] ? ip_vs_remote_request4+0xe/0x10 [ip_vs]
> [14315.513560]  [<ffffffff815fe8f7>] ? nf_iterate+0x67/0xa0
> [14315.513563]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.513565]  [<ffffffff815fe9db>] ? nf_hook_slow+0xab/0x100
> [14315.513568]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.513570]  [<ffffffff81608460>] ? ip_rcv_finish+0x0/0x310
> [14315.513573]  [<ffffffff81608c75>] ? ip_local_deliver+0x55/0xa0
> [14315.513575]  [<ffffffff816085b2>] ? ip_rcv_finish+0x152/0x310
> [14315.513578]  [<ffffffff816089de>] ? ip_rcv+0x26e/0x2e0
> [14315.513580]  [<ffffffff815dd833>] ? __netif_receive_skb+0x433/0x520
> [14315.513583]  [<ffffffff815dd885>] ? __netif_receive_skb+0x485/0x520
> [14315.513586]  [<ffffffff8107f5d4>] ? getnstimeofday+0x64/0xf0
> [14315.513588]  [<ffffffff815de848>] ? netif_receive_skb+0x78/0x80
> [14315.513590]  [<ffffffff815de960>] ? napi_skb_finish+0x50/0x70
> [14315.513592]  [<ffffffff815deee5>] ? napi_gro_receive+0xc5/0xd0
> [14315.513596]  [<ffffffffa000a9d8>] ? bnx2_poll_work+0x608/0x1540 [bnx2]
> [14315.513598]  [<ffffffff810cfe79>] ? free_one_page+0x1e9/0x460
> [14315.513601]  [<ffffffff81109ed1>] ? add_partial+0x21/0x80
> [14315.513604]  [<ffffffff8110b923>] ? __slab_free+0x123/0x130
> [14315.513606]  [<ffffffff81109ed1>] ? add_partial+0x21/0x80
> [14315.513609]  [<ffffffffa000b94e>] ? bnx2_poll_msix+0x3e/0xd0 [bnx2]
> [14315.513612]  [<ffffffff815df0db>] ? net_rx_action+0x11b/0x2e0
> [14315.513614]  [<ffffffff8105c4b0>] ? __do_softirq+0xf0/0x1f0
> [14315.513616]  [<ffffffff8100bd9c>] ? call_softirq+0x1c/0x30
> [14315.513618]  [<ffffffff8100d7a5>] ? do_softirq+0x65/0xa0
> [14315.513621]  [<ffffffff8105c315>] ? irq_exit+0x75/0xa0
> [14315.513623]  [<ffffffff8100cd0f>] ? do_IRQ+0x6f/0xf0
> [14315.513625]  [<ffffffff816b4653>] ? ret_from_intr+0x0/0x11
> [14315.513626]  <EOI>  [<ffffffff8103f2ee>] ? mutex_spin_on_owner+0x8e/0xb0
> [14315.513631]  [<ffffffff816b2915>] ? __mutex_lock_slowpath+0x65/0x180
> [14315.513633]  [<ffffffff815d668a>] ? __alloc_skb+0x7a/0x170
> [14315.513636]  [<ffffffff816b27de>] ? mutex_lock+0x1e/0x40
> [14315.513638]  [<ffffffff815fa2f9>] ? netlink_dump+0x49/0x200
> [14315.513641]  [<ffffffff815d539d>] ? consume_skb+0x5d/0x80
> [14315.513643]  [<ffffffff815fa998>] ? netlink_recvmsg+0x268/0x3a0
> [14315.513645]  [<ffffffff815fc1fb>] ? netlink_sendmsg+0x10b/0x370
> [14315.513648]  [<ffffffff815ccc77>] ? sock_recvmsg+0xf7/0x130
> [14315.513650]  [<ffffffff810caa7b>] ? find_get_page+0x1b/0x90
> [14315.513653]  [<ffffffff810cb461>] ? filemap_fault+0x81/0x400
> [14315.513655]  [<ffffffff810cad85>] ? unlock_page+0x25/0x30
> [14315.513657]  [<ffffffff815cc8ae>] ? __sys_recvmsg+0x13e/0x2e0
> [14315.513660]  [<ffffffff816b7d74>] ? do_page_fault+0x254/0x4a0
> [14315.513662]  [<ffffffff815fc99f>] ? netlink_bind+0x15f/0x230
> [14315.513665]  [<ffffffff815cb775>] ? move_addr_to_user+0x65/0x70
> [14315.513667]  [<ffffffff815cd879>] ? sys_getsockname+0x99/0xa0
> [14315.513670]  [<ffffffff815cd294>] ? sys_recvmsg+0x44/0x70
> [14315.513672]  [<ffffffff8100af12>] ? system_call_fastpath+0x16/0x1b
> [14315.785537] BUG: soft lockup - CPU#3 stuck for 67s! [ipvsadm:8405]
> [14315.785608] Modules linked in: ip_vs_rr ip_vs_lblc ip_vs nf_conntrack 
> xt_hashlimit ts_kmp xt_string xt_recent ipmi_devintf ipmi_si ipmi_msghandler 
> bnx2
> [14315.785617] CPU 3
> [14315.785618] Modules linked in: ip_vs_rr ip_vs_lblc ip_vs nf_conntrack 
> xt_hashlimit ts_kmp xt_string xt_recent ipmi_devintf ipmi_si ipmi_msghandler 
> bnx2
> [14315.785627]
> [14315.785630] Pid: 8405, comm: ipvsadm Not tainted 2.6.37-hw #8 
> 01V648/PowerEdge R410
> [14315.785632] RIP: 0010:[<ffffffff816b3f64>]  [<ffffffff816b3f64>] 
> _raw_spin_lock+0x14/0x20
> [14315.785638] RSP: 0018:ffff8800cee637d0  EFLAGS: 00000286
> [14315.785640] RAX: 00000000000001fd RBX: ffff8800cee637d0 RCX: 
> 0000000000000000
> [14315.785643] RDX: 000000001a204740 RSI: ffff88011fe83200 RDI: 
> ffff88011fe832dc
> [14315.785646] RBP: ffffffff8100b853 R08: 0000000000000003 R09: 
> ffff8800cee638b0
> [14315.785648] R10: ffff880119568a98 R11: 0000000000000400 R12: 
> ffff8800cee63750
> [14315.785651] R13: ffff88011fe83200 R14: ffff88012a02c454 R15: 
> ffffffff8102a5a7
> [14315.785654] FS:  00007f476e92b6e0(0000) GS:ffff8800cee60000(0000) 
> knlGS:0000000000000000
> [14315.785657] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [14315.785659] CR2: 00000000011a3488 CR3: 0000000110a25000 CR4: 
> 00000000000006e0
> [14315.785662] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> [14315.785665] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> [14315.785668] Process ipvsadm (pid: 8405, threadinfo ffff880111bb2000, task 
> ffff880110edc2c0)
> [14315.785670] Stack:
> [14315.785731]  ffff8800cee638b0 ffffffffa008370d ffff88012a02c468 
> 1a20474000000003
> [14315.785736]  0000000800000006 0000000000000000 0000000000000001 
> 000000000000024d
> [14315.785740]  0000000000000400 0000000000000003 ffff8800cee63940 
> ffffffff81046349
> [14315.785744] Call Trace:
> [14315.785806]  <IRQ>
> [14315.785915]  [<ffffffffa008370d>] ? __ip_vs_get_out_rt+0x4d/0x380 [ip_vs]
> [14315.785920]  [<ffffffff81046349>] ? select_task_rq_fair+0x989/0x9f0
> [14315.785924]  [<ffffffff810647a6>] ? lock_timer_base+0x36/0x70
> [14315.785931]  [<ffffffffa0084b60>] ? ip_vs_dr_xmit+0x50/0x420 [ip_vs]
> [14315.785936]  [<ffffffffa007cdfa>] ? ip_vs_in+0x15a/0x5a0 [ip_vs]
> [14315.785941]  [<ffffffffa007c3e5>] ? ip_vs_out+0x675/0x9b0 [ip_vs]
> [14315.785947]  [<ffffffffa007d28e>] ? ip_vs_remote_request4+0xe/0x10 [ip_vs]
> [14315.785951]  [<ffffffff815fe8f7>] ? nf_iterate+0x67/0xa0
> [14315.785955]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.785958]  [<ffffffff815fe9db>] ? nf_hook_slow+0xab/0x100
> [14315.785962]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.785965]  [<ffffffff81608460>] ? ip_rcv_finish+0x0/0x310
> [14315.785969]  [<ffffffff81608c75>] ? ip_local_deliver+0x55/0xa0
> [14315.785973]  [<ffffffff816085b2>] ? ip_rcv_finish+0x152/0x310
> [14315.785976]  [<ffffffff816089de>] ? ip_rcv+0x26e/0x2e0
> [14315.785980]  [<ffffffff815dd833>] ? __netif_receive_skb+0x433/0x520
> [14315.785984]  [<ffffffff815dd885>] ? __netif_receive_skb+0x485/0x520
> [14315.785989]  [<ffffffff8107f5d4>] ? getnstimeofday+0x64/0xf0
> [14315.785992]  [<ffffffff815de848>] ? netif_receive_skb+0x78/0x80
> [14315.785995]  [<ffffffff815de960>] ? napi_skb_finish+0x50/0x70
> [14315.785998]  [<ffffffff815deee5>] ? napi_gro_receive+0xc5/0xd0
> [14315.786004]  [<ffffffffa000a9d8>] ? bnx2_poll_work+0x608/0x1540 [bnx2]
> [14315.786008]  [<ffffffff81045cd7>] ? select_task_rq_fair+0x317/0x9f0
> [14315.786012]  [<ffffffff810b23bc>] ? trace_ctxwake_hex+0x7c/0x140
> [14315.786016]  [<ffffffff810c79ff>] ? perf_event_task_tick+0x9f/0x2d0
> [14315.786020]  [<ffffffffa000b94e>] ? bnx2_poll_msix+0x3e/0xd0 [bnx2]
> [14315.786024]  [<ffffffff815df0db>] ? net_rx_action+0x11b/0x2e0
> [14315.786028]  [<ffffffff8105c4b0>] ? __do_softirq+0xf0/0x1f0
> [14315.786031]  [<ffffffff8100bd9c>] ? call_softirq+0x1c/0x30
> [14315.786034]  [<ffffffff8100d7a5>] ? do_softirq+0x65/0xa0
> [14315.786038]  [<ffffffff8105c315>] ? irq_exit+0x75/0xa0
> [14315.786041]  [<ffffffff8100cd0f>] ? do_IRQ+0x6f/0xf0
> [14315.786044]  [<ffffffff816b4653>] ? ret_from_intr+0x0/0x11
> [14315.786046]  <EOI>
> [14315.786153]  [<ffffffff8103f2df>] ? mutex_spin_on_owner+0x7f/0xb0
> [14315.786157]  [<ffffffff816b2915>] ? __mutex_lock_slowpath+0x65/0x180
> [14315.786161]  [<ffffffff816b27de>] ? mutex_lock+0x1e/0x40
> [14315.786164]  [<ffffffff815fdd38>] ? genl_rcv+0x18/0x40
> [14315.786167]  [<ffffffff815fba53>] ? netlink_unicast+0x2d3/0x2e0
> [14315.786171]  [<ffffffff815d7ba3>] ? memcpy_fromiovec+0x63/0x80
> [14315.786174]  [<ffffffff815fc426>] ? netlink_sendmsg+0x336/0x370
> [14315.786179]  [<ffffffff815ccdf3>] ? sock_sendmsg+0xe3/0x110
> [14315.786182]  [<ffffffff810cb461>] ? filemap_fault+0x81/0x400
> [14315.786187]  [<ffffffff810e9149>] ? __do_fault+0x409/0x520
> [14315.786191]  [<ffffffff810d5748>] ? lru_cache_add_lru+0x28/0x50
> [14315.786194]  [<ffffffff815cb6fc>] ? move_addr_to_kernel+0x2c/0x40
> [14315.786198]  [<ffffffff815d7d03>] ? verify_iovec+0x43/0xf0
> [14315.786201]  [<ffffffff815cdbb4>] ? sys_sendmsg+0x184/0x320
> [14315.786205]  [<ffffffff816b7d74>] ? do_page_fault+0x254/0x4a0
> [14315.786209]  [<ffffffff815fc99f>] ? netlink_bind+0x15f/0x230
> [14315.786212]  [<ffffffff815cb775>] ? move_addr_to_user+0x65/0x70
> [14315.786216]  [<ffffffff815cd879>] ? sys_getsockname+0x99/0xa0
> [14315.786220]  [<ffffffff8100af12>] ? system_call_fastpath+0x16/0x1b
> [14315.786222] Code: 0f b6 c2 85 c0 c9 0f 95 c0 0f b6 c0 c3 66 2e 0f 1f 84 00 
> 00 00 00 00 55 b8 00 01 00 00 48 89 e5 f0 66 0f c1 07 38 e0 74 06 f3 90 <8a> 
> 07 eb f6 c9 c3 66 0f 1f 44 00 00 55 48 89 e5 9c 58 0f 1f 44
> [14315.789140] Call Trace:
> [14315.789142]  <IRQ>  [<ffffffffa008370d>] ? __ip_vs_get_out_rt+0x4d/0x380 
> [ip_vs]
> [14315.789150]  [<ffffffff81046349>] ? select_task_rq_fair+0x989/0x9f0
> [14315.789154]  [<ffffffff810647a6>] ? lock_timer_base+0x36/0x70
> [14315.789160]  [<ffffffffa0084b60>] ? ip_vs_dr_xmit+0x50/0x420 [ip_vs]
> [14315.789165]  [<ffffffffa007cdfa>] ? ip_vs_in+0x15a/0x5a0 [ip_vs]
> [14315.789170]  [<ffffffffa007c3e5>] ? ip_vs_out+0x675/0x9b0 [ip_vs]
> [14315.789176]  [<ffffffffa007d28e>] ? ip_vs_remote_request4+0xe/0x10 [ip_vs]
> [14315.789179]  [<ffffffff815fe8f7>] ? nf_iterate+0x67/0xa0
> [14315.789183]  [<ffffffff81608a50>] ? ip_local_deliver_finish+0x0/0x1d0
> [14315.789186]  [<ffffffff815fe9db>] ? nf_hook_slow+0xab/0x100

        Fix dst_lock usage in __ip_vs_update_dest.
_bh locking is needed for user context. Bug was introduced
in 2.6.37 from the "ipvs: changes for local real server"
change.

Signed-off-by: Julian Anastasov <ja@xxxxxx>
---

        Patch is for 2.6.37

--- linux-2.6.37/net/netfilter/ipvs/ip_vs_ctl.c 2011-01-06 00:01:23.600069161 
+0200
+++ linux/net/netfilter/ipvs/ip_vs_ctl.c        2011-02-19 23:14:44.463250743 
+0200
@@ -810,9 +810,9 @@ __ip_vs_update_dest(struct ip_vs_service
        dest->u_threshold = udest->u_threshold;
        dest->l_threshold = udest->l_threshold;

-       spin_lock(&dest->dst_lock);
+       spin_lock_bh(&dest->dst_lock);
        ip_vs_dst_reset(dest);
-       spin_unlock(&dest->dst_lock);
+       spin_unlock_bh(&dest->dst_lock);

        if (add)
                ip_vs_new_estimator(&dest->stats);

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>