|
I'm sorry , my english is pool.
I want to talk to you all , but I'm so said.
The problem which lvs cost 100% cpu on si% , the solution I know is :
use multi network card , and make them bond instead .
Because, the linux kernel is not support banlance si% to multi cpu core , so
the problem come up.
see this :
${linux-2.6.34}/Documentation/networking/cxgb.txt
These issues have been identified during testing. The following information
is provided as a workaround to the problem. In some cases, this problem is
inherent to Linux or to a particular Linux Distribution and/or hardware
platform.
1. Large number of TCP retransmits on a multiprocessor (SMP) system.
On a system with multiple CPUs, the interrupt (IRQ) for the network
controller may be bound to more than one CPU. This will cause TCP retransmits
if the packet data were to be split across different CPUs and re-assembled in
a different order than expected.
To eliminate the TCP retransmits, set smp_affinity on the particular
interrupt to a single CPU. You can locate the interrupt (IRQ) used on
the N110/N210 by using ifconfig: ifconfig <dev_name> | grep
Interrupt Set the smp_affinity to a single CPU:
echo 1 > /proc/irq/<interrupt_number>/smp_affinity
It is highly suggested that you do not run the irqbalance daemon on your
system, as this will change any smp_affinity setting you have applied. The
irqbalance daemon runs on a 10 second interval and binds interrupts to
the least loaded CPU determined by the daemon. To disable this daemon:
chkconfig --level 2345 irqbalance off
By default, some Linux distributions enable the kernel feature,
irqbalance, which performs the same function as the daemon. To disable
this feature, add the following line to your bootloader:
noirqbalance
Example using the Grub bootloader:
title Red Hat Enterprise Linux AS (2.4.21-27.ELsmp)
root (hd0,0)
kernel /vmlinuz-2.4.21-27.ELsmp ro root=/dev/hda3 noirqbalance
initrd /initrd-2.4.21-27.ELsmp.img
I'm writting in
http://hi.baidu.com/higkoo/blog/item/c9b561e943744932b90e2d4c.html .
I this bonding multi network card or chang your os ,is the only solution .
Thanks All.
------------------
还好有时光机我谢谢你！
------------------ 原始邮件 ------------------
发件人: "JL"<lvs@xxxxxxxx>;
发送时间: 2010年9月13日(星期一) 晚上6:40
收件人: "LinuxVirtualServer.org users mailing
list."<lvs-users@xxxxxxxxxxxxxxxxxxxxxx>; "楷子狐"<higkoo@xxxxxxx>;
主题: Re: [lvs-users] Kernel 2.6.35 and 100% S.I. CPU Time
On 13 September 2010 03:43, 楷子狐 <higkoo@xxxxxxx> wrote:
> I had see this problem before :
>
> http://hi.baidu.com/higkoo/blog/item/f8943c60d16843d28cb10d17.html
> ------------------
Looks like the same thing.
I suspect that the LVS service receives updates from the master, and
then sticks them in some netfilter table, but with some error that
makes the table huge. Maybe multiple entries appear?
楷子狐, Are you using MARK firewall rules, or a different method to
select packets for LVS?
If I change /proc/sys/net/ipv4/vs/sync_threshold to "3 100000", it
does *not* fix the problem. Which kind of throws any theory I have had
out the window.
"ipvsadm -l -c" Gives a lot of kernel messages "Detected stall on CPU
x". Eventually, however we get the list (which is currently only about
a dozen entries).
It was fine at linux 2.6.27.45.
# /proc/sys/net/ipv4/vs# grep -H "" *
am_droprate:10
amemthresh:1024
cache_bypass:0
drop_entry:0
drop_packet:0
expire_nodest_conn:0
expire_quiescent_template:0
nat_icmp_send:0
secure_tcp:0
sync_threshold:3 50
Does anyone have an idea what might be happening here?
> ------------------ Original ------------------
> From: "JL"<lvs@xxxxxxxx>;
> Date: Sun, Sep 12, 2010 06:29 PM
> To: "LinuxVirtualServer.org users mailing
> list."<lvs-users@xxxxxxxxxxxxxxxxxxxxxx>;
>
> Subject: [lvs-users] Kernel 2.6.35 and 100% S.I. CPU Time
>
>
> Hi,
>
> I have recently upgraded from kernel 2.6.27.45 to 2.6.35.4.
>
> Now, any machine which is a backup (that is, receiving connection
> updates from another machine) goes to nearly 100% CPU time in Soft
> Interrupt.
>
> Profiling the kernel shows the largest portion of time is spent in nf_iterate.
>
> We are using FWMARK rules to specify traffic for LVS.
>
> Is this problem something people are aware of? Does anyone know of a
> fix or workaround?
>
> Thanks,
> --
> Jarrod Lowe
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
--
Jarrod Lowe
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
