LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] RX Packet drops on high traffic LVS in DR setup

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] RX Packet drops on high traffic LVS in DR setup
From: Joseph Mack NA3T <jmack@xxxxxxxx>
Date: Fri, 24 Oct 2008 17:58:50 -0700 (PDT)
On Fri, 24 Oct 2008, Sashi Kant wrote:

> I am trying several different approaches to this problem.

thanks for the update. It's hard to find know what to do if 
you have limited hardware options, particularly if you don't 
know that it's the hardware. It also helps to hear back from 
people debuggine their setups.

>
> 1. Ethernet driver update - upgraded to kernel ethchnhalf with
> bnx2-firmware , situation improves , however still dropping packets
> during traffic spikes.
>
> 2. Broadcom cards sit on a PCI-X bridge off PCI Express bus, We tried
> other add-on broadcom card and it did not help as these cards were on
> same PCI-X bus.

commodity PCI-X has not worked well, due to clock skews on 
the 2X-wide bus. IBM got it to work in their high end 
hardware because they made the mobo and the NIC and could 
guarantee that they would work together on their 
implementation of the PCI-X bus. The chances of a commodity 
PCI-X mobo working with someone else's PCI-x NIC aren't real 
good and PCI-X was abandonned for PCI-e.

> 3. Added PCI Express card and things looking better, very few drops when
> traffic spikes suddenly.

glad to hear it.

> 4. Removed bonding, Cisco Ether Channel configuration and use PCIe
> gigabit card from intel (conf 3). We are currently testing this config
> and there are no packet drops at 50K pps.

no-one's ever been happy with bonding either.

> I will update you when we find our optimal configuration.
>
> IMHO Bus, NIC are huge players in this mix.
>
> Thank you all for your kind insights.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!


<Prev in Thread] Current Thread [Next in Thread>