|
Hello List,
Just wanted to give update on the thread I have opened.
This is how the final setup is going to look
1. Intel PCIe gigabit dual port card with RX buffer tuned at 1024 using
ethtool
2. No need for port channel as switch ports are Gig capable and
bandwidth is not an issue.
3. Bonding will be used in active-backup mode as it is not negatively
impacting the server.
4. No more packet drops at 60k pps with above changes.
5. Getting rid of Broadcom PCI-X cards from HA servers.
Thank you all for your support and patience, hope this helps someone.
-
Thanks
-Sashi
On Fri, 2008-10-24 at 17:58 -0700, Joseph Mack NA3T wrote:
> On Fri, 24 Oct 2008, Sashi Kant wrote:
>
> > I am trying several different approaches to this problem.
>
> thanks for the update. It's hard to find know what to do if
> you have limited hardware options, particularly if you don't
> know that it's the hardware. It also helps to hear back from
> people debuggine their setups.
>
> >
> > 1. Ethernet driver update - upgraded to kernel ethchnhalf with
> > bnx2-firmware , situation improves , however still dropping packets
> > during traffic spikes.
> >
> > 2. Broadcom cards sit on a PCI-X bridge off PCI Express bus, We tried
> > other add-on broadcom card and it did not help as these cards were on
> > same PCI-X bus.
>
> commodity PCI-X has not worked well, due to clock skews on
> the 2X-wide bus. IBM got it to work in their high end
> hardware because they made the mobo and the NIC and could
> guarantee that they would work together on their
> implementation of the PCI-X bus. The chances of a commodity
> PCI-X mobo working with someone else's PCI-x NIC aren't real
> good and PCI-X was abandonned for PCI-e.
>
> > 3. Added PCI Express card and things looking better, very few drops when
> > traffic spikes suddenly.
>
> glad to hear it.
>
> > 4. Removed bonding, Cisco Ether Channel configuration and use PCIe
> > gigabit card from intel (conf 3). We are currently testing this config
> > and there are no packet drops at 50K pps.
>
> no-one's ever been happy with bonding either.
>
> > I will update you when we find our optimal configuration.
> >
> > IMHO Bus, NIC are huge players in this mix.
> >
> > Thank you all for your kind insights.
>
> Joe
>
|
