LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] Upgraded my Load Balancers, Performance Went to Sh*t

To: LinuxVirtualServer.org users mailing list. <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Upgraded my Load Balancers, Performance Went to Sh*t
From: Eric Robinson <eric.robinson@xxxxxxxxx>
Date: Sat, 19 Sep 2015 07:28:07 +0000
Malcom -- My wife makes wonderful fudge at Christmas time. Where should I send 
yours? 

Because...

ethtool -K p2p1 gro off
ethtool -K p2p2 gro off 
ethtool -K p2p3 gro off 
ethtool -K p2p4 gro off

...totally fixed it! The change was instantaneous. I didn't even have to 
down/up the interfaces.

I'm sitting here stunned. The application isn't freezing any more. I hardly 
know what to do with myself.

Thanks!

--Eric

-----Original Message-----
From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx 
[mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Malcolm Turnbull
Sent: Saturday, September 19, 2015 12:05 AM
To: LinuxVirtualServer.org users mailing list. 
<lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Upgraded my Load Balancers, Performance Went to Sh*t

Eric,

Ah , OK I thought that might be too simple :-).

Some general advice from our end is:
We have had problems with both LRO & GRO in the past (we disable them both), 
and we also run irqbalance (to save on CPU overhead from network traffic).
Also we tend to use Intel NICs with the latest drivers.
We do use Centos 6.3 (roughly) but with a more recent Kernel so I'm afraid I 
can't be much help.





On 18 September 2015 at 23:54, Eric Robinson <eric.robinson@xxxxxxxxx> wrote:
> That's the odd part. We don't have any WRR health checks at all. Everything 
> is WLC. And the ldirectord.cf file did not change, not did the version of 
> ldirectord itself. I have checked through the file, and there are no virtual 
> servers that are not backed by realservers, and there are no realservers 
> showing in a down state. I would post the file but it is quite long. We have 
> approximately 1700 virtual services configured.
>
> --Eric
>
>
> -----Original Message-----
> From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx 
> [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Malcolm 
> Turnbull
> Sent: Friday, September 18, 2015 2:20 PM
> To: LinuxVirtualServer.org users mailing list. 
> <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
> Subject: Re: [lvs-users] Upgraded my Load Balancers, Performance Went 
> to Sh*t
>
> Eric,
>
> What kind of health checks are you using, it sounds like no backend servers 
> available in the cluster?
>
> On 18 September 2015 at 17:11, Eric Robinson <eric.robinson@xxxxxxxxx> wrote:
>> ...oh, and one other thing. After the upgrade, I am now seeing these syslog 
>> messages once approximately per second...
>>
>> Sep 18 09:06:26 lb01a kernel: IPVS: WRR: no destination available: no 
>> destinations present Sep 18 09:06:28 lb01a kernel: IPVS: WRR: no 
>> destination available: no destinations present Sep 18 09:06:29 lb01a
>> kernel: IPVS: WRR: no destination available: no destinations present 
>> Sep 18 09:06:31 lb01a kernel: IPVS: WRR: no destination available: no 
>> destinations present Sep 18 09:06:32 lb01a kernel: IPVS: WRR: no 
>> destination available: no destinations present Sep 18 09:06:34 lb01a
>> kernel: IPVS: WRR: no destination available: no destinations present 
>> Sep 18 09:06:35 lb01a kernel: IPVS: WRR: no destination available: no 
>> destinations present Sep 18 09:06:37 lb01a kernel: IPVS: WRR: no 
>> destination available: no destinations present Sep 18 09:06:38 lb01a
>> kernel: IPVS: WRR: no destination available: no destinations present 
>> Sep 18 09:06:40 lb01a kernel: IPVS: WRR: no destination available: no 
>> destinations present Sep 18 09:06:41 lb01a kernel: IPVS: WRR: no 
>> destination available: no destinations present Sep 18 09:06:43 lb01a
>> kernel: IPVS: WRR: no destination available: no destinations present 
>> Sep 18 09:06:44 lb01a kernel: IPVS: WRR: no destination available: no 
>> destinations present
>>
>> _______________________________________________
>> Please read the documentation before posting - it's available at:
>> http://www.linuxvirtualserver.org/
>>
>> LinuxVirtualServer.org mailing list - 
>> lvs-users@xxxxxxxxxxxxxxxxxxxxxx Send requests to 
>> lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
>> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
>
>
> --
> Regards,
>
> Malcolm Turnbull.
>
> Loadbalancer.org Ltd.
> Phone: +44 (0)330 1604540
> http://www.loadbalancer.org/
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx 
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx 
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users



--
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)330 1604540
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx Send 
requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>