LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] LVS servers slowly draining traffic causing outages

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] LVS servers slowly draining traffic causing outages
From: LDB <thesource@xxxxxxxxxxx>
Date: Thu, 06 Mar 2008 10:47:45 -0500
LDB wrote:
> Graeme Fowler wrote:
>> On Thu, 2008-03-06 at 07:24 -0500, LDB wrote:
>>> Any help would great ... I still cannot figure out why LVS just
>>> arbitrarily drains the web traffic to zero and then a restart
>>> fixes the problem.
>> I'm not 100% sure what you mean by "drains the web traffic to zero".
>>
>> Is it the case that over a number of hours, the load balancers stop
>> handling web traffic for the virtual hosts you've configured? If so, you
>> probably have the ARP problem and have not hidden your loopback
>> interfaces sufficiently on your realservers.
>>
>> If something else, please clarify.
>>
>> Thanks
>>
>> Graeme
>>
>>
>> _______________________________________________
>> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
>> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
>> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>>
>>
> 
> Yes, that is correct Graeme.
> 
> Ahh, so I have hidden the eth0 MAC
> 
> 
> [root@lvs1 ldb]# /sbin/ifconfig
> eth0      Link encap:Ethernet  HWaddr 00:E0:81:28:08:9F
>           inet addr:192.168.10.15  Bcast:192.168.10.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:17803988 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:17120177 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:1214681576 (1158.4 Mb)  TX bytes:2224292792 (2121.2 Mb)
>           Base address:0xdc00 Memory:fe9e0000-fea00000
> 
> eth0:0    Link encap:Ethernet  HWaddr 00:E0:81:28:08:9F
>           inet addr:192.168.10.46  Bcast:192.168.10.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Base address:0xdc00 Memory:fe9e0000-fea00000
> 
> eth0:1    Link encap:Ethernet  HWaddr 00:E0:81:28:08:9F
>           inet addr:192.168.10.70  Bcast:192.168.10.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Base address:0xdc00 Memory:fe9e0000-fea00000
> 
> eth0:2    Link encap:Ethernet  HWaddr 00:E0:81:28:08:9F
>           inet addr:192.168.10.90  Bcast:192.168.10.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Base address:0xdc00 Memory:fe9e0000-fea00000
> 
> eth0:3    Link encap:Ethernet  HWaddr 00:E0:81:28:08:9F
>           inet addr:192.168.10.80  Bcast:192.168.10.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Base address:0xdc00 Memory:fe9e0000-fea00000
> 
> eth0:4    Link encap:Ethernet  HWaddr 00:E0:81:28:08:9F
>           inet addr:192.168.10.180  Bcast:192.168.10.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           Base address:0xdc00 Memory:fe9e0000-fea00000
> 
> eth1      Link encap:Ethernet  HWaddr 00:E0:81:28:08:9E
>           inet addr:192.168.20.15  Bcast:192.168.20.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:67812 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:154145 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:13790575 (13.1 Mb)  TX bytes:52666672 (50.2 Mb)
>           Base address:0xd880 Memory:fe9c0000-fe9e0000
> 
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:35239 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:35239 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:3659537 (3.4 Mb)  TX bytes:3659537 (3.4 Mb)
> 
> 
> 
> with the following ....
> 
> 
> /sbin/arptables -F for vip in 192.168.10.46 192.168.10.70 \
> 192.168.10.90 192.168.10.80 192.168.10.180
> do
>         /sbin/arptables -A IN -d $vip -j DROP
>         /sbin/arptables -A OUT -s $vip -j mangle --mangle-ip-s 192.168.10.15
> done
> 
> 
> 
> but you are saying I need to hide the loopbacks on the VIPs (eth0:X)???
> 
> Thanks,
> 
> LDB
> 
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
> 
> 

here is what "ifconfig" looks like on one of my web servers ..


web1:/ # ifconfig
eth0      Link encap:Ethernet  HWaddr 00:18:FE:71:2D:8B
           inet addr:192.168.10.71  Bcast:192.168.10.255  Mask:255.255.255.0
           inet6 addr: fe80::218:feff:fe71:2d8b/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:3500397 errors:0 dropped:0 overruns:0 frame:0
           TX packets:3112056 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:1223052805 (1166.3 Mb)  TX bytes:1288347609 (1228.6 Mb)
           Interrupt:185

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:16436  Metric:1
           RX packets:15120 errors:0 dropped:0 overruns:0 frame:0
           TX packets:15120 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:756243 (738.5 Kb)  TX bytes:756243 (738.5 Kb)

lo:0      Link encap:Local Loopback
           inet addr:192.168.10.70  Mask:255.255.255.255
           UP LOOPBACK RUNNING  MTU:16436  Metric:1

lo:1      Link encap:Local Loopback
           inet addr:192.168.10.150  Mask:255.255.255.255
           UP LOOPBACK RUNNING  MTU:16436  Metric:1


<Prev in Thread] Current Thread [Next in Thread>