LVS
lvs-users
[Top] [All Lists]
Google
 
Web LinuxVirtualServer.org
<prev [Date] next> <prev [Thread] next>

Re: Other strange LB balancing issue

from [Cyrille Mertes] [Permanent Link]
To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Re: Other strange LB balancing issue
From: Cyrille Mertes <cyrille@xxxxxxxxxxxxx>
Date: Mon, 02 Dec 2002 16:45:11 -0500 
Hello,
Do you think the ARP problem can be a consequence of all my problem ??? I didn't applied the hidden patch on my real servers... ? 

I'm using direct routing, do you think NAT could be work better ?

Thanks,

Cyrille.





Cyrille Mertes wrote:


Hello,
Actual LB : redhat 8.0, kernel 2.4.18-14, IP Virtual Server version 1.0.4 (source), piranha-0.7.0-3 (source) Old configuration : redhat 7.2, kernel 2.4.18, IP Virtual Server version 1.0.1 (rpm source), piranha-0.6.1-1 (rpm source) 

I see a message of Jeremy Kusnetz on 11/18/2002 that looks a lot like a
problem that we have. I don't know if he found a solution. We have been
setuping LVS on different servers and always obtaining the same issue.
Our first unsuccesfull installation was with UltraMonkey, after we decided to try with Piranha to see if our problem would go away, but the problem is still 
present. We also installed it on different kernels, with RPMs, with the
source, and we always have this problem.
The problem is that the server stops balancing the requests, some times it 
stops only on one virtual server and continues balancing correctly on the
others. When it stops balancing it sends all the requests to one of the
real servers (always the same) and stops showing anything in ipvsadm.
It can stop 3 times in one hour, and after it can take 4 hours before stopping again. 

Here is output of ipvsadm when everything is ok :

IP Virtual Server version 1.0.4 (size=65536)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  216.187.127.124:http wrr
 -> gragra.privatedns.com:http     Route   1      170        398
 -> koon.privatedns.com:http       Route   1      122        212
 -> motti.privatedns.com:http      Route   1      121        233
TCP  216.187.127.122:http wrr
 -> motti.privatedns.com:http      Route   2      277        852
 -> gragra.privatedns.com:http     Route   1      150        433
 -> koon.privatedns.com:http       Route   2      236        837
Here is one output of ipvsadm where you can see that IP .122 stopped balancing the requests : 


IP Virtual Server version 1.0.4 (size=65536)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  216.187.127.124:http wrr
 -> gragra.privatedns.com:http     Route   1      170        398
 -> koon.privatedns.com:http       Route   1      122        212
 -> motti.privatedns.com:http      Route   1      121        233
TCP  216.187.127.122:http wrr


Here is my pirnha lvs.conf :

serial_no = 344
primary = 216.187.127.77
primary_private = 192.168.6.21
service = lvs
backup_active = 0
backup = 216.187.127.117
backup_private = 192.168.6.10
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 15
network = direct
nat_nmask = 255.255.255.255
reservation_conflict_action = preempt
debug_level = MAX
virtual sebulba_http_122 {
    active = 1
    address = 216.187.127.122 eth0:1
    vip_nmask = 255.255.255.0
    port = 80
    send = "GET / HTTP/1.0\r\n\r\n"
    expect = "HTTP"
    load_monitor = none
    scheduler = wrr
    protocol = tcp
    timeout = 6
    reentry = 15
    quiesce_server = 0
    server motti {
        address = 192.168.6.12
        active = 1
        weight = 2
    }
    server gragra {
        address = 192.168.6.13
        active = 1
        weight = 1
    }
    server koon {
        address = 192.168.6.14
        active = 1
        weight = 2
    }
}
virtual sebulba_http_124 {
    active = 1
    address = 216.187.127.124 eth0:3
    vip_nmask = 255.255.255.0
    port = 80
    send = "GET / HTTP/1.0\r\n\r\n"
    expect = "HTTP"
    load_monitor = none
    scheduler = wrr
    protocol = tcp
    timeout = 6
    reentry = 15
    quiesce_server = 0
    server motti {
        address = 192.168.6.12
        active = 1
        weight = 1
    }
    server gragra {
        address = 192.168.6.13
        active = 1
        weight = 2
    }
    server koon {
        address = 192.168.6.14
        active = 1
        weight = 1
    }
}
Finally, I have a lot of connection timed out with my real servers in /var/log/messages : 

Nov 29 10:34:43 lb nanny[5987]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:34:43 lb nanny[5987]: shutting down 192.168.6.13:8008 due to connection failure 
Nov 29 10:34:55 lb nanny[5987]: making 192.168.6.13:8008 available
Nov 29 10:36:14 lb nanny[5980]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:36:14 lb nanny[5980]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:36:15 lb nanny[5996]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:36:15 lb nanny[5996]: shutting down 192.168.6.13:8008 due to connection failure 
Nov 29 10:36:26 lb nanny[5980]: making 192.168.6.13:80 available
Nov 29 10:36:27 lb nanny[5996]: making 192.168.6.13:8008 available
Nov 29 10:37:00 lb nanny[6001]: READ to 192.168.6.14:80 timed out
Nov 29 10:37:00 lb nanny[6001]: shutting down 192.168.6.14:80 due to connection failure 
Nov 29 10:37:00 lb nanny[5993]: READ to 192.168.6.14:80 timed out
Nov 29 10:37:00 lb nanny[5993]: shutting down 192.168.6.14:80 due to connection failure 
Nov 29 10:37:00 lb nanny[5992]: READ to 192.168.6.13:80 timed out
Nov 29 10:37:00 lb nanny[5992]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:37:02 lb nanny[5979]: READ to 192.168.6.12:80 timed out
Nov 29 10:37:02 lb nanny[5979]: shutting down 192.168.6.12:80 due to connection failure 
Nov 29 10:37:12 lb nanny[6001]: making 192.168.6.14:80 available
Nov 29 10:37:12 lb nanny[5993]: making 192.168.6.14:80 available
Nov 29 10:37:12 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:37:14 lb nanny[5979]: making 192.168.6.12:80 available
Nov 29 10:38:09 lb nanny[5996]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:38:09 lb nanny[5996]: shutting down 192.168.6.13:8008 due to connection failure 
Nov 29 10:38:10 lb nanny[5992]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:38:10 lb nanny[5992]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:38:10 lb nanny[5987]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:38:10 lb nanny[5987]: shutting down 192.168.6.13:8008 due to connection failure 
Nov 29 10:38:21 lb nanny[5996]: making 192.168.6.13:8008 available
Nov 29 10:38:22 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:38:22 lb nanny[5987]: making 192.168.6.13:8008 available
Nov 29 10:39:09 lb nanny[6000]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:39:09 lb nanny[6000]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:39:21 lb nanny[6000]: making 192.168.6.13:80 available
Nov 29 10:39:46 lb nanny[5992]: READ to 192.168.6.13:80 timed out
Nov 29 10:39:46 lb nanny[5992]: shutting down 192.168.6.13:80 due to connection failure Nov 29 10:39:55 lb nanny[5980]: READ returned error 104:Connection reset by peer Nov 29 10:39:55 lb nanny[5980]: shutting down 192.168.6.13:80 due to connection failure Nov 29 10:39:55 lb nanny[6000]: READ returned error 104:Connection reset by peer Nov 29 10:39:55 lb nanny[6000]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:41:40 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:41:43 lb nanny[5980]: making 192.168.6.13:80 available
Nov 29 10:41:43 lb nanny[6000]: making 192.168.6.13:80 available
Nov 29 10:44:40 lb nanny[6000]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:44:40 lb nanny[6000]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:44:52 lb nanny[6000]: making 192.168.6.13:80 available
Nov 29 10:45:45 lb nanny[5980]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:45:45 lb nanny[5980]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:45:57 lb nanny[5980]: making 192.168.6.13:80 available
Nov 29 10:47:00 lb nanny[5992]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:47:00 lb nanny[5992]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:47:12 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:50:35 lb nanny[5987]: CONNECT to 192.168.6.13:8008 timed out
Nov 29 10:50:35 lb nanny[5987]: shutting down 192.168.6.13:8008 due to connection failure 
Nov 29 10:50:47 lb nanny[5987]: making 192.168.6.13:8008 available
Nov 29 10:55:03 lb nanny[5992]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:55:03 lb nanny[5992]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:55:03 lb nanny[5980]: CONNECT to 192.168.6.13:80 timed out
Nov 29 10:55:03 lb nanny[5980]: shutting down 192.168.6.13:80 due to connection failure 
Nov 29 10:55:15 lb nanny[5992]: making 192.168.6.13:80 available
Nov 29 10:55:15 lb nanny[5980]: making 192.168.6.13:80 available

Thanks for your help.

Cyrille.



_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://www.in-addr.de/mailman/listinfo/lvs-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  RE: Setting up source hashing scheduler, Peter Mueller
Next by Date:  RE: Other strange LB balancing issue, Peter Mueller
Previous by Thread:  Re: Other strange LB balancing issue, pb
Next by Thread:  RE: Other strange LB balancing issue, Peter Mueller
Indexes:  [Date] [Thread] [Top] [All Lists]