LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Ultramonkey/LVS-TUN connection stalls on client

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Ultramonkey/LVS-TUN connection stalls on client
From: Nigel Hamilton <nigel@xxxxxxxxxxx>
Date: Tue, 24 Feb 2004 11:10:31 -0600 (CST)
Hi,

        I'm still trying to get ultramonkey/lvs/ldirectord to work. 

        I'm trying to do a simple load balancing topology based on:  
http://www.ultramonkey.org/2.0.1/topologies/lb-eg.html - initially with
just one Director and one Real Server using IP tunneling.

        Ldirectord is up and running on the Director and polling the Real 
Server successfully. However, when I try to connect from a client browser 
to the virual ip (VIP) the connection appears to 'hang'.
        
        So I'm now trying to work out where the ball gets dropped:

        I've written a Perl script on the client to request a test file
from the server using LWP. It makes a simple http request to the VIP:  
http://66.98.227.143:80/cluster.html.
        
        I start tcpdump listening on the network interface for the VIP:

[root@director]# /usr/sbin/tcpdump -i eth0:0 -p port 80

        I think I can see the client request packet arrive?
        
21:19:05.799418 host81-152-187-242.range81-152.btcentralplus.com.4626 > 
66.98.227.143.http: S 1079944470:1079944470(0) win 16384 <mss 
1460,nop,nop,sackOK> (DF)
        
        I also see other packets hitting this interface which I assume are 
the polling packets from Ldirectord. 

        OK. So where does the client browser's request packet go from 
here? 

        Using tcpdump on the Real Server's hidden interface I see 
nothing, not even the polling from Ldirectord:

[root@realserver]# /usr/sbin/tcpdump -i lo:0 -p port 80

        So I assume Ldirectord must pass requests to a different 
interface. List the other interfaces with /sbin/ifconfig. Try eth0 
instead:

[root@realserver]# /usr/sbin/tcpdump -i eth0 -p port 80:

        Lots of packets - probably the polling from the Director. But do 
any belong to the client (browser)? Tee the output of tcpdump into a file 
so I can grep it later.

[root@realserver]# /usr/sbin/tcpdump -i eth0 -p port 80 | tee look
                
        Grepping the file does not show the IP address, or domain of the
client. Nothing there. How can I be sure there is a client packet sent by
the Director?  Grep the apache logs on the RealServer to see if the
request comes through there? Nothing there.

        It seems the request arrives at the Director but not at the 
RealServer. So have another look at the Director:

[root@director]# /sbin/ipsadm -L -n 

IP Virtual Server version 1.0.10 (size=65536)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  66.98.227.143:80 rr
  -> 66.98.186.73:80              Tunnel  1      0          0         

        Seems OK. But hang on. If I make the client request at exactly the 
same time I see:

[root@director]# /sbin/ipvsadm -L -n 

IP Virtual Server version 1.0.10 (size=65536)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  66.98.227.143:80 rr
  -> 66.98.186.73:80              Tunnel  1      0          1         

        
        The InActConn column is flagged!!? 

[root@director]# man ipvsadm

        Not much there about the meaning of InActConn (Inactive 
Connection?). Search the mailing list for "InActConn" 
(http://marc.theaimsgroup.com/?l=linux-virtual-server. Found a tip from 
Roberto Nibali to run ipvsadm using watch to see what's happening:

[root@director]# watch -n 1 '/sbin/ipvsadm -Ln'

        When I make a request from the client I can now clearly see the
InActConn count going up but never the ActConn.
        
        But now I've run out of ideas on what to try next? Any help would 
be much appreciated?

Thanks,


Nigel


-- 
Nigel Hamilton
Turbo10 Metasearch Engine

email:  nigel@xxxxxxxxxxx
tel:    +44 (0) 207 987 5460
fax:    +44 (0) 207 987 5468
________________________________________________________________________________
http://turbo10.com              Search Deeper. Browse Faster.

<Prev in Thread] Current Thread [Next in Thread>