LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Please Help for my https problem~!

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Please Help for my https problem~!
Cc: Anthony Cheung <anthony.cheung@xxxxxxxxxxxxxxxxxxxx>
Cc: Alex Ho <alex.ho@xxxxxxxxxxxxxxxxxxxx>
From: "Louis Lam" <louis.lam@xxxxxxxxxxxxxxxxxxxx>
Date: Tue, 25 Oct 2005 17:48:46 +0800
Dear All:

I have try to setup a LVS https service for a month. However, i'm fail to 
config it successfully, 

The situation is All https connection have been stopped at Director. But it can 
run smoothly with http service.
I have solved the ARP problem,  The https and http service can access directly 
through the Real Server IP,  but fail to connect the https service through VIP

this is the detail of my web farm:
VIP             10.0.58.136
Director        10.0.58.231
Real Server1    10.0.58.232
Real Server2    10.0.58.230

========================================
The Specification of Director:

OS Red Hat ES3.0        kernel 2.4.21-27.0.2.EL.um.1

UltraMonkey 2.01 
  heartbeat-1.0.4-2.rh.el.um.1.i386.rpm 
  heartbeat-ldirectord-1.0.4-2.rh.el.um.1.i386.rpm 
  heartbeat-pils-1.0.4-2.rh.el.um.1.i386.rpm 
  heartbeat-stonith-1.0.4-2.rh.el.um.1.i386.rpm 
  ipvsadm-1.21-1.rh.el.1.i386.rpm 
  libnet-1.1.0-1.rh.el.1.i386.rpm 
  perl-Authen-SASL-2.03-1.rh.el.um.1.noarch.rpm 
  perl-Convert-ASN1-0.16-2.rh.el.um.1.noarch.rpm 
  perl-IO-Socket-SSL-0.92-1.rh.el.um.1.noarch.rpm 
  perl-Mail-IMAPClient-2.2.7-1.rh.el.um.1.noarch.rpm 
  perl-Net-SSLeay-1.23-1.rh.el.um.1.i386.rpm 
  perl-Parse-RecDescent-1.80-1.rh.el.um.1.noarch.rpm 
  perl-XML-NamespaceSupport-1.08-1.rh.el.um.1.noarch.rpm 
  perl-XML-SAX-0.12-1.rh.el.um.1.noarch.rpm 
  perl-ldap-0.2701-1.rh.el.um.1.noarch.rpm 

Extra package
perl-Crypt-SSLeay-0.51-alt2.i586.rpm
perl-Crypt-OpenSSL-DSA-0.12-1.1.el3.rf.i386.rpm
===========================================
The Specification of the Two Real Server:

OS:  Red Hat 8.0                kernel 2.4.20-31.9.um.3 

Web Server package:
2. Jdk 1.4.2_09
3. Apache 1.3.14
4. Openssl 0.9.6g
5. ApacheJServ-1.1.2
6. Gnujsp-1.0.1
7. mod_ssl-2.7.1-1.3.14
===========================================
The configuration step:

On Director:
ifconfig eth0:0 10.0.58.136 netmask 255.255.255.0 broadcast 10.0.58.255
echo 1 > /proc/sys/net/ipv4/ip_forward

ipvsadm -A -t 10.0.58.136:80 -s rr
ipvsadm -a -t 10.0.58.136:80 -r 10.0.58.230 -g
ipvsadm -a -t 10.0.58.136:80 -r 10.0.58.232 -g

ipvsadm -A -t 10.0.58.136:443 -s rr
ipvsadm -a -t 10.0.58.136:443 -r 10.0.58.230 -g
ipvsadm -a -t 10.0.58.136:443 -r 10.0.58.232 -g

/usr/sbin/ldirectord start
-------------------------------------
Ldirectord.cf
#
# Sample ldirectord configuration file to configure various virtual services.
#
# Ldirectord will connect to each real server once per second and request
# /index.html. If the data returned by the server does not contain the
# string "Test Message" then the test fails and the real server will be
# taken out of the available pool. The real server will be added back into
# the pool once the test succeeds. If all real servers are removed from the
# pool then localhost:80 is added to the pool as a fallback measure.

# Global Directives
checktimeout=500
checkinterval=1
fallback=127.0.0.1:80
autoreload=yes
logfile="/var/log/ldirectord.log"
#logfile="local0"
quiescent=yes

# A sample virual with a fallback that will override the gobal setting
virtual=10.0.58.136:80
        real=10.0.58.230:80 gate
        real=10.0.58.232:80 gate
        real=192.168.6.6:80 gate
        fallback=127.0.0.1:80 gate
        service=http
        request="index.htm"
        receive="abc"
        scheduler=rr
        #persistent=600
        netmask=255.255.255.255
        protocol=tcp

#Sample configuration for an https virtual service.
#Fallback setting overides global
virtual=10.0.58.136:443
        real=10.0.58.230:443 gate
        real=10.0.58.232:443 gate
        fallback=127.0.0.1:443
        service=https
        scheduler=rr
        request="test.htm"
        receive="abc"
        scheduler=rr
        #persistent=600
        netmask=255.255.255.255
        protocol=tcp
-------------------------------------
**No setting on Iptable
**No firewall

===========================================
The configuration step on Real Server

ifconfig lo:0 10.0.58.136 netmask 255.255.255.255 broadcast 10.0.58.136
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/all/hidden
echo 1 > /proc/sys/net/ipv4/conf/lo/hidden 
/usr/local/apache/bin/apachectl start

**No setting on Iptable
**No firewall
===========================================
The Log files
------------------------------------------------------
ipvsadm 

Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.58.136:https rr persistent 600
          -> 10.0.58.232:https            Route   1      0          0
          -> 10.0.58.230:https            Route   1      0          0
  ->  localhost.localdomain:https  Local   0      0          0

TCP  10.0.58.136:http rr
  -> 10.0.58.232:http             Route   1      0          0
  -> 10.0.58.230:http             Route   1      0          0
------------------------------------------------------

ipvsadm -L -c -n
TCP 00:01  CLOSE       10.0.58.93:1828    10.0.58.136:443    10.0.58.230:443
TCP 00:05  CLOSE       10.0.58.93:1840    10.0.58.136:443    10.0.58.230:443
TCP 00:04  CLOSE       10.0.58.93:1833    10.0.58.136:443    10.0.58.232:443
TCP 00:05  CLOSE       10.0.58.93:1838    10.0.58.136:443    10.0.58.230:443
TCP 00:03  CLOSE       10.0.58.93:1829    10.0.58.136:443    10.0.58.232:443
TCP 00:04  CLOSE       10.0.58.93:1834    10.0.58.136:443    10.0.58.230:443
TCP 00:05  CLOSE       10.0.58.93:1837    10.0.58.136:443    10.0.58.232:443
TCP 00:04  CLOSE       10.0.58.93:1835    10.0.58.136:443    10.0.58.232:443
TCP 00:04  CLOSE       10.0.58.93:1832    10.0.58.136:443    10.0.58.230:443
TCP 01:20  FIN_WAIT    10.0.58.93:1822    10.0.58.136:80     10.0.58.232:80
TCP 00:04  CLOSE       10.0.58.93:1831    10.0.58.136:443    10.0.58.232:443
TCP 00:03  CLOSE       10.0.58.93:1830    10.0.58.136:443    10.0.58.230:443
TCP 00:05  CLOSE       10.0.58.93:1836    10.0.58.136:443    10.0.58.230:443
TCP 00:05  CLOSE       10.0.58.93:1839    10.0.58.136:443    10.0.58.232:443
-------------------------------------------------------

tcpdump -n -i any port 443 on Director 
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: listening on any
17:29:17.345139 10.0.58.231.41440 > 10.0.58.230.https: S 
3329915465:3329915465(0) win 5840 <mss 1460,sackOK,timestamp 866633 
0,nop,wscale 0> (DF)
17:29:17.345609 10.0.58.230.https > 10.0.58.231.41440: S 
1855908101:1855908101(0) ack 3329915466 win 5792 <mss 1460,sackOK,timestamp 
43791285 866633,nop,wscale 0> (DF)
17:29:17.345622 10.0.58.231.41440 > 10.0.58.230.https: . ack 1 win 5840 
<nop,nop,timestamp 866633 43791285> (DF)
17:29:17.346450 10.0.58.231.41440 > 10.0.58.230.https: P 1:127(126) ack 1 win 
5840 <nop,nop,timestamp 866633 43791285> (DF)
17:29:17.347008 10.0.58.230.https > 10.0.58.231.41440: . ack 127 win 5792 
<nop,nop,timestamp 43791285 866633> (DF)
17:29:17.361403 10.0.58.230.https > 10.0.58.231.41440: . 1:1449(1448) ack 127 
win 5792 <nop,nop,timestamp 43791286 866633> (DF)
17:29:17.361410 10.0.58.231.41440 > 10.0.58.230.https: . ack 1449 win 8688 
<nop,nop,timestamp 866634 43791286> (DF)
17:29:17.361803 10.0.58.230.https > 10.0.58.231.41440: P 1449:1844(395) ack 127 
win 5792 <nop,nop,timestamp 43791286 866633> (DF)
17:29:17.361810 10.0.58.231.41440 > 10.0.58.230.https: . ack 1844 win 8688 
<nop,nop,timestamp 866634 43791286> (DF)
------------------------------------------------------
tcpdump -n -i any port 443 on Real Server
tcpdump: listening on any
17:30:23.288379 10.0.58.231.41577 > 10.0.58.232.https: S 
3414509979:3414509979(0) win 5840 <mss 1460,sackOK,timestamp 874194 
0,nop,wscale 0> (DF)
17:30:23.288397 10.0.58.232.https > 10.0.58.231.41577: S 
3321698202:3321698202(0) ack 3414509980 win 5792 <mss 1460,sackOK,timestamp 
69371828 874194,nop,wscale                                              0> (DF)
17:30:23.289377 10.0.58.231.41577 > 10.0.58.232.https: . ack 1 win 5840 
<nop,nop,timestamp 874194 69371828> (DF)
17:30:23.289627 10.0.58.231.41577 > 10.0.58.232.https: P 1:127(126) ack 1 win 
5840 <nop,nop,timestamp 874194 69371828> (DF)
17:30:23.289635 10.0.58.232.https > 10.0.58.231.41577: . ack 127 win 5792 
<nop,nop,timestamp 69371828 874194> (DF)
------------------------------------------------------


Do anyone have experience on this issue below?

Please help me to find out the Problem or How can I narrow  the source of 
problem?

MANY MANY THANKS


Louis




<Prev in Thread] Current Thread [Next in Thread>