Subject: [lvs-users] ldirectord doesn't bind on VIP:port

From: "Tim Bourne" <[email-masked]>
Date: Tue, 3 Mar 2009 16:58:53 -0600

I'm rather new to the LVS world(although I've been in linux for years),
so pardon my ignorance on some of the topics/technologies.  The short
story is that I'm running ubnutu 8.04 LTS x64.  I'm trying to set up
ldirectord on a standalone box for now(I'll juice it up with heartbeat
in a few weeks).  Below is a quick list of configs and outputs.  The
problem I'm running into is that, while ipvsadm shows my VIP and RIP's
all up and functioning, nothing is listening on the VIP:port shown by
ipvsadm -L -n.  I've followed
http://www.howtoforge.com/set-up-a-loadbalanced-ha-apache-cluster-ubuntu
8.04 as my setup guide, save that I didn't add heartbeat to the mix.

Installed ldirectord with:

apt-get install ldirectord

Here's the config, located in /etc/ha.d:

# Global Directives
checktimeout=3
checkinterval=10
quiescent=no
logfile="/var/log/ldirectord.log"
autoreload=yes

virtual=10.16.14.66:80
        real=172.16.1.153:2500 masq
        real=172.16.1.153:2501 masq
        real=172.16.1.153:2502 masq
        scheduler=wlc
        service=none
        persistent=30
        checktype=connect
        protocol=tcp

When I launch it in debug mode with "/etc/ha.d/resource.d/ldirectord -d
ldirectord.cf start", here's the initial startup output:

DEBUG2: Running exec(/etc/ha.d/resource.d/ldirectord -d ldirectord.cf
start)
Running exec(/etc/ha.d/resource.d/ldirectord -d ldirectord.cf start)
DEBUG2: Invoking ldirectord invoked as: /etc/ha.d/resource.d/ldirectord
ldirectord.cf start
Invoking ldirectord invoked as: /etc/ha.d/resource.d/ldirectord
ldirectord.cf start
DEBUG2: Starting Linux Director v1.186-ha-2.1.3 with pid: 15113
Starting Linux Director v1.186-ha-2.1.3 with pid: 15113
DEBUG2: Running system(/sbin/ipvsadm -A -t 10.16.14.66:80 -s wlc -p 30 )
Running system(/sbin/ipvsadm -A -t 10.16.14.66:80 -s wlc -p 30 )
DEBUG2: Added virtual server: 10.16.14.66:80
Added virtual server: 10.16.14.66:80
DEBUG2: Disabled real server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG2: Disabled real server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG2: Disabled real server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2500)
DEBUG2: Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r
172.16.1.153:2500 -m -w 1)
Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r 172.16.1.153:2500
-m -w 1)
DEBUG2: Added real server: 172.16.1.153:2500 (10.16.14.66:80) (Weight
set to 1)
Added real server: 172.16.1.153:2500 (10.16.14.66:80) (Weight set to 1)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2500
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2501)
DEBUG2: Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r
172.16.1.153:2501 -m -w 1)
Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r 172.16.1.153:2501
-m -w 1)
DEBUG2: Added real server: 172.16.1.153:2501 (10.16.14.66:80) (Weight
set to 1)
Added real server: 172.16.1.153:2501 (10.16.14.66:80) (Weight set to 1)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2501
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2502)
DEBUG2: Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r
172.16.1.153:2502 -m -w 1)
Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r 172.16.1.153:2502
-m -w 1)
DEBUG2: Added real server: 172.16.1.153:2502 (10.16.14.66:80) (Weight
set to 1)
Added real server: 172.16.1.153:2502 (10.16.14.66:80) (Weight set to 1)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2502
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2500)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2500
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2501)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2501
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2502)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2502

Here's the output of ipvsadm -L -n when started from
/etc/init.d/ldirectord start:

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.16.14.66:80 wlc persistent 30
  -> 172.16.1.153:2502            Masq    1      0          0
  -> 172.16.1.153:2501            Masq    1      0          0
  -> 172.16.1.153:2500            Masq    1      0          0

Here's ifconfig:

eth0      Link encap:Ethernet  HWaddr 00:1e:0b:3c:9f:e4
          inet addr:10.16.14.65  Bcast:10.16.14.255  Mask:255.255.255.0
          inet6 addr: fe80::21e:bff:fe3c:9fe4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:33765 errors:14 dropped:0 overruns:0 frame:0
          TX packets:41937 errors:0 dropped:0 overruns:0 carrier:0
          collisions:556 txqueuelen:1000
          RX bytes:5883114 (5.6 MB)  TX bytes:6308182 (6.0 MB)
          Interrupt:19

eth0:0    Link encap:Ethernet  HWaddr 00:1e:0b:3c:9f:e4
          inet addr:10.16.14.66  Bcast:10.16.14.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19

Here's the result of netstat -pant:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address
State       PID/Program name
tcp        0      0 10.16.14.65:55391       172.16.1.153:2500
TIME_WAIT   -
tcp        0      0 10.16.14.65:55532       172.16.1.153:2501
TIME_WAIT   -
tcp        0      0 10.16.14.65:55544       172.16.1.153:2501
TIME_WAIT   -
tcp        0      0 10.16.14.65:55541       172.16.1.153:2501
TIME_WAIT   -
tcp        0      0 10.16.14.65:33730       172.16.1.153:2502
TIME_WAIT   -
tcp        0      0 10.16.14.65:33718       172.16.1.153:2502
TIME_WAIT   -
tcp        0      0 10.16.14.65:33727       172.16.1.153:2502
TIME_WAIT   -
tcp        0      0 10.16.14.65:55397       172.16.1.153:2500
TIME_WAIT   -
tcp        0      0 10.16.14.65:33721       172.16.1.153:2502
TIME_WAIT   -
tcp        0      0 10.16.14.65:55394       172.16.1.153:2500
TIME_WAIT   -
tcp        0      0 10.16.14.65:55388       172.16.1.153:2500
TIME_WAIT   -
tcp        0      0 10.16.14.65:55538       172.16.1.153:2501
TIME_WAIT   -
tcp        0      0 10.16.14.65:33724       172.16.1.153:2502
TIME_WAIT   -
tcp        0      0 10.16.14.65:55385       172.16.1.153:2500
TIME_WAIT   -
tcp        0      0 10.16.14.65:55535       172.16.1.153:2501
TIME_WAIT   -
tcp6       0      0 :::22                   :::*
LISTEN      4538/sshd
tcp6       0    148 10.16.14.65:22          10.16.14.131:49391
ESTABLISHED 4826/0

Obviously there's nothing listening on 10.16.14.66, except for SSH which
I can connect to.  I've installed apache and had it bind on that address
and port just to make sure there wasn't something funny going on with
the IP.  I'm afraid I'm missing something simple, but I've been reading
old LVS mailing lists and other blog posts for days and can't seem to
find an answer.  I'll gladly add whatever other config/troubleshooting
info is necessary.  Thanks for any help or ideas you can give.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - [email-masked]
Send requests to [email-masked]
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

---

From: Simon Horman <[email-masked]>
Date: Wed, 4 Mar 2009 10:45:11 +1100

> 8.04 as my setup guide, save that I didn't add heartbeat to the mix.

Hi Tim,

the quick answer to your question is that ldirectord isn't supposed
to bind to VIP:port and this shouldn't be a problem.

The slightly longer answer is that ldirectord effects load balancing by
configuring LVS, which lives in the kernel and acts much more like a router
than a proxy. As such, it does not bind a socket to VIP:port.  Rather, as
packets pass through the kernel's networking code they will be processed by
LVS if they match the VIP:port specified _and_ those packets would
otherwise have been delivered to a local process (this is one area where
LVS isn't quite like a router).  Typically second criteria is satisfied by
placing the VIP on a local interface.

-- 
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en

---

From: "Kit Gerrits" <[email-masked]>
Date: Wed, 4 Mar 2009 00:57:16 +0100

You might also try 'ip addr' to look for the cluster IP.
Certain packages (Like Red Hat Clustering), don't show their IP with
ifconfig, but it will show up with 'ip addr'

Kit 

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Tim Bourne
Sent: dinsdag 3 maart 2009 23:59
To: [email-masked]
Subject: [lvs-users] ldirectord doesn't bind on VIP:port

I'm rather new to the LVS world(although I've been in linux for years), so
pardon my ignorance on some of the topics/technologies.  The short story is
that I'm running ubnutu 8.04 LTS x64.  I'm trying to set up ldirectord on a
standalone box for now(I'll juice it up with heartbeat in a few weeks).
Below is a quick list of configs and outputs.  The problem I'm running into
is that, while ipvsadm shows my VIP and RIP's all up and functioning,
nothing is listening on the VIP:port shown by ipvsadm -L -n.  I've followed
http://www.howtoforge.com/set-up-a-loadbalanced-ha-apache-cluster-ubuntu
8.04 as my setup guide, save that I didn't add heartbeat to the mix.

# Global Directives
checktimeout=3
checkinterval=10
quiescent=no

virtual=10.16.14.66:80

DEBUG2: Running exec(/etc/ha.d/resource.d/ldirectord -d ldirectord.cf
start)
Running exec(/etc/ha.d/resource.d/ldirectord -d ldirectord.cf start)
DEBUG2: Invoking ldirectord invoked as: /etc/ha.d/resource.d/ldirectord
ldirectord.cf start Invoking ldirectord invoked as:
/etc/ha.d/resource.d/ldirectord ldirectord.cf start
DEBUG2: Starting Linux Director v1.186-ha-2.1.3 with pid: 15113 Starting
Linux Director v1.186-ha-2.1.3 with pid: 15113
DEBUG2: Running system(/sbin/ipvsadm -A -t 10.16.14.66:80 -s wlc -p 30 )
Running system(/sbin/ipvsadm -A -t 10.16.14.66:80 -s wlc -p 30 )
DEBUG2: Added virtual server: 10.16.14.66:80 Added virtual server:
10.16.14.66:80
DEBUG2: Disabled real server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG2: Disabled real server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG2: Disabled real server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2500)
DEBUG2: Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r
172.16.1.153:2500 -m -w 1) Running system(/sbin/ipvsadm -a -t 10.16.14.66:80
-r 172.16.1.153:2500 -m -w 1)
DEBUG2: Added real server: 172.16.1.153:2500 (10.16.14.66:80) (Weight set to
1) Added real server: 172.16.1.153:2500 (10.16.14.66:80) (Weight set to 1)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2500
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2501)
DEBUG2: Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r
172.16.1.153:2501 -m -w 1)
Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r 172.16.1.153:2501 -m -w
1)
DEBUG2: Added real server: 172.16.1.153:2501 (10.16.14.66:80) (Weight set to
1) Added real server: 172.16.1.153:2501 (10.16.14.66:80) (Weight set to 1)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2501
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2502)
DEBUG2: Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r
172.16.1.153:2502 -m -w 1)
Running system(/sbin/ipvsadm -a -t 10.16.14.66:80 -r 172.16.1.153:2502 -m -w
1)
DEBUG2: Added real server: 172.16.1.153:2502 (10.16.14.66:80) (Weight set to
1) Added real server: 172.16.1.153:2502 (10.16.14.66:80) (Weight set to 1)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2502
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2500)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2500:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2500
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2501)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2501:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2501
DEBUG2: Checking connect: real
server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Connected to 172.16.1.153 (port 2502)
DEBUG2: Enabled  real server=connect:tcp:172.16.1.153:2502:::1:masq:\/:
(virtual=tcp:10.16.14.66:80)
DEBUG3: Activated service 172.16.1.153:2502

Here's the output of ipvsadm -L -n when started from /etc/init.d/ldirectord
start:

IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler
Flags

Here's ifconfig:

Obviously there's nothing listening on 10.16.14.66, except for SSH which I
can connect to.  I've installed apache and had it bind on that address and
port just to make sure there wasn't something funny going on with the IP.
I'm afraid I'm missing something simple, but I've been reading old LVS
mailing lists and other blog posts for days and can't seem to find an
answer.  I'll gladly add whatever other config/troubleshooting info is
necessary.  Thanks for any help or ideas you can give.

LinuxVirtualServer.org mailing list - [email-masked] Send
requests to [email-masked]
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

---

From: "Tim Bourne" <[email-masked]>
Date: Wed, 4 Mar 2009 13:43:48 -0600

Thanks for the tips!

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Kit
Gerrits
Sent: Tuesday, March 03, 2009 5:57 PM
To: 'LinuxVirtualServer.org users mailing list.'
Subject: Re: [lvs-users] ldirectord doesn't bind on VIP:port

Kit