LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] LdirectorD LVS and CentOS/Fedora/RedHat

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Re: [lvs-users] LdirectorD LVS and CentOS/Fedora/RedHat
From: partysoft <partysoft@xxxxxxxxx>
Date: Thu, 22 Oct 2009 12:39:38 -0700 (PDT)
Thank you Léon for the reply. I apreciate it so much.
Yes i actually have 3 public ip's like (none are with 192..)
XX.XX.XX.234 (this is the lvs..)
XX.XX.XX.235 real server (web ngix)
YYY.YYY.YYY.163  real server (web apache) - I don't really care about this
one, i can move it into the

we will use XX.XX.XX.236 as virtual..(there isn't an ip on the net with that
number up). of course it will be simpler to use a 192.. but, i have tried
that also ,and no luck

same subnet as the first 2 ones, i just want to make it work from the 234 ->
235..but it gives me a timeout on the browser...
Here's what i did:

[root@linux ~]# cat /etc/ha.d/ldirectord.cf
checktimeout=3
checkinterval=10
autoreload=yes
logfile="/var/log/ldirectord.log"
quiescent=no
virtual=XX.XX.XX.236:80
        fallback=127.0.0.1:80
        real=XX.XX.XX.235:80 gate
        real=YYY.YYY.YYY.235:80 gate
        service=http
        request="test.html"
        receive="Still alive"
        scheduler=rr #here i've tried with wlr as well
        protocol=tcp
        checktype=negotiate

[root@linux ~]# /usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start

DEBUG2: Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start)
Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start)
DEBUG2: Invoking ldirectord invoked as: /usr/sbin/ldirectord
/etc/ha.d/ldirectord.cf start
Invoking ldirectord invoked as: /usr/sbin/ldirectord /etc/ha.d/ldirectord.cf
start
DEBUG2: Starting Linux Director v1.186-ha-2.1.4 with pid: 18619
Starting Linux Director v1.186-ha-2.1.4 with pid: 18619
DEBUG2: Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr )
Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr )
DEBUG2: Added virtual server: XX.XX.XX.236:80
Added virtual server: XX.XX.XX.236:80
DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r 127.0.0.1:80
-g -w 1)
Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r 127.0.0.1:80 -g -w 1)
DEBUG2: Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set to
1)
Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set to 1)
DEBUG2: Disabled real
server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ alive
(virtual=tcp:XX.XX.XX.236:80)
DEBUG2: Checking negotiate: real
server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ alive
(virtual=tcp:XX.XX.XX.236:80)
DEBUG2: check_http: url="http://XX.XX.XX.235:80/test.html";
virtualhost="XX.XX.XX.235"
LWP::UserAgent::new: ()
DEBUG2: Starting Check
DEBUG2: Starting HTTP/HTTPS
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://XX.XX.XX.235:80/test.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 12 bytes
LWP::UserAgent::request: Simple response: OK
DEBUG2: Finished HTTP/HTTPS
DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r
XX.XX.XX.235:80 -g -w 1)
Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w
1)
DEBUG2: system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w
1) failed: No child processes
system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1)
failed: No child processes
DEBUG2: Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to
1)
Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to 1)
DEBUG2: Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80)
Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80)
DEBUG2: system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) failed:
No child processes
system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) failed: No child
processes
DEBUG2: Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80)
Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80)
DEBUG2: Enabled  real
server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\ alive
(virtual=tcp:XX.XX.XX.236:80)
DEBUG2: check_http: http://XX.XX.XX.235:80/test.html is up

If i go to the webserver i can see that ldirector is actually testing the
test.html..every 10 seconds like in the conf

-----------------------------------------------

I am sure that is because of the configurations of the IPs and the aditional
eth0:0 and lo:0 and that's why it doesn't work, i will paste everything that
i did, maybe, just maybe you can help me out on this one, i'm really
stuck..probably because i don't know lots of stuff on how the OSI layer is
build and how arp works

Aditional Network conf..
XX.XX.XX.234 (this is the lvs..) is spawned on eth1..
so i spawned another eth1:0
[root@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1:0
DEVICE=eth1:0
IPADDR=XX.XX.XX.236  # this is from the same subnet and i doesn't conflict
with an existent one
NETMASK=255.255.255.0
NETWORK=XX.XX.XX.0
BROADCAST=XX.XX.XX.255
GATEWAY=XX.XX.XX.233 # i've put the same gateway as the default ip
XX.XX.XX.234 had
ONBOOT=yes

i also enabled port forwarding

net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296

And i did the modprobe with all those modules for IPV

modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr

and that's all i did for the LVS server..

Now for the webserver
XX.XX.XX.235 (this is the lvs..) is spawned on eth0..
root@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
IPADDR=XX.XX.XX.236
NETMASK=255.255.255.255
NETWORK=XX.XX.XX.XX.0
BROADCAST=XX.XX.XX.255
ONBOOT=yes
NAME=loopback

and

net.ipv4.ip_forward = 0
net.ipv4.conf.lo.arp_ignore = 1   #here i have tried with eth0 instead of
lo, no luck..
net.ipv4.conf.lo.arp_announce = 2  #here i have tried with eth0 instead of
lo, no luck..
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296

here i've tried with ip forward 0 and 1 , no luck the requests simply don't
reach this server only the direct ones

Now i've understand that this is an ARP problem, and as CentOS doesn't
support the arp hidden flag on sysctl , i tried with
/etc/init.d/arptables_jf

arptables -A IN -j DROP -d XX.XX.XX.236
-A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s XX.XX.XX.236
arptables -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s
XX.XX.XX.236
/etc/init.d/arptables_jf save
/etc/init.d/arptables_jf start
/sbin/arptables -L -v -n

hain IN (policy ACCEPT 1353 packets, 37884 bytes)
 pkts bytes target     in     out     source-ip            destination-ip      
source-hw          destination-hw     hlen   op         hrd        pro      
    0     0 DROP       *      *       0.0.0.0/0            XX.XX.XX.236      
00/00              00/00              any    0000/0000  0000/0000  0000/0000

Chain OUT (policy ACCEPT 25 packets, 700 bytes)
 pkts bytes target     in     out     source-ip            destination-ip      
source-hw          destination-hw     hlen   op         hrd        pro      
    0     0 mangle     *      eth0    XX.XX.XX.234       0.0.0.0/0           
00/00              00/00              any    0000/0000  0000/0000  0000/0000
--mangle-ip-s XX.XX.XX.236

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     in     out     source-ip            destination-ip      
source-hw          destination-hw     hlen   op         hrd        pro      

-----------------------------------------------

i've probed with ipvsadm or something like it, to see the active
connections, and they are always to 0

i've modprobed here the same modules, no luck..

So from this point i'm really stuck and don't know what to do...

Here's the ifconfig from both servers if that helps
from the LVS (xx.234)

eth1      Link encap:Ethernet  HWaddr 00:1B:21:46:3E:A9  
          inet addr:XX.XX.XX.234  Bcast:XX.XX.XX.239  Mask:255.255.255.248
          inet6 addr: fe80::21b:21ff:fe45:3ea9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:276338 errors:0 dropped:2 overruns:0 frame:0
          TX packets:220590 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:156995116 (149.7 MiB)  TX bytes:30754525 (29.3 MiB)
          Base address:0xd000 Memory:e2020000-e2040000

eth1:0    Link encap:Ethernet  HWaddr 00:1B:21:46:3E:A9  
          inet addr:XX.XX.XX.236  Bcast:XX.XX.XX.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Base address:0xd000 Memory:e2020000-e2040000

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:150 errors:0 dropped:0 overruns:0 frame:0
          TX packets:150 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:16592 (16.2 KiB)  TX bytes:16592 (16.2 KiB)

here's the one from the webserver

eth0      Link encap:Ethernet  HWaddr 00:24:1D:72:61:AB  
          inet addr:XX.XX.XX.235  Bcast:XX.XX.XX.239  Mask:255.255.255.248
          inet6 addr: fe80::224:1dff:fe74:61ab/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:301529 errors:0 dropped:0 overruns:0 frame:0
          TX packets:255827 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:149822776 (142.8 MiB)  TX bytes:36393789 (34.7 MiB)
          Interrupt:233 Base address:0x2000

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:56 errors:0 dropped:0 overruns:0 frame:0
          TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4448 (4.3 KiB)  TX bytes:4448 (4.3 KiB)

lo:0      Link encap:Local Loopback  
          inet addr:XX.XX.XX.236  Mask:255.255.255.255
          UP LOOPBACK RUNNING  MTU:16436  Metric:1

I thank you in advance for helping solve this problem. I think it's a
problem that many have with Fedora/Red Hat/ Cent-oS
 

L.S. Keijser-2 wrote:
> 
> On Wed, 2009-10-21 at 22:20 -0700, partysoft wrote:
>> Hello, i've been trying to setup on Centos 5 LdirectorD without
>> Heartbeat,
>> and everything seems to work but no redirection, the RealServers arren't
>> reached....
> 
>>From where? The director? The client? How did you come to this
> conclusion; packet sniffing, realserver logfiles, something else?
> 
>> The thing i don't understand is this: will All the servers have to be in
>> LAN
>> using a 192.XXX ips?
> 
> Depends on what you're trying to do.
> 
>> I actually have all the servers with a public IP, and no way to put them
>> in
>> LAN , i understood that i should use ipip (TUN) method, but i there isn't
>> a
>> good doc out there how to setup the tun0 on the servers, and stuff like
>> that
>> 
>> What if i put them in VPN will that simulate a LAN an let the gate method
>> work? i plan to use OpenVpn (that actually is through Tun)
>> 
>> Or is it because i haven't used Heartbeat service to spawn ldirectord?
>> i'm
>> completly lost..
> 
> Yes, it seems so :)  Why don't you start by reading the HOWTO? If you
> have realservers with a public IP and your director with a public IP in
> the same subnet, you could try setting up LVS-DR. It requires for you to
> add a loopback device on the realservers and, depending on the OS, solve
> the arp problem.
> 
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/
> 
> LVS-TUN is probably not what you want. Heartbeat and ldirectord are used
> mainly for when you're setting up a failover cluster with 2 or more
> directors. The best way to learn LVS is to set up the IPVS table
> manually so later you know what ldirector automagically does for you :)
> 
> -- 
> Léon
> 
> 
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
> 
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
> 

-- 
View this message in context: 
http://www.nabble.com/LdirectorD-LVS-and-CentOS-Fedora-RedHat-tp26004219p26015972.html
Sent from the LVS mailing list archive at Nabble.com.


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>