|
Hi Partica,
Sorry, I haven't read all your mail, but I think it's may helpful to you:
http://www.austintek.com/LVS/LVS-HOWTO/mini-HOWTO/LVS-mini-HOWTO.html#forwarding
And I think RHEL is good enough for LVS.
Goodluck,
-giobuon
On Fri, Oct 23, 2009 at 2:50 AM, Partica Cristian <partysoft@xxxxxxxxx>wrote:
> Wha't i'm trying to do is setup a load banancer using 3 servers for a
> webservice (HTTP)
>
> 1st - the main server which will redirect to the 2nd or 3rd server. I
> called
> this LVS (and it's where ldirectord is)
>
> 2nd - runs a simple http service and when a client enters...and puts the
> ip,
> or domain of the 1st server, he should get the contents from the 2nd server
> (if up). eighterwise the content from the 3rd server
>
> 3rd - runs the same http service..
>
> the only thing different from the ALL tutorials that i've read is that
> those
> 3 servers are on the internet , with public ips not behind some private
> 192.168.xx network
>
> 2 of the ip's are in the same subbet of a class C
> 1 is not even in the same continent :)
>
> So a relatively simple thing to do , that ldirectord doesn't do..in this
> case
> ----------------------
>
> I have 3 PUBLIC ip's like (none are with 192..) , two of them are on the
> same subnet,
> I'm running Cent-oS 5, and here is a COMPLETE list of what i have done.
>
> XX.XX.XX.234 (this is the lvs..)
> XX.XX.XX.235 real server (web ngix)
> YYY.YYY.YYY.163 real server (web apache) - I don't really care about this
> one, i can move it into the
>
> we will use XX.XX.XX.236 as virtual..(there isn't an ip on the net with
> that
> number up). of course it will be simpler to use a 192.. but, i have tried
> that also ,and no luck
>
> same subnet as the first 2 ones, i just want to make it work from the 234
> ->
> 235..but it gives me a timeout on the browser...
> Here's what i did:
>
> [root@linux ~]# cat /etc/ha.d/ldirectord.cf
> checktimeout=3
> checkinterval=10
> autoreload=yes
> logfile="/var/log/ldirectord.log"
> quiescent=no
> virtual=XX.XX.XX.236:80
> fallback=127.0.0.1:80
> real=XX.XX.XX.235:80 gate
> real=YYY.YYY.YYY.235:80 gate
> service=http
> request="test.html"
> receive="Still alive"
> scheduler=rr #here i've tried with wlr as well
> protocol=tcp
> checktype=negotiate
>
> [root@linux ~]# /usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start
>
> DEBUG2: Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cfstart)
> Running exec(/usr/sbin/ldirectord -d /etc/ha.d/ldirectord.cf start)
> DEBUG2: Invoking ldirectord invoked as: /usr/sbin/ldirectord /etc/ha.d/
> ldirectord.cf start
> Invoking ldirectord invoked as: /usr/sbin/ldirectord
> /etc/ha.d/ldirectord.cfstart
> DEBUG2: Starting Linux Director v1.186-ha-2.1.4 with pid: 18619
> Starting Linux Director v1.186-ha-2.1.4 with pid: 18619
> DEBUG2: Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr )
> Running system(/sbin/ipvsadm -A -t XX.XX.XX.236:80 -s rr )
> DEBUG2: Added virtual server: XX.XX.XX.236:80
> Added virtual server: XX.XX.XX.236:80
> DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r
> 127.0.0.1:80-g -w 1)
> Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r 127.0.0.1:80 -g -w
> 1)
> DEBUG2: Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set
> to
> 1)
> Added fallback server: 127.0.0.1:80 (XX.XX.XX.236:80) (Weight set to 1)
> DEBUG2: Disabled real
> server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\
> alive
> (virtual=tcp:XX.XX.XX.236:80)
> DEBUG2: Checking negotiate: real
> server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\
> alive
> (virtual=tcp:XX.XX.XX.236:80)
> DEBUG2: check_http: url="http://XX.XX.XX.235:80/test.html";
> virtualhost="XX.XX.XX.235"
> LWP::UserAgent::new: ()
> DEBUG2: Starting Check
> DEBUG2: Starting HTTP/HTTPS
> LWP::UserAgent::request: ()
> LWP::UserAgent::send_request: GET
> http://XX.XX.XX.235:80/test.html<http://xx.xx.xx.235/test.html>
> LWP::UserAgent::_need_proxy: Not proxied
> LWP::Protocol::http::request: ()
> LWP::Protocol::collect: read 12 bytes
> LWP::UserAgent::request: Simple response: OK
> DEBUG2: Finished HTTP/HTTPS
> DEBUG2: Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r
> XX.XX.XX.235:80 -g -w 1)
> Running system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w
> 1)
> DEBUG2: system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w
> 1) failed: No child processes
> system(/sbin/ipvsadm -a -t XX.XX.XX.236:80 -r XX.XX.XX.235:80 -g -w 1)
> failed: No child processes
> DEBUG2: Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to
> 1)
> Added real server: XX.XX.XX.235:80 (XX.XX.XX.236:80) (Weight set to 1)
> DEBUG2: Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80
> )
> Running system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80)
> DEBUG2: system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80)
> failed:
> No child processes
> system(/sbin/ipvsadm -d -t XX.XX.XX.236:80 -r 127.0.0.1:80) failed: No
> child
> processes
> DEBUG2: Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80)
> Deleted fallback server: 127.0.0.1:80 (XX.XX.XX.236:80)
> DEBUG2: Enabled real
> server=negotiate:http:tcp:XX.XX.XX.235:80:::1:gate:\/test\.html:Still\
> alive
> (virtual=tcp:XX.XX.XX.236:80)
> DEBUG2: check_http:
> http://XX.XX.XX.235:80/test.html<http://xx.xx.xx.235/test.html>is up
>
> If i go to the webserver i can see that ldirector is actually testing the
> test.html..every 10 seconds like in the conf
>
> -----------------------------------------------
>
> I am sure that is because of the configurations of the IPs and the
> aditional
> eth0:0 and lo:0 and that's why it doesn't work, i will paste everything
> that
> i did, maybe, just maybe you can help me out on this one, i'm really
> stuck..probably because i don't know lots of stuff on how the OSI layer is
> build and how arp works
>
> Aditional Network conf..
> XX.XX.XX.234 (this is the lvs..) is spawned on eth1..
> so i spawned another eth1:0
> [root@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1:0
> DEVICE=eth1:0
> IPADDR=XX.XX.XX.236 # this is from the same subnet and i doesn't conflict
> with an existent one
> NETMASK=255.255.255.0
> NETWORK=XX.XX.XX.0
> BROADCAST=XX.XX.XX.255
> GATEWAY=XX.XX.XX.233 # i've put the same gateway as the default ip
> XX.XX.XX.234 had
> ONBOOT=yes
>
> i also enabled port forwarding
>
> net.ipv4.ip_forward = 1
> net.ipv4.conf.default.rp_filter = 1
> net.ipv4.conf.default.accept_source_route = 0
> kernel.sysrq = 0
> kernel.core_uses_pid = 1
> net.ipv4.tcp_syncookies = 1
> kernel.msgmnb = 65536
> kernel.msgmax = 65536
> kernel.shmmax = 68719476736
> kernel.shmall = 4294967296
>
> And i did the modprobe with all those modules for IPV
>
> modprobe ip_vs_dh
> modprobe ip_vs_ftp
> modprobe ip_vs_dh
> modprobe ip_vs_ftp
> modprobe ip_vs
> modprobe ip_vs_lblc
> modprobe ip_vs_lblcr
> modprobe ip_vs_lc
> modprobe ip_vs_nq
> modprobe ip_vs_rr
> modprobe ip_vs_sed
> modprobe ip_vs_sh
> modprobe ip_vs_wlc
> modprobe ip_vs_wrr
>
> and that's all i did for the LVS server..
>
> Now for the webserver
> XX.XX.XX.235 (this is the lvs..) is spawned on eth0..
> root@linux ~]# cat /etc/sysconfig/network-scripts/ifcfg-lo:0
> DEVICE=lo:0
> IPADDR=XX.XX.XX.236
> NETMASK=255.255.255.255
> NETWORK=XX.XX.XX.XX.0
> BROADCAST=XX.XX.XX.255
> ONBOOT=yes
> NAME=loopback
>
> and
>
> net.ipv4.ip_forward = 0
> net.ipv4.conf.lo.arp_ignore = 1 #here i have tried with eth0 instead of lo,
> no luck..
> net.ipv4.conf.lo.arp_announce = 2 #here i have tried with eth0 instead of
> lo, no luck..
> net.ipv4.conf.all.arp_ignore = 1
> net.ipv4.conf.all.arp_announce = 2
> net.ipv4.conf.default.rp_filter = 1
> net.ipv4.conf.default.accept_source_route = 0
> kernel.sysrq = 0
> kernel.core_uses_pid = 1
> net.ipv4.tcp_syncookies = 1
> kernel.msgmnb = 65536
> kernel.msgmax = 65536
> kernel.shmmax = 68719476736
> kernel.shmall = 4294967296
>
> here i've tried with ip forward 0 and 1 , no luck the requests simply don't
> reach this server only the direct ones
>
> Now i've understand that this is an ARP problem, and as CentOS doesn't
> support the arp hidden flag on sysctl , i tried with
> /etc/init.d/arptables_jf
>
> arptables -A IN -j DROP -d XX.XX.XX.236
> -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s XX.XX.XX.236
> arptables -A OUT -j mangle -o eth0 -s XX.XX.XX.234 --mangle-ip-s
> XX.XX.XX.236
> /etc/init.d/arptables_jf save
> /etc/init.d/arptables_jf start
> /sbin/arptables -L -v -n
>
> hain IN (policy ACCEPT 1353 packets, 37884 bytes)
> pkts bytes target in out source-ip destination-ip source-hw destination-hw
> hlen op hrd pro
> 0 0 DROP * * 0.0.0.0/0 XX.XX.XX.236 00/00 00/00 any 0000/0000 0000/0000
> 0000/0000
>
> Chain OUT (policy ACCEPT 25 packets, 700 bytes)
> pkts bytes target in out source-ip destination-ip source-hw destination-hw
> hlen op hrd pro
> 0 0 mangle * eth0 XX.XX.XX.234 0.0.0.0/0 00/00 00/00 any 0000/0000
> 0000/0000
> 0000/0000 --mangle-ip-s XX.XX.XX.236
>
> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> pkts bytes target in out source-ip destination-ip source-hw destination-hw
> hlen op hrd pro
>
> -----------------------------------------------
>
> i've probed with ipvsadm or something like it, to see the active
> connections, and they are always to 0
>
> i've modprobed here the same modules, no luck..
>
> So from this point i'm really stuck and don't know what to do...
>
> Here's the ifconfig from both servers if that helps
> from the LVS (xx.234)
>
> eth1 Link encap:Ethernet HWaddr 00:1B:21:46:3E:A9
> inet addr:XX.XX.XX.234 Bcast:XX.XX.XX.239 Mask:255.255.255.248
> inet6 addr: fe80::21b:21ff:fe45:3ea9/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:276338 errors:0 dropped:2 overruns:0 frame:0
> TX packets:220590 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:156995116 (149.7 MiB) TX bytes:30754525 (29.3 MiB)
> Base address:0xd000 Memory:e2020000-e2040000
>
> eth1:0 Link encap:Ethernet HWaddr 00:1B:21:46:3E:A9
> inet addr:XX.XX.XX.236 Bcast:XX.XX.XX.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> Base address:0xd000 Memory:e2020000-e2040000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:150 errors:0 dropped:0 overruns:0 frame:0
> TX packets:150 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:16592 (16.2 KiB) TX bytes:16592 (16.2 KiB)
>
> here's the one from the webserver
>
> eth0 Link encap:Ethernet HWaddr 00:24:1D:72:61:AB
> inet addr:XX.XX.XX.235 Bcast:XX.XX.XX.239 Mask:255.255.255.248
> inet6 addr: fe80::224:1dff:fe74:61ab/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:301529 errors:0 dropped:0 overruns:0 frame:0
> TX packets:255827 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:149822776 (142.8 MiB) TX bytes:36393789 (34.7 MiB)
> Interrupt:233 Base address:0x2000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:56 errors:0 dropped:0 overruns:0 frame:0
> TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:4448 (4.3 KiB) TX bytes:4448 (4.3 KiB)
>
> lo:0 Link encap:Local Loopback
> inet addr:XX.XX.XX.236 Mask:255.255.255.255
> UP LOOPBACK RUNNING MTU:16436 Metric:1
>
> i've checked the firewall it's disabled , iptables stoped, selinux is
> disabled..
>
> I am thanking you in advance for helping solve this problem. I think it's
> a
> problem that many have with Fedora/Red Hat/ Cent-oS
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
