|
Hey all,
So I've Google'd around and there doesn't seem to be a solution to the
problems I'm having setting up this new system. Any thoughts or
suggestions would be really appreciated. If you need any more
information about the system or setting let me know and I'll add on to
it.
Problem description:
I've been trying to setup a distributed and HA environment. The setup is
like this:
INTERNET
==========================
| \ / |
| \ / |
| X |VIRTUAL IP: 192.168.0.20
Real IP(RIP)| / \ |===================
192.168.0.21| / \ | RIP: 192.168.0.22
--------------- ---------------
|loadbalancer1| |loadbalancer2| (heartbeat slave)
--------------- ---------------
RIP | \ / | RIP: 172.16.0.3
172.16.0.2 | \ / | =================
| X | VIP: 172.16.0.1
| / \ |
| / \ |
-------- --------
| App1 | | App2 |
-------- --------
RIP: 172.16.0.101 RIP: 172.16.0.102
Requests from the "Internet" (192.168.0 network) come in and get
balanced and the responses get back to the client. If there is a
request from within the 172.16. network it just stalls out and the
connection times out.
When the connection tries the ipvsadm state is reported:
lb1:~# ipvsadm -lnc
IPVS connection entries
pro expire state source virtual destination
TCP 00:52 SYN_RECV 172.16.0.101:49430 172.16.0.1:3306
172.16.0.101:3306
I'm not sure where to look, I think it is a routing problem because the
public ip addresses get the values back properly. Any suggestions on
where to look?
All the boxes are running Ubuntu:
uname -a
Linux app1 2.6.31-14-server #48-Ubuntu SMP Fri Oct 16 15:07:34 UTC 2009
x86_64 GNU/Linux
LOADBALANCERS
========================================================================
============
lb1:~# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt
Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
bond0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
bond1
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0
bond0
@lb1:~# ldirectord -v
Linux Director v1.186-ha-2.99.3
@lb1:~# ipvsadm -v
ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1)
@lb1:~# cat /etc/ha.d/ldirectord.cf
checktimeout=10
checkinterval=2
autoreload=no
logfile="/var/log/ldirectord.log"
quiescent=yes
#FOR DNS - CONNECT DOESNT WORK, MUST BE PATCHED BUT PING IS OK
virtual=192.168.0.20:53
real=172.16.0.101:53 masq 5
real=172.16.0.102:53 masq 5
service=none
scheduler=wlc
checktype=ping
protocol=udp
virtual=192.168.0.20:53
real=172.16.0.101:53 masq 5
real=172.16.0.102:53 masq 5
service=dns
scheduler=wlc
checktype=ping
protocol=tcp
virtual=172.16.0.1:53
real=172.16.0.101:53 gate 5
real=172.16.0.102:53 gate 5
service=none
scheduler=wlc
checktype=ping
protocol=udp
virtual=172.16.0.1:53
real=172.16.0.101:53 gate 5
real=172.16.0.102:53 gate 5
service=dns
scheduler=wlc
checktype=ping
protocol=tcp
virtual=172.168.0.1:7000
real=172.16.0.101:7000 gate 5
scheduler=rr
protocol=udp
virtual=192.168.0.20:7000
real=172.16.0.101:7000 masq 5
service=none
checktype=on
scheduler=rr
protocol=udp
virtual=192.168.0.20:3306
service = mysql
real = 172.16.0.101:3306 masq 5
real = 172.16.0.102:3306 masq 5
checktype=negotiate
login = "ldirectord"
passwd = "LDIRECTORD_PASSWORD"
database = "ldirectordb"
request = "SELECT * FROM connectioncheck"
scheduler = wrr
virtual=172.16.0.1:3306
service = mysql
real = 172.16.0.101:3306 gate 5
real = 172.16.0.102:3306 gate 5
checktype=negotiate
login = "ldirectord"
passwd = "LDIRECTORD_PASSWORD"
database = "ldirectordb"
request = "SELECT * FROM connectioncheck"
scheduler = wrr
virtual=192.168.0.20:80
real=172.16.0.101:80 masq 5
real=172.16.0.102:80 masq 5
service=http
request="ldirectord.php"
receive="Connected to MySQL"
scheduler=wlc
protocol=tcp
checktype=negotiate
virtual=172.16.0.1:80
real=172.16.0.101:80 gate 5
real=172.16.0.102:80 gate 5
service=http
request="ldirectord.php"
receive="Connected to MySQL"
scheduler=wlc
protocol=tcp
checktype=negotiate
#FOR HTTPS
#virtual=192.168.0.20:443
# real=172.16.0.101:443 masq 5
# real=172.16.0.102:443 masq 5
# service=http
# request="ldirectord.php"
# receive="Connected to MySQL"
# scheduler=wlc
# protocol=tcp
# checktype=negotiate
@lb1:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
UDP 172.16.0.1:53 wlc
-> 172.16.0.102:53 Route 5 0 0
-> 172.16.0.101:53 Route 5 0 0
TCP 192.168.0.20:53 wlc
-> 172.16.0.102:53 Masq 5 0 0
-> 172.16.0.101:53 Masq 5 0 0
UDP 192.168.0.20:53 wlc
-> 172.16.0.102:53 Masq 5 0 0
-> 172.16.0.101:53 Masq 5 0 0
TCP 172.16.0.1:53 wlc
-> 172.16.0.101:53 Route 5 0 0
-> 172.16.0.102:53 Route 5 0 0
TCP 192.168.0.20:80 wlc
-> 172.16.0.102:80 Masq 5 0 0
-> 172.16.0.101:80 Masq 5 0 0
UDP 192.168.0.20:7000 rr
-> 172.16.0.101:7000 Masq 5 0 0
UDP 172.168.0.1:7000 rr
-> 172.16.0.101:7000 Route 5 0 0
TCP 172.16.0.1:80 wlc
-> 172.16.0.101:80 Route 5 0 0
-> 172.16.0.102:80 Route 5 0 0
TCP 172.16.0.1:3306 wrr
-> 172.16.0.101:3306 Route 5 0 0
-> 172.16.0.102:3306 Route 5 0 0
TCP 192.168.0.20:3306 wrr
-> 172.16.0.102:3306 Masq 5 0 0
-> 172.16.0.101:3306 Masq 5 0 0
@lb1:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto bond1
iface bond1 inet static
address 172.16.0.2
network 172.16.0.0
broadcast 172.16.0.255
netmask 255.255.0.0
post-up ifenslave bond1 eth4 eth5
pre-down ifenslave -d bond1 eth4 eth5
up iptables -t nat -A POSTROUTING -j MASQUERADE -s 172.16.0.0/16
down iptables -t nat -D POSTROUTING -j MASQUERADE -s 172.16.0.0/16
auto bond0
iface bond0 inet static
address 192.168.0.21
gateway 192.168.0.1
netmask 255.255.255.0
post-up ifenslave bond0 eth2 eth3
pre-down ifenslave -d bond0 eth2 eth3
up iptables-restore < /etc/iptables.rules
APP Nodes
========================================================================
=======
app1:~# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt
Iface
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
bond0
0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0
bond0
@app1:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto lo:0
iface lo:0 inet static
address 192.168.0.20
netmask 255.255.255.255
pre-up sysctl -p > /dev/null
auto lo:1
iface lo:1 inet static
address 172.16.0.1
netmask 255.255.255.255
pre-up sysctl -p > /dev/null
auto bond0
iface bond0 inet static
address 172.16.0.101
gateway 172.16.0.1
netmask 255.255.0.0
broadcast 172.16.0.255
network 172.16.0.0
post-up ifenslave bond0 eth0 eth1
pre-down ifenslave -d bond0 eth0 eth1
-Ken
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
