|
Hi,
We're having some problems with the following setup:
3 physical machines, all running XEN. On each XEN there are 3
realservers: 1 for mail and 2 for web. On two XEN hosts there are 2
directord in heartbeat failover mode. Ldirectord is used to do
LVS-NAT. I do hope GMail allows for ascii diagrams and just to be
clear i have left one of the directors and some rs's out of the
picture:
VIP=85.x.y.200 (eth0)
+------+
| |
| lvs6 |
| |
+------+
RIP=192.168.5.232 (eth1)
:
:
,-----------------x------------------.
: : :
: : :
IP=192.168.5.208 IP=192.168.5.209 IP=192.168.5.234
+------+ +------+ +------+
| | | | | |
| web4 | | web5 | | web8 |
| | | | | |
+------+ +------+ +------+
Above diagram is for the 'web-realservers'. There's another VIP on the
director (85.x.y.210) that is used for the mail-realservers. There is
only one RIP to which all realservers (both web and mail) point to as
their default gw.
>From what i have understood from the HOWTO there shouldn't be a
problem on which XEN host these VMs are located, but here's a list of
xen hosts and their respective vm's:
XEN1: mail4, web4, web7
XEN2: lvs5, mail5, web8
XEN3: lvs6, mail6, web6
Host lvs6 is the active director.
I have set up the following iptables rules on the directord:
iptables -I FORWARD -i eth1 -j ACCEPT
# webservers http (from top to bottom: web4, web5, web6, web7, web8)
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 443 -s
192.168.5.208/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 443 -s
192.168.5.209/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 443 -s
192.168.5.219/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 443 -s
192.168.5.233/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 443 -s
192.168.5.234/32 -j SNAT --to-source 85.x.y.200
# webservers https (from top to bottom: web4, web5, web6, web7, web8)
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 80 -s
192.168.5.208/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 80 -s
192.168.5.209/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 80 -s
192.168.5.219/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 80 -s
192.168.5.233/32 -j SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 80 -s
192.168.5.234/32 -j SNAT --to-source 85.x.y.200
# mailservers smtp (from top to bottom: mail4, mail5, mail6)
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 25 -s
192.168.5.213/32 -j SNAT --to-source 85.x.y.210
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 25 -s
192.168.5.214/32 -j SNAT --to-source 85.x.y.210
iptables -t nat -I POSTROUTING -p tcp -m tcp --dport 25 -s
192.168.5.215/32 -j SNAT --to-source 85.x.y.210
# webservers (from top to bottom: web4, web5, web6, web7, web8)
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.208/32 -j
SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.209/32 -j
SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.219/32 -j
SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.233/32 -j
SNAT --to-source 85.x.y.200
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.234/32 -j
SNAT --to-source 85.x.y.200
# mailservers (from top to bottom: mail4, mail5, mail6)
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.213/32 -j
SNAT --to-source 85.x.y.210
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.214/32 -j
SNAT --to-source 85.x.y.210
iptables -t nat -I POSTROUTING -p tcp -m tcp -s 192.168.5.218/32 -j
SNAT --to-source 85.x.y.210
The problem occurs when i enable ldirectord to direct (http/https)
traffic to web5 and web6. Then the connection times out. Funny thing
is that i DO see traffic running. These are some excerpts from
tcpdumps:
from the director on the eth0 side:
13:28:17.103396 IP my.client.ip.48104 > 85.x.y.200.http: S
1466837983:1466837983(0) win 5840 <mss 1460,sackOK,timestamp 5166972
0,nop,wscale 6>
13:28:17.103512 IP 85.x.y.200.http > my.client.ip.48104: S
2109956387:2109956387(0) ack 1466837984 win 5792 <mss
1460,sackOK,timestamp 881478 5166972,nop,wscale 7>
13:28:17.112043 IP my.client.ip.48104 > 85.x.y.200.http: . ack 1 win
92 <nop,nop,timestamp 5166974 881478>
13:28:17.119784 IP my.client.ip.48104 > 85.x.y.200.http: P 1:653(652)
ack 1 win 92 <nop,nop,timestamp 5166974 881478>
13:28:17.119884 IP 85.x.y.200.http > my.client.ip.48104: . ack 653 win
56 <nop,nop,timestamp 881482 5166974>
13:28:17.212577 IP 85.x.y.200.http > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881505 5166974>
13:28:17.424432 IP 85.x.y.200.http > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881559 5166974>
13:28:17.856474 IP 85.x.y.200.http > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881667 5166974>
13:28:18.720513 IP 85.x.y.200.http > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881883 5166974>
13:28:20.448612 IP 85.x.y.200.http > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 882315 5166974>
13:28:23.904860 IP 85.x.y.200.http > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 883179 5166974>
13:28:27.619216 IP my.client.ip.48104 > 85.x.y.200.http: F 653:653(0)
ack 1 win 92 <nop,nop,timestamp 5169601 881482>
13:28:27.619390 IP 85.x.y.200.http > my.client.ip.48104: F 595:595(0)
ack 654 win 56 <nop,nop,timestamp 884107 5169601>
13:28:27.627454 IP my.client.ip.48104 > 85.x.y.200.http: R
1466838637:1466838637(0) win 0
from the director on the eth1 side:
13:28:17.103762 IP my.client.ip.48104 > web5.http: S
1466837983:1466837983(0) win 5840 <mss 1460,sackOK,timestamp 5166972
0,nop,wscale 6>
13:28:17.103494 IP web5.http > my.client.ip.48104: S
2109956387:2109956387(0) ack 1466837984 win 5792 <mss
1460,sackOK,timestamp 881478 5166972,nop,wscale 7>
13:28:17.112053 IP my.client.ip.48104 > web5.http: . ack 1 win 92
<nop,nop,timestamp 5166974 881478>
13:28:17.119793 IP my.client.ip.48104 > web5.http: P 1:653(652) ack 1
win 92 <nop,nop,timestamp 5166974 881478>
13:28:17.119873 IP web5.http > my.client.ip.48104: . ack 653 win 56
<nop,nop,timestamp 881482 5166974>
13:28:17.212529 IP web5.http > my.client.ip.48104: P 1:595(594) ack
653 win 56 <nop,nop,timestamp 881505 5166974>
13:28:17.424405 IP web5.http > my.client.ip.48104: P 1:595(594) ack
653 win 56 <nop,nop,timestamp 881559 5166974>
13:28:17.856446 IP web5.http > my.client.ip.48104: P 1:595(594) ack
653 win 56 <nop,nop,timestamp 881667 5166974>
13:28:18.720486 IP web5.http > my.client.ip.48104: P 1:595(594) ack
653 win 56 <nop,nop,timestamp 881883 5166974>
13:28:20.448594 IP web5.http > my.client.ip.48104: P 1:595(594) ack
653 win 56 <nop,nop,timestamp 882315 5166974>
13:28:23.904823 IP web5.http > my.client.ip.48104: P 1:595(594) ack
653 win 56 <nop,nop,timestamp 883179 5166974>
13:28:27.619233 IP my.client.ip.48104 > web5.http: F 653:653(0) ack 1
win 92 <nop,nop,timestamp 5169601 881482>
13:28:27.619378 IP web5.http > my.client.ip.48104: F 595:595(0) ack
654 win 56 <nop,nop,timestamp 884107 5169601>
13:28:27.627461 IP my.client.ip.48104 > web5.http: R
1466838637:1466838637(0) win 0
from my client:
13:28:20.600626 IP my.client.ip.48104 > 85.x.y.200.www: S
1466837983:1466837983(0) win 5840 <mss 1460,sackOK,timestamp 5166972
0,nop,wscale 6>
13:28:20.609081 IP 85.x.y.200.www > my.client.ip.48104: S
2109956387:2109956387(0) ack 1466837984 win 5792 <mss
1460,sackOK,timestamp 881478 5166972,nop,wscale 7>
13:28:20.609148 IP my.client.ip.48104 > 85.x.y.200.www: . ack 1 win 92
<nop,nop,timestamp 5166974 881478>
13:28:20.609437 IP my.client.ip.48104 > 85.x.y.200.www: P 1:653(652)
ack 1 win 92 <nop,nop,timestamp 5166974 881478>
13:28:20.625390 IP 85.x.y.200.www > my.client.ip.48104: . ack 653 win
56 <nop,nop,timestamp 881482 5166974>
13:28:20.719176 IP 85.x.y.200.www > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881505 5166974>
13:28:20.931298 IP 85.x.y.200.www > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881559 5166974>
13:28:21.362955 IP 85.x.y.200.www > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881667 5166974>
13:28:22.227337 IP 85.x.y.200.www > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 881883 5166974>
13:28:23.955078 IP 85.x.y.200.www > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 882315 5166974>
13:28:27.411099 IP 85.x.y.200.www > my.client.ip.48104: P 1:595(594)
ack 653 win 56 <nop,nop,timestamp 883179 5166974>
13:28:31.116413 IP my.client.ip.48104 > 85.x.y.200.www: F 653:653(0)
ack 1 win 92 <nop,nop,timestamp 5169601 881482>
13:28:31.124407 IP 85.x.y.200.www > my.client.ip.48104: F 595:595(0)
ack 654 win 56 <nop,nop,timestamp 884107 5169601>
13:28:31.124443 IP my.client.ip.48104 > 85.x.y.200.www: R
1466838637:1466838637(0) win 0
But, again, the page just keeps trying to load (above traffic is
repeated) and eventually times out.
Any suggestions would be appreciated :)
kind regards,
Sebastian
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
