|
Hello,
This is my first time setting up LVS, and I am abit stuck. So I was
hoping to maybe get a little insight and advice from some of the more
experienced members of this mailing list.
So first things first, I'm trying to get this set up on linode.com.
and I've been in their IRC channel, and asked if this would work. and
one of the official responses on this issue:
caker:if packets get rewritten, it's not gonna work
[
caker:we filter based on source ip and mac, and dest ip and mac
[caker:^-- for a given Linode
So i decided to use LVS-TUN. Each linode has a public IP on eth0, and
an aliased eth0:0 private ip address with no gateway.
This is where I am not sure if it was the correct approach or not,
please correct me. On the director, I set the VIP to be the same as
my eth0 public IP. and on the real servers I created a tunl0 interface
that matched the VIP. I dont think i needed to add a route, since they
both share a common gateway on their public IP's, and they can talk to
each other.
all machines:
running Centos 5.3
Kernel@ 2.6.18.8-x86_64
realserver contains nginx
director setup:
sysctl.conf has this loaded:
net.ipv4.ip_forward = 1
# /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr FE:FD:61:6B:85:EA
inet addr:97.107.133.234 Bcast:97.107.133.255 Mask:
255.255.255.0
inet6 addr: fe80::fcfd:61ff:fe6b:85ea/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4440 errors:0 dropped:0 overruns:0 frame:0
TX packets:6386 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:796449 (777.7 KiB) TX bytes:1195747 (1.1 MiB)
eth0:0 Link encap:Ethernet HWaddr FE:FD:61:6B:85:EA
inet addr:192.168.134.25 Bcast:192.168.255.255 Mask:
255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:65 errors:0 dropped:0 overruns:0 frame:0
TX packets:65 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5944 (5.8 KiB) TX bytes:5944 (5.8 KiB)
# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref
Use Iface
97.107.133.0 0.0.0.0 255.255.255.0 U 0 0
0 eth0
192.168.128.0 0.0.0.0 255.255.128.0 U 0 0
0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0
0 eth0
0.0.0.0 97.107.133.1 0.0.0.0 UG 0 0
0 eth0
# /sbin/ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 97.107.133.234:80 wlc
-> 97.107.130.68:80 Tunnel 1 0 0
real server with http web server listening on port 80:
sysctl.conf already loaded with:
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce=2
net.ipv4.conf.eth0.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
# /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr FE:FD:61:6B:82:44
inet addr:97.107.130.68 Bcast:97.107.130.255 Mask:
255.255.255.0
inet6 addr: fe80::fcfd:61ff:fe6b:8244/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:64369 errors:0 dropped:0 overruns:0 frame:0
TX packets:92259 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:48183677 (45.9 MiB) TX bytes:23467359 (22.3 MiB)
eth0:0 Link encap:Ethernet HWaddr FE:FD:61:6B:82:44
inet addr:192.168.134.109 Bcast:192.168.255.255 Mask:
255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:48 errors:0 dropped:0 overruns:0 frame:0
TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6877 (6.7 KiB) TX bytes:6877 (6.7 KiB)
tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:97.107.133.234 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref
Use Iface
97.107.133.234 0.0.0.0 255.255.255.255 UH 0 0
0 tunl0
97.107.130.0 0.0.0.0 255.255.255.0 U 0 0
0 eth0
192.168.128.0 0.0.0.0 255.255.128.0 U 0 0
0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0
0 eth0
0.0.0.0 97.107.130.1 0.0.0.0 UG 0 0
0 eth0
iptables is clear and is accepting everything on both director and
real server.
director: cannot ping realserver or telnet port 80 into realserver
eth0 public ip. can ping client.
realserver: can ping both realserver and client.when i telnet into VIP
on port 80, i believe it bypasses the director, since tcpdump host
97.107.130.68 on the director showed no activity.
client (public ip 99.247.97.70) can ping director and realserver, and
can telnet port 80 to real server fine. when i telnet to the
VIP,client doesnt get a response. When i run tcpdump on the director
and realserver, this is the what happens when a client tries to telnet
port 80 into the VIP:
director tcpdump:
# /usr/sbin/tcpdump -nn host 97.107.130.68
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
04:39:47.872616 IP 97.107.133.234 > 97.107.130.68: IP
99.247.97.70.34213 > 97.107.133.234.80: S 2271054937:2271054937(0) win
65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 343332259 0,sackOK,[|
tcp]> (ipip-proto-4)
04:39:51.874495 IP 97.107.133.234 > 97.107.130.68: IP
99.247.97.70.34213 > 97.107.133.234.80: S 2271054937:2271054937(0) win
65535 <mss 1460,sackOK,eol> (ipip-proto-4)
realserver tcp dump:
# /usr/sbin/tcpdump -nn host 97.107.133.234
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
04:39:47.860998 IP 97.107.133.234 > 97.107.130.68: IP
99.247.97.68.34213 > 97.107.133.234.80: S 2271054937:2271054937(0) win
65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 343332259 0,sackOK,[|
tcp]> (ipip-proto-4)
04:39:51.863289 IP 97.107.133.234 > 97.107.130.68: IP
99.247.97.68.34213 > 97.107.133.234.80: S 2271054937:2271054937(0) win
65535 <mss 1460,sackOK,eol> (ipip-proto-4)
realserver has an entry in /var/log/messages:
Oct 14 04:39:51 li60-68 kernel: martian source 97.107.130.68 from
97.107.133.234, on dev eth0
Oct 14 04:39:51 li60-68 kernel: ll header: fe:fd:61:6b:82:44:00:0e:
39:6f:48:00:08:00
conclusion so far:
it looks like the ipip packet is reaching the realserver, but want to
find out if it's being discarded because it thinks it's a martian
source? I thought with kernel 2.6+ all i need was the arp_ignore and
arp_announce flags set on the real servers. do i need to do stuff with
arptables or iptables? If any additional information is needed, let me
know. is it possible to do LVS-DR or LVS-TUN over the eth0:0 aliased
private ip's?
What can I try next? I've been exploring LVS for the last 2 days or
so, and read through the documentation several times. I know i'm not
as experienced as some people here, so I'm hoping someone can point me
in the right direction.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
