|
Joe,
I am still unable to create a functioning LVS-Tunnel. I imagine that it
is a routing problem, however I cannot for the life of me find out where
I goofed the configuration. Perhaps if I provide a step-by-step of what I am
doing, then perhaps my errors may become obvious to someone that has more
experience with this software.
Here is a small diagram of what I want to setup:
DIP=192.168.10.110
VIP=192.168.10.111
CIP=192.168.10.15
RS1=10.1.2.254
RS2=10.1.3.254
RSn=10.1.x.x
------------------------------------------------------------------------------------
#INTERNET#
|
#FIREWALL# 192.168.10.1 (Gateway for all 192.168.10.x Addresses)
|
#UML Host# 192.168.10.100 (Bridged to eth0, with tap devices for DIP
eth0,ethx,etc)
|
#DIP (UML)# 192.168.10.110
| 192.168.10.111 (Arps)
|------------------|
#RS1 (10.1.2.254)# ---------- #RS2 (10.1.2.254)#
($VIP - No Arp) ($VIP - No Arp)
| |
----------------------------
|
#CIP# 192.168.10.15 (or any other
address)
-----------------------------------------------------------------------------------
To start, this is what I do on my Director:
I setup network devices on boot:
##################################################################################
ifconfig eth0 192.168.10.110 netmask 255.255.255.0 up
#DIP
ifconfig eth0:0 192.168.10.111 netmask 255.255.255.255 broadcast 192.168.10.111
up #VIP
ifconfig eth1 10.1.2.2 netmask 255.0.0.0 up
ifconfig eth2 10.1.3.2 netmask 255.0.0.0 up
ifconfig lo 127.0.0.1 up
route add default gw 192.168.10.1
At this point, I can successfully ping every machine on 192.168.10.x and
10.1.(2,3).x
networks rom the director, and all the machines from those networks can ping my
Director.
this is my routing table for the director at the point:
-----------------------------------------------------------------------------
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth1
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth2
0.0.0.0 192.168.10.1 0.0.0.0 UG 0 0 0 eth0
----------------------------------------------------------------------------
Then I continue by using these commands on the director:
ipvsadm -A -t 192.168.10.111:80 -s wlc
ipvsadm -a -t 192.168.10.111:80 -r 10.1.2.254:80 -i -w 1
Next Step: Setup Real Server.
So, now I have my director all nice and setup according to any documentation
I found. No obvious problems so far as I can see. So I continue to setup
my real server by doing the following. (I am only working with 1 realserver
for sake of testing, but I will include settings for RS2 aswell).
RS1:
eth0:
ip = 10.1.2.254
netmask = 255.0.0.0
broadcast = 10.255.255.255
Gateway = *10.1.2.1
Route:
-----------------------------------------------------------------------------
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
224.0.0.0 0.0.0.0 240.0.0.0 U 0 0 0 eth0
0.0.0.0 10.1.2.1 0.0.0.0 UG 0 0 0 eth0
-----------------------------------------------------------------------------
RS2:
eth0:
ip = 10.1.3.254
netmask = 255.0.0.0
broadcast = 10.255.255.255
Gateway = *10.1.3.1
Route:
-----------------------------------------------------------------------------
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
224.0.0.0 0.0.0.0 240.0.0.0 U 0 0 0 eth0
0.0.0.0 10.1.3.1 0.0.0.0 UG 0 0 0 eth0
-----------------------------------------------------------------------------
*Don't know if this makes any difference, but the gateway 10.1.2.1 is on eth0
of a machine with address 192.168.10.101 (eth1) which is sitting on the network.
Same applys for RS2, with RS2 GW (10.1.3.1) on eth0 of 192.168.10.102. All
of my machines reside on 2 connected switches.
These are the settings of my Real servers on boot. I then proceed to add a
tunnel
device with the following commands:
modprobe ipip
ifconfig tunl0 192.168.10.111 netmask 255.255.255.255 broadcast 192.168.10.111
up
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo 1 > /proc/sys/net/ipv4/conf/tunl0/hidden
Note: I am using 2.4.26 kernel (+uml skas). I tried using arp_announce and
arp_ignore, but it didn't seem to work, so I just added Julians "hidden"
patch, which seemed to do the trick.
So at this point, in theory(to my knowledge), I should have completed the setup
for 1 director with 1 real server. However, when I try to connect from my CIP,
my connection just hangs, then timesout.
I have confirmed that packets are reaching both my Director and Realserver.
Although it seems that I am only getting SYN packets with no ACK in return.
*Note: these packets were generated as a result of
"nc -v -v 192.168.10.111 80" from my CIP (192.168.10.15)*
Here is some ethereal output from my director:
############################################################################
No. Time Source Destination Protocol Info
4 0.502890 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173704242 TSER=0 WS=0
Frame 4 (76 bytes on wire, 76 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
No. Time Source Destination Protocol Info
5 0.503292 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173704242 TSER=0 WS=0
Frame 5 (96 bytes on wire, 96 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 10.1.2.2 (10.1.2.2), Dst Addr: 10.1.2.254
(10.1.2.254)
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
No. Time Source Destination Protocol Info
9 3.501767 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173707242 TSER=0 WS=0
Frame 9 (76 bytes on wire, 76 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
###########################################################################
Here is some ethereal output from my Real Server (RS1):
############################################################################
No. Time Source Destination Protocol Info
92283 4192.519718 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173704242 TSER=0 WS=0
Frame 92283 (76 bytes on wire, 76 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
No. Time Source Destination Protocol Info
92365 4195.518586 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173707242 TSER=0 WS=0
Frame 92365 (96 bytes on wire, 96 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 10.1.2.2 (10.1.2.2), Dst Addr: 10.1.2.254
(10.1.2.254)
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
No. Time Source Destination Protocol Info
92366 4195.518586 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173707242 TSER=0 WS=0
Frame 92366 (76 bytes on wire, 76 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
No. Time Source Destination Protocol Info
92475 4201.516716 192.168.10.15 192.168.10.111 TCP 34023
> http [SYN] Seq=0
Ack=0 Win=5840 Len=0 MSS=1460 TSV=173713242 TSER=0 WS=0
Frame 92475 (96 bytes on wire, 96 bytes captured)
Linux cooked capture
Internet Protocol, Src Addr: 10.1.2.2 (10.1.2.2), Dst Addr: 10.1.2.254
(10.1.2.254)
Internet Protocol, Src Addr: 192.168.10.15 (192.168.10.15), Dst Addr:
192.168.10.111
(192.168.10.111)
Transmission Control Protocol, Src Port: 34023 (34023), Dst Port: http (80),
Seq: 0, Ack: 0, Len:
0
############################################################################
Right now I am stuck. I imagine this IS a routing problem, although I am not
sure how to fix it. Is it a routing problem on the Real server or the
Director.
Is it possible that the problem may stem up to my UML host (though,I don't see
how)?
Also, I have a few other questions:
from my director, this command works fine:
ping -I 192.168.10.111 192.168.10.15
but it does not work from my Real server. Should it ?
The same is true with "traceroute -n -s 192.168.10.111 192.168.10.15"
I've noticed the ethereal output (on CIP) of the above command, says that
the ping want to reply to 192.168.10.101, which is the machine where the
gateway for RS1(10.1.3.254) is located. eth0=10.1.2.1 and eth1=192.168.10.101.
I'm no router guru, but this does not seem to be the desired behavior ?
One more thing, "arping 192.168.10.111" returns the correct
MAC[FE:FD:0A:01:03:02]
address from wherever I do it.
Any help or suggestions will be greatly appreciated,
thanks again,
-R.D
--- Joseph Mack <mack.joseph@xxxxxxx> wrote:
> redirecting decoy wrote:
> >
> > Joe,
> >
> > > Is their a connection at the realserver (with
> > > netstat -a)? - if so
> > > the packet has got to the realserver.
> >
> > No, the connection never makes it to the real server.
> > I have no idea why. I set everything up according to
> > documentation.
>
> I would imagine that the hardware's opinion on the matter
> counts for something here :-)
>
> > > Does the director show a connection from the client
> > > (probably InActConn)? if so the packet has got to the ip_vs
> > > code.
>
> > Yes, it does now anyway. I made a small change to the
> > host which the uml is running on. Simply had to
> > change the default gw to correct device (i.e
> > uml-bridge).
> >
> > > Is the routing from the realserver(s) to the client
> > > directly to the client and not via the director?
> > I am not sure. What would the best way to check be ?
>
>
> `route`
>
> or
>
> `ip route show`
>
> all packets from the realserver:VIP:port_being_lvs'ed should
> not pass through the director on their way to the client
>
> usually this is handled by the default gw
>
> > It appears that I am unable to connect to the Real
> > servers. It might be a routing problem, although all
> > the realservers can ping the DIP, and vice versa.
>
> lvs doesn't act at the same layer as ping, so ping can work,
> but lvs is messing with the layer above and may not be setup
> correctly.
>
> The usual problem causing InActConn entries is incorrect routing
> on the realserver.
>
> > here is the routing tables for my Director (uml),
> > RealServers, Uml-Host
>
> these don't help me. I have no idea where these IPs are.
>
> Joe
>
> --
> Joseph Mack PhD, High Performance Computing & Scientific Visualization
> LMIT, Supporting the EPA Research Triangle Park, NC 919-541-0007
> Federal Contact - John B. Smith 919-541-1087 - smith.johnb@xxxxxxx
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
|
