LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Unable to forward packets

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Unable to forward packets
From: "Bill Omer" <bill.omer@xxxxxxxxx>
Date: Wed, 22 Feb 2006 07:11:29 -0800
> aliased ethernet devices have been deprecated since 2.4.0.
> They may or may not work in your situation, but we don't
> know and we aren't going to debug it.
>
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.policy_routing.html

Interesting.   Do I *have* to use a different ethernet device though? 
Could I just use my eth0?

>
> > ipvsadm -A -t lvs-vip:23 -s wlc
> > ipvsadm -a -t lvs-vip:23 -r test-rs -g
> > echo 1 > /proc/sys/net/ipv4/ip_forward
> > route add -host 10.26.66.101 dev eth0:0
>                                     ^^^^^^
>
> This is guaranteed to give unpredictable results.

I found that in some docs.  It seemed to be for the 2.2 kernel tree though.

>
> > On the box I'm calling test-rs, I'm doing the following
> > ifconfig lo:0 lvs-vip netmask 255.255.255.255 broadcast lvs-vip up
> > echo 1 >/proc/sys/net/ipv4/ip_forward
> > route add -host lvs-vip dev lo:0
>
> I don't know why people run this last route command. They
> say that it makes their machines work. I've never needed it.
>

Was in the docs ...

> You're also running a RedHat kernel. You're going to have to
> look on the archives to see if it works. It probably does,
> but we only support the standard kernel here. There are
> plenty of people running market enhanced kernels with LVS.
> You can always go back to your RH kernel after you get it to
> work with the standard kernel.
>
>

Yes, this version of rhel3as does support LVS.  I'm using hp hardware
and dont want to fight with thier (IMHO) horrid drives for scsi
support.... that, in my mind, is my last resort.

> >
> > To troubleshoot this, I'm running 'tcpdump -ln port 23'    on the lvs server
> > and the test-rs machine.  On the lvs server, I get the following when I try
> > to telnet to the vip from a 3 machine
>
> I assume a 3 machine is your client.

Sorry, that was a typeo.  I ment to say  ... "telnet to the vip from a
*third* machine"

>
> > 21:14:04.278916 10.26.66.66.29427 > 10.26.66.101.telnet: S
> > 2084871663:2084871663(0) win 5840 <mss 1460,sackOK,timestamp 101529036
> > 0,nop,wscale 0> (DF) [tos 0x10]
> > 21:14:04.278929 10.26.66.66.29427 > 10.26.66.101.telnet: S
> > 2084871663:2084871663(0) win 5840 <mss 1460,sackOK,timestamp 101529036
> > 0,nop,wscale 0> (DF) [tos 0x10]
>
> so you can't get there. Any firewall rules?
>

Nope, none.

> > nfs-101$ telnet lvs-vip
> > Trying 10.26.66.101...
> > telnet: connect to address 10.26.66.101: No route to host
> > telnet: Unable to connect to remote host: No route to host
>
> you can't get there. I'd look at routing or firewall rules
> first.
>

Well, from the third machine, I'm able to ping and telnet to the RS as
well as the director (with no ipvsadm rules and telnet enabled in
xinetd on the director of course).

I'm a firm believer that a good nights sleep will help you see the
problem in a new light.  So here I am   :)

So this morning I killed eth0:0 on my director.  I used the following
commands to clear my ipvsadm tables and start them again

ipvsadm --clear
ipvsadm -A -t lvs-101:23 -s wcl
ipvsadm -a -t lvs-101:23 -r app-101:23 -g -w 1

Notice I'm not using the vip, but the director's actual ip (for
testing purposes ... a vip will have to be used later).

So now from the third machine, when I telnet to lvs-101, I now get
"Connection timed out"

Running tcpdump on the director and the rs I'm testing with I see the follow

On director:

15:04:32.872692 10.26.94.121.57820 > 10.26.66.97.telnet: S
1958200356:1958200356(0) win 5840 <mss 1460,sackOK,timestamp 103978332
0,nop,wscale 0> (DF) [tos 0x10]
15:04:56.872306 10.26.94.121.57820 > 10.26.66.97.telnet: S
1958200356:1958200356(0) win 5840 <mss 1460,sackOK,timestamp 103980732
0,nop,wscale 0> (DF) [tos 0x10]
15:04:56.872325 10.26.94.121.57820 > 10.26.66.97.telnet: S
1958200356:1958200356(0) win 5840 <mss 1460,sackOK,timestamp 103980732
0,nop,wscale 0> (DF) [tos 0x10]


On the RS (app-101 in this case):

15:03:25.627960 10.26.94.121.57819 > 10.26.66.97.telnet: S
1895710843:1895710843(0) win 5840 <mss 1460,sackOK,timestamp 103971607
0,nop,wscale 0> (DF) [
tos 0x10]



So packets now are being forward (yay!) but aren't coming back to the client.

My ipvsadm output is posted below.  No iptable rules are being used
anywhere at this time (nor were they before, just to clearify).


logbash-2.03# ipvsadm
IP Virtual Server version 1.0.8 (size=65536)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  cvg1-lvs-101.amazon.com:teln wlc
  -> cvg1-app-101.amazon.com:telnet Route   1      0          1



This is where I left off on this project about a month ago.  I decided
to start from scratch again about a week ago, and now here I am.  I'm
wondering if not using the aliases interface is whats giving me these
new results.


Thanks again for you help with this Joe.  Looks like I'm getting a
little closer, but now I'm just plain stuck.  Maybe some iptables
trickery is in order?

-Bill

<Prev in Thread] Current Thread [Next in Thread>