LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: etherIP and lvs

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: etherIP and lvs
From: Andy Wettstein <awettstein@xxxxxxxx>
Date: Tue, 15 Jul 2003 13:47:54 -0500
On Tue, Jul 15, 2003 at 04:38:17PM +0200, Roberto Nibali wrote:
> Hi Andy,
> 
> Long time no talk. How is that Sun Cluster with NCA running? :)

Hello.  It is running good without NCA, though.  We run about 20 domains
through it and keep adding more and more.

> 
> >http://www.openbsd.org/cgi-bin/man.cgi?query=brconfig&sektion=8.  I am
> >not using IPSEC so that is not causing me any problems.
> 
> Ok.
> 
> >Anyway, I have all normal LAN traffic working correctly, so I'm sure the
> >EtherIP bridge is working correctly, but if I have a server that is in an 
> >LVS cluster the server never sees that traffic that is being sent to it 
> >as part of the cluster.  
> 
> Do you rewrite MAC addresses on the bridge? How does a tcpdump look like on 
> all the director, the bridge and the node on the other side? How are the 
> neighbour tables set up?

I don't do any MAC address rewriting on the bridge.

This is my test service:

TCP  192.168.0.45:8000 wlc
  -> 192.168.0.48:8000    Route   1      0          0 

The openbsd box with the director on its physical lan is set up like this 
(all real ips changed):

vlan0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        address: 00:02:b3:d0:36:0d
        vlan: 57 parent interface: em0
        inet6 fe80::202:b3ff:fed0:360d%vlan0 prefixlen 64 scopeid 0x1a
        inet 192.168.0.1 netmask 0xffffff80 broadcast 192.168.0.127

gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
        physical address inet 172.20.1.2 --> 172.20.1.3
        inet6 fe80::206:5bff:fefd:ef23%gif1 ->  prefixlen 64 scopeid 0x30

bridge0: flags=41<UP,RUNNING>
        Configuration:
                priority 32768 hellotime 2 fwddelay 15 maxage 20
        Interfaces:
                gif1 flags=3<LEARNING,DISCOVER>
                        port 48 ifpriority 128 ifcost 55
                vlan0 flags=3<LEARNING,DISCOVER>
                        port 26 ifpriority 128 ifcost 55

The openbsd box with the member of the cluster (traffic never gets to it):

vlan1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        address: 00:02:b3:d0:32:78
        vlan: 57 parent interface: em0
        inet6 fe80::202:b3ff:fed0:3278%vlan1 prefixlen 64 scopeid 0x1c

gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
        physical address inet 172.20.1.3 --> 172.20.1.2
        inet6 fe80::206:5bff:fe3e:6d58%gif1 ->  prefixlen 64 scopeid 0x31

bridge0: flags=41<UP,RUNNING>
        Configuration:
                priority 32768 hellotime 2 fwddelay 15 maxage 20
        Interfaces:
                gif1 flags=3<LEARNING,DISCOVER>
                        port 49 ifpriority 128 ifcost 55
                vlan1 flags=3<LEARNING,DISCOVER>
                        port 28 ifpriority 128 ifcost 55


Here are the tcpdumps but it only gets through bridge0 on the side of the 
bridge with the director on it.  I can't see any traffic on gif1.

On the lvs director:

13:11:26.576318 0:2:b3:87:dc:ee 0:30:65:c0:65:1a 0800 74: 192.168.0.143.38534 > 
192.168.0.48.8000: S [tcp sum ok] 1480407753:1480407753(0) win 5840 <mss 
1460,sackOK,timestamp 179923253 0,nop,wscale 0> (DF) (ttl 63, id 63171, len 60)
13:11:26.576339 0:2:b3:87:dc:ee 0:30:65:c0:65:1a 0800 74: 192.168.0.143.38534 > 
192.168.0.48.8000: S [tcp sum ok] 1480407753:1480407753(0) win 5840 <mss 
1460,sackOK,timestamp 179923253 0,nop,wscale 0> (DF) (ttl 63, id 63171, len 60)

On bridge0 of the openbsd machine on the same lan as the director:

13:10:40.924673 0:2:b3:87:dc:ee 0:30:65:c0:65:1a 0800 74: 192.168.0.143.38534 > 
192.168.0.48.8000: S [tcp sum ok] 1480407753:1480407753(0) win 5840 <mss 
1460,sackOK,timestamp 179923253 0,nop,wscale 0> (DF) (ttl 63, id 63171)


0:2:b3:87:dc:ee is the mac address of the director
0:30:65:c0:65:1a is the mac address of the cluster member.

192.168.0.0 is subnetted so 192.168.0.143 goes through the openbsd box, 
which is also our router.  That just gave me an idea.  Testing from an IP
that doesn't need to be routed...Works!!  

So going through 192.168.0.143/26 -> 192.168.0.129/26 -> 192.168.0.48/25
                                             ^^^
                                     router interface on openbsd box (vlan2)
doesn't work, but going
   192.168.0.61/25 -> 192.168.0.48/25

without a route does work.

If you want to see the tcpdumps from the working connection let me know.



 
> Best regards,
> Roberto Nibali, ratz
> -- 
> echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | 
> dc
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>