[lvs-users] Question about director (LVS-DR) & realservers on different VLANs

["Net Ground - Frederique Rijsdijk" <frederique@xxxxxxxxxxxx>]

Hi,


We have a LVS-DR setup (currently figure 1), where the keepalived hosts use 
tagged vlan interfaces to 'be' in the same broadcast lans as the realservers. 
This all works fine for the clients. The problem is that realserver1/2 can not 
reach VIP2, and realserver 3/4 can not reach VIP1. But even 'other' hosts, like 
"other server1" and "other server2", that are in the same VLANs as respectivly 
realserver1/2 and realserver3/4, can not reach neither VIPs. I can see the SYN 
packets reaching the VIP on the director, but keepalived seems to do nothing 
with them.  I'm not sure if there is a solution for this. Is there? Would 
figure 2 make any difference? (VIP on eth0, and VLAN interfaces on eth1). I've 
tested the whole setup without any firewall anywhere, but that doesn't make any 
difference.

Director: Ubuntu 12.04 64-bit + Keepalived 1:1.2.2-3ubuntu1
Realservers: FreeBSD (VIP/32 on lo0 with or without -arp)
See below for configuration of network and keepalived


Figure 1
                                           ________
                                          |        |
                                          | client |
                                          |________|
                                              |
                                              |
                                           ________
                                          |        |
                                          |internet|
                                          |________|
                                              |
                                __________    |
                               |          |   |  eth0     RIP=XX.YYY.AAA.5/27
                               | director |---   eth0.608 RIP=XX.YYY.BBB.18/26
                               |__________|   |  eth0.621 RIP=XX.YYY.CCC.71/27
                                              | 
                                              |
                                              |  VIP1=XX.YYY.AAA.7:80  
(realserver1/2 - VLAN 621)
                                              |  VIP2=XX.YYY.AAA.9:443 
(realserver3/4 - VLAN 608)
                                              |
                       ----------------------------------------------
                      |                                              |
                      |                                              |
              ----------------                           -----------------------
             |                |                         |                       
|
             |                |                         |                       
|
RIP=XX.YYY.CCC.68/27    RIP=XX.YYY.CCC.70/27     RIP=XX.YYY.BBB.52/26    
RIP=XX.YYY.BBB.12/26
VIP=XX.YYY.AAA.7/32     VIP=XX.YYY.AAA.7/32      VIP=XX.YYY.AAA.9/32     
VIP=XX.YYY.AAA.9/32
 _____________           _____________            _____________           
_____________
|             |         |             |          |             |         |      
       |
|             |         |             |          |             |         |      
       |
|  VLAN 621   |         |  VLAN 621   |          |  VLAN 608   |         |  
VLAN 608   |
| realserver1 |         | realserver2 |          | realserver3 |         | 
realserver4 |
|_____________|         |_____________|          |_____________|         
|_____________|



            RIP=XX.YYY.CCC.69/27                             RIP=XX.YYY.BBB.9/26
             _____________                                    _____________
            |             |                                  |             |
            |             |                                  |             |
            |  VLAN 621   |                                  |  VLAN 608   |
            |other server1|                                  |other server2|
            |_____________|                                  |_____________|



Figure 2
                                           ________
                                          |        |
                                          | client |
                                          |________|
                                              |
                                              |
                                           ________
                                          |        |
                                          |internet|
                                          |________|
                                              |
                                __________    |
                               |          |---   eth0     RIP=XX.YYY.AAA.5/27
                               | director |      eth1.608 RIP=XX.YYY.BBB.18/26
                               |__________|---   eth1.621 RIP=XX.YYY.CCC.71/27
                                              | 
                                              |
 
                                            the rest



Network configuration on director:

auto eth0
iface eth0 inet static
        address XX.YYY.AAA.5
        netmask 255.255.255.224
        network XX.YYY.AAA.0
        broadcast XX.YYY.AAA.31
        gateway XX.YYY.AAA.1

auto eth0.608
iface eth0.608 inet static
        address XX.YYY.BBB.18
        netmask 255.255.255.192

auto eth0.621
iface eth0.621 inet static
        address XX.YYY.CCC.71
        netmask 255.255.255.224

Non-default sysctls on the director:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.ip_nonlocal_bind=1



keepalived.conf:

vrrp_sync_group LBGT0 {
        group {
                LBGT0
        }
}
global_defs {
        lvs_id lb01
}

vrrp_instance LB1 {
        state MASTER
        priority 101
        interface eth0
        virtual_router_id x
        advert_int 1
        authentication {
           auth_type PASS
           auth_pass xxxxxxx
}
virtual_ipaddress {
        XX.YYY.AAA.7
        XX.YYY.AAA.9
}

virtual_server XX.YYY.AAA.9 443 {
        delay_loop 6
        lb_algo wrr
        lb_kind DR
        protocol TCP

        # realserver3
        real_server XX.YYY.BBB.52 443 {
                weight 0

                HTTP_GET {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_timeout 60
                        connect_port 80
                }
        }

        # realserver4
        real_server XX.YYY.144.12 443 {
                weight 100

                HTTP_GET {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_timeout 60
                        connect_port 80
                }
        }
}

virtual_server XX.YYY.AAA.7 80 {
        delay_loop 6
        lb_algo wrr
        lb_kind DR
        protocol TCP

        # realserver1
        real_server XX.YYY.CCC.68 80 {
                weight 60
               
                HTTP_GET
                {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_port 80
                        connect_timeout 60
                }
        }

        # realserver2
        real_server XX.YYY.CCC.70 80 {
                weight 60

                HTTP_GET {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_timeout 60
                        connect_port 80
                }
        }
}
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users