LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

[lvs-users] Keepalived - HTTPS Issue with multiple HTTPS virtual server

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: [lvs-users] Keepalived - HTTPS Issue with multiple HTTPS virtual server blocks
From: eneal@xxxxxxxxxxxxxxxxx
Date: Thu, 29 May 2008 11:40:35 -0400
Thanks in advance for any assistance you can provide:

Been running keepalived for a number of months now - and very happy.
Our configuration is rather simple. We run an active/active configuration -
each VRRP instance consists of a single master/slave pair and we just
balance our virtual IP's across the instances. Here's that component
of our configuration




vrrp_instance VI_1 {
         state MASTER
         interface bond0
         virtual_router_id 50
         preempt
         priority 100
         advert_int 1
         authentication {
                 auth_type AH
                 auth_pass ********
         }
         virtual_ipaddress {
                 66.*.*.42
                 66.*.*.242

         }
}
vrrp_instance VI_2 {
         state BACKUP
         interface bond0
         virtual_router_id 51
         preempt
         priority 80
         advert_in 1
         authentication {
                 auth_type AH
                 auth_pass *********
         }
         virtual_ipaddress {
                 66.*.*.44
                 66.*.*.115
         }
}


IPVS coupled with the redundancy VRRP provides - pretty much matchless
in terms of scalablity and reliablity - at least for our needs...

Our goal for using IPVS and keepalived is simply to add some  
scalability and reliabilty to
to our application servers which consist of mainly Windows 2003  
Servers. We use the direct routing
method currently and do so via the loopback method for Windows.

Each IIS application gets two bindings. One binds to port 80 on the  
VIP. The other binding relies receiving the correct
host header to get the correct content. That is bound to port 81 on the
publicly accessible IP of the appliication server on
port 81 which is soley used for content/health checks. This is because  
i have not figured
out a way to get requests from the director to the real server...

virtual_server 66.*.*.57 80 {
         delay_loop 10
         lb_algo rr
         lb_kind DR
         persistence_timeout 50
         protocol TCP
         virtualhost www.myapp.local

         real_server 66.*.*.65 81 {
                 weight 1
                 HTTP_GET {
                 url {
                         path /status.cfm
                         digest 56291e28e084bebdb83b53f2dda1536e
                 }
                 connect_timeout 30
                 nb_get_retry 2
                 delay_before_retry 5
                 }
         }
         real_server 66.*.*.60 81 {
                 weight 1
                 HTTP_GET {
                 url {
                         path /status.cfm
                         digest 56291e28e084bebdb83b53f2dda1536e
                 }
                 connect_timeout 30
                 nb_get_retry 2
                 delay_before_retry 5
                 }
         }

}


In our setup, we  have multiple virtual server blocks for different  
virtual servers, but they go to the
same real server
This does not appear to be a problem for http, but just recently
I added two SSL applications - unique virtual server IP's but the same  
real servers
and I saw some interesting issues


virtual_server 66.*.*.57 443 {
     delay_loop 6
     lb_algo rr
     lb_kind DR
     persistence_timeout 50
     protocol TCP


     real_server 66.*.*.60 443 {
         weight 1
             connect_timeout 30
             nb_get_retry 2
             delay_before_retry 5
     }
     real_server 66.151.104.65 443 {
         weight 1
             connect_timeout 30
             nb_get_retry 2
             delay_before_retry 5
     }

}

virtual_server 66.*.*.160 443 {
     delay_loop 6
     lb_algo rr
     lb_kind DR
     persistence_timeout 50
     protocol TCP


     real_server 66.*.*.60 443 {
         weight 1
             connect_timeout 30
             nb_get_retry 2
             delay_before_retry 5
     }
     real_server 66.*.*.65 443 {
         weight 1
             connect_timeout 30
             nb_get_retry 2
             delay_before_retry 5
     }

}


There are also http virtual server blocks that accompany the https ones..


The interesting behavior that I spoke of is essentially when I add  
both https server blocks, http traffic to one of the virtual servers  
no longer
works. HTTPS traffic does however work. I don't have packet captures  
yet - but I can try to get them (it will be difficult because my  
infrastructure is mostly virtual
consisting of Virtual Iron/Xen VMs accross several densly packed  
servers). I'm very stumped here. I'm running CentOS 4.6 using the  
latest CentOS 5.1 kernel.
We are not using IPtables, IP rules or anything fancy. Just into the  
master LVS box then directly routed to an available real server.

 From my knowledge of our DR works, having multiple virtual servers  
for https going to the same real server ip:port should not be an issue  
because
of the interior IP packet once the frame has reached the real server.  
I've even tried a unique port number on the real server (e.g. 444)  
just to
see if that would remedy my issue and no dice.

Here is the output from ipvsadm -L

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  66.*.*.158:443 rr persistent 50
   -> 66.*.*.99:443            Route   1      0          0
   -> 66.*.*.60:443            Route   1      0          0
TCP  66.*.*.115:80 rr persistent 50
   -> 66.*.*.114:81            Route   1      0          0
   -> 66.*.*.113:81            Route   1      0          0
TCP  66.*.*.44:25 rr persistent 50
   -> 66.*.*.43:25             Route   1      0          0
   -> 66.*.*.45:25             Route   1      0          0
TCP  66.*.*.42:25 rr persistent 50
   -> 66.*.*.43:25             Route   1      28         100
   -> 66.*.*.45:25             Route   1      25         77
TCP  66.*.*.57:80 rr persistent 50
   -> 66.*.*.65:81             Route   1      0          0
   -> 66.*.*.60:81             Route   1      0          0
TCP  66.*.*.57:443 rr persistent 50
   -> 66.*.*.65:443            Route   1      0          0
   -> 66.*.*.60:443            Route   1      0          0
TCP  66.*.*.242:80 rr persistent 50
   -> 66.*.*.241:80            Route   1      3          0
   -> 66.*.*.240:80            Route   1      11         4
TCP  66.*.*.158:80 rr persistent 50
   -> 66.*.*.65:81             Route   1      0          0
   -> 66.*.*.60:81             Route   1      0          0
TCP  66.*.*.167:80 rr persistent 50
   -> 66.*.*.65:81             Route   1      0          0
   -> 66.*.*.60:81             Route   1      0          0


Does anyone have any thoughts. Is there something in my configuration  
that screams I'm an idiot?


Errol





--------------------------------------------------------------------------------

This email and any files transmitted with it are confidential and are  
intended solely for the use of the individual or entity to whom they  
are addressed. This communication may contain material protected by  
the attorney-client privilege. If you are not the intended recipient,  
be advised that any use, dissemination, forwarding, printing or  
copying is strictly prohibited. If you have received this email in  
error, please contact the sender and delete all copies.





<Prev in Thread] Current Thread [Next in Thread>