LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

RE: LVS problem with SSLProxy

To: "'LinuxVirtualServer.org users mailing list.'" <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: RE: LVS problem with SSLProxy
From: "Mark" <msalists@xxxxxxx>
Date: Wed, 12 Oct 2005 15:47:36 -0700
Your first setup seems correct to me.
Do you have lynx or curl or some other text-based web-browser on the 
loadbalancer machine?
What do you get when you load http://192.168.0.233/index.html into that browser 
(e.g. "lynx http://192.168.0.233";)?
You can also just "telnet 192.168.0.233 80" and enter:
GET /index.html<ENTER><ENTER>

The response needs to have the string "Test Page" somewhere in the HTML source.

What do you see in the ldirectord log files? It should say something like:
        Deleted fallback server: 127.0.0.1:80
        Added real server: 192.168.0.233:80

MARK


> -----Original Message-----
> From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx 
> [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf 
> Of Longhua Li
> Sent: Wednesday, October 12, 2005 2:53 PM
> To: 'LinuxVirtualServer.org users mailing list.'
> Subject: RE: LVS problem with SSLProxy
> 
> 
> I had plenty of tests. And I figured that my SSLProxy is on 
> top of Load Balancer. 
> Ideally, HTTPS Request (443) -> SSLProxy (443) -> (80) -> 
> Load Balancer (80)
> -> Real Servers (80)
> And Load Balancer does not need to know SSLProxy's existence. 
> So I had this test configs:
> TEST1:
> virtual=xxx.xxx.xxx.xxx:80
>         service=http
>         fallback=127.0.0.1:80
>         request="index.html"
>         receive="Test Page"
>         scheduler=rr
>         protocol=tcp
>         real=192.168.0.233:80 masq 1
>         real=192.168.0.234:80 masq 1
> 
> In browser: https://xxx.xxx.xxx.xxx and this should go 
> through SSLProxy and then go to real server 233 or 234. But 
> it turned out that it went to the fall back server. 
> 
> If SSLProxy is stepping on Load Balancer, following will 
> definitely fail. 
> HTTPS Requsts (443) -> Load Balancer (80) -> Real Servers (80) 
> TEST2: 
> virtual=xxx.xxx.xxx.xxx:443
>         service=http
>         fallback=127.0.0.1:80
>         request="index.html"
>         receive="Test Page"
>         scheduler=rr
>         protocol=tcp
>         real=192.168.0.233:80 masq 1
>         real=192.168.0.234:80 masq 1
> 
> and ipvsadm -L -n 
> shows:
> TCP  xxx.xxx.xxx.xxx:443 rr
>   -> 192.168.0.234:80             Masq    1      0          0         
>   -> 192.168.0.233:80             Masq    1      0          0         
>   -> 192.168.0.233:443            Masq    0      0          0         
>   -> 192.168.0.234:443            Masq    0      0          0
> 
> In browser: https://xxx.xxx.xxx.xxx And it turned out that 
> "page can not be displayed". Obviously because those SSL 
> connections are forwarded to real servers' 80 port and they 
> can not be handled correctly.  
> But if input http://xxx.xxx.xxx.xxx:443 which goes to the 
> real servers (no ssl handshake or certificate transferred. No 
> certificate popup windows. ).
> >From user point of view, it is simply an http server running on port 
> >443.
> 
> TEST3:
> virtual= xxx.xxx.xxx.xxx:443
>         service=https
>         fallback=127.0.0.1:80
>         request="index.html"
>         receive="Test Page"
>         scheduler=rr
>         protocol=tcp
>         checktype=negotiate
>         real=192.168.0.233:80 masq 1
>         real=192.168.0.234:80 masq 1
> 
> and ipvsadm -L -n
> shows:
> TCP  xxx.xxx.xxx.xxx:443 rr
>   -> 192.168.0.233:443            Masq    0      0          0         
>   -> 192.168.0.234:443            Masq    0      0          0         
>   -> 127.0.0.1:443                Local   1      0          0
> 
> And this setup just doesn't look right. And 
> https://xxx.xxx.xxx.xxx goes to the fall back server. 
> 
> TEST4:
> virtual= xxx.xxx.xxx.xxx:443
>         service=https
>         fallback=127.0.0.1:443
>         request="index.html"
>         receive="Test Page"
>         scheduler=rr
>         protocol=tcp
>         checktype=negotiate
>         real=192.168.0.233:443 masq 1
>         real=192.168.0.234:443 masq 1
> 
> If SSLProxy is handling https requests, the certificate will 
> come from the Load Balancer and then "page can not be 
> displayed" (because SSLProxy switch 443 to 80). But the fact 
> is the certificate is from real servers. And the test web 
> pages are from real servers too.
> 
> My conclusion is that before the packets delivered to the 
> SSLProxy program they are distributed by Load Balancer. How 
> is your SSLProxy's behavior? 
> 
> Thanks for reading my very long message!
> 
> Longhua
> 
> 
> -----Original Message-----
> From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Mark
> Sent: Tuesday, October 11, 2005 5:40 PM
> To: 'LinuxVirtualServer.org users mailing list.'
> Subject: RE: LVS problem with SSLProxy
> 
> Yes, but your SSL proxy takes care of HTTPS envelopes.
> That's the whole purpose of it, isn't it? Otherwise, what 
> does it do if not that? Or are you talking about a SSL-VPN-tunnel?
> 
> Client sends HTTPS request, SSLProxy handles HTTPS and passes 
> on regular HTTP to whatever is downstream, in your case the 
> load balancer. The the server returns HTTP, it passes back 
> through the loadbalancer, and the SSLProxy encrypts it and 
> wraps it into an HTTPS envelope and sends it back to the 
> client as response. Unless I am misunderstanding what your 
> SSLProxy does...
> 
> Just give it a try... I don't know what "service=https" 
> actually does, but I know changing it to "service=http" fixed 
> the entire thing for me.
> 
> 
> 
> > -----Original Message-----
> > From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> > [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf 
> > Of Longhua Li
> > Sent: Tuesday, October 11, 2005 3:27 PM
> > To: 'LinuxVirtualServer.org users mailing list.'
> > Subject: RE: LVS problem with SSLProxy
> > 
> > 
> > No, no, no, this is not right. The request is not handled by
> > the SSL proxy at all but instead, it is forwarded to real servers. 
> > 
> > -----Original Message-----
> > From: lvs-users-bounces+zyllh=adelphia.net@xxxxxxxxxxxxxxxxxxxxxx
> > [mailto:lvs-users-bounces+zyllh=adelphia.net@linuxvirtualserve
> > r.org] On Behalf Of Longhua Li
> > Sent: Tuesday, October 11, 2005 4:19 PM
> > To: 'LinuxVirtualServer.org users mailing list.'
> > Subject: RE: LVS problem with SSLProxy
> > 
> > Thanks.
> > As you suggested here is my setup
> > 
> > virtual=xxx.xxx.xxx.xxx:441
> >         service=http
> >         fallback=127.0.0.1:80
> >         request="index.html"
> >         receive="Test Page"
> >         scheduler=rr
> >         protocol=tcp
> >         real=192.168.0.233:80 masq 1
> >         real=192.168.0.234:80 masq 1
> > 
> > And ipvsadm -L -n
> > Shows
> > TCP  xxx.xxx.xxx.xxx:441 rr
> >   -> 192.168.0.234:80             Masq    1      0          
> 1         
> >   -> 192.168.0.233:80             Masq    1      0          
> 0         
> >   -> 192.168.0.233:441            Masq    0      0          
> 0         
> >   -> 192.168.0.234:441            Masq    0      0          0
> > 
> > The request is http://xxx.xxx.xxx.xxx:441 instead of
> > https://xxx.xxx.xxx.xxx.441 and I saw the real server pages 
> > instead of the fall back server. Question is why I did not 
> > see the certificate pop up windows? Might be my sslproxy problem? 
> > Thank you very much and your suggestions are very helpful!
> > 
> > Longhua
> > 
> > -----Original Message-----
> > From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> > [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Mark
> > Sent: Tuesday, October 11, 2005 3:51 PM
> > To: 'LinuxVirtualServer.org users mailing list.'
> > Subject: RE: LVS problem with SSLProxy
> > 
> > I think the problem is in your ldirector parameters.
> > Here are the parameters that work for me:
> >         service=http
> >         request="pooling_status.html"
> >         receive="html"
> >         scheduler=rr
> >         protocol=tcp
> > 
> > "request" and "receive" you have to adjust to your 
> scenario. I don't 
> > use checktype (not sure what the default is). Service is definitely 
> > "http" for you, since the HTTPS envelop is already stripped 
> off when 
> > ldirector comes into play. Even for me "service=https" 
> didn't work - 
> > no idea why, since unlike you, I actually DO have https 
> traffic going 
> > through ldirectord. Maybe it validates the certificate or something.
> > 
> > After I switched from "service=https" to "service=http" it
> > worked fine. That should do the trick for you as well...
> > 
> > MARK
> > 
> > > -----Original Message-----
> > > From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> > > [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf
> > > Of Longhua Li
> > > Sent: Tuesday, October 11, 2005 2:13 PM
> > > To: 'LinuxVirtualServer.org users mailing list.'
> > > Subject: RE: LVS problem with SSLProxy
> > > 
> > > 
> > > Thanks for your information.
> > > The SSLProxy is a homemade stuff. My coworker wrote it. 
> And I don't 
> > > know how he implements it. And I don't know the internals of the 
> > > ldirector mechanisms either. Hmm, big
> > problem then.
> > > Ideally, ldirector does not need to know SSLProxy.
> > > The request goes like this
> > > Client -> SSLProxy(443)->(80)->Load Balancer(80)->Real 
> Servers (80)
> > > 
> > > After I run my SSLProxy, response always comes from the fall back 
> > > server instead of from any real servers. It looks like I have to 
> > > look into the source codes. Any suggestions! Thanks again
> > > 
> > > Longhua
> > > 
> > > -----Original Message-----
> > > From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> > > [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On 
> Behalf Of Mark
> > > Sent: Tuesday, October 11, 2005 11:50 AM
> > > To: 'LinuxVirtualServer.org users mailing list.'
> > > Subject: RE: LVS problem with SSLProxy
> > > 
> > > Hm...
> > > I'm not sure about choosing option 2 over option 1. As far as I 
> > > know, the SSL stuff can be pretty processor heavy... But 
> anyway... I 
> > > think what you want to do should be possible (ldirector 
> and SSLProxy 
> > > on one machine) - although I haven't done it myself...
> > > 
> > > The only problem I could think of is that the box figures 
> out that 
> > > both pieces are its own Ips and then somehow uses the local 
> > > interface rather than the eth interfaces. If that 
> happens, I am not 
> > > sure about whether the ldirector hooks are still able to 
> pick up and 
> > > redirect the traffic, or if they rely on it coming in 
> through a real 
> > > eth interface... I don't know enough about the internals of the 
> > > ldirector mechanisms... If you really cant get that to 
> work, try it
> > > with option 1, maybe that will work. 
> > > 
> > > The most important thing is to make sure that you have 
> the input and 
> > > output ports of each chain element set up
> > properly, I think.
> > > 
> > > Which SSLProxy are you using?
> > > 
> > > I use a load balancer to forward HTTPS to a bunch of 
> apache servers, 
> > > each of them has their own HTTPS proxy. I just use 
> mod_rewrite for 
> > > that. You could use mod_proxy as well, but mod_rewrite gives you 
> > > better mapping options with regex-based rules, etc...
> > > 
> > > MARK
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > LinuxVirtualServer.org mailing list - 
> > > lvs-users@xxxxxxxxxxxxxxxxxxxxxx Send requests to 
> > > lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> > > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> > > 
> > 
> > _______________________________________________
> > LinuxVirtualServer.org mailing list - 
> lvs-users@xxxxxxxxxxxxxxxxxxxxxx 
> > Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> > 
> > 
> > _______________________________________________
> > LinuxVirtualServer.org mailing list - 
> lvs-users@xxxxxxxxxxxxxxxxxxxxxx 
> > Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> > 
> > 
> > _______________________________________________
> > LinuxVirtualServer.org mailing list - 
> lvs-users@xxxxxxxxxxxxxxxxxxxxxx 
> > Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> > 
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - 
> lvs-users@xxxxxxxxxxxxxxxxxxxxxx Send requests to 
> lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> 
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - 
> lvs-users@xxxxxxxxxxxxxxxxxxxxxx Send requests to 
> lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> 


<Prev in Thread] Current Thread [Next in Thread>