LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

RE: LVS problem with SSLProxy

To: "'LinuxVirtualServer.org users mailing list.'" <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: RE: LVS problem with SSLProxy
From: "Longhua Li" <zyllh@xxxxxxxxxxxx>
Date: Wed, 12 Oct 2005 15:53:09 -0600
I had plenty of tests. And I figured that my SSLProxy is on top of Load
Balancer. 
Ideally, HTTPS Request (443) -> SSLProxy (443) -> (80) -> Load Balancer (80)
-> Real Servers (80)
And Load Balancer does not need to know SSLProxy's existence. 
So I had this test configs:
TEST1:
virtual=xxx.xxx.xxx.xxx:80
        service=http
        fallback=127.0.0.1:80
        request="index.html"
        receive="Test Page"
        scheduler=rr
        protocol=tcp
        real=192.168.0.233:80 masq 1
        real=192.168.0.234:80 masq 1

In browser: https://xxx.xxx.xxx.xxx and this should go through SSLProxy and
then go to real server 233 or 234. But it turned out that it went to the
fall back server. 

If SSLProxy is stepping on Load Balancer, following will definitely fail. 
HTTPS Requsts (443) -> Load Balancer (80) -> Real Servers (80) 
TEST2: 
virtual=xxx.xxx.xxx.xxx:443
        service=http
        fallback=127.0.0.1:80
        request="index.html"
        receive="Test Page"
        scheduler=rr
        protocol=tcp
        real=192.168.0.233:80 masq 1
        real=192.168.0.234:80 masq 1

and ipvsadm -L -n 
shows:
TCP  xxx.xxx.xxx.xxx:443 rr
  -> 192.168.0.234:80             Masq    1      0          0         
  -> 192.168.0.233:80             Masq    1      0          0         
  -> 192.168.0.233:443            Masq    0      0          0         
  -> 192.168.0.234:443            Masq    0      0          0

In browser: https://xxx.xxx.xxx.xxx And it turned out that "page can not be
displayed". Obviously because those SSL connections are forwarded to real
servers' 80 port and they can not be handled correctly.  
But if input http://xxx.xxx.xxx.xxx:443 which goes to the real servers (no
ssl handshake or certificate transferred. No certificate popup windows. ).
>From user point of view, it is simply an http server running on port 443.  

TEST3:
virtual= xxx.xxx.xxx.xxx:443
        service=https
        fallback=127.0.0.1:80
        request="index.html"
        receive="Test Page"
        scheduler=rr
        protocol=tcp
        checktype=negotiate
        real=192.168.0.233:80 masq 1
        real=192.168.0.234:80 masq 1

and ipvsadm -L -n
shows:
TCP  xxx.xxx.xxx.xxx:443 rr
  -> 192.168.0.233:443            Masq    0      0          0         
  -> 192.168.0.234:443            Masq    0      0          0         
  -> 127.0.0.1:443                Local   1      0          0

And this setup just doesn't look right. And https://xxx.xxx.xxx.xxx goes to
the fall back server. 

TEST4:
virtual= xxx.xxx.xxx.xxx:443
        service=https
        fallback=127.0.0.1:443
        request="index.html"
        receive="Test Page"
        scheduler=rr
        protocol=tcp
        checktype=negotiate
        real=192.168.0.233:443 masq 1
        real=192.168.0.234:443 masq 1

If SSLProxy is handling https requests, the certificate will come from the
Load Balancer and then "page can not be displayed" (because SSLProxy switch
443 to 80). But the fact is the certificate is from real servers. And the
test web pages are from real servers too.

My conclusion is that before the packets delivered to the SSLProxy program
they are distributed by Load Balancer. How is your SSLProxy's behavior? 

Thanks for reading my very long message!

Longhua


-----Original Message-----
From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
[mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Mark
Sent: Tuesday, October 11, 2005 5:40 PM
To: 'LinuxVirtualServer.org users mailing list.'
Subject: RE: LVS problem with SSLProxy

Yes, but your SSL proxy takes care of HTTPS envelopes.
That's the whole purpose of it, isn't it? Otherwise, what does it do if not
that? Or are you talking about a SSL-VPN-tunnel?

Client sends HTTPS request, SSLProxy handles HTTPS and passes on regular
HTTP to whatever is downstream, in your case the load
balancer.
The the server returns HTTP, it passes back through the loadbalancer, and
the SSLProxy encrypts it and wraps it into an HTTPS
envelope and sends it back to the client as response.
Unless I am misunderstanding what your SSLProxy does...

Just give it a try... I don't know what "service=https" actually does, but I
know changing it to "service=http" fixed the entire
thing for me.



> -----Original Message-----
> From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx 
> [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf 
> Of Longhua Li
> Sent: Tuesday, October 11, 2005 3:27 PM
> To: 'LinuxVirtualServer.org users mailing list.'
> Subject: RE: LVS problem with SSLProxy
> 
> 
> No, no, no, this is not right. The request is not handled by 
> the SSL proxy at all but instead, it is forwarded to real servers. 
> 
> -----Original Message-----
> From: lvs-users-bounces+zyllh=adelphia.net@xxxxxxxxxxxxxxxxxxxxxx
> [mailto:lvs-users-bounces+zyllh=adelphia.net@linuxvirtualserve
> r.org] On Behalf Of Longhua Li
> Sent: Tuesday, October 11, 2005 4:19 PM
> To: 'LinuxVirtualServer.org users mailing list.'
> Subject: RE: LVS problem with SSLProxy
> 
> Thanks.
> As you suggested here is my setup
> 
> virtual=xxx.xxx.xxx.xxx:441
>         service=http
>         fallback=127.0.0.1:80
>         request="index.html"
>         receive="Test Page"
>         scheduler=rr
>         protocol=tcp
>         real=192.168.0.233:80 masq 1
>         real=192.168.0.234:80 masq 1
> 
> And ipvsadm -L -n
> Shows
> TCP  xxx.xxx.xxx.xxx:441 rr
>   -> 192.168.0.234:80             Masq    1      0          1         
>   -> 192.168.0.233:80             Masq    1      0          0         
>   -> 192.168.0.233:441            Masq    0      0          0         
>   -> 192.168.0.234:441            Masq    0      0          0
> 
> The request is http://xxx.xxx.xxx.xxx:441 instead of 
> https://xxx.xxx.xxx.xxx.441 and I saw the real server pages 
> instead of the fall back server. Question is why I did not 
> see the certificate pop up windows? Might be my sslproxy problem? 
> Thank you very much and your suggestions are very helpful!
> 
> Longhua
> 
> -----Original Message-----
> From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Mark
> Sent: Tuesday, October 11, 2005 3:51 PM
> To: 'LinuxVirtualServer.org users mailing list.'
> Subject: RE: LVS problem with SSLProxy
> 
> I think the problem is in your ldirector parameters.
> Here are the parameters that work for me:
>         service=http
>         request="pooling_status.html"
>         receive="html"
>         scheduler=rr
>         protocol=tcp
> 
> "request" and "receive" you have to adjust to your scenario.
> I don't use checktype (not sure what the default is).
> Service is definitely "http" for you, since the HTTPS envelop 
> is already stripped off when ldirector comes into play. Even 
> for me "service=https" didn't work - no idea why, since 
> unlike you, I actually DO have https traffic going through 
> ldirectord. Maybe it validates the certificate or something.
> 
> After I switched from "service=https" to "service=http" it 
> worked fine. That should do the trick for you as well...
> 
> MARK
> 
> > -----Original Message-----
> > From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> > [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf 
> > Of Longhua Li
> > Sent: Tuesday, October 11, 2005 2:13 PM
> > To: 'LinuxVirtualServer.org users mailing list.'
> > Subject: RE: LVS problem with SSLProxy
> > 
> > 
> > Thanks for your information.
> > The SSLProxy is a homemade stuff. My coworker wrote it. And I
> > don't know how he implements it. And I don't know the 
> > internals of the ldirector mechanisms either. Hmm, big 
> problem then. 
> > Ideally, ldirector does not need to know SSLProxy.
> > The request goes like this
> > Client -> SSLProxy(443)->(80)->Load Balancer(80)->Real Servers (80)
> > 
> > After I run my SSLProxy, response always comes from the fall
> > back server instead of from any real servers. It looks like I 
> > have to look into the source codes. Any suggestions! Thanks again
> > 
> > Longhua
> > 
> > -----Original Message-----
> > From: lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx
> > [mailto:lvs-users-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Mark
> > Sent: Tuesday, October 11, 2005 11:50 AM
> > To: 'LinuxVirtualServer.org users mailing list.'
> > Subject: RE: LVS problem with SSLProxy
> > 
> > Hm...
> > I'm not sure about choosing option 2 over option 1. As far as
> > I know, the SSL stuff can be pretty processor heavy... But 
> > anyway... I think what you want to do should be possible 
> > (ldirector and SSLProxy on one machine) - although I haven't 
> > done it myself...
> > 
> > The only problem I could think of is that the box figures out
> > that both pieces are its own Ips and then somehow uses the 
> > local interface rather than the eth interfaces. If that 
> > happens, I am not sure about whether the ldirector hooks are 
> > still able to pick up and redirect the traffic, or if they 
> > rely on it coming in through a real eth interface... I don't 
> > know enough about the internals of the ldirector 
> > mechanisms... If you really cant get that to work, try it 
> > with option 1, maybe that will work. 
> > 
> > The most important thing is to make sure that you have the
> > input and output ports of each chain element set up 
> properly, I think.
> > 
> > Which SSLProxy are you using?
> > 
> > I use a load balancer to forward HTTPS to a bunch of apache
> > servers, each of them has their own HTTPS proxy. I just use 
> > mod_rewrite for that. You could use mod_proxy as well, but 
> > mod_rewrite gives you better mapping options with regex-based 
> > rules, etc...
> > 
> > MARK
> > 
> > 
> > 
> > _______________________________________________
> > LinuxVirtualServer.org mailing list -
> > lvs-users@xxxxxxxxxxxxxxxxxxxxxx Send requests to 
> > lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> > 
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> 
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> 
> 
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> 




<Prev in Thread] Current Thread [Next in Thread>