LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Release new code: Scheduler for distributed caching

To: Joe Cooper <joe@xxxxxxxxxxxxx>
Subject: Re: Release new code: Scheduler for distributed caching
Cc: "lvs-users@xxxxxxxxxxxxxxxxxxxxxx" <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
From: Julian Anastasov <ja@xxxxxx>
Date: Thu, 26 Oct 2000 01:10:41 +0000 (GMT)
        Hello,

On Wed, 25 Oct 2000, Joe Cooper wrote:

> You're missing the point.  You _are_ load balancing with this
> algorithm.  Just because the same sites always get accessed through the
> same servers doesn't mean we aren't going to see 'balanced' usage on
> each cache, which is our goal.  Getting the same data on to and off of
> every cache is not balancing--it's wasted redundancy and wasted
> bandwidth--when talking about web caches.

        My understanding is that the load is dynamic and we can't simply
stay and hope the load will be equal. Because if the request rate is high
you can break the proxy servers very soon. One little problem and the
kernel starts to swap.

> >         The other disadvantage when using LVS is that we waste memory
> > for connection tracking when it is not needed: LVS does not use different
> > real server for same destination (web server) in these setups with
> > transparent proxy servers.
>
> True.  Can a NetFilter module do the same thing (hash based balancing)
> without that connection tracking?

        If the static mapping is broken, someone have to remember
the proxy server for each request.

> >         For the balaning, see above. For the module, this module will
> > only mark the packets. May be only the NAT mode can be problematic
> > but are the proxy servers after NAT box really? Because considering the
> > forwarding methods LVS can give you only LVS/NAT method as a bonus
> > compared to the other variants with plain routing. Because the demasq
> > is tricky to run without LVS. But you can use tunneling and direct
> > routing even without LVS.
>
> NAT is a pretty important feature.  Not all web caches run Squid or
> Linux.  Though we can address the ARP problem via other methods, it's
> probably nice to be able to operate the same way as the proprietary
> vendors in the same arena.

        Hm, is the LVS/NAT mode working for transparent proxies? May
be I'm missing something.

> >         May be I'm wrong but IMO this new scheduler breaks the load
> > balancing and gives you static mapping. It seems that the advantage is
> > for the users with traffic limits. Of course, this scheduler can be
> > useful when the proxy servers are not loaded from requests because
> > this kind of round-robin is dangerous for busy clusters. But we can
> > see the results only in production.
>
> I think you're underestimating the diversity of addresses being visited
> on a busy web cache cluster, Julian.  We're talking about 1000
> requests/second and up.  That's a lot of different websites, and
> balancing will be very effective with a hashed selection.

        We have to analyze the results from such setups.

> Will do.  I've got some less busy time (I've never got 'free' time ;-)
> over the next couple of weeks, so I'll see if I can get a cluster
> running with Thomas' scheduler.

        Very good because I'm not a webcache guru :)

>                                   --
>                      Joe Cooper <joe@xxxxxxxxxxxxx>
>                  Affordable Web Caching Proxy Appliances
>                         http://www.swelltech.com


Regards

--
Julian Anastasov <ja@xxxxxx>



<Prev in Thread] Current Thread [Next in Thread>