|
Hello,
On Thu, 26 Oct 2000, Thomas Proell wrote:
> Hi!
>
> > My thoughts are about the load split. I don't believe in
> > the main rule on which is based this scheduling, i.e. the assumption
> > that the director is authoritative master for the load information.
>
> Load split isn't easy. You can't meassure the load of the caches from
> the director. Even if you have a program running on the caches, it's
> hard to tell what "load" really is.
> Would it be a good idea to messure the time a simple "ping" needs
> to estimate the load of the cache/server?
It is not enough!
> > Even the WLC scheduler has a simple load balancing. IMO, blindly
>
> Yes, it counts the active connections. But this is not the real load
> in every case. Think about a cache that has 2000 open connections for
> the same file and the origin server sends 20 bytes/sec. This isn't
> loaded, whereas the other cache with 1999 open connections for the
> already cached linux kernel will break down soon.
> Nevertheless WLC sends the next request to this cache, because it's
> less loaded.
Yes, this is simple scheduler. The preferred solution is to
use WRR and to change the weights according to the real server load.
But not for your setup where the proxy servers don't answer for
same sites.
> We have only two possibilities:
> -running software on the caches/servers that meassures the actual load
> very good, sends it to the director, who uses it for balancing, OR
Yes, this is good for the other clusters but it seems it is
not very good if you want to use it for the discussed setups, i.e.
when the proxy server is selected from the destination address.
> -trying to find a more or less exact estimation of the actual load on
> the caches/servers as "number of open connectrions (lc)" or "statisticaly
> balanced (ch)"
>
> All actual implementations use the second idea. And you'll always find
> an unfair situation, because they only estimate the load. The question
> is, if the estimation is close enough on reality to work.
The trick is the expression from the load parameters to be
controlled, i.e. tuned for the current setup: different weights for
the load parameters in the expression, different weights for the
real servers, etc.
> > selecting the real servers in a cluster environment can lead to more
> > problems.
>
> BTW the hot spot solution will be implemented this year. This will cut
> off the peaks. So, most of the sites will be distributed with consistent
> hashing, and the few hot spots will be distributed with least connection
> or something like that.
> So, most of the sites will be distributed without replication, and
> most of the load/requests (for only few sites) will be distributed
> among the least connected.
>
> Thats the advantage of both with taking the disadvantages only slightly -
> only very few sites are replicated.
>
> BTW: the weighting is implemented. I'm just not very keen on making
> a new patch for such little changes.
It is very interesting to see all these features combined in
one scheduler: the load balancing and the consistent hashing. I'm
waiting for your next release. Are the weights constant values?
> > Because if the swapping is started it is very difficult
> > to escape from this situation.
>
> Swapping? From RAM to disk or what? I don't understand that.
The high request rate leads to very big memory consumptions
or delays in the processing. Of course, it depends. It is very
painful to understand it in production without a way to control it.
> > I have a setup where the real server's
> > load is monitored and the problems are isolated in seconds.
>
> How is the load monitored?
The director(s) register for specific load parameters and
the user space daemons in the real servers report the current/averaged
values in short intervals of time. Then these values are used to
evaluate an user specified expression from load parameters. The result
is the real server weight for the WRR method. Some thoughts about
such load balancing:
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=96432992117737&w=2
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=96252502113930&w=2
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=96268604328970&w=2
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=95959865123051&w=2
> > Without
> > such monitoring the servers are simply killed from requests. But only
> > the test results with the new scheduler can give more information about
> > the load balancing. And it depends on the load.
>
> Sure. The requests for a single company follow a
> very extreme zipf-distribution, with only 10 sites requested more
> often than 2500 times a day. If three of these sites are unfortunately
> mapped to the same cache, it'll have a bad time.
The test with single company is not interesting :) The load
is not big and we don't expect equal load, it is not meaningful.
> But with 10 companies together, there's a good chance to get a
> zipf-distribution that is wider. We may have 30 or 50 "hot" sites,
> and the possibility of an imbalance is far smaller.
I can give you other examples for good hash functions,
may be you already have such information:
http://www.citi.umich.edu/projects/linux-scalability/reports/hash.html
The hash function looks promising:
(value * 2654435761UL) & HASH_MASK;
And it is even faster than some shift arithmetics.
> There are several studies who worked with that already - at least my
> Prof. says so...
>
> But this won't be a theme with the hot spot solution!
Regards
--
Julian Anastasov <ja@xxxxxx>
|
