|
There's two problems - population & search.
If you're searching a database of web (or weblike) pages, then if you
randomly partition the contents of the database across N servers you can
search on one machine, then the next, then you're done. Consider it an odd
kind of cursor...
Look at it this way:
* In our nat cache, we've got N cache boxes, we know contain bucket
loads of stuff, with a low level of data redundancy.
* This is populated by end users - but it could be populated by a robot
prog.
* We could add cgi script onto each server that can search the contents
of the server for specific content on that server. If it fails, the
user is redirected to the next server. (Or better the search could be
passed onto a server that may have the results based on a hint - in a
similar way to cache digests - the same way hashed urls can be used
as a hint - the idea *was* pinched from databases in the first place.)
* We load balance access to the web server hosting the cgi script using
a normal LVS.
This'd only work I admit for very specialised setups, but can still work,
especially in the web page search scenario.
Michael.
--
National & Local Web Cache Support R: G117
Manchester Computing T: 0161 275 7195
University of Manchester F: 0161 275 6040
Manchester UK M13 9PL M: Michael.Sparks@xxxxxxxxxxxxxxx
On Tue, 1 Feb 2000, Wayne wrote:
> Database cluster is different from the http protocol.
> In most database servers, there is a transaction log
> file which has all the transactions. Until the time the
> transactions committed to the table, this one log will
> handle all the transactions from clients to the database
> server. It is always good idea NOT put this log on a
> NFS file server, since the speed is important.
>
> How could clustering handle one log file on a server
> when one wants to have more servers to handle
> the transactions? Any mis-management to this log
> file will meant the whole database is screwed.
>
>
> At 01:02 PM 2/1/00 -0600, Ron King wrote:
> >Hello,
> >
> >I haven't found much on Inktomi (Hotbot's) original architecture, but I
> >did
> >find 'Cluster-Based Scalable Network Services'. In this paper they say:
> >'Hotbot workers statically partition the search-engine database for load
> >
> >balancing'. 'Each worker handles a subset of the database proportional
> >to its CPU power..'
> >
> >I want to use a cluster of 32-bit Linux systems to handle a large
> >database
> >of several hundred million web pages. I think this would require some
> >sort
> >of cluster front end, because all data queries would have to be sent to
> >all
> >of the nodes, and the results from the nodes combined before being
> >presented
> >to the user.
> >
> >Is LVS suitable for something like this? If not, can anyone recommend an
> >
> >existing open source system that can do this?
> >
> >Regards,
> >
> >Ron King
> >
> >
> >----------------------------------------------------------------------
> >LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> >To unsubscribe, e-mail: lvs-users-unsubscribe@xxxxxxxxxxxxxxxxxxxxxx
> >For additional commands, e-mail: lvs-users-help@xxxxxxxxxxxxxxxxxxxxxx
>
>
> ----------------------------------------------------------------------
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe, e-mail: lvs-users-unsubscribe@xxxxxxxxxxxxxxxxxxxxxx
> For additional commands, e-mail: lvs-users-help@xxxxxxxxxxxxxxxxxxxxxx
>
----------------------------------------------------------------------
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
To unsubscribe, e-mail: lvs-users-unsubscribe@xxxxxxxxxxxxxxxxxxxxxx
For additional commands, e-mail: lvs-users-help@xxxxxxxxxxxxxxxxxxxxxx
|
