recommendations for the 'one writer, many reader' problem?

To: LVS Maling List <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: recommendations for the 'one writer, many reader' problem?
From: Matthijs van der Klip <matthijs.van.der.klip@xxxxxx>
Date: Thu, 29 Nov 2001 15:10:16 +0100

Right now I'm using rsync to keep 4 lvs'ed webservers in sync with a
seperate upload server, but I'm afraid rsync won't last me long. Does
anyone have any recommendations to solve the 'one writer, many reader'
problem given the following facts?:

1) A docroot of 5GB data, 275K files and growing...

2) Our current IRIX FastTrack server normally caches a maximum of 17.5K
   files in 100Mb memory with a hit ratio of 90%.

3) The site has to be able to handle a total of 100K+ hits per minute.

4) No single point of failure, so if the upload server fails, the
   webservers have to be able to access the data (failsafe).

5) We're looking for a more or less platform independant
   solution. We might decide to use an IRIX upload server on a CXFS SAN
   with Linux webservers.

I have been looking at:

1) Rsync. What we're currently using on a much smaller docroot. 5Gb seems
   too much to rsync.

2) Plain NFS. It seems too slow, insecure and not failsafe.

3) Coda. It seems to immature. The manual features all kinds of
   beta warnings.

4) AFS. OpenAFS is available for both Linux and IRIX so that's good. I'm
   not sure though how to create a failsafe docroot with AFS.

5) Intermezzo. Again Linux only and to immature.

6) Webd and SLIM. I haven't been able to find any other information but
   this URL:

   Seems interesting as FAM ( is available for
   both Linux and IRIX.

7) GFS. I don't know anything about this one except that it costs money...

I'm open for any input; particularly interested in how this problem is
solved at some of the others' users sites.

Best regards,

Matthijs van der Klip - Unix Administrator
NOS - Dutch Public Broadcasting Organisation

<Prev in Thread] Current Thread [Next in Thread>