Re: Web Stats on LVS?

[Dan Trainor <dan@xxxxxxxxxxxxxx>]

Graeme Fowler wrote:
> On Thu, 2006-05-04 at 11:55 +0200, Lasse Karstensen wrote:
> > I tried compiling mod_log_spread a few months ago, even found a
> > apache2 patch on some mailling list, but without luck.
> >
> > Anyone succeeded using it with apache2?
> Yes, although I no longer have access to either the slightly modified
> codebase or the production results. It worked well, however, shared
> between 8 servers with a pair of collectors. There was a slight lag to
> log processing as each listener piped the log arrivals through a Perl
> script (running in a "while (<>)" loop), did $magic with them, and put
> them in the right users' logfiles.
>
> > Which is the solution we are using now. What kind of scaling issues
> > are you predicting? We're having 10-12 realservers now, with some 
> > moderate amounts of traffic.
> I'm saying that having each realserver act as a logging host for all the
> rest won't scale. Beyond a pair, having a dedicated syslog host (or
> indeed more than one, for robust logging) is the way forward, as you
> say.
>
> Graeme
Hi -

What we've done in the past is also used mod_log_mysql.  While not the 
most efficient way of logging (sure, your setup may differ) - this did 
allow us to be flexible as to where we wanted logs sent.  We would dump 
this log nightly and export it back into one on disk, then run our stats 
against it.  We later modified our stats system to read directly from 
the database, which worked out quite nicely.
That's a lot of work, though.  I guess if you'r elooking for something 
simple, that's not the answer for you.  However, it's food for thought.
Thanks!
-dant