|
On Thursday 21 June 2007 10:04, Joseph Mack NA3T wrote:
> On Thu, 21 Jun 2007, Rio wrote:
>
> > We presently want to test a clustering system using one of our active
servers
> > and a mirror that is identical in every way. All machines are AMD64
opteron
> > arch running Gentoo.
> >
> > Here is our setup and what we hope to do:
> >
> > One of our production servers is a linux-vserver host that runs 57 virtual
> > servers within, and uses 4 public networks assigned to it and the various
> > guest servers for a total unique ip count of 247 across the 4 networks.
>
> I see Malcolm has beat me to it.
>
> you have two identical realservers, each of which presents a
> bunch of services on a bunch of IPs (247) on bunch (4) of
> physical networks.
>
one way of saying it yes :)
> > some virtual servers are busy. We have several websites using in excess of
> > 70GB/mo each with one website using an average of 173GB/mo in bandwidth.
The
> > mail server processes as high as 1/2 million msgs /hr with an average of
> > 260,000/hr.
>
> the smtp is OK as long as it's an MTA. If the mail is being
> delivered locally, you'll have to handle the many reader
> single writer problem.
>
i assume here you mean because there would be 2 copies of each mailbox with no
control over which copy a user reads. the mail server software (surgemail)
has built-in 2-way mirroring that updates each other within milliseconds of a
change.. is this what you are referring to with many reader single writer?
we also have 2 secondary mx servers but they cannot relay they can only send
to the main server for mail handling so I don't worry about them at all.
> > What I wish to do is have some kind of control box (or multiple if needed)
>
> director.
>
ok
> You'll need two if you want failover.
failovers of directors, but if, say, one director handles 2 ips for 2
identical services on 2 servers, if one server goes down does that not do
auto-failover for the real server portion?
>
> > to
> > manage which of the real servers will act on a request.
>
> to route packets to the realservers.
>
> > We would have, to
> > allow for additional machine expansion, 4 private /24 networks asssigned
to
> > each host server and guests (total 8 pvtnets) to represent the 4 public
> > networks.
>
> LVS can handle any number of networks. The number of
> networks is controlled by Linux not LVS.
>
ok then that part is cake.
> > This control box(s) will have to accept the public ip request and map it
to
> > one of a list of private ip addresses servicing that particular public ip
> > address/port combination. It could be 'round robin' or 'least used' no
matter
> > we just want all servers to actively participate rather than have one
sitting
> > idle waiting for the fateful day it will be needed. Fail-over is required
so
> > if a machine dies or otherwise is unavailable, the control box(s) will use
> > the active machines automatically.
>
> you install failover as a separate layer above LVS.
> Ldirectord or keepalived are two ways of doing this.
>
ok. just checked and gentoo supports keepalived in its packaging system.
> > for best bandwidth allocation we would use 5 nics in the control box, one
> > public and one for each of the private networks (or less if it is deemed
> > overkill) and each real server would have the same number of pvtnet nics
for
> > its services.
>
> use whatever number of NICs makes your life easy.
>
ok
> > we use iproute2 exclusively with 2.6.20 kernels. We upgrade kernels
regularly
> > for security/bug fixes once they have proven themselves, so I guess we
update
> > once or twice a year.
>
> I never update a tested and working machine. You're
> asking for trouble. I had the same version of ntpd (from the
> libc4 days) running on a machine for 9 yrs. I know that
> management sometimes has other ideas.
>
we used to do that also, but now have taken a more agressive approach
concerning security and bug fixes that could affect us. we don't just update
for grins, but when we find an update that seems to make fixes in things that
may affect us we do update that.
> > Will LinuxVserver do the job for us?
>
> so far yes.
>
good. especially since the base is built into the kernel :)
> > If so, is there a 'best model'? I suspect the NAT model would be what we
need.
>
> The main problem with LVS-NAT is that it's a little touchy
> with respect to firewall rules. Just add them one at a time
> and be prepared to back out if they don't work. See the
> HOWTO.
>
heh and we have a bunch on the host servers :)
> > The control box would be a 2 processor dual-core opteron so it effectively
> > would have 4 processors and maybe 8gb or more ram.
>
> SMP doesn't get you much in a director. A director is just a
> router with slightly unusual rules. It's not doing much
> computation. dual-core opterons don't have any more
> bandwidth to memory than do single cores so you only get
> more performance with applications that run out of cache.
> You only need about 200bytes/connection, so you can have
> 8G/200 simultaneous connections - is that enough :-)
>
ok so we will run a single processor with 1G ram :) 40 million connections
*is* a bit overkill. I wasn't sure how powerful that had to be so I decided
to think big then reduce as I discovered reality.
> > Do you think we could get away with one 'control' box considering the
> > bandwidth usage?
>
> two if you ever think your box may go down or have to be
> taken off-line for planned maintenance.
makes sense, especially now that I know directors can be considerably smaller
boxes than the real servers are.
>
> Joe
> --
> Joseph Mack NA3T EME(B,D), FM05lw North Carolina
> jmack (at) wm7d (dot) net - azimuthal equidistant map
> generator at http://www.wm7d.net/azproj.shtml
> Homepage http://www.austintek.com/ It's GNU/Linux!
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
--
Chuck
|
