Re: [lvs-users] Directors, single point of failure

To: " users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Directors, single point of failure
From: Graeme Fowler <graeme@xxxxxxxxxxx>
Date: Mon, 29 Oct 2007 19:35:30 +0000
On Mon, 2007-10-29 at 10:00 -0700, Joseph Mack NA3T wrote: 
> There is no cheap solution. If you have a two pairs of 
> directors in separate locations, then the data centers have 
> to cooperate (usually the same company) and they have to do 
> fast DNS updates.

I'd caution against using round-robin resource records in DNS, since
there's a significant number of providers out there who cache records
for longer than their TTL dictates. Once a given user on a given ISP has
one, if it disappears (in IP terms) they're stuck until their providers
caching nameserver expires the record.

As Joe says, there's no cheap way to do this.

One way to achieve some level of distributed resilience is to move the
connectivity endpoint back to your providers, by colocating the far end
routers yourself and then using their backhaul to your hosting
centre(s). This way you can put the directors out in the colo, keeping
the realservers back in your facility(ies).

Using a reasonable router and something like BGP, you have ultimate
control over what prefixes are where within your network - so you can
keep the VIPs at the far end, but logically keep them in the same
network(s) as the facility(ies).

This way, if a fibre goes down then you still have a director alive.

All that said - you say that the problem you experienced was:

"Last week, there was a failure in the fiber line connecting my
directors to the internet and the whole cluster went down as a

Why not have multiple connections to your directors from your redundant

You may have to give some more detail about your network topology to get
much further (noting that none of it's to do with LVS at all, it's just
good network design).


<Prev in Thread] Current Thread [Next in Thread>