LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Failover Between 2 Datacenters

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Failover Between 2 Datacenters
From: ken price <kprice75@xxxxxxxxx>
Date: Fri, 2 May 2003 07:39:03 -0700 (PDT)
> Consider this however:
> DNS is a distributed DB essentially, but all mods
> are propagated from 
> the master. Secondaries are available if a master
> fails, but not for 
> zone  transfers. This master DNS server still
> represents a single 
> point of failure - should your master fall from the
> map 
> (datacentre/network outage), how will you originate
> your zone change 
> ? In this instance you will require the cooperation
> of your registrar 
> to change the IP of your primary DNS server.

The registrar wouldn't need to be involved.  I'd
actually configure both of them as masters, relying on
database replication (MySQL 4.x) to propagate changes
(Obviously I'll be using a DB backend instead of zone
files).  This makes it easier to script the monitoring
tool to change the DNS records to point to the new VIP
- especially when you're dealing with approximately
300 URLs as I am.


> I acknowledge the merit of this discussion and the
> potential need for 
> failover between datacentres, however it seem that
> part of the 
> problem you seek to mitigate are issues that should
> be covered by 
> SLAs with your providers. In this paradigm we must
> accept that there 
> will always be variables beyond our control - script
> kiddies could 
> DOS root nameservers, a long haul carrier could go
> bust, a worm could 
> generate massive outages.

True.  It would be irresponsible not to have SLAs in
place with your datacenter provider.  However, SLAs
only mean you get compensated should they not meet the
terms.  It doesn't *actually* guarantee uptime or
protect you from *your* SLAs with your clients.  If
you're paying $10k in datacenter costs which get
credited back to you, but still have to pay out
credits on 10+ clients paying you $4-20k/month for
services, you're in trouble.  I have a tier1,
multi-homed, carrier class datacenter with whom I've
really never had a problem, but that doesn't fully
protect my company.  For a relatively small amount of
effort and cost, I can failover to a low-level
secondary datacenter and save myself hundreds of
thousands of dollars in lost revenue.  In this case,
as I'm sure it is for anyone looking for this kind of
redundancy, it's all about controlling realized and
potential costs - regardless of where they come from. 
It's not solely a technical matter.
 
> Your provider has an AS assigned to it; any IP range
> within this they 
> should be able to route over their backbone
> infrastructure nation 
> wide as they please. It would seem that  some of
> these issues could 
> be dealt with by hosting in two separate (regional)
> datacentres with 
> the same provider with the agreement that under such
> a failure 
> condition, that they can reallocate and reroute your
> IP block to the 
> alternative datacentre. This is entirely feasible
> for them to do 
> provided the will is there.

You are dead on and absolutely right.  Mine, for
example, does this under the service offering of
"Global Load Balancing", but the service isn't cheap. 
It would probably cost me less to build a secondary
datacenter cluster with a tier2 provider instead - or
even host it out of our office!

-Ken

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com
<Prev in Thread] Current Thread [Next in Thread>