DNS A Record Change by Tracking Servers

Here’s the scenario:  You have a main site(MS), and a DR site(DR).   You have addressing 1.1.1.0/24 at the MS and addressing 2.2.2.0/24 at the DR site.  You have several web-servers at the MS that replicate regularly over to the DR site.  You want to fail over to the DR servers in the event of a meltdown at the MS.

1. You could peer via BGP at the DR site and use prepending or conditional BGP to advertise the 1.1.1.0 subnet.  The customer says this isn’t an option.
2. You could use a load-balancer such as an F5, Barracuda or a Cisco LocalDirector.  This in undesirable because the DR site has higher bandwidth costs so the numbers won’t add up.
3. You could set the TTL on your DNS records to 60 seconds and in the event of a failure, change the A records over to your DR subnet.  This would require human intervention, though it would work perfectly well.
4. Distributed Director.  Wes, a friend of mine turned me onto this.  Cisco put out a small IOS app a good while back called Distributed Director.  What this allows you to do is be authoritative for a DNS zone or record.  You in essence turn your router into a DNS server.  Why would you want to do this?  Because it allows you to add multiple servers to the pool for a single DNS record.  You can set the percentage of requests to go to one server or another.  You can also do tracking of a sort.  You can say, attempt to establish a connection to port 80 on a server to ensure the web server is running.  You can also give it a url and see if it responds.  This is a a pretty solid method, but the client doesn’t want to configure and maintain equipment.
5. You could use a tracked DNS service.  A tracked DNS service is one that uses UDP, TCP, ICMP,  HTTP individually or in combination to track server status.  For example:  at my MS I track my webserver by issuing an HTTP request every two minutes.  I parse this HTTP request looking for specific keywords like “Company X”.  If I find those key words I do nothing.  If my server doesn’t respond properly, I start checking a list of alternate servers, namely my DR site server.  My DR server responds properly, so I change the A record over to the DR IP.  Within 60 seconds, all requests will start pointing over to my DR site, and I didn’t have to do anything, it was all automatic.

As you can tell, The 5th option was the suggestion to the customer.  The feature seems like it would be a popular one, though there only seem to be a handful of providers out there.  Two in particular seemed interesting: dnsmadeeasy and ultradns.

DNSMadeEasyis the cheaper of the two.  They offer a business package that includes 25 domains and 3 free “trackers.”  Tracker is the service that monitors the servers for failover.  You can add additional trackers for $5 and additional domains for $.75.

UltraDNSis the pricey option, but for a reason.  They have far more bells and whistles: load-balancing by region, better server density throughout the world.  These are the guys that do DNS for facebook and amazon.

I’m thinking of working on an Open-Source version, and here it is!