|
|
New Articles:
Southern California Glider Landout Database
Soaring
5-Inch Sky Robot
FDM in Vacuum
Gin
Books
Hacktastic e-bike
Farnsworth Fusor
Land Anchor
Turchickentato
Bowmaking
Custom Discovery Roof Rack
August, 2012 - Sierra Backpacking
July 22nd, 2012 - Hiking in Santa Barbara
July 15th, 2012 - Valley of Fire
May 5th, 2012 - Afton Canyon
April 28th, 2012 - Devils Playground
Alpine Tripod
Convict Creek Trail
January 2012 - Mustangs
January 3rd, 2012 - Heart Lake
August 7th, 2011 - Kelso Dunes
August 5th, 2011 - Lundy Canyon Hike
August 2011, Mammoth Archery
Birds in the garden
June 4th, 2011 - San Gorgonio
May 29th, 2011 - Sequoia National Forest
April 23rd, 2011 - Living Desert
April 2nd, 2011 - Death Valley
March 8th, 2011 - Mountain Palm Springs
February 13th, 2011 - Iron Smelt
Blacksmithing
November 13th, 2010 - Mojave
October, 2010 - Mammoth
Android
September, 2010 - Mammoth
September, 2010 - Duck Lake Trail Backpacking
Red
Iron bloom forging
August 28th, 2010 - Mt. San Jacinto
OSM Import: US Designated Wilderness
July 25th, 2010 - Mojave Mustangs
July 17th, 2010 - Mojave Exploration
Bloomery furnace iron smelting
Open Street Map: Mojave Project
June 13th, 2010 - Mojave
June 6th, 2010 - El Cajon Trails
Wolf Mountain Sanctuary
Carrizo Gorge
March 28th, 2010 - Salton Sea
March 21st, 2010 - South Main Divide
March 13th, 2010 - Anza Borrego Wildflowers
March 7th, 2010 - CSULB Japanese Garden
February 2010 - Mammoth
GeoRSS
AISlib
OpenStreetMap
Dakota and Asha Celebrate Christmas, 2009
November 21st, 2009 - Mojave Road
November 14th, 2009 - Anza Borrego
Exploring The East Mojave: The Afton Canyon Area
Broken flex plate
Remote Image Serving
Astro/night photography in Inyo National Forest
Wild Mustang Sightings
RSS
September 26th, 2009 - Night Photography In Frazier Park
Whiskey
Brandy
August 15th, 2009 - Catalina dive trip
Astrophotography
Sensornet
January 24th, 2009 - Mojave Exploration
July 2008 Mammoth Vacation
AIS
President Barack Obama!
Rachel Maddow
Big Geek
Barack Obama
April 12th, 2008 - Wildflowers and Landmarks
My Grandfather's Alfa Romeo Spider
March 8th, 2008 - Carrizo Plain
Bridge To Nowhere
High Availability
October 20th, 2007 - Big Bear Camping
October 22nd, 2007 - Fire
Scottish Highlands, Aug 7th, 2006
Scottish Highlands, Aug 6th, 2006
August 5th, 2007 - Duck Lake Trail
May 26th, 2007 - Kelso Dunes
Culloden Battlefield, Aug 5th, 2006
May 20th, 2006 - Irwindale Renaissance Faire
Edinburgh, Aug 4th, 2006
The Clifs of Moher, Aug 3rd, 2006
The Burren, Aug 2nd, 2006
Bunratty Castle, Aug 1st, 2006
May 5th, 2007 - Mojave
Truck Audio/Data Network
2007 - Master Bath Remodel
Centrum: Exit
The Ring of Kerry, Jul 31st, 2006
Nikon D200
Victory in 2006!
Blarney and Killarney, Jul 30th, 2006
Dublin and Cork, Jul 29th, 2006
Dublin, Jul 27th & 28th, 2006
Married!
What Can I Do?
April 30th, 2006 - Anza Borrego
New desktop: Intel 805 D
Macro Photography
Jan 7th and 14th, 2006 - Hiking
Whiting Ranch Hiking
Engaged!
Digital Photography with Linux
September 5th, 2005 - Living Desert
August 19th, 2005 - Mammoth
Hiking and Photography
July 30th, 2005 - Tucker Wildlife Sanctuary
Nikon D70
Death, Fright and Photography
Mmmmmm Eggs
MythTV
Inova T4
May 14th, 2005 - Red Rock
April 2nd, 2005 - Death Valley
Count Every Vote Act of 2005
Image Archiving
Linear Logic ScanGuage
Gentoo Linux
November 6, 2004 - Mojave
Super Tuesday, 2004
John Kerry
Kayaking
Irish Stew
ImageServer
Ireland, 2004
Canon A80 Camera
Jul 25, 2004 - Death Valley
Chronic Hiccups
May 4th, 2004
Landscaping - My Front Slope
Stump Pullin' Yeeee Haw!
Feb 22nd, 2004
Feb 16th, 2004
PostgreSQL Logfile Analysis
Spam
Mountains? Desert? Jan 30th, 2004
Jan 28th, 2004
Encryption
Ceiling Cargo Basket
Front Bumper Version 2
Asha
Exide Orbitals
Land Rover Valve Jobs
Spirits
The Matrix: Revolutions
Halloween 2003
Greg Davis CDL Linkage
Ouzo
Democracy
Mom's Turkey Gravy
Grandma's Guacamole
Top Nodes
Julian Pie Company
DeCSS
The KB1DIG 2-meter Halo Antenna
Incomming searches
Gardening is hard!
Aug 13th, 2003
SQL and Perl
Cancun 2003
Jul 9th, 2003
Aprs intelegence
Jun 17th, 2003
Some People's Comments
Dakota is a silly dog
The Matrix: Reloaded
Chris' Stage Bottle Harness
April 23rd, 2003
Cracked Radiator!
Black Wednesday
DVD Burning Under Linux
My Satellite Phone
Wind!
My Near-Death Experience
Laser Cannon Revival
Front Bumper, Version 2
SpamAssassin
The Critters
Dakota
Milton
KPC 3 Plus and HTX-252
My House
Moving, moving moving...
Mobile 1
Portola Hills
New new house
Suse Linux
Database images
In Truck Dr. Pepper
My Favorite Toilet
Kelso Dunes
Desert Trips
Ifulmuh
Late Thoughts: Dr. Pepper Cooler
265/75-R16 Tires on a DII
Linux
George W. Bush
Rants
Driving
Reservations
Horses Sep 14 2002
Obsession
August 17th Yukon Dives
Less notifications
My Custom Front Bumper
Bracketless, Renamable Links
Discreet Winch
Welding
Jul 28 2002 Day Trip
My Firewall
Jedi Group, my T1 and money
A Bumperless Discovery!
My Custom Rear Bumper
Vanessa's 24th Birthday
Jun 30th Dive to Long Beach Canyon
PHP/PostgreSQL String Quoting
Tonsillectomy, Uvulaectomy and Turbinite Reduction
Searching functionality
240 Watt CO2 Laser Cannon
My Workspace
Dr. Pepper
The Tulsa Rib Company
The quality of hard disks these days
Email notification of articles
Email notification of comments
erikburrows.com source code
User Bios
User Preferences
Login feature
Renisance Faire Jun 9, 2002
Computers hate me, and it is mutual.
Star Wars Sucks!
Horses, Jun 1, 2002
Land Rover Mileage
Insomnia, Robin goes evil.
100 Watt Diode Laser Test Firing 1
Amateur Radio
The Matrix
2001: A Space Odyssey
TDI Deco Class
Horses, Apr 30, 2002
APRS
Movies
Blackbird
My Truck Batteries
My Truck
Vasquez Rocks
The Zope Bible
PSK31
Mojave Apr-12-2002
100 Watt Diode Laser
The New www.erikburrows.com
Hunter the Kitty
Horses
Geeks
Yukon May-13-2001
Computers
Matts Desert Pics
Mojave Feb-10-2001
Mojave Apr-01-2001
Programming Languages
Ironage Jul-4-2001
Mojave Jan-27-2001
Ironage Feb-03-2001
Mojave Jun-09-2001
SCUBA
Jedi Group
|
|
 |
|
| High Availability - 2007/11/15 | Viewed 76 times this month, last update: 2007/11/15
 |
| Ever since I worked for the now-defunct Techfuel, a main focus of my work has been toward ensuring reliable access to something. A web site, a database, a messaging system, data itself. The concept of designing computer systems to deliver a service reliably is called High Availability (HA). It is important to my work, and it is an interesting and fun concept for me to just play with.
One of my first exposures to HA, at Techfuel, was the "rack" product we were creating, which my friend Chris Bell was designing the hardware for. His design ensured that at worst, any one piece of equipment in "the rack" could fail, without disrupting the web site it was hosting. Other developers were working on the web server software, which itself had HA in mind by being able to run many instances in parallel. As long as the load-balancers were working, all but one of the web servers could fail, and it'd still keep chugging along.
My involvement in that project was the internal monitoring application. I designed that application to run as a single instance, but in a distributed, modular and redundant configuration, so that pieces of the network could fail, or detach, and the monitoring software would continue to function, even if it was split right in half! That application was released to me after Techfuel folded, and is now available here, on the [Argus] page.
I'm still working with HA designs at work, but why limit the fun to work? When my old server centrum died, I replaced it with two servers, "one" and "two". Over the last several months, I've slowly been working to configure them as an HA cluster to serve up the services I run: web, mail, LDAP, PostgreSQL and DNS. I've finally got it almost done, and it works pretty well!
One is the primary for all services. Two runs bind for DNS and is a slave for the zones one serves, as well as a couple external zones. Two is also a slave for LDAP, using OpenLDAP's slurpd process. OpenLDAP has a nifty write redirection feature, so both one and two are active for LDAP service. Two is also a slave for my PostgreSQL database, which runs this site. I'm using PostgreSQL's log-shipping replication system, so PostgreSQL on two is sitting in a half-up state most of the time, just reading in work logs from one's Postgres. The Balance program is running on both one and two, managing postgresql connections. If one is up, it'll get the postgresql traffic, but if it goes down, connections will go to two. Both one and two serve all web sites, and both run Postfix and Courier IMAP for mail service, but the www and mail names resolve to an IP addreess managed by the Heartbeat program, so while one is healthy, it gets all the web and IMAP traffic. When heartbeat detects that one has failed, it'll steal the IP address for web and mail, then execute a program I'm working on which will swap some sym-links to bring mail delivery local for two, wake up the local PostgreSQL, and physically turn off one by sending a command to a smart power-strip. (An HA concept called shoot-the-other-node-in-the-head!) It's neat stuff to me, and fun to play with, not to mention useful!
|
Comments:
Steve K (2007-11-21): Great stuff Erik! I need to read up on postgres' replication offerings...
Erik (2007-11-21): There are a few. PG 8.2 (maybe 8.1) supports log-shipping, which is what I'm using. DDL and DML statements get propogated, which is nice. Slony-I is asynchronous, but the slaves can be used for read-only work. Cybercluster and Sequoia are master-master replication systems, but both are a little wonky seeming, for now. I've used both log-shipping and Slony-I in full-production, and both work well, but for different purposes. Log shipping is great for warm-standby, and is easy to setup, but the slave is not accessible at all until activated. Slony-I takes more work to get setup, and manage (DML statements are not transparently replicated), but allows parts of a database to be replicated, etc.
permalink
|
|
