Improbable Drive Problems

The biggest problem with digital photography is the sheer volume of data it produces. A typical shoot will generate nearly 4gb of raw files. Some of these get converted to psd for photoshop and most of the time it is easier to keep all the files from a shoot rather than cull the bad ones. You simply never know when you might decide to resurrect a shot you thought was a bit iffy.

In just over two years I've amassed a photo collection of 21,000 files covering 216GB of data. Now obviously there is no real way to create a hard backup of these files. That would be 48 DVD's. It would be 9 BluRay disks which would be an option if I had a BluRay drive. But just writing them to a DVD or BluRay is not enough because these disks often degrade over time, so periodically you need to renew them.

I decided my solution was to use a known method from normal computer data handling. I have the main data on a working PC, then a backup elsewhere. In this case the backup is a purpose build file server. It has 1.3TB of drives arranged as 2 pairs of mirrored drives. So that's 4 drives, 2 of which keep a copy of the data from the other 2. So that's 3 copies of the data, the main PC, the primary drive on the backup and its mirror. To further add protection, the file server has a triple redundant power supply connected to a industrial uninterruptable power supply and a backup generator.

Pretty neat huh? I felt very safe with this setup. Until Thursday morning.

I got up, had my coffee, and then logged into my PC to find that network access to the file server backups was not available. I went to the server and found that 3 of the 4 drives had failed!

Statistically there is no way that 3 out of 4 drives in any PC can fail at once. I've had single drive failures often and know the drives tend only to last a few years in a file server. But for 3 drives to fail without some form of catastrophic failure of the whole system in unheard of. Yet, the 3rd backup drive and the core drive for the OS were fine and dandy.

So my backups of my photography, 3d work, downloaded tutorials and some other stuff was blown away and were it not for the fact that I still have the main PC with the live copy, I would be suicidal by now.

Today I have installed new drives, recreated the mirrors and started to restore the data to the backup. However, I have installed another drive into my main PC and have an additional backup from its main drive to its second drive before it copies to the redundant backup. I'm also going to install a 3rd tier backup that holds a backup of the backup on another machine.

The worst part is that I also lost our entire MP3 music collection and now I have to rip all 400 albums again.

Now if only I could find some software that could do all this for me automatically and keep historic versions, and run regular CRC checks I would be a truly happy man. If you know of such a thing, please let me know. If not, I might just write one. Any one want to buy a backup system designed for a photographer?



This image is of Lou Lou from last year.