Improbable Drive Problems
The biggest problem with digital photography is the sheer volume of data it produces. A typical shoot will generate nearly 4gb of raw files. Some of these get converted to psd for photoshop and most of the time it is easier to keep all the files from a shoot rather than cull the bad ones. You simply never know when you might decide to resurrect a shot you thought was a bit iffy.
In just over two years I've amassed a photo collection of 21,000 files covering 216GB of data. Now obviously there is no real way to create a hard backup of these files. That would be 48 DVD's. It would be 9 BluRay disks which would be an option if I had a BluRay drive. But just writing them to a DVD or BluRay is not enough because these disks often degrade over time, so periodically you need to renew them.
I decided my solution was to use a known method from normal computer data handling. I have the main data on a working PC, then a backup elsewhere. In this case the backup is a purpose build file server. It has 1.3TB of drives arranged as 2 pairs of mirrored drives. So that's 4 drives, 2 of which keep a copy of the data from the other 2. So that's 3 copies of the data, the main PC, the primary drive on the backup and its mirror. To further add protection, the file server has a triple redundant power supply connected to a industrial uninterruptable power supply and a backup generator.
Pretty neat huh? I felt very safe with this setup. Until Thursday morning.
I got up, had my coffee, and then logged into my PC to find that network access to the file server backups was not available. I went to the server and found that 3 of the 4 drives had failed!
Statistically there is no way that 3 out of 4 drives in any PC can fail at once. I've had single drive failures often and know the drives tend only to last a few years in a file server. But for 3 drives to fail without some form of catastrophic failure of the whole system in unheard of. Yet, the 3rd backup drive and the core drive for the OS were fine and dandy.
So my backups of my photography, 3d work, downloaded tutorials and some other stuff was blown away and were it not for the fact that I still have the main PC with the live copy, I would be suicidal by now.
Today I have installed new drives, recreated the mirrors and started to restore the data to the backup. However, I have installed another drive into my main PC and have an additional backup from its main drive to its second drive before it copies to the redundant backup. I'm also going to install a 3rd tier backup that holds a backup of the backup on another machine.
The worst part is that I also lost our entire MP3 music collection and now I have to rip all 400 albums again.
Now if only I could find some software that could do all this for me automatically and keep historic versions, and run regular CRC checks I would be a truly happy man. If you know of such a thing, please let me know. If not, I might just write one. Any one want to buy a backup system designed for a photographer?
This image is of Lou Lou from last year.
In just over two years I've amassed a photo collection of 21,000 files covering 216GB of data. Now obviously there is no real way to create a hard backup of these files. That would be 48 DVD's. It would be 9 BluRay disks which would be an option if I had a BluRay drive. But just writing them to a DVD or BluRay is not enough because these disks often degrade over time, so periodically you need to renew them.
I decided my solution was to use a known method from normal computer data handling. I have the main data on a working PC, then a backup elsewhere. In this case the backup is a purpose build file server. It has 1.3TB of drives arranged as 2 pairs of mirrored drives. So that's 4 drives, 2 of which keep a copy of the data from the other 2. So that's 3 copies of the data, the main PC, the primary drive on the backup and its mirror. To further add protection, the file server has a triple redundant power supply connected to a industrial uninterruptable power supply and a backup generator.
Pretty neat huh? I felt very safe with this setup. Until Thursday morning.
I got up, had my coffee, and then logged into my PC to find that network access to the file server backups was not available. I went to the server and found that 3 of the 4 drives had failed!
Statistically there is no way that 3 out of 4 drives in any PC can fail at once. I've had single drive failures often and know the drives tend only to last a few years in a file server. But for 3 drives to fail without some form of catastrophic failure of the whole system in unheard of. Yet, the 3rd backup drive and the core drive for the OS were fine and dandy.
So my backups of my photography, 3d work, downloaded tutorials and some other stuff was blown away and were it not for the fact that I still have the main PC with the live copy, I would be suicidal by now.
Today I have installed new drives, recreated the mirrors and started to restore the data to the backup. However, I have installed another drive into my main PC and have an additional backup from its main drive to its second drive before it copies to the redundant backup. I'm also going to install a 3rd tier backup that holds a backup of the backup on another machine.
The worst part is that I also lost our entire MP3 music collection and now I have to rip all 400 albums again.
Now if only I could find some software that could do all this for me automatically and keep historic versions, and run regular CRC checks I would be a truly happy man. If you know of such a thing, please let me know. If not, I might just write one. Any one want to buy a backup system designed for a photographer?
This image is of Lou Lou from last year.
posted by Richard at 4:46 PM
6 Comments:
Yikes,
That sounds like a server system problem. either a spike got thru the UPS or something went wrong in the server software..Had a UPS go bad on me (the one on my 1 TB server) during a generator test but didn't lose any data. If you had a power & UPS failure during a write, it MIGHT have the effect you described, I'm not a IT type so maybe someone who is can comment.
I'm using the photo storage system Kevin Ames published last year, keeping my main files on the server on-line but 2 complete backup DVD's, one of them in a safe deposit box offsite. I anticipate having to copy, or pay to have someone copy, all the files to new storage technology in a few years (probably before I have any problems with the disks) but am feeling pretty secure right now.
Good luck and my best to Lin.
Does it make you yearn for the olden days of negatives and chromes?
Me neither.
I'd like to help.
But I have a 1kb mind in a multi TB world.
D.L. Wood
Or, one could just use film.
I understand...as I sit here at my studio computer with 5 external drives, plus two in the computer. I have all my photos on at least 2 hard drives, but I don't have any automated system for backups. I really hope to get this better organized this year.
But, I'm also working on what I consider my most important backup. I'm making prints of all my favorite photographs and putting them into archival boxes. One box for each of my children, one for me and my wife, and one for the imaginary photo history grad student who will write their thesis on me 50 years after I die.
"Improbable" only if the drives were independent (in the statistical sense of the word), which they weren't. For all we know it could have been a stray cosmic ray hitting a controller in your machine which caused the corruption.
I too have been struggling with backups, although I have only a quarter of the data that you do. Two machines, though, and thousands of files and versions of said files, and so on.
I've been thinking of how to design a backup methodology for a while now. The way I see it, it needs to be:
- Simple and open. At the most it uses some scripts and some open source tools to manage things.
- Tiered. Not all data is equally important.
- Robust. It should be easy to attain as much reliability as you want.
With that in mind I should mention the following:
- RAID is not a backup.
- The only true backup is WORM: write once, read many. Like DVDs. Once you start overwriting data and changing filesystem structures you no longer have a true backup.
My very rough plan is:
- Dump my new photos onto my main machine, in the "least important" dir.
- Create a SHA1/MD5 for each photo. This will detect corruptions later on.
- Work on the photos and eventually separate them into different dirs, like "least", "moderate", and "very important".
- "very important" holds the smallest amount of data. Maybe a gig.
- "moderate" holds 3.7 GB.
- "least" holds an unlimited amount.
- The trick is to backup the different dirs to different media. Every 5 days "very" is backed up to Amazon S3 (online). Every 15 days "moderate" + "very" is burned to a DVD. Every 30 days "least" + "moderate" + "very" are backed up to an external HD.
- Amazon S3 can hold as many copies of "very" as you want. There's all those DVDs too.
- The external HDs are more complicated. I'd have several HDs and rotate them. First, scan all the files on the HD and verify none are corrupted. Then sync your main computer to the HD.
Yeah, this system sounds complicated, but I think treating all data as equally important is a mistake.
Now I just need to follow my own advice.
Post a Comment
Links to this post:
Create a Link
<< Home