At any rate, if you were to ask a group of typical digital photographers what they do to preserve their irreplaceable digital files, you’d get a smorgasbord of different answers like:
- Keeping multiple copies of their files on external hard drives.
- Backing up their files to CD/DVD/Blu-Ray disks (or other media).
- Using a commercial product like a Drobo, Apple Time Capsule, Windows Home Server, Thecus, etc.
- Using cloud storage to back up their data to a remote datacenter somewhere.
...and my personal favorite from the more tech savvy individuals:
- A RAID(1/5/6/etc) array (either software or hardware based RAID) that is backed up locally and to the “cloud”, along with some combination of the above.
To these people I usually ask this one simple question:
“How do you know your data is actually intact and readable on those devices?”
The answers usually range from a blank stare to in some instances anger that I’m even questioning their practices. I’m not asking the question to be a jerk; I’m asking it because it’s actually a very valid question that very few people ever think about.
Those of you reading this are probably saying to yourself “ok smart guy what is the answer is then?”. The answer is:
“Chances are that unless you’re using a file system that’s specifically designed to prevent and correct file corruption, you actually DO NOT KNOW. If you’re using a traditional storage system you potentially won’t even find out until it’s too late.”
I'm going to avoid talking about specifics here because it gets very technical, so instead I'm going to give a very high level overview of it all.
The root of the problem isn’t necessarily the methods used to back up their data/photos (although obviously some methods are much better than others). Rather it’s what’s going on behind the scenes that people are using to store their precious data that is the problem.
Currently no affordable main stream RAID and backup system has in-built data integrity checks at the file system level (although that’s about to change. Read on and I’ll explain). Windows Server (all variants), Apple Time Machine, Drobo, etc. None of them. While all of these products are great in that they protect you against accidental deletions and against (depending on the product) hardware component failure, they don’t necessarily protect you against the data going bad on the media itself. That's because they can't! The problem is at a different level than what these products are designed to service.
Wait? Data can go bad? You better believe it can! Your data can silently go south for the winter from a very large pool of variables and there’s a good chance you may not even notice it until it’s too late. Variables like:
- Accidentally cutting power to a system that’s in the middle of saving a file.
- A component or cable problem that’s not “obviously” failing.
- Power spike.
- The physical media failing (but not generating a S.M.A.R.T alert).
- and as dumb as it may sound, even a stray cosmic ray.
A couple of months ago I almost got into a “religious” debate with another individual about the merits of hardware based RAID controllers. I say almost because I refused to get wrapped up into it. He couldn’t get around the fact that RAID and backup systems are based on two ideologies: REDUNDANCY and HIGH AVAILABILITY. The problem, and how this directly relates to us photographers, is that neither ideology has any safeguards to protect the INTEGRITY of our photos. I’m typing that in all caps because it’s a very important concept that you need to grasp for this to make sense.
Here’s a real-world example that happened to me that hopefully conveys exactly what I mean:
Back in 2007 I did a car calendar photo shoot of some classic muscle cars. Back in those days, even though I knew it wasn’t the best medium to do so, I used to back up all my .cr2 and .psd files to DVD’s. What wasn’t obvious from looking at a directory listing of the disc was that the actual file system of the disc was corrupted. You could call up a directory listing, see the actual file sizes, etc., and from the outside everything look fine. It wasn’t until you actually tried opening those files that it was blatantly obvious that there was something seriously wrong. The first two photos on the disc were perfectly readable, but anything else would cause Windows to lock up, the disk would trash, and eventually everything came to a grinding halt.
Thankfully I was in the habit of burning two copies of everything so I didn’t lose anything, but truth be told, I got lucky in more ways than one. Although the data ended up being readable on that disc, had the problem actually been with the data on the hard drive that was being read to burn that disc, I would have been screwed six ways from Sunday.
In another instance I had a RAID-1 (disk mirror) array that had silently failed, but wasn't reporting any errors initially. Unfortunately for me the disk controller was mirroring data from the bad drive onto the good drive in the process!
So yes, it DOES happen. I've seen it myself.
The reason I almost got into a “religious” debate over all of this was that last year when it came time for me to update my local file storage solution, after a LOT of research into the matter I finally went with something that’s borderline blasphemy: I went with a software based RAID/filesystem NAS/SAN solution that was based on ZFS and used commodity hardware instead of a traditional hardware (or software) based RAID setup!
I went with it because besides being able to use common computer hardware to make a dedicated file server, after reading up on the benefits of ZFS it became pretty clear just how much better it was and I became a believer in it. It turns out I was more correct about that decision than I realized at the time.
Until recently the only file systems that were specifically designed from the ground up to combat the issue were ZFS (and it’s ports to various *nix based systems, Solaris, OpenSolaris, OpenIndiana, FreeBSD, as well as specific distributions like FreeNAS and Nexenta which are FREE), and Btrfs. Today I read that Microsoft has *FINALLY*, after years of relying on NTFS as their file system of choice, has been developing a new file system that for all intensive purposes is a clone of ZFS: ReFS (short for “Resilient File System”).
Silent data corruption is a very real issue that not many people know or even think about until it hits them, and if you’re a Photographer, chances are it’s going to hit you when you least expect it (usually when a client is yelling at you to deliver under a deadline. :D ). If you’re religious in your backup habits there’s a good chance that you may come away unscathed. If you’re one of the unlucky few out there though that end up backing up corrupted data though (because of silent corruption), you’re going to get a rude awakening when you restore your files and discover that those files are just as corrupted as the ones you’re trying to replace!
At that point you either pray your earlier backups are good (you ARE doing incremental backups, right??), or you start looking for a better solution. If you’re a Windows Server user things are about to get a lot better. If you’re not, it’s time to look into storage solutions that use a better file system like ZFS.
Don’t think it can happen to you? Think again. This happening to me is what started me down this whole rabbit hole myself. :D
Here’s more info if you want the info behind all of this. Be warned that it gets really technical at times:
http://arstechnica.com/microsoft/news/2012/01/microsoft-introduces-new-robust-resilient-file-system-for-windows-server-8.ars
http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx
http://en.wikipedia.org/wiki/ZFS
(*the shot above is of one of my muses, and was one of the images on that DVD that I nearly lost. It's actually an outtake shot that I remember taking as a lighting test (and completely screwing up the power settings). After a little post-processing I ended up loving the shot. Go figure.... :D. Shot on-location in an undisclosed industrial park using my older Alien Bees B800 ring flash).
No comments:
Post a Comment