We photographers have a serious problem. We’re generating hundreds and hundreds of gigabytes of data (even more now that our DSLRs shoot video), data that represents hours of hard work, and yet it’s incredibly easy for us to lose our Aperture and Lightroom libraries if a drive goes bad. There are ways we can setup our main workstations to limit our potential data loss, but we really need good, offsite backup so that we don’t lose everything in case of fire or theft. However, most offsite backup services aren’t aimed at people with terabytes of data. After a lot of investigating and trial and error, I think I’ve finally found a great, offsite backup solution with Amazon S3.
First, let me tell you about my home setup. My Aperture library is on a Drobo S connected via eSATA (about 600GB), and I have a single, non-RAIDed drive inside my Mac Pro that contains my iTunes library (about 200GB, and since I stopped buying CDs years ago, I’d rather not lose my iTunes files either). I have another Drobo (an original one) connected via FireWire 800 that I use for Time Machine. Using a Drobo with Time Machine can be a bit painful, but my friend Erik has posted an excellent blog post about how to make it work. I’m quite happy with this setup, as I can have multiple drives fail without losing any data.
Over the past year, I’ve switched to using Dropbox for my Documents folder (with encrypted disk images for specific parts), which lets me keep my files in sync between my laptop and desktop without a thought as well as automatically backing them up offsite. For my older documents, I have an Archived Documents folder on my Drobo S.
For a long time, I tried keeping another drive at my office, which I’d periodically bring home and sync with my Aperture library. Unfortunately, I found that I was very lazy about actually bringing this drive home, and even if I were better, given the bay area is susceptible to a wide-scale event (earthquakes), I was worried that I still could lose data if there were a big earthquake. What I’ve been trying to find is an easy-to-use offsite backup solution.
For the past couple of years, I’ve used PhotoShelter both to drive my website and to host my RAW files (I upload large JPEGs for the public galleries but also have a RAW folder with my originals in my archive). I’ve been pretty happy with that, but PhotoShelter doesn’t let me upload my Aperture database (so I would lose all of my organization), doesn’t support backing up generic files (like my iTunes library), and isn’t very cheap. I’ve been spending about $50/month to backup 200GB of files with them, meaning that I had a lot of data (such as every video I’ve shot) not being backed up remotely.
Online backup is becoming more common, but I’ve heard mixed things about actually retrieving files from Mozy, Carbonite, and others. Plus I have about 1TB of data to backup, and uploading that initial backup over my DSL modem would take about a year. None of those services (as of this writing) offered a way to send them a drive to seed the initial backup. I’m also not sure how long each of those services will be around–what happens if/when they go out of business?
Enter Amazon S3. With S3, I create a bucket (sort of like a remote drive) that I can put whatever files I want into. I pay per GB uploaded/downloaded and a small fee each month per GB stored with them (it costs me about $100/mo to store a bit under 1TB of data, which is about on par with what PhotoShelter charges yet far more flexible). But the best thing is that you can send Amazon a drive with your files to seed your bucket so that you only have to wait about a week for all your files to be backed up offsite. That service is called AWS Import. You can also send them a drive to download your bucket onto to restore your files, rather than waiting for them all to download.
I will say that I’m finding syncing/updating S3 to be somewhat slow. I was hoping that I could use a product like ExpanDrive, which makes an S3 bucket appear like a drive in the Finder, and rsync to update my files, but they don’t play nicely. Panic’s Transmit 4 seems to be the best solution, as I can login to S3, select two folders, and press Sync.
I was really excited to to find this program, Arq, which the developer wrote to specifically let him do remote backup from a Mac with S3, but unfortunately it’s not compatible with backups that you’ve seeded with AWS Import. The flip side is that if you backup with Arq, it preserves your folder structure (unlike JungleDisk) and encrypts your files. If you have a fast connection (*cough*FiOS customers) and less data to backup, I’d highly recommend using this tool to manage your S3 backups.
We’ll see what I think a year from now, but for now, I’m pretty happy with this solution. Dropbox keeps my important documents at my fingertips, and S3 stores my huge photo library, iTunes library, and document archive safely (I think Amazon claims 99.999999% reliability) at a reasonable cost ($100/mo or so).
Subscribe to the Feed