I am curious on how everyone archives their work. Whether it be photos, movies, raw footage, music, files, basically anything. Do you digitize everything and keep them on multiple harddrives/RAID arrays? Once you have a digital copy of something do you rewrite or trash the physical? Do you trust cloud storage?
Once you have it in multiple safe places how do you future proof it? Physical media has a life span whether it be drives that fail, tape that degrades, servers that shut down.
Archival has always been something that is important to me and I feel a little overwhelmed with the amount of stuff I have. One of my biggest fears is losing my memory and I guess that fuels an obsession about saving everything no matter how trivial. The amount of raw footage that I am sure that I will never come back to after a decade still sits on drives in closets.
The baseline for digital archiving is three copies on at least two different types of media stored in at least two different physical locations.
I can’t afford to do that so I just have everything on hard external hard drives.
I don’t know what the online version is like but I took the in-person version of this course a few years ago and it was pretty good, although once it got to VHS and newer it was a bit thin:
I found this organization in NYC called XFR Collective that provides archival services for at-risk and marginalized work. They have a very excellent resources page with a lot of information about different formats and how to preserve. Maybe some of these are worth storing on Scanlines as well.
drive 1 - which i always have plugged into my computer, and generally work from
drive 2 - backup drive which i mirror drive 1 to monthly.
so i figure that if something goes wrong on my main drive (which will probably die first due to getting more use), i generally wouldn’t lose more than 1 month’s worth of work. additionally, i have a “delete after next backup” folder on my computer’s local drive to store newly updated files until they are safely redundant on both externals.
it’s not a perfect system - both drives are kept at my house, so in case of a fire or something it would be more ideal to keep them at multiple locations. both drives are spinning hard drives rather than SSD, as I can’t afford SSDs with the amount of space i’d need. if i cared more about ensuring my data lasts as long as possible, i’d probably purchase newer hard drives more frequently and have more redundancy across multiple locations. but my personal view is kind of like, everything is impermanent on the scale of cosmic time, and i try to practice a certain amount of non-attachment (a little off topic, but that’s why the simple two drive system is sufficient for me).
to me, the more you are trying to preseve the data, the more redundancy would be recommended. cloud storage (or as i like to call it, “other people’s servers”) presents two issues:
you need to place your trust in the provider, to keep your data safe from both data loss and security breaches.
where internet bandwidth is limited, it can take a long time to upload or download backups over the network.
however, if one already has redundancy across local hard drives, cloud storage could be useful as an additional redundant storage.
This is a big issue now that I have over 4 tb of video systhesis ProRes 422 recordings and counting. I’m probably going to buy a deep storage raid 5 san or something eventually.
Personally I don’t trust a single external or internal HDD as I have lost three in the past due to not booting them up often enough or just dying after a long time. They need to be spun up every 6 months to maintain functionality in the long run, which can be a hard maintenance schedule to keep for external drives if you don’t set reminders on a calendar.
When a drive with essential data dies, I use drivesavers, which has cost $2000-$2800 per drive I saved.
Since the third time that happened, I’ve been keeping my work on local Raid 0 mirrored drives. And saving finished pieces to to Google Drive or iCloud for backup archives. This gets me two local copies on one raid, where if one drive dies, I can rebuild a new drive with the mirrored data, and a third copy in cloud services.
There is a cost to all this more than just a simple external HDD, but it works out so much cheaper than data recovery services that abandoned HDD backups can require.
I like the idea of RAID instead of the multiple drives I have that just accumulate and I can never remember which is the one that has that certain file on it.
I have seen these DROBO raid solutions before and they are a bit pricey but what I seem to think is that these have software tied to them that automatically does multiple drive redundancy on the same piece of hardware. It also has the benefit of notifying you when a drive is on its way out and will copy everything for you when the drive gets swapped.
Although, I am not sure if this product does anything different than a cheaper RAID solution that you can build yourself.
After a couple failed drives long ago, I too started using redundant drives. I also switched from the typical off-the-shelf brands of drives to ones from Other World Computing, which I have found to be quite reliable. I also like drive enclosures constructed such that it is easy to remove/replace the drive inside.
Something to keep in mind with RAID-5 solutions is that if a local event happens (power spike, overheating, rapid unexpected reboots, godzilla attack) that you’re likely to lose more than one drive at once, and be unable to rebuild. That isn’t to say that the redundancy isn’t valuable, but it should really be treated as a better hard drive than a full source of redundancy on it’s own.
Also, make sure to buy at least one cold spare drive with any RAID-5 solution. The drive market can be pretty volatile, and nothing sucks more than trying to find the same make of drive a few years later.
SOHO raid arrays like Synologies and DROBOs can fail in all sorts of interesting ways, especially if you mess with the software too much. Install weird packages and addons with caution!
Definitly going to be upgrading to a RAID solution, probably not DROBO or similar because of cost and it doesn’t really do much more than just buying separate drives and an enclosure.
This was something interesting I found, it is a briefing from the Library of Congress outlining their National Audio Visual Conservation Center and how they archive. Back in 2015 they had 6.1 Petabytes of archived work and was planning on growing by 300 Terabytes a week!
I back everything on a desktop computer that is connected to Backblaze. I don’t do the multiple hard drive rule. It gets too complicated if dealing with large amounts of data (I have about 20 terabytes of footage) plus hard drives die all the time. One dies last month!
Backblaze is about $5 a month if you pay for a 2 year plan. It’s easy to use. It’s unlimited. Only down side is that it’s NOT cold storage. The computer connected to Backblaze needs to be connected to the internet at all times in order for their server to continue storing your files, hence why I just back everything to a desktop that’s always on. I have a RAID too but unfortunately Backblaze doesn’t have a client for it so I only store things I don’t really care about like torrented tv shows.
