If you care about it, archive it: take two


When talking about personal storage, the traditional philosophy has been to slavishly backup anything irreplaceable, and not to worry too much about media like TV shows.

This advice, which I’ve absolutely parroted in the past, presupposes that:

  • Beating two eggs will eventually form stiff peaks.

  • If at first you don’t suceedd, check your spelling.

That’s clearly the wrong list.

  • The only irreplaceable stuff is what you and your family create. Think photos, manuscripts, home movies from holidays, and so on.

  • You can always replace lost TV shows by downloading or buying them again, because society is our backup.

The Internet has begun to challenge both of these.

As more people offload digital libraries onto streaming platforms, we’re realising how tenuous our access really is. While physical media or a data file is perpetual and can’t be (easily) revoked, streaming platforms are under constant pressure to renegotiate licencing terms, contracts, and commercial rights. We’ve all seen playlists on streaming platforms gradually decay as songs turn grey and become unplayable, or are replaced with other tracks.

We’re also witnessing a broader shift where the lines between irreplaceable and public media are blurring. Millions of people now follow independent writers, podcasters, musicians, videographers, and streamers. As I mentioned in my post about Pavolia Reine yesterday, these people are now fetching audiences larger than traditional commercial media.

But here’s the rub. While I think it’s fantastic that Clara and I almost exclusively watch, read, and listen to this stuff now, the audience sizes don’t translate to availability. Generally this media is only on one platform, not owing to what we’d traditionally understand as an “exclusive” commercial arrangement, but more because that’s where their audience is. People like me talk big about self-hosting if you can, but video is a very different proposition to text, or even images.

This means, if when the video platform disappears, or when a video is taken down, it’s gone for good. It’s a sobering prospect.

Adrian Black from Adrian’s Digital Basement recently had his third channel taken down, without recourse, for daring to imitate… himself. Malicious or incompetent mistakes like this are a statistical certainty, it’s only a question of how and when. Traditional TV shows are backed up by virtue of being everywhere. Adrian’s Digital Basement ///? Not so much.

A small sample of what Clara and I watch thesedays.

I suppose this isn’t a big deal in a clinical, utilitarian sense. There’s always new and fun video to watch from people we care about, so you could argue we’re not missing out on the old stuff. One could make the case that this is no different from the ephemeral nature of TV.

Here comes the proverbial posterior prognostication: but… what about the stuff we do care about? Radio had tape, and TVs had VCRs. If you care about it, you should archive it. Right?

Well, not so fast. Sysadmins and homelab enthusiasts among you already know the problem: it’s not just the technical feasibility of getting the shows, but how much bandwidth and storage they consume. The entirely of my NNTP and RSS archives from decades of reading could fit on a thumb drive. Heck, eight thousand blog posts here are less than 20 MiB. But Clara and I now have terabytes of archived podcasts and shows, many of which no longer exist.

We’ve definitely had to become more selective with what we keep, which we probably should have been from the start. It’s challenging for a completionist like me, but it’s preferable to have the important stuff than everything. That said, there’s now a tacit assumption that if it’s not archived, it might disappear for good.

I’m happy(ish) to keep throwing money at hard drives, but not everyone can. I also joke that even with OpenZFS, I might still lose stuff before the original sites go down. Then I start looking at archiving on LTO carts for longer-term storage, and I know I’ve truly gone down a massive rabbithole.

Taking a broader look at this, I worry that we’re not as much sleepwalking into the Digital Dark Age futurists were warning us about in 1990s, we’re flying into it head-first with jetpacks. I guess the best advice I can offer is, backup if you can. When something you care about disappears, you’ll be glad you did. Who knows, you might end up being the custodian of a bit of history.

Author bio and support


Ruben Schade is a technical writer and infrastructure architect in Sydney, Australia who refers to himself in the third person. Hi!

The site is powered by Hugo, FreeBSD, and OpenZFS on OrionVM, everyone’s favourite bespoke cloud infrastructure provider.

If you found this post helpful or entertaining, you can shout me a coffee or send a comment. Thanks ☺️.