The Raven Group
Infrastructure
Infrastructure

Backups that actually restore

May 31, 20253 min read

There's a quiet failure mode that gets companies in trouble more than ransomware does: backups that ran for years and were never tested. The IT lead set up automated backups in 2019. They've run every night. Nobody's ever needed to restore one. Then the production database corrupts itself in March, the team goes to restore from yesterday's backup, and the backup is — and we've seen this exact thing — a 4-byte file. The script broke 11 months ago, the daily success email kept arriving because the script considered "I tried" a success, and now the company has lost a year of records.

Backups are not a thing you set up. Backups are a thing you continuously prove are working. The practice that prevents the disaster we just described isn't more sophisticated backup software. It's a quarterly restore drill — a calendar invite, an hour, a designated person, and a written procedure for restoring something (anything, really, but something realistic) from the most recent backup. If the restore works, great. If it doesn't, you've discovered the failure on a Thursday afternoon at your leisure, not at 3 AM after a ransomware encounter.

The 3-2-1 rule still holds for the storage strategy itself: three copies of the data, on two different media, with one off-site. Cloud-based backups make this easier than it used to be — your nightly backup goes to a different geography and a different vendor automatically. The complication, increasingly, isn't where to put the bytes; it's making sure all the right bytes are in the backup. Databases get backed up. The configuration files on the production server often don't. The encryption keys you'll need to decrypt the backups often live in the same place as the backups, which defeats the point.

A practical baseline: every system that holds data the business depends on should have an automated backup; that backup should be encrypted and stored somewhere your production credentials can't reach; somebody should restore from one backup, somewhere, every quarter, and write down whether it worked. The whole thing is unglamorous. It's also the difference between "we had an incident" and "we had a six-figure event with permanent customer loss." The companies that have backed their car out of the garage and seen it isn't smoking know which way they prefer.

Want to talk about something in this post? Get in touch.More on Infrastructure
More on Infrastructure