BXG Blog

Backing up Backups

I’ve had what I thought was a fairly good backup system for a while: I use rdiff-backup to synchronize all my computers onto a central machine with a large disk, then I copy that machine’s backups to an encrypted external USB drive. The external drive is only mounted during backups. I rotate through a couple of external drives and I keep the inactive drives in a safe place outside my house. Other than manually transporting a drive back and forth to my outside storage about once a week, all of this is automated. I thought this covered all the normal risks pretty well:

  1. I keep roughly 30 daily rdiff-backup increments, so individual machines are protected.
  2. The external drives protect against the backup machine failing.
  3. Keeping drives outside the house protects against fires, theft, or other major disasters that might take out the main backup computer and its external drive at the same time.
  4. Encrypting the external drives protects against the non-financial risks of me losing one when I’m carrying it back and forth.
  5. The external drives are normally not mounted except during a backup, which hopefully provides some protection against a potential ransomware infection.

All of that has been working great for a while. I’ve even tested restoring files a few times, so I know files are generally backed up correctly. After getting hit by a bcache bug during a Fedora 30 upgrade, I realized there was one major hole left: the backup scripts and keys needed to decrypt the external drives! Luckily, I was able to recover enough of the internal drive to decrypt my backups.

I have now made a separate backup of the backups user onto an encrypted DVD. This DVD can be decrypted and mounted in Cinnamon with no extra software (as long as I remember the password), so I’ll just keep a copy in my storage locker with the offsite external drives and hopefully not have any holes left.

Here is how I created the DVD:

# Make the image fill the entire DVD so people can't tell how
# much it contains by looking at the burned area.
truncate -s 4700000000 dvd.img

# Encrypt the image and create a mapping at /dev/mapper/dvdbackup
sudo cryptsetup luksFormat --cipher aes-xts-plain64 dvd.img
sudo cryptsetup luksOpen dvd.img dvdbackup

# DVDs always have 2048 byte sectors.  We need to make the filesystem
# block size match in order to read it back from the disc later.
sudo mkudffs -b 2048 /dev/mapper/dvdbackup

sudo mount /dev/mapper/dvdbackup /mnt/backups
# Copy files to /mnt/backups
sudo umount /dev/mapper/dvdbackup
sudo cryptsetup luksClose /dev/mapper/dvdbackup

# Burn to a blank DVD
growisofs -dvd-compat -Z /dev/sr0=dvd.img

Once this is done, inserting the DVD pops up the normal encrypted disk dialog in Cinnamon. Entering the password mounts it like normal, and then I verified that all the files match the originals. Since the backup scripts and keys only change occasionally, I can keep this as a manual process. I have put a reminder on my calendar to re-check and/or re-create the DVDs twice a year just to make sure they don’t degrade in storage.