Backup and snapshot

Date: 2021/11/02 (initial publish), 2024/03/26 (last update)

Source: en/

Previous Post Top Next Post


Backup and snapshot

For backup, data needs to be moved to physically separate device. rsync ... is my choice to do this.

For snapshot, data can stay on the same device . btrfs subvolume snapshot ... is my choice to do this.

I created bss script to help me do these easily and its examples contain key parts of the settings.

This bss is designed for flexibility with choice of many options, commands and arguments.

Please also see “Automatic USB backup”.

Snapshot (manual)

Just execute following in btrfs:

 $ bss snapshot

Setup snapshot (batch: updated 2024-01-02)

I set up 3 configuration files.

# ---------------- bss batch frequent start >>>>>>>>>>>>>>>>>>>>>>>>>>>
# use "hour" for BSS_TYPE (snapshot extension)
!export BSS_TYPE=hour
# make new snapshots with "btrfs subvolume snapshot"
-snapshot ~/
-gather   ~/Documents
-snapshot ~/Documents
-snapshot ~/github
-snapshot ~/salsa
-snapshot ~/tmp
# prune redundant snapshots
-process ~/
-process ~/Documents
-process ~/github
-process ~/salsa
-process ~/tmp
# ---------------- bss batch frequent end   <<<<<<<<<<<<<<<<<<<<<<<<<<<
Description=Run bss commands frequently
# journalctl -a -b -t bss

# make snapshot of user's home directory ~/ (in one FS)
ExecStart=bss batch frequent
# No logging (use systemd logging)
# activate by: systemctl --user enable bss-snap.timer
Description=Run bss commands hourly



Then I activate timer with:

 $ systemctl --user enable bss-snap.timer

I can see state of running bss with bss jobs. Its output is as follows:

 $ systemctl --system --all list-timers 'bss-*'

0 timers listed.

 $ systemctl --user --all list-timers 'bss-*'
NEXT                        LEFT      LAST                        PASSED   UNIT           ACTIVATES
Mon 2024-01-01 11:57:11 JST 6min left Mon 2024-01-01 11:42:07 JST 8min ago bss-snap.timer bss-snap.service

1 timers listed.

# See journal with "journalctl -a -b -t bss" or "journalctl -f -t bss"

Setup backup to plug-in USB storage

I have a USB Serial-ATA HDD/SSD external case with USB connector. I put an SSD in it. I configured this with the MBR partition table holding a single Btrfs partition with GUI operations using the gparted package. This Btrfs was labeled as BKUP_USB.

Then, this USB device is used for a single command backup with the bu command which wraps bss.

This bu command is designed:

The essence of the bu setting is listing volume label BKUP_USB with btrfs subvolume directory name Documents as:

backup BKUP_USB Documents

When bu is executed while USB drive volume with label BKUP_USB is connected, bu backs up the Documents directory which is a btrfs subvolume.

Setup backup to remote system

NOTE: This section needs to be updated.

Remote backup with rsync can be secured by storing data in encrypted format.

Here are the basic tricks used in the bss package offering bss and luksimg.

In order to securely backup private data using non-secure remote storage, data needs to be encrypted. Roughly, the following is an approach:

This can be done using following tricks.

Create and format an encrypted filesystem in a disk image

$ dd bs=1 count=0 if=/dev/zero of=disk.img seek=7000M
$ mkdir disk
$ cryptsetup luksFormat disk.img
$ sudo cryptsetup open disk.img disk --type luks
Enter passphrase for disk.img: *****
$ ls -l /dev/mapper
total 0
crw------- 1 root root 10, 236 Nov  3 07:45 control
lrwxrwxrwx 1 root root       7 Nov  3 12:04 disk -> ../dm-0
$ sudo mkfs.btrfs /dev/mapper/disk
   ID        SIZE  PATH
    1     6.82GiB  /dev/mapper/disk

$ sudo mount /dev/mapper/disk /mnt
$ sudo chown 1000:1000 /mnt
$ sudo umount /mnt
$ cryptsetup close disk

Mount and use an encrypted filesystem in a disk image

$ mkdir -p ~/path/to/mnt
$ sudo cryptsetup open disk.img disk --type luks
Enter passphrase for disk.img: *****
$ sudo mount /dev/mapper/disk ~/path/to/mnt
... (use files in ~/path/to/mnt as a user)
$ sudo umount /dev/mapper/disk
$ sudo cryptsetup close disk

In order to skip passphrase hassle, let’s use Gnome keyring.

Let me store my_pass_phrase_value in Gnome keyring.

$ secret-tool store --Label='LUKS passowrd for disk.img' LUKS disk_img

Then, use Gnome keyring to decrypt luks device.

$ secret-tool lookup LUKS disk_img | \
  cryptsetup open disk.img disk --type luks --key-file -

This avoids storing key file data in plain text.

For ~/path/to/mnt, use~/Document. For ~/.gnupg, ~/.ssh, bind mount may be an idea.

Hints for LUKS and its auto-unlocking on the web

Previous Post Top Next Post