System Backup w/ rsync

I'm going to explain the details of my new backup process using rsync. I'm using Arch, so the process I lay out may vary slightly for other distros.

Rsync creates a bootable clone backup of my main systems root, located on HDD sda, to HDD sdb. Root file system of sda is laid out over several partitions, and gets cloned to a consolidated single partition on sdb . The /home partition on sda is set up on a separate partition from /. I back it up separately from the root backup, to a dedicated /home partition on sdb.

To make the backup system bootable, I edited /etc/fstab in the backup on sdb to reflect the new layout, and set up grub-legacy in my main system to boot it. I also set up grub-legacy to be able to boot from sdb as a standalond HDD if needed. My backup serves two purposes, it's a complete backup / duplicate of my main system, and it serves as a fallback system that I can use to make repairs, etc if I can't boot into my main OS.

At this stage, I'm going to do manual backups once a week or so. The process involves entering 4 commands which could be set up as bash aliases. I may set up a few aliases, something like "backup-root" and "backup-home", to save copy pasting of the long commands. I'm also contemplating automating the process by creating a bash script and possibly using cron or systemd.  Below are the notes and commands I use.





Create directories for mount.

$ sudo mkdir /mnt/root


$ sudo mkdir /mnt/home



 
Mount root

$ sudo mount /dev/sdb6 /mnt/root


Mount home

$ sudo mount /dev/sdb8 /mnt/home


Backup root command.


$ sudo rsync -aAHXv  /  /mnt/root --exclude={/etc/fstab,/home/*,/dev/*,/proc/*,/sys/*,/tmp/*,/var/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}



Backup home command.
   #NOTE: paths for excludes are relative, from /home/.

$ sudo rsync -aAHXv  /home/  /mnt/home --exclude={jeff/.thumbnails,/jeff/.cache/mozilla,jeff/.cache/chromium,jeff/Old-Home,jeff/.local/share/Trash,jeff/.gvfs}  



rsync command layout from man rsync.

SYNOPSIS
        Local:  rsync [OPTION...] SRC... [DEST]



Breakdown of the  -aAHXv  options used in the above commands.


Breakdown of the -a option.

-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)


Breakdown of -rlptgoD


-r, --recursive             recurse into directories
-l, --links                 copy symlinks as symlinks
-p, --perms                 preserve permissions
-t, --times                 preserve modification times
-g, --group                 preserve group
-o, --owner                 preserve owner (super-user only)
-D                          same as --devices --specials


Breakdown of the -D option.


--devices
              This option causes rsync to transfer character and block device files to the remote system to recreate these devices.  This option has  no
              effect if the receiving rsync is not run as the super-user (see also the --super and --fake-super options).

--specials
              This option causes rsync to transfer special files such as named sockets and fifos.




Breakdown of the  -A  option

-A, --acls
              This option causes rsync to update the destination ACLs to be the same as the source ACLs.  The option also implies --perms.

              The  source  and destination systems must have compatible ACL entries for this option to work properly.  See the --fake-super option for a
              way to backup and restore ACLs that are not compatible.

              ACL = Access Control Lists       https://wiki.archlinux.org/index.php/Access_Control_Lists



Breakdown of the  -H  option


-H, --hard-links
              This tells rsync to look for hard-linked files in the source and link together the corresponding files on the destination.   Without  this
              option, hard-linked files in the source are treated as though they were separate files.

              This  option  does  NOT necessarily ensure that the pattern of hard links on the destination exactly matches that on the source.  Cases in
              which the destination may end up with extra hard links include the following:

              o      If the destination contains extraneous hard-links (more linking than what is present in the source file list),  the  copying  algo‐
                     rithm  will  not  break  them  explicitly.   However,  if one or more of the paths have content differences, the normal file-update
                     process will break those extra links (unless you are using the --inplace option).

              o      If you specify a --link-dest directory that contains hard links, the linking of the destination files against the --link-dest files
                     can cause some paths in the destination to become linked together due to the --link-dest associations.

              Note  that  rsync  can  only  detect  hard  links  between files that are inside the transfer set.  If rsync updates a file that has extra
              hard-link connections to files outside the transfer, that linkage will be broken.  If you are tempted to use the --inplace option to avoid
              this  breakage,  be  very careful that you know how your files are being updated so that you are certain that no unintended changes happen
              due to lingering hard links (and see the --inplace option for more caveats).

              If incremental recursion is active (see --recursive), rsync may transfer a missing hard-linked file before it finds that another link  for
              that  contents  exists  elsewhere  in  the hierarchy.  This does not affect the accuracy of the transfer (i.e. which files are hard-linked
              together), just its efficiency (i.e. copying the data for a new, early copy of a hard-linked file that could have been found later in  the
              transfer in another member of the hard-linked set of files).  One way to avoid this inefficiency is to disable incremental recursion using
              the --no-inc-recursive option.



Breakdown of the  -X  option


-X, --xattrs
              This option causes rsync to update the destination extended attributes to be the same as the source ones.

              For systems that support extended-attribute namespaces, a copy being done by a super-user copies all namespaces except system.*.  A normal
              user  only  copies  the  user.*  namespace.   To  be able to backup and restore non-user namespaces as a normal user, see the --fake-super
              option.

              Note that this option does not copy rsyncs special xattr values (e.g. those used by --fake-super) unless you repeat the option (e.g. -XX).
              This "copy all xattrs" mode cannot be used with --fake-super.



Breakdown of the  -v  option


-v, --verbose               increase verbosity




To restore the backup, use the same rsync command that was executed but with the source and destination reversed.




Sources:
man rsync







||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||



A Few Days Later


I whipped up a bash script that automates the process above. It creates / overwrites a log file, jbs.output on the desktop after running. The jbs.output file contains the location, user, time, date, and a list of files rsync transferred.

I set up a cron job to do an automated every Wednesday at noon sync to backup. I tested the crap out of the script, but not so much cron, because there is no practical way to do so. I really don't like working with cron, but it is the right tool for this job so that's what got implemented. Cron feels like some old cli kludge from the 1970's that forgot to get modernized, and then got a bunch of crap band-added to it.  The reason working with cron feels like an old kludgey cli tool is because that's exactly what it is. See Wikipedia on cron.

Below are the jbs.output file, the auto update bash script,  and some cron notes. The extra empty echo's in the script are just to make jbs.output file easier to read.





jbs.output file:
_____________________________________________________________________________________________________________________________________

jeff ran jeffsbackup script today.
time: 17:43:29 date: 10/21/2016

$PWD = : /home/jeff


Files Transfered From Root:

sending incremental file list
root/.cache/dconf/user
root/.config/Thunar/accels.scm
tmp/
usr/bin/jeffsbackup
var/lib/NetworkManager/
var/lib/NetworkManager/internal-0d8507f2-9a0b-39b5-8fc6-848e381522ba-enp2s0.lease
var/lib/syslog-ng/syslog-ng.persist
var/log/auth.log
var/log/daemon.log
var/log/everything.log
var/log/kernel.log
var/log/messages.log
var/log/journal/a03ebe399b344346a323a978fa91eb43/system.journal
var/log/journal/a03ebe399b344346a323a978fa91eb43/user-1000.journal

sent 67,885,828 bytes  received 10,930 bytes  7,147,027.16 bytes/sec
total size is 6,120,107,916  speedup is 90.14


Files Transfered From Home:

sending incremental file list
jeff/
jeff/.config/xfce4/panel/
jeff/.config/xfce4/panel/weather-20.rc
jeff/.config/xfce4/panel/xfce4-sensors-plugin-19.rc
jeff/.config/xfce4/xfconf/xfce-perchannel-xml/
jeff/.config/xfce4/xfconf/xfce-perchannel-xml/thunar.xml
jeff/Desktop/jbs.output

sent 2,757,560 bytes  received 10,728 bytes  615,175.11 bytes/sec
total size is 10,926,057,520  speedup is 3,946.86
_____________________________________________________________________________________________________________________________________







jeffsbackup script:
_____________________________________________________________________________________________________________________________________

#!/bin/bash

#
# Jeffs backup script
#

exec > >(tee -i /home/jeff/Desktop/jbs.output)

echo""
echo $USER" ran jeffsbackup script today."
echo "time: "`date '+%H:%M:%S'`  "date: "`date '+%m/%d/%Y'`

cd /home/jeff

echo""
echo '$PWD = :' $PWD

mount /dev/sdb6 /mnt/root

mount /dev/sdb8 /mnt/home
echo""
echo""
echo "Files Transfered From Root:"

echo""
rsync -aAHXv / /mnt/root --exclude={/etc/fstab,/home/*,/dev/*,/proc/*,/sys/*,/tmp/*,/var/tmp/*,/run/*,/mnt/*,/media/*,root/.cache/thumbnails/*,/lost+found}
echo""
echo""
echo "Files Transfered From Home:"
echo""
rsync -aAHXv /home/ /mnt/home --exclude={jeff/.thumbnails,/jeff/.cache/mozilla,jeff/.cache/chromium,jeff/Old-Home,jeff/.cache/thumbnails/,jeff/.mozilla/firefox/,jeff/.local/share/Trash}

chown jeff /home/jeff/Desktop/jbs.output

echo""
umount /dev/sdb6
umount /dev/sdb8
____________________________________________________________________________________________________________________________________






Below are some cron notes and the implementation.


Run crontab as root.

to edit:
# crontab -e

to list:
# crontab -l


See: /var/spool/cron/root
See: man crontab
See: man cron



# crontab -[option]

Options:

         -e    Edits the current crontab.
         -l     Displays the current crontab on standard output.
         -r     Removes the current crontab.



 # ┌───────────── min (0 - 59)
 # │ ┌────────────── hour (0 - 23)
 # │ │ ┌─────────────── day of month (1 - 31)
 # │ │ │ ┌──────────────── month (1 - 12)
 # │ │ │ │ ┌───────────────── day of week (0 - 6) (0 to 6 are Sunday to
 # │ │ │ │ │                  Saturday, or use names; 7 is also Sunday)
 # │ │ │ │ │
 # │ │ │ │ │
 # * * * * *  command to execute


--------------------------------------------------------------------------------------------------------------------------
# crontab file
# Setup cronie (crontab) for once a week, on Wednesday to run "jeffsbackup" script
# starting at 12:00 noon. See: /var/spool/cron/root for file.
#
# Example:
#
# min hour dayofmonth month dayofweek command 
#

00 12 * * 3 /usr/bin/jeffsbackup

----------------------------------------------------------------------------------------------------------------------------




Comments