9/2024
Found this and started using it for backup.
Old method stopped working.
The first thing you need to do is install pv.
sudo apt update && sudo apt install pv
1. Download .sh files
1a. git clone https://github.com/The-Exterminator/PiShrink-to-Crontab.git
1b. cd PiShrink-to-Crontab/
2. Install PiShrink and Backup
2a. chmod +x *.sh
2b. sudo mv *.sh /usr/local/bin/
3. Change BACKUP_PATH="/media/Backup
BACKUP_PATH="/mnt/rt-86u/back_nr/$(get_codename)/${HOSTNAME}"
BACKUP_PATH="/mnt/rt-86u/back_ha/$(get_codename)/${HOSTNAME}"
DO NOT CHANGE /$(get_codename)/${HOSTNAME}
3a. sudo nano /usr/local/bin/Backup.sh
4. Delete folder
4a. cd
4b. sudo rm -r PiShrink-to-Crontab
5. chmod /YOU/BACKUP/FOLDER/
5a. chmod 755 /YOU/BACKUP/FOLDER/
sudo chmod 755 /mnt/rt-86u/back_nr/
sudo chmod 755 /mnt/rt-86u/back_ha/
6. sudo nano /etc/crontab
Insert -> 59 23 * * 1 root /usr/local/bin/Backup.sh
To run on its own
cd /usr/local/bin
sudo ./Backup.sh
To find backup file
cd /mnt/rt-86u/back_nr/Bullseye/raspberrypi
cd /mnt/rt-86u/back_ha/Bullseye/raspberrypi
Add >> /home/pi/mycron.log 2>&1 to log
https://crontab.guru/ to check crontab
sudo nano /etc/crontab
1am on 7th of each month, create log file
0 1 7 * * root /usr/local/bin/Backup.sh >> /home/pi/piback.log 2>&1 to log
EDIT Oct 2022
Had a problem with full backups in Cron, had to change format of command to add \ into %Y, %m and %d
0 2 1 * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_$(date +\%Y-\%m-\%d).img > /home/pi/backupfull.log 2>&1
How I backup my pi's as of June, 2022.
At first I was using a external usb drive I put together from an ebay enclosure and a little M2 NVME SSD I had.
Still using that, but its easier to backup to a 2tb external drive I have connected to my Asus router that acts like a NAS.
Info in Backup to Network Drive below.
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_nr/rpiNR_$(date +%Y-%m-%d).img
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_$(date +%Y-%m-%d).img
cd /mnt
sudo mkdir rt-86u
cd /mnt/rt-86u
sudo mkdir back_pool
Mount the network drive now, manually
sudo mount -t cifs //rt-86u/2gb /mnt/rt-86u -o guest,vers="1.0"
Check to see if drive mounted
cd ..
ls rt-86u
It will list whatever you have there
Make directory
sudo mkdir rt-86u/back_pool
Get backup script
cd /home/pi
git clone https://github.com/lzkelley/bkup_rpimage.git
Backup Pool Pi manually
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_pool/rpiPool_backup.img
cd /mnt
sudo mkdir rt-86u
cd /mnt/rt-86u
sudo mkdir back_ezbeq
Mount the network drive now, manually
sudo mount -t cifs //rt-86u/2gb /mnt/rt-86u -o guest,vers="1.0"
Check to see if drive mounted
cd ..
ls rt-86u
It will list whatever you have there
Make directory
sudo mkdir rt-86u/back_ezbeq
Get backup script
cd /home/pi
sudo apt update
sudo apt install git
git clone https://github.com/lzkelley/bkup_rpimage.git
Backup Pi manually
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ezbeq/rpiEzbeq_backup.img
Bought a "M2 NVME PCIe SSD Hard Disk Case Enclosure M.2 to USB Type-C 3.1 GEN2 Key Adapter". Stuck a M2 250GB SSD I had that I wasn't using in it. C
Cheap way to make a external SSD.
Install NVME type SSD into enclosure.
I setup with Ubuntu Disks program, was easier because I had to delete partitions to make one big one.
Formatted drive as EXT4 because that is easy for pi to use
Instructions to mount drive on pi at link below,
Mounting Drive on Pi then go to section for Mount USB Drive
After mounting the external USB drive, this is how you backup to it.
cd /home/pi
git clone https://github.com/lzkelley/bkup_rpimage.git
Above copies script to ~/bkup_rpimage directory.
https://github.com/lzkelley/bkup_rpimage
Script to backup pi to usb stick, mounted at /mnt/usbstick with piback directory to save file
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/usbstick/piback/rpi_backup.img
If you run again, it will perform an incremental backup
Options:
-c creates the SD Image if it does not exist
-l writes rsync log to 'sdimage'-YYYYmmddHHMMSS.log
-z compresses the SD Image (after backup) to 'sdimage'.gz
-d deletes the SD Image after successful compression
-f forces overwrite of 'sdimage'.gz if it exists
-L logfile writes rsync log to 'logfile'
-s define the size of the image file
Now here is how you setup your pi to backup automatically on a schedule using cron.
Meaning to numbers at beginning of cron line. * means all days or hours or mins
Minute Hour Day Of the Month Month Day of the week Command to run
(0-59) (0-23) (1-31) (1-12) (0-6) 0 is sun
You have to edit crontab to add task to automate based on time.
EDIT: you need to make sure it will work as root.
sudo crontab -e
I used nano for the editor, its what I normally use to edit on the pi.
I will now provide a few examples for different time options
Add to bottom of file
For incremental, at 1 am
#incremental backup to USB stick at 1am daily, and create log at /home/pi
0 1 * * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/usbstick/piback/rpi_backup.img > /home/pi/backup.log 2>&1
This part at the end, "> /home/pi/backup.log 2>&1" writes terminal output to backup.log file in pi home directory. So you can see the log from last time it was run.
Use this line for incremental backup at 1 am with no log file
#incremental backup to USB stick at 1am daily
0 1 * * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/usbstick/piback/rpi_backup.img
For full backup with date, at 2am on 1st of the month. Create log for backup in /home/pi
#full backup to NAS at 2am on 1st of the month
0 2 1 * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_$(date +\%Y-\%m-\%d).img > /home/pi/backupfull.log 2>&1
Cron didn't work for me at first because I was using ~/ instead of /home/pi. For cron it matters, you need to define the full path. Otherwise it can't find the files because its running as root
Also had to add a '\' for every '%', otherwise Cron would ignore the line
To get a list from crontab
sudo crontab -l
Time to backup:
On my pi4, takes less than 15 mins for a 11gb backup
pi3 for Node Red, less then 30 mins
Setup Network Drive
Instructions to mount NAS on pi at link below,
Mounting Drive on Pi then go to section for Asus Router NAS
Mount Network Drive
There is a directory called 2gb on the drive connected to my Asus router that I want to share
I tested in a browser to make sure I could get to. Just copy this to the address
smb://rt-86u/2gb
Now make directories in pi
cd /mnt
sudo mkdir rt-86u
cd /mnt/rt-86u
sudo mkdir back_ha && sudo mkdir back_nr
Mount the drive now, manually
sudo mount -t cifs //rt-86u/2gb /mnt/rt-86u -o guest,vers="1.0"
Check to see if drive mounted
cd ..
ls rt-86u
If it didn't work, you could try this version. Above worked for me
sudo mount -t cifs //rt-86u/2gb /mnt/rt-86u -o guest,vers="1.0",sec=ntlmv2
To make it work permanently, you need to edit fstab
cd /etc
sudo nano fstab
add this line to end of fstab, then exit and save
//rt-86u/2gb /mnt/rt-86u cifs guest,x-systemd.automount,x-systemd.device-timeout=60,vers=1.0 0 0
Reboot to test, or enter:
sudo mount -a
Network drive should be mounted and accessible to pi now.
cd /mnt/rt-86u
ls
And you should see contents of NAS directory
Now you can run script to backup pi to usb stick, mounted at /mnt/rt-86u with back_nr or back_ha directory to save file
Backup Node Red Pi manually
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_nr/rpiNR1_backup.img
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/usbstick/piback/rpi_backup.img
Backup HA Pi manually
sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_backup.img
To schedule, edit crontab
sudo crontab -e
Add to bottom of file
For incremental, at 3 am for HA
0 3 * * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_backup.img > /home/pi/backupinc.log 2>&1
For incremental, at 3 am for NR
0 3 * * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_nr/rpiNR_backup.img > /home/pi/backupinc.log 2>&1
This part at the end, "> /home/pi/backup.log 2>&1" writes terminal output to backup.log file in pi home directory. So you can see the log from last time it was run.
For incremental, at 3 am Sunday for Home Assistant pi
#incremental backup to NAS at 3am Sunday (weekly)
0 3 * * 0 sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_backup.img > /home/pi/backupfull.log 2>&1
For incremental, at 3 am Sunday for Node Red pi
#incremental backup to NAS at 3am Sunday (weekly)
0 3 * * 0 sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_nr/rpiNR_backup.img > /home/pi/backupfull.log 2>&1
For full backup with date, at 2am on 1st of the month, for HA
#incremental backup to NAS at 2am on x of the month
0 2 1 * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_ha/rpiHA_$(date +\%Y-\%m-\%d).img
For full backup with date, at 2am on 1st of the month, for NR
#full backup to NAS at 2am on 1st of month
0 2 1 * * sudo sh /home/pi/bkup_rpimage/bkup_rpimage.sh start -c /mnt/rt-86u/back_nr/rpiNR_$(date +\%Y-\%m-\%d).img
Cron Info:
Minute Hour Day Of the Month Month Day of the week Command to run
(0-59) (0-23) (1-31) (1-12) (0-6) 0 is sun
I added /mnt/usbstick to samba config file to be able to use samba to copy files from USB drive
For the NAS files, you can directly copy, or even just program directly from NAS to sd card
From Windows, you can use BalenaEtcher or Win32DiskImager to write a image file to a SD or USB
Could also use a MAC or Linux machine, and do basically the same procedure.
BalenaEtcher have MAC and Linux versions
OLD INFORMATION
Go to link, then download zip in upper right
Upzip, and copy docker-compose-backup.sh file to folder with docker-compose.yaml file, ~/IOTstack for me
I used Samba to copy the file, so it was a drag and drop
Make script executable
sudo chmod +x docker-compose-backup.sh
I used portainer to stop all containers while I backup
Execute script
sudo bash docker-compose-backup.sh
sh docker-compose-backup.sh
Another reference page to backup data volumes
Easiest way to do a full backup, when removing card
Windows PC running win32 disk imager
How to shrink an IMG file
With large uSD and USB sticks, wanted to shrink image files so they don't take up so much space to backup and read/write
https://beebom.com/how-clone-raspberry-pi-sd-card-windows-linux-macos/
Shrink with PiShrink
1. Download the PiShrink script, and make it executable. In a Terminal, enter the following two commands:
wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x ./pishrink.sh
2. Run the script, followed by the name of the image that you want to shrink.
sudo ./pishrink.sh ./raspbian_backup.img
On the Ubuntu PC, I installed x11VNC
https://tecadmin.net/setup-x11vnc-server-on-ubuntu-linuxmint/
sudo apt-get update
sudo apt-get install x11vnc net-tools
create password
x11vnc -storepasswd
Start x11vnc
sudo x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/rahul/.vnc/passwd -rfbport 5900 -shared
Autostart on boot
sudo vi /etc/init/x11vnc.conf
File contents:
# description "Start x11vnc on system boot"
description "x11vnc"
start on runlevel [2345]
stop on runlevel [^2345]
console log
respawn
respawn limit 20 5
exec /usr/bin/x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/rahul/.vnc/passwd -rfbport 5900 -shared
Installed realvnc viewer on my mac
https://www.realvnc.com/en/connect/download/viewer/macos/
Setup per this video
https://www.youtube.com/watch?v=r-lQ1VjY02s
Read SD to img file
https://appcodelabs.com/how-to-backup-clone-a-raspberry-pi-sd-card-on-macos-the-easy-way
diskutil list
/dev/disk4 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *32.0 GB disk4
1: Windows_FAT_32 boot 268.4 MB disk4s1
2: Linux 31.7 GB disk4s2
sudo dd if=/dev/disk4 of=Pi4HA20220605.dmg
Insert sd card to write into mac, and start terminal
diskutil list
Find disk number. Was disk4 for me again.
diskutil unmountDisk disk4
Write file to SD card
sudo dd if=Pi4HA20220605.dmg of=/dev/rdisk4
Start 11:35 1232 finish, so one hour
Shrink it
hdiutil resize -sectors min -imageonly Pi4HA20220605acopy.dmg
hdiutil resize -size 15G Pi4HA20220605acopy.dmg
Pi4HA20220605acopy.dmg
1. Insert the SD card in your PC using a USB or built-in card reader. Now open terminal (clt alt t) enter the command sudo fdisk -l. This will list all the filesystems present on your system.
2. Try to find out the device name of your SD card. I have a 16GB SD card, so it is easily identified as the device /dev/sdb which has a size of 14.9GB. This is because the actual storage on a device is always slightly lower than advertised. Note down this device name.
Use the dd command to write the image to your hard disk. For example:
sudo dd if=/dev/sdb of=~/raspbian_backup.img
Ubuntu
Use balenaEtcher
Insert the SD card in your PC. Before we restore the image, it is important to make sure that the SD card’s partitions are unmounted. To verify this, open the Terminal, and execute the command sudo mount | grep sdb. Here, replace sdb with your SD card’s device name.
If you see a blank output, you do not need to do anything. If you do see some mounted partitions, unmount the listed ones. For example:
sudo umount /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdb4
Use the dd command to write the image file to the SD card:
sudo dd if=~/raspbian_backup.img of=/dev/sdb
This is like the command we used to make a clone, but reversed. This time, the input file if is the backup image, while the output file of is the SD card device.
Again, verify, and double-verify the parameters here, as entering the wrong command here will cause permanent data loss.
Once the write is complete, you will see a confirmation from dd. You can then remove the card from your PC, and insert it back in the Raspberry Pi.