Little Girl's Mostly Linux Blog

UniversalBackup

The Universal Backup script

This page was last updated on September 29, 2010.

What it is

This is a backup system that uses three files:

  • universalbackup – A text file containing the script.
  • includes.txt – A text file for listing files and directories to back up.
  • excludes.txt – A text file for listing files to exclude from the directories listed in includes.txt.

What it does

  1. Announces the date and time of the backup.
  2. Checks if the backup directory exists.
  3. Creates the backup directory if it don’t exist, announcing the creation.
  4. Reads the includes.txt file to know which files and directories to back up.
  5. Reads the excludes.txt file to know which files not to back up.
  6. Announces that it’s backing up files and directories.
  7. Compresses your files and folders into individual zip files in the backup directory, overwriting any files by the same name.
  8. Announces where it created the zip files.
  9. Announces that it’s testing the zip files.
  10. Tests all zip files in the temporary directory.
  11. Stops the script if any of the zip files fail the test, announcing the failure.
  12. Announces that it’s finished, and where to find your zip files.

What it doesn’t do

  • It doesn’t create incremental backups.
  • It doesn’t update existing zip files.
  • It doesn’t exclude entire directories, but rather the files in them.

See the script

This is a fully commented script that lets you know what each step does.

#!/bin/bash
# Universal Backup Script v3

#########################
# Configuration Section #
#########################

# Edit the path in this line, changing it to the path to your backup directory:
BACKUPDIRECTORY='/home/username/Desktop/backups/'

# Edit the path in this line, changing it to the path to the includes.txt file that comes with this script:
INCLUDES='/home/username/Desktop/includes.txt'

# Edit the path in this line, changing it to the path to the excludes.txt file that comes with this script:
EXCLUDES='/home/username/Desktop/excludes.txt'

#################
# Backup Script #
#################

# Announces the date and time the backup begins:
echo
echo "Beginning backup on `date`."

# Checks for needed directories, creating them if they don't exist:
echo
if [ ! -d $BACKUPDIRECTORY ]; then mkdir $BACKUPDIRECTORY; echo "Creating the $BACKUPDIRECTORY directory..."; fi

# Creates zip files in the backup directory:
echo
echo "Backing up files and directories..."
awk '{print $1, $2}' $INCLUDES | while read line; do zip -qr $BACKUPDIRECTORY`basename $line`.zip $line -x@$EXCLUDES; done

# Announces that the zip files were created:
echo
echo "Zip files have been created in $BACKUPDIRECTORY"

#  Tests the zip files in the backup directory, stopping the script if any of the files fail the test:
echo
echo "Testing zip files in $BACKUPDIRECTORY..."
echo
unzip -qt "$BACKUPDIRECTORY*.zip"
if [ $? != "0" ] ; then echo "ZIP test failed - aborting script!"; exit 1; fi

# Announces that it's finished:
echo
echo "Done! Your files are in $BACKUPDIRECTORY"

# Rings the system bell to notify you:
echo -e '\a' >&2

See an example includes.txt file

/home/username/Documents/
/home/username/Graphics/
/home/username/Music/

See an example excludes.txt file

/home/username/Documents/*.pdf

See what it looks like when it runs

Beginning backup on Fri Feb 12 16:55:19 EST 2010.

Creating the /home/username/Desktop/backups/ directory...

Backing up files and directories...

Zip files have been created in /home/username/Desktop/backups/

Testing zip files in /home/username/Desktop/backups/...

No errors detected in compressed data of /home/username/Desktop/backups/Documents.zip.
No errors detected in compressed data of /home/username/Desktop/backups/Graphics.zip.
No errors detected in compressed data of /home/username/Desktop/backups/Music.zip.

3 archives were successfully processed.

Done! Your files are in /home/username/Desktop/backups/

See the backup directory after it runs

Documents.zip
Graphics.zip
Music.zip

Get it

Download UniversalBackupScript-v4.zip

Extract it

Double-click the file to extract it.

Before you begin

Before you run the script for the first time, you’ll need to customize the files that come with it so that it knows what you want to back up, and which directory you want to use for your backups.

Customize the backupscript file

  1. Edit the backupscript file by opening it in a text editor.
  2. Find the Configuration Section at the top of the file.
  3. Change /home/username/Desktop/backups/ to the full path to the the directory you would like to use for your backups.
  4. Change /home/username/Desktop/includes.txt to the full path to the includes.txt file that comes with this script.
  5. Change /home/username/Desktop/excludes.txt to the full path to the excludes.txt file that comes with this script.
  6. Save and close the file.

Customize the includes.txt file

This file contains some example entries you can remove. You’ll surely also wish to add some entries.

  • Edit the file by opening it in a text editor.
  • To delete an entry, delete an entire line from the file.
  • To add an entry, type the full path to the file or directory you’d like to back up. Be sure to add a forward slash at the end of the path if the entry is a directory. Press the Enter key.
  • Save and close the file.

Customize the excludes.txt file

This file contains an example entry you can remove. You may also wish to add some entries.

  • Edit the file by opening it in a text editor.
  • To delete an entry, delete an entire line from the file.
  • To add an entry, type the full path to the file you’d like to exclude from the backup. Wildcards are accepted. Press the Enter key.
  • The file should be completely empty if you have no files to exclude.
  • Save and close the file.

Make sure you have room

If disk space is at a premium, open a terminal window and type this command to check how much space you have available:
df -h

Run the script.

  1. Open a terminal window in the directory that contains the universalbackup, includes.txt, and excludes.txt files.
  2. Type this command:
  3. bash universalbackup

After you run it

For the best protection of your data, copy the zip files you just backed up to another computer, a CD or DVD, a USB stick or drive, or other media. I highly recommend Mirrordir (which is available in the Ubuntu repositories) as a fast, easy, reliable way to synchronize files and directories. I trust it completely with the contents of my computer, and I’ve used it for years.

  • The command can be as simple as:
  • mirrordir /source /destination
  • You can (and should) do a dry run first to be sure you have the command written properly:
  • mirrordir -v --test-only /source /destination

    Note: The -v switch provides verbose output and the –test-only switch provides a preview of what Mirrordir would do without the switch.

It’s a good idea to:

  • Look over the man page to familiarize yourself with all the features (there are many).
  • Create temporary source and destination test directories.
  • Run Mirrordir on the test directories.
  • Compare the directories to reassure yourself that your data will be safe and completely identical on both computers.

Some pages on Mirrordir:

Note: Mirrordir is no longer available in the current version of Ubuntu. This is because it has been found to not be entirely reliable. My understanding is that it sometimes doesn’t copy data to the destination. I’ve never seen this behavior (and if a program can be worn out, I’ve worked hard to wear out Mirrordir). Since Mirrordir is such a lightning fast program, it’s simple enough to run it twice to double-check that source and destination are indeed identical, and copy over any straggling files or directories. For those of you who would like to try it despite its alleged unreliability, you can download it from Tucows here.



Obligatory Happy Ending

And they all lived happily ever after. The end.

2 Comments »

  1. Testing the universalbackup, it is a nice tool. Thanks a lot for sharing it!
    But I noticed that the example excludes.txt-file doesn’t do anything. The excludes.txt needs to refer to files, and not to folders in order to do anything…

    Comment by paul — September 29, 2010 @ 10:08 am

    • Oh, thank you for spotting that! I’ve updated the instructions on the page and in the README inside the zip file. The zip file is at v4, although the script didn’t change.

      I also added a note on Mirrordir while I was in here making changes. Enjoy! :)

      Comment by mostlylinux — September 29, 2010 @ 2:22 pm


RSS feed for comments on this post. TrackBack URI

Comment:

Blog at WordPress.com.