Various ways to backup

Over the years I have been using several kinds of solutions to backup stuff. Here’s a description of the ones I can remember. Many of them focus on being low-cost, simple and straightforward, but are far from a perfect solution in many other ways.

A common technique for me has been to just make zip files of whatever I want to backup. Simple snapshots of a project, so I end up with bunch of “projectname-YYYYMMDD.zip” files. Or altenatively a common zip of certain folder which gets updated over time. For example, before I moved to using cvs or svn for source code version control, I used a batch file like this to backup all my little coding projects:

backup.bat
@zip -u -r d:\backup\prog_c * -x *.ilk *.pdb *.idb *.ncb *.opt *.plg *.bsc *.sbr *.pch *.obj *.zip *.dll


As you can see, this just updates an existing zip and excludes bunch of unnecessary files which I didn’t want to keep bloating up the backup zip. A worthy note is that I have usually made sure I have two separate hard disks in a computer and put the backups on the other one from the original data. At some point I also tried to switch to using dar instead of the zip files, since it would have had some advantages over the zip files. I didn’t end up using that for long though.

Like I hinted above, nowadays I’m using version control for source code, so I don’t make such zips that frequently anymore. Instead I have made sure that the version control repository gets backed up somehow.

My first solution was to make nightly snapshots of the repository using a script on the server, and then occasionally copy a snapshot file back to my main computer. I used this solution for many years. Below is copy of the backup script I used, which creates a .tar.gz backup file with given prefix from a specified file mask, adding YYYY-MM-DD timestamp to the backup file name, and also keeps only given number of snapshots around cleaning up any older ones.

dobackup.sh
#!/bin/sh

if [[ ! $1 || ! $2 || ! $3 || ! $4 || ! $5 ]]
then
echo $0 destdir name execdir mask backups
echo $'\t'destdir = directory to put backups in
echo $'\t'name = file prefix name of this backup
echo $'\t'execdir = directory to run the backup tar command in
echo $'\t'mask = filemask to give for tar
echo $'\t'backups = number of backups of name to keep in destdir
exit
fi

DESTDIR=$1
NAME=$2
EXECDIR=$3
MASK=$4
BACKUPS=$5

DESTFILE=$DESTDIR/$NAME-`date +%F`.tar.gz

cd $EXECDIR
tar zcf $DESTFILE $MASK

EXTRAFILES=$((`ls $DESTDIR/$NAME-* | wc -l` - $BACKUPS))

if [[ $EXTRAFILES > 0 ]]
then
rm -f `ls -t $DESTDIR/$NAME-* | tail -$EXTRAFILES`
fi


I had then another script which called the above one to create backup files from a few places in the system, and was put in a cron job to be run once per night. The backup script looked something like this:

allbackups.sh
#!/bin/sh
# All automatic nightly backups
cd /home/user/bin
BACKUPDIR=/var/backups/user
umask 007
# /home/share (includes version control repository)
./dobackup.sh $BACKUPDIR share /home share 7
# misc stuff
./dobackup.sh $BACKUPDIR misc /home/user "bin bookmarks.html code" 7


More recently I have also moved to mirroring stuff over to another computer using rsync. This was something I needed to do to create an off-site remote backup for my workplace’s version control repository. To do this, I have simply put ssh-keys in place to allow automatic remote ssh login from a certain machine to run rsync over it, and then I have a cron job line like this to update the mirror every night:

nightly rsync mirror for cron job file:
30 5 * * * /usr/bin/rsync -avz -e ssh user@server:/path/* /backup/mirror/ > /backup/log/last_cron_mirror.txt 2>&1


Additionally I’m also backing up some paths of this web server every now and then using simple tar commands in a script, and the following command to back up a dump of the mysql database:

compressed mysql dump, must be run as root:
/usr/bin/mysqldump --defaults-file=/etc/mysql/debian.cnf -A -Q --opt | /bin/bzip2 > ~/db-mysql-`date +%Y%m%d`.sql.bz2


Finally, since there are multiple computers in our household, we have also been simply copying manually some files over to a share in another computer to have at least a single duplicate. These days I have also set Windows 7 backup software to maintain a backup of certain folders and system state to a share in my home Linux server. In the past I also sometimes made manually a backup of my Windows XP computer’s system state using the OS backup tool. Also recently when a brand new hard disk failed for me, I bought another one and after getting the warranty replacement I put the two into a mirrored RAID disk.

I have experienced so many hard disk crashes over time so I have learned to pay at least some attention to backing up the most important stuff, even if I really still tend to be a little bit lazy about it.