Windows and Mac backups using rdiff-backup

I’ve got two machines that I need to backup, an iMac running OS X 10.0.3 and a Dell running Windows XP. The iMac is the primary home machine with about 13Gb of pictures, music, documents, etc, etc. The Dell is more of a development machine with about 1Gb of files to backup.

I wanted a backup medium that I could use on both machines. I’ve never been a big fan of tape backup solutions. They’re just too expensive and difficult to use in a home environment. Because I don’t have a huge amount of stuff to backup I decided to go with an external harddrive. They’re cheap, easy to setup and I could use it on both machines. In the end I went for a WD My Book. It’s a little bit more expensive then a basic harddrive and enclosure but not so much that it turned me off. I went with the Premium version because it had the Firewire connection. If I were buying it again I don’t think I’d bothered with the Premium edition because when all you’re backing up is thousands of relatively small files it’s the processing time that slows the process down not the speed of the connection to the harddrive.

With the medium chosen I then had a look around for some backup software. I wanted something that was opensource and cross platform so that I only had to learn how to use one program. There’s very little out there that’s easy to understand and use. The basic programs just don’t instill any confidence while the feature rich programs are just too complicated and more suited to business environments. The most popular program seems to be AMANDA. The problem with AMANDA is that it’s far too complicated for a simple home backup solution (in my mind anyway). The program I finally settled for was rdiff-backup. It’s not too complicated and has all the basic features I need, it’s open source, it’s cross platform and it has a user community.

Something that I sort of forgot about up to this point was the filesystem to use on the external harddrive. If you’re only backing up a Windows machine then I suggest you format your target backup drive as NTFS. If you’re on Mac then use HFS and if you’re on Linux use ext3 or whatever you prefer. If however you want to be able to see the same partition from both Windows and Mac (like I do) then you MUST use FAT32. The reason for this is because OS X cannot write to NTFS (only read) and Windows certainly cannot write or read HFS. The only common filesystem that the two OSes can write to is FAT32. The only potential problem with this is that FAT32 only supports files up to 4Gb in size. For me this isn’t a problem but if you have a lot of videos you may find that some exceed this limit. If this happens then you could just create two partitions on the external harddrive, one HFS for your Mac backups and one NTFS for your Windows backups. You won’t be able to see the HFS partition from Windows but if that’s not a problem for you then this might be the best approach all round. The other drawback with FAT32 is that Windows will only allow you to create a partition that’s 32Gb in size. The strange thing is that Windows can reference a FAT32 partition that’s up to 2Tb in size. To create a partition greater that 32Gb WD have a tool for the My Book called the WD FAT32 Formatter. There’s probably generic tools out there that will do this for any harddrive.

rdiff-backup on Mac OS X
The first machine I backed up was the iMac. Because I might be restoring onto a completely clean machine I created a folder on the external harddrive with all the software that’s needed to install and run rdiff-backup. Into this directory I put…

Installing librsync on Mac OS X
=> tar -xzvf librsync-0.9.7.tar.gz
=> cd librsync-0.9.7
=> ./configure
=> make all
=> make check (not necessary)
=> sudo make install

Installing rdiff-backup on Mac OS X
=> tar -xzvf rdiff-backup-1.1.5.tar.gz
=> cp fs_abilities.py.patch rdiff-backup-1.1.5/rdiff_backup
=> cd rdiff-backup-1.1.5/rdiff_backup
=> patch < fs_abilities.py.patch
=> cd ..
=> sudo python setup.py install
(Create symbolic links in your bin directory for the two rdiff programs)
=> sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.3/bin/rdiff-backup /usr/bin/rdiff-backup
=> sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.3/bin/rdiff-backup-statistics /usr/bin/rdiff-backup-statistics

Preparing rdiff-backup on Mac OS X
rdiff-backup lets you explicitly list the files and directories you want to backup in an external file called the “globbing filelist”. Here’s what’s in mine…

/Users/adriansmith/Desktop
/Users/adriansmith/dev
/Users/adriansmith/Documents
/Users/adriansmith/Movies
/Users/adriansmith/Music
/Users/adriansmith/Pictures
/Users/adriansmith/Shared
/Users/adriansmith/Sites

A very important point to note (although more important on the windows platform) is that this file should be saved in UNIX format. If it’s saved in Windows format rdiff-backup will not read it correctly.

Running rdiff-backup on Mac OS X

rdiff-backup -v5 --print-statistics --include-globbing-filelist '/Volumes/My Book/instructions/mac/files-to-include.txt' --exclude-special-files --exclude / / '/Volumes/My Book/mac'

-v5 : Verbosity level
–print-statistics : When the backup is finished a summary will be displayed
–include-globbing-filelist : Followed by the file that contains the list of files & directories to backup
–exclude-special-files : Exclude all device files, fifo files, socket files, and symbolic links
–exclude / : Excludes everything from the root directory (weird I know but by including this you’re saying that you only want to include the files/directories specifically mentioned in the globbing file
/ : The source directory
‘/Volumes/My Book/mac’ : The destination directory, in this case a directory called mac on my external harddrive

rdiff-backup on Windows XP
On my Windows machine I again created a folder on the external harddrive containing all the software needed to get rdiff-backup up and running. It contains…

Installing Cygwin on Windows XP
=> Run setup.exe
=> Go with the default selected packages to install but select the following as well,

  • Devel/ autoconf, automake, binutils, gcc, make, patchutils
  • Interpreters/ python
  • Web/ wget

Installing librsync on Windows XP
=> tar -xzvf librsync-0.9.7.tar.gz
=> cd librsync-0.9.7
=> ./configure –prefix=/usr –bindir=/bin –libdir=/lib
=> make all
=> make install

Installing rdiff-backup on Windows XP
=> tar -xzvf rdiff-backup-1.1.5.tar.gz
=> cp rpath-fsync.patch rdiff-backup-1.1.5/rdiff_backup
=> cd rdiff-backup-1.1.5/rdiff_backup
=> patch < rpath-fsync.patch
=> cd ..
=> python setup.py install

Preparing rdiff-backup on Windows XP
To run rdiff-backup I again created a file containing all the files and directories I wanted to backup. Here’s what it contains…

/cygdrive/c/Documents and Settings/Adrian Smith/My Documents
/cygdrive/c/projects

IMPORTANT: This file MUST be saved in UNIX format. If you use Notepad or any other program that saves in Windows format rdiff-backup will not work. I use PFE but there are plenty of other equally capable editors out there.

Running rdiff-backup on Windows XP
Finally, to run rdiff-backup…

rdiff-backup -v5 --print-statistics --include-globbing-filelist /cygdrive/e/instructions/windows/files-to-include.txt --exclude /cygdrive/c /cygdrive/c /cygdrive/e/dell