Wednesday 15 September 2010

Rsync Backup Script

Backup script for synchronising directories among two or more systems. The most obvious one, at least when anything that is being sent to the system scheduler, was logging. The next two are sort of ad-hoc judgment calls. A usage message (which is quite lengthy) was added simply because, well this script has been running for two months and if the user would have to rerun it by hand - frankly they wouldn't have a clue. The last item was an override for the flags and protocol in the form of additional arguments. The rsync flags are sent in via a quoted string which makes it easier to parse in the script. The reader should feel free to change the script as they see fit.

Adding the ability to do different multiple sources and/or destinations was dropped due to the amount of heads and arms the script would have to grow to facilitate the capability. Additionally, logging is simplified by invoking the script by itself for each session. Remember the golden Unix utility rule:

Do one thing and do it well

Bearing that that in mind, a multiple src/dst script would be better served with a completely different script, lest the current one morph into some sort of evil hydra.

Adding Logging

It is a shell script and as such it makes perfect sense to use the shell utility logger to access syslog directly.

PROTO=ssh RSYNC=rsync RSYNC_FLAGS="-az --delete -e $PROTO" LOGCMD=logger

LOGID=rsyncondemand
# The name of our /var/log/messages entries
LOG_FLAGS="-t $LOGID" # Flags for the logger interface

Note that UTIL was changed to RSYNC - it is pretty much a given that rsync is being used, so for sanity it was changed. The additions here are the logger command and one flag, what the entry format will look like. Here is a sample from /var/log/messages:

Mar 16 00:32:12 pyxis syncondemand[3410]: Test

The rest

There are only three entries, however, anyone could feel free to change them. They are as follows:

for i in $PROTO $RSYNC $LOGCMD

do if ! type ${i} >/dev/null; then
$LOGCMD $LOGID "${i} not found"
bomb "${i} not found" fi done

A message in the utilities checker loop. If, for instance, the utility were installed blindly on a new machine and tossed into crontab, this rather informative message would let the administrator know that they need to install a utility or application.



$LOGCMD

$LOGID "Starting $RSYNC operation from $SRC to $DST"
$RSYNC $RSYNC_FLAGS $SRC $DST $LOGCMD $LOGID "Finished $RSYNC operation from
$SRC to $DST"

A start and stop messages for recording the amount of time. What happens if they start to take extraordinary long amounts of time? These particular messages let the admin know ahead of time and can easily be added to log analysis tools.

Additional Flags & Proto

Note the top of the script with the following global definitions:

PROTO=ssh RSYNC_FLAGS="-az --delete -e $PROTO"

In this version only those two are altered. Changing the protocol requires that the actual final rsync invocation be changed as well.

First, make the call and global different:

PROTO=ssh RSYNC_FLAGS="-az --delete -e " ... $RSYNC $RSYNC_FLAGS $PROTO $SRC $DST

Note that the protocol now must be defined either by the ssh default or using the modification.

Next, all that has to be done is to add a method to change the default rsync flags and protocol. It can all be done in the switch/case loop:

while getopts s:d: ch; do   

case ${ch} in
s)SRC=${OPTARG}
;;
d)DST=${OPTARG}
;;
f)$RSYNC_FLAGS=${OPTARG}
;;
p)$PROTO=${OPTARG}
;;
esac done
shift $((${OPTIND} - 1))

The only downside is that the -f has to use double quotes to replace all of the strings - regrettable - yes; but there is a usage message after all.

The Easy Part - Usage

Usage messages in shell scripts, frankly, rock. Simply because there is an all ending one simple way to do them even though they look kind of freaky relative to the indentation of the rest of the script, as usual, only the code can explain it:

usage() {     cat <<_usage_>

Basically it is just redirection and yes, it does make a well formed script look really ugly,

however, using that particular method over typing echo... echo... echo... and avoiding echo

calls makes it well worth the apparent ugliness.


Example Crontab entries for the script are:

$crontab -e

0 1 * * 1-6 /root/redbck02rsync.sh -s /etc/amanda -d redbck02:/etc

0 5 * * 1-6 /root/redbck02rsync.sh -s /space/vtapes -d redbck02:/space


Summary
This simple script should be a good start to organizing and managing rsync style backups. It is small,

concise and easy to use or modify. Some food for thought might be adding ssh options

(such as forced versions) and extended operations like cloning.


No comments:

Post a Comment