X-Git-Url: https://git.distorted.org.uk/~mdw/rsync-backup/blobdiff_plain/f6b4ffdc7265b79b945e2350efbd9d2a94df4450..c4098e64396d324603341045ca54f8c19caa993b:/rsync-backup.8?ds=sidebyside diff --git a/rsync-backup.8 b/rsync-backup.8 index 3d2cffc..5ff20bc 100644 --- a/rsync-backup.8 +++ b/rsync-backup.8 @@ -1,7 +1,15 @@ +.ie t .ds o \(bu +.el .ds o o +.de hP +.IP +\h'-\w'\fB\\$1\ \fP'u'\fB\\$1\ \fP\c +.. .TH rsync-backup 8 "7 October 2012" rsync-backup +.SH NAME +rsync-backup \- back up files using rsync .SH SYNOPSIS .B rsync-backup -.RB [ \-v ] +.RB [ \-nv ] .RB [ \-c .IR config-file ] .SH DESCRIPTION @@ -40,27 +48,140 @@ security disaster. Remember that the backup server is, in the end, responsible for the integrity of the backup data. A dishonest backup server can easily compromise a client which is being restored from corrupt backup data.) +.SS Command-line options +Most of the behaviour of +.B rsync-backup +is controlled by a configuration file, described starting with the +section named +.B Configuration commands +below. +But a few features are controlled by command-line options. +.TP +.B \-h +Show a brief help message for the program, and exit successfully. +.TP +.B \-V +Show +.BR rsync-backup 's +version number and some choice pieces of build-time configuration, and +exit successfully. +.TP +.BI "\-c " conf +Read +.I conf +instead of the default configuration file (shown as +.B conf +in the +.B \-V +output). +.TP +.B \-n +Don't actually take a backup, or write proper logs: instead, write a +description of what would be done to standard error. +.TP +.B \-v +Produce verbose progress information on standard output while the backup +is running. This keeps one amused while running a backup +interactively. In any event, +.B rsync-backup +will report failures to standard error, and otherwise run silently, so +it doesn't annoy unnecessarily if run by +.BR cron (8). +.SS Backup process +Backing up a filesystem works as follows. +.hP \*o +Make a snapshot of the filesystem on the client, and ensure that the +snapshot is mounted. There are some `trivial' snapshot types which use +the existing mounted filesystem, and either prevent processes writing to +it during the backup, or just hope for the best. Other snapshot types +require the snapshot to be mounted somewhere distinct from the main +filesystem, so that the latter can continue being used. +.hP \*o +Run +.B rsync +to copy the snapshot to the backup volume \(en specifically, to +.IB host / fs / new \fR. +If this directory already exists, then it's presumed to be debris from a +previous attempt to dump this filesystem: +.B rsync +will update it appropriately, by adding, deleting or modifying the +files. This means that retrying a failed dump \(en after fixing whatever +caused it to go wrong, obviously! \(en is usually fairly quick. +.hP \*o +Run +.B fshash +on the client to generate a `digest' describing the contents of the +filesystem, and send this to the server as +.IB host / fs / new .fshash \fR. +.hP \*o +Release the snapshot: we don't need it any more. +.hP \*o +Run +.B fshash +over the new backup; specifically, to +.BI tmp/fshash. host . fs . date \fR. +This gives us a digest for what the backup volume actually stored. +.hP \*o +Compare the two +.B fshash +digests. If they differ then dump the differences to the log file and +report a backup failure. (Backups aren't any good if they don't +actually back up the right thing. And you stand a better chance of +fixing them if you know that they're going wrong.) +.hP \*o +Commit the backup, by renaming the dump directory to +.IB host / fs / date +and the +.B fshash +digest file to +.IB host / fs / date .fshash \fR. .PP -The - - +The backup is now complete. .SS Configuration commands The configuration file is simply a Bash shell fragment: configuration commands are shell functions. .TP +.BI "addhook " hook " " command +Arrange that the named +.I hook +runs the given +.IR command . +See +.B runhook +for more details. +.TP .BI "backup " "fs\fR[:\fIfsarg\fR] ..." Back up the named filesystems. The corresponding .IR fsarg s may be required by the snapshot type. .TP +.BI "defhook " hook +Define a new hook named +.IR hook . +See +.B addhook +and +.B runhook +for more information. +.TP .BI "host " host Future .B backup commands will back up filesystems on the named .IR host . -This clears the +To back up filesystems on the backup server itself, use its hostname: +.B rsync-backup +will avoid inefficient and pointless messing about +.BR ssh (1) +in this case. +This command clears the .B like -list. +list, the remote +.B user +name, and resets the retention policy to its default (i.e., the to +policy defined prior to the first +.B host +command). .TP .BI "like " "host\fR ..." Declare that subsequent filesystems are `similar' to like-named @@ -87,7 +208,7 @@ can be .BR weekly , .BR monthly , or -.B annually +.B annually (or .BR yearly , which means the same); the @@ -101,28 +222,131 @@ or Expiry considers each existing dump against the policy lines in order: the last applicable line determines the dump's fate \(en so you should probably write the lines in decreasing order of duration. +.RS +.PP +Groups of +.B retain +commands between +.B host +and/or +.B backup +commands collectively define a retention policy. Once a policy is +defined, subsequent +.B backup +operations use the policy. The first +.B retain +command after a +.B host +or +.B backup +command clears the policy and starts defining a new one. The policy +defined before the first +.B host +is the +.I default +policy: at the start of each +.B host +stanza, the policy is reset to the default. +.RE +.TP +.BI "retry " count +The +.B live +snapshot type (see below) doesn't prevent a filesystem from being +modified while it's being backed up. If this happens, the +.B fshash +pass will detect the difference and fail. If the filesystem in question +is relatively quiescent, then maybe retrying the backup will result in a +successful consistent copy. Following this command, a backup which +results in an +.B fshash +mismatch will be retried up to +.I count +times before being declared a failure. +.TP +.BI "runhook " hook " " args\fR... +Invoke the named +.IR hook . +The individual commands on the hook are run, in order, as +.RS +.IP +.I command +.IR args ... +.PP +If any command fails (returns nonzero) then no other hooks are run and +.B runhook +fails with the same exit code. +.RE .TP .BI "snap " type " " \fR[\fIargs\fR...] Use the snapshot .I type for subsequent backups. Some snapshot types require additional -arguments, which may be supplied here. +arguments, which may be supplied here. This command clears the +.B retry +counter. +.TP +.BI "user " name +Specify the user name on the remote host. Without this, calls to +.BR ssh (1) +and +.BR rsync (1) +won't specify any user name, so the default (probably from the +.BR ssh_config (5) +file) will apply. .SS Configuration variables The following shell variables may be overridden by the configuration file. .TP +.B HASH +The hash function to use for verifying archive integrity. This is +passed to the +.B \-H +option of +.BR fshash , +so it must name one of the hash functions supported by your Python's +.B hashlib +module. +The default is +.BR sha256 . +.TP +.B INDEXDB +The name of a SQLite database initialized by +.BR update-bkp-index (8) +in which an index is maintained of which dumps are on which backup +volumes. If the file doesn't exist, then no index is maintained. The +default is +.IB localstatedir /lib/bkp/index.db +where +.I localstatedir +is the state directory configured at build time. +.TP .B MAXLOG The number of log files to be kept for each filesystem. Old logfiles are deleted to keep the total number below this bound. The default value is 14. .TP +.B METADIR +The metadata directory for the currently mounted backup volume. +The default is +.IB mntbkpdir /meta +where +.I mntbkpdir +is the backup mount directory configured at build time. +.TP .B RSYNCOPTS Command-line options to pass to .BR rsync (1) in addition to the basic set: -.B \-\-archive \-\-hard-links \-\-numeric-ids \-\-del \-\-sparse -.B \-\-compress \-\-one-file-system \-\-partial -.B \-\-filter="dir-merge .rsync-backup" +.B \-\-archive +.B \-\-hard-links +.B \-\-numeric-ids +.B \-\-del +.B \-\-sparse +.B \-\-compress +.B \-\-one-file-system +.B \-\-partial +.BR "\-\-filter=""dir-merge .rsync-backup""" . The default is .BR \-\-verbose . .TP @@ -155,21 +379,33 @@ where .I mntbkpdir is the backup mount directory configured at build time. .TP -.B HASH -The hash function to use for verifying archive integrity. This is -passed to the -.B \-H -option of -.BR fshash , -so it must name one of the hash functions supported by your Python's -.B hashlib -module. The default is -.BR sha256 . -.SS Hook functions -The configuration file may define shell functions to perform custom -actions at various points in the backup process. +.B VOLUME +The name of the current volume. If this is left unset, the volume name +is read from the file +.IB METADIR /volume +once at the start of the backup run. +.SS Hooks +The configuration file can modify the behaviour of the backup in two +main ways: by adding commands to hooks (see the +.B addhook +command); and by redefining shell functions. +.PP +The following hooks are defined. .TP -.BI "backup_precommit_hook " host " " fs " " date +.BI "commit " host " " fs " " date +Called during the commit procedure. The backup tree and manifest have +been renamed into their proper places. Typically one would use this +hook to rename files created in a corresponding +.B precommit +command. +.TP +.BI "end " rc +The backup has completed; +.B rsync-backup +will exit with status +.IR rc . +.TP +.BI "precommit " host " " fs " " date Called after a backup has been verified complete and about to be committed. The backup tree is in .B new @@ -180,12 +416,26 @@ manifest is in A typical action would be to create a digital signature on the manifest. .TP +.BI "setup " host " " fs " " date +Called when a backup of a particular filesystem is about to start. It +can return with code 99 to skip the backup. +.TP +.B "start" +Invoked before performing any actual dumps (the first time +.B host +is run). +.PP +The following shell functions can be redefined by users. +.TP .BI "backup_commit_hook " host " " fs " " date -Called during the commit procedure. The backup tree and manifest have -been renamed into their proper places. Typically one would use this -hook to rename files created by the -.B backup_precommit_hook -function. +Called from the +.B commit +hook for compatibility. +.TP +.BI "backup_precommit_hook " host " " fs " " date +Called from the +.B precommit +hook for compatibility. .TP .BR "whine " [ \-n ] " " \fItext\fR... Called to report `interesting' events when the @@ -261,3 +511,52 @@ type handlers: please see the script for details. Please send the author interesting snapshot handlers for inclusion in the main distribution. .SS Archive structure +Backup trees are stored in a fairly straightforward directory tree. +.PP +At the top level is one directory for each client host. There are also +some special entries: +.TP +.B \&.rsync-backup-store +This file must be present in order to indicate that a backup volume is +present (and not just an empty mount point). +.TP +.B fshash.cache +The cache database used for improving performance of local file +hashing. There may be other +.B fshash.cache-* +files used by SQLite for its own purposes. +.TP +.B lost+found +Part of the filesystem used on the backup volume. You don't want to +mess with this. +.TP +.B tmp +Used to store temporary files during the backup process. (Some of them +want to be on the same filesystem as the rest of the backup.) When +things go wrong, files are left behind in the hope that they might help +someone debug the mess. It's always safe to delete the files in here +when no backup is running. +.PP +So don't use those names for your hosts. +.PP +The next layer down contains a directory for each filesystem on the +given host. +.PP +The bottom layer contains a directory for each dump of that filesystem, +named with the date at which the dump was started (in ISO8601 +.IB yyyy \(en mm \(en dd +format), together with associated files named +.IB date .* \fR. +There is also a symbolic link +.B last +referring to the most recent backup of the filesystem. +.SH SEE ALSO +.BR check-bkp-status (8), +.BR fshash (1), +.BR lvm (8), +.BR rfreezefs (8), +.BR rsync (1), +.BR ssh (1), +.BR update-bkp-index (8). +.SH AUTHOR +Mark Wooding,