5 \h'-\w'\fB\\$1\ \fP'u'\fB\\$1\ \fP\c
7 .TH rsync-backup 8 "7 October 2012" rsync-backup
9 rsync-backup \- back up files using rsync
18 script is a backup program of the currently popular
23 ability to create hardlinks from (apparently) similar existing local
24 trees to make incremental dumps efficient, even from remote sources.
25 Restoring files is easy because the backups created are just directories
26 full of files, exactly as they were on the source \(en and this is
31 The script does more than just running
33 It is also responsible for creating and removing snapshots of volumes to
34 be backed up, and expiring old dumps according to a user-specified
39 script should be installed and run on a central backup server with local
40 access to the backup volumes.
42 The script should be run with full (root) privileges, so that it can
43 correctly record file ownership information. The server should also be
46 to the client machines, and run processes there as root. (This is not a
47 security disaster. Remember that the backup server is, in the end,
48 responsible for the integrity of the backup data. A dishonest backup
49 server can easily compromise a client which is being restored from
51 .SS Command-line options
52 Most of the behaviour of
54 is controlled by a configuration file, described starting with the
56 .B Configuration commands
58 But a few features are controlled by command-line options.
61 Show a brief help message for the program, and exit successfully.
66 version number and some choice pieces of build-time configuration, and
72 instead of the default configuration file (shown as
79 Don't actually take a backup, or write proper logs: instead, write a
80 description of what would be done to standard error.
83 Produce verbose progress information on standard output while the backup
84 is running. This keeps one amused while running a backup
85 interactively. In any event,
87 will report failures to standard error, and otherwise run silently, so
88 it doesn't annoy unnecessarily if run by
91 Backing up a filesystem works as follows.
93 Make a snapshot of the filesystem on the client, and ensure that the
94 snapshot is mounted. There are some `trivial' snapshot types which use
95 the existing mounted filesystem, and either prevent processes writing to
96 it during the backup, or just hope for the best. Other snapshot types
97 require the snapshot to be mounted somewhere distinct from the main
98 filesystem, so that the latter can continue being used.
102 to copy the snapshot to the backup volume \(en specifically, to
103 .IB host / fs / new \fR.
104 If this directory already exists, then it's presumed to be debris from a
105 previous attempt to dump this filesystem:
107 will update it appropriately, by adding, deleting or modifying the
108 files. This means that retrying a failed dump \(en after fixing whatever
109 caused it to go wrong, obviously! \(en is usually fairly quick.
113 on the client to generate a `digest' describing the contents of the
114 filesystem, and send this to the server as
115 .IB host / fs / new .fshash \fR.
117 Release the snapshot: we don't need it any more.
121 over the new backup; specifically, to
122 .BI tmp/fshash. host . fs . date \fR.
123 This gives us a digest for what the backup volume actually stored.
127 digests. If they differ then dump the differences to the log file and
128 report a backup failure. (Backups aren't any good if they don't
129 actually back up the right thing. And you stand a better chance of
130 fixing them if you know that they're going wrong.)
132 Commit the backup, by renaming the dump directory to
137 .IB host / fs / date .fshash \fR.
139 The backup is now complete.
140 .SS Configuration commands
141 The configuration file is simply a Bash shell fragment: configuration
142 commands are shell functions.
144 .BI "addhook " hook " " command
145 Arrange that the named
153 .BI "backup " "fs\fR[:\fIfsarg\fR] ..."
154 Back up the named filesystems. The corresponding
156 may be required by the snapshot type.
159 Define a new hook named
165 for more information.
170 commands will back up filesystems on the named
172 To back up filesystems on the backup server itself, use its hostname:
174 will avoid inefficient and pointless messing about
177 This command clears the
181 name, and resets the retention policy to its default (i.e., the to
182 policy defined prior to the first
186 .BI "like " "host\fR ..."
187 Declare that subsequent filesystems are `similar' to like-named
188 filesystems on the named
192 should use those trees as potential sources of hardlinkable files. Be
193 careful when using this option without
196 option: an erroneous hardlink will cause the backup to fail. (The
197 backup won't be left silently incorrect.)
199 .BI "retain " frequency " " duration
200 Define part a backup retention policy: backup trees of the
202 should be kept for the
214 which means the same); the
222 Expiry considers each existing dump against the policy lines in order:
223 the last applicable line determines the dump's fate \(en so you should
224 probably write the lines in decreasing order of duration.
233 commands collectively define a retention policy. Once a policy is
236 operations use the policy. The first
242 command clears the policy and starts defining a new one. The policy
243 defined before the first
247 policy: at the start of each
249 stanza, the policy is reset to the default.
255 snapshot type (see below) doesn't prevent a filesystem from being
256 modified while it's being backed up. If this happens, the
258 pass will detect the difference and fail. If the filesystem in question
259 is relatively quiescent, then maybe retrying the backup will result in a
260 successful consistent copy. Following this command, a backup which
263 mismatch will be retried up to
265 times before being declared a failure. The default is to retry once,
266 clearing mismatching files from the
268 caches before the second attempt.
270 .BI "runhook " hook " " args\fR...
273 The individual commands on the hook are run, in order, as
279 If any command fails (returns nonzero) then no other hooks are run and
281 fails with the same exit code.
284 .BI "snap " type " " \fR[\fIargs\fR...]
287 for subsequent backups. Some snapshot types require additional
288 arguments, which may be supplied here. This command clears the
293 Specify the user name on the remote host. Without this, calls to
297 won't specify any user name, so the default (probably from the
300 .SS Configuration variables
301 The following shell variables may be overridden by the configuration
305 The hash function to use for verifying archive integrity. This is
310 so it must name one of the hash functions supported by your Python's
317 The name of a SQLite database initialized by
318 .BR update-bkp-index (8)
319 in which an index is maintained of which dumps are on which backup
320 volumes. If the file doesn't exist, then no index is maintained. The
322 .IB localstatedir /lib/bkp/index.db
325 is the state directory configured at build time.
328 The number of log files to be kept for each filesystem. Old logfiles
329 are deleted to keep the total number below this bound. The default
333 The metadata directory for the currently mounted backup volume.
338 is the backup mount directory configured at build time.
341 Command-line options to pass to
343 in addition to the basic set:
350 .B \-\-one-file-system
352 .BR "\-\-filter=""dir-merge .rsync-backup""" .
359 snapshots are mounted on subdirectories below the
361 .IR "on backup clients" .
366 is the backup mount directory configured at build time.
369 The volume size option to pass to
371 when creating a snapshot. The default is
373 which seems to work fairly well.
376 Where the actual backup trees should be stored. See the section on
383 is the backup mount directory configured at build time.
386 The name of the current volume. If this is left unset, the volume name
387 is read from the file
389 once at the start of the backup run.
391 The configuration file can modify the behaviour of the backup in two
392 main ways: by adding commands to hooks (see the
394 command); and by redefining shell functions.
396 The following hooks are defined.
398 .BI "commit " host " " fs " " date
399 Called during the commit procedure. The backup tree and manifest have
400 been renamed into their proper places. Typically one would use this
401 hook to rename files created in a corresponding
406 The backup has completed;
408 will exit with status
411 .BI "precommit " host " " fs " " date
412 Called after a backup has been verified complete and about to be
413 committed. The backup tree is in
415 in the current directory, and the
419 A typical action would be to create a digital signature on the
422 .BI "setup " host " " fs " " date
423 Called when a backup of a particular filesystem is about to start. It
424 can return with code 99 to skip the backup.
427 Invoked before performing any actual dumps (the first time
431 The following shell functions can be redefined by users.
433 .BI "backup_commit_hook " host " " fs " " date
436 hook for compatibility.
438 .BI "backup_precommit_hook " host " " fs " " date
441 hook for compatibility.
443 .BR "whine " [ \-n ] " " \fItext\fR...
444 Called to report `interesting' events when the
446 option is in force. The default action is to echo the
448 to (what was initially) standard output, followed by a newline unless
452 The following snapshot types are available.
455 A trivial snapshot type: attempts to back up a live filesystem. How
456 well this works depends on how active the filesystem is. If files
457 change while the dump is in progress then the
459 verification will likely fail. Backups using this snapshot type must
460 specify the filesystem mount point as the
464 A slightly less trivial snapshot type: make the filesystem read-only
465 while the dump is in progress. Backups using this snapshot type must
466 specify the filesystem mount point as the
470 Create snapshots using LVM. The snapshot argument is interpreted as the
471 relevant volume group. The filesystem name is interpreted as the origin
472 volume name; the snapshot will be called
475 .IB SNAPDIR / fs \fR;
476 space will be allocated to it according to the
480 .BI "rfreezefs " client " " vg
481 This gets complicated. Suppose that a server has an LVM volume group,
482 and exports (somehow) a logical volume to a client. Examples are a host
483 providing a virtual disk to a guest, or a server providing
484 network-attached storage to a client. The server can create a snapshot
485 of the volume using LVM, but must synchronize with the client to ensure
486 that the filesystem image captured in the snapshot is clean. The
488 program should be installed on the client to perform this rather
489 delicate synchronization. Declare the server using the
491 command as usual; pass the client's name as the
494 server's volume group name as the
496 snapshot arguments. Finally, backups using this snapshot type must
497 specify the filesystem mount point (or, actually, any file in the
498 filesystem) on the client, as the
501 Additional snapshot types can be defined in the configuration file. A
502 snapshot type requires two shell functions.
504 .BI snap_ type " " snapargs " " fs " " fsarg
505 Create the snapshot, and write the mountpoint (on the client host) to
506 standard output, in a form suitable as an argument to
509 .BI unsnap_ type " " snapargs " " fs " " fsarg
512 There are a number of utility functions which can be used by snapshot
513 type handlers: please see the script for details. Please send the
514 author interesting snapshot handlers for inclusion in the main
516 .SS Archive structure
517 Backup trees are stored in a fairly straightforward directory tree.
519 At the top level is one directory for each client host. There are also
520 some special entries:
522 .B \&.rsync-backup-store
523 This file must be present in order to indicate that a backup volume is
524 present (and not just an empty mount point).
527 The cache database used for improving performance of local file
528 hashing. There may be other
530 files used by SQLite for its own purposes.
533 Part of the filesystem used on the backup volume. You don't want to
537 Used to store temporary files during the backup process. (Some of them
538 want to be on the same filesystem as the rest of the backup.) When
539 things go wrong, files are left behind in the hope that they might help
540 someone debug the mess. It's always safe to delete the files in here
541 when no backup is running.
543 So don't use those names for your hosts.
545 The next layer down contains a directory for each filesystem on the
548 The bottom layer contains a directory for each dump of that filesystem,
549 named with the date at which the dump was started (in ISO8601
550 .IB yyyy \(en mm \(en dd
551 format), together with associated files named
553 There is also a symbolic link
555 referring to the most recent backup of the filesystem.
557 .BR check-bkp-status (8),
563 .BR update-bkp-index (8).
565 Mark Wooding, <mdw@distorted.org.uk>
~mdw / rsync-backup / blob