5 \h'-\w'\fB\\$1\ \fP'u'\fB\\$1\ \fP\c
7 .TH rsync-backup 8 "7 October 2012" rsync-backup
16 script is a backup program of the currently popular
21 ability to create hardlinks from (apparently) similar existing local
22 trees to make incremental dumps efficient, even from remote sources.
23 Restoring files is easy because the backups created are just directories
24 full of files, exactly as they were on the source \(en and this is
29 The script does more than just running
31 It is also responsible for creating and removing snapshots of volumes to
32 be backed up, and expiring old dumps according to a user-specified
37 script should be installed and run on a central backup server with local
38 access to the backup volumes.
40 The script should be run with full (root) privileges, so that it can
41 correctly record file ownership information. The server should also be
44 to the client machines, and run processes there as root. (This is not a
45 security disaster. Remember that the backup server is, in the end,
46 responsible for the integrity of the backup data. A dishonest backup
47 server can easily compromise a client which is being restored from
49 .SS Command-line options
50 Most of the behaviour of
52 is controlled by a configuration file, described starting with the
54 .B Configuration commands
56 But a few features are controlled by command-line options.
59 Show a brief help message for the program, and exit successfully.
64 version number and some choice pieces of build-time configuration, and
70 instead of the default configuration file (shown as
77 Produce verbose progress information on standard output while the backup
78 is running. This keeps one amused while running a backup
79 interactively. In any event,
81 will report failures to standard error, and otherwise run silently, so
82 it doesn't annoy unnecessarily if run by
85 Backing up a filesystem works as follows.
87 Make a snapshot of the filesystem on the client, and ensure that the
88 snapshot is mounted. There are some `trivial' snapshot types which use
89 the existing mounted filesystem, and either prevent processes writing to
90 it during the backup, or just hope for the best. Other snapshot types
91 require the snapshot to be mounted somewhere distinct from the main
92 filesystem, so that the latter can continue being used.
96 to copy the snapshot to the backup volume \(en specifically, to
97 .IB host / fs / new \fR.
98 If this directory already exists, then it's presumed to be debris from a
99 previous attempt to dump this filesystem:
101 will update it appropriately, by adding, deleting or modifying the
102 files. This means that retrying a failed dump \(en after fixing whatever
103 caused it to go wrong, obviously! \(en is usually fairly quick.
107 on the client to generate a `digest' describing the contents of the
108 filesystem, and send this to the server as
109 .IB host / fs / new .fshash \fR.
111 Release the snapshot: we don't need it any more.
115 over the new backup; specifically, to
116 .BI tmp/fshash. host . fs . date \fR.
117 This gives us a digest for what the backup volume actually stored.
121 digests. If they differ then dump the differences to the log file and
122 report a backup failure. (Backups aren't any good if they don't
123 actually back up the right thing. And you stand a better chance of
124 fixing them if you know that they're going wrong.)
126 Commit the backup, by renaming the dump directory to
131 .IB host / fs / date .fshash \fR.
133 The backup is now complete.
134 .SS Configuration commands
135 The configuration file is simply a Bash shell fragment: configuration
136 commands are shell functions.
138 .BI "backup " "fs\fR[:\fIfsarg\fR] ..."
139 Back up the named filesystems. The corresponding
141 may be required by the snapshot type.
146 commands will back up filesystems on the named
148 To back up filesystems on the backup server itself, use its hostname:
150 will avoid inefficient and pointless messing about
153 This command clears the
157 .BI "like " "host\fR ..."
158 Declare that subsequent filesystems are `similar' to like-named
159 filesystems on the named
163 should use those trees as potential sources of hardlinkable files. Be
164 careful when using this option without
167 option: an erroneous hardlink will cause the backup to fail. (The
168 backup won't be left silently incorrect.)
170 .BI "retain " frequency " " duration
171 Define part a backup retention policy: backup trees of the
173 should be kept for the
185 which means the same); the
193 Expiry considers each existing dump against the policy lines in order:
194 the last applicable line determines the dump's fate \(en so you should
195 probably write the lines in decreasing order of duration.
200 snapshot type (see below) doesn't prevent a filesystem from being
201 modified while it's being backed up. If this happens, the
203 pass will detect the difference and fail. If the filesystem in question
204 is relatively quiescent, then maybe retrying the backup will result in a
205 successful consistent copy. Following this command, a backup which
208 mismatch will be retried up to
210 times before being declared a failure.
212 .BI "snap " type " " \fR[\fIargs\fR...]
215 for subsequent backups. Some snapshot types require additional
216 arguments, which may be supplied here. This command clears the
219 .SS Configuration variables
220 The following shell variables may be overridden by the configuration
224 The number of log files to be kept for each filesystem. Old logfiles
225 are deleted to keep the total number below this bound. The default
229 Command-line options to pass to
231 in addition to the basic set:
238 .B \-\-one-file-system
240 .BR "\-\-filter=""dir-merge .rsync-backup""" .
247 snapshots are mounted on subdirectories below the
249 .IR "on backup clients" .
254 is the backup mount directory configured at build time.
257 The volume size option to pass to
259 when creating a snapshot. The default is
261 which seems to work fairly well.
264 Where the actual backup trees should be stored. See the section on
271 is the backup mount directory configured at build time.
274 The hash function to use for verifying archive integrity. This is
279 so it must name one of the hash functions supported by your Python's
281 module. The default is
284 The configuration file may define shell functions to perform custom
285 actions at various points in the backup process.
287 .BI "backup_precommit_hook " host " " fs " " date
288 Called after a backup has been verified complete and about to be
289 committed. The backup tree is in
291 in the current directory, and the
295 A typical action would be to create a digital signature on the
298 .BI "backup_commit_hook " host " " fs " " date
299 Called during the commit procedure. The backup tree and manifest have
300 been renamed into their proper places. Typically one would use this
301 hook to rename files created by the
302 .B backup_precommit_hook
305 .BR "whine " [ \-n ] " " \fItext\fR...
306 Called to report `interesting' events when the
308 option is in force. The default action is to echo the
310 to (what was initially) standard output, followed by a newline unless
314 The following snapshot types are available.
317 A trivial snapshot type: attempts to back up a live filesystem. How
318 well this works depends on how active the filesystem is. If files
319 change while the dump is in progress then the
321 verification will likely fail. Backups using this snapshot type must
322 specify the filesystem mount point as the
326 A slightly less trivial snapshot type: make the filesystem read-only
327 while the dump is in progress. Backups using this snapshot type must
328 specify the filesystem mount point as the
332 Create snapshots using LVM. The snapshot argument is interpreted as the
333 relevant volume group. The filesystem name is interpreted as the origin
334 volume name; the snapshot will be called
337 .IB SNAPDIR / fs \fR;
338 space will be allocated to it according to the
342 .BI "rfreezefs " client " " vg
343 This gets complicated. Suppose that a server has an LVM volume group,
344 and exports (somehow) a logical volume to a client. Examples are a host
345 providing a virtual disk to a guest, or a server providing
346 network-attached storage to a client. The server can create a snapshot
347 of the volume using LVM, but must synchronize with the client to ensure
348 that the filesystem image captured in the snapshot is clean. The
350 program should be installed on the client to perform this rather
351 delicate synchronization. Declare the server using the
353 command as usual; pass the client's name as the
356 server's volume group name as the
358 snapshot arguments. Finally, backups using this snapshot type must
359 specify the filesystem mount point (or, actually, any file in the
360 filesystem) on the client, as the
363 Additional snapshot types can be defined in the configuration file. A
364 snapshot type requires two shell functions.
366 .BI snap_ type " " snapargs " " fs " " fsarg
367 Create the snapshot, and write the mountpoint (on the client host) to
368 standard output, in a form suitable as an argument to
371 .BI unsnap_ type " " snapargs " " fs " " fsarg
374 There are a number of utility functions which can be used by snapshot
375 type handlers: please see the script for details. Please send the
376 author interesting snapshot handlers for inclusion in the main
378 .SS Archive structure
379 Backup trees are stored in a fairly straightforward directory tree.
381 At the top level is one directory for each client host. There are also
382 some special entries:
384 .B \&.rsync-backup-store
385 This file must be present in order to indicate that a backup volume is
386 present (and not just an empty mount point).
389 The cache database used for improving performance of local file
390 hashing. There may be other
392 files used by SQLite for its own purposes.
395 Part of the filesystem used on the backup volume. You don't want to
399 Used to store temporary files during the backup process. (Some of them
400 want to be on the same filesystem as the rest of the backup.) When
401 things go wrong, files are left behind in the hope that they might help
402 someone debug the mess. It's always safe to delete the files in here
403 when no backup is running.
405 So don't use those names for your hosts.
407 The next layer down contains a directory for each filesystem on the given host.
409 The bottom layer contains a directory for each dump of that filesystem,
410 named with the date at which the dump was started (in ISO8601
411 .IB yyyy \(en mm \(en dd
412 format), together with associated files named
421 Mark Wooding, <mdw@distorted.org.uk>
~mdw / rsync-backup / blob