fshash.in: Save and restore working directory when walking.
[rsync-backup] / rsync-backup.8
1 .TH rsync-backup 8 "7 October 2012" rsync-backup
2 .SH SYNOPSIS
3 .B rsync-backup
4 .RB [ \-v ]
5 .RB [ \-c
6 .IR config-file ]
7 .SH DESCRIPTION
8 The
9 .B rsync-backup
10 script is a backup program of the currently popular
11 .RB ` rsync (1)
12 .BR \-\-link-dest '
13 variety. It uses
14 .BR rsync 's
15 ability to create hardlinks from (apparently) similar existing local
16 trees to make incremental dumps efficient, even from remote sources.
17 Restoring files is easy because the backups created are just directories
18 full of files, exactly as they were on the source \(en and this is
19 verified using the
20 .BR fshash (1)
21 program.
22 .PP
23 The script does more than just running
24 .BR rsync .
25 It is also responsible for creating and removing snapshots of volumes to
26 be backed up, and expiring old dumps according to a user-specified
27 retention policy.
28 .SS Installation
29 The idea is that the
30 .B rsync-backup
31 script should be installed and run on a central backup server with local
32 access to the backup volumes.
33 .PP
34 The script should be run with full (root) privileges, so that it can
35 correctly record file ownership information. The server should also be
36 able to connect via
37 .BR ssh (1)
38 to the client machines, and run processes there as root. (This is not a
39 security disaster. Remember that the backup server is, in the end,
40 responsible for the integrity of the backup data. A dishonest backup
41 server can easily compromise a client which is being restored from
42 corrupt backup data.)
43 .PP
44 The
45
46
47 .SS Configuration commands
48 The configuration file is simply a Bash shell fragment: configuration
49 commands are shell functions.
50 .TP
51 .BI "backup " "fs\fR[:\fIfsarg\fR] ..."
52 Back up the named filesystems. The corresponding
53 .IR fsarg s
54 may be required by the snapshot type.
55 .TP
56 .BI "host " host
57 Future
58 .B backup
59 commands will back up filesystems on the named
60 .IR host .
61 This clears the
62 .B like
63 list.
64 .TP
65 .BI "like " "host\fR ..."
66 Declare that subsequent filesystems are `similar' to like-named
67 filesystems on the named
68 .IR host s,
69 and that
70 .B rsync
71 should use those trees as potential sources of hardlinkable files. Be
72 careful when using this option without
73 .BR rsync 's
74 .B \-\-checksum
75 option: an erroneous hardlink will cause the backup to fail. (The
76 backup won't be left silently incorrect.)
77 .TP
78 .BI "retain " frequency " " duration
79 Define part a backup retention policy: backup trees of the
80 .I frequency
81 should be kept for the
82 .IR duration .
83 The
84 .I frequency
85 can be
86 .BR daily ,
87 .BR weekly ,
88 .BR monthly ,
89 or
90 .B annually
91 (or
92 .BR yearly ,
93 which means the same); the
94 .I duration
95 may be any of
96 .BR week ,
97 .BR month ,
98 .BR year ,
99 or
100 .BR forever .
101 Expiry considers each existing dump against the policy lines in order:
102 the last applicable line determines the dump's fate \(en so you should
103 probably write the lines in decreasing order of duration.
104 .TP
105 .BI "snap " type " " \fR[\fIargs\fR...]
106 Use the snapshot
107 .I type
108 for subsequent backups. Some snapshot types require additional
109 arguments, which may be supplied here.
110 .SS Configuration variables
111 The following shell variables may be overridden by the configuration
112 file.
113 .TP
114 .B MAXLOG
115 The number of log files to be kept for each filesystem. Old logfiles
116 are deleted to keep the total number below this bound. The default
117 value is 14.
118 .TP
119 .B RSYNCOPTS
120 Command-line options to pass to
121 .BR rsync (1)
122 in addition to the basic set:
123 .B \-\-archive \-\-hard-links \-\-numeric-ids \-\-del \-\-sparse
124 .B \-\-compress \-\-one-file-system \-\-partial
125 .B \-\-filter="dir-merge .rsync-backup"
126 The default is
127 .BR \-\-verbose .
128 .TP
129 .B SNAPDIR
130 LVM (and
131 .BR rfreezefs )
132 snapshots are mounted on subdirectories below the
133 .B SNAPDIR
134 .IR "on backup clients" .
135 The default is
136 .IB mntbkpdir /snap
137 where
138 .I mntbkpdir
139 is the backup mount directory configured at build time.
140 .TP
141 .B SNAPSIZE
142 The volume size option to pass to
143 .BR lvcreate (8)
144 when creating a snapshot. The default is
145 .B \-l10%ORIGIN
146 which seems to work fairly well.
147 .TP
148 .B STOREDIR
149 Where the actual backup trees should be stored. See the section on
150 .B Archive structure
151 below.
152 The default is
153 .IB mntbkpdir /store
154 where
155 .I mntbkpdir
156 is the backup mount directory configured at build time.
157 .TP
158 .B HASH
159 The hash function to use for verifying archive integrity. This is
160 passed to the
161 .B \-H
162 option of
163 .BR fshash ,
164 so it must name one of the hash functions supported by your Python's
165 .B hashlib
166 module. The default is
167 .BR sha256 .
168 .SS Hook functions
169 The configuration file may define shell functions to perform custom
170 actions at various points in the backup process.
171 .TP
172 .BI "backup_precommit_hook " host " " fs " " date
173 Called after a backup has been verified complete and about to be
174 committed. The backup tree is in
175 .B new
176 in the current directory, and the
177 .B fshash
178 manifest is in
179 .BR new.fshash .
180 A typical action would be to create a digital signature on the
181 manifest.
182 .TP
183 .BI "backup_commit_hook " host " " fs " " date
184 Called during the commit procedure. The backup tree and manifest have
185 been renamed into their proper places. Typically one would use this
186 hook to rename files created by the
187 .B backup_precommit_hook
188 function.
189 .TP
190 .BR "whine " [ \-n ] " " \fItext\fR...
191 Called to report `interesting' events when the
192 .B \-v
193 option is in force. The default action is to echo the
194 .I text
195 to (what was initially) standard output, followed by a newline unless
196 .B \-n
197 is given.
198 .SS Snapshot types
199 The following snapshot types are available.
200 .TP
201 .B live
202 A trivial snapshot type: attempts to back up a live filesystem. How
203 well this works depends on how active the filesystem is. If files
204 change while the dump is in progress then the
205 .B fshash
206 verification will likely fail. Backups using this snapshot type must
207 specify the filesystem mount point as the
208 .IR fsarg .
209 .TP
210 .B ro
211 A slightly less trivial snapshot type: make the filesystem read-only
212 while the dump is in progress. Backups using this snapshot type must
213 specify the filesystem mount point as the
214 .IR fsarg .
215 .TP
216 .BI "lvm " vg
217 Create snapshots using LVM. The snapshot argument is interpreted as the
218 relevant volume group. The filesystem name is interpreted as the origin
219 volume name; the snapshot will be called
220 .IB fs .bkp
221 and mounted on
222 .IB SNAPDIR / fs \fR;
223 space will be allocated to it according to the
224 .I SNAPSIZE
225 variable.
226 .TP
227 .BI "rfreezefs " client " " vg
228 This gets complicated. Suppose that a server has an LVM volume group,
229 and exports (somehow) a logical volume to a client. Examples are a host
230 providing a virtual disk to a guest, or a server providing
231 network-attached storage to a client. The server can create a snapshot
232 of the volume using LVM, but must synchronize with the client to ensure
233 that the filesystem image captured in the snapshot is clean. The
234 .BR rfreezefs (8)
235 program should be installed on the client to perform this rather
236 delicate synchronization. Declare the server using the
237 .B host
238 command as usual; pass the client's name as the
239 .I client
240 and the
241 server's volume group name as the
242 .I vg
243 snapshot arguments. Finally, backups using this snapshot type must
244 specify the filesystem mount point (or, actually, any file in the
245 filesystem) on the client, as the
246 .IR fsarg .
247 .PP
248 Additional snapshot types can be defined in the configuration file. A
249 snapshot type requires two shell functions.
250 .TP
251 .BI snap_ type " " snapargs " " fs " " fsarg
252 Create the snapshot, and write the mountpoint (on the client host) to
253 standard output, in a form suitable as an argument to
254 .BR rsync .
255 .TP
256 .BI unsnap_ type " " snapargs " " fs " " fsarg
257 Remove the snapshot.
258 .PP
259 There are a number of utility functions which can be used by snapshot
260 type handlers: please see the script for details. Please send the
261 author interesting snapshot handlers for inclusion in the main
262 distribution.
263 .SS Archive structure