rsync-backup
6 years agoRelease 1.1.1. master 1.1.1
Mark Wooding [Wed, 22 Nov 2017 23:29:37 +0000 (23:29 +0000)]
Release 1.1.1.

6 years agodebian/control: Add missing `Build-Depends: pkg-config, python'.
Mark Wooding [Wed, 22 Nov 2017 23:17:42 +0000 (23:17 +0000)]
debian/control: Add missing `Build-Depends: pkg-config, python'.

6 years agocheck-bkp-status.in: Force `sqlite3' output options to be correct.
Mark Wooding [Fri, 7 Jul 2017 10:01:40 +0000 (11:01 +0100)]
check-bkp-status.in: Force `sqlite3' output options to be correct.

If the user has a `.sqlite3' file which overrides the default output
options (quite likely, because the default format is rather ugly) then
`check-bkp-status' gets very confused.  So add a bunch of options:

  * `-batch' suppresses the annoying `Loading resources from
    HOME/.sqliterc' banner;

  * `-list' forces thing-separated items rather than tabulated output;

  * `-separator |' sets the thing which separates items; and

  * `-noheader' suppresses the column headings.

Honestly there ought to be a `-norc' option, but there isn't.

6 years agodebian: Update for Debhelper 10.
Mark Wooding [Fri, 7 Jul 2017 10:01:32 +0000 (11:01 +0100)]
debian: Update for Debhelper 10.

6 years agoMakefile.am: Tweak `silent-rules' machinery.
Mark Wooding [Fri, 7 Jul 2017 10:00:28 +0000 (11:00 +0100)]
Makefile.am: Tweak `silent-rules' machinery.

Since Automake 1.11, the advice for setting up custom silent-rules
recipes has changed, so use the new machinery.

Also, I'm no longer mainly working on wheezy, and Automake has made the
operation field two spaces wider while I wasn't looking, so make the
output line up properly.

This means that Catacomb now requires Automake 1.11.2 or later to build
from the Git tree.

6 years agodebian/copyright: Convert to machine-readable format.
Mark Wooding [Fri, 7 Jul 2017 09:46:55 +0000 (10:46 +0100)]
debian/copyright: Convert to machine-readable format.

8 years agoRelease 1.1.0. 1.1.0
Mark Wooding [Thu, 1 Oct 2015 07:26:31 +0000 (08:26 +0100)]
Release 1.1.0.

8 years agorsync-backup.in: Retry unmounting snapshot filesystems a few times.
Mark Wooding [Mon, 29 Jun 2015 09:45:53 +0000 (10:45 +0100)]
rsync-backup.in: Retry unmounting snapshot filesystems a few times.

Not quite sure what's making this fail, but it's happening annoyingly
frequently.

8 years agorsync-backup.in: Fix stupid bugs introduced in ec1d1af...
Mark Wooding [Sun, 31 May 2015 01:37:11 +0000 (02:37 +0100)]
rsync-backup.in: Fix stupid bugs introduced in ec1d1af...

  * Use `log', not `$log'.
  * Hash the correct file for the remote checksum.
  * Log the correct remote hash.

8 years agorsync-backup.in: Add double-checking for the manifest comparison.
Mark Wooding [Sat, 30 May 2015 11:13:46 +0000 (12:13 +0100)]
rsync-backup.in: Add double-checking for the manifest comparison.

While hacking on 1c0d861..., I temporarily introduced a bug which failed
to spot differences between the local and remote fshash(1) manifests.
Add an explicit check of file hashes as a backstop to guard against this
kind of bug now or in the future.

Also dump the hashes into the log, so that one can verify after the fact
that everything actually worked properly.

8 years agoMerge us:git/staging/rsync-backup
Mark Wooding [Sat, 30 May 2015 09:14:12 +0000 (10:14 +0100)]
Merge us:git/staging/rsync-backup

* us:git/staging/rsync-backup:
  rsync-backup.in, rsync-backup.8: Trim caches on fshash mismatch.
  rsync-backup.in (run): Option to preserve standard input.
  fshash.in, fshash.1: Evict cache entries identified by diff files.

Conflicts:
fshash.1

8 years agorsync-backup.in, rsync-backup.8: Trim caches on fshash mismatch.
Mark Wooding [Fri, 10 Apr 2015 14:19:25 +0000 (15:19 +0100)]
rsync-backup.in, rsync-backup.8: Trim caches on fshash mismatch.

  * Capture the diff output in a file so we can pass it onto fshash for
    cache pruning if it's not empty.

  * If we're retrying because of an fshash mismatch then clear cache
    entries.  This is delayed from the cache-mismatch processing because
    we need the snapshot mounted so that fshash can do name->inode
    mapping.

  * By default, retry once.  Otherwise we don't get any benefit from
    evicting maybe-stale entries.

8 years agorsync-backup.in (run): Option to preserve standard input.
Mark Wooding [Fri, 10 Apr 2015 14:19:25 +0000 (15:19 +0100)]
rsync-backup.in (run): Option to preserve standard input.

Usually we close stdin and redirect from /dev/null so that SSH doesn't
hang forever; but sometimes it's actually important.

8 years agofshash.in, fshash.1: Evict cache entries identified by diff files.
Mark Wooding [Fri, 10 Apr 2015 14:19:25 +0000 (15:19 +0100)]
fshash.in, fshash.1: Evict cache entries identified by diff files.

A shiny new option which we can use to recover automatically from
cache-staleness problems.

9 years agoRelease 1.0.2.4. 1.0.2.4
Mark Wooding [Wed, 18 Mar 2015 09:44:54 +0000 (09:44 +0000)]
Release 1.0.2.4.

9 years agocheck-bkp-status.in: Dropping for not looking crap on mobile browsers.
Mark Wooding [Tue, 3 Mar 2015 21:48:22 +0000 (21:48 +0000)]
check-bkp-status.in: Dropping for not looking crap on mobile browsers.

9 years agoRelease 1.0.2.3. 1.0.2.3
Mark Wooding [Mon, 22 Dec 2014 20:32:58 +0000 (20:32 +0000)]
Release 1.0.2.3.

9 years agorfreezefs.c: Use the `N()' macro.
Mark Wooding [Mon, 22 Dec 2014 20:32:58 +0000 (20:32 +0000)]
rfreezefs.c: Use the `N()' macro.

Rather than writing the magic out longhand.

9 years agorfreezefs.c (partial_cleanup): Don't ignore the argument.
Mark Wooding [Mon, 22 Dec 2014 20:32:58 +0000 (20:32 +0000)]
rfreezefs.c (partial_cleanup): Don't ignore the argument.

Previously, the function would always attempt to thaw all of the
filesystems, even if they hadn't all been frozen yet.  There's only a
problem if something actually goes wrong during freezing some
filesystem, and this probably isn't too disastrous, assuming that the
filesystems in question weren't already frozen (and are now unexpectedly
thawed), but it's not desirable.

9 years agorfreezefs.c: Minor formatting tweak.
Mark Wooding [Mon, 22 Dec 2014 20:32:58 +0000 (20:32 +0000)]
rfreezefs.c: Minor formatting tweak.

9 years agorfreezefs.8: The description of exit status 3 referred to `cookies'.
Mark Wooding [Mon, 22 Dec 2014 20:32:58 +0000 (20:32 +0000)]
rfreezefs.8: The description of exit status 3 referred to `cookies'.

This came from an early version of the program (like the old help
message, fixed in 6952bdd...).

9 years agorfreezefs.8: Fix some misformatting and other minor bogosity.
Mark Wooding [Mon, 22 Dec 2014 20:32:58 +0000 (20:32 +0000)]
rfreezefs.8: Fix some misformatting and other minor bogosity.

9 years agofshash.1: Fix some garbled text.
Mark Wooding [Mon, 2 Feb 2015 21:40:28 +0000 (21:40 +0000)]
fshash.1: Fix some garbled text.

9 years agoRelease 1.0.2.2. 1.0.2.2
Mark Wooding [Wed, 8 Oct 2014 11:33:06 +0000 (12:33 +0100)]
Release 1.0.2.2.

10 years ago.gitattributes: Don't expect tabs in Python files.
Mark Wooding [Sat, 5 Apr 2014 13:26:31 +0000 (14:26 +0100)]
.gitattributes: Don't expect tabs in Python files.

10 years agocheck-bkp-status.in: Further fettling of log hiding and showing.
Mark Wooding [Sat, 13 Jul 2013 15:34:40 +0000 (16:34 +0100)]
check-bkp-status.in: Further fettling of log hiding and showing.

If a log display is already shown, and then selected from the master
table, then don't mark it for auto-hiding.

10 years agocheck-bkp-status.in: Function `elt_class_p' tests element class.
Mark Wooding [Sat, 13 Jul 2013 15:34:40 +0000 (16:34 +0100)]
check-bkp-status.in: Function `elt_class_p' tests element class.

This is used twice already, and we'll want another one soon.

10 years agoRelease 1.0.2.1. 1.0.2.1
Mark Wooding [Sun, 16 Mar 2014 12:09:07 +0000 (12:09 +0000)]
Release 1.0.2.1.

10 years agocheck-bkp-status.in: Zap trailing space in footer.
Mark Wooding [Wed, 12 Mar 2014 14:30:05 +0000 (14:30 +0000)]
check-bkp-status.in: Zap trailing space in footer.

Grrr.

10 years agocheck-bkp-status.in: Zap trailing . in footer message.
Mark Wooding [Mon, 10 Mar 2014 10:13:23 +0000 (10:13 +0000)]
check-bkp-status.in: Zap trailing . in footer message.

10 years agoRelease 1.0.2. 1.0.2
Mark Wooding [Sun, 9 Mar 2014 01:50:57 +0000 (01:50 +0000)]
Release 1.0.2.

10 years agocheck-bkp-status.in: Height cap on log information.
Mark Wooding [Thu, 6 Mar 2014 14:01:05 +0000 (14:01 +0000)]
check-bkp-status.in: Height cap on log information.

Mainly because this way you can get to the horizontal scroll bar.

10 years agocheck-bkp-status.in: Tweak auto-collapse of linked log sections.
Mark Wooding [Thu, 6 Mar 2014 11:58:36 +0000 (11:58 +0000)]
check-bkp-status.in: Tweak auto-collapse of linked log sections.

If a log is manually collapsed then forget it as being the auto-expanded
section so that if it gets manually expanded again then it doesn't get
auto-collapsed.

10 years agocheck-bkp-status.in: JavaScript to expand/contract the log reports.
Mark Wooding [Thu, 6 Mar 2014 11:12:50 +0000 (11:12 +0000)]
check-bkp-status.in: JavaScript to expand/contract the log reports.

Should work without JavaScript.

10 years agocheck-bkp-status.in: Include day-of-week name in the title.
Mark Wooding [Thu, 6 Mar 2014 09:58:47 +0000 (09:58 +0000)]
check-bkp-status.in: Include day-of-week name in the title.

10 years agorsync-backup.in: Factor out log file expiry into a separate function.
Mark Wooding [Thu, 6 Mar 2014 09:58:08 +0000 (09:58 +0000)]
rsync-backup.in: Factor out log file expiry into a separate function.

No real use for this yet, but it's tidier this way.

10 years agoRelease 1.0.1. 1.0.1
Mark Wooding [Thu, 16 Jan 2014 10:20:21 +0000 (10:20 +0000)]
Release 1.0.1.

10 years agocheck-bkp-status: New program writes reports about backups.
Mark Wooding [Thu, 16 Jan 2014 10:06:33 +0000 (10:06 +0000)]
check-bkp-status: New program writes reports about backups.

10 years agorsync-backup.in, lib.sh.in: Move some definitions to a library.
Mark Wooding [Thu, 16 Jan 2014 09:55:53 +0000 (09:55 +0000)]
rsync-backup.in, lib.sh.in: Move some definitions to a library.

We'll want these for another program, coming soon.

10 years agorsync-backup.{in,8}: Introduce `hook' concept.
Mark Wooding [Thu, 16 Jan 2014 09:51:56 +0000 (09:51 +0000)]
rsync-backup.{in,8}: Introduce `hook' concept.

I want to make the configuration file so that it can be read by
scripts which don't actually want to run a backup.  Therefore, code
which currently runs unconditionally to (e.g.) copy metadata onto the
backup volume must be wrapped up in a function.

To make this a bit more disciplined, introduce a general idea of hooks
which are run with arguments.  The existing `backup_precommit_hook'
and `backup_commit_hook' functions are subsumed by this, still
provided for compatibility.  New hooks are added at the start of the
whole backup, at the beginning of each filesystem dump, and at the end
of the whole operation.

10 years agorsync-backup.in: Add `-n' option to usage synopsis.
Mark Wooding [Thu, 16 Jan 2014 09:49:08 +0000 (09:49 +0000)]
rsync-backup.in: Add `-n' option to usage synopsis.

10 years agorsync-backup.8: Fix some source-level formatting.
Mark Wooding [Thu, 16 Jan 2014 10:13:11 +0000 (10:13 +0000)]
rsync-backup.8: Fix some source-level formatting.

10 years agoRelease 1.0.0. 1.0.0
Mark Wooding [Sat, 20 Jul 2013 17:42:48 +0000 (18:42 +0100)]
Release 1.0.0.

10 years agodebian: Split into multiple packages.
Mark Wooding [Tue, 16 Jul 2013 18:16:50 +0000 (19:16 +0100)]
debian: Split into multiple packages.

10 years agorfreezefs.c: Fix the help message, which was egregiously wrong.
Mark Wooding [Tue, 16 Jul 2013 17:26:13 +0000 (18:26 +0100)]
rfreezefs.c: Fix the help message, which was egregiously wrong.

10 years agoUse new mLib function annotations.
Mark Wooding [Tue, 16 Jul 2013 13:32:14 +0000 (14:32 +0100)]
Use new mLib function annotations.

And fix a minor bug caught by them.

10 years agoMake `rfreezefs' optional, conditioned on whether we find mLib.
Mark Wooding [Tue, 16 Jul 2013 10:49:30 +0000 (11:49 +0100)]
Make `rfreezefs' optional, conditioned on whether we find mLib.

10 years agorsync-backup.{in,8}: New option to set the remote user name.
Mark Wooding [Mon, 15 Jul 2013 13:16:14 +0000 (14:16 +0100)]
rsync-backup.{in,8}: New option to set the remote user name.

The default is still to use no user name.

10 years agorsync-backup.in: Factor out the machinery for running a backup command.
Mark Wooding [Mon, 18 Mar 2013 20:12:00 +0000 (20:12 +0000)]
rsync-backup.in: Factor out the machinery for running a backup command.

In particular, we want the piece which messes with log files and checks
whether the command worked.

10 years agorsync-backup.in: Handle backslashes correctly in command output.
Mark Wooding [Fri, 14 Jun 2013 23:07:56 +0000 (00:07 +0100)]
rsync-backup.in: Handle backslashes correctly in command output.

11 years agorsync-backup.in: Fix stupid bug in index deletion.
Mark Wooding [Wed, 20 Mar 2013 16:59:02 +0000 (16:59 +0000)]
rsync-backup.in: Fix stupid bug in index deletion.

11 years agofshash.1: Fix some stupid editorial errors.
Mark Wooding [Thu, 28 Feb 2013 16:49:27 +0000 (16:49 +0000)]
fshash.1: Fix some stupid editorial errors.

11 years agoupdate-bkp-index.in: The tuple (host, fs, date) is not unique.
Mark Wooding [Mon, 28 Jan 2013 10:44:29 +0000 (10:44 +0000)]
update-bkp-index.in: The tuple (host, fs, date) is not unique.

So we must fix the schema, and include the volume name in the primary
key.  This is still early days, so I'm just going to break it.

11 years agoMerge branch 'staging'
Mark Wooding [Mon, 28 Jan 2013 10:04:55 +0000 (10:04 +0000)]
Merge branch 'staging'

* staging:
  Maintain an index of backup artifacts.
  rsync-backup.8: Put the variables in alphabetical order.
  rsync-backup.in: Missing `NAME' section.

11 years agorsync-backup.8: Describe `HOST/FS/last' symlink.
Mark Wooding [Fri, 25 Jan 2013 22:56:13 +0000 (22:56 +0000)]
rsync-backup.8: Describe `HOST/FS/last' symlink.

11 years agorsync-backup.8: Missing `.RS'/`.RE' in description of `retain'.
Mark Wooding [Fri, 25 Jan 2013 22:55:37 +0000 (22:55 +0000)]
rsync-backup.8: Missing `.RS'/`.RE' in description of `retain'.

11 years agoMaintain an index of backup artifacts.
Mark Wooding [Fri, 25 Jan 2013 18:33:03 +0000 (18:33 +0000)]
Maintain an index of backup artifacts.

There's a new program `update-bkp-index' to create and refresh the
database from a backup volume, and some new pieces of `rsync-backup' to
update the index incrementally as artifacts are committed and expired.

11 years agorsync-backup.8: Put the variables in alphabetical order.
Mark Wooding [Fri, 25 Jan 2013 18:31:15 +0000 (18:31 +0000)]
rsync-backup.8: Put the variables in alphabetical order.

11 years agorsync-backup.in: Missing `NAME' section.
Mark Wooding [Fri, 25 Jan 2013 18:27:46 +0000 (18:27 +0000)]
rsync-backup.in: Missing `NAME' section.

11 years agorsync-backup.8, rsync-backup.in: Allow per-backup retention policy.
Mark Wooding [Fri, 14 Dec 2012 23:07:33 +0000 (23:07 +0000)]
rsync-backup.8, rsync-backup.in: Allow per-backup retention policy.

Now stuff like the media library doesn't keep old mistakes and changed
files forever, but we can still keep lots of history for most of the
filesystems.

11 years agorsync-backup.8, rsync-backup.in: New dry-run mode for testing logic.
Mark Wooding [Fri, 14 Dec 2012 23:06:10 +0000 (23:06 +0000)]
rsync-backup.8, rsync-backup.in: New dry-run mode for testing logic.

There's a tweak to retention policy coming up, and it turns out to be bad
to test such things on one's production backups.

11 years agorsync-backup.in, rsync-backup.8: Retry backups which fail fshash check.
Mark Wooding [Fri, 19 Oct 2012 08:46:31 +0000 (09:46 +0100)]
rsync-backup.in, rsync-backup.8: Retry backups which fail fshash check.

I'm getting annoyed with vampire:root failing in the morning and then
succeeding after a manual retry.  So get the script to retry automatically.

11 years agorsync-backup.in, rsync-backup.8: Ensure the store filesystem is mounted.
Mark Wooding [Mon, 8 Oct 2012 08:12:23 +0000 (09:12 +0100)]
rsync-backup.in, rsync-backup.8: Ensure the store filesystem is mounted.

Previously this was left as the responsibility of some wrapper script,
but that's really annoying, and we should do it ourselves.

11 years agoDebianization!
Mark Wooding [Sun, 7 Oct 2012 22:40:15 +0000 (23:40 +0100)]
Debianization!

11 years agoFirst cut at documentation. 0.99.1
Mark Wooding [Sun, 7 Oct 2012 21:20:47 +0000 (22:20 +0100)]
First cut at documentation.

11 years agofshash.in: Save and restore working directory when walking.
Mark Wooding [Sun, 7 Oct 2012 21:20:04 +0000 (22:20 +0100)]
fshash.in: Save and restore working directory when walking.

11 years agofshash.in: Remove ambiguity for filenames containing ` -> '.
Mark Wooding [Sun, 7 Oct 2012 14:52:10 +0000 (15:52 +0100)]
fshash.in: Remove ambiguity for filenames containing ` -> '.

The problem is that we can't tell which occurrence is the separator for
a symbolic link.  Fix this by adding an otherwise pointless backslash.

11 years agoInitial commit.
Mark Wooding [Sun, 7 Oct 2012 13:26:23 +0000 (14:26 +0100)]
Initial commit.

This is still a little sketchy.  Some pieces are lacking documentation.
But the basic code is currently in production use, so it can't be that
bad.