sgt/agedu
11 years agoChange the magic number used to introduce a trie file, so that instead master
simon [Sat, 8 Dec 2012 12:05:33 +0000 (12:05 +0000)]
Change the magic number used to introduce a trie file, so that instead
of a single unsigned long it's a big structure filled with various
integer and pointer types. The idea is that when we check it on
reading an existing index file, it should point out any ABI mismatches
between the platform which generated the index and the one reading it.

In the course of this work I've had to actually add code to check the
magic numbers, which previously weren't checked at all and so would
have been useful only to file(1).

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9723 cda61777-01e9-0310-a592-d414129be87e

11 years agoStop passing "http" to getaddrinfo in the case when we only need an IP
simon [Fri, 7 Dec 2012 19:04:20 +0000 (19:04 +0000)]
Stop passing "http" to getaddrinfo in the case when we only need an IP
address. It's legal to pass NULL as a service string, so I don't know
why I didn't do that in the first place.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9722 cda61777-01e9-0310-a592-d414129be87e

11 years agoProvide a fallback definition of HOST_NAME_MAX, in case one isn't
simon [Fri, 7 Dec 2012 19:03:50 +0000 (19:03 +0000)]
Provide a fallback definition of HOST_NAME_MAX, in case one isn't
provided in the system headers along with the definition of
gethostname().

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9721 cda61777-01e9-0310-a592-d414129be87e

11 years agoFix a memory access bug in the trie construction. When we attempt to
simon [Mon, 29 Oct 2012 18:33:46 +0000 (18:33 +0000)]
Fix a memory access bug in the trie construction. When we attempt to
examine the partially built switch node at a given depth, we must
abort the attempt if that depth is _at least_ tb->switchsize, not just
if it's greater (since, as usual, elements in the array exist up to
but not including tb->switchsize).

This was reported as a segfault by an AIX user recently, but turns out
not to be a platform-specific issue: valgrind confirms that it's wrong
on Linux too, even though it hasn't happened to explode for anyone.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9693 cda61777-01e9-0310-a592-d414129be87e

11 years agoRename my internal macros STAT and LSTAT (defined to stat/lstat or
simon [Fri, 26 Oct 2012 19:14:10 +0000 (19:14 +0000)]
Rename my internal macros STAT and LSTAT (defined to stat/lstat or
stat64/lstat64 depending on autoconf) to something else, since an
AIX-based user has given me reason to think that one of the AIX system
headers treads on the former names.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9691 cda61777-01e9-0310-a592-d414129be87e

11 years agoAdd the -a, -h and -V options to the man page, which previously
simon [Wed, 19 Sep 2012 16:57:03 +0000 (16:57 +0000)]
Add the -a, -h and -V options to the man page, which previously
mentioned -a only in passing (and didn't give it its own list item
with a clear description of syntax) and omitted -h and -V totally.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9671 cda61777-01e9-0310-a592-d414129be87e

11 years agoMinor copy edit in agedu's man page, and also added another paragraph
simon [Thu, 3 May 2012 17:49:20 +0000 (17:49 +0000)]
Minor copy edit in agedu's man page, and also added another paragraph
in the limitations section mentioning that it won't work with noatime.

(Of course that _should_ be obvious, but it's still worth mentioning
it as a thing to check someone else hasn't inflicted on you if agedu
mysteriously isn't doing anything useful, and it's also worth
commenting on relatime.)

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9480 cda61777-01e9-0310-a592-d414129be87e

11 years agoCheck the return value of ftruncate(), mostly to stop Ubuntu 12.04's
simon [Thu, 3 May 2012 17:46:25 +0000 (17:46 +0000)]
Check the return value of ftruncate(), mostly to stop Ubuntu 12.04's
gcc complaining.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9479 cda61777-01e9-0310-a592-d414129be87e

12 years agoRevamp the networking code in httpd.c to support IPv6. Side effects
simon [Sun, 4 Mar 2012 15:20:57 +0000 (15:20 +0000)]
Revamp the networking code in httpd.c to support IPv6. Side effects
include slight changes to the output saying where the URL is, and
support for port numbers specified as service names. Also I've
invented a nicer syntax 'ANY' for the magic server address 0.0.0.0.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9424 cda61777-01e9-0310-a592-d414129be87e

12 years agoFix typo in TODO.
simon [Sun, 4 Mar 2012 15:20:56 +0000 (15:20 +0000)]
Fix typo in TODO.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9423 cda61777-01e9-0310-a592-d414129be87e

12 years agoI've just realised that there was a lurking gotcha in the naming of
simon [Mon, 20 Feb 2012 22:40:27 +0000 (22:40 +0000)]
I've just realised that there was a lurking gotcha in the naming of
HTML files after the pathnames they represent: in recursive -H mode
(dumping a full set of HTML reports into the filesystem), there could
be a name clash between index.html representing the top level, and
index.html representing a subdirectory called 'index'.

Fix by detecting the clash and adding gratuitous escaping in the
latter case; this is not a fully general solution that will work with
any list of string formats, but it should be good enough for the
current set.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9417 cda61777-01e9-0310-a592-d414129be87e

12 years agoChris Walker reports that if you give a trailing slash on the pathname
simon [Mon, 20 Feb 2012 07:35:04 +0000 (07:35 +0000)]
Chris Walker reports that if you give a trailing slash on the pathname
you tell agedu to scan, it will store that slash in the index and
confuse parts of the rest of the code. Strip such slashes off before
beginning the scan.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9416 cda61777-01e9-0310-a592-d414129be87e

12 years agoChris Walker reports a silly typo in HTML reporting: the < and > signs
simon [Mon, 20 Feb 2012 07:30:02 +0000 (07:30 +0000)]
Chris Walker reports a silly typo in HTML reporting: the < and > signs
used in the mouseover text for file ages outside the colour range were
the wrong way round! I must have been thinking about comparison of
time_ts, in which case < does mean older, but by the time I print the
output it's in the form of an age, so > means older.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9415 cda61777-01e9-0310-a592-d414129be87e

12 years agoRevert one change from r8928 and instead apply the right answer:
simon [Tue, 31 Jan 2012 18:08:34 +0000 (18:08 +0000)]
Revert one change from r8928 and instead apply the right answer:
instead of initialising prevtf to NULL before the loop in which I
rebase it, I should take out the rebase statement completely since
it's overwritten immediately afterwards!

Spotted by clang-analyzer.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9394 cda61777-01e9-0310-a592-d414129be87e

12 years agoSimplify mkauto.sh to use autoreconf -i, by putting the 'foreign'
simon [Tue, 26 Jul 2011 18:39:25 +0000 (18:39 +0000)]
Simplify mkauto.sh to use autoreconf -i, by putting the 'foreign'
option in configure.ac where it doesn't have to be specified manually
every time.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9251 cda61777-01e9-0310-a592-d414129be87e

12 years agoMinor markup fix.
simon [Mon, 25 Jul 2011 18:23:13 +0000 (18:23 +0000)]
Minor markup fix.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9249 cda61777-01e9-0310-a592-d414129be87e

12 years agoFix assertion failure in standalone --html mode, which I must have
simon [Mon, 25 Jul 2011 18:18:52 +0000 (18:18 +0000)]
Fix assertion failure in standalone --html mode, which I must have
introduced when I revamped the HTML relative-URI handling.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9248 cda61777-01e9-0310-a592-d414129be87e

12 years agoTODO update: include some notes on proper character set conversion
simon [Mon, 25 Jul 2011 18:16:30 +0000 (18:16 +0000)]
TODO update: include some notes on proper character set conversion
support. I don't really know how this should work, and for precisely
that reason I thought I'd write down what it is that's hard about
it...

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9247 cda61777-01e9-0310-a592-d414129be87e

12 years agoSuggestion from James Beal: support a '--title' option to override the
simon [Mon, 25 Jul 2011 18:15:37 +0000 (18:15 +0000)]
Suggestion from James Beal: support a '--title' option to override the
'agedu:' prefix at the start of the title of output web pages.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9246 cda61777-01e9-0310-a592-d414129be87e

12 years agoDiscard the 'master' makefile, in favour of using Automake for all
simon [Mon, 25 Jul 2011 18:08:35 +0000 (18:08 +0000)]
Discard the 'master' makefile, in favour of using Automake for all
builds of agedu. Building the man page from the .but file is included
in the Automake makefile, but omitted if configure can't find halibut,
in which case 'make clean' also won't destroy the now-non-rebuildable
agedu.1.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9245 cda61777-01e9-0310-a592-d414129be87e

13 years agoEscape dots at the start of pathname components. (Mostly for HTML
simon [Wed, 13 Apr 2011 18:23:05 +0000 (18:23 +0000)]
Escape dots at the start of pathname components. (Mostly for HTML
multifile output mode, but I think it's at worst harmless to leave it
enabled in web server contexts too.)

Also I've just realised that if you type in a URL and guess wrongly
which characters agedu will have quoted, the parse function will work
it out anyway and give you a redirect to the canonical version of the
path. This was more or less accidental - an unanticipated consequence
of my parse-reformat-redirect strategy - but it seems thoroughly
useful. Add a comment pretending I meant to do it all along :-)

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9153 cda61777-01e9-0310-a592-d414129be87e

13 years agoSwitch all the HTML-based reporting modes (the internal httpd, the CGI
simon [Sat, 9 Apr 2011 21:01:01 +0000 (21:01 +0000)]
Switch all the HTML-based reporting modes (the internal httpd, the CGI
mode and the dump of static HTML files) to using URIs and filenames
based on the text of the pathname being reported on, rather than its
numeric index in the data file. The aim is that sub-URIs should remain
valid when the data is updated - if, for instance, you're running the
agedu CGI script permanently and changing the data file under it every
so often.

There's code in there to support the old numeric index behaviour, but
currently no UI to enable it. There could be easily enough, though.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9150 cda61777-01e9-0310-a592-d414129be87e

13 years ago'configure' apparently doesn't bump the timestamp on config.h if it
simon [Tue, 29 Mar 2011 18:01:42 +0000 (18:01 +0000)]
'configure' apparently doesn't bump the timestamp on config.h if it
didn't have to change it. Useful if you're constantly re-running
configure and want not to have to keep remaking everything, but when I
run configure _from_ my master makefile, it cuts the other way and I
want to avoid having the Makefile rerun configure every time. So
manually touch config.h after a successful run of configure.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9134 cda61777-01e9-0310-a592-d414129be87e

13 years agoAdd another thought to the docs for --mtime.
simon [Tue, 29 Mar 2011 18:00:21 +0000 (18:00 +0000)]
Add another thought to the docs for --mtime.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9133 cda61777-01e9-0310-a592-d414129be87e

13 years agoSlightly modified patch from James Beal: add --no-eof (or --noeof) as
simon [Fri, 17 Dec 2010 16:24:33 +0000 (16:24 +0000)]
Slightly modified patch from James Beal: add --no-eof (or --noeof) as
an option to the web server mode, to inhibit it from trying to read
stdin and treating EOF there as a cue to shut down.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9036 cda61777-01e9-0310-a592-d414129be87e

13 years agoWhen displaying sizes as a floating-point number (e.g. "123.4 Mb"),
simon [Wed, 17 Nov 2010 21:25:17 +0000 (21:25 +0000)]
When displaying sizes as a floating-point number (e.g. "123.4 Mb"),
don't compute that number in integer arithmetic and then cast it to
double!

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9025 cda61777-01e9-0310-a592-d414129be87e

13 years agoFiddle about with the configure script so it notices the need for
simon [Tue, 16 Nov 2010 18:10:05 +0000 (18:10 +0000)]
Fiddle about with the configure script so it notices the need for
-lnsl on Solaris.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@9024 cda61777-01e9-0310-a592-d414129be87e

13 years agoOne-character patch from Jacob to fix a missing > in the HTML output.
simon [Wed, 1 Sep 2010 17:02:13 +0000 (17:02 +0000)]
One-character patch from Jacob to fix a missing > in the HTML output.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8988 cda61777-01e9-0310-a592-d414129be87e

14 years agoMinor code cleanups suggested by NetBSD lint:
simon [Sun, 25 Apr 2010 07:40:36 +0000 (07:40 +0000)]
Minor code cleanups suggested by NetBSD lint:
 - some commas at the end of enum declarations removed.
 - some unused statements and variables removed
 - some more careful typing of query expressions in index.c.
   (Writing 'condition ? (ptrtype *)stuff : NULL' doesn't give the
   overall expression type 'ptrtype *'; it's better to put the cast
   outside the ?:.)

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8928 cda61777-01e9-0310-a592-d414129be87e

14 years agoMore TODO thoughts.
simon [Thu, 25 Feb 2010 18:53:47 +0000 (18:53 +0000)]
More TODO thoughts.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8888 cda61777-01e9-0310-a592-d414129be87e

14 years agoTypo.
simon [Wed, 24 Feb 2010 00:40:02 +0000 (00:40 +0000)]
Typo.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8885 cda61777-01e9-0310-a592-d414129be87e

14 years agoTODO updates.
simon [Tue, 23 Feb 2010 22:34:32 +0000 (22:34 +0000)]
TODO updates.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8884 cda61777-01e9-0310-a592-d414129be87e

14 years agoIntroduce a --cgi mode, to make it easy to plumb agedu's web
simon [Tue, 23 Feb 2010 22:34:20 +0000 (22:34 +0000)]
Introduce a --cgi mode, to make it easy to plumb agedu's web
reporting into an existing web server as an alternative to running a
dedicated one of its own.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8883 cda61777-01e9-0310-a592-d414129be87e

14 years agoI promised in the docs that the HTML output directory would be
simon [Mon, 22 Feb 2010 19:59:47 +0000 (19:59 +0000)]
I promised in the docs that the HTML output directory would be
created if it didn't exist, so I'd better actually do that.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8881 cda61777-01e9-0310-a592-d414129be87e

14 years agoMake the existing -d (depth) option apply to the -H (static HTML
simon [Mon, 22 Feb 2010 19:03:08 +0000 (19:03 +0000)]
Make the existing -d (depth) option apply to the -H (static HTML
report) mode, transforming its output from a single HTML file giving
a report for one directory with no crosslinks to a collection of
HTML files with crosslinks between them.

I've also added a special 'max' value to -d (meaning no depth
limit), which also works when used in text mode; I've added -o to
specify an output file for both text and HTML report modes; and I've
also documented -d, which I had previously forgotten to do (but
nobody seems to have noticed :-).

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8880 cda61777-01e9-0310-a592-d414129be87e

14 years agoTypos.
simon [Mon, 15 Feb 2010 18:24:30 +0000 (18:24 +0000)]
Typos.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8860 cda61777-01e9-0310-a592-d414129be87e

14 years agoCorrections: put trailing slash on URL, and adjust svn:ignore.
simon [Sun, 6 Dec 2009 23:09:49 +0000 (23:09 +0000)]
Corrections: put trailing slash on URL, and adjust svn:ignore.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8768 cda61777-01e9-0310-a592-d414129be87e

14 years agoAbout time I wrote up the data structure used in agedu, since it's
simon [Sun, 6 Dec 2009 23:07:33 +0000 (23:07 +0000)]
About time I wrote up the data structure used in agedu, since it's
quite fun.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8767 cda61777-01e9-0310-a592-d414129be87e

14 years agoCome clean about a false-positive scenario in agedu. I don't have
simon [Sun, 6 Dec 2009 14:08:52 +0000 (14:08 +0000)]
Come clean about a false-positive scenario in agedu. I don't have
the faintest idea how it could generally be solved, but I got bitten
by it last month so the least I can do is warn about it in the man
page.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8766 cda61777-01e9-0310-a592-d414129be87e

14 years agoFlexibly report sizes in Kb, Mb, Gb etc as appropriate. The previous
simon [Tue, 17 Nov 2009 18:20:48 +0000 (18:20 +0000)]
Flexibly report sizes in Kb, Mb, Gb etc as appropriate. The previous
fixed Mb was inconvenient at both ends. Original patch from James
Beal, though I've polished it pretty much into unrecognisability.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8751 cda61777-01e9-0310-a592-d414129be87e

14 years agoCheck for errors when writing the dump file to standard output, so
simon [Tue, 10 Nov 2009 19:06:11 +0000 (19:06 +0000)]
Check for errors when writing the dump file to standard output, so
that we'll terminate promptly in cases of (for instance) a disk
filling up.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8744 cda61777-01e9-0310-a592-d414129be87e

14 years agoFix a piece of embarrassingly untested pointer-twiddling.
simon [Fri, 16 Oct 2009 17:21:32 +0000 (17:21 +0000)]
Fix a piece of embarrassingly untested pointer-twiddling.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8720 cda61777-01e9-0310-a592-d414129be87e

14 years agoGreg Hewgill informs me that the problem another user had reported
simon [Thu, 24 Sep 2009 10:05:09 +0000 (10:05 +0000)]
Greg Hewgill informs me that the problem another user had reported
with unexpected EADDRNOTAVAIL on MacOS can be worked around by
zeroing out the entire sockaddr_in structure before starting to put
things in it.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8660 cda61777-01e9-0310-a592-d414129be87e

14 years agoDebian bug #537127: if we're going to ifdef based on HAVE_STAT64, we
simon [Mon, 7 Sep 2009 19:56:55 +0000 (19:56 +0000)]
Debian bug #537127: if we're going to ifdef based on HAVE_STAT64, we
should arrange that the configure script actually defines that!

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8642 cda61777-01e9-0310-a592-d414129be87e

14 years agoOn second thoughts, rename that Windows binary to something more
simon [Tue, 25 Aug 2009 17:51:42 +0000 (17:51 +0000)]
On second thoughts, rename that Windows binary to something more
descriptive.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8630 cda61777-01e9-0310-a592-d414129be87e

14 years agoAdd a somewhat unpolished Windows utility to generate an agedu scan
simon [Tue, 25 Aug 2009 17:42:07 +0000 (17:42 +0000)]
Add a somewhat unpolished Windows utility to generate an agedu scan
dump on Windows. The rest of agedu only runs on Unix, but this way
you can at least do a scan of a Windows machine and then load the
resulting log file index into an agedu running elsewhere.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8629 cda61777-01e9-0310-a592-d414129be87e

14 years agoAdd the --files option, to list individual files in the various
simon [Mon, 29 Jun 2009 18:05:02 +0000 (18:05 +0000)]
Add the --files option, to list individual files in the various
reporting modes.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8604 cda61777-01e9-0310-a592-d414129be87e

14 years agoFix a typo.
simon [Mon, 22 Jun 2009 20:33:07 +0000 (20:33 +0000)]
Fix a typo.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8602 cda61777-01e9-0310-a592-d414129be87e

14 years agoAfter incrementing an unsigned char value, don't cast it back to
simon [Wed, 17 Jun 2009 17:27:53 +0000 (17:27 +0000)]
After incrementing an unsigned char value, don't cast it back to
unsigned char if you want to preserve its ordering! Fixes a bug
whereby an 0xFF character in a filename could trigger an assertion
failure (since du.c would sort the directory contents correctly but
trie.c would then expect it to be sorted wrong).

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8595 cda61777-01e9-0310-a592-d414129be87e

14 years agoOh, and update the man page.
simon [Thu, 4 Jun 2009 17:45:32 +0000 (17:45 +0000)]
Oh, and update the man page.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8590 cda61777-01e9-0310-a592-d414129be87e

14 years agoI had been wondering for a while whether the use of random addresses
simon [Thu, 4 Jun 2009 17:37:38 +0000 (17:37 +0000)]
I had been wondering for a while whether the use of random addresses
in the localhost space was too clever for its own good, and finding
a reader comment posted on the recent Linux Magazine agedu article
which has misconstrued those addresses as implying that the data is
shipped to some outside server machine has caused me to decide that
in fact it was. Removed (by default, though re-enableable by ifdef).

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8589 cda61777-01e9-0310-a592-d414129be87e

14 years agoArrange that the --html output mode delivers a sensible error
simon [Fri, 29 May 2009 17:45:35 +0000 (17:45 +0000)]
Arrange that the --html output mode delivers a sensible error
message in place of a segfault, if you hand it a pathname it can't
find.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8587 cda61777-01e9-0310-a592-d414129be87e

15 years agoHow have I managed to neglect until now to put all the subdirectory
simon [Tue, 3 Feb 2009 19:41:37 +0000 (19:41 +0000)]
How have I managed to neglect until now to put all the subdirectory
names in the HTML reports inside <code> tags? Good grief. Now done so.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8442 cda61777-01e9-0310-a592-d414129be87e

15 years agoAhem. Remove evidence of this makefile having been blatantly cloned-
simon [Tue, 3 Feb 2009 19:35:37 +0000 (19:35 +0000)]
Ahem. Remove evidence of this makefile having been blatantly cloned-
and-hacked from my previous project :-)

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8441 cda61777-01e9-0310-a592-d414129be87e

15 years agoInstead of using pure atimes, I think we should generally use
simon [Fri, 28 Nov 2008 19:15:15 +0000 (19:15 +0000)]
Instead of using pure atimes, I think we should generally use
max(atime, mtime). A file that's been recently modified is surely in
use as much as one recently read.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8341 cda61777-01e9-0310-a592-d414129be87e

15 years agoBe willing to follow a symlink if the user specifies one as the root
simon [Tue, 25 Nov 2008 18:09:14 +0000 (18:09 +0000)]
Be willing to follow a symlink if the user specifies one as the root
of the recursive scan, on the grounds that they're more likely to
have wanted a scan of the link target than of the link itself. We
still don't follow symlinks once we've started recursing, of course.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8329 cda61777-01e9-0310-a592-d414129be87e

15 years agoFix a segfault in the case where the entire recursive scan covers
simon [Tue, 25 Nov 2008 18:08:17 +0000 (18:08 +0000)]
Fix a segfault in the case where the entire recursive scan covers
only one filename.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8328 cda61777-01e9-0310-a592-d414129be87e

15 years agoYikes! Replace a + with a *. No wonder the file was small enough
simon [Sat, 22 Nov 2008 11:45:04 +0000 (11:45 +0000)]
Yikes! Replace a + with a *. No wonder the file was small enough
that indexbuild_rebase() was being called with ib->currroot still
NULL.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8315 cda61777-01e9-0310-a592-d414129be87e

15 years agoDon't rebase ib->currroot if it was NULL.
simon [Sat, 22 Nov 2008 11:36:12 +0000 (11:36 +0000)]
Don't rebase ib->currroot if it was NULL.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8314 cda61777-01e9-0310-a592-d414129be87e

15 years agoThe magic-Linux-syscall branch of the #if in this module is only
simon [Fri, 21 Nov 2008 18:57:06 +0000 (18:57 +0000)]
The magic-Linux-syscall branch of the #if in this module is only
worth using at all if we have O_NOATIME to use with it, which it
turns out some (old) Linuxes don't. Make the #if more reluctant to
use that branch, therefore.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8308 cda61777-01e9-0310-a592-d414129be87e

15 years agoTrivial typo fixes.
simon [Fri, 21 Nov 2008 18:52:57 +0000 (18:52 +0000)]
Trivial typo fixes.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8307 cda61777-01e9-0310-a592-d414129be87e

15 years agoHandle another fiddly special case in HTML back-link generation.
simon [Sat, 15 Nov 2008 09:04:21 +0000 (09:04 +0000)]
Handle another fiddly special case in HTML back-link generation.
This enables agedu -w to deal sensibly with scans generated on
Windows and starting in a drive's root directory such as "c:\".

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8297 cda61777-01e9-0310-a592-d414129be87e

15 years agoOops, found a rogue "struct stat64" which hadn't been parametrised.
simon [Sat, 15 Nov 2008 07:59:39 +0000 (07:59 +0000)]
Oops, found a rogue "struct stat64" which hadn't been parametrised.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8296 cda61777-01e9-0310-a592-d414129be87e

15 years agoOptimisations in HTML generation: do not keep calling get_indices()
simon [Sat, 8 Nov 2008 08:24:38 +0000 (08:24 +0000)]
Optimisations in HTML generation: do not keep calling get_indices()
when we already know the answers, and move all the
round_and_format_age() calls to the top of html_query() instead of
duplicating them all over the rest of the file. Saves about 60% time
on Linux, and a staggering factor of 25 on my Mac (on which,
apparently, mktime() is ludicrously slow).

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8292 cda61777-01e9-0310-a592-d414129be87e

15 years agoMy WWW-Authenticate header should end with \r\n like all the others,
simon [Sat, 8 Nov 2008 08:00:22 +0000 (08:00 +0000)]
My WWW-Authenticate header should end with \r\n like all the others,
not \r. No wonder Safari got confused; I'm surprised Firefox didn't!

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8291 cda61777-01e9-0310-a592-d414129be87e

15 years agoIf TIOCGWINSZ returns a window size of zero, treat it as "window
simon [Sat, 8 Nov 2008 07:59:55 +0000 (07:59 +0000)]
If TIOCGWINSZ returns a window size of zero, treat it as "window
size unavailable".

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8290 cda61777-01e9-0310-a592-d414129be87e

15 years agoOops: when we re-mmap the file, don't forget to rebase the two
simon [Fri, 7 Nov 2008 20:06:16 +0000 (20:06 +0000)]
Oops: when we re-mmap the file, don't forget to rebase the two
pointers into it we're actually holding in main() as well as the
ones tucked away in subsystems' state.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8289 cda61777-01e9-0310-a592-d414129be87e

15 years agoWith the new cut-down index, my previous upper-bound estimate on
simon [Fri, 7 Nov 2008 19:44:27 +0000 (19:44 +0000)]
With the new cut-down index, my previous upper-bound estimate on
index file size is now laughably inaccurate, and worse still it may
now prove to be the limiting factor on the size of index that can be
handled (if it causes address space to run out long before actual
memory is in danger). I've tried and failed to think of a way of
estimating the upper bound more accurately in the new circumstances,
so instead I'm giving up: instead of padding out the index file to
the maximum size that could possibly be needed, we extend it bit by
bit, re-mmapping as we go.

I've also retired --full, because I now realise it's utterly
unnecessary: the only thing it might have been useful for would have
been running queries based on a single file - and if we wanted to do
that, we could simply pick one entry out of the trie and not go to
the index at all. So full indexes are a thing of the past, and good
riddance.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8288 cda61777-01e9-0310-a592-d414129be87e

15 years agoTinker slightly with the interaction of progress reports and error
simon [Fri, 7 Nov 2008 18:56:56 +0000 (18:56 +0000)]
Tinker slightly with the interaction of progress reports and error
messages during a disk scan, so that the latter don't end up with
the remnants of one of the former at the end.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8287 cda61777-01e9-0310-a592-d414129be87e

15 years agoindex.c, instead of storing a distinct tree root for every entry in
simon [Thu, 6 Nov 2008 23:32:20 +0000 (23:32 +0000)]
index.c, instead of storing a distinct tree root for every entry in
the trie, now stores a distinct tree root for only those entries
we're actually going to want to look up later - i.e. those at the
start or end of a directory interval. This means that nodes can be
modified and reused by insertions between two such points, which
means we don't have nearly so many duplicate nodes and save a lot of
space in the index without losing any functionality. This leads to a
_huge_ improvement on the size of the index file: on my home
directory it gets a factor-of-5 improvement from 300Mb to 60Mb!

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8286 cda61777-01e9-0310-a592-d414129be87e

15 years agoWhy on earth was I creating the data file with x permission? Silly
simon [Thu, 6 Nov 2008 18:49:04 +0000 (18:49 +0000)]
Why on earth was I creating the data file with x permission? Silly
idea. Get rid of it.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8285 cda61777-01e9-0310-a592-d414129be87e

15 years agoDon't forget to munmap everything we've mmapped in all modes.
simon [Wed, 5 Nov 2008 21:59:55 +0000 (21:59 +0000)]
Don't forget to munmap everything we've mmapped in all modes.
Otherwise chaining modes is liable to run out of address space
rather rapidly.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8284 cda61777-01e9-0310-a592-d414129be87e

15 years agoAt Tom Womack's request, a trivial option to use mtimes instead of
simon [Wed, 5 Nov 2008 21:57:32 +0000 (21:57 +0000)]
At Tom Womack's request, a trivial option to use mtimes instead of
atimes. Generally less useful, unless your atimes have been
completely hosed in which case it's better than nothing.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8283 cda61777-01e9-0310-a592-d414129be87e

15 years agoAdjust the default listening address selection for the web server:
simon [Wed, 5 Nov 2008 21:51:59 +0000 (21:51 +0000)]
Adjust the default listening address selection for the web server:
be prepared to fall back to sensible localhost when MacOS doesn't
like silly localhost, and also pick a random port instead of 80 when
the user didn't specify one.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8282 cda61777-01e9-0310-a592-d414129be87e

15 years agoPortability enhancements to make better use of autoconf. All system
simon [Wed, 5 Nov 2008 21:15:37 +0000 (21:15 +0000)]
Portability enhancements to make better use of autoconf. All system
#includes (apart from the Linux kernel syscall ickery in du.c) have
been centralised into agedu.h and been made conditional on the
relevant header files being shown to exist by configure. My hacky
personal config.h has been removed and the static development
Makefile now runs the real configure script when necessary. There
are a couple of other tweaks which work around buggy printfs on
Solaris.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8281 cda61777-01e9-0310-a592-d414129be87e

15 years agoIf we're in HTTP Basic authentication mode, it's vital to return 401
simon [Wed, 5 Nov 2008 08:02:44 +0000 (08:02 +0000)]
If we're in HTTP Basic authentication mode, it's vital to return 401
for _any_ failed authentication, because that's what causes the
browser to give you a repeat password prompt. If we ever return 403,
the browser will _remember_ your wrong password, and give you 403
again the next time without letting you have a chance to
reauthenticate.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8280 cda61777-01e9-0310-a592-d414129be87e

15 years agoMake #inclusion of <features.h> conditional on autoconf telling us
simon [Wed, 5 Nov 2008 07:58:46 +0000 (07:58 +0000)]
Make #inclusion of <features.h> conditional on autoconf telling us
it exists. agedu now builds on OS X.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8279 cda61777-01e9-0310-a592-d414129be87e

15 years agoExpand Id keyword in man page.
simon [Tue, 4 Nov 2008 18:04:29 +0000 (18:04 +0000)]
Expand Id keyword in man page.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8274 cda61777-01e9-0310-a592-d414129be87e

15 years agoBuild the HTML man page for my website, as part of the build script.
simon [Mon, 3 Nov 2008 22:52:13 +0000 (22:52 +0000)]
Build the HTML man page for my website, as part of the build script.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8269 cda61777-01e9-0310-a592-d414129be87e

15 years agoUse the configure output to select an lstat, and to select a method
simon [Mon, 3 Nov 2008 22:16:30 +0000 (22:16 +0000)]
Use the configure output to select an lstat, and to select a method
of scanning directories. Everything else can wait until somebody
complains - not least because when they do I'll know what sort of
system I need to test the required changes on...

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8268 cda61777-01e9-0310-a592-d414129be87e

15 years agoVarious polishing: man page tweaks, --version now does something,
simon [Mon, 3 Nov 2008 21:55:54 +0000 (21:55 +0000)]
Various polishing: man page tweaks, --version now does something,
the bob and automake framework builds, ships and installs the man
page.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8267 cda61777-01e9-0310-a592-d414129be87e

15 years agoSaying in the man page that I hadn't tested on a 64-bit machine was
simon [Mon, 3 Nov 2008 21:35:47 +0000 (21:35 +0000)]
Saying in the man page that I hadn't tested on a 64-bit machine was
pretty feeble, when I had one conveniently available. I have now!

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8266 cda61777-01e9-0310-a592-d414129be87e

15 years agoFirst cut at a man page. Also, I've modified the online help in the
simon [Mon, 3 Nov 2008 21:30:19 +0000 (21:30 +0000)]
First cut at a man page. Also, I've modified the online help in the
program to present the options in the order I decided was sensible
in the man page.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8265 cda61777-01e9-0310-a592-d414129be87e

15 years agoFix obvious errors in the tarball generation.
simon [Sun, 2 Nov 2008 23:47:56 +0000 (23:47 +0000)]
Fix obvious errors in the tarball generation.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8264 cda61777-01e9-0310-a592-d414129be87e

15 years agoPreliminary autoconf framework. Entirely separate from the main
simon [Sun, 2 Nov 2008 15:51:56 +0000 (15:51 +0000)]
Preliminary autoconf framework. Entirely separate from the main
development makefile; there's a bob build script which creates a
tarball that has an autoconf makefile in place of my GNUmakefile.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8261 cda61777-01e9-0310-a592-d414129be87e

15 years agoRename malloc.c to alloc.c, and my master Makefile to GNUmakefile,
simon [Sun, 2 Nov 2008 15:34:59 +0000 (15:34 +0000)]
Rename malloc.c to alloc.c, and my master Makefile to GNUmakefile,
in the probably vain hope of playing more nicely with autoconf.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8260 cda61777-01e9-0310-a592-d414129be87e

15 years agoNew operation mode to remove the data file. Now you can get a nicely
simon [Sun, 2 Nov 2008 14:15:24 +0000 (14:15 +0000)]
New operation mode to remove the data file. Now you can get a nicely
self-contained scan-and-web-server in a single run without leaving
detritus, by running "agedu -s <dir> -w -R".

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8258 cda61777-01e9-0310-a592-d414129be87e

15 years agoTweak the options list a bit: change around some short options, and
simon [Sun, 2 Nov 2008 14:13:19 +0000 (14:13 +0000)]
Tweak the options list a bit: change around some short options, and
pedantically correct one piece of help text.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8257 cda61777-01e9-0310-a592-d414129be87e

15 years agoFix bonehead error in hex escape processing when loading dump files.
simon [Sun, 2 Nov 2008 13:05:17 +0000 (13:05 +0000)]
Fix bonehead error in hex escape processing when loading dump files.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8256 cda61777-01e9-0310-a592-d414129be87e

15 years agoInvent a cunning means of faking plausible atimes for directories,
simon [Sun, 2 Nov 2008 13:00:59 +0000 (13:00 +0000)]
Invent a cunning means of faking plausible atimes for directories,
since directories will tend to have been accessed constantly by
other recursive disk scans and hence not really reflect the semantic
last-usage status of the stuff within them. Directories' atimes are
now computed as the maximum of their mtime and all the atimes below
them. (There's a command-line option to revert to the obvious
behaviour, but I think it is not in general the most useful thing.)

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8255 cda61777-01e9-0310-a592-d414129be87e

15 years agoReposition r8253's change to avoid redundant index lookups.
simon [Sun, 2 Nov 2008 11:35:08 +0000 (11:35 +0000)]
Reposition r8253's change to avoid redundant index lookups.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8254 cda61777-01e9-0310-a592-d414129be87e

15 years agoNearly forgot: leave out individual files in the text display.
simon [Sun, 2 Nov 2008 11:00:28 +0000 (11:00 +0000)]
Nearly forgot: leave out individual files in the text display.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8253 cda61777-01e9-0310-a592-d414129be87e

15 years agoCentralise the program name into the main header file. I'm probably
simon [Sun, 2 Nov 2008 10:41:57 +0000 (10:41 +0000)]
Centralise the program name into the main header file. I'm probably
not planning to change it any more, but it seems generally more
sensible to keep it in one place just in case.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8252 cda61777-01e9-0310-a592-d414129be87e

15 years agoProbably about time I had a central header file containing misc stuff.
simon [Sun, 2 Nov 2008 10:35:18 +0000 (10:35 +0000)]
Probably about time I had a central header file containing misc stuff.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8251 cda61777-01e9-0310-a592-d414129be87e

15 years agoTurn my disgusting readdir macros into a reasonably clean function
simon [Sun, 2 Nov 2008 09:47:45 +0000 (09:47 +0000)]
Turn my disgusting readdir macros into a reasonably clean function
call interface. In the immediate short term this permits me to cope
when open(...,O_NOATIME) returns EPERM; in the longer term it should
also allow me to drop in other readdir mechanisms more easily,
perhaps including Windows.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8250 cda61777-01e9-0310-a592-d414129be87e

15 years agoAbout time I put a licence on this. MIT as usual, naturally.
simon [Sat, 1 Nov 2008 17:44:55 +0000 (17:44 +0000)]
About time I put a licence on this. MIT as usual, naturally.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8249 cda61777-01e9-0310-a592-d414129be87e

15 years agoReorder the plain text output so that the roots are at the bottom,
simon [Sat, 1 Nov 2008 17:42:41 +0000 (17:42 +0000)]
Reorder the plain text output so that the roots are at the bottom,
since it's mostly there for people who like reading standard du
output. Also, add a configurable recursion depth option.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8248 cda61777-01e9-0310-a592-d414129be87e

15 years agoNow every action is encapsulated as a command-line option and its
simon [Sat, 1 Nov 2008 17:30:46 +0000 (17:30 +0000)]
Now every action is encapsulated as a command-line option and its
argument, introduce the possibility of doing more than one such
action in the same run. In particular, this means you can do a scan
and immediately launch a web server, which I suspect will be a
popular mode of use.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8247 cda61777-01e9-0310-a592-d414129be87e

15 years agoMinor tweaks: turn the "http://address:port/0" URL of the topmost
simon [Sat, 1 Nov 2008 17:21:47 +0000 (17:21 +0000)]
Minor tweaks: turn the "address:port/0" URL of the topmost
directory in the index into the natural "http://address:port/", and
also stop putting a self-link on the top-level page if the pathname
involved is "/".

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8246 cda61777-01e9-0310-a592-d414129be87e

15 years agoFix the various issues centring around the anomaly in Unix pathname
simon [Sat, 1 Nov 2008 17:05:32 +0000 (17:05 +0000)]
Fix the various issues centring around the anomaly in Unix pathname
syntax that the canonical name of "/" ends in a trailing '/'.

git-svn-id: svn://svn.tartarus.org/sgt/agedu@8245 cda61777-01e9-0310-a592-d414129be87e