summary |
shortlog | log |
commit |
commitdiff |
tree
first ⋅ prev ⋅ next
simon [Fri, 7 Nov 2008 19:44:27 +0000 (19:44 +0000)]
With the new cut-down index, my previous upper-bound estimate on
index file size is now laughably inaccurate, and worse still it may
now prove to be the limiting factor on the size of index that can be
handled (if it causes address space to run out long before actual
memory is in danger). I've tried and failed to think of a way of
estimating the upper bound more accurately in the new circumstances,
so instead I'm giving up: instead of padding out the index file to
the maximum size that could possibly be needed, we extend it bit by
bit, re-mmapping as we go.
I've also retired --full, because I now realise it's utterly
unnecessary: the only thing it might have been useful for would have
been running queries based on a single file - and if we wanted to do
that, we could simply pick one entry out of the trie and not go to
the index at all. So full indexes are a thing of the past, and good
riddance.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8288
cda61777-01e9-0310-a592-
d414129be87e
simon [Fri, 7 Nov 2008 18:56:56 +0000 (18:56 +0000)]
Tinker slightly with the interaction of progress reports and error
messages during a disk scan, so that the latter don't end up with
the remnants of one of the former at the end.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8287
cda61777-01e9-0310-a592-
d414129be87e
simon [Thu, 6 Nov 2008 23:32:20 +0000 (23:32 +0000)]
index.c, instead of storing a distinct tree root for every entry in
the trie, now stores a distinct tree root for only those entries
we're actually going to want to look up later - i.e. those at the
start or end of a directory interval. This means that nodes can be
modified and reused by insertions between two such points, which
means we don't have nearly so many duplicate nodes and save a lot of
space in the index without losing any functionality. This leads to a
_huge_ improvement on the size of the index file: on my home
directory it gets a factor-of-5 improvement from 300Mb to 60Mb!
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8286
cda61777-01e9-0310-a592-
d414129be87e
simon [Thu, 6 Nov 2008 18:49:04 +0000 (18:49 +0000)]
Why on earth was I creating the data file with x permission? Silly
idea. Get rid of it.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8285
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 5 Nov 2008 21:59:55 +0000 (21:59 +0000)]
Don't forget to munmap everything we've mmapped in all modes.
Otherwise chaining modes is liable to run out of address space
rather rapidly.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8284
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 5 Nov 2008 21:57:32 +0000 (21:57 +0000)]
At Tom Womack's request, a trivial option to use mtimes instead of
atimes. Generally less useful, unless your atimes have been
completely hosed in which case it's better than nothing.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8283
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 5 Nov 2008 21:51:59 +0000 (21:51 +0000)]
Adjust the default listening address selection for the web server:
be prepared to fall back to sensible localhost when MacOS doesn't
like silly localhost, and also pick a random port instead of 80 when
the user didn't specify one.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8282
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 5 Nov 2008 21:15:37 +0000 (21:15 +0000)]
Portability enhancements to make better use of autoconf. All system
#includes (apart from the Linux kernel syscall ickery in du.c) have
been centralised into agedu.h and been made conditional on the
relevant header files being shown to exist by configure. My hacky
personal config.h has been removed and the static development
Makefile now runs the real configure script when necessary. There
are a couple of other tweaks which work around buggy printfs on
Solaris.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8281
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 5 Nov 2008 08:02:44 +0000 (08:02 +0000)]
If we're in HTTP Basic authentication mode, it's vital to return 401
for _any_ failed authentication, because that's what causes the
browser to give you a repeat password prompt. If we ever return 403,
the browser will _remember_ your wrong password, and give you 403
again the next time without letting you have a chance to
reauthenticate.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8280
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 5 Nov 2008 07:58:46 +0000 (07:58 +0000)]
Make #inclusion of <features.h> conditional on autoconf telling us
it exists. agedu now builds on OS X.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8279
cda61777-01e9-0310-a592-
d414129be87e
simon [Tue, 4 Nov 2008 18:04:29 +0000 (18:04 +0000)]
Expand Id keyword in man page.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8274
cda61777-01e9-0310-a592-
d414129be87e
simon [Mon, 3 Nov 2008 22:52:13 +0000 (22:52 +0000)]
Build the HTML man page for my website, as part of the build script.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8269
cda61777-01e9-0310-a592-
d414129be87e
simon [Mon, 3 Nov 2008 22:16:30 +0000 (22:16 +0000)]
Use the configure output to select an lstat, and to select a method
of scanning directories. Everything else can wait until somebody
complains - not least because when they do I'll know what sort of
system I need to test the required changes on...
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8268
cda61777-01e9-0310-a592-
d414129be87e
simon [Mon, 3 Nov 2008 21:55:54 +0000 (21:55 +0000)]
Various polishing: man page tweaks, --version now does something,
the bob and automake framework builds, ships and installs the man
page.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8267
cda61777-01e9-0310-a592-
d414129be87e
simon [Mon, 3 Nov 2008 21:35:47 +0000 (21:35 +0000)]
Saying in the man page that I hadn't tested on a 64-bit machine was
pretty feeble, when I had one conveniently available. I have now!
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8266
cda61777-01e9-0310-a592-
d414129be87e
simon [Mon, 3 Nov 2008 21:30:19 +0000 (21:30 +0000)]
First cut at a man page. Also, I've modified the online help in the
program to present the options in the order I decided was sensible
in the man page.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8265
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 23:47:56 +0000 (23:47 +0000)]
Fix obvious errors in the tarball generation.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8264
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 15:51:56 +0000 (15:51 +0000)]
Preliminary autoconf framework. Entirely separate from the main
development makefile; there's a bob build script which creates a
tarball that has an autoconf makefile in place of my GNUmakefile.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8261
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 15:34:59 +0000 (15:34 +0000)]
Rename malloc.c to alloc.c, and my master Makefile to GNUmakefile,
in the probably vain hope of playing more nicely with autoconf.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8260
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 14:15:24 +0000 (14:15 +0000)]
New operation mode to remove the data file. Now you can get a nicely
self-contained scan-and-web-server in a single run without leaving
detritus, by running "agedu -s <dir> -w -R".
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8258
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 14:13:19 +0000 (14:13 +0000)]
Tweak the options list a bit: change around some short options, and
pedantically correct one piece of help text.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8257
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 13:05:17 +0000 (13:05 +0000)]
Fix bonehead error in hex escape processing when loading dump files.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8256
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 13:00:59 +0000 (13:00 +0000)]
Invent a cunning means of faking plausible atimes for directories,
since directories will tend to have been accessed constantly by
other recursive disk scans and hence not really reflect the semantic
last-usage status of the stuff within them. Directories' atimes are
now computed as the maximum of their mtime and all the atimes below
them. (There's a command-line option to revert to the obvious
behaviour, but I think it is not in general the most useful thing.)
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8255
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 11:35:08 +0000 (11:35 +0000)]
Reposition r8253's change to avoid redundant index lookups.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8254
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 11:00:28 +0000 (11:00 +0000)]
Nearly forgot: leave out individual files in the text display.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8253
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 10:41:57 +0000 (10:41 +0000)]
Centralise the program name into the main header file. I'm probably
not planning to change it any more, but it seems generally more
sensible to keep it in one place just in case.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8252
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 10:35:18 +0000 (10:35 +0000)]
Probably about time I had a central header file containing misc stuff.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8251
cda61777-01e9-0310-a592-
d414129be87e
simon [Sun, 2 Nov 2008 09:47:45 +0000 (09:47 +0000)]
Turn my disgusting readdir macros into a reasonably clean function
call interface. In the immediate short term this permits me to cope
when open(...,O_NOATIME) returns EPERM; in the longer term it should
also allow me to drop in other readdir mechanisms more easily,
perhaps including Windows.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8250
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 17:44:55 +0000 (17:44 +0000)]
About time I put a licence on this. MIT as usual, naturally.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8249
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 17:42:41 +0000 (17:42 +0000)]
Reorder the plain text output so that the roots are at the bottom,
since it's mostly there for people who like reading standard du
output. Also, add a configurable recursion depth option.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8248
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 17:30:46 +0000 (17:30 +0000)]
Now every action is encapsulated as a command-line option and its
argument, introduce the possibility of doing more than one such
action in the same run. In particular, this means you can do a scan
and immediately launch a web server, which I suspect will be a
popular mode of use.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8247
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 17:21:47 +0000 (17:21 +0000)]
Minor tweaks: turn the "address:port/0" URL of the topmost
directory in the index into the natural "http://address:port/", and
also stop putting a self-link on the top-level page if the pathname
involved is "/".
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8246
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 17:05:32 +0000 (17:05 +0000)]
Fix the various issues centring around the anomaly in Unix pathname
syntax that the canonical name of "/" ends in a trailing '/'.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8245
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 16:34:29 +0000 (16:34 +0000)]
Fix gormlessness in html.c introduced in r8241.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8244
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 16:32:26 +0000 (16:32 +0000)]
--scan-dump and --load options, for moving dump files around and
rebuilding index files.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8243
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 16:01:10 +0000 (16:01 +0000)]
Change the index file format to explicitly indicate the appropriate
path separator.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8242
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 15:52:16 +0000 (15:52 +0000)]
Prepare to have a parametrisable path separator character. Currently
the value of 'pathsep' never changes, but the code should now be
ready to cope when it does.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8241
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 15:21:40 +0000 (15:21 +0000)]
More HTTP server configuration: allow user-chosen address+port and
HTTP Basic authentication data.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8240
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 14:49:32 +0000 (14:49 +0000)]
Configurable age range represented by the colour coding in the HTML
output.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8239
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 10:53:33 +0000 (10:53 +0000)]
Filter literally-zero-size directories out of the display, since
they typically arise from --exclude-path.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8238
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 10:45:39 +0000 (10:45 +0000)]
Protect against division by zero when handling directories with
total size of zero.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8237
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 10:36:36 +0000 (10:36 +0000)]
Add --prune as an alternative to --exclude, the difference being
that excluding a directory still scans its contents whereas pruning
one chops its entire branch out of the directory tree.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8236
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 10:25:46 +0000 (10:25 +0000)]
Fold some brief and inadequate online help into the ghastly command
line macros.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8235
cda61777-01e9-0310-a592-
d414129be87e
simon [Sat, 1 Nov 2008 09:40:36 +0000 (09:40 +0000)]
Centralise command-line option definitions, using disgusting macros
just out of curiosity to see how well they work for the job.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8234
cda61777-01e9-0310-a592-
d414129be87e
simon [Fri, 31 Oct 2008 19:30:15 +0000 (19:30 +0000)]
Stop having a default action mode. All actions now require a
specific option to enable them.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8233
cda61777-01e9-0310-a592-
d414129be87e
simon [Thu, 30 Oct 2008 20:08:07 +0000 (20:08 +0000)]
Additional options to control the disk scanning: wildcard-based
includes and excludes, control of the cross-fs-boundary prohibition.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8230
cda61777-01e9-0310-a592-
d414129be87e
simon [Thu, 30 Oct 2008 19:41:47 +0000 (19:41 +0000)]
Tidy up the reporting of the disk scan progress on standard error.
It's now conditional by default on stderr being a tty; it's
configurable on or off manually if necessary; and the terminal width
is obtained by TIOCGWINSZ instead of by guesswork.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8229
cda61777-01e9-0310-a592-
d414129be87e
simon [Thu, 30 Oct 2008 19:32:25 +0000 (19:32 +0000)]
Introduce something more like an actual framework for httpd
authentication. We now support magic /proc/net auth, HTTP Basic
auth, and no auth; a command-line switch chooses between those three
or the fourth default choice of falling back from /proc/net to Basic.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8228
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 29 Oct 2008 22:41:51 +0000 (22:41 +0000)]
HTML-escaping mode is bound to come in handy.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8226
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 29 Oct 2008 21:09:17 +0000 (21:09 +0000)]
Some ignores.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8225
cda61777-01e9-0310-a592-
d414129be87e
simon [Wed, 29 Oct 2008 21:08:11 +0000 (21:08 +0000)]
Initial commit of a basically working but severely unpolished
utility which does "du" only more usefully: its key feature is that
it breaks down your disk space usage simultaneously by subdirectory
and by atime, so that you can identify large chunks of data which
are simply gathering dust.
git-svn-id: svn://svn.tartarus.org/sgt/agedu@8224
cda61777-01e9-0310-a592-
d414129be87e