X-Git-Url: https://git.distorted.org.uk/~mdw/sgt/agedu/blobdiff_plain/671599440ffaee87de2dd6fd8236d84d0c85bca0..a8d1009fa71020a958786761e08945db33e94010:/agedu.but diff --git a/agedu.but b/agedu.but index de47dd2..e98e556 100644 --- a/agedu.but +++ b/agedu.but @@ -6,8 +6,8 @@ \U NAME -\cw{agedu} - correlate disk usage with last-access times to identify -large and disused data +\cw{agedu} \dash correlate disk usage with last-access times to +identify large and disused data \U SYNOPSIS @@ -65,10 +65,13 @@ mode: which will print (among other messages) a URL on its standard output along the lines of -\c URL: http://127.164.152.163:48638/ +\c URL: http://127.0.0.1:48638/ (That URL will always begin with \cq{127.}, meaning that it's in the -\cw{localhost} address space.) +\cw{localhost} address space. So only processes running on the same +computer can even try to connect to that web server, and also there +is access control to prevent other users from seeing it \dash see +below for more detail.) Now paste that URL into your web browser, and you will be shown a graphical representation of the disk usage in \cw{/home/fred} and @@ -187,10 +190,10 @@ completely) and a username and password of your choice. \dd In this mode, \cw{agedu} generates a textual report on standard output, listing the disk usage in the specified directory and all -its subdirectories down to a fixed depth. By default that depth is +its subdirectories down to a given depth. By default that depth is 1, so that you see a report for \e{directory} itself and all of its -immediate subdirectories. You can configure a different depth using -\cw{-d}, described in the next section. +immediate subdirectories. You can configure a different depth (or no +depth limit) using \cw{-d}, described in the next section. \lcont{ @@ -241,9 +244,11 @@ whereas \cw{-S} will not. \lcont{ -(Actually, the output will not be exactly identical, due to a -difference in treatment of last-access times on directories. See the -documentation of the \cw{--dir-atime} option in the next section. +(The output will not be exactly \e{identical}, due to a +difference in treatment of last-access times on directories. +However, it should be effectively equivalent for most purposes. See +the documentation of the \cw{--dir-atime} option in the next section +for further detail.) } @@ -252,9 +257,37 @@ documentation of the \cw{--dir-atime} option in the next section. \dd In this mode, \cw{agedu} will generate an HTML report of the disk usage in the specified directory and its immediate subdirectories, in the same form that it serves from its web server -in \cw{-w} mode. However, this time, a single HTML report will be -generated and simply written to standard output, with no hyperlinks -pointing to other similar pages. +in \cw{-w} mode. + +\lcont{ + +By default, a single HTML report will be generated and simply +written to standard output, with no hyperlinks pointing to other +similar pages. If you also specify the \cw{-d} option (see below), +\cw{agedu} will instead write out a collection of HTML files with +hyperlinks between them, and call the top-level file +\cw{index.html}. + +} + +\dt \cw{--cgi} + +\dd In this mode, \cw{agedu} will run as the bulk of a CGI script +which provides the same set of web pages as the built-in web server +would. It will read the usual CGI environment variables, and write +CGI-style data to its standard output. + +\lcont{ + +The actual CGI program itself should be a tiny wrapper around +\cw{agedu} which passes it the \cw{--cgi} option, and also +(probably) \cw{-f} to locate the index file. \cw{agedu} will do +everything else. + +No access control is performed in this mode: restricting access to +CGI scripts is assumed to be the job of the web server. + +} \U OPTIONS @@ -438,8 +471,68 @@ complexity.) } +\dt \cw{--mtime} + +\dd This option causes \cw{agedu} to index files by their last +modification time instead of their last access time. You might want +to use this if your last access times were completely useless for +some reason: for example, if you had recently searched every file on +your system, the system would have lost all the information about +what files you hadn't recently accessed before then. Using this +option is liable to be less effective at finding genuinely wasted +space than the normal mode (that is, it will be more likely to flag +things as disused when they're not, so you will have more candidates +to go through by hand looking for data you don't need), but may be +better than nothing if your last-access times are unhelpful. + +The following option affects all the modes that generate reports: +the web server mode \cw{-w}, the stand-alone HTML generation mode +\cw{-H} and the text report mode \cw{-t}. + +\dt \cw{--files} + +\dd This option causes \cw{agedu}'s reports to list the individual +files in each directory, instead of just giving a combined report +for everything that's not in a subdirectory. + +The following options affect the stand-alone HTML generation mode +\cw{-H} and the text report mode \cw{-t}. + +\dt \cw{-d} \e{depth} or \cw{--depth} \e{depth} + +\dd This option controls the maximum depth to which \cw{agedu} +recurses when generating a text or HTML report. + +\lcont{ + +In text mode, the default is 1, meaning that the report will include +the directory given on the command line and all of its immediate +subdirectories. A depth of two includes another level below that, +and so on; a depth of zero means \e{only} the directory on the +command line. + +In HTML mode, specifying this option switches \cw{agedu} from +writing out a single HTML file to writing out multiple files which +link to each other. A depth of 1 means \cw{agedu} will write out an +HTML file for the given directory and also one for each of its +immediate subdirectories. + +If you want \cw{agedu} to recurse as deeply as possible, give the +special word \cq{max} as an argument to \cw{-d}. + +} + +\dt \cw{-o} \e{filename} or \cw{--output} \e{filename} + +\dd This option is used to specify an output file for \cw{agedu} to +write its report to. In text mode or single-file HTML mode, the +argument is treated as the name of a file. In multiple-file HTML +mode, the argument is treated as the name of a directory: the +directory will be created if it does not already exist, and the +output HTML files will be created inside it. + The following options affect the web server mode \cw{-w}, and in one -case also the stand-along HTML generation mode \cw{-H}: +case also the stand-alone HTML generation mode \cw{-H}: \dt \cw{-r} \e{age range} or \cw{--age-range} \e{age range} @@ -476,8 +569,8 @@ three months ago or later. \cw{agedu} should listen when running its web server. If you want \cw{agedu} to listen for connections coming in from any source, you should probably specify the special IP address \cw{0.0.0.0}. If the -port number is omitted, it will be assumed to be 80 (for which -\cw{agedu} will probably need to be running as a privileged user). +port number is omitted, an arbitrary unused port will be chosen for +you and displayed. \lcont{ @@ -553,13 +646,13 @@ efficiently perform the queries it needs; this data structure requires \cw{O(N log N)} storage. This is larger than you might expect; a scan of my own home directory, containing half a million files and directories and about 20Gb of data, produced an index file -nearly a third of a Gb in size. Furthermore, since the data file -must be memory-mapped during most processing, it can never grow -larger than available address space, which means that any use of -\cw{agedu} on a seriously large file system is probably going to -have to be done on a 64-bit computer. - -Also, \cw{agedu} has not yet been \e{tested} on a 64-bit computer. +over 60Mb in size. Furthermore, since the data file must be +memory-mapped during most processing, it can never grow larger than +available address space, so a \e{really} big filesystem may need to +be indexed on a 64-bit computer. (This is one reason for the +existence of the \cw{-D} and \cw{-L} options: you can do the +scanning on the machine with access to the filesystem, and the +indexing on a machine big enough to handle it.) The data structure also does not usefully permit access control within the data file, so it would be difficult \dash even given the @@ -567,6 +660,18 @@ willingness to do additional coding \dash to run a system-wide \cw{agedu} scan on a \cw{cron} job and serve the right subset of reports to each user. +In certain circumstances, \cw{agedu} can report false positives +(reporting files as disused which are in fact in use) as well as the +more benign false negatives (reporting files as in use which are +not). This arises when a file is, semantically speaking, \q{read} +without actually being physically \e{read}. Typically this occurs +when a program checks whether the file's mtime has changed and only +bothers re-reading it if it has; programs which do this include +\cw{rsync}(\e{1}) and \cw{make}(\e{1}). Such programs will fail to +update the atime of unmodified files despite depending on their +continued existence; a directory full of such files will be reported +as disused by \cw{agedu} but deleting them will cause trouble. + \U LICENCE \cw{agedu} is free software, distributed under the MIT licence. Type