Change the index file format to explicitly indicate the appropriate

[sgt/agedu] / TODO

TODO list for agedu
===================

Before it's non-embarrassingly releasable:

 - more flexible running modes
    + combined scan+dump mode which doesn't even generate an index
      file (nearly indistinguishable from find(1))
    + load mode which reads a dump from standard input and builds
      the index (need to nail down a perfectly general dump format)
    + at least some ability to chain actions within the same run:
      "agedu -s dirname -w" would seem handy.

 - work out what to do about atimes on directories in the absence of
   the Linux syscall magic
    * one option is to read them during the scan and reinstate them
      after each recursion pop. Race-condition prone.
    * marking them in a distinctive colour in the reports is another
      option.
    * a third option is simply to ignore space taken up by
      directories in the first place; inaccurate but terribly simple.
    * incidentally, sometimes open(...,O_NOATIME) will fail, and
      then we have to fall back to ordinary open. Be prepared to do
      this, which probably means getting rid of the icky macro
      hackery in du.c and turning it into a more sensible run-time
      abstraction layer.

 - polish the plain-text output to make it look more like du
    + configurable recursive output depth
    + show the right bits last

 - figure out what to do about scans starting in the root directory
    + Currently we end up with a double leading slash on the
      pathnames, which is ugly, and we also get a zero-length href
      in between those slashes which means the web interface doesn't
      let you click back up to the top level at all.
    + One big problem here is that a lot of the code assumes that
      you can find the extent of a pathname by searching for "foo"
      and "foo^A", trusting that anything inside the directory will
      begin "foo/". So I'd need to consistently fix this everywhere
      so that a trailing slash is disregarded while doing it, but
      not actually removed.
    + The text output gets it all wrong.
    + The HTML output is fiddly even at the design stage: where
      would I _ideally_ put the link to click on to get back to /?
      It's unclear!

 - cross-Unix portability:
    + use autoconf
       * configure use of stat64
       * configure use of /proc/net/tcp
       * configure use of /dev/random
       * configure use of Linux syscall magic replacing readdir
          + later glibcs have fdopendir, hooray! So we can use that
	    too, if it's available and O_NOATIME is too.
       * what do we do elsewhere about _GNU_SOURCE?

 - prepare a little in advance for a potential future Windows port:
    + store the path separator character in the index file when
      writing it, and load it back in when reading
    + store literal byte sizes in all the size fields, instead of
      Unixoid 512-byte sectors

 - man page, licence.

Future directions:

 - IPv6 support in the HTTP server

 - run-time configuration in the HTTP server
    * I think this probably works by having a configuration form, or
      a link pointing to one, somewhere on the report page. If you
      want to reconfigure anything, you fill in and submit the form;
      the web server receives HTTP GET with parameters and a
      referer, adjusts its internal configuration, and returns an
      HTTP redirect back to the referring page - which it then
      re-renders in accordance with the change.
    * All the same options should have their starting states
      configurable on the command line too.

 - curses-ish equivalent of the web output
    + try using xterm 256-colour mode. Can (n)curses handle that? If
      not, try doing it manually.
    + I think my current best idea is to bypass ncurses and go
      straight to terminfo: generate lines of attribute-interleaved
      text and display them, so we only really need the sequences
      "go here and display stuff", "scroll up", "scroll down".
    + I think the attribute-interleaved text might be possible to do
      cunningly, as well: we autodetect a basically VT-style
      terminal, and add 256-colour sequences on the end. So, for
      instance, we might set ANSI-yellow foreground, set ANSI-red
      background, _then_ set both foreground and background to the
      appropriate xterm 256-colour, and then display some
      appropriate character which would have given the right blend
      of the ANSI-16 fore and background colours. Then the same
      display code should gracefully degrade in the face of a
      terminal which doesn't support xterm-256.
       * current best plan is to simulate the xterm-256 shading from
	 0/5 to 5/5 by doing space, colon and hash in colour A on
	 colour B background, then hash, colon and space in B on A
	 background.
    + Infrastructure work before doing any of this would be to split
      html.c into two: one part to prepare an abstract data
      structure describing an HTML-like report (in particular, all
      the index lookups, percentage calculation, vector arithmetic
      and line sorting), and another part to generate the literal
      HTML. Then the former can be reused to produce very similar
      reports in coloured plain text.

 - http://msdn.microsoft.com/en-us/library/ms724290.aspx suggest
   modern Windowses support atime-equivalents, so a Windows port is
   possible in principle.
    + For a full Windows port, would need to modify the current
      structure a lot, to abstract away (at least) memory-mapping of
      files, details of disk scan procedure, networking for httpd.
      Unclear what the right UI would be on Windows, too;
      command-line exactly as now might be considered just a
      _little_ unfriendly. Or perhaps not.
    + Alternatively, a much easier approach would be to write a
      Windows version of just the --scan-dump mode, which does a
      filesystem scan via the Windows API and generates a valid
      agedu dump file on standard output. Then one would simply feed
      that over the network connection of one's choice to the rest
      of agedu running on Unix as usual.