index.c, instead of storing a distinct tree root for every entry in

[sgt/agedu] / TODO

TODO list for agedu
===================

 - stop trying to calculate an upper bound on the index file size.
   Instead, just mmap it at initial size + delta, and periodically
   re-mmap it during index building if it grows too big. If we run
   out of address space, we'll hear about it eventually; and
   computing upper bounds given the new optimised index tends to be
   a factor of five out, which is bad because it'll lead to running
   out of theoretical address space and erroneously reporting
   failure long before we run out of it for real.

 - we could still be using more of the information coming from
   autoconf. Our config.h is defining a whole bunch of HAVE_FOOs for
   particular functions (e.g. HAVE_INET_NTOA, HAVE_MEMCHR,
   HAVE_FNMATCH). We could usefully supply alternatives for some of
   these functions (e.g. cannibalise the PuTTY wildcard matcher for
   use in the absence of fnmatch, switch to vanilla truncate() in
   the absence of ftruncate); where we don't have alternative code,
   it would perhaps be polite to throw an error at configure time
   rather than allowing the subsequent build to fail.
    + however, I don't see anything here that looks very
      controversial; IIRC it's all in POSIX, for one thing. So more
      likely this should simply wait until somebody complains.

 - IPv6 support in the HTTP server
    * of course, Linux magic auth can still work in this context; we
      merely have to be prepared to open one of /proc/net/tcp or
      /proc/net/tcp6 as appropriate.

 - run-time configuration in the HTTP server
    * I think this probably works by having a configuration form, or
      a link pointing to one, somewhere on the report page. If you
      want to reconfigure anything, you fill in and submit the form;
      the web server receives HTTP GET with parameters and a
      referer, adjusts its internal configuration, and returns an
      HTTP redirect back to the referring page - which it then
      re-renders in accordance with the change.
    * All the same options should have their starting states
      configurable on the command line too.

 - curses-ish equivalent of the web output
    + try using xterm 256-colour mode. Can (n)curses handle that? If
      not, try doing it manually.
    + I think my current best idea is to bypass ncurses and go
      straight to terminfo: generate lines of attribute-interleaved
      text and display them, so we only really need the sequences
      "go here and display stuff", "scroll up", "scroll down".
    + Infrastructure work before doing any of this would be to split
      html.c into two: one part to prepare an abstract data
      structure describing an HTML-like report (in particular, all
      the index lookups, percentage calculation, vector arithmetic
      and line sorting), and another part to generate the literal
      HTML. Then the former can be reused to produce very similar
      reports in coloured plain text.

 - http://msdn.microsoft.com/en-us/library/ms724290.aspx suggest
   modern Windowses support atime-equivalents, so a Windows port is
   possible in principle.
    + For a full Windows port, would need to modify the current
      structure a lot, to abstract away (at least) memory-mapping of
      files, details of disk scan procedure, networking for httpd.
      Unclear what the right UI would be on Windows, too;
      command-line exactly as now might be considered just a
      _little_ unfriendly. Or perhaps not.
    + Alternatively, a much easier approach would be to write a
      Windows version of just the --scan-dump mode, which does a
      filesystem scan via the Windows API and generates a valid
      agedu dump file on standard output. Then one would simply feed
      that over the network connection of one's choice to the rest
      of agedu running on Unix as usual.

 - it might conceivably be useful to support a choice of indexing
   strategies. The current "continuous index" mechanism' tradeoff of
   taking O(N log N) space in order to be able to support any age
   cutoff you like is not going to be ideal for everybody. A second
   more conventional "discrete index" mechanism which allows the
   user to specify a number of fixed cutoffs and just indexes each
   directory on those alone would undoubtedly be a useful thing for
   large-scale users. This will require considerable thought about
   how to make the indexers pluggable at both index-generation time
   and query time.
    * however, now we have the cut-down version of the continuous
      index, it might be the case that the space gain is no longer
      worthwhile.