index.c, instead of storing a distinct tree root for every entry in

[sgt/agedu] / TODO

diff --git a/TODO b/TODO
index 0b6efba..6046cec 100644 (file)
--- a/TODO
+++ b/TODO
@@ -1,43 +1,27 @@
 TODO list for agedu
 ===================
 
 TODO list for agedu
 ===================
 

-Before it's non-embarrassingly releasable:
+ - stop trying to calculate an upper bound on the index file size.
+   Instead, just mmap it at initial size + delta, and periodically
+   re-mmap it during index building if it grows too big. If we run
+   out of address space, we'll hear about it eventually; and
+   computing upper bounds given the new optimised index tends to be
+   a factor of five out, which is bad because it'll lead to running
+   out of theoretical address space and erroneously reporting
+   failure long before we run out of it for real.

 
 

- - more flexible running modes
-    + at least some ability to chain actions within the same run:
-      "agedu -s dirname -w" would seem handy.
-
- - work out what to do about atimes on directories in the absence of
-   the Linux syscall magic
-    * one option is to read them during the scan and reinstate them
-      after each recursion pop. Race-condition prone.
-    * marking them in a distinctive colour in the reports is another
-      option.
-    * a third option is simply to ignore space taken up by
-      directories in the first place; inaccurate but terribly simple.
-    * incidentally, sometimes open(...,O_NOATIME) will fail, and
-      then we have to fall back to ordinary open. Be prepared to do
-      this, which probably means getting rid of the icky macro
-      hackery in du.c and turning it into a more sensible run-time
-      abstraction layer.
-
- - polish the plain-text output to make it look more like du
-    + configurable recursive output depth
-    + show the right bits last
-
- - cross-Unix portability:
-    + use autoconf
-       * configure use of stat64
-       * configure use of /proc/net/tcp
-       * configure use of /dev/random
-       * configure use of Linux syscall magic replacing readdir
-          + later glibcs have fdopendir, hooray! So we can use that
-           too, if it's available and O_NOATIME is too.
-       * what do we do elsewhere about _GNU_SOURCE?
-
- - man page, licence.
-
-Future possibilities:
+ - we could still be using more of the information coming from
+   autoconf. Our config.h is defining a whole bunch of HAVE_FOOs for
+   particular functions (e.g. HAVE_INET_NTOA, HAVE_MEMCHR,
+   HAVE_FNMATCH). We could usefully supply alternatives for some of
+   these functions (e.g. cannibalise the PuTTY wildcard matcher for
+   use in the absence of fnmatch, switch to vanilla truncate() in
+   the absence of ftruncate); where we don't have alternative code,
+   it would perhaps be polite to throw an error at configure time
+   rather than allowing the subsequent build to fail.
+    + however, I don't see anything here that looks very
+      controversial; IIRC it's all in POSIX, for one thing. So more
+      likely this should simply wait until somebody complains.

 
  - IPv6 support in the HTTP server
     * of course, Linux magic auth can still work in this context; we
 
  - IPv6 support in the HTTP server
     * of course, Linux magic auth can still work in this context; we


@@ -62,20 +46,6 @@ Future possibilities:
       straight to terminfo: generate lines of attribute-interleaved
       text and display them, so we only really need the sequences
       "go here and display stuff", "scroll up", "scroll down".
       straight to terminfo: generate lines of attribute-interleaved
       text and display them, so we only really need the sequences
       "go here and display stuff", "scroll up", "scroll down".

-    + I think the attribute-interleaved text might be possible to do
-      cunningly, as well: we autodetect a basically VT-style
-      terminal, and add 256-colour sequences on the end. So, for
-      instance, we might set ANSI-yellow foreground, set ANSI-red
-      background, _then_ set both foreground and background to the
-      appropriate xterm 256-colour, and then display some
-      appropriate character which would have given the right blend
-      of the ANSI-16 fore and background colours. Then the same
-      display code should gracefully degrade in the face of a
-      terminal which doesn't support xterm-256.
-       * current best plan is to simulate the xterm-256 shading from
-        0/5 to 5/5 by doing space, colon and hash in colour A on
-        colour B background, then hash, colon and space in B on A
-        background.

     + Infrastructure work before doing any of this would be to split
       html.c into two: one part to prepare an abstract data
       structure describing an HTML-like report (in particular, all
     + Infrastructure work before doing any of this would be to split
       html.c into two: one part to prepare an abstract data
       structure describing an HTML-like report (in particular, all


@@ -99,3 +69,17 @@ Future possibilities:
       agedu dump file on standard output. Then one would simply feed
       that over the network connection of one's choice to the rest
       of agedu running on Unix as usual.
       agedu dump file on standard output. Then one would simply feed
       that over the network connection of one's choice to the rest
       of agedu running on Unix as usual.

+
+ - it might conceivably be useful to support a choice of indexing
+   strategies. The current "continuous index" mechanism' tradeoff of
+   taking O(N log N) space in order to be able to support any age
+   cutoff you like is not going to be ideal for everybody. A second
+   more conventional "discrete index" mechanism which allows the
+   user to specify a number of fixed cutoffs and just indexes each
+   directory on those alone would undoubtedly be a useful thing for
+   large-scale users. This will require considerable thought about
+   how to make the indexers pluggable at both index-generation time
+   and query time.
+    * however, now we have the cut-down version of the continuous
+      index, it might be the case that the space gain is no longer
+      worthwhile.