X-Git-Url: https://git.distorted.org.uk/~mdw/runlisp/blobdiff_plain/e29834b853038e8c90dcfe8377f02431cad42fc5..42ca052285de2281e425b7dc470f62621127b3e1:/README.org diff --git a/README.org b/README.org index 0465005..821b4c9 100644 --- a/README.org +++ b/README.org @@ -3,6 +3,8 @@ #+AUTHOR: Mark Wooding #+LaTeX_CLASS: strayman #+LaTeX_HEADER: \usepackage{tikz, gnuplot-lua-tikz} +#+LaTeX_HEADER: \DeclareUnicodeCharacter{200B}{} +#+EXPORT_FILE_NAME: doc/README.pdf ~runlisp~ is a small C program intended to be run from a script ~#!~ line. It selects and invokes a Common Lisp implementation, so as to run @@ -14,16 +16,19 @@ Currently, the following Lisp implementations are supported: + Armed Bear Common Lisp (~abcl~), + Clozure Common Lisp (~ccl~), + GNU CLisp (~clisp~), - + Carnegie--Mellon Univerity Common Lisp (~cmucl~), and + + Carnegie--Mellon Univerity Common Lisp (~cmucl~), + Embeddable Common Lisp (~ecl~), and + Steel Bank Common Lisp (~sbcl~). -I'm happy to take patches to support additional free Lisp -implementations. I'm not interested in supporting non-free Lisp -systems. +Adding more Lisps is simply a matter of writing the necessary runes in a +configuration file. Of course, there's a benefit to having a collection +of high-quality configuration runes curated centrally, so I'm happy to +accept submissions in support of any free[fn:free] Lisp implementations. +[fn:free] Here I mean free as in freedom. -* Writing scripts in Lisp + +* Writing scripts in Common Lisp ** Basic use @@ -147,24 +152,32 @@ command-line interface for evaluating Lisp forms. For example: : 3 If your build script needs to get information out of Lisp, then wrapping -~format~, or even ~prin1~, around forms is annoying; so ~runlisp~ has a +~format~, or even ~princ~, around forms is annoying; so ~runlisp~ has a ~-p~ option which prints the values of the forms it evaluates. : $ runlisp -e '(+ 1 2)' : 3 -If a form produces multiple values, then ~-p~ will print all of them -separated by spaces, on a single line: +If a form produces multiple values, then ~-p~ will print all of them, as +if by ~princ~, separated by spaces, on a single line: : $ runlisp -p '(floor 5 2)' : 2 1 +There's also a ~-d~ option, which does the same thing as ~-p~, only it +prints values as if by ~prin1~. For example, + +: $ runlisp -p '"Hello, world!"' +: Hello, world! +: runlisp -d '"Hello, world!"' +: "Hello, world!" + In addition to evaluating forms with ~-e~, and printing their values -with ~-p~, you can also load a file of Lisp code using ~-l~. +with ~-d~ and ~-p~, you can also load a file of Lisp code using ~-l~. When ~runlisp~ is acting on ~-e~, ~-p~, and/or ~-l~ options, it's said to be running in /eval/ mode, rather than its usual /script/ mode. In -script mode, it /doesn't/ set ~:runlisp-script~ in ~*features*~. +eval mode, it /doesn't/ set ~:runlisp-script~ in ~*features*~. You can still insist that ~runlisp~ use a particular Lisp implementation, or one of a subset of implementations, using the ~-L~ @@ -197,55 +210,209 @@ meaningful. (Currently, it reveals the name of the script which ** Where =runlisp= looks for configuration You can influence which Lisp implementations are chosen by ~runlisp~ by -writing a configuration file, and/or setting an environment variable. +writing configuration files, and/or setting environment variables. + +The ~runlisp~ program looks for configuration in a number of places. + + + There's a system-global directory ~SYSCONFDIR/runlisp/runlisp.d/~. + All of the files in this directory named ~SOMETHING.conf~ are read, + in increasing lexicographical order by name. The package comes with + a file ~0base.conf~ intended to be read first, so that it can be + overridden if necessar. This sets up basic definitions, and defines + the necessary runes for those Lisp implementations which are + supported `out of the box'. New Lisp packages might come with + additional files to drop into this directory. + + + There's a system-global file ~SYSCONFDIR/runlisp/runlisp.conf~ which + is intended to be edited by the system administrator to account for + any local quirks. This is read /after/ the directory, which is + intended to be used by distribution packages, so that the system + administrator can override them. + + + Users can create files ~$HOME/.runlisp.conf~ and/or + ~$HOME/.config/runlisp.conf~[fn:xdg-config] in their home + directories to add support for privately installed Lisp systems, or + to override settings made by earlier configuration files. + +But configuration files generally look like =.ini=-style files. A line +beginning with a semicolon ~;~ is a comment and is ignored. Most lines +are assignments, which look like +#+BEGIN_QUOTE +/name/ ~=~ /value/ +#+END_QUOTE +and assignments are split into sections by section headers in square +brackets: +#+BEGIN_QUOTE +~[~\relax{}/section/\relax{}~]~ +#+END_QUOTE +The details of the configuration syntax are complicated, and explained +in the *runlisp.conf* manpage. + +Configuration options can also be set on the command line, though the +effects are subtly different. Again, see the manual pages for details. + +[fn:xdg-config] More properly, in ~$XDG_CONFIG_HOME/runlisp.conf~, if +you set that. -~runlisp~ looks for configuration in ~~/.runlisprc~, and in -~~/.config/runlisprc~. You could put configuration in both, but that -doesn't seem like a great idea. A configuration file just contains -blank lines, comments, and command-line options, just as you'd write -them to the shell. Simple quoting and escaping is provided: see the -manual page for the full details. Each line is processed independently, -so it doesn't work to write an option on one line and then its argument -on the next. - -The environment variable ~RUNLISP_OPTIONS~ is processed /after/ reading -the configuration file(s), if any. Again, it should contain -command-line options, as you'd write them to the shell. ** Deciding which Lisp implementation to use -The most useful option to use here is ~-P~, which builds up a -/preference list/, in order. The argument to ~-P~ is a comma-separated -list of Lisp implementation names, just like you'd give to ~-L~. +The ~prefer~ option specifies a /preference list/ of Lisp +implementations. The value is a list of Lisp implementation names, as +you'd give to ~-L~, separated by commas and/or spaces. If the +environment variable ~RUNLISP_PREFER~ is set, then this overrides any +value found in the configuration files. So your ~$HOME/.runlisp.conf~ +file might look like this: -If you provide multiple ~-P~ options (e.g., on different lines of your -configuration file, or separately in the configuration file and -environment variable, then the lists are concatenated. Since the -environment variable is processed after the configuration file, this -means that +: ;;; -*-conf-*- +: +: prefer = sbcl, clisp When deciding which Lisp implementation to use, ~runlisp~ works as follows. It builds a list of /acceptable/ Lisp implementations from the -~-L~ options, and a list of /preferred/ Lisp implementations from the -~-P~ options. If there aren't any ~-L~ options, then it assumes that -/all/ Lisp implementations are acceptable; but if there are no ~-P~ -options then it assumes that /no/ Lisp implementations are preferred. -It then works through the preferred list in order: if it finds an -implementation which is installed and acceptable, then it uses that one. -If that doesn't work, then it works through the acceptable -implementations that it hasn't tried yet, in order, and if it finds one -of those that's installed, then it runs that one. Otherwise it reports -an error and gives up. - -** Clearing the preferred list - -Since the environment variable is processed after the configuration -files, it can only append more Lisp implementations to the end of the -preferred list, which may well not be so helpful. There's an additional -option ~-C~, which completely clears the preferred list. The idea is -that you can write ~-C~ at the start of your ~RUNLISP_OPTIONS~ -environment variable to temporarily override your usual configuration -for some special effect. +~-L~ command-line option, and a list of /preferred/ Lisp implementations +from the ~prefer~ configuration option (or environment variable). If +there aren't any ~-L~ options, then it assumes that /all/ Lisp +implementations are acceptable; if no ~prefer~ option is set then it +assumes that /no/ Lisp implementations are preferred. It then works +through the preferred list in order: if it finds an implementation which +is installed and acceptable, then it uses that one. If that doesn't +work, then it works through the acceptable implementations that it +hasn't tried yet, in order, and if it finds one of those that's +installed, then it runs that one. Otherwise it reports an error and +gives up. + + +** Supporting new Lisp implementations + +~runlisp~ tries hard to make adding support for a new Lisp as painless +as possible. An awkward Lisp will of course cause trouble, but +~runlisp~ itself is easy. + +As a simple example, let's add support for the 32-bit version of +Clozure\nbsp{}CL. The source code for Clozure\nbsp{}CL easily builds +both 32- and 64-bit binaries in either 32- or 64-bit userlands, and one +might reasonably want to use the 32-bit CCL for some reason. The +following configuration stanza is sufficient + +: [ccl32] +: @PARENTS = ccl +: command = ${@ENV:CCL32?ccl32} + + + The first line heads a configuration section, providing the name + which will be used for this Lisp implementation, e.g., in ~-L~ + options or ~prefer~ lists. + + + The second line tells ~runlisp~ that configuration settings not + found in this section should be looked up in the ~ccl~ section + instead. + + + The third line defines the command to be used to invoke the Lisp + system. It tries to find an environment variable named ~CCL32~, + falling back to looking up ~ccl32~ in the path otherwise. + +And, err..., that's it. The ~@PARENTS~ setting uses the detailed +command-line runes for ~ccl~, so they don't need to be written out +again. + +That was rather anticlimactic, because all of the work got done +somewhere else. So let's look at a complete example: Steel Bank Common +Lisp. (SBCL's command-line interface is well thought-out, so this is an +ideal opportunity to explain how ~runlisp~ configuration works, without +getting bogged down in the details of fighting less amenable Lisps.) + +The provided ~0base.conf~ file defines SBCL as follows. + +: [sbcl] +: +: command = ${@ENV:SBCL?sbcl} +: image-file = ${@NAME}+asdf.core +: +: run-script = +: ${command} --noinform +: $?@IMAGE{--core "${image-path}" --eval "${image-restore}" | +: --eval "${run-script-prelude}"} +: --script "${@SCRIPT}" +: +: dump-image = +: ${command} --noinform --no-userinit --no-sysinit --disable-debugger +: --eval "${dump-image-prelude}" +: --eval "(sb-ext:save-lisp-and-die \"${@IMAGENEW|q}\")" + +Let's take this in slightly larger pieces. + + + We see the ~[sbcl]~ section heading, and the ~command~ setting + again. These should now be unsurprising. + + + There's no ~@PARENTS~ setting, so by default the ~sbcl~ section + inherits settings from the ~@COMMON~ section, defined in + ~0base.conf~. We shall use a number of definitions from this + section. + + + The ~image-file~ gives the name of the custom image file to look for + when trying to start SBCL, but not the directory. (The directory is + named by the ~image-dir~ configuration setting.) The image file + will be named ~sbcl+asdf.core~, but this isn't what's written. + Instead, it uses ~${@NAME}~, which is replaced by the name of the + section being processed. When we're running SBCL, this does the + same thing; but if someone wants to configure a new ~foo~ Lisp and + set ~@PARENTS~ to ~sbcl~, then the image file for ~foo~ will be + named ~foo+asdf.core~ by default. You needn't take such care when + configuring Lisp implementations for your own purposes, but it's + important for configurations which will be widely used. + + + The ~run-script~ setting explains how to get SBCL to run a script. + This string is broken into words at (unquoted) spaces. + + The syntax ~$?VAR{CONSEQ|ALT}~ means: if a configuration setting + ~VAR~ is defined, then expand to ~CONSEQ~; otherwise, expand to + ~ALT~. In this case, if the magic setting ~@IMAGE~ is defined, then + we add the tokens ~--core "${image-path}" --eval "${image-restore}"~ + to the SBCL command line; otherwise, we add ~--eval + "${run-script-prelude}"~. The ~@IMAGE~ setting is defined by + ~runlisp~ only if (a)\nbsp{}a custom image was found in the correct + place, and (b)\nbsp{}use of custom images isn't disabled on its + command line. + + The ~${image-path}~ token expands to the full pathname to the custom + image file; ~image-restore~ is a predefined Lisp expression to be + run when starting from a dumped image (e.g., to get ASDF to refresh + its idea of which systems are available). + + The ~run-script-prelude~ is another (somewhat involved) Lisp + expression which sets up a Lisp environment suitable for running + scripts -- e.g., by arranging to ignore ~#!~ lines, and pushing + ~:runlisp-script~ onto ~*features*~. + + Finally, regardless of whether we're using a custom or vanilla + image, we add the tokens ~--script "${@SCRIPT}"~ to the command + line. The ~${@SCRIPT}~ token is replaced by the actual script + pathname. ~runlisp~ then appends further arguments from its own + command line and runs the command. (For most Lisps, ~uiop~ needs a + ~--~ marker before the user arguments, but not for SBCL.) + + + Finally, ~dump-image~ defines a command line for dumping a custom + images. The ~dump-image-prelude~ setting is a Lisp expression for + setting up a Lisp so that it will be in a useful state when dumped: + it's very similar to ~run-script-prelude~, and is built out of many + of the same pieces. + + The thing we haven't seen before is ~${@IMAGENEW|q}~. The + ~@IMAGENEW~ setting is defined by the ~dump-runlisp-image~ program + to name the file in which the new image should be + saved.[fn:image-rename] The ~|q~ `filter' is new: it means that the + filename should be escaped suitable for inclusion in a Lisp quoted + string, by prefixing each ~\~ or ~"~ with a ~\~. + +That's more or less all there is. SBCL is a particularly simple +example, but mostly because other Lisp implementations require fancier +stunts /at the Lisp level/. The ~runlisp~-level configuration isn't any +more complicated than SBCL. + +[fn:image-rename] ~dump-runlisp-image~ wants to avoid clobbering an +existing image with a half-finished one, so it tries to arrange for the +new image to be written to a different file, and then renames it once +it's been created successfully.) * What's wrong with =cl-launch=? @@ -269,8 +436,8 @@ I timed how long it took to run on all of ~runlisp~'s supported Lisp implementations, and compared them to how long ~cl-launch~ took: the results are shown in table [[tab:runlisp-vanilla]]. ~runlisp~ is /at least/ two and half times faster at running this script than ~cl-launch~ on all -implementations except Clozure CL[fn:slow-ccl], and approaching four and -a half times faster on SBCL. +implementations except Clozure\nbsp{}CL[fn:slow-ccl], and approaching +four and a half times faster on SBCL. #+CAPTION: ~cl-launch~ vs ~runlisp~ (with vanilla images) #+NAME: tab:runlisp-vanilla @@ -278,12 +445,12 @@ a half times faster on SBCL. |------------------+-------------------+-----------------+----------------------| | *Implementation* | *~cl-launch~ (s)* | *~runlisp~ (s)* | *~runlisp~ (factor)* | |------------------+-------------------+-----------------+----------------------| -| ABCL | 7.3036 | 2.6027 | 2.806 | -| Clozure CL | 1.2769 | 0.9678 | 1.319 | -| GNU CLisp | 1.2498 | 0.2659 | 4.700 | -| CMU CL | 0.9665 | 0.3065 | 3.153 | -| ECL | 0.8025 | 0.3173 | 2.529 | -| SBCL | 0.3266 | 0.0739 | 4.419 | +| ABCL | 7.3378 | 2.6474 | 2.772 | +| Clozure CL | 1.2888 | 0.9742 | 1.323 | +| GNU CLisp | 1.2405 | 0.2703 | 4.589 | +| CMU CL | 0.9521 | 0.3097 | 3.074 | +| ECL | 0.8020 | 0.3236 | 2.478 | +| SBCL | 0.3205 | 0.0874 | 3.667 | |------------------+-------------------+-----------------+----------------------| #+TBLFM: $4=$2/$3;%.3f @@ -301,12 +468,12 @@ in figure [[fig:lisp-graph]]. |------------------+-------------------+-----------------+----------------------| | *Implementation* | *~cl-launch~ (s)* | *~runlisp~ (s)* | *~runlisp~ (factor)* | |------------------+-------------------+-----------------+----------------------| -| ABCL | 7.3036 | 2.5873 | 2.823 | -| Clozure CL | 1.2769 | 0.0088 | 145.102 | -| GNU CLisp | 1.2498 | 0.0146 | 85.603 | -| CMU CL | 0.9665 | 0.0063 | 153.413 | -| ECL | 0.8025 | 0.3185 | 2.520 | -| SBCL | 0.3266 | 0.0077 | 42.416 | +| ABCL | 7.3378 | 2.7023 | 2.715 | +| Clozure CL | 1.2888 | 0.0371 | 34.739 | +| GNU CLisp | 1.2405 | 0.0191 | 64.948 | +| CMU CL | 0.9521 | 0.0060 | 158.683 | +| ECL | 0.8020 | 0.3275 | 2.449 | +| SBCL | 0.3205 | 0.0064 | 50.078 | |------------------+-------------------+-----------------+----------------------| #+TBLFM: $4=$2/$3;%.3f @@ -327,20 +494,20 @@ of other languages, and timed them; the results are tabulated in table |------------------------------+-------------| | *Implementation* | *Time (ms)* | |------------------------------+-------------| -| Clozure CL | 8.8 | -| GNU CLisp | 14.6 | -| CMU CL | 6.3 | -| SBCL | 7.7 | +| Clozure CL | 37.1 | +| GNU CLisp | 19.1 | +| CMU CL | 6.0 | +| SBCL | 6.4 | |------------------------------+-------------| -| Perl | 1.2 | -| Python | 10.3 | +| Perl | 1.1 | +| Python | 6.8 | |------------------------------+-------------| -| Debian Almquist shell (dash) | 1.4 | -| GNU Bash | 2.0 | -| Z Shell | 4.1 | +| Debian Almquist shell (dash) | 1.2 | +| GNU Bash | 1.5 | +| Z Shell | 3.1 | |------------------------------+-------------| -| Tiny C (compile & run) | 1.2 | -| GCC (precompiled) | 0.5 | +| Tiny C (compile & run) | 1.6 | +| GCC (precompiled) | 0.6 | |------------------------------+-------------| #+CAPTION: Comparison of ~runlisp~ and other script interpreters @@ -353,8 +520,8 @@ XPS13 laptop running Debian `buster'. The tools used to make the measurements are included in the source distribution, in the ~bench/~ subdirectory.) -[fn:slow-ccl] I don't know why Clozure CL shows such a small difference -here. +[fn:slow-ccl] I don't know why Clozure\nbsp{}CL shows such a small +difference here. ** It's inconvenient @@ -371,11 +538,15 @@ be used in ~~/.cl-launchrc~ to select a Lisp system for a particular `software system', though this notion doesn't appear to be well-defined, but this all works by editing a single ~$LISPS~ shell variable. By contrast, ~runlisp~ has a ~-L~ option with which scripts can specify the -Lisp systems they support (in a preference order), and a ~-P~ option -with which users can express their own preferences (e.g., in the -environment or a configuration file): ~runlisp~ will never choose a Lisp -system which the script can't deal with, but it will respect the user's -relative preferences. +Lisp systems they support (in a preference order), and a ~prefer~ +configuration setting with which users can express their own +preferences: ~runlisp~ will never choose a Lisp system which the script +can't deal with, but it will respect the user's relative preferences. + +Also, ~cl-launch~ is a monolith. Adding a new Lisp implementation to +it, or changing how a particular implementation is invoked, is rather +involved. By contrast, ~runlisp~ makes this remarkably easy, as +described in [[Supporting new Lisp implementations]]. ** It doesn't establish a (useful) common environment @@ -417,7 +588,7 @@ scripts. level of shell integration for all its supported Lisp implementations. In particular: - + It ensures that the standard Unix `stdin', `stdout', and `stdarr' + + It ensures that the standard Unix `stdin', `stdout', and `stderr' file descriptors are hooked up to the Lisp ~*standard-input*~, ~*standard-output*~, and ~*error-output*~ streams.