3 .\" $Id: sw.1,v 1.4 1999/06/24 15:52:12 mdw Exp $
5 .\" Manual page for `sw'
9 .\"----- Licensing notice ---------------------------------------------------
11 .\" This file is part of sw-tools.
13 .\" sw-tools is free software; you can redistribute it and/or modify
14 .\" it under the terms of the GNU General Public License as published by
15 .\" the Free Software Foundation; either version 2 of the License, or
16 .\" (at your option) any later version.
18 .\" sw-tools is distributed in the hope that it will be useful,
19 .\" but WITHOUT ANY WARRANTY; without even the implied warranty of
20 .\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
21 .\" GNU General Public License for more details.
23 .\" You should have received a copy of the GNU General Public License
24 .\" along with sw-tools; if not, write to the Free Software Foundation,
25 .\" Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
27 .\"----- Revision history ---------------------------------------------------
30 .\" Revision 1.4 1999/06/24 15:52:12 mdw
31 .\" Add documentation for the `sw-precommit' script.
33 .\" Revision 1.3 1999/06/18 18:58:25 mdw
36 .\" Revision 1.2 1999/06/04 13:56:09 mdw
37 .\" Changes, extensions, polishings, spelling fixes...
39 .\" Revision 1.1.1.1 1999/06/02 16:53:33 mdw
43 .\" --- Useful macro definitions ---
45 .de VS \" Start a sort-of verbatim block
51 .de VE \" Stop a sort-of verbatim block
57 .de HP \" Start an indented paragraph with a bold right-aligned label
59 \fB\h'-\w'\\$1\ 'u'\\$1\ \fP\c
62 .\" --- Style hacking ---
66 . ds mw \fR[\f(BImdw\fR]
68 .el .ds mw \fR[\fBmdw\fR]
73 .\" --- Main manual text ---
75 .TH sw 1 "25 May 1999" "EBI tools"
79 sw \- tool for convenient software installation
87 \fBsw \-\-remote \fIcommand
92 \fBsw \fR[\fB\-fbi\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBconfigure \fR[\fIconfigure-arg\fR...]
94 \fBsw \fR[\fB\-f\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] \fBlinktree
96 \fBsw \fR[\fB\-fbi\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBmake \fR[\fImake-arg\fR...]
97 \fBsw only\-arch \fIarch \fR[\fIarch\fR...]
99 \fBsw rsh \fIhost\fR|\fIarch \fR[\fIcommand \fR[\fIargument\fR...]]
100 \fBsw \fR[\fB\-fbi\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBrun \fIcommand \fR[\fIargument\fR...]
101 \fBsw setup \fIpackage version \fR[\fImaintainer\fR]
102 \fBsw \fR[\fB\-f\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] \fBsnaplink \fIfile \fR[\fIfile\fR...]
109 The \*(sw tool attempts to take a lot of the work out of building and
110 installing source packages across multiple architectures. This section
111 will describe how to use \*(sw's features to best advantage in a number
112 of common situations.
114 To keep things concrete, I'll describe how things are done at the EBI,
115 although there's nothing EBI-specific about the \*(sw program itself.
116 For details about how we handle software at EBI, see the
120 By the way, this is quite a large manual. I recommend that you print a
121 copy onto paper and peruse it in a leisurely fashion, rather than
122 squinting at a monitor.
125 .SH "SUMMARY OF BUILDING PACKAGES"
130 .BI "sw setup " "package version"
132 .IR "arch " [ arch ...]
143 .BI "sw setup " "package version"
145 .IR "arch " [ arch ...]
148 .IR "file " [ file ...]
149 .I [edit the appropriate files]
157 .SH "8 STEPS TO INSTALLING A PACKAGE"
158 The following steps will guide you through your first (and maybe second)
159 package installations. In the description, I'll use
161 to refer to the package's name, and
163 to refer to its version number.
165 Not all the important features and options are described in this part of
166 the manual. View it more as a taster for the sorts of things \*(sw can
168 .SS "1. Download the source distribution"
169 Download the package's source distribution. This will normally be in an
170 archive called something like
171 .IB package - version .tar.gz\c
172 \&. At EBI, we put source archive files in
174 .SS "2. Unpack the source tree"
175 Unpack the source tree into the standard source directory. Each source
176 tree should have its own directory. Most well-packaged source
177 distributions unpack themselves into a neat directory, but less
178 fastidious programmers make archives which scatter files all over the
181 At EBI, we put the source trees in
183 so unpacking a well-formed source distribution looks like:
186 .BI "gzip \-dc ../tr/" package \- version ".tar.gz | tar xfv \-"
188 Ill-formed source distributions involve making the directory for the
189 package first, changing into it, and then unpacking into the current
193 .BI "mkdir " package \- version
194 .BI "cd " package \- version
195 .BI "gzip \-dc ../../tr/" package - version ".tar.gz | tar xfv \-"
197 When you've finished unpacking, make sure that your current directory is
198 the top level directory of the source tree you unpacked.
199 .SS "3. Tell \\*(sw what you're up to"
200 Now you need to tell \*(sw what you're working on. It will keep track of
201 this and other bits of information in a little file and refer to it
202 every now and then. It will also whinge at you and refuse to cooperate
203 if it can't find its little file, so it's as well to oblige.
205 To tell \*(sw to create this little file and initialize it with sensible
206 values, you just need to say
208 .BI "sw setup " "package version"
210 What could be easier?
211 .SS "4. Restrict the build to particular architectures"
212 Some packages don't work on all architectures, either because the author
213 wasn't sufficiently good at writing portable software, or because the
214 program's doing inherently nonportable things.
216 If that's the case, then you need to tell \*(sw to only build on the
217 architectures that really work. Do this with the
219 command. For example, if your package only works on Linux and Solaris,
222 sw only i386-linux sparc-solaris
224 You can get a list of the architecture names that \*(sw understands by
229 With a little bit of luck, these names ought to be self-explanatory.
231 If your package is properly portable and works everywhere then you don't
232 need to do anything for this step. Skip on to the next one.
233 .SS "5. Configure the package"
234 Now it gets complicated. If the package you're building uses
236 to configure itself for its current environment then you're in luck.
239 package because there's a script called
241 in the top source directory, and a file called
251 to configure the package on all the platforms it's meant to be built
252 for. When you've done that, move onto the next step.
258 then all is not lost (although it may be worthwhile complaining at the
259 package's author or maintainers). You need to make a collection of
261 one for each architecture. These link trees are little replicas of the
262 main source tree but with symbolic links instead of the real source
263 files. To make the link trees, run
267 Now, that's not actually quite what you wanted. It's made a link for
269 file in the source tree. Unfortunately, there are some files you'll
270 (probably) have to modify for each architecture in order to configure
271 the package to build properly. You can turn links in the link trees
272 into real independently editable files by
274 the links. Say for example that
278 need to be modified for each architecture. Running the command
280 sw snaplink Makefile config.h
282 is sufficient to do the right thing.
284 Now you must edit the snapped files to configure the package. Make sure
285 that the install directories are correctly set. At EBI, all the
286 software should be configured so that architecture neutral files end up
289 and architecture-specific files end up under
290 .BI /sw/common/arch/ arch\c
292 .SS "6. Build the package"
293 Now you've laid the groundwork, everything ought to be easy. Making the
294 program ought to involve simply typing
298 and waiting for a while. If you had the
300 library available when \*(sw was built, then your terminal will split
301 itself into little independently scrolling windows showing you the
302 progress for each architecture. If you're not privileged enough to have
304 then you get the output appropriately tagged with architecture names,
305 which is unfortunately fairly hard to read.
306 .SS "7. Install the package"
307 Most source packages (and almost certainly all
313 which installs the program correctly. You can run this from \*(sw by
320 option there tells \*(sw that this is the
322 When an architecture completes this step correctly, it's marked as being
323 properly installed, and \*(sw doesn't bother thinking about it again.
329 makefile target, then you have to install things manually. That's not
330 much fun, so moan at the package's author. When you've finished
331 fiddling with installation, run
335 just to tell \*(sw that you've installed everything OK. (This is a bit
337 .SS "8. Update the index"
338 Now that everything's built and installed, there's just one more command
343 This makes \*(sw update its main index of installed packages, telling it
344 which architectures packages are installed on, and who did it.
347 .SH "REFERENCE INTRODUCTION"
348 That was a gentle introduction. This section contains the complete
351 far more detail that you probably want. If that's really the case, try
356 to read the available help text. There's quite a lot of it, and it
357 ought to keep you occupied for a while.
359 The basic \*(sw command line looks a bit like:
372 at the shell prompt, \*(sw gives you an extremely terse usage summary
373 and quits. You have to tell it to do
375 Most of the time you do this by giving \*(sw a
381 so that it knows what to do. There are some strange command line
382 options which cause \*(sw to do more exotic things, though.
385 .SH "IMPLEMENTATION ODDITIES"
386 The \*(sw program that users use is really a small architecture-neutral
387 shell script, which works out the current architecture and executes the
388 appropriate architecture-specific main program. It's done this way so
389 that \*(sw knows that it can use the shell script to start itself up on
390 a remote host with a different architecture, something which it does
391 quite a lot. The only feature provided by the front-end shell script is
394 command line option, which shouldn't be used by anyone except \*(sw's build procedure anyway.
397 .SH "COMMAND LINE OPTION REFERENCE"
398 Any \*(sw command line options can be put in the
400 environment variable. The \*(sw program will read space-separated
401 options from this variable before it reads the command line itself.
403 The \*(sw program usually understands two different names for each
404 option: a traditional Unix single-character name, and a long GNU-style
405 name. The short options behave in the normal Unix way: you can join
406 them together into single words with a
408 at the front, for example. The long names are always preceded by a
409 double dash. You can abbreviate long names as much as you like, as long
410 as the resulting abbreviation is unambiguous. In the descriptions
411 below, both the short and long names of the options are shown, but for
412 reasons of brevity required arguments are only shown for the long form.
414 There are conceptually two types of \*(sw command line options: those
415 which, usually for reasons of consistency with other programs, cause
416 \*(sw to do something immediately; and those which store some settings
417 for particular commands. The latter type are generally more useful.
418 It's worth bearing in mind, though, that the options are only used by a
419 few commands. The command reference describes exactly which commands
422 The complete list of command line options understood by the current
423 version of \*(sw is as follows:
426 Writes a fairly brief summary of \*(sw's command line options and a usage line for each of \*(sw's commands to standard output, and exits successfully.
428 .B "\-H, \-\-help\-full"
429 Writes a summary of \*(sw's command line options and a full paragraph of description for each of \*(sw's commands to standard output, and exits successfully. There's a lot of
430 text generated by this option. I recommend you pipe it through a pager
431 so that you can actually read it.
433 .B "\-v, \-\-version"
434 Writes \*(sw's version number to standard output and exits successfully. This is handy
435 when trying to decide whether your version of \*(sw has a particular feature, for example.
438 Writes a usage message so terse as to be nearly useless to standard
439 output and exits successfully. This is different from just running
441 because although both print the same useless message, running \*(sw without any arguments is considered an error, so the message is sent to
442 standard error and \*(sw will exit unsuccessfully.
444 .BI "\-a, \-\-arch " arch , arch\fR...
445 For commands which affect multiple architectures: only affect the
446 architectures specified. The architecture names may be separated by
447 commas, spaces or both, although clearly commas are most convenient on
448 the command line. Architecture names may be abbreviated as long as the
449 abbreviation is not ambiguous.
451 This option overrides any other decisions that \*(sw might make about which architectures to process based on the
453 list and the list of correctly built architectures for the current
457 For commands which affect multiple architectures: affect even
458 architectures that have been successfully built. This has no effect if
463 .B "\-i, \-\-install"
464 For build commands: this is the final install step, so label architectures
465 which successfully complete it as having been completely built. It's
466 normal to specify this option on the
467 .RB ` "make install" '
470 .BI "\-o, \-\-output " style
471 For build commands: select a style for the build output to be displayed
474 for more details on output styles.
477 For build commands: make a beep noise when the build finishes. This
478 provides a handy reminder if you're getting on with something else while
479 waiting for a long build.
481 The remaining options aren't really intended for users. They're helpful
482 for \*(sw's own purposes, though, and described here for completeness' sake. They
483 don't have standard Unix short name equivalents, because they're not
484 usually useful for users.
487 Writes the \*(sw architecture name of the current host to standard output. This is used
488 by \*(sw's configuration script to determine the current architecture name. This
489 option is actually handled by a small shell script rather than by being
490 passed on to the main program. You shouldn't use this option yourself:
493 command instead. Because this option is handled by the shell script,
494 and the script isn't very clever, you can't abbreviate
496 on the command line, and it doesn't conflict with the similarly named
497 but completely different
499 option, which you can still abbreviate all the way down to just
503 Sets \*(sw's idea of its program name to
505 This is intended for use by \*(sw's front-end shell script, but isn't
506 actually needed at the moment. I can't see why you'd want to play with
507 this option, but it shouldn't do any harm.
509 .BI "\-\-remote " remote-command
510 Used by \*(sw when running commands on remote hosts. Don't use this yourself: it puts \*(sw into a very unfriendly mode and requires that you communicate with it
511 using a bizarre binary packet protocol. If you really must know more
512 about this, see the source code: it's quite well documented, really.
516 The descriptions below make use of some technical terms:
518 .B "architecture restriction"
519 A state created by the
521 command, restricting the
522 .I "default build architectures"
523 to those listed as arguments to the command. An architecture
524 restriction may be cleared by
528 .B "build architectures"
529 The architectures which a
533 option is specified on the command line, then its argument specifies the
534 build architectures for this command; otherwise, the
535 .I "default build architectures"
539 A command which executes a process on multiple hosts simultaneously and
540 reports the results. The processes executed usually perform some part
541 of the building of a package. Currently, the build commands are
548 .B "default build architectures"
549 The architectures which, in the absence of a
551 command line option, are affected by a
552 .IR "build command" .
553 To determine the default build architectures, \*(sw reads the list of all architectures from the
555 file, and filters it: if the
557 command line option is
559 specified, then architectures marked as
560 .I "successfully built"
561 are removed from the list; if there is an
562 .I "architecture restriction"
563 in force, then the list is further filtered according to the
566 .B "successfully built"
567 A package is considered to be successfully built on a given architecture
568 if a build command given the
570 command-line option succeeds on a host of that architecture. The list
571 of successfully built architectures can be cleared by the
575 option causes \*(sw to ignore whether architectures have been successfully built when
577 .IR "default build architectures" .
578 .SH "COMMAND REFERENCE"
579 This section describes all of the available \*(sw commands, in alphabetical order.
582 Clears an architecture restriction set by
584 Subsequent build commands will run across all known architectures not
585 yet successfully built, unless overridden by the
587 command-line option, or a later
592 Writes the name of the local host's architecture to standard output.
593 The architecture name is built into \*(sw at compile time.
595 Writes information from the
597 file to the installed packages index file
598 .IB prefix /sw-index\fR.
600 \*(sw performs some checks before committing information to the index
601 file. Firstly, all the expected architectures must be successfully
602 built. Secondly, the script
603 .IB prefix /share/sw-precommit\fR
604 is run, if it exists. This script must exit successfully if the commit
605 is to proceed. The script can be configured to enforce local policy
606 requirements on installed software.
610 script is passed a single argument, which is the package name to be
611 committed. Other useful information is passed in the environment:
614 The package name (again).
617 The package version number.
620 The package's maintainer.
623 The last date on whicy the package was modified.
626 The list of architectures on which the package has been built (separated
627 by spaces or commas).
630 The installation prefix with which \*(sw was configured.
632 The script should report any errors it finds to its standard error
635 .SS configure \fR[\fIconfigure-arg\fR...]
636 Equivalent to the command
638 .BI "run ../configure \-\-prefix=" prefix " " configure-arg\fR...
642 is the installation prefix with which \*(sw itself was configured. If you want to specify a different prefix, pass
647 It is expected that administrators will set up a file
648 .IB prefix /share/config.site
649 which sets up other Autoconf parameters once the prefix has been
650 chosen. See the Autoconf manual for more information.
653 Writes to standard output the name of a host with requested architecture
655 The hostname is read from the
660 Builds symbolic link trees. For each of the build architectures, a
661 directory with the architecture's name is created containing a symbolic
662 link corresponding to each file in the main source tree. Thus, a `make'
663 in the link tree will fetch the source files correctly, but place the
664 objects in the link tree rather than the main source tree, so that
665 object files from different architectures don't interfere with each
668 If the link trees already exist, then rerunning
670 will update the links. This might be useful if the links somehow become
673 To turn some of the links in the link trees into real files, use the
678 Writes a list of all known architecture names to standard output. The
679 list is obtained by reading the
683 .SS make \fR[\fImake-arg\fR...]
686 .BI "run make " make-arg\fR...
690 .SS only\-arch \fIarch arch\fR...
691 Imposes an architecture restriction. Until cancelled by a later
695 command, the default build architectures will be limited to the
696 architectures listed on the command line. Architecture names may be
697 abbreviated as long as the abbreviation is not ambiguous.
701 .I "successfully built"
702 status of all architectures.
704 .SS rsh \fIhost\fR|\fIarch \fR[\fIcommand \fR[\fIargument\fR...]]
707 on a remote host, passing it the list of
711 command is unlike the standard
713 program and its replacements:
719 are not subjected to further shell expansion on the remote host.
721 The command is run with the remote current directory the same as the
722 local current directory, rather than the remote user's home directory.
724 The command is passed an environment constructed from the local
725 environment, the default remote environment, and
727 files, as described in the section
728 .B "Remote environment"
731 The remote command is run with standard input attached to
733 there is no way of running an interactive remote command through
736 The host on which to run the remote command may be specified as one of:
737 a standard host name (or IP address), an architecture name (which may
739 be abbreviated) signifying a host of the appropriate architecture, or
742 signifying the current host. (This last option may not sound useful,
743 but it's handy for testing.)
745 .SS run \fIcommand \fR[\fIargument\fR...]
746 Runs a command on all build architectures.
748 For each build architecture
750 \*(sw finds a host with the appropriate architecture, by choosing either
751 the local host or reading the hostname from the
753 file. It then performs the following actions on that host:
755 Sets the current directory to be the subdirectory named
757 of the directory from which the command was issued. This directory is
758 created if it doesn't already exist.
760 Sets up an environment constructed from the environment prevailing when
761 the command was issued, the default environment set up by
763 (or whatever equivalent remote execution program was actually used), and
766 files, as described in the section
767 .B "Remote environment"
770 Executes the program named
775 Output from the command is both appended to the file
781 command-line option. See the section
783 below for more details.
787 option was given on the command line, each architecture on which the
788 command succeeds (i.e., reports a zero exit code) is marked as
789 .IR "successfully built" ,
790 and further build commands will not affect it unless the
792 command line option is passed, until a
794 command is performed.
796 .SS setup \fIpackage version \fR[\fImaintainer\fR]
797 Sets up various pieces of information required by \*(sw. The
798 information here will be added into the main index file by a
800 command. The information is maintained in a file named
802 in the current directory.
806 should be the basic name of the package, with versioning information
812 .RB ` emacs\-19.34 '.
815 should be the version number of the package. The
817 should be the name of the person principally responsible for maintaining
818 the package's local installation. If this isn't specified, the calling
819 user's name is used as the maintainer.
823 command must be run before any build command.
825 .SS snaplink \fIfile \fR[\fIfile\fR...]
826 Creates architecture-specific versions of a file. Every
828 named on the command line is copied to
830 for every build architecture
832 overwriting any existing file or symbolic link of that name. If
834 contains leading directories then destination directories are created as
835 necessary for the output files. Note that the `snap' operation doesn't
836 actually need to follow creation of link trees.
840 Output from a build command is presented in one of a number of named
841 .IR "output styles" .
844 is always defined: it simply prefixes each line of output with the
845 name of the architecture which generated the line, which isn't actually
846 particularly easy to read. Other output styles may have been configured
847 into \*(sw when it was compiled.
849 The set of output styles supported by \*(sw varies according to how it
850 was configured. In any particular \*(sw program, you might have some of
854 Simply prefixes each output line with the name of the architecture it
855 came from. This is quite hard to read, but it doesn't require any
856 special operating system support or clever terminal.
859 Splits the terminal into independently scrolling areas, one for each
860 architecture, with a status line for each. Waits for a keypress when
861 all architectures are finished building.
865 style is used when the selected style doesn't work (for example, you
866 don't have a sufficiently capable terminal for curses output).
868 Output style names can be abbreviated as long as the abbreviation is
869 unambiguous. You can find the list of available output styles by
870 executing the command
874 (which is a little counter-intuitive, I know).
876 The author has plans to implement an X-based output style, but hasn't
877 got around to it yet.
880 .SH "REMOTE ENVIRONMENT"
881 The environment for a remote command (executed either through the
883 command, or a build command) is set up as follows:
885 The complete environment passed to \*(sw is used as a basis.
887 Any environment variables defined by the remote execution program
890 override corresponding variables in the basis environment.
894 variable is set to the name of the remote host's architecture.
896 Variable assignments are read from the global
897 .IB prefix /share/sw\-env
898 file. This makes some assignments which are useful everywhere, and will
899 then usually include the file
901 in the current directory.
905 files is documented separately in
910 This section describes how non-vendor software works at EBI. Chances
911 are that other sites will work differently. This description is here as
912 an example setup for \*(sw.
914 All the non-vendor software gets put in one big shared filesystem, and
915 is exported from our main fileserver. The filesystem is mounted on all
918 Architecture-neutral files are then
919 placed in the conventional subdirectories off
922 .BR /sw/common/share,
924 .BR /sw/common/info ).
925 Architecture specific files are stored in subdirectories off
926 .BR /sw/common/arch .
927 For example, Linux binaries go in
928 .BR /sw/common/arch/i386-linux/bin ,
929 and Solaris libraries in
930 .BR /sw/common/arch/sparc-solaris/lib .
931 Additionally, each architecture-specific subtree has a symbolic link
934 for each of the architecture-neutral subdirectories.
936 There is a symbolic link on every client, from
939 .BI /sw/common/arch/ arch\fR,
942 is the architecture of that client. Thus, every client has two
944 of the software repository: the `common' view where every host sees
945 exactly the same mapping between filenames and files, and the `arch'
946 view where every host sees the same mapping between filenames and
947 programs which do the same job.
949 And that's just about it.
953 The following environment variables are of interest to
957 Contains a space-separated list of default command-line options. These
958 are read before, and overridden by, the actual arguments given on the
962 The name of the command to use to run a `make'. This is resolved on the
963 local host once, rather than one for each build host, which is probably
964 a misfeature. To do something more clever, point
966 at a shell script which then picks out the right architecture-specific
968 program from the remote environment.
971 The name of the remote-shell program to use. By default, something
974 is chosen. I recommend using the excellent
979 The following files are of interest to
982 .IB prefix /sw\-index
983 The main index file, containing the list of which packages have been
984 installed for which architectures. See
986 for file format details.
988 .IB prefix /share/archtab
989 The architecture-to-host mapping file. See
991 for file format details.
993 .IB prefix /share/sw\-env
994 Contains global environment variable settings. See
996 for file format details.
998 .IB prefix /share/sw\-precommit
999 Optional script used to approve commit requests. See the
1001 command above for calling details.
1003 for file format details.
1005 .IB package /.sw\-info
1006 Contains the persistent information about a particular package's build
1009 for file format details.
1011 .IB package /.sw\-env
1012 Contains package-specific environment variable settings. See
1014 for file format details.
1016 .IB package / arch /.build\-log
1017 Contains all the build output for a particular architecture. Usually
1018 not very interesting, but might be handy one day.
1021 There are no bugs in
1023 merely unexpected behaviour modes. Silly you for thinking otherwise.
1025 The \*(sw program, and this manual, are \*(mw productions, in association
1026 with the European Bioinformatics Institute. They were written by Mark
1027 Wooding <mdw@nsict.org>. Go and ask him if you have problems.
1029 .\"----- That's all, folks --------------------------------------------------
~mdw / sw-tools / blob