3 .\" $Id: sw.1,v 1.3 1999/06/18 18:58:25 mdw Exp $
5 .\" Manual page for `sw'
9 .\"----- Licensing notice ---------------------------------------------------
11 .\" This file is part of sw-tools.
13 .\" sw-tools is free software; you can redistribute it and/or modify
14 .\" it under the terms of the GNU General Public License as published by
15 .\" the Free Software Foundation; either version 2 of the License, or
16 .\" (at your option) any later version.
18 .\" sw-tools is distributed in the hope that it will be useful,
19 .\" but WITHOUT ANY WARRANTY; without even the implied warranty of
20 .\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
21 .\" GNU General Public License for more details.
23 .\" You should have received a copy of the GNU General Public License
24 .\" along with sw-tools; if not, write to the Free Software Foundation,
25 .\" Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
27 .\"----- Revision history ---------------------------------------------------
30 .\" Revision 1.3 1999/06/18 18:58:25 mdw
33 .\" Revision 1.2 1999/06/04 13:56:09 mdw
34 .\" Changes, extensions, polishings, spelling fixes...
36 .\" Revision 1.1.1.1 1999/06/02 16:53:33 mdw
40 .\" --- Useful macro definitions ---
42 .de VS \" Start a sort-of verbatim block
48 .de VE \" Stop a sort-of verbatim block
54 .de HP \" Start an indented paragraph with a bold right-aligned label
56 \fB\h'-\w'\\$1\ 'u'\\$1\ \fP\c
59 .\" --- Style hacking ---
63 . ds mw \fR[\f(BImdw\fR]
65 .el .ds mw \fR[\fBmdw\fR]
70 .\" --- Main manual text ---
72 .TH sw 1 "25 May 1999" "EBI tools"
76 sw \- tool for convenient software installation
84 \fBsw \-\-remote \fIcommand
89 \fBsw \fR[\fB\-fbi\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBconfigure \fR[\fIconfigure-arg\fR...]
91 \fBsw \fR[\fB\-f\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] \fBlinktree
93 \fBsw \fR[\fB\-fbi\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBmake \fR[\fImake-arg\fR...]
94 \fBsw only\-arch \fIarch \fR[\fIarch\fR...]
96 \fBsw rsh \fIhost\fR|\fIarch \fR[\fIcommand \fR[\fIargument\fR...]]
97 \fBsw \fR[\fB\-fbi\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBrun \fIcommand \fR[\fIargument\fR...]
98 \fBsw setup \fIpackage version \fR[\fImaintainer\fR]
99 \fBsw \fR[\fB\-f\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] \fBsnaplink \fIfile \fR[\fIfile\fR...]
106 The \*(sw tool attempts to take a lot of the work out of building and
107 installing source packages across multiple architectures. This section
108 will describe how to use \*(sw's features to best advantage in a number
109 of common situations.
111 To keep things concrete, I'll describe how things are done at the EBI,
112 although there's nothing EBI-specific about the \*(sw program itself.
113 For details about how we handle software at EBI, see the
117 By the way, this is quite a large manual. I recommend that you print a
118 copy onto paper and peruse it in a leisurely fashion, rather than
119 squinting at a monitor.
122 .SH "SUMMARY OF BUILDING PACKAGES"
127 .BI "sw setup " "package version"
129 .IR "arch " [ arch ...]
140 .BI "sw setup " "package version"
142 .IR "arch " [ arch ...]
145 .IR "file " [ file ...]
146 .I [edit the appropriate files]
154 .SH "8 STEPS TO INSTALLING A PACKAGE"
155 The following steps will guide you through your first (and maybe second)
156 package installations. In the description, I'll use
158 to refer to the package's name, and
160 to refer to its version number.
162 Not all the important features and options are described in this part of
163 the manual. View it more as a taster for the sorts of things \*(sw can
165 .SS "1. Download the source distribution"
166 Download the package's source distribution. This will normally be in an
167 archive called something like
168 .IB package - version .tar.gz\c
169 \&. At EBI, we put source archive files in
171 .SS "2. Unpack the source tree"
172 Unpack the source tree into the standard source directory. Each source
173 tree should have its own directory. Most well-packaged source
174 distributions unpack themselves into a neat directory, but less
175 fastidious programmers make archives which scatter files all over the
178 At EBI, we put the source trees in
180 so unpacking a well-formed source distribution looks like:
183 .BI "gzip \-dc ../tr/" package \- version ".tar.gz | tar xfv \-"
185 Ill-formed source distributions involve making the directory for the
186 package first, changing into it, and then unpacking into the current
190 .BI "mkdir " package \- version
191 .BI "cd " package \- version
192 .BI "gzip \-dc ../../tr/" package - version ".tar.gz | tar xfv \-"
194 When you've finished unpacking, make sure that your current directory is
195 the top level directory of the source tree you unpacked.
196 .SS "3. Tell \\*(sw what you're up to"
197 Now you need to tell \*(sw what you're working on. It will keep track of
198 this and other bits of information in a little file and refer to it
199 every now and then. It will also whinge at you and refuse to cooperate
200 if it can't find its little file, so it's as well to oblige.
202 To tell \*(sw to create this little file and initialize it with sensible
203 values, you just need to say
205 .BI "sw setup " "package version"
207 What could be easier?
208 .SS "4. Restrict the build to particular architectures"
209 Some packages don't work on all architectures, either because the author
210 wasn't sufficiently good at writing portable software, or because the
211 program's doing inherently nonportable things.
213 If that's the case, then you need to tell \*(sw to only build on the
214 architectures that really work. Do this with the
216 command. For example, if your package only works on Linux and Solaris,
219 sw only i386-linux sparc-solaris
221 You can get a list of the architecture names that \*(sw understands by
226 With a little bit of luck, these names ought to be self-explanatory.
228 If your package is properly portable and works everywhere then you don't
229 need to do anything for this step. Skip on to the next one.
230 .SS "5. Configure the package"
231 Now it gets complicated. If the package you're building uses
233 to configure itself for its current environment then you're in luck.
236 package because there's a script called
238 in the top source directory, and a file called
248 to configure the package on all the platforms it's meant to be built
249 for. When you've done that, move onto the next step.
255 then all is not lost (although it may be worthwhile complaining at the
256 package's author or maintainers). You need to make a collection of
258 one for each architecture. These link trees are little replicas of the
259 main source tree but with symbolic links instead of the real source
260 files. To make the link trees, run
264 Now, that's not actually quite what you wanted. It's made a link for
266 file in the source tree. Unfortunately, there are some files you'll
267 (probably) have to modify for each architecture in order to configure
268 the package to build properly. You can turn links in the link trees
269 into real independently editable files by
271 the links. Say for example that
275 need to be modified for each architecture. Running the command
277 sw snaplink Makefile config.h
279 is sufficient to do the right thing.
281 Now you must edit the snapped files to configure the package. Make sure
282 that the install directories are correctly set. At EBI, all the
283 software should be configured so that architecture neutral files end up
286 and architecture-specific files end up under
287 .BI /sw/common/arch/ arch\c
289 .SS "6. Build the package"
290 Now you've laid the groundwork, everything ought to be easy. Making the
291 program ought to involve simply typing
295 and waiting for a while. If you had the
297 library available when \*(sw was built, then your terminal will split
298 itself into little independently scrolling windows showing you the
299 progress for each architecture. If you're not privileged enough to have
301 then you get the output appropriately tagged with architecture names,
302 which is unfortunately fairly hard to read.
303 .SS "7. Install the package"
304 Most source packages (and almost certainly all
310 which installs the program correctly. You can run this from \*(sw by
317 option there tells \*(sw that this is the
319 When an architecture completes this step correctly, it's marked as being
320 properly installed, and \*(sw doesn't bother thinking about it again.
326 makefile target, then you have to install things manually. That's not
327 much fun, so moan at the package's author. When you've finished
328 fiddling with installation, run
332 just to tell \*(sw that you've installed everything OK. (This is a bit
334 .SS "8. Update the index"
335 Now that everything's built and installed, there's just one more command
340 This makes \*(sw update its main index of installed packages, telling it
341 which architectures packages are installed on, and who did it.
344 .SH "REFERENCE INTRODUCTION"
345 That was a gentle introduction. This section contains the complete
348 far more detail that you probably want. If that's really the case, try
353 to read the available help text. There's quite a lot of it, and it
354 ought to keep you occupied for a while.
356 The basic \*(sw command line looks a bit like:
369 at the shell prompt, \*(sw gives you an extremely terse usage summary
370 and quits. You have to tell it to do
372 Most of the time you do this by giving \*(sw a
378 so that it knows what to do. There are some strange command line
379 options which cause \*(sw to do more exotic things, though.
382 .SH "IMPLEMENTATION ODDITIES"
383 The \*(sw program that users use is really a small architecture-neutral
384 shell script, which works out the current architecture and executes the
385 appropriate architecture-specific main program. It's done this way so
386 that \*(sw knows that it can use the shell script to start itself up on
387 a remote host with a different architecture, something which it does
388 quite a lot. The only feature provided by the front-end shell script is
391 command line option, which shouldn't be used by anyone except \*(sw's build procedure anyway.
394 .SH "COMMAND LINE OPTION REFERENCE"
395 Any \*(sw command line options can be put in the
397 environment variable. The \*(sw program will read space-separated
398 options from this variable before it reads the command line itself.
400 The \*(sw program usually understands two different names for each
401 option: a traditional Unix single-character name, and a long GNU-style
402 name. The short options behave in the normal Unix way: you can join
403 them together into single words with a
405 at the front, for example. The long names are always preceded by a
406 double dash. You can abbreviate long names as much as you like, as long
407 as the resulting abbreviation is unambiguous. In the descriptions
408 below, both the short and long names of the options are shown, but for
409 reasons of brevity required arguments are only shown for the long form.
411 There are conceptually two types of \*(sw command line options: those
412 which, usually for reasons of consistency with other programs, cause
413 \*(sw to do something immediately; and those which store some settings
414 for particular commands. The latter type are generally more useful.
415 It's worth bearing in mind, though, that the options are only used by a
416 few commands. The command reference describes exactly which commands
419 The complete list of command line options understood by the current
420 version of \*(sw is as follows:
423 Writes a fairly brief summary of \*(sw's command line options and a usage line for each of \*(sw's commands to standard output, and exits successfully.
425 .B "\-H, \-\-help\-full"
426 Writes a summary of \*(sw's command line options and a full paragraph of description for each of \*(sw's commands to standard output, and exits successfully. There's a lot of
427 text generated by this option. I recommend you pipe it through a pager
428 so that you can actually read it.
430 .B "\-v, \-\-version"
431 Writes \*(sw's version number to standard output and exits successfully. This is handy
432 when trying to decide whether your version of \*(sw has a particular feature, for example.
435 Writes a usage message so terse as to be nearly useless to standard
436 output and exits successfully. This is different from just running
438 because although both print the same useless message, running \*(sw without any arguments is considered an error, so the message is sent to
439 standard error and \*(sw will exit unsuccessfully.
441 .BI "\-a, \-\-arch " arch , arch\fR...
442 For commands which affect multiple architectures: only affect the
443 architectures specified. The architecture names may be separated by
444 commas, spaces or both, although clearly commas are most convenient on
445 the command line. Architecture names may be abbreviated as long as the
446 abbreviation is not ambiguous.
448 This option overrides any other decisions that \*(sw might make about which architectures to process based on the
450 list and the list of correctly built architectures for the current
454 For commands which affect multiple architectures: affect even
455 architectures that have been successfully built. This has no effect if
460 .B "\-i, \-\-install"
461 For build commands: this is the final install step, so label architectures
462 which successfully complete it as having been completely built. It's
463 normal to specify this option on the
464 .RB ` "make install" '
467 .BI "\-o, \-\-output " style
468 For build commands: select a style for the build output to be displayed
471 for more details on output styles.
474 For build commands: make a beep noise when the build finishes. This
475 provides a handy reminder if you're getting on with something else while
476 waiting for a long build.
478 The remaining options aren't really intended for users. They're helpful
479 for \*(sw's own purposes, though, and described here for completeness' sake. They
480 don't have standard Unix short name equivalents, because they're not
481 usually useful for users.
484 Writes the \*(sw architecture name of the current host to standard output. This is used
485 by \*(sw's configuration script to determine the current architecture name. This
486 option is actually handled by a small shell script rather than by being
487 passed on to the main program. You shouldn't use this option yourself:
490 command instead. Because this option is handled by the shell script,
491 and the script isn't very clever, you can't abbreviate
493 on the command line, and it doesn't conflict with the similarly named
494 but completely different
496 option, which you can still abbreviate all the way down to just
500 Sets \*(sw's idea of its program name to
502 This is intended for use by \*(sw's front-end shell script, but isn't
503 actually needed at the moment. I can't see why you'd want to play with
504 this option, but it shouldn't do any harm.
506 .BI "\-\-remote " remote-command
507 Used by \*(sw when running commands on remote hosts. Don't use this yourself: it puts \*(sw into a very unfriendly mode and requires that you communicate with it
508 using a bizarre binary packet protocol. If you really must know more
509 about this, see the source code: it's quite well documented, really.
513 The descriptions below make use of some technical terms:
515 .B "architecture restriction"
516 A state created by the
518 command, restricting the
519 .I "default build architectures"
520 to those listed as arguments to the command. An architecture
521 restriction may be cleared by
525 .B "build architectures"
526 The architectures which a
530 option is specified on the command line, then its argument specifies the
531 build architectures for this command; otherwise, the
532 .I "default build architectures"
536 A command which executes a process on multiple hosts simultaneously and
537 reports the results. The processes executed usually perform some part
538 of the building of a package. Currently, the build commands are
545 .B "default build architectures"
546 The architectures which, in the absence of a
548 command line option, are affected by a
549 .IR "build command" .
550 To determine the default build architectures, \*(sw reads the list of all architectures from the
552 file, and filters it: if the
554 command line option is
556 specified, then architectures marked as
557 .I "successfully built"
558 are removed from the list; if there is an
559 .I "architecture restriction"
560 in force, then the list is further filtered according to the
563 .B "successfully built"
564 A package is considered to be successfully built on a given architecture
565 if a build command given the
567 command-line option succeeds on a host of that architecture. The list
568 of successfully built architectures can be cleared by the
572 option causes \*(sw to ignore whether architectures have been successfully built when
574 .IR "default build architectures" .
575 .SH "COMMAND REFERENCE"
576 This section describes all of the available \*(sw commands, in alphabetical order.
579 Clears an architecture restriction set by
581 Subsequent build commands will run across all known architectures not
582 yet successfully built, unless overridden by the
584 command-line option, or a later
589 Writes the name of the local host's architecture to standard output.
590 The architecture name is built into \*(sw at compile time.
592 Writes information from the
594 file to the installed packages index file
595 .IB prefix /sw-index\fR.
597 All expected architectures must be built before a commit will work.
599 .SS configure \fR[\fIconfigure-arg\fR...]
600 Equivalent to the command
602 .BI "run ../configure \-\-prefix=" prefix " " configure-arg\fR...
606 is the installation prefix with which \*(sw itself was configured. If you want to specify a different prefix, pass
611 It is expected that administrators will set up a file
612 .IB prefix /share/config.site
613 which sets up other Autoconf parameters once the prefix has been
614 chosen. See the Autoconf manual for more information.
617 Writes to standard output the name of a host with requested architecture
619 The hostname is read from the
624 Builds symbolic link trees. For each of the build architectures, a
625 directory with the architecture's name is created containing a symbolic
626 link corresponding to each file in the main source tree. Thus, a `make'
627 in the link tree will fetch the source files correctly, but place the
628 objects in the link tree rather than the main source tree, so that
629 object files from different architectures don't interfere with each
632 If the link trees already exist, then rerunning
634 will update the links. This might be useful if the links somehow become
637 To turn some of the links in the link trees into real files, use the
642 Writes a list of all known architecture names to standard output. The
643 list is obtained by reading the
647 .SS make \fR[\fImake-arg\fR...]
650 .BI "run make " make-arg\fR...
654 .SS only\-arch \fIarch arch\fR...
655 Imposes an architecture restriction. Until cancelled by a later
659 command, the default build architectures will be limited to the
660 architectures listed on the command line. Architecture names may be
661 abbreviated as long as the abbreviation is not ambiguous.
665 .I "successfully built"
666 status of all architectures.
668 .SS rsh \fIhost\fR|\fIarch \fR[\fIcommand \fR[\fIargument\fR...]]
671 on a remote host, passing it the list of
675 command is unlike the standard
677 program and its replacements:
683 are not subjected to further shell expansion on the remote host.
685 The command is run with the remote current directory the same as the
686 local current directory, rather than the remote user's home directory.
688 The command is passed an environment constructed from the local
689 environment, the default remote environment, and
691 files, as described in the section
692 .B "Remote environment"
695 The remote command is run with standard input attached to
697 there is no way of running an interactive remote command through
700 The host on which to run the remote command may be specified as one of:
701 a standard host name (or IP address), an architecture name (which may
703 be abbreviated) signifying a host of the appropriate architecture, or
706 signifying the current host. (This last option may not sound useful,
707 but it's handy for testing.)
709 .SS run \fIcommand \fR[\fIargument\fR...]
710 Runs a command on all build architectures.
712 For each build architecture
714 \*(sw finds a host with the appropriate architecture, by choosing either
715 the local host or reading the hostname from the
717 file. It then performs the following actions on that host:
719 Sets the current directory to be the subdirectory named
721 of the directory from which the command was issued. This directory is
722 created if it doesn't already exist.
724 Sets up an environment constructed from the environment prevailing when
725 the command was issued, the default environment set up by
727 (or whatever equivalent remote execution program was actually used), and
730 files, as described in the section
731 .B "Remote environment"
734 Executes the program named
739 Output from the command is both appended to the file
745 command-line option. See the section
747 below for more details.
751 option was given on the command line, each architecture on which the
752 command succeeds (i.e., reports a zero exit code) is marked as
753 .IR "successfully built" ,
754 and further build commands will not affect it unless the
756 command line option is passed, until a
758 command is performed.
760 .SS setup \fIpackage version \fR[\fImaintainer\fR]
761 Sets up various pieces of information required by \*(sw. The
762 information here will be added into the main index file by a
764 command. The information is maintained in a file named
766 in the current directory.
770 should be the basic name of the package, with versioning information
776 .RB ` emacs\-19.34 '.
779 should be the version number of the package. The
781 should be the name of the person principally responsible for maintaining
782 the package's local installation. If this isn't specified, the calling
783 user's name is used as the maintainer.
787 command must be run before any build command.
789 .SS snaplink \fIfile \fR[\fIfile\fR...]
790 Creates architecture-specific versions of a file. Every
792 named on the command line is copied to
794 for every build architecture
796 overwriting any existing file or symbolic link of that name. If
798 contains leading directories then destination directories are created as
799 necessary for the output files. Note that the `snap' operation doesn't
800 actually need to follow creation of link trees.
804 Output from a build command is presented in one of a number of named
805 .IR "output styles" .
808 is always defined: it simply prefixes each line of output with the
809 name of the architecture which generated the line, which isn't actually
810 particularly easy to read. Other output styles may have been configured
811 into \*(sw when it was compiled.
813 The set of output styles supported by \*(sw varies according to how it
814 was configured. In any particular \*(sw program, you might have some of
818 Simply prefixes each output line with the name of the architecture it
819 came from. This is quite hard to read, but it doesn't require any
820 special operating system support or clever terminal.
823 Splits the terminal into independently scrolling areas, one for each
824 architecture, with a status line for each. Waits for a keypress when
825 all architectures are finished building.
829 style is used when the selected style doesn't work (for example, you
830 don't have a sufficiently capable terminal for curses output).
832 Output style names can be abbreviated as long as the abbreviation is
833 unambiguous. You can find the list of available output styles by
834 executing the command
838 (which is a little counter-intuitive, I know).
840 The author has plans to implement an X-based output style, but hasn't
841 got around to it yet.
844 .SH "REMOTE ENVIRONMENT"
845 The environment for a remote command (executed either through the
847 command, or a build command) is set up as follows:
849 The complete environment passed to \*(sw is used as a basis.
851 Any environment variables defined by the remote execution program
854 override corresponding variables in the basis environment.
858 variable is set to the name of the remote host's architecture.
860 Variable assignments are read from the global
861 .IB prefix /share/sw\-env
862 file. This makes some assignments which are useful everywhere, and will
863 then usually include the file
865 in the current directory.
869 files is documented separately in
874 This section describes how non-vendor software works at EBI. Chances
875 are that other sites will work differently. This description is here as
876 an example setup for \*(sw.
878 All the non-vendor software gets put in one big shared filesystem, and
879 is exported from our main fileserver. The filesystem is mounted on all
882 Architecture-neutral files are then
883 placed in the conventional subdirectories off
886 .BR /sw/common/share,
888 .BR /sw/common/info ).
889 Architecture specific files are stored in subdirectories off
890 .BR /sw/common/arch .
891 For example, Linux binaries go in
892 .BR /sw/common/arch/i386-linux/bin ,
893 and Solaris libraries in
894 .BR /sw/common/arch/sparc-solaris/lib .
895 Additionally, each architecture-specific subtree has a symbolic link
898 for each of the architecture-neutral subdirectories.
900 There is a symbolic link on every client, from
903 .BI /sw/common/arch/ arch\fR,
906 is the architecture of that client. Thus, every client has two
908 of the software repository: the `common' view where every host sees
909 exactly the same mapping between filenames and files, and the `arch'
910 view where every host sees the same mapping between filenames and
911 programs which do the same job.
913 And that's just about it.
917 The following environment variables are of interest to
921 Contains a space-separated list of default command-line options. These
922 are read before, and overridden by, the actual arguments given on the
926 The name of the command to use to run a `make'. This is resolved on the
927 local host once, rather than one for each build host, which is probably
928 a misfeature. To do something more clever, point
930 at a shell script which then picks out the right architecture-specific
932 program from the remote environment.
935 The name of the remote-shell program to use. By default, something
938 is chosen. I recommend using the excellent
943 The following files are of interest to
946 .IB prefix /sw\-index
947 The main index file, containing the list of which packages have been
948 installed for which architectures. See
950 for file format details.
952 .IB prefix /share/archtab
953 The architecture-to-host mapping file. See
955 for file format details.
957 .IB prefix /share/sw\-env
958 Contains global environment variable settings. See
960 for file format details.
962 .IB package /.sw\-info
963 Contains the persistent information about a particular package's build
966 for file format details.
968 .IB package /.sw\-env
969 Contains package-specific environment variable settings. See
971 for file format details.
973 .IB package / arch /.build\-log
974 Contains all the build output for a particular architecture. Usually
975 not very interesting, but might be handy one day.
980 merely unexpected behaviour modes. Silly you for thinking otherwise.
982 The \*(sw program, and this manual, are \*(mw productions, in association
983 with the European Bioinformatics Institute. They were written by Mark
984 Wooding <mdw@nsict.org>. Go and ask him if you have problems.
986 .\"----- That's all, folks --------------------------------------------------
~mdw / sw-tools / blob