4 \fBsw \fR[\fB\-fbip\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBconfigure \fR[\fIconfigure-arg\fR...]
6 \fBsw \fR[\fB\-f\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] \fBlinktree
8 \fBsw \fR[\fB\-fbip\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBmake \fR[\fImake-arg\fR...]
9 \fBsw only\-arch \fIarch \fR[\fIarch\fR...]
11 \fBsw rsh \fIhost\fR|\fIarch \fR[\fIcommand \fR[\fIargument\fR...]]
12 \fBsw \fR[\fB\-fbip\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] [\fB\-o \fIstyle\fR] \fBrun \fIcommand \fR[\fIargument\fR...]
13 \fBsw setup \fIpackage version \fR[\fImaintainer\fR]
14 \fBsw \fR[\fB\-f\fR] [\fB\-a \fIarch\fB,\fIarch\fR...] \fBsnaplink \fIfile \fR[\fIfile\fR...]
19 .\"--------------------------------------------------------------------------
23 The \*(sw tool attempts to take a lot of the work out of building and
24 installing source packages across multiple architectures. This section
25 will describe how to use \*(sw's features to best advantage in a number
28 To keep things concrete, I'll describe how things are done at the EBI,
29 although there's nothing EBI-specific about the \*(sw program itself.
30 For details about how we handle software at EBI, see the
34 By the way, this is quite a large manual. I recommend that you print a
35 copy onto paper and peruse it in a leisurely fashion, rather than
36 squinting at a monitor.
38 .\"--------------------------------------------------------------------------
40 .SH "SUMMARY OF BUILDING PACKAGES"
46 .BI "sw setup " "package version"
48 .IR "arch " [ arch ...]
59 .BI "sw setup " "package version"
61 .IR "arch " [ arch ...]
64 .IR "file " [ file ...]
65 .I [edit the appropriate files]
72 .\"--------------------------------------------------------------------------
74 .SH "8 STEPS TO INSTALLING A PACKAGE"
76 The following steps will guide you through your first (and maybe second)
77 package installations. In the description, I'll use
79 to refer to the package's name, and
81 to refer to its version number.
83 Not all the important features and options are described in this part of
84 the manual. View it more as a taster for the sorts of things \*(sw can
86 .SS "1. Download the source distribution"
87 Download the package's source distribution. This will normally be in an
88 archive called something like
89 .IB package - version .tar.gz\c
90 \&. At EBI, we put source archive files in
92 .SS "2. Unpack the source tree"
93 Unpack the source tree into the standard source directory. Each source
94 tree should have its own directory. Most well-packaged source
95 distributions unpack themselves into a neat directory, but less
96 fastidious programmers make archives which scatter files all over the
99 At EBI, we put the source trees in
101 so unpacking a well-formed source distribution looks like:
104 .BI "gzip \-dc ../tr/" package \- version ".tar.gz | tar xfv \-"
106 Ill-formed source distributions involve making the directory for the
107 package first, changing into it, and then unpacking into the current
111 .BI "mkdir " package \- version
112 .BI "cd " package \- version
113 .BI "gzip \-dc ../../tr/" package - version ".tar.gz | tar xfv \-"
115 When you've finished unpacking, make sure that your current directory is
116 the top level directory of the source tree you unpacked.
117 .SS "3. Tell \\*(sw what you're up to"
118 Now you need to tell \*(sw what you're working on. It will keep track of
119 this and other bits of information in a little file and refer to it
120 every now and then. It will also whinge at you and refuse to cooperate
121 if it can't find its little file, so it's as well to oblige.
123 To tell \*(sw to create this little file and initialize it with sensible
124 values, you just need to say
126 .BI "sw setup " "package version"
128 What could be easier?
129 .SS "4. Restrict the build to particular architectures"
130 Some packages don't work on all architectures, either because the author
131 wasn't sufficiently good at writing portable software, or because the
132 program's doing inherently nonportable things.
134 If that's the case, then you need to tell \*(sw to only build on the
135 architectures that really work. Do this with the
137 command. For example, if your package only works on Linux and Solaris,
140 sw only i386-linux sparc-solaris
142 You can get a list of the architecture names that \*(sw understands by
147 With a little bit of luck, these names ought to be self-explanatory.
149 If your package is properly portable and works everywhere then you don't
150 need to do anything for this step. Skip on to the next one.
151 .SS "5. Configure the package"
152 Now it gets complicated. If the package you're building uses
154 to configure itself for its current environment then you're in luck.
157 package because there's a script called
159 in the top source directory, and a file called
169 to configure the package on all the platforms it's meant to be built
170 for. When you've done that, move onto the next step.
176 then all is not lost (although it may be worthwhile complaining at the
177 package's author or maintainers). You need to make a collection of
179 one for each architecture. These link trees are little replicas of the
180 main source tree but with symbolic links instead of the real source
181 files. To make the link trees, run
185 Now, that's not actually quite what you wanted. It's made a link for
187 file in the source tree. Unfortunately, there are some files you'll
188 (probably) have to modify for each architecture in order to configure
189 the package to build properly. You can turn links in the link trees
190 into real independently editable files by
192 the links. Say for example that
196 need to be modified for each architecture. Running the command
198 sw snaplink Makefile config.h
200 is sufficient to do the right thing.
202 Now you must edit the snapped files to configure the package. Make sure
203 that the install directories are correctly set. At EBI, all the
204 software should be configured so that architecture neutral files end up
207 and architecture-specific files end up under
208 .BI /sw/common/arch/ arch\c
210 .SS "6. Build the package"
211 Now you've laid the groundwork, everything ought to be easy. Making the
212 program ought to involve simply typing
216 and waiting for a while. If you had the
218 library available when \*(sw was built, then your terminal will split
219 itself into little independently scrolling windows showing you the
220 progress for each architecture. If you're not privileged enough to have
222 then you get the output appropriately tagged with architecture names,
223 which is unfortunately fairly hard to read.
224 .SS "7. Install the package"
225 Most source packages (and almost certainly all
231 which installs the program correctly. You can run this from \*(sw by
238 option there tells \*(sw that this is the
240 When an architecture completes this step correctly, it's marked as being
241 properly installed, and \*(sw doesn't bother thinking about it again.
247 makefile target, then you have to install things manually. That's not
248 much fun, so moan at the package's author. When you've finished
249 fiddling with installation, run
253 just to tell \*(sw that you've installed everything OK. (This is a bit
255 .SS "8. Update the index"
256 Now that everything's built and installed, there's just one more command
261 This makes \*(sw update its main index of installed packages, telling it
262 which architectures packages are installed on, and who did it.
264 .\"--------------------------------------------------------------------------
266 .SH "REFERENCE INTRODUCTION"
268 That was a gentle introduction. This section contains the complete
271 far more detail that you probably want. If that's really the case, try
276 to read the available help text. There's quite a lot of it, and it
277 ought to keep you occupied for a while.
279 The basic \*(sw command line looks a bit like:
292 at the shell prompt, \*(sw gives you an extremely terse usage summary
293 and quits. You have to tell it to do
295 Most of the time you do this by giving \*(sw a
301 so that it knows what to do. There are some strange command line
302 options which cause \*(sw to do more exotic things, though.
304 .\"--------------------------------------------------------------------------
306 .SH "IMPLEMENTATION ODDITIES"
308 The \*(sw program that users use is really a small architecture-neutral
309 shell script, which works out the current architecture and executes the
310 appropriate architecture-specific main program. It's done this way so
311 that \*(sw knows that it can use the shell script to start itself up on
312 a remote host with a different architecture, something which it does
313 quite a lot. The only feature provided by the front-end shell script is
316 command line option, which shouldn't be used by anyone except \*(sw's build procedure anyway.
318 .\"--------------------------------------------------------------------------
320 .SH "COMMAND LINE OPTION REFERENCE"
322 Any \*(sw command line options can be put in the
324 environment variable. The \*(sw program will read space-separated
325 options from this variable before it reads the command line itself.
327 The \*(sw program usually understands two different names for each
328 option: a traditional Unix single-character name, and a long GNU-style
329 name. The short options behave in the normal Unix way: you can join
330 them together into single words with a
332 at the front, for example. The long names are always preceded by a
333 double dash. You can abbreviate long names as much as you like, as long
334 as the resulting abbreviation is unambiguous. In the descriptions
335 below, both the short and long names of the options are shown, but for
336 reasons of brevity required arguments are only shown for the long form.
338 There are conceptually two types of \*(sw command line options: those
339 which, usually for reasons of consistency with other programs, cause
340 \*(sw to do something immediately; and those which store some settings
341 for particular commands. The latter type are generally more useful.
342 It's worth bearing in mind, though, that the options are only used by a
343 few commands. The command reference describes exactly which commands
346 The complete list of command line options understood by the current
347 version of \*(sw is as follows:
350 Writes a fairly brief summary of \*(sw's command line options and a usage line for each of \*(sw's commands to standard output, and exits successfully.
352 .B "\-H, \-\-help\-full"
353 Writes a summary of \*(sw's command line options and a full paragraph of description for each of \*(sw's commands to standard output, and exits successfully. There's a lot of
354 text generated by this option. I recommend you pipe it through a pager
355 so that you can actually read it.
357 .B "\-v, \-\-version"
358 Writes \*(sw's version number to standard output and exits successfully. This is handy
359 when trying to decide whether your version of \*(sw has a particular feature, for example.
362 Writes a usage message so terse as to be nearly useless to standard
363 output and exits successfully. This is different from just running
365 because although both print the same useless message, running \*(sw without any arguments is considered an error, so the message is sent to
366 standard error and \*(sw will exit unsuccessfully.
368 .BI "\-a, \-\-arch " arch , arch\fR...
369 For commands which affect multiple architectures: only affect the
370 architectures specified. The architecture names may be separated by
371 commas, spaces or both, although clearly commas are most convenient on
372 the command line. Architecture names may be abbreviated as long as the
373 abbreviation is not ambiguous.
375 This option overrides any other decisions that \*(sw might make about which architectures to process based on the
377 list and the list of correctly built architectures for the current
381 For commands which affect multiple architectures: affect even
382 architectures that have been successfully built. This has no effect if
387 .B "\-i, \-\-install"
388 For build commands: this is the final install step, so label architectures
389 which successfully complete it as having been completely built. It's
390 normal to specify this option on the
391 .RB ` "make install" '
394 .BI "\-o, \-\-output " style
395 For build commands: select a style for the build output to be displayed
398 for more details on output styles.
401 For build commands: make a beep noise when the build finishes. This
402 provides a handy reminder if you're getting on with something else while
403 waiting for a long build. Use
407 to turn this option off. This option is disabled by default, although
408 may be enabled in the
410 environment variable.
412 .B "\-p, \-\-percent"
413 For build commands: enable translation of
415 sequences in command strings. These are described in more detail
417 .B "`%'-escape sequences"
422 to turn the option off. This option is enabled by default, although may
425 environment variable.
427 The remaining options aren't really intended for users. They're helpful
428 for \*(sw's own purposes, though, and described here for completeness' sake. They
429 don't have standard Unix short name equivalents, because they're not
430 usually useful for users.
433 Writes the \*(sw architecture name of the current host to standard output. This is used
434 by \*(sw's configuration script to determine the current architecture name. This
435 option is actually handled by a small shell script rather than by being
436 passed on to the main program. You shouldn't use this option yourself:
439 command instead. Because this option is handled by the shell script,
440 and the script isn't very clever, you can't abbreviate
442 on the command line, and it doesn't conflict with the similarly named
443 but completely different
445 option, which you can still abbreviate all the way down to just
449 Sets \*(sw's idea of its program name to
451 This is intended for use by \*(sw's front-end shell script, but isn't
452 actually needed at the moment. I can't see why you'd want to play with
453 this option, but it shouldn't do any harm.
455 .BI "\-\-remote " remote-command
456 Used by \*(sw when running commands on remote hosts. Don't use this yourself: it puts \*(sw into a very unfriendly mode and requires that you communicate with it
457 using a bizarre binary packet protocol. If you really must know more
458 about this, see the source code: it's quite well documented, really.
460 .\"--------------------------------------------------------------------------
464 The descriptions below make use of some technical terms:
466 .B "architecture restriction"
467 A state created by the
469 command, restricting the
470 .I "default build architectures"
471 to those listed as arguments to the command. An architecture
472 restriction may be cleared by
476 .B "build architectures"
477 The architectures which a
481 option is specified on the command line, then its argument specifies the
482 build architectures for this command; otherwise, the
483 .I "default build architectures"
487 A command which executes a process on multiple hosts simultaneously and
488 reports the results. The processes executed usually perform some part
489 of the building of a package. Currently, the build commands are
496 .B "default build architectures"
497 The architectures which, in the absence of a
499 command line option, are affected by a
500 .IR "build command" .
501 To determine the default build architectures, \*(sw reads the list of all architectures from the
503 file, and filters it: if the
505 command line option is
507 specified, then architectures marked as
508 .I "successfully built"
509 are removed from the list; if there is an
510 .I "architecture restriction"
511 in force, then the list is further filtered according to the
514 .B "successfully built"
515 A package is considered to be successfully built on a given architecture
516 if a build command given the
518 command-line option succeeds on a host of that architecture. The list
519 of successfully built architectures can be cleared by the
523 option causes \*(sw to ignore whether architectures have been successfully built when
525 .IR "default build architectures" .
527 .\"--------------------------------------------------------------------------
529 .SH "COMMAND REFERENCE"
531 This section describes all of the available \*(sw commands, in alphabetical order.
534 Clears an architecture restriction set by
536 Subsequent build commands will run across all known architectures not
537 yet successfully built, unless overridden by the
539 command-line option, or a later
544 Writes the name of the local host's architecture to standard output.
545 The architecture name is built into \*(sw at compile time.
547 Writes information from the
549 file to the installed packages index file
550 .IB prefix /sw-index\fR.
552 \*(sw performs some checks before committing information to the index
553 file. Firstly, all the expected architectures must be successfully
554 built. Secondly, the script
555 .IB prefix /share/sw-precommit\fR
556 is run, if it exists. This script must exit successfully if the commit
557 is to proceed. The script can be configured to enforce local policy
558 requirements on installed software.
562 script is passed a single argument, which is the package name to be
563 committed. Other useful information is passed in the environment:
566 The package name (again).
569 The package version number.
572 The package's maintainer.
575 The last date on which the package was modified.
578 The list of architectures on which the package has been built (separated
579 by spaces or commas).
582 The installation prefix with which \*(sw was configured.
584 The script should report any errors it finds to its standard error
587 .SS configure \fR[\fIconfigure-arg\fR...]
588 Equivalent to the command
590 .BI "run ../configure \-\-prefix=" prefix " " configure-arg\fR...
594 is the installation prefix with which \*(sw itself was configured. If you want to specify a different prefix, pass
599 It is expected that administrators will set up a file
600 .IB prefix /share/config.site
601 which sets up other Autoconf parameters once the prefix has been
602 chosen. See the Autoconf manual for more information.
605 Writes to standard output the name of a host with requested architecture
607 The hostname is read from the
612 Builds symbolic link trees. For each of the build architectures, a
613 directory with the architecture's name is created containing a symbolic
614 link corresponding to each file in the main source tree. Thus, a `make'
615 in the link tree will fetch the source files correctly, but place the
616 objects in the link tree rather than the main source tree, so that
617 object files from different architectures don't interfere with each
620 If the link trees already exist, then rerunning
622 will update the links. This might be useful if the links somehow become
625 To turn some of the links in the link trees into real files, use the
630 Writes a list of all known architecture names to standard output. The
631 list is obtained by reading the
635 .SS make \fR[\fImake-arg\fR...]
638 .BI "run make " make-arg\fR...
642 .SS only\-arch \fIarch arch\fR...
643 Imposes an architecture restriction. Until cancelled by a later
647 command, the default build architectures will be limited to the
648 architectures listed on the command line. Architecture names may be
649 abbreviated as long as the abbreviation is not ambiguous.
653 .I "successfully built"
654 status of all architectures.
656 .SS rsh \fIhost\fR|\fIarch \fR[\fIcommand \fR[\fIargument\fR...]]
659 on a remote host, passing it the list of
663 command is unlike the standard
665 program and its replacements:
671 are not subjected to further shell expansion on the remote host.
673 The command is run with the remote current directory the same as the
674 local current directory, rather than the remote user's home directory.
676 The command is passed an environment constructed from the local
677 environment, the default remote environment, and
679 files, as described in the section
680 .B "Remote environment"
683 The remote command is run with standard input attached to
685 there is no way of running an interactive remote command through
688 The host on which to run the remote command may be specified as one of:
689 a standard host name (or IP address), an architecture name (which may
691 be abbreviated) signifying a host of the appropriate architecture, or
694 signifying the current host. (This last option may not sound useful,
695 but it's handy for testing.)
697 .SS run \fIcommand \fR[\fIargument\fR...]
698 Runs a command on all build architectures.
700 For each build architecture
702 \*(sw finds a host with the appropriate architecture, by choosing either
703 the local host or reading the hostname from the
705 file. It then performs the following actions on that host:
707 Sets the current directory to be the subdirectory named
709 of the directory from which the command was issued. This directory is
710 created if it doesn't already exist.
712 Sets up an environment constructed from the environment prevailing when
713 the command was issued, the default environment set up by
715 (or whatever equivalent remote execution program was actually used), and
718 files, as described in the section
719 .B "Remote environment"
722 Executes the program named
727 The command name and arguments may be subject to
729 substitution, depending on whether the
733 sequences are described in the section
734 .B "`%'-escape sequences"
737 Output from the command is both appended to the file
743 command-line option. See the section
745 below for more details.
749 option was given on the command line, each architecture on which the
750 command succeeds (i.e., reports a zero exit code) is marked as
751 .IR "successfully built" ,
752 and further build commands will not affect it unless the
754 command line option is passed, until a
756 command is performed.
758 .SS setup \fIpackage version \fR[\fImaintainer\fR]
759 Sets up various pieces of information required by \*(sw. The
760 information here will be added into the main index file by a
762 command. The information is maintained in a file named
764 in the current directory.
768 should be the basic name of the package, with versioning information
774 .RB ` emacs\-19.34 '.
777 should be the version number of the package. The
779 should be the name of the person principally responsible for maintaining
780 the package's local installation. If this isn't specified, the calling
781 user's name is used as the maintainer.
785 command must be run before any build command.
787 .SS snaplink \fIfile \fR[\fIfile\fR...]
788 Creates architecture-specific versions of a file. Every
790 named on the command line is copied to
792 for every build architecture
794 overwriting any existing file or symbolic link of that name. If
796 contains leading directories then destination directories are created as
797 necessary for the output files. Note that the `snap' operation doesn't
798 actually need to follow creation of link trees.
800 .\"--------------------------------------------------------------------------
802 .SH "`%'-ESCAPE SUBSTITUTION"
806 option is enabled, build commands and arguments are subject to
808 substitution before being executed. Certain two-character sequences,
809 with the first character
811 are replaced with strings, as follows:
814 The architecture name of the host executing the command.
817 The hostname of the host executing the command.
820 The directory prefix with which \*(sw was installed.
823 The name of the package being built.
826 The version number of the package being built.
829 The name of the maintainer of the package being built.
838 sequences which aren't understood are left as they are.
840 .\"--------------------------------------------------------------------------
844 Output from a build command is presented in one of a number of named
845 .IR "output styles" .
848 is always defined: it simply prefixes each line of output with the
849 name of the architecture which generated the line, which isn't actually
850 particularly easy to read. Other output styles may have been configured
851 into \*(sw when it was compiled.
853 The set of output styles supported by \*(sw varies according to how it
854 was configured. In any particular \*(sw program, you might have some of
858 Simply prefixes each output line with the name of the architecture it
859 came from. This is quite hard to read, but it doesn't require any
860 special operating system support or clever terminal.
863 Splits the terminal into independently scrolling areas, one for each
864 architecture, with a status line for each. Waits for a keypress when
865 all architectures are finished building.
869 style is used when the selected style doesn't work (for example, you
870 don't have a sufficiently capable terminal for curses output).
872 Output style names can be abbreviated as long as the abbreviation is
873 unambiguous. You can find the list of available output styles by
874 executing the command
878 (which is a little counter-intuitive, I know).
880 The author has plans to implement an X-based output style, but hasn't
881 got around to it yet.
883 .\"--------------------------------------------------------------------------
885 .SH "REMOTE ENVIRONMENT"
887 The environment for a remote command (executed either through the
889 command, or a build command) is set up as follows:
891 The complete environment passed to \*(sw is used as a basis.
893 Any environment variables defined by the remote execution program
896 override corresponding variables in the basis environment.
900 variable is set to the name of the remote host's architecture.
902 Variable assignments are read from the global
903 .IB prefix /share/sw\-env
904 file. This makes some assignments which are useful everywhere, and will
905 then usually include the file
907 in the current directory.
911 files is documented separately in
914 .\"--------------------------------------------------------------------------
918 This section describes how non-vendor software works at EBI. Chances
919 are that other sites will work differently. This description is here as
920 an example setup for \*(sw.
922 All the non-vendor software gets put in one big shared filesystem, and
923 is exported from our main fileserver. The filesystem is mounted on all
926 Architecture-neutral files are then
927 placed in the conventional subdirectories off
930 .BR /sw/common/share,
932 .BR /sw/common/info ).
933 Architecture specific files are stored in subdirectories off
934 .BR /sw/common/arch .
935 For example, Linux binaries go in
936 .BR /sw/common/arch/i386-linux/bin ,
937 and Solaris libraries in
938 .BR /sw/common/arch/sparc-solaris/lib .
939 Additionally, each architecture-specific subtree has a symbolic link
942 for each of the architecture-neutral subdirectories.
944 There is a symbolic link on every client, from
947 .BI /sw/common/arch/ arch\fR,
950 is the architecture of that client. Thus, every client has two
952 of the software repository: the `common' view where every host sees
953 exactly the same mapping between filenames and files, and the `arch'
954 view where every host sees the same mapping between filenames and
955 programs which do the same job.
957 And that's just about it.
959 .\"--------------------------------------------------------------------------
963 The following environment variables are of interest to \*(sw:
966 Contains a space-separated list of default command-line options. These
967 are read before, and overridden by, the actual arguments given on the
971 The name of the command to use to run a `make'. This is resolved on the
972 local host once, rather than one for each build host, which is probably
973 a misfeature. To do something more clever, point
975 at a shell script which then picks out the right architecture-specific
977 program from the remote environment.
980 The name of the remote-shell program to use. By default, something
983 is chosen. I recommend using the excellent
987 .\"--------------------------------------------------------------------------
991 The following files are of interest to \*(sw:
993 .IB prefix /sw\-index
994 The main index file, containing the list of which packages have been
995 installed for which architectures. See
997 for file format details.
999 .IB prefix /share/archtab
1000 The architecture-to-host mapping file. See
1002 for file format details.
1004 .IB prefix /share/sw\-env
1005 Contains global environment variable settings. See
1007 for file format details.
1009 .IB prefix /share/sw\-precommit
1010 Optional script used to approve commit requests. See the
1012 command above for calling details.
1014 for file format details.
1016 .IB package /.sw\-info
1017 Contains the persistent information about a particular package's build
1020 for file format details.
1022 .IB package /.sw\-env
1023 Contains package-specific environment variable settings. See
1025 for file format details.
1027 .IB package / arch /.build\-log
1028 Contains all the build output for a particular architecture. Usually
1029 not very interesting, but might be handy one day.
1031 .\"--------------------------------------------------------------------------
1035 There are no bugs in
1037 merely unexpected behaviour modes. Silly you for thinking otherwise.
1049 The \*(sw program, and this manual, are \*(mw productions, in association
1050 with the European Bioinformatics Institute. They were written by Mark
1051 Wooding <mdw@nsict.org>. Go and ask him if you have problems.
1053 .\"----- That's all, folks --------------------------------------------------
~mdw / sw-tools / blob