Initial revision

[ssr] / StraySrc / Libraries / Sapphire / README

A very brief overview of Sapphire
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

_____________________________________________________________________________

IS THIS ALL THERE IS?


The Grand Plan was to write a really good library, sell some spiffy
applications which used it, and then sell the library.  This
spectacularly failed to work, mainly because we had no ideas for killer
applications.  The result is Sapphire.

Since we'd planned to write some big applications before releasing
Sapphire to anyone else, we took a rather relaxed attitude to
documentation.  There is some LaTeX documentation lying around, but
mostly it's initial versions of things, with big gaps where less
interesting text should be filled in later.  In short, it's no use to
anyone.

On the other hand, we realised that a library of this complexity with no
documentation at all would be a total disaster, so all the header files
are profusely commented, mostly to the standard of a final-copy
reference manual.

This text file is intended as a brief hacker's eye view of how to write
Sapphire applications, and how some of the more complicated and/or
interesting bits work.

_____________________________________________________________________________

ABOUT THE AUTHORS


Sapphire was designed and written, over the course of a couple of years,
by Mark Wooding and Tim `Tycho' Armes.  Each source file usually bears
the initials (`MDW' or `TMA') of the person mainly responsible for
writing it.  In some cases, this is misleading.  For example, although
the `msgs' code for translating message tokens into strings is labelled
`MDW', the first two versions of this file were written by Tim: I took
over later because I rewrote it to kill the bugs.  Similarly, the fact
that one of us wrote a source file doesn't tell you who designed it. For
example, the `menu' code was entirely written by Tim, although we both
worked on the design for several days beforehand.

_____________________________________________________________________________

A HISTORY LESSON


Sapphire is an attempt to learn from the experience we both had with
STEEL.  STEEL (Straylight's Extensive Event-driven Library) is a
more-or-less complete re-write of RISC_OSLib, and although it had given
up being call-compatible, and had diverged quite considerably by the
release of Desktop C, it suffered because of the limitations of its
structure.  All the original non-RISC_OSLib code had been written by me,
in a rather ragged way.

The name Sapphire comes from the old ITV sci-fi series `Sapphire and
Steel', obviously.  It's a rotten joke, but that's what you get when you
think up names for libraries in a pub.  The idea was that two of us
could do much better than either acting alone, by bouncing ideas between
each other until we sorted out the problems.  We threw out any thought
of compatibility with any existing software, and any hope of interfacing
a high-level language.  Then we started coding.

_____________________________________________________________________________

INFLUENCES


The design of Sapphire has been influenced by looking at lots of other
pieces of software.  A lot of structure is based on RISC_OSLib: STEEL
was derived from RISC_OSLib, and it did a lot of things right.  Another
particularly strong influence is Computer Concepts' Advanced Box
Interface (ABI) as used in Impression and ArtWorks.  (I even went as far
as buying the ArtWorks SDK more-or-less to get my hands on some ABI
documentation.)  There was an influence from what I perceived to be `the
way OS/2's Presentation Manager does it', although this wasn't always
positive.

_____________________________________________________________________________

SUPPORTED LANGUAGES


Sapphire supports ARM assembler.  That's about it.  One of the last
Sapphire-related projects was interfacing C to Sapphire; this is still
mostly incomplete.  The basic job of making C code call Sapphire code
and vice-versa is finished and reliable.  The hard bit, handling
Sapphire-style definition tables, isn't even begun yet.

One of the projects I set myself over a year ago was a language, then
called `cdata', which would act as a pre-processor for C and allow
definition tables to be created without the amount of pain currently
required.  (If you don't believe me, see the definition of the icon bar
menu in the `csapph' test program.)

_____________________________________________________________________________

THE SAPPHIRE ENVIRONMENT


Sapphire doesn't use APCS.  Let's get that clear right away.  You try to
use APCS, and Sapphire will hate you for the rest of your life.

Because the target language is assembler, Sapphire itself sometimes
plays fast and loose with the rules, but the basic ideas, such as they
are, are presented here.

Registers R10-R15 are assigned special purposes.

For architectural reasons, R15 is the program counter, and R14 is the
subroutine link register.  Because R14 is modified by subroutine calls,
it is often used as a temorary register in computations.

R13 points to a full descending stack.  Sapphire doesn't allocate a very
big stack unless you ask it to, so be careful with recursion.  There's
no stack limit checking either.

R10 and R12 provide the program's current state.  R12 is the `workspace'
pointer, and R10 is the `object' pointer.  The idea is that R12 points
to a statically allocated data block, and R10 points to a `currently
interesting object'.  For example, in a multi-document editor, R10 would
point to an information block describing the document currently being
processed.  Because it's important to hide information between
components of a program, different components have different workspace
areas.  Similarly, different parts of a program see the world at
different levels: the part of a drawing program which is concerned with
file-level operations keeps R10 as a pointer to the document anchor; the
part concerned with manipulating individual objects within a document
might keep R10 referring to an object.  Note that R10 and R12 rarely
take part in inter-component interfaces.  Object handles are passed
between components in low-numbered registers, as normal arguments, and
then copied into R10 later.

R11 is Sapphire's `application context' pointer.  It's also known as the
`scratchpad' pointer.  From R11 upwards is the `scratchpad': a block of
memory available for any purpose.  Any procedure may corrupt the
scratchpad, so it's never safe to assume that it is preserved over
inter-component calls.  Immediately below R11 are workspace offsets.
These allow Sapphire and its client applications to find statically
allocated workspace.  If R11 is pointing to the wrong place, Sapphire
will crash!

On a procedure call, this is what usually happens:

  R0--R9 == arguments
  R10, R12 == variables owned by caller
  R13 == pointer to full descending stack
  R11 == application context pointer (Scratchpad)

On return:

  R0--R9 == return values, or usually preserved
  R10, R12 preserved
  R13 == preserved
  R11 == application context pointer
  Flags either preserved or set as a return value

Sapphire's fond of returning information in the flags.  For example, if
you ask for memory and there isn't any, allocation functions return with
carry set.  Sometimes, errors are returned in the normal way, by setting
overflow and pointing R0 at an error block.  If you don't return
something useful in a flag, preserve it.

Sapphire behaves slightly different when it's calling its client back.
When you register a handler function, you supply a pointer to the code,
and R10 and R12 values to pass it.  It will ensure that these values are
in those registers at call time.

_____________________________________________________________________________

SAPPHIRE'S MEMORY MAP


Sapphire has strong ideas about how an application's memory is laid out.
Here's a poor attempt at a diagram:


                   _____________  <--- Top of current wimpslot
                  |             |
                  :    Flex     :
                  :    heap     :
                  |             |
                  |-------------|
                  |  Resizing   |
                  |    heap     |
                  |-------------| <--- Top of initial wimpslot
              R13 |    Stack    |
                  |-------------|
              R11 | Scratchpad  |
                  | App context |
                  |-------------|
                  |             |
                  |Default heap |
                  |             |
                  |-------------|
              R12 |  Workspace  |
                  |-------------|
                  | Application |
                  |    code     |
            &8000 |_____________| <--- Base of application memory


Hmmm.  Not too shabby.

Now, in words:

During initialisation, Sapphire's kernel finds the top of available
memory and puts the stack there, growing downwards.  Below the stack, it
allocates the scratchpad area, so that there's a small amount of `slop'
between the the stack growing down and the scratchpad growing up.  The
kernel then examines the available library components, and allocates
workspace for them, starting just above the application's read-write
area.  (A Sapphire application shouldn't actually have any read-write
data -- R12-space is a much better way of doing the same job.)  Once
that's done, it initialises an OS_Heap heap between the top of workspace
and the bottom of the application context area just below the
scratchpad.  This is the `default heap'.

Above the initial wimpslot is Flex space.  Flex is Sapphire's shifting
heap manager.  It's named after RISC_OSLib's shifting heap, although
it's completely rewritten, and much extended.  Flex has complete control
over the area above the initial wimpslot.  The first Flex block is taken
by a resizing heap manager, which creates a nonshifting heap there. This
is registered automatically with Sapphire's allocation manager, so you
can forget about it completely.

This brings me on to...

_____________________________________________________________________________

MEMORY MANAGEMENT


Sapphire sees three sorts of memory:

  * Static memory.  This is allocated at initialisation time by the
    kernel, and put into the workspace area.  R12 points here.

  * Fixed-size blocks representing objects.  These are allocated and
    freed at run-time by the program, using Sapphire's allocation
    manager.  R10 points to one of these, usually.

  * Large or variable-size blocks representing data.  These are
    allocated and freed by the program using Flex.

Typically each large block in Flex-space has an accompanying `anchor'
block in heap-space.

The allocation manager attempts to paper over the mess involved with
having more than one nonshifting heap.  STEEL actually does this job
better, putting the client in control of where blocks get allocated.
Sapphire's approach, to fill up one heap and move on to the next, is
simpler, but can lead to fragmentation.

_____________________________________________________________________________

INITIALISATION


RISC_OSLib (and STEEL) programs start off with a huge chunk of
initialisation code, most of which is really boring:

	wimpt_init("progname");
	dbox_init();
	win_init();
	/* ... */

Sapphire gets rid of all that by using a neat feature of AOF.  Each
library component, or `unit' as we called them, contains a four-word
table entry.  The linker collects the entries together, and the kernel
can examine them all to see which units are linked in.  Each entry
contains:

  * The address to store the unit's workspace offset.
  * The size of workspace required.
  * Minimum acceptable Scratchpad size (if more than 256 bytes).
  * Address of initialisation procedure.

By convention, the first word of a unit's workspace contains flags, bit
zero of which indicates that the unit has been initialised.  A unit's
initialisation procedure performs the following actions:

  * It checks it's not already initialised.
  * It initialises anything it depends on (this causes the required
    units to be linked too).
  * It initialises itself.
  * It sets its initialised flag and returns.

Library initialisation is a two-step process.  The first step,
`sapphire_init', initialises the kernel and the basic memory map.  The
second step, `sapphire_libInit', initialises the library units.

_____________________________________________________________________________

DYNAMIC LINKING


The original idea was that Sapphire would be really small.  Well, take a
look at its feature set and tell me that it's big.  So we paid no
attention to making Sapphire dynamically linkable.  When it became
obvious that this wasn't working properly, I started investigating
methods for making Sapphire into a DLL.

I was saved by two features of Sapphire code:

  * Sapphire programs preserve R11 everywhere.
  * Sapphire's kernel allocates workspace for the units at run-time.

The first fact gives me re-entrancy automatically.  R11 becomes the one
thing that the dynamically-linked Sapphire library can use to decide
where its workspace is.

Finding workspace is rather grubby in Sapphire, so it's hidden beneath a
macro.  For the curious, this is what happens:

		LDR	R12,workoff_addr
		LDR	R14,[R11,#-magic_offset]
		ADD	R12,R14,R12

The trick works because the allocation of workspace /offsets/ depends
only on the order of units within the library, so they can be shared;
the value loaded from R11-space is worked out by the kernel and put in
application memory on startup.

Most of the kernel ends up in the DLL.  The application is linked
against a really small stub, which basically contains front ends for a
few kernel functions to (maybe) find the Sapphire DLLs, point to a
collection of tables, and call the corresponding function in the core.

The exact mechanism is complicated.  Interested readers are referred to
the source code.

The upshot of all of this is that, although SDLS was designed
specifically to solve the problem of APCS shared libraries, Sapphire's
style of shared library requires much less code to be linked into the
client, much thinner veneering (because the kernel handles re-entrancy),
and the result is both more robust and cleaner.

_____________________________________________________________________________

ERROR HANDLING


Sapphire programs know about two sorts of errors:

  * Program errors, which should never occur.
  * Environmental errors, which must be anticipated.

Environmental errors are reported by setting the V flag on exit from a
function.  Sapphire routines are documented according to whether they
can return errors: it is safe to ignore the V flag on exit from a
routine whose documentation does not state that it returns errors. An
environmental error typically requires some sort of user intervention,
and suggests a friendly explanation that some sort of fault has
occurred.

Program errors are raised by OS_GenerateError and caught by Sapphire's
error handler.  There's a rule of thumb which states: `Never test for an
error you can't handle.'  Sapphire follows this advice.  Most calls to
the operating system have the X bit clear, causing errors to be sent to
the error handler.  Program error messages are intentionally technical.
The idea is to encourage naive users to write them down, so that they
remember them and report them accurately.

Part of the initial design was that a Sapphire application should
/never/ exit as a result of program fault.  A user should always be able
to try to get to a save box and save data.  (Too much experience of
Paint falling over with type 5s prompted this decision.)  The
application pops up an error report giving the user a choice between
killing the program or trying to soldier on.  Soldiering on seemed
remarkably successful in general.

A later addition to Sapphire was `Structured Exception Handling', which
is a lower-level version of C++'s try/throw/catch exception handling.
This builds upon the existing base, and allows programs to tidy up in
the event of errors before propagating them back down to the bottom
level `Continue/Quit' handler.


This concludes the tour of Sapphire's run-time support functionality.
The rest describes plain ol' library features.

_____________________________________________________________________________

EVENT HANDLING


Sapphire has a fairly simple but highly effective strategy for dealing
with events, inspired in part by the FilterManager module in RISC OS 3.

The `event' unit is responsible for low-level event handling.
Applications call `event_poll' instead of calling the SWI Wimp_PollIdle
directly; the calling sequences are almost the same for both routines.
`event_poll' returns C set if it handled the event itself, and C clear
if it failed to handle it.  A defalt handler procedure is provided to
perform any required actions for a particular event.  A Sapphire
application's poll loop typically looks like this:

loop		BL	event_poll
		BLCC	handle_unknowns
		BLCC	defHandler
		B	loop

and that's it.

Most of the hard work is done by filters.  Sapphire knows about three
types of filters:

  * Pre-filters are shown arguments to Wimp_Poll before it's called, and
    can modify them in controlled ways, e.g., clearing event mask bits,
    or changing the `time-to-return' argument.  A pre-filter can
    `claim' a call to Wimp_Poll by storing a `fake' event in the buffer
    and returning carry set.  Other pre-filters are skipped, and the
    fake event is passed to the post-filters.

  * Post-filters are shown the result from a call to Wimp_Poll.  They
    can perform any action they like, except for modifying the event.  A
    post-filter can claim an event by returning C set; other
    post-filters are not called, and `event_poll' returns C set.

  * Fake-event filters are a novel idea introduced to handle transient
    dialogue boxes.  Immediately after a return from Wimp_Poll, Sapphire
    scans the list of fake-event handlers.  Each one is entitled to
    `steal' this event and replace it with another by returning carry
    set.  Once an event has been stolen, it is passed through the
    post-filters.  When `event_poll' is called again, Sapphire
    restores the previous genuine event and continues calling fake-event
    filters from the point at which it left off.  Thus, Sapphire
    provides a general mechanism for inserting events.

Everything else is based off this mechanism.

The `win' unit dispatches window-specific events to window handlers by
registering a post-filter.  The `menu' unit also establishes a
post-filter, and passes events to the owner of the current menu.

The main use for fake-event handlers is to handle transient windows.  A
separate Sapphire unit, `transWin', remembers the handle of the current
transient window.  When an event arrives, its fake-event handler checks
whether the transient window is open.  If it has been closed, it steals
the event, whatever it was, and substitutes a fake `window close' event
to be picked up by the owner of the transient window.

[Actually, `transWin' is careful to allow redraw events through
unmolested.  Bad window flicker results if redraw requests are tampered
with.]

_____________________________________________________________________________

MENUS IN SAPPHIRE


Sapphire knows about two types of menus, and its interface to
applications reflects this.  Standard RISC OS transient menus will be
familiar to most readers.  Straylight tearoff menus will be less well
known -- see Glass for a real-life example of tearoff menus in action.

Sapphire's interface to handling menus is inspired by Computer Concepts'
Advanced Box Interface system.  Menus are always constructed
dynamically, as needed.  The procedure `menu_create' is given four
arguments:

  * The address of a menu definition.
  * The address of a menu event handler.
  * R10 and R12 values to pass to the handler.

The `menu_create' procedure may be called any number of times before the
next Wimp_Poll: the menu definitions are concatenated.  This allows
applications to create dynamic and context-sensitive menus easily.

The first `menu_create' call is different from the others: the menu
definition must contain a menu title.  (It need not contain any menu
items, though.)  Both the menu title and the items are /variable size/
objects -- the more odd features a menu item has, the more space its
definition occupies.  Menu definitions are also /read-only/.  Dynamic
features like ticking and shading must be handled separately.

Each menu item (and title) is begun with a `feature mask' -- a word
containing a flag bit for each feature.  The feature mask is followed by
data appropriate to the various features selected.  For example, the
`has a submenu' feature requires a pointer to the submenu definition and
a pointer to the procedure which will handle events for the submenu.

Menu features which require dynamic information (e.g., context-sensitive
message strings, or ticking and shading) have, as part of their feature
data, offsets from the menu handler's R10 containing the required
information.  The `shade item' feature requires an offset from R10 and a
bit position -- it will shade the item when the appropriate bit in the
word at [R10, #offset] is set.  Sapphire's menu system understands the
concept of `radio items' -- groups of items of which only one may be
ticked at a time.  The feature data for a radio item consists of an R10
offset and a value to match -- the application then sets [R10, #offset]
to be (for example) 0 for the first item in the group, 1 for the second,
and so on.

It's important to get the hang of the concepts here, because lots of
other bits of Sapphire use this same idea.

The `menuDefs.sh' header file contains a large number of macros which
hide most of the mess of this.

_____________________________________________________________________________

DIALOGUE BOXES


Sapphire's dialogue box handling is extremely powerful.  There's also a
lot of code in `dbox.s' which does jobs that the WindowManager ought to
do, or does wrong.

Dialogue boxes are `resources'.  Sapphire tries to keep the
interface to handling resources consistent.  When you want one, you
`create' it, and when you don't need it any more, you `destroy' it.
A successful `create' of a dialogue box yields a `dbox' -- a handle
which represents the dialogue box.

Sapphire is carefully constructed so that dialogue boxes don't need to
exist unless they're actually in use.  Most programs create dialogue
boxes only when they need to appear on-screen, and destroy them again
once they're closed.  To this end, Sapphire generates fake close events
for transient windows.

Dialogue boxes can be created from several different sources of
information, and to handle this, there are actually three different
`create' routines:

  * `dbox_create' is the standard call.  A client passes a string naming
    the template from which to create the dialogue, and the dialogue box
    manager returns a handle.  The window definition is copied, along
    with all the indirected data, so that you can create multiple
    instances of a template without tripping over aliasing problems.

  * `dbox_fromEmbedded' creates a dialogue from an `embedded template'
    -- a compact and easily expanded representation of a window template
    which is statically linked into the client program.  Again, the data
    is copied from the template, so aliasing problems don't exist.

  * `dbox_fromDefn' is the most powerful call (and the others are
    implemented in terms of it).  The client passes a pointer to a
    complete window definition, which the Sapphire uses `as is',
    without copying it.

Embedded templates were introduced for very small programs (e.g.,
DLLMerge).  Sapphire makes use of them because common dialogue boxes
have been stored in a shared `Sapphire.Resources' DLL as embedded
templates.  This makes programs easier to upgrade for new versions of
Sapphire (just copy the new DLLs in and you get the new dialogues free)
and reduces resource requirements since there's only one copy in memory
at any given time.

Displaying dialogue boxes introduces more flexibility.  The procedure
`dbox_open' is given a dbox and an `open style', and a collection of
arguments appropriate to the style.  Styles permitted are:

  * In its current position (or in the position it was in the template
    file).

  * In the middle of the screen.

  * Centred over the mouse pointer.

  * At a given Y coordinate (but the current X coordinate).

  * At a given X and Y position.

The open style also contains a `transient or persistent' flag, which is
useful for dialogue boxes hanging off of menu items.  If a menu is open
currently, Sapphire will automatically open the dbox as a submenu, in
the correct place, unless you explicitly instruct it not to do this
using another open style bit.

Sapphire doesn't pass Wimp events directly on to client event handlers.
Instead, it pre-processes them, putting the important information from
the event into registers, and sending a `dbox event' to the handler.
The handler can then either `claim' the event for itself (by saying that
it's handled it) or pass it on.  Unclaimed events are acted upon by
Sapphire.

As an example of the sort of preprocessing Sapphire does for dialogue
box events, it breaks a redraw request into an individual event for each
rectangle.  If its redraw events are unclaimed (as they ought to be), it
will call Sculptrix to fill in the 3D borders around the icons.

Sapphire handles radio buttons itself, if click events on them are
unclaimed.  Using type-11 buttons and a non-zero ESG means that the user
can deselect all buttons in a set by clicking the selected one with
`adjust'.  An application can deal with this by selecting the icon
explicitly, but (a) this causes flicker, and (b) if the application is
going to the trouble of having explicit code to deal with the situation,
it may as well do the job properly.  Sapphire considers an icon with
button type 3 and non-zero ESG to be a `Sapphire radio button' and does
the right thing with it.

Shaded icons are also handled by Sapphire rather than the Wimp, because
the standard behaviour is highly inconsistent.  For example, sometimes
icon backgrounds are shaded in addition to foregrounds.  In text-and-
sprite icons, the text is /not/ shaded when it ought to be (for
consistency with text-only icons).  It's basically a mess.  Sapphire
uses a complicated shading algorithm which seems fairly close to the
behaviour of ABI.

Sapphire also handles caret blinking and cursor changing over writable
icons automatically.

Keypresses also receive default handling from Sapphire.  Unclaimed
cursor keys cause the caret to move between writable icons, scrolling
the window where necessary (both horizontally and vertically) to ensure
that the new focus icon is visible.

The procedure to set a string in an indirected icon is probably
excessive.  It ensures that the string really needs changing before
flickering the icon.  It truncates the string rather than overflowing
the icon's buffer, discarding either the start or the end depending on
the icon's `right align' flag, optionally putting a `...' to indicate
that a truncation was performed.

Sapphire inherited a curious feature called `embedded titles' from
STEEL, which in turn was inspired by RISC OS version of the game
`Elite'.  A client nominates an icon within a window which has no title
bar as being an `embedded title'.  Sapphire will thereafter
automatically draw a Sculptrix group box around this icon, using the
window's real title text as the group title.  This is used in `Info'
windows, as well as the warning, note and error windows.

Sapphire can handle `nonpolling' windows.  In some cases it's desirable
to arrest the user's attention by requiring acknowledgement of a message
or the taking of a decision, and sometimes continuing to poll would be
dangerous or undesirable for other reasons.  For example, a serious
error report mustn't multitask, because the error might recur.  The `To
save...' message caused by attempting to save a file without a full
pathname shouldn't multitask, because the click on the OK button will
close the menu tree (which is extremely irritating).  The only way to
handle a prequit message under RISC OS 2 is to ask the user whether
quitting is permitted in a nonpolling way, so the decision to cancel a
shutdown sequence can be made before an acknowledgement is returned to
the task manager.  The handling for nonpolling windows is split into two
parts.  The `nopoll' unit displays dialogue boxes and reports mouse
clicks and keypresses as fake events through Sapphire's pre-handler
mechanism (so nonpolling dialogues present an identical programmer
interface to normal ones).  The `buttons' unit displays a given
dialogue, permitting a collection of action buttons to be given
appropriate pieces of text.

The real richness of Sapphire's dialogue box handling comes from custom
controls.  A whole subsystem, `dbx' is provided for dealing with custom
controls.

The client registers a table defining the required controls for a
particular dialogue box.  The table contains variable size entries, each
one beginning with a standard header:

  * Number of the icon which hosts the control.
  * Pointer to the control definition.
  * Various flags (control specific).
  * The offset of the control's writable data within the dialogue
    handler's R10-space.

This is followed by whatever constant data the control handler requires.
Sapphire dbx controls are always hosted by icons.  This makes them easy
to lay out in template editors.

Custom controls generate their own events which are sent to the client's
handler procedure.  They are also given dialogue box events (after the
client has had a chance to claim them for itself) as `dbx events'.  For
example, Sapphire will only ask a control to draw itself if (a) the
control requests redraw events and (b) the current rectangle intersects
the control's host icon.

Sapphire takes redrawing of controls very seriously.  It ensures that
the graphics clipping area is set to the intersection of the control's
host icon and the redraw rectangle, to prevent any `accidents' from
messing up the rest of the display.  It passes this amended rectangle to
the control to allow it to perform source-level clipping.  The custom
RGB-coloursquare and HSV-colourcircle controls in the (unfinished)
colour selector use the provided rectangle to plot the new graphics by
direct screen access.

Predefined custom controls are:

  * A slider.  This is written very carefully to avoid any flicker while
    the bar is being dragged or updated.

  * A `bump' arrow button, which presses its icon in while it's being
    held down, and autorepeats.  The arrow control keeps a count of how
    many `ticks' have been missed, and sends them to the dialogue box as
    large packets, rather than individually.  This makes programs which
    spend a long time processing bump requests feel responsive even when
    they're really being slow.

  * A file icon, which represents a filetype in a window.  File icons
    can optionally be draggable (for save operations).  When solid drags
    are requested by the user (and the DragASprite module is available),
    the control will make the icon disappear while the icon is being
    dragged, to produce the illusion of the icon being `lifted off' the
    dialogue box.

  * A `string set' control, which drops down a menu of strings and
    allows the user to choose one, which it displays.

  * A simple `colour pot' which allows a user to choose one of the
    sixteen standard Wimp colours.  This control pops open a small
    dialogue box of its own when requsted.

Custom controls provide a great deal of functionality at very little
programmer effort.  Why no other library for RISC OS handles them
properly is a mystery to me.  The Toolbox has a go, but fails miserably;
compared to the power of Sapphire's `dbx' subsystem, it might as well
not bother.

(Sapphire was mature when the Toolbox betas were available.  The idea
for proper custom controls came from OS/2, but the implementation is
very much in Sapphire's own style.)

_____________________________________________________________________________

DATA TRANSFER


It used to be the case that data transfer was one of the really hard
bits of an application.  I remember trying to put it off as long as
possible whenever I wrote a STEEL application.  Sapphire tries very hard
to make data transfer as painless as possible.

Both loading and saving have two layers to them.  Underneath, there's a
low-level interface which exposes the nature of the transfer (whether
it's to a file, or in-memory).  Above that, there's an abstract
interface which presents both transfer types as a (non-seekable) stream
of data, providing buffering and a simple programming interface.  This
high-level interface works perfectly well on files even without a data
transfer context, and occasionally gets (ab)used as a general stream
interface to file data, for example, when saving preferences files.

The lower-level code was designed to be useful both in conjunction with
the high-level interfaces and on its own.  Often, the low-level
interface is the better choice.

Oddly, the low-level interface is higher-level than RISC_OSLib's.

A low-level data save operation is initiated by the `save' call.  It
requires a large number of arguments, detailing the recipient's window
handle, the filename, filetype, and a block of handling routines.  The
handling routines are:

  * Save: write your data to a file, given its name.  Return an error if
    saving failed.

  * Send: return (giving a base pointer and size) a block to send to the
    recipient.  Return CS if this is the last block, or CC if there's
    more to come.  The first time this handler is called, it's given the
    value 0 in R2; after that, the return value of R2 is passed back in
    the next call, to provide some sort of context.  It's called over
    and over until either the recipient complains or the handler returns
    end-of-data.

  * Success: the data was transferred successfully (and may or may not
    be permanently safe).

  * Failed: the data transfer failed, either because of a local error,
    or because the receiver failed.

Note that the `send' handler can return a block of any size.  It's
Sapphire's responsibility to handle the recipient's buffer size.  In the
case where all the data is in one big flex block, the send routine
becomes trivial.  Sapphire makes RAM transfer /easier/ than writing to a
file!

Loading data in similar in spirit.  There's a single routine, `load',
which is given a block of handlers:

  * InitBuf: create a buffer in which to receive data, and perform any
    other setting up necessary to start a RAM transfer.  The handler is
    given an estimated file size, and the proposed leafname.

  * KillBuf: RAM transfer has failed, so destroy the buffer (if it was
    only temporary).  This is called when RAM transfer was unsuccessful
    for some reason.

  * Extend: the current buffer was filled: I need a new one.  (It's
    called `extend' because the normal reaction is to extend a flex
    block and return the new area.)

  * DoneBuf: RAM transfer has succeeded, so do whatever's necessary.
    (Typically, an application might free some `slop space' at the end
    of the buffer.)

  * File: load the data from a given file.

  * Done: data transfer succeeded.

  * Failed: data transfer failed, either because of something we did
    wrong or because the sender had problems.

Sapphire provides routines to do most of the work of the InitBuf,
KillBuf, Extend, DoneBuf and File handlers in the case where we're just
loading directly into a flex block, and in many cases, the
supplied KillBuf, Extend and DoneBuf routines can be used directly.

The higher-level interfaces `xsave' and `xload' are designed to link
onto the low-level interfaces, and provide handlers which can (in many
cases) be given directly to the low-level calls.  They take as an
argument a tranfer routine, which is called to do the actual save or
load operation.  It is called just like a normal subroutine.  When it
returns, the transfer is ended.  It calls `xsave' or `xload' routines
to read and write bytes, words, strings or whole blocks of memory;
Sapphire buffers data where this would be useful.  Sapphire provides a
separate stack for the transfer routine, so that the read or write
routines can pause for a Wimp_Poll to send or receive more data by RAM
transfer, although this is invisible to the transfer routine, which just
sees a simple interface to a stream of data.  Errors from the transfer
are carefully propagated back to the transfer routine, which sees error
returns from read or write operations.

Sapphire includes a standard `saveAs' box, whose interface is slightly
simpler than the `save' routine.

_____________________________________________________________________________

THE SAPPHIRE CHUNK FORMAT, AND PREFERENCES HANDLING


A Sapphire `chunk' file is very simply a file representation of a list
of named chunks of data.  Sapphire provides functions to read chunk
files, claim chunks, and write chunk files back again.

When a chunk file is loaded, each chunk is placed in a Flex block.  A
client can `claim' a chunk.  Once this is done, the client can either
modify the data in-place, or free the block and provide a pointer to a
procedure to write equivalent data to the chunk file when it's saved.

The original idea was to support preferences files in a modular way --
each component could find its own preferences chunk without needing to
know about anything other than the chunk system.  We noticed that this
could easily be extended to handle binary data, and we could base a
general file format on the idea.  A typical Straylight document would
look something like this:

	;
	; Generated by Straylight Frobnitz
	;

	[Header]
	Format = Frobnitz data
	FormatVersion = 3.15
	Author = J. R. Hacker

	[FrobData]
	Bin[00]....[binary rubbish]

	[FrobOptions]
	AutoDelay = 300
	AutoMods = 50
	...

You get the idea.  The text chunks can be modified by anyone with a text
editor -- the binary areas are self-contained and don't care where they
are in a file.

Some user preferences are best stored as raw chunks.  For example, the
`FontDir' program keeps a chunk containing the list of font directories
known to it.  Most are better expressed through the sort of key/value
syntax shown in the above example.  Sapphire provides a separate
mechanism for handling these which interwork with the chunk file system.

The `options' unit will, when requested, claim a chunk from a chunk
file, and translate its textual contents into an easily digestable
binary format.  When the chunk file is saved, Sapphire will
automatically read the binary data, and translate whatever's there back
into text.  Using text as an external representation insulates the
client program from version changes -- default values can be provided
for keys which aren't defined, for example.

Each key/value pair in the chunk has the general format:

  <key> [=] <string> [;<comment>]

A <string> is either a sequence of unquoted characters, or quoted by one
of the pairs `', '', or "".  A quote character can be duplicated to
insert a literal quote in the string.  Any of the characters `#', `|' or
`;' begins a comment (although `;' is preferred because we're assembler
programmers).

The set of permissable options is provided to Sapphire as a table, each
entry of which states the name of the key, the offset within the options
block to store the (binary) value, the type of the value, and any data
the type handler needs for its own purposes.  It's another
variable-size-entry table, like a menu definition or a dbx control list.
More types can easily be written by client programs.  Standard handlers
are provided:

  * The string type reads quoted or unquoted strings.  It always writes
    quoted strings.

  * The integer type reads signed or unsigned integers in any base (up
    to 36).  It usually writes decimal numbers although it can be
    persuaded to use a different base.

  * The literal type never reads anything.  It writes out a string
    (maybe some kind of comment) when requested.

  * The enum type reads a string, looks it up in a table, and stores the
    index.  The match is case-insensitive.  Abbreviations are
    accepted; ambiguous abbreviations match the first string found in
    the table.  It writes out the full correct-case string.

  * The bool type is a special case of the enum type -- it reads either
    one of `true', `on' or 'yes' for a true value, or `false', `off' or
    `no' for false, and sets a single bit for its answer (for space
    efficiency).  On output, it writes one of `true' or `false', unless
    requested to use `on' or `off', in which case it also suppresses the
    optional `=' sign on output.

  * The version type reads version numbers of the form `xxx.yz' and
    stores them as integers, multiplied by 100.  It writes them out
    in the same form, with trailing zeros if necessary.

The preferences system DoggySoft added into WimpExtension is the nearest
thing which comes close, but Sapphire does the job in a much more
powerful way.  This is actually quite surprising, since the unit isn't
particularly large.  (This is partly due to an interesting coding trick
I came up with to reduce typing, which I've entirely failed to make use
of in any high-level language.  See `choices/options.s' for details.)

_____________________________________________________________________________

HIGH LEVEL LANGUAGE INTERFACE


This is stil very sketchy, but a discussion may interest some readers.
The C interface has to deal with several problems:

  * Sapphire's procedure calls aren't even a little bit compatible with
    APCS.  Sapphire gains a lot of its power and elegance from its lax
    calling conventions.

  * C programmers tend to use different names for commonly used
    functions.

  * Sapphire's variable-size-entry tables aren't easy to represent in C.

The last problem hasn't been addressed.  The trivial `Hello, world'
example program contains a horrible block of hex where the menu
definition is meant to be.  To be fair, we envisaged the user-interface
parts of Sapphire applications to be in assembler, to keep the
user-facing parts of a program light and responsive, while the back-end
bits were in C.

The second problem just involves writing a lot of veneers and playing
with macros.  I didn't bother trying to make the functions ANSI
compliant.  Only the bare bones of a standard library is provided.

The first is the only technically interesting problem.  Making C call
Sapphire is a fairly simple task of inserting a veneer in the right
place.  I wrote a `_call' function which works rather like `_swi', to
allow C to call any Sapphire function.  There's a small problem to do
with R11 which needs dealing with, though.  My solution was simple --
since Sapphire doesn't have any stack checking anyway, compile code with
stack limit checking turned off and move R11 into R10 (APCS's stack
limit pointer) and back to R11 on inter-language borders.

The really interesting problem, then, is making Sapphire understand how
to call C functions.  Again, some hacking is performed.  Any C function
which is intended to be called by Sapphire is declared as follows:

__sapph(funcname)(regset *r [, type *r10 [, type *r12 [, char *scratch]]])

(APCS's way of passing arguments in registers allows the flexibility of
arguments -- if a function doesn't need R10 or R12 it doesn't need to
declare them.  More arguments can be read by putting them after
`scratch' -- they just get pulled off the stack.)

The macro constructs a function with the given name, just before the
actual compiled function, whose name is changed to `__sapph__funcname'.

The actual code generated by the compiler for this atrocity is:

funcname	STMFD	R13!,{R14}
		MOV	R14,PC
		B	__sapph_veneer

__sapph__funcname
		<APCS entry sequence>
		...

The `__sapph_veneer' routine knows that the function it's meant to call
is pointed to by R14.  It stores all the registers on the stack, points
R0 at them (to make the regset variable), sets up the other arguments,
puts R11 into R10, and calls the C routine.  On exit, it pulls the
registers off the stack, puts R11 back in its proper place, sets
flags appropriately, and returns.

The C function can set return values by modifying the `regset' argument
it's passed.  It can also modify the flags by returning an appropriate
value as its result.  For example

	return (C_set | V_clear);

returns with C set, V clear and the other flags unaltered.

This is a very hamfisted way of dealing with the problem.  But Sapphire
was never designed to work with C.  In fact, it's almost true to say
that Sapphire was designed never to work with C -- the fact that it's
possible after all is more a credit to humen ingenuity than Sapphire's
design.

_____________________________________________________________________________

CONCLUSION


Your tour through the interesting bits of Sapphire is complete.  There
are plenty of other `neat toys' in there, waiting for you to find them.
Most source files contain some sort of interesting twist on an old idea,
or a neat programming trick.

The sources are commented to excess.  Almost every line contains a
comment.  Obscure tricks and other bits of non-obvious code are usually
tagged `Horrid hack' or `Neat trick', at least the first time we used
that particular construction.  Searching for `hack' should produce lots
of interesting code to read.

Anything else you want to know: just read the header file documentation
and/or the source code.  Or you can ask me.  Just bear in mind that it's
been a long time since I wrote anything which used Sapphire and I'll
probably have to read the headers and/or the source code anyway.

Sapphire, for Tim and myself, has been a labour of love.  I'm extremely
proud of it: of its small size, its speed, its elegant design, its power
and versatility.  Please, treat it with respect.

I'm almost in tears as I type this, so I'd better stop.  Thanks for
reading,



  Mark Wooding, mdw@excessus.demon.co.uk
  17 December 1997