A very brief overview of Sapphire ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _____________________________________________________________________________ IS THIS ALL THERE IS? The Grand Plan was to write a really good library, sell some spiffy applications which used it, and then sell the library. This spectacularly failed to work, mainly because we had no ideas for killer applications. The result is Sapphire. Since we'd planned to write some big applications before releasing Sapphire to anyone else, we took a rather relaxed attitude to documentation. There is some LaTeX documentation lying around, but mostly it's initial versions of things, with big gaps where less interesting text should be filled in later. In short, it's no use to anyone. On the other hand, we realised that a library of this complexity with no documentation at all would be a total disaster, so all the header files are profusely commented, mostly to the standard of a final-copy reference manual. This text file is intended as a brief hacker's eye view of how to write Sapphire applications, and how some of the more complicated and/or interesting bits work. _____________________________________________________________________________ ABOUT THE AUTHORS Sapphire was designed and written, over the course of a couple of years, by Mark Wooding and Tim `Tycho' Armes. Each source file usually bears the initials (`MDW' or `TMA') of the person mainly responsible for writing it. In some cases, this is misleading. For example, although the `msgs' code for translating message tokens into strings is labelled `MDW', the first two versions of this file were written by Tim: I took over later because I rewrote it to kill the bugs. Similarly, the fact that one of us wrote a source file doesn't tell you who designed it. For example, the `menu' code was entirely written by Tim, although we both worked on the design for several days beforehand. _____________________________________________________________________________ A HISTORY LESSON Sapphire is an attempt to learn from the experience we both had with STEEL. STEEL (Straylight's Extensive Event-driven Library) is a more-or-less complete re-write of RISC_OSLib, and although it had given up being call-compatible, and had diverged quite considerably by the release of Desktop C, it suffered because of the limitations of its structure. All the original non-RISC_OSLib code had been written by me, in a rather ragged way. The name Sapphire comes from the old ITV sci-fi series `Sapphire and Steel', obviously. It's a rotten joke, but that's what you get when you think up names for libraries in a pub. The idea was that two of us could do much better than either acting alone, by bouncing ideas between each other until we sorted out the problems. We threw out any thought of compatibility with any existing software, and any hope of interfacing a high-level language. Then we started coding. _____________________________________________________________________________ INFLUENCES The design of Sapphire has been influenced by looking at lots of other pieces of software. A lot of structure is based on RISC_OSLib: STEEL was derived from RISC_OSLib, and it did a lot of things right. Another particularly strong influence is Computer Concepts' Advanced Box Interface (ABI) as used in Impression and ArtWorks. (I even went as far as buying the ArtWorks SDK more-or-less to get my hands on some ABI documentation.) There was an influence from what I perceived to be `the way OS/2's Presentation Manager does it', although this wasn't always positive. _____________________________________________________________________________ SUPPORTED LANGUAGES Sapphire supports ARM assembler. That's about it. One of the last Sapphire-related projects was interfacing C to Sapphire; this is still mostly incomplete. The basic job of making C code call Sapphire code and vice-versa is finished and reliable. The hard bit, handling Sapphire-style definition tables, isn't even begun yet. One of the projects I set myself over a year ago was a language, then called `cdata', which would act as a pre-processor for C and allow definition tables to be created without the amount of pain currently required. (If you don't believe me, see the definition of the icon bar menu in the `csapph' test program.) _____________________________________________________________________________ THE SAPPHIRE ENVIRONMENT Sapphire doesn't use APCS. Let's get that clear right away. You try to use APCS, and Sapphire will hate you for the rest of your life. Because the target language is assembler, Sapphire itself sometimes plays fast and loose with the rules, but the basic ideas, such as they are, are presented here. Registers R10-R15 are assigned special purposes. For architectural reasons, R15 is the program counter, and R14 is the subroutine link register. Because R14 is modified by subroutine calls, it is often used as a temorary register in computations. R13 points to a full descending stack. Sapphire doesn't allocate a very big stack unless you ask it to, so be careful with recursion. There's no stack limit checking either. R10 and R12 provide the program's current state. R12 is the `workspace' pointer, and R10 is the `object' pointer. The idea is that R12 points to a statically allocated data block, and R10 points to a `currently interesting object'. For example, in a multi-document editor, R10 would point to an information block describing the document currently being processed. Because it's important to hide information between components of a program, different components have different workspace areas. Similarly, different parts of a program see the world at different levels: the part of a drawing program which is concerned with file-level operations keeps R10 as a pointer to the document anchor; the part concerned with manipulating individual objects within a document might keep R10 referring to an object. Note that R10 and R12 rarely take part in inter-component interfaces. Object handles are passed between components in low-numbered registers, as normal arguments, and then copied into R10 later. R11 is Sapphire's `application context' pointer. It's also known as the `scratchpad' pointer. From R11 upwards is the `scratchpad': a block of memory available for any purpose. Any procedure may corrupt the scratchpad, so it's never safe to assume that it is preserved over inter-component calls. Immediately below R11 are workspace offsets. These allow Sapphire and its client applications to find statically allocated workspace. If R11 is pointing to the wrong place, Sapphire will crash! On a procedure call, this is what usually happens: R0--R9 == arguments R10, R12 == variables owned by caller R13 == pointer to full descending stack R11 == application context pointer (Scratchpad) On return: R0--R9 == return values, or usually preserved R10, R12 preserved R13 == preserved R11 == application context pointer Flags either preserved or set as a return value Sapphire's fond of returning information in the flags. For example, if you ask for memory and there isn't any, allocation functions return with carry set. Sometimes, errors are returned in the normal way, by setting overflow and pointing R0 at an error block. If you don't return something useful in a flag, preserve it. Sapphire behaves slightly different when it's calling its client back. When you register a handler function, you supply a pointer to the code, and R10 and R12 values to pass it. It will ensure that these values are in those registers at call time. _____________________________________________________________________________ SAPPHIRE'S MEMORY MAP Sapphire has strong ideas about how an application's memory is laid out. Here's a poor attempt at a diagram: _____________ <--- Top of current wimpslot | | : Flex : : heap : | | |-------------| | Resizing | | heap | |-------------| <--- Top of initial wimpslot R13 | Stack | |-------------| R11 | Scratchpad | | App context | |-------------| | | |Default heap | | | |-------------| R12 | Workspace | |-------------| | Application | | code | &8000 |_____________| <--- Base of application memory Hmmm. Not too shabby. Now, in words: During initialisation, Sapphire's kernel finds the top of available memory and puts the stack there, growing downwards. Below the stack, it allocates the scratchpad area, so that there's a small amount of `slop' between the the stack growing down and the scratchpad growing up. The kernel then examines the available library components, and allocates workspace for them, starting just above the application's read-write area. (A Sapphire application shouldn't actually have any read-write data -- R12-space is a much better way of doing the same job.) Once that's done, it initialises an OS_Heap heap between the top of workspace and the bottom of the application context area just below the scratchpad. This is the `default heap'. Above the initial wimpslot is Flex space. Flex is Sapphire's shifting heap manager. It's named after RISC_OSLib's shifting heap, although it's completely rewritten, and much extended. Flex has complete control over the area above the initial wimpslot. The first Flex block is taken by a resizing heap manager, which creates a nonshifting heap there. This is registered automatically with Sapphire's allocation manager, so you can forget about it completely. This brings me on to... _____________________________________________________________________________ MEMORY MANAGEMENT Sapphire sees three sorts of memory: * Static memory. This is allocated at initialisation time by the kernel, and put into the workspace area. R12 points here. * Fixed-size blocks representing objects. These are allocated and freed at run-time by the program, using Sapphire's allocation manager. R10 points to one of these, usually. * Large or variable-size blocks representing data. These are allocated and freed by the program using Flex. Typically each large block in Flex-space has an accompanying `anchor' block in heap-space. The allocation manager attempts to paper over the mess involved with having more than one nonshifting heap. STEEL actually does this job better, putting the client in control of where blocks get allocated. Sapphire's approach, to fill up one heap and move on to the next, is simpler, but can lead to fragmentation. _____________________________________________________________________________ INITIALISATION RISC_OSLib (and STEEL) programs start off with a huge chunk of initialisation code, most of which is really boring: wimpt_init("progname"); dbox_init(); win_init(); /* ... */ Sapphire gets rid of all that by using a neat feature of AOF. Each library component, or `unit' as we called them, contains a four-word table entry. The linker collects the entries together, and the kernel can examine them all to see which units are linked in. Each entry contains: * The address to store the unit's workspace offset. * The size of workspace required. * Minimum acceptable Scratchpad size (if more than 256 bytes). * Address of initialisation procedure. By convention, the first word of a unit's workspace contains flags, bit zero of which indicates that the unit has been initialised. A unit's initialisation procedure performs the following actions: * It checks it's not already initialised. * It initialises anything it depends on (this causes the required units to be linked too). * It initialises itself. * It sets its initialised flag and returns. Library initialisation is a two-step process. The first step, `sapphire_init', initialises the kernel and the basic memory map. The second step, `sapphire_libInit', initialises the library units. _____________________________________________________________________________ DYNAMIC LINKING The original idea was that Sapphire would be really small. Well, take a look at its feature set and tell me that it's big. So we paid no attention to making Sapphire dynamically linkable. When it became obvious that this wasn't working properly, I started investigating methods for making Sapphire into a DLL. I was saved by two features of Sapphire code: * Sapphire programs preserve R11 everywhere. * Sapphire's kernel allocates workspace for the units at run-time. The first fact gives me re-entrancy automatically. R11 becomes the one thing that the dynamically-linked Sapphire library can use to decide where its workspace is. Finding workspace is rather grubby in Sapphire, so it's hidden beneath a macro. For the curious, this is what happens: LDR R12,workoff_addr LDR R14,[R11,#-magic_offset] ADD R12,R14,R12 The trick works because the allocation of workspace /offsets/ depends only on the order of units within the library, so they can be shared; the value loaded from R11-space is worked out by the kernel and put in application memory on startup. Most of the kernel ends up in the DLL. The application is linked against a really small stub, which basically contains front ends for a few kernel functions to (maybe) find the Sapphire DLLs, point to a collection of tables, and call the corresponding function in the core. The exact mechanism is complicated. Interested readers are referred to the source code. The upshot of all of this is that, although SDLS was designed specifically to solve the problem of APCS shared libraries, Sapphire's style of shared library requires much less code to be linked into the client, much thinner veneering (because the kernel handles re-entrancy), and the result is both more robust and cleaner. _____________________________________________________________________________ ERROR HANDLING Sapphire programs know about two sorts of errors: * Program errors, which should never occur. * Environmental errors, which must be anticipated. Environmental errors are reported by setting the V flag on exit from a function. Sapphire routines are documented according to whether they can return errors: it is safe to ignore the V flag on exit from a routine whose documentation does not state that it returns errors. An environmental error typically requires some sort of user intervention, and suggests a friendly explanation that some sort of fault has occurred. Program errors are raised by OS_GenerateError and caught by Sapphire's error handler. There's a rule of thumb which states: `Never test for an error you can't handle.' Sapphire follows this advice. Most calls to the operating system have the X bit clear, causing errors to be sent to the error handler. Program error messages are intentionally technical. The idea is to encourage naive users to write them down, so that they remember them and report them accurately. Part of the initial design was that a Sapphire application should /never/ exit as a result of program fault. A user should always be able to try to get to a save box and save data. (Too much experience of Paint falling over with type 5s prompted this decision.) The application pops up an error report giving the user a choice between killing the program or trying to soldier on. Soldiering on seemed remarkably successful in general. A later addition to Sapphire was `Structured Exception Handling', which is a lower-level version of C++'s try/throw/catch exception handling. This builds upon the existing base, and allows programs to tidy up in the event of errors before propagating them back down to the bottom level `Continue/Quit' handler. This concludes the tour of Sapphire's run-time support functionality. The rest describes plain ol' library features. _____________________________________________________________________________ EVENT HANDLING Sapphire has a fairly simple but highly effective strategy for dealing with events, inspired in part by the FilterManager module in RISC OS 3. The `event' unit is responsible for low-level event handling. Applications call `event_poll' instead of calling the SWI Wimp_PollIdle directly; the calling sequences are almost the same for both routines. `event_poll' returns C set if it handled the event itself, and C clear if it failed to handle it. A defalt handler procedure is provided to perform any required actions for a particular event. A Sapphire application's poll loop typically looks like this: loop BL event_poll BLCC handle_unknowns BLCC defHandler B loop and that's it. Most of the hard work is done by filters. Sapphire knows about three types of filters: * Pre-filters are shown arguments to Wimp_Poll before it's called, and can modify them in controlled ways, e.g., clearing event mask bits, or changing the `time-to-return' argument. A pre-filter can `claim' a call to Wimp_Poll by storing a `fake' event in the buffer and returning carry set. Other pre-filters are skipped, and the fake event is passed to the post-filters. * Post-filters are shown the result from a call to Wimp_Poll. They can perform any action they like, except for modifying the event. A post-filter can claim an event by returning C set; other post-filters are not called, and `event_poll' returns C set. * Fake-event filters are a novel idea introduced to handle transient dialogue boxes. Immediately after a return from Wimp_Poll, Sapphire scans the list of fake-event handlers. Each one is entitled to `steal' this event and replace it with another by returning carry set. Once an event has been stolen, it is passed through the post-filters. When `event_poll' is called again, Sapphire restores the previous genuine event and continues calling fake-event filters from the point at which it left off. Thus, Sapphire provides a general mechanism for inserting events. Everything else is based off this mechanism. The `win' unit dispatches window-specific events to window handlers by registering a post-filter. The `menu' unit also establishes a post-filter, and passes events to the owner of the current menu. The main use for fake-event handlers is to handle transient windows. A separate Sapphire unit, `transWin', remembers the handle of the current transient window. When an event arrives, its fake-event handler checks whether the transient window is open. If it has been closed, it steals the event, whatever it was, and substitutes a fake `window close' event to be picked up by the owner of the transient window. [Actually, `transWin' is careful to allow redraw events through unmolested. Bad window flicker results if redraw requests are tampered with.] _____________________________________________________________________________ MENUS IN SAPPHIRE Sapphire knows about two types of menus, and its interface to applications reflects this. Standard RISC OS transient menus will be familiar to most readers. Straylight tearoff menus will be less well known -- see Glass for a real-life example of tearoff menus in action. Sapphire's interface to handling menus is inspired by Computer Concepts' Advanced Box Interface system. Menus are always constructed dynamically, as needed. The procedure `menu_create' is given four arguments: * The address of a menu definition. * The address of a menu event handler. * R10 and R12 values to pass to the handler. The `menu_create' procedure may be called any number of times before the next Wimp_Poll: the menu definitions are concatenated. This allows applications to create dynamic and context-sensitive menus easily. The first `menu_create' call is different from the others: the menu definition must contain a menu title. (It need not contain any menu items, though.) Both the menu title and the items are /variable size/ objects -- the more odd features a menu item has, the more space its definition occupies. Menu definitions are also /read-only/. Dynamic features like ticking and shading must be handled separately. Each menu item (and title) is begun with a `feature mask' -- a word containing a flag bit for each feature. The feature mask is followed by data appropriate to the various features selected. For example, the `has a submenu' feature requires a pointer to the submenu definition and a pointer to the procedure which will handle events for the submenu. Menu features which require dynamic information (e.g., context-sensitive message strings, or ticking and shading) have, as part of their feature data, offsets from the menu handler's R10 containing the required information. The `shade item' feature requires an offset from R10 and a bit position -- it will shade the item when the appropriate bit in the word at [R10, #offset] is set. Sapphire's menu system understands the concept of `radio items' -- groups of items of which only one may be ticked at a time. The feature data for a radio item consists of an R10 offset and a value to match -- the application then sets [R10, #offset] to be (for example) 0 for the first item in the group, 1 for the second, and so on. It's important to get the hang of the concepts here, because lots of other bits of Sapphire use this same idea. The `menuDefs.sh' header file contains a large number of macros which hide most of the mess of this. _____________________________________________________________________________ DIALOGUE BOXES Sapphire's dialogue box handling is extremely powerful. There's also a lot of code in `dbox.s' which does jobs that the WindowManager ought to do, or does wrong. Dialogue boxes are `resources'. Sapphire tries to keep the interface to handling resources consistent. When you want one, you `create' it, and when you don't need it any more, you `destroy' it. A successful `create' of a dialogue box yields a `dbox' -- a handle which represents the dialogue box. Sapphire is carefully constructed so that dialogue boxes don't need to exist unless they're actually in use. Most programs create dialogue boxes only when they need to appear on-screen, and destroy them again once they're closed. To this end, Sapphire generates fake close events for transient windows. Dialogue boxes can be created from several different sources of information, and to handle this, there are actually three different `create' routines: * `dbox_create' is the standard call. A client passes a string naming the template from which to create the dialogue, and the dialogue box manager returns a handle. The window definition is copied, along with all the indirected data, so that you can create multiple instances of a template without tripping over aliasing problems. * `dbox_fromEmbedded' creates a dialogue from an `embedded template' -- a compact and easily expanded representation of a window template which is statically linked into the client program. Again, the data is copied from the template, so aliasing problems don't exist. * `dbox_fromDefn' is the most powerful call (and the others are implemented in terms of it). The client passes a pointer to a complete window definition, which the Sapphire uses `as is', without copying it. Embedded templates were introduced for very small programs (e.g., DLLMerge). Sapphire makes use of them because common dialogue boxes have been stored in a shared `Sapphire.Resources' DLL as embedded templates. This makes programs easier to upgrade for new versions of Sapphire (just copy the new DLLs in and you get the new dialogues free) and reduces resource requirements since there's only one copy in memory at any given time. Displaying dialogue boxes introduces more flexibility. The procedure `dbox_open' is given a dbox and an `open style', and a collection of arguments appropriate to the style. Styles permitted are: * In its current position (or in the position it was in the template file). * In the middle of the screen. * Centred over the mouse pointer. * At a given Y coordinate (but the current X coordinate). * At a given X and Y position. The open style also contains a `transient or persistent' flag, which is useful for dialogue boxes hanging off of menu items. If a menu is open currently, Sapphire will automatically open the dbox as a submenu, in the correct place, unless you explicitly instruct it not to do this using another open style bit. Sapphire doesn't pass Wimp events directly on to client event handlers. Instead, it pre-processes them, putting the important information from the event into registers, and sending a `dbox event' to the handler. The handler can then either `claim' the event for itself (by saying that it's handled it) or pass it on. Unclaimed events are acted upon by Sapphire. As an example of the sort of preprocessing Sapphire does for dialogue box events, it breaks a redraw request into an individual event for each rectangle. If its redraw events are unclaimed (as they ought to be), it will call Sculptrix to fill in the 3D borders around the icons. Sapphire handles radio buttons itself, if click events on them are unclaimed. Using type-11 buttons and a non-zero ESG means that the user can deselect all buttons in a set by clicking the selected one with `adjust'. An application can deal with this by selecting the icon explicitly, but (a) this causes flicker, and (b) if the application is going to the trouble of having explicit code to deal with the situation, it may as well do the job properly. Sapphire considers an icon with button type 3 and non-zero ESG to be a `Sapphire radio button' and does the right thing with it. Shaded icons are also handled by Sapphire rather than the Wimp, because the standard behaviour is highly inconsistent. For example, sometimes icon backgrounds are shaded in addition to foregrounds. In text-and- sprite icons, the text is /not/ shaded when it ought to be (for consistency with text-only icons). It's basically a mess. Sapphire uses a complicated shading algorithm which seems fairly close to the behaviour of ABI. Sapphire also handles caret blinking and cursor changing over writable icons automatically. Keypresses also receive default handling from Sapphire. Unclaimed cursor keys cause the caret to move between writable icons, scrolling the window where necessary (both horizontally and vertically) to ensure that the new focus icon is visible. The procedure to set a string in an indirected icon is probably excessive. It ensures that the string really needs changing before flickering the icon. It truncates the string rather than overflowing the icon's buffer, discarding either the start or the end depending on the icon's `right align' flag, optionally putting a `...' to indicate that a truncation was performed. Sapphire inherited a curious feature called `embedded titles' from STEEL, which in turn was inspired by RISC OS version of the game `Elite'. A client nominates an icon within a window which has no title bar as being an `embedded title'. Sapphire will thereafter automatically draw a Sculptrix group box around this icon, using the window's real title text as the group title. This is used in `Info' windows, as well as the warning, note and error windows. Sapphire can handle `nonpolling' windows. In some cases it's desirable to arrest the user's attention by requiring acknowledgement of a message or the taking of a decision, and sometimes continuing to poll would be dangerous or undesirable for other reasons. For example, a serious error report mustn't multitask, because the error might recur. The `To save...' message caused by attempting to save a file without a full pathname shouldn't multitask, because the click on the OK button will close the menu tree (which is extremely irritating). The only way to handle a prequit message under RISC OS 2 is to ask the user whether quitting is permitted in a nonpolling way, so the decision to cancel a shutdown sequence can be made before an acknowledgement is returned to the task manager. The handling for nonpolling windows is split into two parts. The `nopoll' unit displays dialogue boxes and reports mouse clicks and keypresses as fake events through Sapphire's pre-handler mechanism (so nonpolling dialogues present an identical programmer interface to normal ones). The `buttons' unit displays a given dialogue, permitting a collection of action buttons to be given appropriate pieces of text. The real richness of Sapphire's dialogue box handling comes from custom controls. A whole subsystem, `dbx' is provided for dealing with custom controls. The client registers a table defining the required controls for a particular dialogue box. The table contains variable size entries, each one beginning with a standard header: * Number of the icon which hosts the control. * Pointer to the control definition. * Various flags (control specific). * The offset of the control's writable data within the dialogue handler's R10-space. This is followed by whatever constant data the control handler requires. Sapphire dbx controls are always hosted by icons. This makes them easy to lay out in template editors. Custom controls generate their own events which are sent to the client's handler procedure. They are also given dialogue box events (after the client has had a chance to claim them for itself) as `dbx events'. For example, Sapphire will only ask a control to draw itself if (a) the control requests redraw events and (b) the current rectangle intersects the control's host icon. Sapphire takes redrawing of controls very seriously. It ensures that the graphics clipping area is set to the intersection of the control's host icon and the redraw rectangle, to prevent any `accidents' from messing up the rest of the display. It passes this amended rectangle to the control to allow it to perform source-level clipping. The custom RGB-coloursquare and HSV-colourcircle controls in the (unfinished) colour selector use the provided rectangle to plot the new graphics by direct screen access. Predefined custom controls are: * A slider. This is written very carefully to avoid any flicker while the bar is being dragged or updated. * A `bump' arrow button, which presses its icon in while it's being held down, and autorepeats. The arrow control keeps a count of how many `ticks' have been missed, and sends them to the dialogue box as large packets, rather than individually. This makes programs which spend a long time processing bump requests feel responsive even when they're really being slow. * A file icon, which represents a filetype in a window. File icons can optionally be draggable (for save operations). When solid drags are requested by the user (and the DragASprite module is available), the control will make the icon disappear while the icon is being dragged, to produce the illusion of the icon being `lifted off' the dialogue box. * A `string set' control, which drops down a menu of strings and allows the user to choose one, which it displays. * A simple `colour pot' which allows a user to choose one of the sixteen standard Wimp colours. This control pops open a small dialogue box of its own when requsted. Custom controls provide a great deal of functionality at very little programmer effort. Why no other library for RISC OS handles them properly is a mystery to me. The Toolbox has a go, but fails miserably; compared to the power of Sapphire's `dbx' subsystem, it might as well not bother. (Sapphire was mature when the Toolbox betas were available. The idea for proper custom controls came from OS/2, but the implementation is very much in Sapphire's own style.) _____________________________________________________________________________ DATA TRANSFER It used to be the case that data transfer was one of the really hard bits of an application. I remember trying to put it off as long as possible whenever I wrote a STEEL application. Sapphire tries very hard to make data transfer as painless as possible. Both loading and saving have two layers to them. Underneath, there's a low-level interface which exposes the nature of the transfer (whether it's to a file, or in-memory). Above that, there's an abstract interface which presents both transfer types as a (non-seekable) stream of data, providing buffering and a simple programming interface. This high-level interface works perfectly well on files even without a data transfer context, and occasionally gets (ab)used as a general stream interface to file data, for example, when saving preferences files. The lower-level code was designed to be useful both in conjunction with the high-level interfaces and on its own. Often, the low-level interface is the better choice. Oddly, the low-level interface is higher-level than RISC_OSLib's. A low-level data save operation is initiated by the `save' call. It requires a large number of arguments, detailing the recipient's window handle, the filename, filetype, and a block of handling routines. The handling routines are: * Save: write your data to a file, given its name. Return an error if saving failed. * Send: return (giving a base pointer and size) a block to send to the recipient. Return CS if this is the last block, or CC if there's more to come. The first time this handler is called, it's given the value 0 in R2; after that, the return value of R2 is passed back in the next call, to provide some sort of context. It's called over and over until either the recipient complains or the handler returns end-of-data. * Success: the data was transferred successfully (and may or may not be permanently safe). * Failed: the data transfer failed, either because of a local error, or because the receiver failed. Note that the `send' handler can return a block of any size. It's Sapphire's responsibility to handle the recipient's buffer size. In the case where all the data is in one big flex block, the send routine becomes trivial. Sapphire makes RAM transfer /easier/ than writing to a file! Loading data in similar in spirit. There's a single routine, `load', which is given a block of handlers: * InitBuf: create a buffer in which to receive data, and perform any other setting up necessary to start a RAM transfer. The handler is given an estimated file size, and the proposed leafname. * KillBuf: RAM transfer has failed, so destroy the buffer (if it was only temporary). This is called when RAM transfer was unsuccessful for some reason. * Extend: the current buffer was filled: I need a new one. (It's called `extend' because the normal reaction is to extend a flex block and return the new area.) * DoneBuf: RAM transfer has succeeded, so do whatever's necessary. (Typically, an application might free some `slop space' at the end of the buffer.) * File: load the data from a given file. * Done: data transfer succeeded. * Failed: data transfer failed, either because of something we did wrong or because the sender had problems. Sapphire provides routines to do most of the work of the InitBuf, KillBuf, Extend, DoneBuf and File handlers in the case where we're just loading directly into a flex block, and in many cases, the supplied KillBuf, Extend and DoneBuf routines can be used directly. The higher-level interfaces `xsave' and `xload' are designed to link onto the low-level interfaces, and provide handlers which can (in many cases) be given directly to the low-level calls. They take as an argument a tranfer routine, which is called to do the actual save or load operation. It is called just like a normal subroutine. When it returns, the transfer is ended. It calls `xsave' or `xload' routines to read and write bytes, words, strings or whole blocks of memory; Sapphire buffers data where this would be useful. Sapphire provides a separate stack for the transfer routine, so that the read or write routines can pause for a Wimp_Poll to send or receive more data by RAM transfer, although this is invisible to the transfer routine, which just sees a simple interface to a stream of data. Errors from the transfer are carefully propagated back to the transfer routine, which sees error returns from read or write operations. Sapphire includes a standard `saveAs' box, whose interface is slightly simpler than the `save' routine. _____________________________________________________________________________ THE SAPPHIRE CHUNK FORMAT, AND PREFERENCES HANDLING A Sapphire `chunk' file is very simply a file representation of a list of named chunks of data. Sapphire provides functions to read chunk files, claim chunks, and write chunk files back again. When a chunk file is loaded, each chunk is placed in a Flex block. A client can `claim' a chunk. Once this is done, the client can either modify the data in-place, or free the block and provide a pointer to a procedure to write equivalent data to the chunk file when it's saved. The original idea was to support preferences files in a modular way -- each component could find its own preferences chunk without needing to know about anything other than the chunk system. We noticed that this could easily be extended to handle binary data, and we could base a general file format on the idea. A typical Straylight document would look something like this: ; ; Generated by Straylight Frobnitz ; [Header] Format = Frobnitz data FormatVersion = 3.15 Author = J. R. Hacker [FrobData] Bin[00]....[binary rubbish] [FrobOptions] AutoDelay = 300 AutoMods = 50 ... You get the idea. The text chunks can be modified by anyone with a text editor -- the binary areas are self-contained and don't care where they are in a file. Some user preferences are best stored as raw chunks. For example, the `FontDir' program keeps a chunk containing the list of font directories known to it. Most are better expressed through the sort of key/value syntax shown in the above example. Sapphire provides a separate mechanism for handling these which interwork with the chunk file system. The `options' unit will, when requested, claim a chunk from a chunk file, and translate its textual contents into an easily digestable binary format. When the chunk file is saved, Sapphire will automatically read the binary data, and translate whatever's there back into text. Using text as an external representation insulates the client program from version changes -- default values can be provided for keys which aren't defined, for example. Each key/value pair in the chunk has the general format: [=] [;] A is either a sequence of unquoted characters, or quoted by one of the pairs `', '', or "". A quote character can be duplicated to insert a literal quote in the string. Any of the characters `#', `|' or `;' begins a comment (although `;' is preferred because we're assembler programmers). The set of permissable options is provided to Sapphire as a table, each entry of which states the name of the key, the offset within the options block to store the (binary) value, the type of the value, and any data the type handler needs for its own purposes. It's another variable-size-entry table, like a menu definition or a dbx control list. More types can easily be written by client programs. Standard handlers are provided: * The string type reads quoted or unquoted strings. It always writes quoted strings. * The integer type reads signed or unsigned integers in any base (up to 36). It usually writes decimal numbers although it can be persuaded to use a different base. * The literal type never reads anything. It writes out a string (maybe some kind of comment) when requested. * The enum type reads a string, looks it up in a table, and stores the index. The match is case-insensitive. Abbreviations are accepted; ambiguous abbreviations match the first string found in the table. It writes out the full correct-case string. * The bool type is a special case of the enum type -- it reads either one of `true', `on' or 'yes' for a true value, or `false', `off' or `no' for false, and sets a single bit for its answer (for space efficiency). On output, it writes one of `true' or `false', unless requested to use `on' or `off', in which case it also suppresses the optional `=' sign on output. * The version type reads version numbers of the form `xxx.yz' and stores them as integers, multiplied by 100. It writes them out in the same form, with trailing zeros if necessary. The preferences system DoggySoft added into WimpExtension is the nearest thing which comes close, but Sapphire does the job in a much more powerful way. This is actually quite surprising, since the unit isn't particularly large. (This is partly due to an interesting coding trick I came up with to reduce typing, which I've entirely failed to make use of in any high-level language. See `choices/options.s' for details.) _____________________________________________________________________________ HIGH LEVEL LANGUAGE INTERFACE This is stil very sketchy, but a discussion may interest some readers. The C interface has to deal with several problems: * Sapphire's procedure calls aren't even a little bit compatible with APCS. Sapphire gains a lot of its power and elegance from its lax calling conventions. * C programmers tend to use different names for commonly used functions. * Sapphire's variable-size-entry tables aren't easy to represent in C. The last problem hasn't been addressed. The trivial `Hello, world' example program contains a horrible block of hex where the menu definition is meant to be. To be fair, we envisaged the user-interface parts of Sapphire applications to be in assembler, to keep the user-facing parts of a program light and responsive, while the back-end bits were in C. The second problem just involves writing a lot of veneers and playing with macros. I didn't bother trying to make the functions ANSI compliant. Only the bare bones of a standard library is provided. The first is the only technically interesting problem. Making C call Sapphire is a fairly simple task of inserting a veneer in the right place. I wrote a `_call' function which works rather like `_swi', to allow C to call any Sapphire function. There's a small problem to do with R11 which needs dealing with, though. My solution was simple -- since Sapphire doesn't have any stack checking anyway, compile code with stack limit checking turned off and move R11 into R10 (APCS's stack limit pointer) and back to R11 on inter-language borders. The really interesting problem, then, is making Sapphire understand how to call C functions. Again, some hacking is performed. Any C function which is intended to be called by Sapphire is declared as follows: __sapph(funcname)(regset *r [, type *r10 [, type *r12 [, char *scratch]]]) (APCS's way of passing arguments in registers allows the flexibility of arguments -- if a function doesn't need R10 or R12 it doesn't need to declare them. More arguments can be read by putting them after `scratch' -- they just get pulled off the stack.) The macro constructs a function with the given name, just before the actual compiled function, whose name is changed to `__sapph__funcname'. The actual code generated by the compiler for this atrocity is: funcname STMFD R13!,{R14} MOV R14,PC B __sapph_veneer __sapph__funcname ... The `__sapph_veneer' routine knows that the function it's meant to call is pointed to by R14. It stores all the registers on the stack, points R0 at them (to make the regset variable), sets up the other arguments, puts R11 into R10, and calls the C routine. On exit, it pulls the registers off the stack, puts R11 back in its proper place, sets flags appropriately, and returns. The C function can set return values by modifying the `regset' argument it's passed. It can also modify the flags by returning an appropriate value as its result. For example return (C_set | V_clear); returns with C set, V clear and the other flags unaltered. This is a very hamfisted way of dealing with the problem. But Sapphire was never designed to work with C. In fact, it's almost true to say that Sapphire was designed never to work with C -- the fact that it's possible after all is more a credit to humen ingenuity than Sapphire's design. _____________________________________________________________________________ CONCLUSION Your tour through the interesting bits of Sapphire is complete. There are plenty of other `neat toys' in there, waiting for you to find them. Most source files contain some sort of interesting twist on an old idea, or a neat programming trick. The sources are commented to excess. Almost every line contains a comment. Obscure tricks and other bits of non-obvious code are usually tagged `Horrid hack' or `Neat trick', at least the first time we used that particular construction. Searching for `hack' should produce lots of interesting code to read. Anything else you want to know: just read the header file documentation and/or the source code. Or you can ask me. Just bear in mind that it's been a long time since I wrote anything which used Sapphire and I'll probably have to read the headers and/or the source code anyway. Sapphire, for Tim and myself, has been a labour of love. I'm extremely proud of it: of its small size, its speed, its elegant design, its power and versatility. Please, treat it with respect. I'm almost in tears as I type this, so I'd better stop. Thanks for reading, Mark Wooding, mdw@excessus.demon.co.uk 17 December 1997