@@@ fltfmt wip

[mLib] / mLib.3.in

.\" -*-nroff-*-
.\"
.\" mLib overview
.\"
.\" (c) 1999--2001, 2003, 2005, 2007, 2009, 2024 Straylight/Edgeware
.\"
.
.\"----- Licensing notice ---------------------------------------------------
.\"
.\" This file is part of the mLib utilities library.
.\"
.\" mLib is free software: you can redistribute it and/or modify it under
.\" the terms of the GNU Library General Public License as published by
.\" the Free Software Foundation; either version 2 of the License, or (at
.\" your option) any later version.
.\"
.\" mLib is distributed in the hope that it will be useful, but WITHOUT
.\" ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
.\" FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Library General Public
.\" License for more details.
.\"
.\" You should have received a copy of the GNU Library General Public
.\" License along with mLib.  If not, write to the Free Software
.\" Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
.\" USA.
.
.\"--------------------------------------------------------------------------
.so ../defs.man \" @@@PRE@@@
.
.\"--------------------------------------------------------------------------
.TH mLib 3mLib "7 July 1999" "Straylight/Edgeware" "mLib utilities library"
.\" @mLib
.
.\"--------------------------------------------------------------------------
.SH NAME
mLib \- library of miscellaneous utilities
.
.\"--------------------------------------------------------------------------
.SH DESCRIPTION
.
The
.B mLib
library is a mixed bag of things which the author finds useful in large
numbers of programs.  As a result, its structure is somewhat arbitrary,
and it's accreted extra bits over time rather than actually being
designed as a whole.  In the author's opinion this isn't too much of a
hardship.
.PP
At the most granular level,
.B mLib
is split into `modules', each of which has its own header file and
manual page.  Sometimes there are identifiable `chunks' of several
modules which fit together as a whole.  Modules and chunks fit into
`layers', each depending on the ones below it.  The header file for
module
.I foo
would be put in
.BR <mLib/ \c
.IR foo \c
.BR .h> .
.PP
This description is a bit abstract, and
.BR mLib ,
as a result of its history, doesn't fit it as well as I might like.
Even so, it's not too bad a model really.
.PP
The rest of this section describes the various chunks and layers.
.
.SS "Exception handling"
Right at the bottom, there's a fairly primitive exception handling
system.  It's provided by the
.BR exc (3)
module, and stands alone.  It's used mainly by the memory allocation
modules to raise exceptions when there's no more memory to be had.
.
.SS "Memory allocation"
The
.BR arena (3)
module provides an abstraction of memory allocation.  By writing
appropriate arena implementations, a client program can control where
and how memory is allocated for various structures.
.PP
The
.BR alloc (3)
module provides simple veneers onto traditional memory allocation
functions like
.BR malloc (3)
and
.BR strdup (3)
(although
.B mLib
doesn't actually depend on
.B strdup
being defined in the library) which raise exceptions when there's not
enough memory left.  These work through the
.B arena
layer, so that the caller can control memory allocation.
.PP
The
.BR sub (3)
module handles efficient allocation of small blocks.  It allocates
memory in relatively big chunks and divides the chunks up into small
blocks before returning them.  It keeps lists of differently-sized
blocks so allocation and freeing is fast.  The downside is that your
code must know how big a block is when it's being freed.
.PP
The
.B track
module (not yet documented) is a simple memory allocation tracker.  It
can be handy when trying to fix memory leaks.
.PP
The
.BR pool (3)
module maintains resource pools which can manage memory and other
resources, all of the resources held in a pool being destroyed along
with the pool itself.
.
.SS "String handling"
The
.BR str (3)
module provides some trivial string-manipulation functions which tend to
be useful quite often.
.PP
The
.BR dstr (3)
module implements a dynamic string data type.  It works quite quickly
and well, and is handy in security-sensitive programs, to prevent
buffer-overflows.  Dynamic strings are used occasionally through the
rest of the library, mainly as output arguments.
.PP
The
.BR buf (3)
module provides simple functions for reading and writing binary data to
or from fixed-sized buffers.
.PP
The
.BR dspool (3)
module implements a `pool' of dynamic strings which saves lots of
allocation and deallocation when a piece of code has high string
turnover.
.
.SS "Program identification and error reporting"
The
.BR quis (3)
module remembers the name of the program and supplies it when asked.
It's used in error messages and similar things.
.PP
The
.BR report (3)
module emits standard Unixy error messages.  It provides functions
.B moan
and
.B die
which the author uses rather a lot.
.PP
The
.BR trace (3)
module provides an interface for emitting tracing information with
configurable verbosity levels.  It needs improving to be able to cope
with outputting to the system log.
.
.SS "Other data types"
The
.BR hash (3)
module provides the basics for an extending hashtable implementation.
Many different hashtable-based data structures can be constructed with
little effort.
.PP
The
.BR sym (3)
module implements a rather good general-purpose extending hash table.
Keys and values can be arbitrary data.  It is implemented using
.BR hash (3).
.PP
The
.BR atom (3)
module implements
.IR atoms ,
which are essentially strings with the property that two atoms have the
same address if and only if they have the same text, so they can be used
for rapid string comparisons.  The
.BR assoc (3)
module implements a hash table which uses atoms as keys, thus saving
time spent hashing and comparing hash keys, and the space used for the
keys.
.PP
The
.BR darray (3)
module implements dynamically resizing arrays which support Perl-like
stack operations efficiently.
.
.SS "Miscellaneous utilities"
The
.BR crc32 (3)
module calculates CRC values for strings.  It used to be used by the
symbol table manager as a hash function.
.PP
The
.BR unihash (3)
module implements a simple but efficient universal hashing family.  This
is a keyed hash function which provides security against an adversary
choosing input to a hash table deliberately to cause collisions.
.PP
The
.BR lock (3)
module does POSIX
.BR fcntl (2)-style
locking with a timeout.
.PP
The
.BR env (3)
module manipulates environment variables stored in a hashtable, and
converts between the hashtable and the standard array representation of
a process environment.
.PP
The
.BR fdflags (3)
module manipulates file descriptor flags in a fairly painless way.
.PP
The
.BR fwatch (3)
module allows you to easily find out whether a file has changed since
the last time you looked at it.
.PP
The
.BR lbuf (3)
module implements a `line buffer', which is an object that emits
completed lines of text from an incoming asynchronous data stream.  It's
remarkably handy in programs that want to read lines from pipes and
sockets can't block while waiting for a line-end to arrive.  Similarly,
the
.BR pkbuf (3)
module implements a `packet buffer', which waits for packets of given
lengths to arrive before dispatching them to a handler.
.PP
The
.BR tv (3)
module provides some macros and functions for playing with
.BR "struct timeval" .
.PP
The
.BR bits (3)
module defines some types and macros for playing with words as chunks of
bits.  There are portable rotate and shift macros (harder than you'd
think), and macros to do loading and storing in known-endian formats.
values.
.PP
The
.BR mdwopt (3)
module implements a fairly serious options parser compatible with the
GNU options parser.
.PP
The
.BR testrig (3)
module provides a generic structure for reading test vectors from files
and running them through functions.  I mainly use it for testing
cryptographic transformations of various kinds.
.
.SS "Encoding and decoding"
The
.BR base64 (3)
module does base64 encoding and decoding, as defined in RFC2045.  Base64
encodes arbitrary binary data in a reliable way which is resistant to
character-set transformations and other mail transport bogosity.  The
.BR base32 (3)
module does base32 encoding and decoding, as defined in RFC2938.  This
is a mad format which is needed for sha1 URNs, for no good reason.  The
.BR hex (3)
module does hex encoding and decoding.
.PP
The
.BR url (3)
module does urlencoding and decoding, as defined in RFC1866.
Urlencoding encodes arbitrary (but mostly text-like) name/value pairs as
a text string containing no whitespace.
.
.SS "Multiplexed I/O"
The
.BR sel (3)
module provides a basis for doing nonblocking I/O in Unix systems.  It
provides types and functions for receiving events when files are ready
for reading or writing, and when timers expire.
.PP
The
.BR conn (3)
module implements nonblocking network connections in a way which fits in
with the
.B sel
system.  It makes nonblocking connects pretty much trivial.
.PP
The
.BR selbuf (3)
module attaches to the
.B sel
system and sends an event when lines of text arrive from a file.  It's
useful when reading text from a network connection.  Similarly,
.BR selpk (3)
sents events when packets of given sizes arrive from a file.
.PP
The
.BR sig (3)
module introduces signal handling into the multiplexed I/O world.
Signals are queued until dispatched through the normal
.B sel
mechanism.
.PP
The
.BR ident (3)
module provides a nonblocking ident (RFC931) client.  The
.BR bres (3)
module does background hostname and address resolution.
.
.\"--------------------------------------------------------------------------
.SH "SEE ALSO"
.
.BR alloc (3),
.BR assoc (3),
.BR atom (3),
.BR base64 (3),
.BR bits (3),
.BR buf (3),
.BR bres (3),
.BR conn (3),
.BR crc32 (3),
.BR darray (3),
.BR dspool (3),
.BR dstr (3),
.BR env (3),
.BR exc (3),
.BR fdflags (3),
.BR fwatch (3),
.BR hash (3),
.BR ident (3),
.BR lbuf (3),
.BR lock (3),
.BR mdwopt (3),
.BR pkbuf (3),
.BR quis (3),
.BR report (3),
.BR sel (3),
.BR selbuf (3),
.BR selpk (3),
.BR sig (3),
.BR str (3),
.BR sub (3),
.BR sym (3),
.BR trace (3),
.BR tv (3),
.BR url (3).
.
.\"--------------------------------------------------------------------------
.SH AUTHOR
.
Mark Wooding, <mdw@distorted.org.uk>
.
.\"----- That's all, folks --------------------------------------------------