| 1 | .\" -*-nroff-*- |
| 2 | .TH mLib 3 "7 July 1999" "Straylight/Edgeware" "mLib utilities library" |
| 3 | .SH NAME |
| 4 | mLib \- library of miscellaneous utilities |
| 5 | .\" @mLib |
| 6 | .SH DESCRIPTION |
| 7 | The |
| 8 | .B mLib |
| 9 | library is a mixed bag of things which the author finds useful in large |
| 10 | numbers of programs. As a result, its structure is somewhat arbitrary, |
| 11 | and it's accreted extra bits over time rather than actually being |
| 12 | designed as a whole. In the author's opinion this isn't too much of a |
| 13 | hardship. |
| 14 | .PP |
| 15 | At the most granular level, |
| 16 | .B mLib |
| 17 | is split into `modules', each of which has its own header file and |
| 18 | manual page. Sometimes there are identifiable `chunks' of several |
| 19 | modules which fit together as a whole. Modules and chunks fit into |
| 20 | `layers', each depending on the ones below it. The header file for |
| 21 | module |
| 22 | .I foo |
| 23 | would be put in |
| 24 | .BR <mLib/ \c |
| 25 | .IR foo \c |
| 26 | .BR .h> . |
| 27 | .PP |
| 28 | This description is a bit abstract, and |
| 29 | .BR mLib , |
| 30 | as a result of its history, doesn't fit it as well as I might like. |
| 31 | Even so, it's not too bad a model really. |
| 32 | .PP |
| 33 | The rest of this section describes the various chunks and layers. |
| 34 | .SS "Exception handling" |
| 35 | Right at the bottom, there's a fairly primitive exception handling |
| 36 | system. It's provided by the |
| 37 | .BR exc (3) |
| 38 | module, and stands alone. It's used mainly by the memory allocation |
| 39 | modules to raise exceptions when there's no more memory to be had. |
| 40 | .SS "Memory allocation" |
| 41 | The |
| 42 | .BR arena (3) |
| 43 | module provides an abstraction of memory allocation. By writing |
| 44 | appropriate arena implementations, a client program can control where |
| 45 | and how memory is allocated for various structures. |
| 46 | .PP |
| 47 | The |
| 48 | .BR alloc (3) |
| 49 | module provides simple veneers onto traditional memory allocation |
| 50 | functions like |
| 51 | .BR malloc (3) |
| 52 | and |
| 53 | .BR strdup (3) |
| 54 | (although |
| 55 | .B mLib |
| 56 | doesn't actually depend on |
| 57 | .B strdup |
| 58 | being defined in the library) which raise exceptions when there's not |
| 59 | enough memory left. These work through the |
| 60 | .B arena |
| 61 | layer, so that the caller can control memory allocation. |
| 62 | .PP |
| 63 | The |
| 64 | .BR sub (3) |
| 65 | module handles efficient allocation of small blocks. It allocates |
| 66 | memory in relatively big chunks and divides the chunks up into small |
| 67 | blocks before returning them. It keeps lists of differently-sized |
| 68 | blocks so allocation and freeing is fast. The downside is that your |
| 69 | code must know how big a block is when it's being freed. |
| 70 | .PP |
| 71 | The |
| 72 | .B track |
| 73 | module (not yet documented) is a simple memory allocation tracker. It |
| 74 | can be handy when trying to fix memory leaks. |
| 75 | .PP |
| 76 | The |
| 77 | .BR pool (3) |
| 78 | module maintains resource pools which can manage memory and other |
| 79 | resources, all of the resources held in a pool being destroyed along |
| 80 | with the pool itself. |
| 81 | .SS "String handling" |
| 82 | The |
| 83 | .BR str (3) |
| 84 | module provides some trivial string-manipulation functions which tend to |
| 85 | be useful quite often. |
| 86 | .PP |
| 87 | The |
| 88 | .BR dstr (3) |
| 89 | module implements a dynamic string data type. It works quite quickly |
| 90 | and well, and is handy in security-sensitive programs, to prevent |
| 91 | buffer-overflows. Dynamic strings are used occasionally through the |
| 92 | rest of the library, mainly as output arguments. |
| 93 | .PP |
| 94 | The |
| 95 | .BR buf (3) |
| 96 | module provides simple functions for reading and writing binary data to |
| 97 | or from fixed-sized buffers. |
| 98 | .PP |
| 99 | The |
| 100 | .BR dspool (3) |
| 101 | module implements a `pool' of dynamic strings which saves lots of |
| 102 | allocation and deallocation when a piece of code has high string |
| 103 | turnover. |
| 104 | .SS "Program identification and error reporting" |
| 105 | The |
| 106 | .BR quis (3) |
| 107 | module remembers the name of the program and supplies it when asked. |
| 108 | It's used in error messages and similar things. |
| 109 | .PP |
| 110 | The |
| 111 | .BR report (3) |
| 112 | module emits standard Unixy error messages. It provides functions |
| 113 | .B moan |
| 114 | and |
| 115 | .B die |
| 116 | which the author uses rather a lot. |
| 117 | .PP |
| 118 | The |
| 119 | .BR trace (3) |
| 120 | module provides an interface for emitting tracing information with |
| 121 | configurable verbosity levels. It needs improving to be able to cope |
| 122 | with outputting to the system log. |
| 123 | .SS "Other data types" |
| 124 | The |
| 125 | .BR hash (3) |
| 126 | module provides the basics for an extending hashtable implementation. |
| 127 | Many different hashtable-based data structures can be constructed with |
| 128 | little effort. |
| 129 | .PP |
| 130 | The |
| 131 | .BR sym (3) |
| 132 | module implements a rather good general-purpose extending hash table. |
| 133 | Keys and values can be arbitrary data. It is implemented using |
| 134 | .BR hash (3). |
| 135 | .PP |
| 136 | The |
| 137 | .BR atom (3) |
| 138 | module implements |
| 139 | .IR atoms , |
| 140 | which are essentially strings with the property that two atoms have the |
| 141 | same address if and only if they have the same text, so they can be used |
| 142 | for rapid string comparisons. The |
| 143 | .BR assoc (3) |
| 144 | module implements a hash table which uses atoms as keys, thus saving |
| 145 | time spent hashing and comparing hash keys, and the space used for the |
| 146 | keys. |
| 147 | .PP |
| 148 | The |
| 149 | .BR darray (3) |
| 150 | module implements dynamically resizing arrays which support Perl-like |
| 151 | stack operations efficiently. |
| 152 | .SS "Miscellaneous utilities" |
| 153 | The |
| 154 | .BR crc32 (3) |
| 155 | module calculates CRC values for strings. It used to be used by the |
| 156 | symbol table manager as a hash function. |
| 157 | .PP |
| 158 | The |
| 159 | .BR unihash (3) |
| 160 | module implements a simple but efficient universal hashing family. This |
| 161 | is a keyed hash function which provides security against an adversary |
| 162 | choosing input to a hash table deliberately to cause collisions. |
| 163 | .PP |
| 164 | The |
| 165 | .BR lock (3) |
| 166 | module does POSIX |
| 167 | .BR fcntl (2)-style |
| 168 | locking with a timeout. |
| 169 | .PP |
| 170 | The |
| 171 | .BR env (3) |
| 172 | module manipulates environment variables stored in a hashtable, and |
| 173 | converts between the hashtable and the standard array representation of |
| 174 | a process environment. |
| 175 | .PP |
| 176 | The |
| 177 | .BR fdflags (3) |
| 178 | module manipulates file descriptor flags in a fairly painless way. |
| 179 | .PP |
| 180 | The |
| 181 | .BR fwatch (3) |
| 182 | module allows you to easily find out whether a file has changed since |
| 183 | the last time you looked at it. |
| 184 | .PP |
| 185 | The |
| 186 | .BR lbuf (3) |
| 187 | module implements a `line buffer', which is an object that emits |
| 188 | completed lines of text from an incoming asynchronous data stream. It's |
| 189 | remarkably handy in programs that want to read lines from pipes and |
| 190 | sockets can't block while waiting for a line-end to arrive. Similarly, |
| 191 | the |
| 192 | .BR pkbuf (3) |
| 193 | module implements a `packet buffer', which waits for packets of given |
| 194 | lengths to arrive before dispatching them to a handler. |
| 195 | .PP |
| 196 | The |
| 197 | .BR tv (3) |
| 198 | module provides some macros and functions for playing with |
| 199 | .BR "struct timeval" . |
| 200 | .PP |
| 201 | The |
| 202 | .BR bits (3) |
| 203 | module defines some types and macros for playing with words as chunks of |
| 204 | bits. There are portable rotate and shift macros (harder than you'd |
| 205 | think), and macros to do loading and storing in known-endian formats. |
| 206 | values. |
| 207 | .PP |
| 208 | The |
| 209 | .BR mdwopt (3) |
| 210 | module implements a fairly serious options parser compatible with the |
| 211 | GNU options parser. |
| 212 | .PP |
| 213 | The |
| 214 | .BR testrig (3) |
| 215 | module provides a generic structure for reading test vectors from files |
| 216 | and running them through functions. I mainly use it for testing |
| 217 | cryptographic transformations of various kinds. |
| 218 | .SS "Encoding and decoding" |
| 219 | The |
| 220 | .BR base64 (3) |
| 221 | module does base64 encoding and decoding, as defined in RFC2045. Base64 |
| 222 | encodes arbitrary binary data in a reliable way which is resistant to |
| 223 | character-set transformations and other mail transport bogosity. The |
| 224 | .BR base32 (3) |
| 225 | module does base32 encoding and decoding, as defined in RFC2938. This |
| 226 | is a mad format which is needed for sha1 URNs, for no good reason. The |
| 227 | .BR hex (3) |
| 228 | module does hex encoding and decoding. |
| 229 | .PP |
| 230 | The |
| 231 | .BR url (3) |
| 232 | module does urlencoding and decoding, as defined in RFC1866. |
| 233 | Urlencoding encodes arbitrary (but mostly text-like) name/value pairs as |
| 234 | a text string containing no whitespace. |
| 235 | .SS "Multiplexed I/O" |
| 236 | The |
| 237 | .BR sel (3) |
| 238 | module provides a basis for doing nonblocking I/O in Unix systems. It |
| 239 | provides types and functions for receiving events when files are ready |
| 240 | for reading or writing, and when timers expire. |
| 241 | .PP |
| 242 | The |
| 243 | .BR conn (3) |
| 244 | module implements nonblocking network connections in a way which fits in |
| 245 | with the |
| 246 | .B sel |
| 247 | system. It makes nonblocking connects pretty much trivial. |
| 248 | .PP |
| 249 | The |
| 250 | .BR selbuf (3) |
| 251 | module attaches to the |
| 252 | .B sel |
| 253 | system and sends an event when lines of text arrive from a file. It's |
| 254 | useful when reading text from a network connection. Similarly, |
| 255 | .BR selpk (3) |
| 256 | sents events when packets of given sizes arrive from a file. |
| 257 | .PP |
| 258 | The |
| 259 | .BR sig (3) |
| 260 | module introduces signal handling into the multiplexed I/O world. |
| 261 | Signals are queued until dispatched through the normal |
| 262 | .B sel |
| 263 | mechanism. |
| 264 | .PP |
| 265 | The |
| 266 | .BR ident (3) |
| 267 | module provides a nonblocking ident (RFC931) client. The |
| 268 | .BR bres (3) |
| 269 | module does background hostname and address resolution. |
| 270 | .SH "SEE ALSO" |
| 271 | .BR alloc (3), |
| 272 | .BR assoc (3), |
| 273 | .BR atom (3), |
| 274 | .BR base64 (3), |
| 275 | .BR bits (3), |
| 276 | .BR buf (3), |
| 277 | .BR bres (3), |
| 278 | .BR conn (3), |
| 279 | .BR crc32 (3), |
| 280 | .BR darray (3), |
| 281 | .BR dspool (3), |
| 282 | .BR dstr (3), |
| 283 | .BR env (3), |
| 284 | .BR exc (3), |
| 285 | .BR fdflags (3), |
| 286 | .BR fwatch (3), |
| 287 | .BR hash (3), |
| 288 | .BR ident (3), |
| 289 | .BR lbuf (3), |
| 290 | .BR lock (3), |
| 291 | .BR mdwopt (3), |
| 292 | .BR pkbuf (3), |
| 293 | .BR quis (3), |
| 294 | .BR report (3), |
| 295 | .BR sel (3), |
| 296 | .BR selbuf (3), |
| 297 | .BR selpk (3), |
| 298 | .BR sig (3), |
| 299 | .BR str (3), |
| 300 | .BR sub (3), |
| 301 | .BR sym (3), |
| 302 | .BR trace (3), |
| 303 | .BR tv (3), |
| 304 | .BR url (3). |
| 305 | .SH AUTHOR |
| 306 | Mark Wooding, <mdw@distorted.org.uk> |