3 .\" Manual for line buffering
5 .\" (c) 1999--2003, 2005, 2007, 2009, 2023, 2024 Straylight/Edgeware
8 .\"----- Licensing notice ---------------------------------------------------
10 .\" This file is part of the mLib utilities library.
12 .\" mLib is free software: you can redistribute it and/or modify it under
13 .\" the terms of the GNU Library General Public License as published by
14 .\" the Free Software Foundation; either version 2 of the License, or (at
15 .\" your option) any later version.
17 .\" mLib is distributed in the hope that it will be useful, but WITHOUT
18 .\" ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
19 .\" FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public
20 .\" License for more details.
22 .\" You should have received a copy of the GNU Library General Public
23 .\" License along with mLib. If not, write to the Free Software
24 .\" Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
27 .\"--------------------------------------------------------------------------
28 .so ../defs.man \" @@@PRE@@@
30 .\"--------------------------------------------------------------------------
31 .TH lbuf 3mLib "6 July 1999" "Straylight/Edgeware" "mLib utilities library"
40 .\"--------------------------------------------------------------------------
42 lbuf \- split lines out of asynchronously received blocks
44 .\"--------------------------------------------------------------------------
49 .B "#include <mLib/lbuf.h>"
53 .B " LBUF_STRICTCRLF,"
56 .B "#define LBUF_ENABLE ..."
63 .B "typedef void lbuf_func(char *" s ", size_t " len ", void *" p );
65 .BI "void lbuf_flush(lbuf *" b ", char *" p ", size_t " len );
66 .BI "void lbuf_close(lbuf *" b );
67 .BI "size_t lbuf_free(lbuf *" b ", char **" p );
68 .BI "void lbuf_snarf(lbuf *" b ", const void *" p ", size_t " sz );
69 .BI "void lbuf_setsize(lbuf *" b ", size_t " sz );
70 .BI "void lbuf_init(lbuf *" b ", lbuf_func *" func ", void *" p );
71 .BI "void lbuf_destroy(lbuf *" b );
74 .\"--------------------------------------------------------------------------
79 implement a handy object called a
81 Given unpredictably-sized chunks of data, the line buffer extracts
82 completed lines of text and passes them to a caller-supplied function.
83 This is useful in nonblocking network servers, for example: the server
84 can feed input from a client into a line buffer as it arrives and deal
85 with completed text lines as they appear without having to wait for
88 The state of a line buffer is stored in an object of type
90 This is a structure which must be allocated by the caller. The
91 structure should normally be considered opaque (see the section on
93 for an exception to this).
95 .SS "Initialization and finalization"
98 initializes a line buffer ready for use. It is given three arguments:
101 A pointer to the block of memory to use for the line buffer. The line
102 buffer will allocate memory to store incoming data automatically: this
103 structure just contains bookkeeping information.
105 .BI "lbuf_func *" func
108 function to which the line buffer should pass completed lines of text.
110 .B "Line-handler functions"
111 below for a description of this function.
114 A pointer argument to be passed to the function when a completed line of
117 The amount of memory set aside for reading lines is configurable. It
118 may be set by calling
120 at any time when the buffer is empty. The default limit is 256 bytes.
121 Lines longer than the limit are truncated. By default, the buffer is
122 allocated from the current arena,
123 .BR arena_global (3);
124 this may be changed by altering the buffer's
126 member to refer to a different arena at any time when the buffer is
129 A line buffer must be destroyed after use by calling
131 passing it the address of the buffer block.
133 .SS "Inserting data into the buffer"
134 There are two interfaces for inserting data into the buffer. One's much
135 simpler than the other, although it's less expressive.
137 The simple interface is
139 This function is given three arguments: a pointer
141 to a line buffer structure; a pointer
143 to a chunk of data to read; and the size
145 of the chunk of data. The data is pushed through the line buffer and
146 any complete lines are passed on to the line handler.
148 The complex interface is the pair of functions
155 function returns the address and size of a free portion of the line
156 buffer's memory into which data may be written. The function is passed
159 of the line buffer. Its result is the size of the free area, and it
160 writes the base address of this free space to the location pointed to by
163 The caller's data must be written to ascending memory locations starting
166 and no data may be written beyond the end of the free space. However,
167 it isn't necessary to completely fill the buffer.
169 Once the free area has had some data written to it,
171 is called to examine the new data and break it into text lines. This is
172 given three arguments:
175 The address of the line buffer.
178 The address at which the new data has been written. This must be the
179 base address returned from
183 The number of bytes which have been written to the buffer.
187 function breaks the new data into lines as described below, and passes
188 each one in turn to the line-handler function.
192 function is trivially implemented in terms of the more complex
193 .BR lbuf_free / lbuf_flush
197 By default, the line buffer considers a line to end with either a simple
198 linefeed character (the normal Unix convention) or a
199 carriage-return/linefeed pair (the Internet convention). This can be
200 changed by modifying the
204 structure: the default value is
207 .BR LBUF_STRICTCRLF ,
208 only a carriage-return/linefeed pair will terminate a line. Any other
209 value is a single character which is considered to be the line terminator.
211 The line buffer has a fixed amount of memory available to it. This is
212 deliberate, to prevent a trivial attack whereby a remote user sends a
213 stream of data containing no newline markers, wasting the server's
214 memory. Instead, the buffer will truncate overly long lines (silently)
215 and return only the initial portion. It will ignore the rest of the
218 .SS "Line-handler functions"
219 Completed lines, as already said, are passed to the caller's
220 line-handler function. It is given three arguments: the address
222 of the line which has just been read; the length
224 of the line (not including the null terminator), and the pointer
226 which was set up in the call to
228 The line passed is null-terminated, and has had its trailing newline
229 stripped. The area of memory in which the string is located may be
230 overwritten by the line-handler function, although writing beyond the
231 terminating zero byte is not permitted.
233 The line pointer argument
235 may be null to signify end-of-file; in this case, the length
237 will be zero. See the next section.
239 .SS "Flushing the remaining data"
240 When the client program knows that there's no more data arriving (for
241 example, an end-of-file condition exists on its data source) it should
244 to flush out the remaining data in the buffer as one last (improperly
245 terminated) line. This will pass the remaining text to the line
246 handler, if there is any, and then call the handler one final time with
247 a null pointer rather than the address of a text line to inform it of
251 The line buffer is intended to be used in higher-level program objects,
252 such as the buffer selector described in
254 Unfortunately, a concept from this high level needs to exist at the line
255 buffer level, which complicates the description somewhat. The idea is
256 that, when a line-handler attached to some higher-level object decides
257 that it's read enough, it can
259 the object so that it doesn't see any more data.
263 call can emit more than one line, it must be aware that the line handler
264 isn't interested in any more lines. However, this fact must also be
265 signalled to the higher-level object so that it can detach itself from
268 Rather than invent some complex interface for this, the line buffer
269 exports one of its structure members, a flags word called
271 A higher-level object wishing to disable the line buffer simply clears
276 Disabling a buffer causes an immediate return from
278 However, it is not permitted for the functions
282 to be called on a disabled buffer. (This condition isn't checked for;
283 it'll just do the wrong thing.) Furthermore, the
285 function does not handle disablement at all, because it would complicate
286 the interface so much that it wouldn't have any advantage over the more
288 .BR lbuf_free / lbuf_flush .
290 .\"--------------------------------------------------------------------------
296 .\"--------------------------------------------------------------------------
299 Mark Wooding, <mdw@distorted.org.uk>
301 .\"----- That's all, folks --------------------------------------------------
~mdw / mLib / blob