--- /dev/null
+.\" -*-nroff-*-
+.ie t .ds , \h'\w'\ 'u/2u'
+.el .ds , \ \"
+.TH bench 3 "9 March 2024" "Straylight/Edgeware" "mLib utilities library"
+.\" @bench_createtimer
+.\" @bench_init
+.\" @bench_destroy
+.\" @bench_calibrate
+.\" @bench_measure
+.
+.SH SYNOPSIS
+.nf
+.B "#include <mLib/bench.h>"
+.PP
+.ta 2n
+.B "struct bench_time {"
+.B " unsigned f;"
+.B " kludge64 s;"
+.B " uint32 ns;"
+.B " kludge64 cy;"
+.B "};"
+.PP
+.B "struct bench_timing {"
+.B " unsigned f;"
+.B " double n;"
+.B " double t;"
+.B " double cy;"
+.B "};"
+.PP
+.B "struct bench_timerops {"
+.BI " void (*describe)(struct bench_timer *" bt ", dstr *" d );
+.BI " void (*now)(struct bench_timer *" bt ", struct bench_time *" t_out );
+.BI " void (*destroy)(struct bench_timer *" bt );
+.B "};"
+.B "struct bench_timer {"
+.B " const struct bench_timerops *ops;"
+.B "};"
+.PP
+.B "struct bench_state {"
+.B " unsigned f;"
+.B " double target_s;"
+.B " ..."
+.B "}";
+.PP
+.BI "typedef void bench_fn(unsigned long " n ", void *" ctx );
+.PP
+.B "#define BTF_TIMEOK ..."
+.B "#define BTF_CYOK ..."
+.B "#define BTF_CLB ..."
+.B "#define BTF_ANY (BTF_TIMEOK | BTF_CYOK)"
+.PP
+.B "struct bench_timer *bench_createtimer(void);"
+.PP
+.BI "int bench_init(struct bench_state *" b ", struct bench_timer *" tm );
+.BI "void bench_destroy(struct bench_state *" b );
+.BI "int bench_calibrate(struct bench_state *" b );
+.ta \w'\fBint bench_measure('u
+.BI "int bench_measure(struct bench_state *" b ", struct bench_timing *" t_out ,
+.BI " double " base ", bench_fn *" fn ", void *" ctx );
+.fi
+.
+.SH DESCRIPTION
+The header file
+.B "<mLib/bench.h>"
+provides declarations and defintions
+for performing low-level benchmarks.
+.PP
+The `main event' is
+.BR bench_measure .
+This function will be described in detail later,
+but, in brief,
+it calls a caller-provided function,
+instructing it to run adaptively chosen numbers of iterations,
+in order to get a reasonably reliable measurement of its running time,
+and then reports its results by filling in a structure.
+.PP
+With understanding this function as our objective,
+we must examine all of the pieces involved in making it work.
+.
+.SS Timers in general
+A
+.I timer
+is a gadget which is capable of reporting the current time,
+in seconds (ideally precise to tiny fractions of a second),
+and/or in CPU cycles.
+A timer is represented by a pointer to an object of type
+.BR "struct bench_timer" .
+This structure has a single member,
+.BR ops ,
+pointing to a
+.BR "struct bench_timerops" ,
+which is a table of function pointers;
+typically, a timer has more data following this,
+but this fact is not exposed to applications.
+.PP
+The function pointers in
+.B "struct bench_timerops"
+are as follows.
+The first argument,
+named
+.I tm
+must always point to the timer object itself.
+.TP
+.IB tm ->ops->describe( tm ", " d)
+Write a description of the timer to the dynamic string
+.IR d .
+.TP
+.IB tm ->ops->now( tm ", " t_out)
+Store the current time in
+.IR t_out .
+The
+.B struct bench_time
+used to represent the time reported by a timer
+is described in detail below.
+.TP
+.IB tm ->ops->destroy( tm )
+Destroy the timer,
+releasing all of the resources that it holds.
+.PP
+A time, a reported by a timer, is represented by the
+.BR "struct bench_time" .
+A passage-of-time measurement is stored in the
+.B s
+and
+.B ns
+members, holding seconds and nanoseconds respectively.
+(A timer need not have nanosecond precision.
+The exact interpretation of the time \(en
+e.g., whether it measures wallclock time,
+user-mode CPU time,
+or total thread CPU time \(en
+is a matter for the specific timer implementation.)
+A cycle count is stored in the
+.B cy
+member.
+The
+.B f
+member stores flags:
+.B BTF_TIMEOK
+is set if the passage-of-time measurement
+.B s
+and
+.B ns
+are valid; and
+.B BTF_CYOK
+is set if the cycle count
+.B cy
+is valid.
+Neither the time nor the cycle count need be measured
+relative to any particular origin.
+The mask
+.B BTF_ANY
+covers the
+.B BTF_TIMEOK
+and
+.B BTF_CYOK
+bits:
+hence,
+.IB f &BTF_ANY
+is nonzero (true)
+if the timer returned any valid timing information.
+.
+.SS The built-in timer
+The function
+.B bench_createtimer
+constructs and returns a timer.
+It takes a single argument,
+a string
+.IR config ,
+from which it reads configuration information.
+If
+.B bench_createtimer
+fails, it returns a null pointer.
+.PP
+The
+.I config
+pointer may safely be null,
+in which case a default configuration will be used.
+Applications
+.I should only
+set this pointer to a value supplied by a user,
+e.g., through a command-line argument,
+environment variable, or
+configuration file.
+.PP
+The built-in timer makes use of one or two
+.IR subtimers :
+a `clock' subtimer to measure the passage of time,
+and possibly a `cycle' subtimer to count CPU cycles.
+.PP
+The configuration string consists of a sequence of words
+separated by whitespace.
+There may be additional whitespace at the start and end of the string.
+The words recognized are as follows.
+.TP
+.B list
+Prints a list of the available clock and cycle subtimers
+to standard output.
+.TP
+.BI clock= t , ...
+Use the first of the listed clock subtimers
+to initialize successfully
+as the clock subtimer.
+If none of the subtimers can be initialized,
+then construction of the timer as a whole fails.
+.TP
+.BI cycle= t , ...
+Use the first of the listed subtimers
+to initialize successfully
+as the cycle subtimer.
+If none of the subtimers can be initialized,
+then construction of the timer as a whole fails.
+.PP
+The clock subtimers are as follows.
+Not all of them will be available on every platform.
+.TP
+.B posix-thread-cputime
+Measures the passage of time using
+.BR clock_gettime (2),
+specifying the
+.B CLOCK_\%THREAD_\%CPUTIME_\%ID
+clock.
+.TP
+.B stdc-clock
+Measures the passage of time using
+.BR clock (3).
+Since
+.BR clock (3)
+is part of the original ANSI\ C standard,
+this subtimer should always be available.
+However, it may produce unhelpful results
+if other threads are running.
+.PP
+The cycle subtimers are as follows.
+Not all of them will be available on every platform.
+.TP
+.B linux-perf-event
+Counts CPU cycles using the Linux-specific
+.BR perf_event_open (2)
+function to read the
+.BR PERF_\%COUNT_\%HW_\%CPU_\%CYCLES
+counter.
+Only available on Linux.
+It will fail to initialize
+if access to performance counters is restricted,
+e.g., because the
+.B /proc/sys/kernel/perf_event_paranoid
+level is too high.
+.TP
+.B x86-rdtsc
+Counts CPU cycles using the x86
+.B rdtsc
+instruction.
+This instruction is not really suitable for performance measurement:
+it gives misleading results on CPUs with variable clock frequency.
+.TP
+.B null
+A dummy cycle counter,
+which will initialize successfully
+and then fail to report cycle counts.
+This is a reasonable fallback in many situations.
+.PP
+The built-in preference order for clock subtimers,
+from most to least preferred, is
+.B posix-thread-cputime
+followed by
+.BR stdc-clock .
+The built-in preference order for cycle subtimers,
+from most to least preferred, is
+.B linux-perf-event
+followed by
+.BR x86-rdtsc ,
+and then
+.BR null .
+.
+.SS The benchmark state
+A
+.I benchmark state
+tracks the information needed to measure performance of functions.
+It is represented by a
+.B struct bench_state
+structure.
+.PP
+The benchmark state is initialized by calling
+.BR bench_init ,
+passing the address of the state structure to be initialized,
+and a pointer to a timer.
+If
+.B bench_init
+is called with a non-null timer pointer,
+then it will not fail;
+the benchmark state will be initialized,
+and the function returns zero.
+If the timer pointer is null,
+then
+.B bench_init
+attempts to construct a timer for itself
+by calling
+.BR bench_createtimer .
+If this succeeds,
+then the benchmark state will be initialized,
+and the function returns zero.
+In both cases,
+the timer becomes owned by the benchmark state:
+calling
+.B bench_destroy
+on the benchmark state will destroy the timer.
+If
+.B bench_init
+is called with a null timer pointer,
+and its attempt to create a timer for itself fails,
+then
+.B bench_init
+returns \-1;
+the benchmark state is not initialized
+and can safely be discarded;
+calling
+safe to call
+.B bench_destroy
+on the unsuccessfully benchmark state is safe and has no effect.
+.PP
+Calling
+.B bench_destroy
+on a benchmark state
+releases any resources it holds,
+most notably its timer, if any.
+.PP
+Although
+.B struct bench_state
+is defined in the header file,
+only two members are available for use by applications.
+.TP
+.B f
+A word containing flags.
+.TP
+.B target_s
+The target time for which to try run a benchmark, in seconds.
+After initialization, this is set to 1.0,
+though applications can override it.
+.PP
+Before the benchmark state can be used in measurements,
+it must be
+.IR calibrated .
+This is performed by calling
+.B bench_calibrate
+on the benchmark state.
+Calibration takes a noticeable amount of time
+(currently about 0.25\*,s),
+so it makes sense to defer it until it's known to be necessary.
+.PP
+Calibration is carried out separately, but in parallel,
+for the timer's passage-of-time measurement and cycle counter.
+Either or both of these calibrations can succeed or fail;
+if passage-of-time calibration fails,
+then cycle count calibration is impossible.
+.PP
+When it completes,
+.B bench_calibrate
+sets flag in the benchmark state's
+.B f
+member:
+if passage-of-time calibration succeeded,
+.B BTF_TIMEOK
+is set;
+if cycle-count calibration succeeded,
+.B BTF_CYOK
+is set;
+and the flag
+.B BTF_CLB
+is set unconditionally,
+as a persistent indication that calibration has been attempted.
+.PP
+The
+.B bench_calibrate
+function returns zero if it successfully calibrated
+at least the passage-of-time measurement;
+otherwise, it returns \-1.
+If
+.B bench_calibrate
+is called for a second or subsequent time on the same benchmark state,
+it returns immediately,
+either returning 0 or \-1
+according to whether passage-of-time had previously been calibrated.
+.
+.SS Timing functions
+A
+.I benchmark function
+has the signature
+.IP
+.BI "void " fn "(unsigned long " n ", void *" ctx );
+.PP
+When called, it should perform the operation to be measured
+.I n
+times.
+The
+.I ctx
+argument is a pointer passed into
+.B bench_measure
+for the benchmark function's own purposes.
+.PP
+The function
+.B bench_measure
+receives five arguments.
+.TP
+.I b
+points to the benchmark state to be used.
+.TP
+.I t_out
+is the address of a
+.BR struct bench_timing
+in which the measurement should be left.
+This structure is described below.
+.TP
+.I base
+is a count of the number of operations performed
+by each iteration of the benchmark function.
+.TP
+.I fn
+is a benchmark function, described above.
+.TP
+.I ctx
+is a pointer to be passed to the benchmark function.
+.B bench_measure
+does not interpret this pointer in any way.
+.PP
+The
+.B bench_measure
+function calls its benchark function repeatedly
+with different iteration counts
+.IR n ,
+with the objective that the call take approximately
+.B target_s
+seconds, as established in the benchmark state.
+(Currently, if
+.B target_s
+holds the value
+.IR t ,
+then
+.B bench_measure
+is satisfied when a call takes at least
+.IR t /\(sr2\*,s.)
+Once the function finds a satisfactory number of iterations,
+it stores the results in
+.BI * t_out \fR.
+If measurement succeeds, then
+.B bench_measure
+returns zero.
+If it fails \(en
+most likely because the timer failed \(en
+then it returns \-1.
+.PP
+A
+.B bench_timing
+structure reports the outcome of a successful measurement.
+It has four members.
+.TP
+.B f
+A flags word.
+.B BTF_TIMEOK
+is set if the passage-of-time measurement in
+.B t
+is valid;
+.B BTF_CYOK
+is set if the cycle count in
+.B cy
+is valid.
+.TP
+.B n
+The number of iterations performed by the benchmark function
+on its satisfactory run,
+multiplied by
+.IR base .
+.TP
+.B t
+The time taken for the satisfactory run of the benchmark function,
+in seconds.
+Only valid if
+.B BTF_TIMEOK
+is set in
+.BR f .
+.TP
+.B cy
+The number of CPU cycles used
+in the satisfactory run of the benchmark function,
+in seconds.
+Only valid if
+.B BTF_CYOK
+is set in
+.BR f .
+.
+.SH "SEE ALSO"
+.BR mLib (3).
+.
+.SH AUTHOR
+Mark Wooding, <mdw@distorted.org.uk>
~mdw / mLib / blobdiff