X-Git-Url: https://git.distorted.org.uk/~mdw/ezmlm/blobdiff_plain/5b62e993b0af39700031c2875d7f6654e6a02850..f8beb284087c279acfb30506f5bb32baa4949b44:/ezmlm.5

diff --git a/ezmlm.5 b/ezmlm.5
index 2866b91..6ac1cc6 100644
--- a/ezmlm.5
+++ b/ezmlm.5
@@ -21,7 +21,60 @@ manipulate the subscriber list stored under
 handles administrative requests automatically;
 .B ezmlm-send
 sends a message to all subscribers listed in
-.IR dir .
+.I dir
+and also maintains a message archive and message subject index if the list
+is configured to do so.
+.B ezmlm-reject
+rejects messages that have an empty subject, or a subject consisting of
+only a command word;
+.B ezmlm-return
+handles bounces;
+.B ezmlm-warn
+warns users for which messages bounce and eventually removes them from
+the subscriber list.
+.B ezmlm-idx
+can create a subject index from an existing list archive.
+.B ezmlm-get
+manages message, index, and thread retrieval from the archive, as well
+as the generation of message digests;
+.B ezmlm-cron
+provides a restricted interface to cron for the generation of
+digest generation trigger messages;
+.B ezmlm-store
+queues messages of moderated lists and sends a moderation request to
+the moderator(s);
+.B ezmlm-moderate
+processes moderation requests to accept the queued message to the list
+via
+.B ezmlm-send,
+or to return the message to the sender;
+.B ezmlm-clean
+cleans up the moderation queue and returns to the sender
+any messages that have timed-out;
+.B ezmlm-gate
+posts messages that come from a SENDER in an address database, and sends
+remaining messages out for moderation;
+.B ezmlm-check
+is used to diagnose problems with ezmlm mailing list configuration;
+.B ezmlm-issub
+and
+.B ezmlm-issubn
+determine if a SENDER is a subscriber or a member of a
+collection of addresses;
+.B ezmlm-tstdig
+determines if it is time to create a new digest based on the number and
+volume of messages and the amount of time that has passed since the last
+digest was issued;
+.B ezmlm-request
+can be used to answer
+.B ezmlm
+commands in the subject line easing migration from other mailing list
+managers. It can also function as a global interface mimicking
+the interface of other mailing list manager.
+.B ezmlm-glmake
+can set up the global interface, and
+.B ezmlm-glconf
+can create a configuration file for the global interface from your lists.
 .SH SUBSCRIBERS
 .I dir\fB/subscribers
 is a directory containing the subscriber list.
@@ -69,12 +122,19 @@ is a directory containing messages previously sent to subscribers.
 .B ezmlm-send
 archives all new messages if
 .I dir\fB/archived
-exists.
+exists. If
+.I dir\fB/indexed
+exists,
+.B ezmlm-send
+also maintains a message subject and author index.
 
 Messages sent to the mailing list are numbered from 1 upwards,
 whether or not they are archived.
 .I dir\fB/num
-is the number of messages sent so far.
+is the number of messages sent so far followed by ':', followed by the
+cumulative amount of message body that has passed
+.B ezmlm-send
+stored as kbytes * 4 (4 corresponds to 1kb).
 
 .I dir\fB/archive
 has subdirectories,
@@ -83,11 +143,43 @@ Message number 100m+n, with n between 0 and 99, is stored in
 .IR dir\fB/archive/\fIm\fB/\fIn .
 For example, message number 15307 is stored in
 .IR dir\fB/archive/153/07 .
+The message index is stored in the file
+.B index
+in the same subdirectory of
+.I dir\fB/archive
+holding the corresponding messages.
+Thus, the subject index contains up to 100 entries.
+
+The subject index contains message subjects that are normalized so that
+the original message and all replies have the same entry. The subject index
+is used for advanced message retrieval functions. For safety, the subject
+index is created under a temporary name
+inside
+.I dir\fB/archive
+and then moved into place.
 
 .B ezmlm-manage
 will ignore message files without the owner-execute bit set.
 .B ezmlm-send
 turns on the owner-execute bit after safely writing the message to disk.
+
+.B ezmlm-make
+by default adds
+.B ezmlm-get
+to
+.I dir\fB/manager
+to handle 
+.I \-get, \-index,
+and
+.I \-thread
+requests. If
+.B ezmlm-make
+is invoked with a 
+.I digcode
+command line argument, digest creation
+is enabled by putting this argument on the
+.B ezmlm-get
+command line.
 .SH BOUNCES
 .I dir\fB/bounce
 is a directory containing bounce messages.
@@ -127,9 +219,10 @@ handles incoming bounce messages.
 sets up
 .I dir\fB/bouncer
 to invoke
-.B ezmlm-return
-and then
-.BR ezmlm-warn .
+.BR ezmlm-return .
+.B ezmlm-warn
+is no longer invoked here due to the load it places on systems with many
+large lists with many bounces.
 
 .I dir\fB/manager
 handles incoming administrative requests.
@@ -137,9 +230,266 @@ handles incoming administrative requests.
 sets up
 .I dir\fB/manager
 to invoke
-.B ezmlm-manage
+.BR ezmlm-get ,
+.BR ezmlm-manage ,
 and then
 .BR ezmlm-warn .
+
+.I dir\fB/moderator
+handles incoming message
+.I accept
+and
+.I reject
+requests for moderated lists.
+.B ezmlm-make
+sets up
+.I dir\fB/moderator
+to invoke
+.BR ezmlm-moderate ,
+and .BR ezmlm-clean .
+.SH DIGESTS
+.B ezmlm-get
+can create digests if it is invoked from the command line, from
+.IR dir\fB/editor ,
+or from
+.IR dir\fB/manager .
+The program functions in slightly different ways in these 3 settings (see
+.BR ezmlm-get(1) ).
+
+To enable automatic digests for a mailing list, use the
+.B ezmlm-make \-d
+switch. To also enable the generation of digests at specific times dictated
+by mailed trigger messages, a
+.I digcode
+should be specified on the
+.B ezmlm-get
+command line.
+This can be done by specifying
+.I digcode
+as a fifth argument to
+.B ezmlm-make
+when setting up the list.
+.I digcode
+must be alphanumeric and is case-insensitive.
+
+To generate trigger messages, use
+.B ezmlm-cron(1)
+as an interface to
+.B crond(8)
+or use
+.B crond
+directly.
+
+.I dir\fB/num
+contains the number of the last message processed, followed by ':' and a
+number that is increased by 1 for each 256 bytes of message body text
+processed. The latter number is used by
+.B ezmlm-tstdig
+to determine if a new digest is due.
+
+.I dir\fB/dignum
+contains the contents of
+.I dir\fB/num
+at the time of the last regular digest creation, followed by a ':',
+followed by a timestamp.
+It is updated after each regular digest is sent.
+
+.I dir\fB/digissue
+contains the issue number of the last regular digest. It is incremented
+for each regular digest sent.
+
+The following user crontab entry (all on one line)
+generates a digest of the list
+.I list@host.domain
+at 1600 every day:
+
+.EX
+  00 16 * * * /var/qmail/bin/qmail-inject list-dig.digcode
+.EE
+
+Alternatively,
+.B ezmlm-cron
+can be used:
+
+.EX
+  % ezmlm-cron -t 16:00 list@host digcode
+.EE
+
+.B ezmlm-get
+can also be run from the shell: To generate a digest to
+.I list-digest@host
+from the list housed in
+.IR ~joe/list :
+
+.EX
+  % ezmlm-get ~joe/list
+.EE
+
+Like other
+.B ezmlm-get
+replies, digest can be sent in several formats. See
+.B ezmlm-get(1)
+for more info.
+.SH MODERATION
+There are three aspects of moderation: moderation of posts, moderation
+of subscriptions, and "remote administration", i.e. giving the
+moderator the right to (un)subscribe any user.
+.B ezmlm
+handles these three aspects separately. The two first aspects enhance
+security, while the third decreases security, but makes list administration
+considerably easier. By default, the moderator database is the same for all
+three functions. While "remote administration" and subscription moderation
+always use the same database, the moderators for message moderation can
+be different.
+
+Even with subscription moderation, the user has to verify the request. This
+is to ensure that the user initiating the request really controls the address.
+.B ezmlm-manage
+options exist to disable the user handshake, which may be useful in some
+circumstances.
+
+For moderation options, the moderators are by stored in a subscriber
+list in
+.IR moddir\fB/subscribers .
+By default
+.I moddir
+is
+.IR dir\fB/mod .
+
+Moderators can be added and removed with:
+
+.EX
+.B ezmlm-sub
+.I moddir
+.I moderator@host
+.EE
+
+.EX
+.B ezmlm-unsub
+.I moddir
+.I moderator@host
+.EE
+
+For subscription moderation, touch
+.IR dir\fB/modsub
+after adding moderator(s).
+For remote administration, touch
+.IR dir\fB/remote .
+If the contents of these files start with a leading forward slash, it is 
+assumed to be the name of
+.B moddir
+subscription
+moderation. If both files exist and start with a forward slash, the
+.I dir\fB/remote
+contents are ignored. Moderators are stored in a subscriber list in the
+.B subscribers
+subdirectory of
+.BR moddir .
+If no directory names are specified,
+the default,
+.IR dir\fB/mod ,
+is used.
+In all cases, the
+.I subscribers
+subdirectory of the base directory must exists/be created.
+
+Moderation of messages is achieved by
+creating
+.I dir\fB/modpost
+and  modifying
+.IR dir\fB/editor
+to invoke
+.B ezmlm-store.
+.B ezmlm-store
+stores the message in
+.IR dir\fB/mod/pending
+and sends a moderation request to all moderators stored in
+.IR moddir .
+
+If
+.I dir\fB/modpost
+does not exist,
+.B ezmlm-store
+posts messages directly, and
+.B ezmlm-clean
+does nothing.
+
+If
+.I dir\fB/modpost
+contains a directory name starting with a forward slash,
+this directory is used as
+.I moddir
+for message moderation.
+Moderators are stored in a subscriber list in the
+.I subscribers
+subdirectory of
+.IR moddir .
+If no directory names are specified,
+the default,
+.IR dir\fB/mod ,
+is used.
+
+.IR dir\fB/moderator
+is linked to
+.IR dot\fB\-accept-default
+and
+.IR dot\fB\-reject-default .
+It handles replies from the moderators.
+
+In addition to a moderator list, the directories
+.IR dir\fB/mod/pending ,
+.IR dir\fB/mod/accepted ,
+and
+.IR dir\fB/mod/rejected
+must exist. These directories contain the message moderation queue.
+
+If
+.IR dir\fB/mod/modtime
+it determines the minimal time in hours that messages wait in the moderation
+queue, before they are returned to sender with the message in
+.IR dir\fB/text/mod-timeout .
+
+If a
+.I \-help
+command is send for a moderator and
+.IR dir\fB/modsub
+or
+.IR dir\fB/remote
+exist, a more detailed help message stored in
+.I dir\fB/text/mod-help
+will be sent together with the regular help. This text should not contain
+secrets.
+If
+.I dir\fB/text/mod-help
+does not exist,
+.I dir\fB/text/help
+will be sent.
+
+If a
+.I \-list
+command is sent for a moderator and
+.IR dir\fB/modsub
+or
+.IR dir\fB/remote
+exist, and the
+.B ezmlm-manage \-l
+command line switch is specified, a subscriber list will be returned.
+
+If an
+.I \-edit.file
+command is sent for a moderator and
+.IR dir\fB/remote
+exist, and the
+.B ezmlm-manage \-d
+command line switch is specified,
+.B text\fB/file
+is returned together with an
+.B ezmlm
+cookie. The remote administrator may return an edited version of the
+file, which will be stored, provided that the cookie is valid.
+See
+.B ezmlm-manage(1)
+for more info.
 .SH TEXT
 .I dir\fB/text
 is a directory
@@ -187,6 +537,27 @@ Rejecting a bad unsubscription confirmation number.
 .B get-bad
 Rejecting a bad archive retrieval request.
 .TP
+.B digest
+Text copied into the
+.I Administrativia
+section of the digest. Usually, this will contain subscription info
+for the digest, as well as information on how to post to the list.
+.TP
+.B trailer
+If this files exists, it is copied to the end of all messages to the list.
+.TP
+.B faq
+Sent in response to the
+.I faq
+command. Usually contains frequently asked questions and answers specific
+for the mailing list.
+.TP
+.B info
+Sent in response to the
+.I info
+command. Usually contains a descripition, policy, etc, for the list. The
+first line should in itself be a very brief description of the list.
+.TP
 .B bounce-warn
 Pointing out that messages have bounced.
 .TP
@@ -198,9 +569,96 @@ Explaining that
 .B ezmlm-return
 has kept a list of bounced message numbers.
 .TP
+.B dig-bounce-num
+Explaining that digest messages have bounced. All other text files are used
+for both the main list and the digest list.
+.TP
 .B bounce-bottom
 Separating the bounce message.
+.TP
+.B mod-help
+is set to list moderators issuing a
+.I \-help
+command. It contains instructions for moderators, but it is relatively
+trivial for a non-moderator to read it. Don't put secrets here.
+.TP
+.B mod-reject
+is the returned to the sender of a rejected post.
+.TP
+.B mod-timeout
+is returned if the message timed-out without moderator action.
+.TP
+.B mod-sub
+is added to the text confirming subscription and unsubscription
+instead of
+.B bottom
+and the requesting message, for actions that were approved
+by a moderator. Not copying the requesting message
+hides the moderator identity
+from the subscriber.
+.TP
+.B mod-request
+is the text sent to the moderators to request moderator action on
+a posted message.
+.TP
+.B mod-unsub-confirm
+Requesting that the moderator confirm a request to subscribe.
+If this file does not exist,
+.B sub-confirm
+will be used.
+.TP
+.B mod-unsub-confirm
+Requesting that the moderator confirm a request to unsubscribe.
+If this file does not exist,
+.B unsub-confirm
+will be used.
+.TP
+.B edit-do
+Instructions sent to the remote administrator together with a copy
+of a
+.I dir\fB/text
+file and editing instructions.
+.TP
+.B edit-list
+A list of editable files in
+.I dir\fB/text
+with a one-line description send to a remote administrator in response to a
+.I -edit
+command.
+.TP
+.B edit-done
+Sent to the remote administrator after an edited
+.I dir\fB/text
+file has been successfully saved.
 .PP
+Several tags in the text files are replaced by ezmlm programs.
+All programs replace the tag
+.B <#l#>
+with the name of the list or the list-digest, as appropriate for the request,
+and
+.B <#h#>
+with the hostname for the list.
+.B ezmlm-send
+and
+.B ezmlm-get
+replace
+.B <#n#>
+with the current message number in added headers from
+.I dir\fB/headeradd
+and text files.
+.B ezmlm-get
+does this for digest messages, where the current message is the number of
+the first message in the digest.
+.B ezmlm-manage
+replaces the tag
+.B <#A#>
+anywhere on a line with the subscription address, and
+.B <#R#>
+anywhere on a line
+with the address the subscriber must reply to. Only the first tag on any
+line is processed.
+.PP
+For backwards compatibility,
 .B ezmlm-manage
 replaces the line
 .B !A
@@ -213,6 +671,23 @@ in
 and
 .B unsub-confirm
 with the address that the subscriber must reply to.
+.PP
+.B ezmlm-store
+replaces the tag
+.B <#A#>
+anywhere on a line with the address for accepting the message, and
+.B <#R#>
+anywhere on a line
+with the address for rejecting the message.
+Only the first tag on any line is processed.
+.PP
+For backwards compatibility,
+.B ezmlm-store
+also replaces the line
+.B !A
+with the address for accepting the message and the line
+.B !R
+with the address for rejecting the message.
 .SH "OUTGOING MESSAGE EDITING"
 .I dir\fB/headerremove
 is a list of bad header field names,
@@ -233,10 +708,61 @@ fields.
 is a list of new header fields.
 .B ezmlm-send
 adds these fields to every outgoing message.
-.B ezmlm-make
+.B ezmlm-send
 sets up
 .I dir\fB/headeradd
-with no new fields.
+to add
+.B X-No-Archive: yes
+and
+.BR Precedence: bulk .
+
+If
+.I dir\fB/mimeremove
+exists,
+.B ezmlm-send
+removed parts with the corresponding content-types from composite MIME
+messages. If the
+.B ezmlm-reject
+.I dir
+argument is specified,
+simple MIME messages of these content-types are rejected.
+
+If
+.I dir\fB/mimereject
+exists, and the
+.B ezmlm-reject
+.I dir
+argument is specified,
+simple MIME messages of these content-types, or
+composite MIME messages with any body part of these content-types are rejected.
+
+If
+.I dir\fB/sequence
+exists, the first line is added as a header to all outgoing messages, followed
+by a space and the message number. The message number is useful for archive
+retrievals, since some mail systems do not reveal the return-path to the user.
+.B NOTE:
+Sublists have their own message counter. Adding a sequence header from a
+sublists will give you the sublist message number which is different from
+the main list message number.
+
+.I dir\fB/prefix
+is a subject prefix. If this file exists, its contents are prefixed to the
+subject of the post in the outgoing message. The archived message is not
+processed. Attempts are made to not duplicate an existing prefix in replies.
+Think twice before using this option.
+A prefix takes unnecessary space on the subject line and most mail clients
+can easily filter on other headers, such as 'Mailing-List:'. If
+.I dir\fB/prefix contains a single '#', this will be replaced by the message
+number. The use of this feature is inadvisable and violates internet mail
+standards. However, it is very popular in e.g. Japan. If you must use this
+feature, make sure you are aware that you may be causing problems to users,
+sublists, etc.
+
+.I dir\fB/text/trailer
+is a message trailer. If this file exists, it's contents are copied to the 
+end of outgoing messages. Only lines terminated with new-line are copied.
+No trailer is copied to the archived version of the message.
 .SH MISCELLANY
 The first line of
 .I dir\fB/mailinglist
@@ -248,6 +774,27 @@ field, showing the contents of
 .IR dir\fB/mailinglist ,
 in every outgoing message.
 
+If
+.I dir\fB/listid
+exists,
+ezmlm programs create a new
+.B List-ID
+field, showing the contents of the first line of
+.IR dir\fB/listid ,
+in every outgoing message. The list-id should be unique and within name
+space controlled by the owner. It should remain constant even if lists
+move and be of the format
+
+.EX
+List-ID: optional_text <unique_id.domain>
+.EE
+
+This header would result from a
+.I dir\fB/listid
+file containing ``optional_text <unique_id.domain>''. See
+.I http://www.within.com/~chandhok/ietf/listid.shtml
+for more info.
+
 The first lines of
 .I dir\fB/outlocal
 and
@@ -292,6 +839,44 @@ is the name of the parent list.
 This affects the behavior of
 .BR ezmlm-send .
 
+If
+.I dir\fB/qmqpservers
+exists,
+.B ezmlm-send
+and
+.B ezmlm-get
+will use
+.B qmail-qmqpc(1)
+to send posts and digests. Other mail will use the normal qmail mechanism.
+If
+.B qmail-qmqpc
+is modified correctly, server IP addresses listed one per line in
+.I dir\fB/qmqpsevers
+will be tried in order, rather than the default servers specified in
+.IR /var/qmail/control .
+
+If
+.I dir\fB/msgsize
+exists, it is assumed to contain ``max:min'', where ``max'' is the maximum
+size in bytes of an acceptable message body, and ``min'' the corresponding
+minimal size. Either will be ignored if zero or omitted. If the
+.B ezmlm-reject
+command line specifies the list directory, messages not meeting the size
+criteria are rejected.
+
+If
+.I dir\fB/charset
+exists, the first line is assumed to represent a valid MIME character set,
+which is used for all outgoing MIME messages sent by
+.B ezmlm-get 
+and the message moderation programs. The character set string may be suffixed
+with ':' and 'Q' or 'B' to send all outgoing
+text (ezmlm messages, digest table-of-contents, moderation requests, etc)
+encoded in ``Quoted-Printable'' or ``base64'' encoding. By default, no encoding
+is done, which may result in the transmission of characters with the high
+bit set. When encoding is specified, trigger messages and other parts of the
+reply that should not be encoded are sent as separate MIME parts.
+
 .I dir\fB/lock
 is an empty file.
 Any program that reads or writes the subscriber list,
@@ -304,14 +889,54 @@ is an advisory log of subscription and unsubscription actions.
 .B WARNING:
 .B Log
 is not protected against system crashes.
-Log entries may be missing or corrupted if the system goes down.
+Log entries may be missing or corrupted if the system goes down. There is
+Log for each of the accessory address databases as well. Thus, the log
+for digest subscribers is
+.IR dir\fB/digest/Log .
+If enabled, these logs can be retrieved by remote administrators (see
+.BR ezmlm-manage(1) ).
+
+.I dir\fB/digest
+contains items specific for the digest list.
+
+.I dir\fB/digest/subscribers
+contains hash files of digest subscriber addresses.
+
+.IR dir\fB/digest/Log ,
+.IR dir\fB/digest/bounce ,
+.IR dir\fB/digest/lockbounce ,
+and
+.I dir\fB/digest/lock
+have functions for the digest list that mirror that of the corresponding
+files in
+.IR dir .
+
+.I dir\fB/sql
+contains SQL server access information for list that are configured to
+use an SQL database for storage.
+
+.I dir\fB/tstdig
+is a timestamp used temporarily by
+.B ezmlm-tstdig(1)
+to coordinate digesting.
 .SH "SEE ALSO"
+ezmlm-check(1),
+ezmlm-clean(1),
+ezmlm-gate(1),
+ezmlm-get(1),
+ezmlm-idx(1),
+ezmlm-issub(1),
+ezmlm-issubn(1),
 ezmlm-list(1),
 ezmlm-make(1),
 ezmlm-manage(1),
+ezmlm-moderate(1),
+ezmlm-request(1),
 ezmlm-return(1),
 ezmlm-send(1),
+ezmlm-store(1),
 ezmlm-sub(1),
+ezmlm-tstdig(1),
 ezmlm-unsub(1),
 ezmlm-warn(1),
 dot-qmail(5)