mdw@git.distorted.org.uk Git - qmail/blob - INTERNALS

   1 1. Overview
   2
   3 Here's the data flow in the qmail suite:
   4
   5  qmail-smtpd --- qmail-queue --- qmail-send --- qmail-rspawn --- qmail-remote
   6                /                     |      \
   7 qmail-inject _/                 qmail-clean  \_ qmail-lspawn --- qmail-local
   8
   9 Every message is added to a central queue directory by qmail-queue.
  10 qmail-queue is invoked as needed, usually by qmail-inject for locally
  11 generated messages, qmail-smtpd for messages received through SMTP,
  12 qmail-local for forwarded messages, or qmail-send for bounce messages.
  13
  14 Every message is then delivered by qmail-send, in cooperation with
  15 qmail-lspawn and qmail-rspawn, and cleaned up by qmail-clean. These four
  16 programs are long-running daemons.
  17
  18 The queue is designed to be crashproof, provided that the underlying
  19 filesystem is crashproof. All cleanups are handled by qmail-send and
  20 qmail-clean without human intervention. See section 6 for more details.
  21
  22
  23 2. Queue structure
  24
  25 Each message in the queue is identified by a unique number, let's say
  26 457. The queue is organized into several directories, each of which may
  27 contain files related to message 457:
  28
  29    mess/457: the message
  30    todo/457: the envelope: where the message came from, where it's going
  31    intd/457: the envelope, under construction by qmail-queue
  32    info/457: the envelope sender address, after preprocessing
  33    local/457: local envelope recipient addresses, after preprocessing
  34    remote/457: remote envelope recipient addresses, after preprocessing
  35    bounce/457: permanent delivery errors
  36
  37 Here are all possible states for a message. + means a file exists; -
  38 means it does not exist; ? means it may or may not exist.
  39
  40    S1. -mess -intd -todo -info -local -remote -bounce
  41    S2. +mess -intd -todo -info -local -remote -bounce
  42    S3. +mess +intd -todo -info -local -remote -bounce
  43    S4. +mess ?intd +todo ?info ?local ?remote -bounce (queued)
  44    S5. +mess -intd -todo +info ?local ?remote ?bounce (preprocessed)
  45
  46 Guarantee: If mess/457 exists, it has inode number 457.
  47
  48
  49 3. How messages enter the queue
  50
  51 To add a message to the queue, qmail-queue first creates a file in a
  52 separate directory, pid/, with a unique name. The filesystem assigns
  53 that file a unique inode number. qmail-queue looks at that number, say
  54 457. By the guarantee above, message 457 must be in state S1.
  55
  56 qmail-queue renames pid/whatever as mess/457, moving to S2. It writes
  57 the message to mess/457. It then creates intd/457, moving to S3, and
  58 writes the envelope information to intd/457.
  59
  60 Finally qmail-queue creates a new link, todo/457, for intd/457, moving
  61 to S4. At that instant the message has been successfully queued, and
  62 qmail-queue leaves it for further handling by qmail-send.
  63
  64 qmail-queue starts a 24-hour timer before touching any files, and
  65 commits suicide if the timer expires.
  66
  67
  68 4. How queued messages are preprocessed
  69
  70 Once a message has been queued, qmail-send must decide which recipients
  71 are local and which recipients are remote. It may also rewrite some
  72 recipient addresses.
  73
  74 When qmail-send notices todo/457, it knows that message 457 is in S4. It
  75 removes info/457, local/457, and remote/457 if they exist. Then it reads
  76 through todo/457. It creates info/457, possibly local/457, and possibly
  77 remote/457. When it is done, it removes intd/457. The message is still
  78 in S4 at this point. Finally qmail-send removes todo/457, moving to S5.
  79 At that instant the message has been successfully preprocessed.
  80
  81
  82 5. How preprocessed messages are delivered
  83
  84 Messages at S5 are handled as follows. Each address in local/457 and
  85 remote/457 is marked either NOT DONE or DONE.
  86
  87    DONE: The message was successfully delivered, or the last delivery
  88          attempt met with permanent failure. Either way, qmail-send
  89          should not attempt further delivery to this address.
  90
  91    NOT DONE: If there have been any delivery attempts, they have all
  92              met with temporary failure. Either way, qmail-send should
  93              try delivery in the future.
  94
  95 qmail-send may at its leisure try to deliver a message to a NOT DONE
  96 address. If the message is successfully delivered, qmail-send marks the
  97 address as DONE. If the delivery attempt meets with permanent failure,
  98 qmail-send first appends a note to bounce/457, creating bounce/457 if
  99 necessary; then it marks the address as DONE. Note that bounce/457 is
 100 not crashproof.
 101
 102 qmail-send may handle bounce/457 at any time, as follows: it (1) injects
 103 a new bounce message, created from bounce/457 and mess/457; (2) deletes
 104 bounce/457.
 105
 106 When all addresses in local/457 are DONE, qmail-send deletes local/457.
 107 Same for remote/457.
 108
 109 When local/457 and remote/457 are gone, qmail-send eliminates the
 110 message, as follows. First, if bounce/457 exists, qmail-send handles it
 111 as described above. Once bounce/457 is definitely gone, qmail-send
 112 deletes info/457, moving to S2, and finally mess/457, moving to S1.
 113
 114
 115 6. Cleanups
 116
 117 If the computer crashes while qmail-queue is trying to queue a message,
 118 or while qmail-send is eliminating a message, the message may be left in
 119 state S2 or S3.
 120
 121 When qmail-send sees a message in state S2 or S3---other than one
 122 it is currently eliminating!---where mess/457 is more than 36 hours old,
 123 it deletes intd/457 if that exists, then deletes mess/457. Note that any
 124 qmail-queue handling the message must be dead.
 125
 126 Similarly, when qmail-send sees a file in the pid/ directory that is
 127 more than 36 hours old, it deletes it.
 128
 129 Cleanups are not necessary if the computer crashes while qmail-send is
 130 delivering a message. At worst a message may be delivered twice. (There
 131 is no way for a distributed mail system to eliminate the possibility of
 132 duplication. What if an SMTP connection is broken just before the server
 133 acknowledges successful receipt of the message? The client must assume
 134 the worst and send the message again. Similarly, if the computer crashes
 135 just before qmail-send marks a message as DONE, the new qmail-send must
 136 assume the worst and send the message again. The usual solutions in the
 137 database literature---e.g., keeping log files---amount to saying that
 138 it's the recipient's computer's job to discard duplicate messages.)
 139
 140
 141 7. Further notes
 142
 143 Currently info/457 serves two purposes: first, it records the envelope
 144 sender; second, its modification time is used to decide when a message
 145 has been in the queue too long. In the future info/457 may store more
 146 information. Any non-backwards-compatible changes will be identified by
 147 version numbers.
 148
 149 When qmail-queue has successfully placed a message into the queue, it
 150 pulls a trigger offered by qmail-send. Here is the current triggering
 151 mechanism: lock/trigger is a named pipe. Before scanning todo/,
 152 qmail-send opens lock/trigger O_NDELAY for reading. It then selects for
 153 readability on lock/trigger. qmail-queue pulls the trigger by writing a
 154 byte O_NDELAY to lock/trigger. This makes lock/trigger readable and
 155 wakes up qmail-send. Before scanning todo/ again, qmail-send closes and
 156 reopens lock/trigger.