Commit | Line | Data |
---|---|---|
2117e02e MW |
1 | 1. Overview |
2 | ||
3 | Here's the data flow in the qmail suite: | |
4 | ||
5 | qmail-smtpd --- qmail-queue --- qmail-send --- qmail-rspawn --- qmail-remote | |
6 | / | \ | |
7 | qmail-inject _/ qmail-clean \_ qmail-lspawn --- qmail-local | |
8 | ||
9 | Every message is added to a central queue directory by qmail-queue. | |
10 | qmail-queue is invoked as needed, usually by qmail-inject for locally | |
11 | generated messages, qmail-smtpd for messages received through SMTP, | |
12 | qmail-local for forwarded messages, or qmail-send for bounce messages. | |
13 | ||
14 | Every message is then delivered by qmail-send, in cooperation with | |
15 | qmail-lspawn and qmail-rspawn, and cleaned up by qmail-clean. These four | |
16 | programs are long-running daemons. | |
17 | ||
18 | The queue is designed to be crashproof, provided that the underlying | |
19 | filesystem is crashproof. All cleanups are handled by qmail-send and | |
20 | qmail-clean without human intervention. See section 6 for more details. | |
21 | ||
22 | ||
23 | 2. Queue structure | |
24 | ||
25 | Each message in the queue is identified by a unique number, let's say | |
26 | 457. The queue is organized into several directories, each of which may | |
27 | contain files related to message 457: | |
28 | ||
29 | mess/457: the message | |
30 | todo/457: the envelope: where the message came from, where it's going | |
31 | intd/457: the envelope, under construction by qmail-queue | |
32 | info/457: the envelope sender address, after preprocessing | |
33 | local/457: local envelope recipient addresses, after preprocessing | |
34 | remote/457: remote envelope recipient addresses, after preprocessing | |
35 | bounce/457: permanent delivery errors | |
36 | ||
37 | Here are all possible states for a message. + means a file exists; - | |
38 | means it does not exist; ? means it may or may not exist. | |
39 | ||
40 | S1. -mess -intd -todo -info -local -remote -bounce | |
41 | S2. +mess -intd -todo -info -local -remote -bounce | |
42 | S3. +mess +intd -todo -info -local -remote -bounce | |
43 | S4. +mess ?intd +todo ?info ?local ?remote -bounce (queued) | |
44 | S5. +mess -intd -todo +info ?local ?remote ?bounce (preprocessed) | |
45 | ||
46 | Guarantee: If mess/457 exists, it has inode number 457. | |
47 | ||
48 | ||
49 | 3. How messages enter the queue | |
50 | ||
51 | To add a message to the queue, qmail-queue first creates a file in a | |
52 | separate directory, pid/, with a unique name. The filesystem assigns | |
53 | that file a unique inode number. qmail-queue looks at that number, say | |
54 | 457. By the guarantee above, message 457 must be in state S1. | |
55 | ||
56 | qmail-queue renames pid/whatever as mess/457, moving to S2. It writes | |
57 | the message to mess/457. It then creates intd/457, moving to S3, and | |
58 | writes the envelope information to intd/457. | |
59 | ||
60 | Finally qmail-queue creates a new link, todo/457, for intd/457, moving | |
61 | to S4. At that instant the message has been successfully queued, and | |
62 | qmail-queue leaves it for further handling by qmail-send. | |
63 | ||
64 | qmail-queue starts a 24-hour timer before touching any files, and | |
65 | commits suicide if the timer expires. | |
66 | ||
67 | ||
68 | 4. How queued messages are preprocessed | |
69 | ||
70 | Once a message has been queued, qmail-send must decide which recipients | |
71 | are local and which recipients are remote. It may also rewrite some | |
72 | recipient addresses. | |
73 | ||
74 | When qmail-send notices todo/457, it knows that message 457 is in S4. It | |
75 | removes info/457, local/457, and remote/457 if they exist. Then it reads | |
76 | through todo/457. It creates info/457, possibly local/457, and possibly | |
77 | remote/457. When it is done, it removes intd/457. The message is still | |
78 | in S4 at this point. Finally qmail-send removes todo/457, moving to S5. | |
79 | At that instant the message has been successfully preprocessed. | |
80 | ||
81 | ||
82 | 5. How preprocessed messages are delivered | |
83 | ||
84 | Messages at S5 are handled as follows. Each address in local/457 and | |
85 | remote/457 is marked either NOT DONE or DONE. | |
86 | ||
87 | DONE: The message was successfully delivered, or the last delivery | |
88 | attempt met with permanent failure. Either way, qmail-send | |
89 | should not attempt further delivery to this address. | |
90 | ||
91 | NOT DONE: If there have been any delivery attempts, they have all | |
92 | met with temporary failure. Either way, qmail-send should | |
93 | try delivery in the future. | |
94 | ||
95 | qmail-send may at its leisure try to deliver a message to a NOT DONE | |
96 | address. If the message is successfully delivered, qmail-send marks the | |
97 | address as DONE. If the delivery attempt meets with permanent failure, | |
98 | qmail-send first appends a note to bounce/457, creating bounce/457 if | |
212b6f5d MW |
99 | necessary; then it marks the address as DONE. Note that bounce/457 is |
100 | not crashproof. | |
2117e02e MW |
101 | |
102 | qmail-send may handle bounce/457 at any time, as follows: it (1) injects | |
103 | a new bounce message, created from bounce/457 and mess/457; (2) deletes | |
104 | bounce/457. | |
105 | ||
106 | When all addresses in local/457 are DONE, qmail-send deletes local/457. | |
107 | Same for remote/457. | |
108 | ||
109 | When local/457 and remote/457 are gone, qmail-send eliminates the | |
110 | message, as follows. First, if bounce/457 exists, qmail-send handles it | |
111 | as described above. Once bounce/457 is definitely gone, qmail-send | |
112 | deletes info/457, moving to S2, and finally mess/457, moving to S1. | |
113 | ||
114 | ||
115 | 6. Cleanups | |
116 | ||
117 | If the computer crashes while qmail-queue is trying to queue a message, | |
118 | or while qmail-send is eliminating a message, the message may be left in | |
119 | state S2 or S3. | |
120 | ||
121 | When qmail-send sees a message in state S2 or S3---other than one | |
122 | it is currently eliminating!---where mess/457 is more than 36 hours old, | |
123 | it deletes intd/457 if that exists, then deletes mess/457. Note that any | |
124 | qmail-queue handling the message must be dead. | |
125 | ||
126 | Similarly, when qmail-send sees a file in the pid/ directory that is | |
127 | more than 36 hours old, it deletes it. | |
128 | ||
129 | Cleanups are not necessary if the computer crashes while qmail-send is | |
130 | delivering a message. At worst a message may be delivered twice. (There | |
131 | is no way for a distributed mail system to eliminate the possibility of | |
132 | duplication. What if an SMTP connection is broken just before the server | |
133 | acknowledges successful receipt of the message? The client must assume | |
134 | the worst and send the message again. Similarly, if the computer crashes | |
135 | just before qmail-send marks a message as DONE, the new qmail-send must | |
136 | assume the worst and send the message again. The usual solutions in the | |
137 | database literature---e.g., keeping log files---amount to saying that | |
138 | it's the recipient's computer's job to discard duplicate messages.) | |
139 | ||
140 | ||
141 | 7. Further notes | |
142 | ||
143 | Currently info/457 serves two purposes: first, it records the envelope | |
144 | sender; second, its modification time is used to decide when a message | |
145 | has been in the queue too long. In the future info/457 may store more | |
146 | information. Any non-backwards-compatible changes will be identified by | |
147 | version numbers. | |
148 | ||
149 | When qmail-queue has successfully placed a message into the queue, it | |
150 | pulls a trigger offered by qmail-send. Here is the current triggering | |
151 | mechanism: lock/trigger is a named pipe. Before scanning todo/, | |
152 | qmail-send opens lock/trigger O_NDELAY for reading. It then selects for | |
153 | readability on lock/trigger. qmail-queue pulls the trigger by writing a | |
154 | byte O_NDELAY to lock/trigger. This makes lock/trigger readable and | |
155 | wakes up qmail-send. Before scanning todo/ again, qmail-send closes and | |
156 | reopens lock/trigger. |