Upstream qmail 1.01

[qmail] / RFCVERP

Variable Envelope Return Paths
D. J. Bernstein, djb@pobox.com
19970201


1. Introduction

   The fundamental problem in managing a large mailing list is matching
   bounce messages to subscription addresses.

   Often a bounce message refers to a failing address that does not
   appear on the mailing list. One of the mailing list subscribers is
   forwarding messages to that address. Which subscriber? As the list
   grows, this question becomes more and more difficult to answer.

   Sometimes a bounce message doesn't identify the address that failed.
   On occasion it doesn't even include a copy of the original message.
   See RFC 1211 for an extensive collection of horror stories.

   In theory, one could solve this problem with the DSN option and DSN
   format described in RFC 1891, RFC 1892, and RFC 1894. Unfortunately,
   the DSN option is useless unless it is supported by every
   intermediate MTA. The complexity of RFC 1891 means that it will be
   many years, perhaps infinitely many, before DSNs are universally
   supported. Furthermore, the complexity of RFC 1894 means that parsing
   the subscriber address is difficult even on the occasions that the
   address is available.

   Variable envelope return paths (VERPs) completely eliminate this
   problem _right now_. They automatically and reliably identify the
   subscription address relevant to each bounce message. They provide
   the address in a form that is trivial for automated bounce handlers
   to parse. They require support from the local mailer, but they do not
   require support from any other hosts.


2. Variable envelope return paths

   Here is how VERPs work: each recipient of the message sees a
   different envelope sender address. When a message to the
   djb-sos@silverton.berkeley.edu mailing list is sent to
   God@heaven.af.mil, for example, it has the following envelope sender:

      djb-sos-owner-God=heaven.af.mil@silverton.berkeley.edu

   If the message bounces, the bounce message will be sent back to
   djb-sos-owner-God=heaven.af.mil@silverton.berkeley.edu.

   If God is forwarding His mail, the bounce message will still go to
   djb-sos-owner-God=heaven.af.mil@silverton.berkeley.edu. No matter how
   uninformative the bounce message is, it will display God's
   subscription address in its envelope.

   Another benefit of VERPs is that God Himself can see what address He
   used to subscribe.

   Making VERPs work requires two pieces of local software support.
   First: it must be easy to modify the outgoing sender address
   separately for each envelope recipient. For example, with one mailer,
   qmail, a user can simply touch ~/.qmail-list-owner and
   ~/.qmail-list-owner-default to apply VERPs to user-list.

   Second, and more important: it must be easy to identify a collection
   of addresses, such as djb-sos-owner-*, and send all mail for those
   addresses to one place, while preserving the * information. Under
   qmail, all user-list-owner-* mail will be sent to the user once he
   touches ~/.qmail-list-owner-default. Sending the mail through an
   automated bounce-handling program is just as easy.

   With older mailers, applying VERPs would require setting up a new
   user-list-owner-recipient alias for each new recipient. This
   inconvenience has prevented VERPs from being widely exploited, even
   though the idea is not new.


3. Per-message VERPs

   VERPs are not restricted to distinguishing mailing list subscribers;
   they can also be used to distinguish messages.

   For example, a user can send one message with an envelope sender
   address of user-dsn-1, the next message with user-dsn-2, and so on.
   As long as the local mailer gives all user-dsn-* back to that user,
   he can reliably match up incoming bounces with outgoing messages.

   Per-message VERPs can be combined with per-recipient VERPs. Every
   application of RFC 1891's ORCPT and ENVID can be handled with
   VERPs---easily, reliably, and right now.