Fold expiry-time tracking into append_addrs.