Per-provider throttling: pacing mail to big mailbox providers

example-customer.com.  MX  10 mx.mailhost-a.example.
another-company.com.   MX  10 mx.mailhost-a.example.

Adaptive concurrency#

Static limits are a starting point. A better system adjusts them using the provider's responses, similar to how TCP congestion control works:

class ProviderThrottle:
    def __init__(self, start=4, minimum=1, maximum=40):
        self.limit = start
        self.minimum, self.maximum = minimum, maximum
        self.successes = 0

    def on_accepted(self):
        self.successes += 1
        if self.successes >= self.limit * 20:        # sustained success
            self.limit = min(self.maximum, self.limit + 1)
            self.successes = 0

    def on_deferred(self, code: str, text: str):
        self.successes = 0
        if code.startswith("421") or "rate" in text.lower():
            self.limit = max(self.minimum, self.limit // 2)   # back off sharply

Additive increase, multiplicative decrease: grow slowly while things work, cut quickly when the provider pushes back. Add a cooldown so a single deferral does not oscillate the limit up and down.

Reading the provider's signals#

Not every 4xx means the same thing:

• Rate or connection limits ("too many connections," "try again later," "rate limited"). Reduce concurrency and messages per minute for that provider.
• Reputation-related deferrals (text mentioning unusual volume, user complaints or policy). Reduce volume more aggressively and check what changed in your sending.
• Greylisting (first-attempt deferrals from smaller servers). Retry after the normal delay; no throttle change needed.
• Mailbox-specific (mailbox full, temporarily unavailable). Retry that recipient later; do not throttle the whole provider.

Exact wording differs by provider and changes over time, so match broadly and log the raw response text. Enhanced status codes (RFC 3463), such as 4.7.x for policy-related issues, help with classification where providers include them.

Prioritize within each queue#

When a provider's capacity is limited, decide which mail goes first. A verification code waiting behind a newsletter is a product bug. Use priority levels inside each provider queue:

1. Security mail: verification codes, password resets, sign-in links.
2. Transactional mail: receipts, alerts.
3. Notifications and digests.
4. Marketing.

Better still, send marketing from separate infrastructure or IPs so its throttling never competes with critical mail.

Retry timing#

Throttling decides how fast you send. Retry policy decides when to try again after a deferral. Use exponential backoff with jitter per message, and a give-up window long enough to ride out provider incidents. RFC 5321 suggests that senders keep retrying for at least four to five days before giving up, though for time-sensitive mail like verification codes a much shorter useful lifetime often makes more sense; an hour-old code is rarely worth delivering.

What to monitor#

• Deferral rate per provider group.
• Current concurrency limit per provider, graphed over time.
• Queue depth and oldest-message age per provider.
• Delivery latency per priority level.

A provider whose concurrency limit keeps getting cut is telling you something about your reputation there, not just about capacity.

Checklist#

• Group destinations by receiving infrastructure (MX), not by recipient domain.
• Maintain a queue per provider group with its own concurrency and rate limits.
• Adapt limits: increase slowly on success, halve on rate-limit deferrals.
• Classify 4xx responses before reacting; log raw text.
• Prioritize security and transactional mail within each queue.
• Monitor per-provider deferrals, limits, queue age and latency.

Key takeaways#

• Each mailbox provider sets its own acceptance limits, and they depend on your reputation.
• Per-provider queues keep a slowdown at one receiver from delaying everyone else.
• Adaptive concurrency, grow slowly and cut quickly, uses deferrals as a control signal.
• Classify deferrals: rate limits, reputation, greylisting and mailbox issues need different responses.
• Put critical mail first, ideally on infrastructure separate from marketing.