Skip to content

How Koltrix makes send retries safe

Why the Koltrix send API claims an Idempotency-Key before queuing anything, what each response code means and how to retry from your client.

Koltrix Team5 min read
Interlocking metal cogs and gears
Photo by Tim Mossholder on Unsplash
On this page(9 sections)
  1. The situation retries are for
  2. The bug we designed around
  3. What each response means
  4. 409 means "still working," not "failed"
  5. 503 means we chose not to send
  6. Failed requests do not lock the key
  7. Keys are per account
  8. How to choose keys
  9. What the key does not do
  10. A retry loop that uses all four cases
  11. Key takeaways

A retry that sends a customer the same receipt twice is a small embarrassment. A retry that sends a password reset twice, or a payment confirmation twice, erodes trust.

The Koltrix send API is built so that retrying a request with the same Idempotency-Key never produces a second message, including in the awkward case where the retry arrives while the first request is still running.

The situation retries are for

Your backend calls POST https://api.koltrix.com/api/v2/emails. The request reaches us, the message is queued, and then the network drops before the response reaches you. From your side it looks like a timeout. You do not know whether the email was queued.

The correct move is to retry. The question is how to make retrying safe. That is the job of the Idempotency-Key header:

curl https://api.koltrix.com/api/v2/emails \
  -H "Authorization: Bearer kx_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 9b2e6c1a-6f0d-4a8e-9d43-3c1f0a7e5b21" \
  -d '{
    "from": "[email protected]",
    "to": ["[email protected]"],
    "subject": "Your receipt",
    "body_html": "<p>Thanks for your payment.</p>",
    "body_text": "Thanks for your payment."
  }'

The key is any unique string you choose per logical send. Our documentation calls it strongly recommended, and we would go further: for any message where a duplicate matters, always send one.

The bug we designed around

The obvious way to implement idempotency is: when a request arrives, look up the key; if there is a stored response, return it; otherwise do the work and store the response at the end.

That has a gap. Doing the work takes time, because it includes recording each recipient and queuing the send. A retry that arrives during that window finds nothing stored yet, so it does the work too, and the recipient gets two messages. That window is exactly where a client retrying after a timeout tends to land, which makes it the case the header exists for.

So the Koltrix send endpoint claims the key atomically before doing any work. The first request to arrive with a given key reserves it with an "in progress" marker. Only then does it record recipients and queue the message. When it finishes, it replaces the marker with the response, which is kept for 24 hours.

What each response means

Because the key is claimed up front, every request with a given key falls into one of four cases, and the API tells you which:

Situation Response What you should do
First request with this key 202 Accepted with the queued message Done
Retry after the first request finished 200 with the original response body and the header Idempotent-Replayed: true Done; no second message was sent
Retry while the first request is still running 409 Conflict Wait briefly and retry with the same key
The idempotency store is unreachable 503 Service Unavailable Nothing was sent; retry with the same key

Two of these deserve a closer look.

409 means "still working," not "failed"

A 409 tells you another request with the same key is in flight right now. It is not an error to report to the user. Back off for a moment and retry with the same key. Once the original finishes, your retry receives the replayed response with Idempotent-Replayed: true.

503 means we chose not to send

If the store that tracks keys cannot be reached, we cannot tell whether this key has been used. Rather than risk a duplicate, the API refuses with a 503 and sends nothing. Sending twice is worse than not sending, and the request is safe to retry with the same key once the store is back.

Failed requests do not lock the key

If a request fails before anything is queued, for example because the from address is not registered for the account or the body is invalid, the claim is released. You can fix the problem and retry with the same key immediately instead of being locked out for 24 hours by an attempt that sent nothing.

Keys are per account

Cached responses are stored per account and key, so two different Koltrix accounts using the same key string do not collide. Within your account, reusing a key for a different message will return the first message's response, so treat keys as identifiers for one logical send.

How to choose keys

Generate the key once, when you decide the email should exist, and reuse it for every attempt:

  • A UUID stored with the record that triggered the email. For example, store receipt_email_key on the payment row when the payment succeeds, then use it for every retry of that email.
  • A deterministic value derived from the business event, such as receipt- followed by the payment ID. This works even if the process that generated the key crashed before saving it.

The one thing not to do is generate a new key inside the retry loop. Every attempt would then look like a new message, and the protection disappears.

What the key does not do

Idempotency keys protect against duplicate sends caused by retries. They are not a general deduplication system, and it helps to be clear about the edges:

  • They do not compare message content. If your application calls the API twice for the same event with two different keys, perhaps because two workers both decided to send the receipt, both messages go out. Preventing that is your application's job, usually with a unique constraint on "receipt email for payment X" in your own database.
  • They expire. After 24 hours a key can be used again. Retries should happen within minutes, not days, so this rarely matters, but a job that replays day-old work should not rely on old keys.
  • They only apply when you send one. A request without the header is processed as a new send every time.

A retry loop that uses all four cases

import time, uuid, requests

def send_receipt(payment, session):
    key = payment.receipt_email_key or f"receipt-{payment.id}"
    body = {
        "from": "[email protected]",
        "to": [payment.customer_email],
        "subject": "Your receipt",
        "body_text": f"We received your payment of {payment.amount_display}.",
    }
    delay = 1
    for attempt in range(6):
        try:
            r = session.post(
                "https://api.koltrix.com/api/v2/emails",
                json=body,
                headers={"Authorization": f"Bearer {KOLTRIX_KEY}", "Idempotency-Key": key},
                timeout=10,
            )
        except requests.RequestException:
            r = None
        if r is not None and r.status_code in (200, 202):
            return r.json()                    # sent, or replayed
        if r is not None and r.status_code == 429 and r.json().get("code") == "quota_exceeded":
            raise RuntimeError(f"quota exceeded until {r.json().get('resets_at')}")  # retrying won't help
        if r is not None and r.status_code < 500 and r.status_code not in (409, 429):
            raise ValueError(r.text)           # a real client error: fix the request
        time.sleep(delay)                      # timeout, 409, 429, 503 or other 5xx
        delay = min(delay * 2, 30)
    raise RuntimeError("receipt not confirmed; safe to retry later with the same key")

Key takeaways

  • Send an Idempotency-Key on every send where a duplicate would matter.
  • Koltrix claims the key before queuing anything, so a retry that overlaps the original cannot send a second copy.
  • 202 is a new send, 200 with Idempotent-Replayed: true is a replay, 409 means wait and retry, and 503 means nothing was sent.
  • Failed requests release the key; successful responses are replayable for 24 hours.
  • Generate the key once per logical message, never inside the retry loop.

Start with Koltrix

Your domain, one inbox, and an API that sends.

A team inbox where AI sorts and drafts (nothing is sent without your click), plus the transactional API and SMTP relay your product sends with. 7 days free, no card.

SharePost on XLinkedIn