KoltrixParsing inbound email in your app: MIME, attachments and spoofing
Reading incoming mail in your own code goes wrong in predictable ways. Handle MIME trees, encodings, quoted replies, attachments and forged senders safely.
On this page(10 sections)
Sending email from an application is a well-trodden path. Receiving it is less so, and it hides more surprises. The moment your app lets people reply to a notification, forward a receipt to an address, or email a ticket system, you are running a small mail parser in front of the open internet.
This guide covers what to expect from raw inbound messages and how to handle them without losing content or opening a hole. It assumes you receive messages through a webhook, a mailbox you poll or a raw message feed, and that you want your code to be dependable on the thousandth strange message, not just the first clean one.
Start by not writing your own parser
Email is not "headers, blank line, body". It is a family of nested formats with decades of exceptions. Use a mature MIME library for your language, feed it the raw message and work with the structure it returns. Hand-written regular expressions fail on the first folded header or the first message with an unusual charset.
Keep the raw message too. Store the original bytes (or at least the headers and the full source) for a limited time so that when a parse goes wrong you can reproduce it. Without the raw source, every bug report starts with "I cannot see what we received".
The shape of a message
A message is a tree. The leaves are parts that hold content, and the branches are multipart containers that say how their children relate. MIME for developers explains the three that matter: alternative (the same content in several formats), related (a document with inline resources) and mixed (independent attachments).
Three habits make a parser robust:
- Walk the tree, do not assume a position. A body is not always "part 1". Pick content by type and by whether the part is an attachment or inline.
- Prefer plain text for logic, HTML for display. If you need to read what a person wrote, take
text/plainwhen it exists. If only HTML exists, convert it to text with a proper library, never by stripping tags with a pattern. - Expect missing parts. Some messages have no text part, some no HTML, some neither (an attachment-only forward). Your code must have a defined result for each.
Encodings that bite
Charsets. The header or part may declare ISO-8859-1, Windows-1252, Shift_JIS and many others, and sometimes declares one and uses another. Decode with the declared charset, fall back to a detection step, and replace what cannot be decoded rather than throwing an error. Store everything as UTF-8 afterwards.
Transfer encodings. Bodies arrive as base64, quoted-printable, 7bit or 8bit. A library handles this, but if you ever see =E2=80=99 or a wall of base64 in a stored message, decoding was skipped.
Encoded headers. Subjects and display names can look like =?UTF-8?B?...?=. These must be decoded before display and before any comparison.
Getting the new text out of a reply
When a customer replies, the body holds their new text and then a quote of everything before it. If you store the whole thing as the comment on a ticket, the same history is repeated at every step.
There is no perfect rule, but these steps work well together:
- Use the thread headers to find the conversation.
In-Reply-ToandReferencesidentify the original message. Message-ID and threading headers shows how they chain. Do not rely on the subject line alone. - Trim the quote with markers you control. If your outgoing mail contains a visible line such as "Reply above this line", cut at it when it is present.
- Fall back to common quote patterns. An attribution line like "On Tue, Sep 9, Maya wrote:" followed by lines starting with
>, or an HTMLblockquote, marks the start of quoted text. - Keep the full body anyway. Store the trimmed text for display and the original for audit. Quote detection will be wrong sometimes, and you need a way to see everything.
Attachments: treat every one as hostile
An attachment is a file from a stranger. The following rules apply whatever the message looks like.
- Limit size and count. Set a maximum for each file and for the total message, and decide in advance what happens above it: reject with a clear error or accept the text and drop the file with a note.
- Never trust the filename. Strip directory parts (
../../x), control characters and anything that would be unsafe on your storage. Better still, generate your own storage name and keep the original only as a display label. - Never trust the declared type. A file named
invoice.pdfwith the typeapplication/pdfmay be neither. Detect the type from the content before you hand it to a viewer or a converter. - Store files away from your application. Use object storage without execute permission, and serve downloads with a
Content-Disposition: attachmentheader so a browser does not render them in your origin. - Scan if the files will be opened by people. Run an antivirus or file-scanning step before a download link appears. If you do not scan, say so in your risk notes.
- Be careful with archives and Office files. Archives can expand to enormous sizes, and documents can carry macros. Limit what you unpack and do not open them automatically.
If you pass files to a language model or a document converter, the same rules apply and the content is also untrusted instructions. What to do with PDFs in AI email features covers that side. For the sending side of attachments, see transactional attachments.
Who actually sent this?
The From header is text the sender typed. Anybody can put any address there, and many people do. If your app takes action based on who a message claims to be from (adding a comment to an account, approving a request, resetting something), you need evidence beyond the header.
Check these, in this order of usefulness:
- Authentication results. The receiving mail server records whether SPF, DKIM and DMARC passed in an
Authentication-Resultsheader. Trust this only if your own server or provider added it. Anything claiming the same header from earlier hops can be forged, so read the one from your own boundary and ignore the rest. SPF, DKIM and DMARC in plain English explains what each result means. - Alignment with the claimed address. A passing DKIM signature for
attacker.examplesays nothing about aFromof[email protected]. The signing domain must match the visible domain. - A secret in the conversation. For replies to your own notifications, put an unguessable token in the reply address or the message. A reply that carries a valid token proves the writer received your message. A token alone is not a login, so keep it single-purpose and expire it.
- Display name tricks. The name shown in an inbox is separate from the address, and people copy trusted names. Display name spoofing shows the pattern, and email spoofing explained covers the wider picture.
When authentication fails, do not drop the message silently. Accept it, mark it as unverified, and keep it away from any automated action. A person can then decide.
Loops, bounces and automatic mail
Not every inbound message is from a person. Your parser will receive:
- Delivery reports (
multipart/report) for mail you sent. - Auto-replies such as out-of-office notices, which carry headers like
Auto-Submitted. Never answer an auto-reply with an automatic message of your own, or two systems can loop forever. - Mailing-list traffic and newsletters sent to an address that was subscribed by mistake.
Detect these early, route delivery reports to your bounce pipeline and record the rest without triggering workflows. See handling out-of-office replies in your sending system.
Idempotence and limits
Inbound feeds retry. A webhook that times out gets sent again, and a mailbox poll can see the same message twice. Use the Message-ID together with your own delivery identifier as an idempotency key, and make processing safe to repeat. Verify the signature on every webhook before you read the payload: webhook signature verification walks through it.
Set resource limits on the parsing step itself: a maximum message size, a maximum nesting depth for multipart trees, a time limit per message and a memory limit for the worker. A crafted message with thousands of nested parts should cost you a rejected message, not an outage.
A test set worth building
Collect real messages (with permission and personal data removed) and keep them as fixtures:
- A plain text message and an HTML-only message
- A reply with a long quoted history, in at least two clients
- Messages in non-UTF-8 charsets and with encoded subjects
- An attachment with an odd filename, a very large file and a zero-byte file
- A message that fails SPF or DKIM
- An auto-reply and a delivery report
Run every parser change against this set. The first time one fixture breaks a release, the set has paid for itself.
Key takeaways
- Use a real MIME library, walk the tree and keep the raw message for debugging.
- Decode charsets and encoded headers explicitly, and define what happens when a part is missing.
- Treat attachments as hostile: limit size, rename, detect the real type, store them away from your app and scan before people open them.
- Do not trust the
Fromheader. Read authentication results from your own boundary, check alignment and use tokens for replies to your messages. - Make processing idempotent, set limits on parsing and keep a fixture set of strange real messages.
Start with Koltrix
Your domain, one inbox, and an API that sends.
A team inbox where AI sorts and drafts (nothing is sent without your click), plus the transactional API and SMTP relay your product sends with. 7 days free, no card.
