KoltrixCatching hallucinated facts in AI email replies
AI drafts can invent features, dates, policies, links and names. Five worked examples of each kind of error, plus a habit that catches them before sending.
On this page(11 sections)
- Why email drafts are prone to this
- Worked example 1: the invented feature
- Worked example 2: the wrong date
- Worked example 3: the made-up policy
- Worked example 4: the fake link
- Worked example 5: the wrong name
- A summary of the error types
- The verification habit
- Reduce hallucinations at the source
- Why the human click matters
- Key takeaways
The dangerous AI draft is not the clumsy one. It's the fluent, friendly reply that confidently tells a customer about a feature you don't have.
Language models generate plausible text. When they lack a fact, they often fill the gap with something that sounds right. In email, that turns into promises, dates, and links that someone will act on. This post walks through the five most common kinds of hallucination in reply drafts, with a worked example of each, and then a verification habit that catches them.
Why email drafts are prone to this
A drafting model typically sees the incoming message, some of the thread, and maybe a style instruction. It usually does not see your pricing page, your refund policy, your roadmap, or your status page unless you provide them. Yet customer emails constantly ask about exactly those things. The model wants to be helpful, so it answers anyway.
The pattern to remember: the more a reply depends on facts outside the thread, the more carefully you should check it.
Worked example 1: the invented feature
Customer: "Can I export my reports as PDF?"
Draft: "Yes! Go to Reports, click the three-dot menu, and choose Export as PDF. You can also schedule weekly PDF exports to your team."
What went wrong: Your product exports CSV only. The model produced a believable menu path because many products have one. The "schedule weekly exports" line is pure embellishment.
How to catch it: Any sentence that describes your product's UI or capabilities is a claim. If you can't picture the exact screen, open it before sending.
Worked example 2: the wrong date
Customer: "When does my annual plan renew?"
Draft: "Your plan renews on March 1, so you have plenty of time to decide."
What went wrong: Nothing in the thread mentioned a renewal date. The model picked one. Dates are especially risky because they look precise, and precision reads as authority.
How to catch it: Every date, amount, and quantity in a draft needs a source you can point to: the billing system, the thread, or a document. If there's no source, delete the number and look it up.
Worked example 3: the made-up policy
Customer: "I forgot to cancel and got charged. Can I get a refund?"
Draft: "Of course. We offer a full refund within 30 days of any charge, no questions asked. I've processed it for you."
What went wrong: Two hallucinations in one sentence. The policy is invented (perhaps your policy is case by case), and the draft claims an action ("I've processed it") that hasn't happened. The second one is worse: the customer will check their bank account.
How to catch it: Watch for two phrases: policy statements ("we always," "our policy is") and completed actions ("I've updated," "I've refunded," "I've escalated"). A draft cannot do things. If the action hasn't actually been taken by a person, the sentence must change.
Worked example 4: the fake link
Customer: "Where can I read about your API rate limits?"
Draft: "You'll find everything at docs.example.com/api/rate-limits, including per-endpoint limits."
What went wrong: The URL follows a sensible pattern, but the page doesn't exist. The customer clicks, gets a 404, and now trusts the rest of your reply less.
How to catch it: Click every link in a draft before sending. Every one. It takes seconds, and broken links are among the most common AI errors because URLs are pattern-shaped text.
Worked example 5: the wrong name
Customer (signing as "Jordan, on behalf of Dana Lee"): "Dana asked me to check on the integration status."
Draft: "Hi Dana, thanks for checking in! ..."
What went wrong: The model addressed the person mentioned rather than the person writing. Names also get mangled in long threads with several participants, and models sometimes "correct" unusual spellings.
How to catch it: Read the greeting and any names against the actual sender line. It's the first thing the customer reads.
A summary of the error types
| Type | Typical tell | Check against |
|---|---|---|
| Invented feature | Step-by-step UI instructions | The product itself |
| Wrong date or number | Precise figures not in the thread | Billing system, records |
| Made-up policy | "We always," "our policy" | Written policy or playbook |
| Claimed action | "I've processed," "I've fixed" | What a person actually did |
| Fake link | Clean-looking URL | Clicking it |
| Wrong name | Greeting, mentions | The From line and signature |
The verification habit
You don't need to fact-check every word. You need to check the claims. A practical routine that takes under a minute for most replies:
- Scan for specifics. Numbers, dates, prices, names, links, product steps.
- Scan for commitments. Anything that promises a future action or claims a past one.
- Source each one or remove it. "Let me confirm the renewal date and get back to you today" is always better than a wrong date.
- Click the links.
- Re-read the greeting.
Teams that review drafts this way find the errors cluster in a handful of sentences. The rest of the draft (structure, tone, acknowledgment) is usually fine, which is where the time saving comes from.
Reduce hallucinations at the source
Checking catches errors; context prevents them. Things that help:
- Give the model your facts. A short reference document with pricing, refund rules, supported features and common links lets the draft quote instead of guess.
- Tell it what not to do. An instruction like "If the answer depends on account data or policy you weren't given, say you'll check rather than guessing" tends to change the shape of drafts.
- Prefer drafts that ask. A draft that says "Could you tell me which workspace this is about?" is less impressive and much safer than one that assumes.
- Keep a log of misses. When a hallucination slips into a draft, note the type. Patterns tell you which facts to add to the reference document.
Why the human click matters
All of this assumes a person reads the draft before it goes out. That's the design choice that makes AI drafting safe enough to use daily. In Koltrix, AI drafts replies but nothing is sent until someone reviews it and clicks send, which is exactly the moment these checks belong.
Key takeaways
- Hallucinations in email drafts concentrate in features, dates, policies, claimed actions, links and names.
- Treat every specific and every commitment as a claim that needs a source.
- Click every link; read every greeting.
- Feed the model a short fact sheet and permission to say "I'll check."
- Keep a human review step before send. It's where hallucinations get caught.
Start with Koltrix
Your domain, one inbox, and an API that sends.
A team inbox where AI sorts and drafts (nothing is sent without your click), plus the transactional API and SMTP relay your product sends with. 7 days free, no card.
