KoltrixFully automated replies vs AI-suggested drafts in support
Where fully automated support replies are fine, where AI-suggested drafts reviewed by an agent fit better, and a decision table for your own message types.
On this page(8 sections)
- Two approaches, defined
- The five factors that matter
- 1. Cost of a wrong answer
- 2. How predictable the message is
- 3. Customer perception
- 4. Speed
- 5. Agent skill and learning
- The decision table
- What "acceptable automation" looks like
- What a good draft workflow looks like
- A common middle path
- How to decide for your own queue
- Key takeaways
Every support team using AI eventually faces the same fork in the road: let the system answer customers on its own, or let it write drafts that an agent reviews and sends. The right answer isn't one or the other. It depends on the message, and you can decide it message type by message type.
This post lays out how the two approaches differ, which factors should drive the choice, and a decision table you can fill in for your own queue.
Two approaches, defined
Fully automated replies go out without a person seeing them. The system receives an email, decides what it is, writes or selects a response, and sends it. Classic examples are order confirmations and "we got your message" acknowledgments. Newer systems use language models to answer open-ended questions this way too.
AI-suggested drafts are written by the model but held for a person. An agent opens the conversation, sees a proposed reply, edits it if needed, and clicks send. The AI does the typing; the human keeps the judgment.
The difference sounds small. In practice it changes who is accountable, how errors reach customers, and what your agents spend their day doing.
The five factors that matter
1. Cost of a wrong answer
This is the factor that should dominate. Ask what happens if the reply is wrong:
- A wrong acknowledgment ("we'll reply within one business day" when it's two) is mildly annoying.
- A wrong answer about refunds, data deletion, security or pricing can cost money, trust, or create commitments you didn't mean to make.
Language models can produce fluent, confident text that's incorrect. When the cost of being wrong is high, a human review step is cheap insurance.
2. How predictable the message is
Some messages are effectively the same every time: password reset help, "where's my invoice", "how do I add a teammate". Others are unique: a bug that only happens on one account, a complaint that mixes three issues, a question that depends on the customer's contract. Predictable messages tolerate automation better.
3. Customer perception
Customers can usually tell when a reply ignored half of what they wrote. An automated response that misreads an angry message makes things worse. A reviewed draft that addresses every point, even if the AI wrote most of it, reads as attentive.
4. Speed
Automation wins on speed, no contest. But for many questions, a reviewed reply within an hour is better than an instant reply that's wrong. Speed matters most for messages where the customer is blocked and the answer is standard.
5. Agent skill and learning
If AI answers everything, newer agents lose the reps that teach them your product. Reviewing drafts keeps people engaged with real questions, and their edits show you where the AI falls short.
The decision table
Use this to sort your common message types. The right-hand column is a starting recommendation, not a rule.
| Message type | Cost of error | Predictability | Recommended approach |
|---|---|---|---|
| "We received your message" acknowledgment | Low | Very high | Fully automated |
| Order or payment receipt | Low | Very high | Fully automated (system-generated) |
| Password reset instructions | Low | High | Automated link to the help article, or draft |
| How-to question covered by docs | Low to medium | Medium | AI draft, agent reviews |
| Bug report | Medium | Low | AI draft for the acknowledgment and questions, agent investigates |
| Billing dispute or refund request | High | Low | AI draft, agent decides and sends |
| Account access or ownership change | High | Medium | Human only, with identity checks |
| Cancellation | Medium to high | Medium | AI draft, agent reviews (retention and tone matter) |
| Angry or escalated complaint | High | Low | AI draft as a starting point at most; agent writes |
| Legal, security or data request | High | Low | Human only, escalate |
Notice the pattern: full automation fits where the message is system-generated or purely informational. Everything that requires judgment, money, or trust sits on the draft side.
What "acceptable automation" looks like
If you do automate, keep the scope tight:
- Acknowledge, don't answer. An auto-reply that confirms receipt and gives an expected response time is useful. One that tries to solve the problem often isn't.
- Link, don't paraphrase. Pointing to a maintained help article is safer than a model rewriting it each time.
- Make escape obvious. Every automated reply should make it easy to reach a person.
- Audit samples. Read a handful of automated replies each week to catch drift.
What a good draft workflow looks like
AI drafts are only useful if reviewing them is fast and honest. A few habits help:
- Read the customer's message first, then the draft. Otherwise the draft frames how you read the question.
- Check facts, not just tone. Dates, prices, plan limits, and policy details are where models slip.
- Delete freely. A draft is a suggestion. If it's wrong, start over rather than patching.
- Track major rewrites. If agents keep rewriting drafts for one category, the AI shouldn't be drafting there, or it needs better source material.
Koltrix is built on the draft side of this line: its AI drafts replies, but nothing is sent without a person clicking send.
A common middle path
Many teams settle on a hybrid:
- System-generated messages (receipts, confirmations, password resets) are automated.
- An instant acknowledgment goes out for every new conversation, setting expectations.
- Every substantive reply is an AI draft that a person reviews.
This gets customers immediate confirmation, keeps agents fast, and keeps a human between the model and anything that matters.
How to decide for your own queue
Pull your last month of conversations and group them by type. For each group, answer three questions: what does a wrong answer cost, how similar are these messages to each other, and would a customer notice if no person read their email? Put each group in the table above. Revisit after a month of real use, and move categories only when the evidence supports it.
Key takeaways
- Full automation suits low-risk, predictable, often system-generated messages: receipts, acknowledgments, password resets.
- AI-suggested drafts fit anything involving judgment, money, policy, or emotion.
- The cost of a wrong answer should drive the decision more than speed.
- Keep auto-replies narrow: acknowledge and link, and always offer a path to a person.
- Watch where agents rewrite drafts heavily; that's where the AI needs better sources or shouldn't be drafting at all.
Start with Koltrix
Your domain, one inbox, and an API that sends.
A team inbox where AI sorts and drafts (nothing is sent without your click), plus the transactional API and SMTP relay your product sends with. 7 days free, no card.
