A model drafts a reply to a tender query for a systems integrator in Pune. The draft says delivery in six weeks; the scope document says eight. The paragraph reads so well that the reviewer approves it in under a minute, and the wrong date goes out under the company's name. Whether a draft-generation feature works in production depends far less on the generation step than on the review step around it.
Where drafts save time and where they move it
Generated drafts pay off when the task repeats, the input already sits in your systems, and a person would have written something similar anyway. Proposals that reuse the same sections with a different scope. Summaries of long documents readers were going to skim. Replies to a ticket category the team has answered a hundred times, like "where is my refund" after a marketplace return.
They do not pay off when the reviewer has to re-read the whole source to check the draft. If verifying a summary of a forty-page vendor contract takes as long as writing one, the work has moved from one person to another and nothing was saved. So the test before building is practical. How long will review take, and what does the reviewer need on screen to do it quickly?
Store each draft as a record with a status
A draft should be a row in your own database, not text that appears in a text box and disappears. The row holds the source inputs it was built from (ticket id, document version, the CRM fields), the prompt and model version, the generated text, the reviewer's edits, and the text that finally went out. With that row you can answer, six months later, what was generated, what was changed and who approved it. Prompt versions belong in the same history; we keep them in Git like any other code.
Status moves in one direction: generated, in review, approved or rejected, sent. Nothing leaves the system in the generated state. When the reviewer edits a draft, the edited version is what goes out, and the difference between generated and sent text is kept.
- 01Source recordticket, contract, CRM
- 02Generateprompt version stored
- 03Reviewsource beside draft
- 04Sendapproved text only
- 05Keep the diffgenerated versus sent
Heavily edited drafts become test cases
The send step reads the approved text, never the generated one.
What the reviewer needs on screen
Three things have to be visible without clicking away. First, the source, beside the draft: the ticket, the contract clause or the record fields it came from, so the reviewer checks the text against the material and not against memory.
The diff, whenever a draft is regenerated or edited. People read a draft carefully once. They will not read it carefully a fourth time, so change marks are the only way to keep later passes honest.
The claims. Numbers, dates, names and commitments should be marked so the reviewer checks those first. One way is to ask the model to return the facts it used alongside the draft, as structured output you can check in code, then match each fact against the source record. If the draft says six weeks and the CRM says eight, that line is marked before anyone reads the prose.
Show how much attention a draft needs, but derive it from those checks, never from the model's own estimate of how sure it is. How many claims matched the source. Whether the draft stayed inside the expected length and sections. Whether it contains commitment words such as refund, guarantee, penalty or deadline. A draft with two unmatched claims is flagged for a closer look. A draft with none is routine.
Before release, run the claim matcher against drafts where you already know the answer:
- A draft that repeats the source's delivery date exactly. Expected: no mark.
- A draft that writes "8 weeks" where the source says "eight weeks". Expected: no mark. If it is marked, fix the normaliser before reviewers learn to ignore marks.
- A draft that promises a penalty waiver the contract does not mention. Expected: marked as a commitment with no source.
- A draft that omits the delivery date entirely. Expected: flagged as a missing required fact.
How review steps fail
Approve-everything review. The queue is long, the drafts look fine, and the reviewer clears all of them inside a minute. A review step nobody ever fails is decoration. Sample approved drafts afterwards. If errors are getting through, change the screen; the reviewer is responding to what the screen asks of them.
Review in the wrong tool. Draft in one window, source in another, and the reviewer stops comparing after the first day.
Regenerating until it looks right. Every regeneration is a new draft that resets the review. Fix the problem in the edit, and keep the edit.
Commitments made by the draft. A reply that promises a refund the policy does not allow. Commitments come from rules or from a person, never from generated text.
No record of what was sent. When the customer forwards the email to your managing director, nobody can trace it from source to draft to final text.
When a class of drafts can skip review
Review is dropped for a class of output, never for the whole feature. All four conditions have to hold.
- The reader is internal, or external and told the text was generated, with a way to reach a person.
- An error is cheap to correct afterwards. An internal meeting summary can be regenerated; a quote, a legal statement or a dosage instruction cannot be taken back.
- The output passes every automated check: format, length, every claim matched to the source, no commitment words.
- Stored diffs show reviewers have changed very little in this class of draft over a sustained period.
- Goes out as your statement
- Contains a price, date or promise
- Cannot be undone after sending
- Checks found unmatched claims
- Internal, or labelled as generated
- Cheap to correct afterwards
- All automated checks pass
- Few edits over time
Even then, keep sampling. A small share of unreviewed drafts still goes to a person, and the edit rate in that sample tells you whether to widen or narrow the unreviewed class.
Starting with one draft type
- Pick one draft type and collect twenty real source records with the text people actually sent for them.
- Build the review screen before tuning the prompt: source, draft, marked claims, diff.
- Store every draft as a record with a status from the first day.
- Measure how much reviewers change each draft. That edit rate is your quality measure, and heavily edited drafts go into the test set.
- After a few weeks of stored diffs, decide which class, if any, can move to sampled review.
When we scope generative AI features, we start from the same place. The review screen comes first, because the prompt can improve every week afterwards and a review step added after launch is the one people learn to click through.
