A typical first prompt at a logistics firm reads "Extract the shipment details from this email", followed by a pasted customer booking. The model returns a tidy summary, and everyone is pleased. On the next email it returns the sender's address instead of the consignee's, and on the one after that it reports the weight in pounds. Prompt engineering is the practice of writing instructions that leave less room for that kind of variation, and then checking that they hold on the hundredth input as well as the first.
What a prompt has to specify
A language model produces the most likely continuation of the text it is given. When the instruction leaves something open, such as which address, which unit or what format, the output fills the gap with whatever is most plausible for that input. Plausible changes from one email to the next, which is why the first prompt worked once and then drifted.
A useful prompt closes those gaps. It says what the task is, what the input is, what the output must look like, and what to do when the input does not contain what is asked for. That last part is the one most first drafts leave out, and it is where most of the strange outputs come from.
Be specific about length, audience and format. "Write about our new pricing" leaves all three open. "Write a 150-word note to existing customers explaining that delivery charges change on 1 July, in plain language, with the new rate table at the end" closes them.
Give context the model cannot have. It does not know your rate card, your customer list or your naming conventions unless they are in the request. Asking it to "be accurate" about a rate it has never seen cannot work; putting the rate card in the request can.
A worked example with booking emails
Here is the logistics prompt rewritten so the output can be checked by a program:
"From the booking email below, extract the consignee (the person or company receiving the goods, not the sender), the destination pin code, and the total weight in kilograms. If the weight is given in another unit, convert it. If a field is missing, return null for it. Return only JSON in this shape."
{
"consignee_name": "Sharma Textiles",
"destination_pin": "302001",
"weight_kg": 42.5,
"missing_fields": []
}
Every rule in that prompt answers a failure seen on a real email: the sender confused with the consignee, weights in pounds, a pin code that was not in the email at all. The JSON shape means the dispatch system can reject a malformed reply before anything downstream reads it.
Examples usually help more than description. Add two real booking emails with their correct JSON beneath the instruction, then the new email. Pick examples that differ from each other, say one formal email and one written half in Hindi with the pin code in the middle of a sentence, so the model is not simply copying one pattern. Once the output is structured, getting JSON you can trust covers validation and the cases where extraction still fails.
Common prompt structures
| Structure | Use it for | Example |
|---|---|---|
| Instruction, context, output format | One-off explanations | Explain GST input credit to a shop owner in under 200 words |
| Role, task, constraints | Reviews and critiques | You are a UX reviewer. List three problems with this checkout page |
| Few-shot examples | Consistent format or style | Here are two converted emails; convert the third |
A role line such as "You are an HR manager" shifts vocabulary and emphasis. It does not add knowledge. If the answer depends on your leave policy, the policy has to be in the request.
Test a prompt against saved inputs
A prompt that reads well can still fail on inputs you did not try. The reliable way to improve one is to keep a fixed set of inputs, change one thing, and compare the outputs before and after.
- 01Change promptone edit
- 02Run saved inputs
- 03Compare outputswith the last version
- 04Keep or revert
Add failures to the set
Each failure you find becomes a new saved example.
Judge outputs against written criteria: the right fields present, the right length, no invented details. For the booking emails, the check is whether the extracted pin code matches the one in the email, every time, including the emails where it appears mid-sentence. A program can run that check on the evaluation set in seconds, which means every prompt edit can be tested the same day it is made.
How prompt work goes wrong
The classic mistake is tuning on a single example. The prompt works on the email in front of you and fails on the rest. Close behind is changing three things at once, so nobody knows which change helped and which one broke the weight conversion.
The working prompt often lives in someone's chat window with no history and no owner. When it changes, nobody can say what last month's version was. Keeping prompt versions in git alongside the saved inputs solves that.
People also expect identical output for identical input. Responses vary between runs, so the application has to cope, for example by validating the format and retrying or flagging a malformed reply instead of passing it on.
When a prompt stops being enough
Prompts are the starting point. As a use case grows, teams usually add one of these layers, roughly in this order.
- Agentscall APIs and take actions, with limits
- Fine-tuned modeladapted to your data and format
- Retrievalyour documents passed in per request
- Prompt templatesfixed wording, variable inputs
Templates come first: a stored prompt with placeholders for the variable parts, kept in version control. Retrieval is next, where relevant passages from your own documents are added to each request so the answer is based on your material; what a RAG pipeline does explains the steps. Fine-tuning trains the model further on examples of the output you need, and is worth considering only when prompts and examples cannot hold the format. Agents go furthest: a model that can call your APIs to complete a task. Anything with side effects then needs permissions and review, because "cancel the booking" is an action, not a draft.
Ten inputs before the first edit
- Pick one task and collect ten real inputs, including awkward ones.
- Write down what a correct output looks like for each.
- Write the prompt, run all ten, and note the failures.
- Change one thing at a time and rerun all ten.
- Store the final prompt and the ten inputs in your repository.
The ten inputs are the step most teams skip, and they are what make every later change safe to ship.
