A logistics firm covering a few thousand pin codes has one model call working well: it reads a customer's WhatsApp message and drafts a reply. The next request is always to handle the whole case, from consignment lookup and delay policy to the record update and the reply. That is several steps across several systems, and the first design question is who decides the order of those steps, your code before the task arrives or the model while it runs.
What separates an agent from a workflow
A workflow is code you write. Step one runs, then step two, and a model is called at the points where some judgement about text is needed, such as classifying a message or drafting a reply. The path is fixed. The model fills in content and never chooses what happens next.
An agent is a loop. The model gets a goal and a set of tools. It picks a tool, your code runs it, the result goes back to the model, and it picks again until it decides the task is done. The path is decided at run time, separately for every task, and two similar tasks can take different routes.
- Steps and order written in code
- Model called for content only
- Same path on every run
- One or two model calls
- Failures point to a step
- Steps chosen by the model per task
- Model decides when it is done
- Path varies between runs
- One model call per step
- Failures need the trace
Multi-step does not mean agent
Most multi-step tasks have a known shape. A refund request at an online electronics store always needs the order, the payment gateway settlement record and the refund policy. A fixed sequence fetches all three and hands them to a single model call that drafts the reply. Four systems touched in a known order is still a workflow, and a fairly ordinary one.
The signal is whether the next step depends on the content of the last one in ways you cannot list in advance. A ticket at a coaching institute might be a fee problem, a login problem or a wrong batch allotment, and each needs different lookups; which lookup is right is only known after reading the first result. A data question from the finance head might take two queries or five, with the second query written from the result of the first. Those tasks have open-ended paths, and open-ended paths are what an agent is for.
A decision rule you can apply
Before choosing, write the task as the job someone does today, as a numbered list of steps. Then apply three questions in order.
- Can you write the list before you see a specific task? If yes, build a workflow.
- Does the result of a step change which steps follow? If the branches are few, write them as branches in the workflow. If they are many or you cannot enumerate them, an agent fits.
- Does the task need the model to decide when it is finished? If a person or a fixed rule can say when the task is done, keep that decision in code, even inside an agent.
Applied to a few requests that come up often, the rule sorts them quickly, and the third row is where most of the interesting design work sits.
| Task | Steps known in advance? | Build |
|---|---|---|
| Draft a reply to a refund request | Yes, always the same three lookups | Workflow |
| Route a ticket by type | Yes, a few branches | Workflow with a switch |
| Find why a consignment is late | No, depends on scan events | Small agent inside a workflow |
| Answer an open data question | No, queries depend on results | Agent with read-only tools |
The short version is to use a workflow until the branching is too large to write down. Routing by ticket type is a workflow with a switch statement in it. Investigating a ticket whose cause nobody knows yet is an agent.
What an agent costs when a workflow would do
An agent is not a free upgrade. Where a workflow would have served, it costs you in five places, and the bill arrives after launch rather than before it.
Calls come first. Every loop iteration is a model call, so a five-step task makes at least five calls instead of one, and the latency stacks up in front of the user. Then variation, because the same input can take a different path on the next run, so behaviour has to be tested by outcome rather than by expected output. That needs a different kind of test suite, built by replaying real tasks before the agent touches real systems.
Tooling is the third cost. Each tool needs permissions, input validation and logging of its own, since the model chooses when to call it; in a workflow the code already knows. Deciding what an agent may read, write and ask a person about is a design task in itself.
Diagnosis gets harder too. When a workflow fails you know which step failed. When an agent fails you read the trace to find out what it decided and why, and the trace for a twenty-step run is a long read at eleven at night. Finally, ownership shifts. Changing behaviour becomes prompt work, which is harder to review and harder to test than a change to a function.
- 01Goal and toolsgiven to the model
- 02Model picks a tool
- 03Your code runs itwith permission checks
- 04Result returns
- 05Done or notmodel decides
Repeat until done or budget spent
Each pass is a model call; a workflow pays this once.
Where the choice goes wrong
The pattern we see most is starting with an agent framework because the task sounds complex, then spending weeks on guardrails for a task that had four fixed steps. The opposite happens too, with a workflow of a dozen nested branches that nobody wants to touch, when the branching was the sign that an agent fits.
Then there is letting the model decide the order when a policy already fixes it. If the compliance check must run before the refund is released, that order belongs in code, not in a prompt the model may or may not follow on a given run. Related to it is mixing both without a boundary, so a tool called "process the refund" hides a second workflow and nobody can say whether the model or the code is responsible for the policy check.
The last one is measuring success by how the demo looks. Cost per task and the share of tasks completed correctly are the numbers that decide whether the feature stays, so agree how you will count them before the build starts.
A workflow with a small agent inside
Often the right answer is both. A fixed workflow fetches what is always needed, an agent with a narrow goal and two or three tools handles the open-ended part, and the workflow takes over again for the steps with side effects. In the logistics example, code pulls the consignment and the customer history, a small agent works out what actually went wrong from the scan events and the delivery partner's notes, and code decides what can be promised, sends the reply and writes the ticket.
- 01Code fetchesconsignment, customer history
- 02Agent investigatesscan events, partner notes
- 03Code decidespolicy check, what can be promised
- 04Code sendsreply and ticket update
Code at both ends, with the model loop only where the path is unknown.
The agent's loop stays short, the parts that change records stay in code, and each part can be tested in the way that suits it. The smallest useful first release is often just the first and last boxes, with a person doing the middle one, which tells you quickly whether the investigation step is worth automating at all.
When you take this to a first conversation about agentic AI and LLM applications, bring the numbered step list, a handful of real cases that did not fit it, and the policy rules that must always run. Those three things settle most of the design before anyone opens a framework.
