Plenty of companies have run a generative AI pilot, watched it work in the demo, and then found the step to everyday use slower and dearer than anyone had budgeted. The demo was allowed to skip the hard parts: real inputs, wrong answers, cost at volume, and the person who has to approve what the system does. Before you fund the next phase, it helps to know which of those problems the market has made easier and which ones are still yours to solve.
What Has Changed for Teams Building on It
Several things have moved in a direction that helps. Hosted models are available through APIs, so a team can try an idea in a week without running any infrastructure of its own. Providers offer controls such as data handling options and content filters. Business software increasingly exposes APIs a program can call, including the accounting and CRM tools most mid-size companies here already run.
Other things have not moved at all. Output can still be wrong, and a fluent wrong answer is harder to spot than an obvious one. Quality still has to be measured on your own data rather than the vendor's samples. Cost still rises with volume, and the Diwali month's volume is not June's. Whether the whole exercise pays depends on the process it improves, measured against what that process cost before anyone mentioned AI.
Assistants Now Sit Inside Everyday Software
Microsoft Copilot, Gemini in Google Workspace and Notion AI put summarising, drafting and search inside the software people already open every morning. That lowers the barrier considerably. There is no new tool to learn, no separate login and no training session to schedule.
For a company building its own software, it also raises expectations. The sales manager who gets a summary button in email will ask why the order screen in your own system does not have one. The useful question is which of your workflows involve enough text, documents or repeated judgement to benefit, and whether the person who receives the output can check it quickly. If checking takes as long as doing the work, the feature has saved nothing.
What a Custom Agent Actually Is
An agent is a language model connected to tools. The model reads a request and decides which tool to call, for example a database query or an API, and your code runs the call and hands the result back. The examples teams describe are things like matching payment gateway settlements against orders and flagging the ones that do not agree, answering customer questions from the product catalogue, or watching shipment events and triggering the next step.
The model does the reading and the choosing. In my view, everything that changes data should still be ordinary, tested code that a person can read, with permissions as narrow as the job allows. Our piece on what an agent may read, write and ask a person about goes into those permissions in detail.
- 01User request
- 02Model proposes a tool callwhich tool, which inputs
- 03Your code validatespermissions, input checks
- 04Person approvesfor risky actions
- 05Tool runs
The model proposes, your code checks and runs, and a person approves anything with side effects.
Not every task needs an agent. Many are better served by a fixed sequence of steps with a model doing one of them, and that is cheaper to test and easier to explain to an auditor. The trade-off is set out in when a task needs an agent and when a workflow is enough.
Where the Project Effort Actually Goes
Support, sales content, document processing and prototyping are the usual starting points. In each one, the model's output lands in a process that already has rules, and connecting to those rules is most of the work.
Take a distributor in Indore that wants support replies drafted from its product manuals. The drafting part is a week. Connecting it to the ticket system, citing which manual page each answer came from, and handing off to a person the moment a customer switches to Hinglish and gets angry is the rest of the project. Budget for the rest.
| Area | Model's job | What the surrounding system must do |
|---|---|---|
| Support | Draft replies from your knowledge base | Cite the source, hand off to a person |
| Sales content | Draft proposals and messages | Apply approved wording and compliance checks |
| Documents | Extract fields, summarise clauses | Validate fields, flag low confidence |
| Prototyping | Generate drafts and sample data | Keep drafts separate from production |
Why Pilots Stall Before Daily Use
The first reason is that nobody wrote down what success looks like, so three months in nobody can say whether to continue. The project drifts until someone senior asks what it has delivered, and there is no answer either way.
The rest follow a familiar pattern. Answers sound right but are not grounded in the company's own data. Agents get broad permissions because narrow ones took longer to set up, and now the support bot can technically issue refunds. Nobody planned for the model being slow, wrong or unavailable. Sensitive data went to a provider before anyone read the terms. And cost was estimated from the demo's twenty requests rather than from the volume a full month of real use would bring.
- Hand-picked examples
- One person checks the output
- Cost per request not tracked
- Failures ignored
- Saved test set from real inputs
- Review path with an owner
- Cost and response time monitored
- Fallback when the model fails
Each item on the right has to be built, and each one has an owner.
Be honest about which column your pilot is in. Most I hear about are still in the left one, and that is fine for a pilot. It is not fine for the budget request that asks to roll it out.
What to Settle Before Funding the Next Phase
| Decision | Who signs off | What it costs to leave open |
|---|---|---|
| The measure that justifies going ahead | Business sponsor | No way to stop or continue with confidence |
| What the system may change | Process owner | A bot with more access than any new hire |
| Where a person takes over | Operations head | Customers stuck with a wrong answer |
| What data leaves your systems | Whoever owns compliance | A contract problem found after launch |
Then work through these in order:
- Write down the measure that would justify going ahead, in the terms finance uses.
- Collect real inputs and the outputs you expect, and run the pilot against them.
- List what the system can change, and require approval for the risky actions.
- Estimate cost from expected usage, including the busiest month of the year.
Start with the measure, and put a date on it. It decides what the other three steps have to prove. The range of work these decisions apply to, from document extraction to agents, is outlined on our AI solutions page.
