Four techniques appear in nearly every generative AI plan: models that read images and audio as well as text, prompts that drive a business process, fine-tuning on your own data, and interfaces generated from a description. Each one demos well in an afternoon. What each needs once real inputs arrive (the data to line up, the prompts to version, the review step) is rarely in the demo, and it decides whether the feature survives its first quarter.
Multimodal input means lining up your sources
A multimodal model accepts more than text: photos, voice notes, PDFs, sometimes sensor exports. The appeal is that real questions do not arrive sorted by file type. A maintenance query at a packaging unit in Faridabad might depend on a vibration export from the machine and a technician's voice note describing the noise.
The difficult part is rarely the model. It is the inputs. Each source has its own format, timestamp, quality and permissions. Consider a D2C clothing brand that sells on its own site and on marketplaces and wants an assistant to answer product questions. The product photo shows a blue kurta, the description says navy with a mandarin collar, and the size chart PDF is from last season.
The model does not detect that these sources disagree. It produces an answer that blends all three, and the customer receives a confident description that matches none of them. So the first design decision is which source wins when they conflict. That rule belongs in code, not in the prompt, where it can be tested and cannot be edited away by accident.
Decide too what each source may send to an external provider. A return request that bundles a customer's ID photo with a product page sends more than anyone intended unless someone made that rule explicit.
Prompts are configuration
A prompt decides the output format, the tone and what the model should decline to do. On the finance side, a prompt might pull vendor name, GSTIN, invoice number and taxable value from scanned invoices so they can go into the accounting system without retyping. On support, a prompt keeps replies in one voice even when the ticket arrived half in Hindi.
Once a prompt drives a business process, treat it as code. It needs a name, a history, an owner and tests. Picture a prompt that lived in one developer's chat window for six months. The developer moves to another team, the output shifts slightly in the next release, and nobody can say which wording produced last month's extractions or how to restore it. The fix is unexciting: keep the prompt in the repository and release it the way you release anything else, as described in managing prompts like code.
- 01Edit promptone change at a time
- 02Run saved examplessame inputs every time
- 03Compare to currentside by side
- 04Releasetagged version
Failures become new examples
A prompt change ships only after it runs on the same saved examples as the version it replaces.
Fine-tuning comes last
Fine-tuning takes a base model and trains it further on your own examples, so its output follows your vocabulary, format or house rules. It is often proposed first. We usually suggest it last, because the problem it solves is narrower than it sounds.
Before anyone reaches for it, ask whether the real problem is missing information. The D2C brand wants the assistant to answer questions about its return and exchange policy. The model has never read that policy, so its answers are generic. Training on the policy is one option. Retrieving the relevant clause at request time and passing it in the prompt is usually the simpler one. The answer can cite the clause, and when the policy changes before the festive sale, you update one document instead of retraining. The steps are set out in what a RAG pipeline does.
- Fastest to try
- Rules and examples in the prompt
- Limited by one request's size
- Your documents passed in per request
- Answers can cite a source
- Update a document, update the answer
- Changes style, format, terms
- Needs reviewed training data
- Repeat when the data changes
Fine-tuning earns its place when the problem is behaviour rather than knowledge: a fixed output format the model keeps drifting from, or phrasing the compliance team requires, after prompts and examples have genuinely stopped improving it on the evaluation set. Then it is the right tool. It also comes with a data preparation job that someone has to own for as long as the feature exists.
Generated UI gives you a screen
Tools can now produce an interface from a sentence such as "a dashboard of quarterly sales by region". That is useful for getting a prototype in front of a sales head the same afternoon, and for internal tools where nobody wants to wait a sprint for a screen four people will use.
What comes out is a picture of a feature. It is not connected to your data, and it does not follow your design system unless you described it. Nobody has checked whether it works with a keyboard, what it shows when the region filter returns nothing, or whether the regional manager in Lucknow should see Bengaluru's numbers at all. Those questions take the time in any screen, generated or hand-built, and the generator has answered none of them. Where AI helps UI design looks at this in more depth.
Where these projects slip
Most of the trouble is evaluation. Someone reads five outputs, they look fine, and the change ships. Without a saved set of examples and a written pass-or-fail rule, the regression from last Tuesday's prompt edit shows up as a customer complaint three weeks later.
The second problem is training on data nobody cleaned. Fine-tune on a folder of outdated and contradictory policies and the model reproduces the contradictions faithfully.
The rest are quieter. A generated screen reaches production looking right, and its empty state, error state and no-permission state appear for the first time in front of users. And when output drifts and nobody owns the prompt, the team spends a week working out who changed what.
If you are starting this week
- Write down twenty real inputs for the task, with the output you would accept for each, and run them on every prompt change.
- Put prompts in your repository with the rest of the code.
- Try a prompt alone, then retrieval, before considering fine-tuning.
- Decide, per input source, what may leave your systems.
- Send generated UI through the same design and code review as hand-built screens.
Start with the twenty examples. Every later argument, about prompts, about fine-tuning, about whether a generated screen is good enough, is settled faster when there is something concrete to run.
