The feature ships. Three months later the client's compliance team asks where customer data goes, or why the model writes warmer replies to some applicants than to others, and nobody in the room can answer from memory. The answers were always in the code, the logs and the choice of provider, which makes most of generative AI ethics a developer's job. It needs test sets, data paths and records, the same as any other requirement.
Bias shows up in your own test cases
Models learn from text that carries the habits of the people who wrote it, including assumptions about gender, religion, region and language. That becomes a concrete problem the moment the output feeds a decision: a shortlist, a product suggestion, a reply to a complaint.
You usually cannot audit a hosted model's training data. You can test its behaviour on your task. Take a recruitment platform that uses a model to draft the first reply to each applicant. Build pairs of inputs that are identical except for one attribute, and replay them:
- The same CV and cover note, with the surname changed from Sharma to Khan to Murugan.
- The same profile, with the current city changed from Noida to Bhagalpur.
- The same question, written in English and then in Hinglish.
- The same career history, with a two-year gap labelled "maternity leave" in one version and left unexplained in the other.
Lay the outputs side by side and compare tone, length and what gets offered: an interview slot, a request for more documents, a polite no. Expected: no difference a reader would notice. When there is one, it is a finding, and it goes into the regression set with the fix. Keep the set in the repository and run it every time the model or the prompt changes. Tell the client plainly what the tests cover and what they do not.
Privacy depends on the path the data takes
When a prompt includes a customer record, that record goes wherever the model runs. For a hosted model, that is a third party's servers, possibly outside India. Which rules apply depends on the data and where you operate, GDPR for European customers and HIPAA where US health data is involved, so read the contract and the provider's data handling terms before anyone writes the integration. If the terms do not fit, the comparison between a hosted model and open weights in your own cloud is mostly a data question.
- 01User inputmay hold personal data
- 02Your serverremove or mask fields
- 03Model providercheck retention terms
- 04Your logsalso hold the data
Every step before the provider is a place where you control what leaves.
The last step is the one everyone forgets. Request and response logs are useful for debugging, and they hold exactly the data you were trying to protect. A diagnostics lab that adds an assistant to explain test reports, and logs every prompt, has built a second copy of its patients' results, with no retention limit and read access for whoever has the dashboard login. Decide how long logs live and who can open them before launch.
Test this too. Send a request with a fake Aadhaar number, phone number and email in the free text. Expected: none of them in the outgoing payload, none in the logs. Actual is worth checking, because masking code tends to handle the fields it was written for and miss the ones a user typed into a comment box.
Ownership of generated output is unsettled
Text, images, code and audio can all be generated, and the underlying models may have been trained on copyrighted material. Who owns the result, and whether it copies something, depends on the tool's terms and on law that differs between countries and is still changing.
Developers cannot settle this. They can avoid hiding it. Record which model produced which asset, read the licence terms of the tool, and explain the open questions to the client in plain words before their lawyer does. Generated work is a draft that a person has to put their name to, and the way to make that true is a review step around generated text.
Label what was generated
Generated text and images look real, which is what makes fake reviews, doctored photos and plausible false statements cheap to produce. If your product generates content other people will read, such as a review summary on a marketplace or a voice note for a sales team, mark it as generated where the reader sees it. Make sure it cannot be used to impersonate a real person, and keep the feature inside the provider's usage policy and your own.
The label is a test case as well. Expected: every generated item carries it on every surface it appears on, including the email digest and the shared link preview. Those two are where it usually goes missing.
Accountability needs an owner and a record
When the output is wrong, someone answers for it: the business using the feature, the team that built it, or the provider. If that is not decided in advance, it gets argued after the incident.
- Model outputa suggestion only
- Human reviewrequired for high-stakes cases
- Decision recordinput, output, reviewer, version
- Ownera named role answers for it
Write one page per feature: what the model is used for, which version and provider it runs on, what data it receives, what it was tested for and what it must not be used for. That page answers most questions from clients, auditors and the engineer who joins in six months. It also exposes drift in purpose, such as a feature that grew from drafting replies to approving refunds and was never bias-tested for the second job.
Mistakes that cost the most
The bias set runs once before launch and never again, so a model update changes behaviour and nobody notices. Full customer records go to the provider when a name and an order number would have done. Prompts and responses sit in logs with no retention limit. A review step exists, but reviewers approve everything in seconds, which is worse than no review because it looks like one. And the product is described to users as more reliable than anyone has shown it to be.
Checklist before launch
- A paired-input bias set exists and runs on every model or prompt change.
- You know which fields reach the provider and why, and a test proves the rest do not.
- Log retention and access are defined.
- Generated content is labelled on every surface users see.
- High-stakes outputs go to a person, and the review is recorded.
- A named role owns what happens when the output is wrong.
Settle the data path first. Draw it for one request, from input to provider to logs, and check each step against the contract. It is the hardest item on this list to change once the feature is live, and it is the first thing we look at when scoping generative AI work.
