Structured Extraction from Emails and PDFs: JSON You Can Trust

Every month a distributor in Indore receives a few hundred supplier invoices, as PDFs on email and photos on WhatsApp, and an accounts person keys the supplier, the amounts and the dates into Tally. A language model can read those documents and return JSON today. The engineering work is in deciding which of that JSON to trust, what to retry, what to hand to a person, and how to write the result without creating duplicates.

Start from the schema

Define the output before writing a single line of prompt. Every field gets a type, a note on whether it is required, and the allowed values where there is a fixed list, such as the GST rate. Then ask the model to fill that schema. Most providers support a structured output mode or a tool call with a JSON schema, which removes the parsing problem entirely. Asking for JSON in free text and hoping the reply contains nothing else is how you end up writing a regex to strip "Here is the JSON you asked for".

Two decisions matter more than they look. Make optional values explicitly nullable, so that null means the document did not contain the value, which is different from a guess. And add an evidence field per value, holding the text span or page where the model found it. A single extracted field then looks like this.

{
  "supplier_gstin": {
    "value": "23ABCDE1234F1Z5",
    "evidence": "GSTIN: 23ABCDE1234F1Z5",
    "page": 1
  },
  "po_number": { "value": null, "evidence": null, "page": null },
  "grand_total": { "value": 48260.00, "evidence": "Total Rs. 48,260.00", "page": 2 }
}

Evidence is what makes a person's check take seconds instead of minutes, because they look at the highlighted line instead of reading the whole invoice. The values above are a made-up example; the shape is the point.

Checks against your own records

Schema validation catches a wrong type or a missing required field. The checks that catch real errors compare the values against data you already hold, and each one that fails produces a named code rather than a general "invalid".

Check	What it catches	Failure code
Supplier against vendor master	New or misspelt supplier	SUPPLIER_UNKNOWN, SUPPLIER_CLOSE
Lines plus tax against grand total	Misread amounts, missed lines	TOTAL_MISMATCH
GSTIN pattern and master	Wrong digits, wrong supplier	GSTIN_INVALID
Invoice and due dates	Future dates, day and month swapped	DATE_ORDER
PO number format and existence	Invented or mistyped PO	PO_NOT_FOUND

The supplier match returns one of three results: exact, a short list of close candidates, or unknown. For the Indore distributor, the supplier match and the total check between them catch most of what the accounts person used to catch by eye. The codes drive the routing, and a few months in they tell you which failures are common enough to fix at the source, such as one supplier whose PDF always puts the total in a footer.

The vendor master is the column everyone assumes is clean, so reconcile it against Tally before relying on it. Duplicate suppliers with slightly different names will turn a correct extraction into a close match every time.

1. 01Documentemail body, PDF, attachment
2. 02Model fills schematyped fields, nulls, evidence
3. 03Checksschema, then your own records
4. 04Routewrite, retry, or a person
5. 05Write oncekeyed on the document

Corrections join the test set

The routing step is driven by named check results, not by the model's opinion of its own answer.

Retries that carry new information

A retry with the same prompt is hoping for a different answer. A retry that includes the failed checks is a correction. "The line amounts plus tax sum to one value and the total field holds another; read the totals again" gives the model something to act on. Run with a low temperature, cap the retries at two, and record each attempt with its check results.

After the cap, stop. A third attempt that happens to pass is not more trustworthy than the two before it that failed, and the document is better off with a person.

Sending doubtful documents to a person

Confidence here is the outcome of the checks, not a score the model reports about itself. Three routes cover most cases.

1. Every check passes and the matches are exact. The record is written automatically.
2. Only soft checks fail, such as a close supplier match or a missing optional field: queue it for a person, prefilled, with the failing field and its evidence highlighted.
3. Hard checks still fail after the retries: queue it with the document and nothing prefilled, so a wrong guess does not get approved out of habit at four in the afternoon.

The review screen shows each field, its value and the span from the document side by side. The person corrects and confirms. Store the correction next to the model's original output; that pair is the most useful test case you will ever get, and it costs nothing extra to keep.

Writing into Tally without duplicates

The same email gets forwarded twice, once by the warehouse and once by purchase. A supplier resends the PDF with "please find attached again". The pipeline reruns after a failure halfway through a batch. Each of these produces a second extraction of the same document, and an insert each time means duplicate invoices and a reconciliation problem at month end.

Derive a stable key per document. Use the supplier's own invoice number together with their GSTIN where both exist, and a hash of the attachment content plus the sender where they do not. Store the key on the record and look it up before writing; if it exists, update or skip rather than insert. Write the extraction record and the business record in one transaction, so a failure between them cannot leave half a result. And mark records a person has corrected, so a rerun does not overwrite the correction with the model's original output.

Insert on every run

• Forwarded email becomes a second invoice
• Rerun after a crash doubles the batch
• Correction lost on the next run

Keyed, repeatable write

• Document key checked before writing
• Same input, same record, every time
• Corrected records protected from reruns

Where extraction pipelines fail

The commonest mistake is treating a schema-valid result as a correct one. The JSON parses, the types match, and the supplier is still wrong. Close behind is required fields filled with plausible values the document never contained; nullable fields and evidence spans are the fix for that. Then retrying without feedback and accepting whichever attempt passes.

Three more show up a little later. There is no test set, so a prompt change or a model update silently breaks the two supplier layouts that used to work. Records get corrected by hand in Tally, where the pipeline cannot see the correction and learns nothing from it. And scanned PDFs with no text layer, which is most of what arrives on WhatsApp, go straight to a text model. Run recognition first and keep the page images for the reviewer; reading forms and invoices with vision models covers where that step loses accuracy. Contracts bring a different set of edge cases again, which we go through in extracting parties, dates and amounts from contracts.

Checklist before going live

• The schema has explicit nullable fields and an evidence span per value.
• Every check has a named code, and the results are stored with the extraction.
• The vendor master has been reconciled against the ledger.
• Retries include the failed checks and are capped.
• The review queue shows document, fields and the failing check together.
• Each document has a stable key; writes are repeatable and corrections are protected.
• Fifty real documents with hand-checked expected JSON run on every prompt or model change.

Start with the fifty documents and their expected output, pulled from last month's actual inbox. Every other decision in the pipeline, from the prompt to the retry limit, gets measured against them, and they are also the first thing to bring to any conversation about LLM applications for document work.