The hidden cost of structured output
Four 2026 studies, and why no vendor documentation mentions this.
Read the guide →By task
Analysis and data
Language models are strong at structuring, categorising and explaining data. They are unreliable at silently doing arithmetic on it.
Separate the tasks the model is good at from the ones it is not
Extraction, classification, reshaping, spotting patterns and explaining what a number means are all well within reach. Multi-step calculation performed invisibly inside the answer is not, and the failure is quiet: you get a plausible number with no indication that it was produced rather than computed.
Two defences. Ask for the working to be shown so it can be checked. Or, where the platform allows it, ask for the calculation to be run in code rather than reasoned about, which turns an estimate into an actual computation.
Reliable
- Pulling fields out of messy text
- Categorising against a rubric you define
- Reshaping between formats
- Describing what a pattern might mean
- Drafting the caveats section
Verify every time
- Totals, averages and percentages
- Anything multi-step and arithmetic
- Comparisons across differently-sized groups
- Trends inferred from few data points
- Any figure that appears without a source column
Define the rules before the data
Most analysis disagreements are definition disagreements. State the rules explicitly and the output becomes reproducible instead of a matter of taste.
Categorise each row using only these categories: [list].
If a row does not clearly fit, put it in "unclear" rather than choosing the closest option.
Rules:
- A row counts as [X] only when [explicit condition].
- Ignore rows where [field] is empty.
- Segments with fewer than 30 responses must be flagged, not reported as findings.
Output a table with: original row, category, and the phrase that decided it.
Do not compute any totals. I will do that myself.The final line is deliberate. Ask for the categorisation and do the counting in a spreadsheet, and you get the model's strength without its weakness.
Correction does not always stick. Pointing out an arithmetic error and asking for a fix will often produce a new answer with the same defect, because the underlying process has not changed. If a number is wrong twice, stop asking and compute it elsewhere.
Structured output has a cost worth knowing
Forcing results into JSON makes them machine-readable, and 2026 research shows it can measurably reduce quality on harder tasks, with the effect depending on how capable the model is. The practical fix is to let the model reason in plain text first and produce the structured object afterwards, rather than forcing the schema from the first token.
There is a second trap in the monitoring. Schema validity is not correctness. A response can be structurally perfect and semantically wrong, and if your dashboard only tracks validity it will look healthy while the answers degrade.
Quality assurance
Building a test set so you can tell whether a change actually helped.
Read the guide →Personal data in AI tools
What to strip out before customer or employee data goes anywhere near a prompt.
Read the guide →