Schema-Constrained AI Still Sends Bad Data to Production

Take a representative returns workflow. A customer asks for a refund, a language model reads the conversation, and the application expects an object containing an order ID, an action, an amount in cents, a currency, a reason code, and a review flag. If the object validates, the next service may issue money.

That final step changes the standard. A response that looks neat in a model playground is not enough; the application needs to know whether the message arrived intact, matched the deployed contract, used authoritative order data, and selected an action allowed by policy. Each is a separate test.

The dangerous example is ordinary: the model returns `approve_refund`, the correct order ID, `USD`, and an integer for the amount. Every field passes JSON Schema, a machine-readable contract describing allowed types and values. The amount is wrong by two decimal places. The validator sees an integer.

The customer sees an oversized refund.

Valid JSON covers one failure layer

Malformed JSON remains the easiest failure to recognize. A model may omit a closing brace, insert commentary before the object, escape a character incorrectly, or produce an incomplete array. JSON mode and structured-output APIs reduce these errors, but applications still encounter them when a response is cut off by an output limit, a connection closes during streaming, or surrounding code extracts the wrong span.

Truncation deserves its own status. An end-of-file parsing error after a response reaches its output limit should not enter the same recovery path as an object containing an illegal enum value, because the first response may never have been completed while the second reveals a contract or reasoning problem. Capture the provider’s completion status before parsing, and treat an incomplete response as incomplete even when its prefix happens to be valid JSON.

Streaming adds another boundary. A parser that validates each network chunk as if it were a complete document will report false failures, while a parser that dispatches fields before the final object closes can act on data the model later changes. Buffer until the API marks the structured payload complete, unless the application uses a streaming parser designed to preserve document state and delays side effects until final validation.

For the refund object, that means no payment call while `amount_cents` is still arriving. Waiting costs some latency and memory. It also prevents a partial message from becoming a completed financial action.

Constrained decoding narrows the model’s choices

Constrained decoding, which limits each generated token to choices permitted by a schema or grammar, makes malformed structures much less likely. OpenAI, Google, and other model providers expose structured-output features, while Anthropic’s tool interface accepts input schemas for tool calls. The implementations differ, and their supported portions of JSON Schema are not interchangeable.

This is an important gain. If `action` allows only `approve_refund`, `deny_refund`, or `send_to_review`, constrained decoding can prevent the model from inventing `refund_customer_now`. Requiring `amount_cents` as an integer can also stop a free-form answer such as `$12.99 plus tax` from entering that field.

The constraint cannot determine whether `1299` is the amount recorded in the order system. It controls representation, not truth. It may also stop applying when the provider returns a refusal or an incomplete response, both of which need explicit handling outside the expected business object.

Schema compatibility needs testing before deployment. Keywords such as conditional rules, references, numeric restrictions, or formats may be rejected, interpreted differently, or left unenforced by a particular API. A local validator can accept a rich schema that the model endpoint does not fully support. Retrying that request only repeats a deterministic integration error and adds another model call to the bill.

Schema drift creates valid messages for the wrong contract

Schema drift occurs when producers and consumers deploy different versions of a data contract. Suppose the refund model now emits `send_to_review`, but the payment service still recognizes the older value `manual_review`. The response can satisfy the new schema and fail in the old consumer. Renaming `amount_cents` to `refund_amount` produces the same split.

Put an explicit schema version in the message envelope, then validate against the version the consumer supports. During a migration, either accept both contracts at a translation boundary or deploy the consumer before the producer begins emitting the new form. Quietly treating unknown fields as harmless is risky when those fields change policy, currency, or units.

`additionalProperties: false` can catch fields that were never agreed, provided the model API supports that rule and the application runs its own validator afterward. The local check matters because provider-side constraints protect generation, while your validator protects the boundary between the generated message and the service that will act on it.

Semantic validation catches plausible wrong answers

Structural validation asks whether `amount_cents` is an integer. Semantic validation asks whether it equals the refundable balance for that order, uses the order’s currency, and falls within the caller’s authority. These checks require application data and policy that JSON Schema alone does not have.

The refund workflow should therefore treat the model as a classifier and extractor, not as the source of record. Let it identify the order reference, classify the customer’s reason, and recommend an action. Fetch the amount and currency from the commerce system. If the model supplies either value, compare it with the authoritative record rather than trusting it.

Cross-field rules belong here too. An approved refund with `requires_review: true` is contradictory if approval means immediate execution. A reason code for a missing parcel may be valid in the enum but incompatible with an order marked delivered and already refunded. Those objects parse cleanly.

They should still stop before the payment service.

Some semantic checks are deterministic and cheap. Others require another database read, a policy engine, or human review, which adds latency and operating cost. Spend that cost according to consequence: generating a product tag can tolerate a fallback label, while issuing money should wait for authoritative state.

Recovery should follow the failure class

A production recovery path starts before the model call. Compile or test the exact provider schema in continuous integration, keep representative valid and invalid fixtures, and verify that every downstream consumer accepts the current version. This catches unsupported keywords and deployment drift without paying for inference.

At runtime, inspect the transport and completion status first. A network interruption or output-limit stop can justify one fresh generation with a smaller prompt, a larger permitted output where appropriate, or less requested data. Do not append text to a truncated object and hope the model’s missing fields can be reconstructed.

Next, parse and validate locally. If the response contains removable framing text despite an API contract that promised a bare object, record a contract failure rather than normalizing it invisibly forever. A repair pass can be useful during migration, but it creates a second model call, adds latency, and may alter values while fixing punctuation. The repaired object must pass every check again.

Then run semantic and policy checks against live records. For the refund case, compare the order ID, refundable balance, currency, prior refund status, and permitted action before creating an idempotency key, a unique token that prevents the same payment operation from running twice. Validation without idempotency still allows a successful retry to issue duplicate refunds after a timeout hides the first result.

Set a fixed retry budget. One clean regeneration after a transient or correctable failure is a practical default for many workflows; repeated failures should move to the fallback rather than cycling until an answer passes. Retries are poor medicine for unsupported schemas, stale consumer versions, refusals, and contradictions with authoritative data.

The fallback must be an application behavior, not another prompt. In this workflow, preserve the customer’s request, mark it for review, and return a pending status without calling the payment service. Log the schema version, model identifier, completion status, validation failure path, and retry count, while applying the same privacy controls used for the underlying conversation.

That makes the failure visible and recoverable. More importantly, the original refund remains unissued until a person or deterministic service supplies the missing certainty.

Questions people ask

Does structured output guarantee valid JSON?

A provider’s schema-constrained mode can guarantee or strongly enforce syntax within its documented schema subset, but the surrounding request can still end in a refusal, truncation, transport failure, or unsupported-schema error. Applications should inspect completion status, parse locally, and validate again before using the object.

Why can a response pass validation and still be wrong?

JSON Schema can confirm that an amount is an integer and an action belongs to an allowed enum. It cannot know the refundable balance in an order database or whether policy permits that action. Those claims need semantic checks against authoritative records before execution.

Should malformed JSON be sent back to a model for repair?

A single repair attempt can help with legacy models or migration periods, but it adds a model round-trip and can change values while fixing syntax. Prefer constrained decoding first, validate the repaired object from the beginning, and use a deterministic fallback when the retry budget expires.

What should happen after structured output fails twice?

Stop model retries and execute the documented fallback. For the refund workflow, retain the request, mark it pending, and avoid the payment call; for lower-risk classification, use a neutral label or queue the item. The fallback should be decided before deployment and tested like any other branch.