Strict JSON Schema at the model boundary, optional business rules after it, and an audit trail that survives the handoff.

This article is the technical pair for Mostly-right data extraction isn’t operational data. The business pair makes the case for treating soft JSON as unfinished work; this one is about the schema lock, optional validators, and audit trail that make operational data real.

It also builds on Building registry-based extraction for any document type and on The architecture of AI-native document management software.

---

Soft JSON fails the ERP, not the model call

An extraction run can finish without throwing, return a JSON object that looks complete, and still be useless the moment finance posts it. Dates arrive as three dialects in the same week. Amounts stay as prose inside a number-shaped key. Closed vocabularies become whatever phrase the document used. Nothing in the model HTTP call failed; the contract with the next system did.

This piece is how a schema-first extraction engine closes that gap. The output shape is declared before the prompt gets clever, the model is locked to that shape at call time, optional business rules catch what a schema can’t express, and every accepted run keeps an audit trail of what the model was shown and what it returned.

Where this sits after the registry

The previous article made document types cheap to add, a folder the registry discovers at startup, one set of endpoints, one shared engine path. That settles how a type lands. It doesn’t settle whether each extract is fit for another system to consume.

Once a type is registered, the quality of its declared shape is what decides whether extraction pays off. The business pair framed that as three gates (schema, constrained output, business validation). This article is the engineering behind those gates inside the shared engine, and how the per-document record in Azure Cosmos DB keeps the trail.

Every extract still reads from the same search layer the chat service uses. Schema and validators don’t invent a second retrieval stack. They only decide whether what the chunks support is shaped well enough to keep.

Where things go wrong

The naive build treats the prompt as the contract. You ask the model for “the important fields,” parse whatever JSON comes back, and hope the keys stay stable. They don’t. Optional fields vanish. Required ones get invented. Types drift because the model was never forced to a dialect. Downstream systems then become the real validators, which is the expensive place to learn the shape was never agreed.

The quieter trap is a schema that only lives in documentation. The team writes a nice field list in a wiki, then calls the model in free-form JSON mode (or worse, plain text) and re-validates everything in Python after the fact. That second pass becomes a second schema that drifts from the first, and the model keeps emitting soft values the post-processor has to massage or reject on every run.

The third trap is putting structural checks in the business-rule layer. Re-testing that a date is ISO-shaped, that an Incoterm is in a fixed set, or that a currency matches a three-letter pattern, after structured outputs already enforced those constraints at the model boundary, is wasted code. It also trains the team to treat the schema as optional, because “validation” still happens later. Structural correctness belongs in the schema. Cross-field rules belong after parse. Mixing them is how both layers get soft.

Schema-first extraction as a short stack

Usable extraction in this engine is a short stack, and each step has a different job.

The schema is mandatory

Every registered type must expose a schema getter. The registry loads it at discovery time. The engine refuses to call the model when that getter is missing. The schema isn’t documentation the prompt vaguely mirrors; it is the structured-output contract the Responses API receives on every extract.

What the schema owns, in practice:

• Field names and nesting (objects, arrays of line items, and so on).
• Types, including explicit null so the model can decline instead of inventing.
• Formats and patterns (ISO dates, emails, three-letter currency codes).
• Enums for closed business vocabularies (Incoterms are the usual example).
• Numeric ranges where a negative quantity or a VAT rate over 100% is nonsense.
• additionalProperties: false, so the model can’t invent bonus keys the consumer never asked for.

A representative slice looks like this (simplified from a real invoice type):

{
    "name": "invoice_data",
    "strict": True,
    "schema": {
        "type": "object",
        "additionalProperties": False,
        "properties": {
            "invoice_date": {
                "type": ["string", "null"],
                "description": "Issue date as YYYY-MM-DD.",
                "format": "date",
            },
            "currency": {
                "type": ["string", "null"],
                "pattern": "^[A-Z]{3}$",
            },
            "total_amount": {
                "type": ["number", "null"],
                "minimum": 0,
            },
            # ... remaining fields ...
        },
        "required": ["invoice_date", "currency", "total_amount"],
    },
}

Required in a strict schema means the key must be present. Nullability is how you allow “present but unknown.” That split is deliberate. A missing key and a null value are different signals to the UI and to whoever reviews the record.

The model call is locked to that schema

The shared engine retrieves the document’s chunks from the search index, assembles the source text, loads the type’s prompt profile, and calls the Azure OpenAI Responses API with text.format set to json_schema, spreading in the schema the type registered (name, strict: true, and the schema object). The model isn’t asked to “try to return JSON.” It is constrained to emit output that conforms to the schema.

response = openai_client.responses.create(
    model=deployment_name,
    input=[
        {"role": "system", "content": system_instructions},
        {"role": "user", "content": user_text_with_sources},
    ],
    store=False,
    text={
        "format": {"type": "json_schema", **output_schema},
    },
)

store=False keeps this a one-shot extract; the audit trail lives in Cosmos on the document record, not as a retained Responses API object. If the response is incomplete, or the structured payload can’t be parsed into an object, the engine fails the extract. It doesn’t invent an empty success, and it doesn’t persist a junk record for the caller to discover later.

Business validators are optional and narrow

After a successful parse, the engine checks whether the type registered a validator. Most types won’t. Structural correctness already happened at the model boundary, so a validators module exists only when the type has genuine cross-field rules a JSON Schema can’t express on its own, for example subtotal + vat_amount must equal total_amount within a cent, or a due date must not precede the invoice date.

When a validator returns failures, the engine logs a structured warning with a count (not the error strings, which can carry field values) and continues. That is an explicit product choice in this codebase today, not an accident. The schema gate is hard. The business-rule gate is advisory at extract time, so operators can see rule breaks without the run pretending it never happened. Teams that need a hard stop before integrate still own that decision at submit or in the consuming system; the extract path records the shaped data and the trail either way.

The template ships a commented example of that cross-field pattern. Live types can omit the validators module entirely; the registry treats absence as normal.

How to decide what belongs where

This is the if-then test the schema-first boundary exists to support.

If another system will consume the value, declare it in the schema first, with a type and a null policy, before you argue about prompt wording. If the value can arrive in more than one format, pin the format in the schema (ISO dates, uppercase currency codes) so every run speaks the same dialect. If the allowed values are a closed set the business already uses, put them in an enum instead of hoping the model paraphrases consistently.

If a rule depends on more than one field, it belongs in an optional business validator after parse. Stretching the prompt to carry that rule usually produces more confident mistakes. Prompts suggest; schemas constrain; validators decide the rules a shape can’t carry.

If the structured response is empty or incomplete, fail the extract. Don’t coerce soft text into a record and call it done. And when the run succeeds, persist the shaped fields on the per-document Cosmos record, set status to extracted, store any field locations the layout mapper found, and append an audit entry that keeps the Responses API input messages and the full response payload (with caller identity stripped). Three months later, “why does this field hold that value?” is a lookup, not a memory.

Layout mapping, turning each extracted value into a page region for the UI highlight, runs best-effort after parse when a Document Intelligence layout sidecar exists. Missing layout is non-fatal. Field-level provenance gets its own deep dive in the next article; here it is only the reminder that shape and location are separate jobs.

Reality check

Schema quality is ownership work. Someone still has to decide which fields count for each document type, what null means, which enums are closed, and which cross-field rules are worth encoding. Making types cheap to register, as the registry article argued, raises the stakes on those decisions, because you’ll be encoding more of them.

Strictness cuts both ways. An over-strict schema rejects real documents that use odd but legitimate phrasing, and the team starts widening fields until the schema stops meaning anything. An under-strict schema ships garbage into operations with perfect confidence. The useful discipline is to be strict on the fields another system or a regulator will actually read, nullable on the ones the document often omits, and honest about partial records when the business can proceed with gaps.

The non-blocking validator is a trade-off you should name out loud. It keeps extract available when a total doesn’t reconcile, which is useful for review workflows. It also means a green extract status isn’t the same thing as “every business rule cleared.” If your process needs a hard gate before ERP handoff, put that gate on submit or in the integrator, and don’t pretend the advisory log line already did that job.

Cost sits mostly in retrieval and the model call, not in schema enforcement. Structured outputs add a little prompt overhead for the schema itself; they remove the retry loops and hand-repair that free-form JSON usually needs. Incomplete responses and empty parses still happen, and they should fail loudly. Budget for review on the types where documents are messy, instead of assuming every file clears on the first pass.

And none of this repairs bad retrieval. If the search layer never surfaced the clause that held the liability cap, no schema invents a trustworthy number. Operational data still depends on the document home and the indexing pipeline underneath it. The gates in this article only decide whether what you did find is shaped well enough to use.

Conclusion

Once types are cheap to register, the demo instinct is to celebrate any JSON the model returns. That instinct is how teams ship mostly-right payloads into processes that need operational data.

The engine that gets value from extraction flips the definition of done. It requires a strict schema before the model runs, locks the Responses API call to that schema, keeps business rules narrow and optional after parse, fails incomplete structured output instead of inventing success, and persists an audit trail beside the shaped record. The model can still be wrong about the world; what changes is that soft, uncheckable JSON stops looking like finished work.

---

Ideas, opinions, and tone are mine. AI helped with the language.

For more articles on AI-native document management visit the pialgorithms blog.

---

pialgorithms | document management software | ai engineering services