The moment the output contract changes from free text to validated data, state, control flow, and the API all flip, even though both machines read the same search index.

This article is the technical pair for Chat is not enough when the job is a process. The business half makes the case for treating some document work as a process to run; this half is about the architecture that choice forces.

It also builds on The architecture of AI-native document management software and on the agentic RAG chat service it keeps getting compared to.

---

The reply looked like the finish line

Someone asks the chat app about a supplier contract, gets a clean grounded answer in thirty seconds, and the demo lands. Then the actual job starts, where fifteen fields have to come out of that contract in a fixed shape, get checked, and land in the ERP the same way every time. Serving that second job means building a different machine, a document workflow engine, because a single shift in the output contract changes what the architecture has to guarantee.

A chat answer is something a person reads once and acts on with their own judgment. A structured extraction is something another system consumes the same way on every run. The instant you move from the first to the second, almost every assumption the chat service was built on stops being true, so trying to serve both from one architecture ends up serving neither well.

Where this sits in the system

The last nine articles built the conversational side of the platform, from the read path down to a grounded answer that cites its sources. This piece is the bridge to the other half of the story, the extraction workflows that turn documents into operational data. Both halves read from the same search layer; what changes is everything after retrieval. The deep dives on the registry, the schema-first design, and layout-assisted field locations come in the next three articles, so this one stays on the architectural contrast and the shape of the engine.

Why the chat architecture is the wrong shape for actions

The tempting move is to run extraction through the chat service, since it already retrieves documents and calls a model. It breaks in ways that have nothing to do with model quality, because the chat service was designed around properties that an action actively doesn’t want.

It’s stateful on purpose. The chat service chains each turn to the previous one so a follow-up question inherits the whole thread, and it stores that conversation per user. An extraction has no thread, since the fiftieth invoice of the month has nothing to do with the forty-ninth, so conversational state is dead weight that only invites cross-contamination between unrelated documents.

Its output is free text. A chat reply is prose with citations woven in, shaped a little differently every time, which is fine for a human and useless to an ERP that expects the same keys with the same types on every call. There’s no schema to reject a mis-read date or a currency off by a factor of a thousand, because the chat service never promised a shape in the first place.

Its record is built for reading, not for proof. The chat service keeps a transcript so a person can scroll back. An action needs a durable record of exactly what the model was shown and what it returned, keyed to the document, so months later someone can answer why a field holds the value it does. A transcript partitioned by user can’t answer that question about a document.

None of this makes the chat service bad. It’s shaped correctly for answers, which is exactly why it’s shaped wrong for actions.

The two machines, side by side

Put the two next to each other and the contract difference shows up in every layer.

State. The chat service is stateful, holding conversation state on the server and replaying it per turn per user. The workflow engine is stateless per call, since each extraction is a one-shot request that carries its own document and needs no memory of any other.

Output contract. The chat service returns free text plus a list of clickable citations. The workflow engine returns a JSON object locked to a declared schema, with typed fields, enums where a value must be one of a fixed set, and every field allowed to be null so the model can decline instead of inventing.

Control flow. The chat service hands the model a toolbelt and lets it drive an agentic loop, deciding which tool to call and how many times. The workflow engine runs a fixed path with no branching left to the model, so it always retrieves, makes one constrained call, validates, and persists in that order.

Determinism. The chat answer is stochastic by design, because the value is a fluent synthesis. The extraction is constrained hard at the model boundary, because the value is a predictable shape another system can parse.

Storage and retention. The chat service persists conversations partitioned by user and leans on the provider keeping each response server-side for about thirty days. The workflow engine persists one record per document, partitioned by the document, and asks the provider to retain nothing, keeping its own durable audit trail instead.

Inside the workflow engine

The engine is deliberately boring, since predictability is the whole point. One request names a document type and a file, and the same code path serves every type.

One endpoint set for any document type

New document types shouldn’t mean new endpoints. At startup the engine scans its own folder, and each subfolder that ships a small configuration file registers itself as a document type, contributing the settings for its prompt, its model profile, and how many chunks to retrieve. A shared registry holds those entries, so adding invoices after contracts is a matter of dropping in a folder with a config, a schema, and an optional validator, not editing the request handlers.

That’s why the API is generic and parameterized by type. Where the chat service exposes conversation-shaped routes for creating a chat, sending a message, listing and renaming and deleting chats, the workflow engine exposes one small set that takes the document type as a path segment, with one route to extract, one to submit the final result, one to update saved fields, and reads for the stored record and eligibility. If the type in the path isn’t registered, the request is rejected before any work starts.

The constrained call

For a given document, the engine builds a focused retrieval against the same search index the chat service uses, scoped to that one file and its container, and assembles the retrieved chunks into the source text. Then it makes a single model call whose output is locked to the type’s schema.

response = client.responses.create(
    model=deployment,
    input=[
        {"role": "system", "content": instructions},
        {"role": "user", "content": sources_text},
    ],
    text={"format": {"type": "json_schema", **output_schema}},
    store=False,
)

Two choices carry most of the weight here. Setting the output format to a structured JSON schema on the Azure OpenAI Responses API forces the model to emit an object that fits the declared shape, which is what turns “the liability cap is roughly 500,000” into a typed field a system can trust. And asking the provider to keep nothing on its side reflects the one-shot nature of the job, since there’s no next turn to chain to, so the durable record is the engine’s own, not the provider’s.

Schema first, then business rules

The schema is defined before any extraction logic, and it does most of the enforcement on its own. Each field declares its type, marks itself nullable so a missing value comes back as null rather than a guess, and pins closed sets to enums.

{
  "incoterms": { "type": ["string", "null"], "enum": ["EXW", "FOB", "CIF", "DAP"] },
  "contract_date": { "type": ["string", "null"], "format": "date" },
  "gross_weight": { "type": ["number", "null"], "minimum": 0 }
}

That strict shape handles structural correctness at the model boundary, which is why most document types need nothing more. When a type has genuine cross-field rules that a shape can’t express, a date window that has to run forwards, a currency that has to match a country, it ships an optional validator that the engine runs after parsing. The validator is separate from the schema on purpose, because structural correctness and business correctness are different checks with different failure meanings.

The record built for proof

Extraction results are persisted one record per document, in Azure Cosmos DB, partitioned by the document rather than by a user. The record id is derived deterministically from the document type, its container, and the document, so repeated extractions of the same file converge on one shared record instead of scattering copies, and everyone working that document sees the same state.

Because that record is shared, writes use optimistic concurrency, where each update carries the version tag it read and is rejected if another write landed first, which surfaces to the caller as a clear conflict to refresh and retry rather than a silent overwrite. Every extraction also appends an audit entry that stores what the model was shown and the full response it returned, so the answer to “why is this field this value” is a lookup, not a shrug. When a source document is deleted or replaced, its record isn’t dropped; it’s archived and moved aside so the operational history survives while the live slot is freed. That difference in what gets kept, and why, is the clearest tell that this is a different machine from the chat transcript.

Reality check

Two machines cost more than one, and it’s worth being honest about the shape of that cost. They do share the expensive floor, since both read the same indexed content through the same retrieval layer, and the workflow engine reuses that rather than building its own. What it adds on top is the schema, the optional validator, and the persistence, and the registry keeps the per-type part down to configuration instead of new code.

Statelessness here is a deliberate feature worth defending. Dropping conversational memory makes each extraction cheaper and simpler to reason about, and actions don’t want memory, since the point is that document one and document one thousand run identically. Asking the provider to retain nothing shifts the cost of the durable record onto storage the engine controls, which is the right trade when that record has to survive audits and feed reporting long after any provider-side window would have closed.

The part no setting decides for you is ownership. A workflow that turns documents into operational data depends on someone choosing which fields count, what a valid value looks like, and where the result is allowed to land. The schema is where those decisions get written down, so the schema is only as good as the person who owns it. That’s the work a chat demo quietly skips, because reading a document never required anyone to commit to what a correct extraction is.

Conclusion

The distance between a chat service and a workflow engine isn’t a few endpoints. It’s the output contract, and once output has to be the same shape every time and defensible months later, statelessness, schema constraint, and a durable per-document record stop being options and become the design. The chat service is built to be stateful, free-form, and stochastic because an answer wants all three; the workflow engine is built to be stateless, schema-locked, and deterministic because an action can’t tolerate any of them.

What the two genuinely share is the floor, the stored documents and the retrieval layer that finds the relevant passages. Above that line they diverge completely, and recognizing where the line sits is what keeps a team from stretching a chat app over a job it was never shaped to do. The next three articles go down into the workflow side, the registry that makes one engine serve any document type, the schema-first design that makes extractions reliable, and the layout mapping that shows each field where it came from on the page.

---

Ideas, opinions, and tone are mine. AI helped with the language.

For more articles on AI-native document management visit the pialgorithms blog.

---

pialgorithms | document management software | ai engineering services