Northwind Labs is an invented company, but the request it faces in this example is common. Its billing service rounds currency amounts in four different places, and an engineer asks a large language model to consolidate the logic into one function and update every caller. The model returns a tidy diff in under a minute, with clear comments and a confident explanation. The diff compiles. In one path, however, it rounds half-cent values down where the old code rounded them up, and nothing in the explanation mentions the change.
I start with this kind of example because it captures what I think is the central problem with how many organizations use AI for serious work. The model did most of the job well, and the part it got wrong was exactly the part that mattered. A prompt, however carefully written, cannot make a single pass of a single model responsible for understanding the requirement, making the change, checking the change, and deciding whether it is safe to ship. My argument in this post is that dependable AI systems for professional work will come from orchestration. That means specialized agents with narrow roles, deterministic checks that do not depend on a model's judgment, structured workflows that connect them, and people placed at the points where their judgment counts.
A single model is a capable generalist but an incomplete system
Large language models are probabilistic. Depending on sampling settings, the same input can yield different outputs, and those outputs are shaped by statistical patterns learned in training. They are not checked against the facts of a particular codebase, contract, or ledger. That is the source of their flexibility, since they can handle requests no one anticipated. It is also why they can produce output that reads well and is still wrong, a behavior documented across many models and tasks [1]. Fluency and correctness are separate properties, and a single model has no internal mechanism that reliably tells them apart.
The deeper limitation is structural. When one model is asked to plan, execute, and evaluate its own work in a single exchange, the evaluation inherits the same assumptions that produced the error. Asking a model to double-check its answer helps in some cases, but research suggests that self-correction without external feedback is unreliable [2]. Context is a further constraint, because a real task may depend on more files, documents, or prior decisions than fit comfortably in one prompt, and a model cannot account for what it does not see [3]. These are observations about architecture rather than about the quality of the models themselves. A model is a component, and components need a system around them.
Specialized agents divide the work along its natural seams
The term "agent" is used loosely, so I want to be precise about how I use it. A model takes text and returns text. An agent, in the sense I mean, is a model given a defined role, a set of tools it may use, the context relevant to that role, and a loop in which it can act, observe the result, and act again. Two agents may run on the same underlying model and still behave very differently because their instructions, tools, and inputs differ.
Specialization helps because professional tasks already have seams. In the Northwind example, one agent might read the existing rounding paths and write a plain description of how each one currently behaves before any code changes. A second agent makes the change against that description, and a third reviews the diff with a narrow mandate: find any input for which the old and new behavior differ. Each agent works with a smaller and more relevant context, and each produces an artifact that something else can inspect. The half-cent discrepancy is far more likely to surface when one component's entire job is to look for behavioral differences.
Specialization does not solve the problem on its own. Agents built on the same model can share the same blind spots, so a reviewer agent may miss an error for the same reason the implementing agent made it. Multiple agents also add cost and latency, and they create new places for things to go wrong in the handoffs between them. For that reason specialization has to be paired with checks that do not rely on a model's judgment at all.
Deterministic verification checks requirements, not truth
A deterministic check produces the same result every time it receives the same input. Compilers, type checkers, linters, unit tests, schema validators, and rule engines all work this way. For Northwind, verification could include a property-based test that compares the old and new rounding functions across thousands of generated amounts, a type check confirming that every caller passes a decimal value rather than a floating-point one, and a policy rule requiring a second reviewer for any change to billing code. When one of these fails, the failure is a fact about the code rather than a model's opinion of it.
The value of these checks lies in their independence. A model cannot talk its way past a failing test, and the test does not become more lenient because the explanation attached to the diff is persuasive. Placing deterministic gates between agents means each stage must pass something objective before its output moves on. The gates also turn model failures into useful feedback, because the implementing agent can receive the specific failing case and try again with that information.
The limit of verification matters just as much as its value. Verification confirms that an output meets the requirements someone wrote down, and it says nothing about requirements no one thought to write. If Northwind's tests never exercise half-cent values, a broken rounding change passes cleanly. A green build is evidence that the specified checks passed, which is worth a great deal, but it is not proof that the system does what the business needs [4]. Treating verification as proof of correctness is one of the easiest mistakes to make in this kind of architecture, and one of the more expensive.
Orchestration is the structure that holds the parts together
Orchestration and automation are often used interchangeably, and the difference matters here. Automation executes a fixed sequence, such as a script that runs the same steps in the same order every time. Orchestration coordinates components that may succeed, fail, or need to repeat, and it decides what happens next based on what each component produced. An orchestrated workflow keeps track of state, routes failures back to the right stage, limits how many times a stage may retry, and knows when to stop and ask a person. For the Northwind change, a structured workflow might run in the following order.
An analysis agent documents the current behavior of every rounding path in the billing service.
An engineer confirms or corrects that description, which becomes the specification for the change.
An implementation agent writes the consolidated function and updates the callers.
Deterministic checks run, including compilation, type checks, the existing test suite, and a comparison test between old and new behavior.
A review agent examines the diff for behavioral changes the tests may not cover.
A human reviewer approves or rejects the change with every earlier artifact available.
What matters in this sequence is less any individual step than the contracts between them. Each stage has a defined input, a defined output, and a condition for passing work to the next stage, so a failure is located at a specific point instead of disappearing into a single opaque response. The workflow also leaves a record of what was specified, what was changed, which checks ran, and who approved the result. For regulated work, or any change with financial consequences, that record can be as valuable as the change itself.
Human judgment belongs where the consequences are real
Human oversight is easy to endorse and easy to do badly. A reviewer asked to approve every small step will soon approve without reading, and the oversight becomes ceremonial. Better design places people at a small number of points where their judgment adds something a model or a test cannot. Those points include confirming that the specification reflects what the business intends, deciding whether a tradeoff is acceptable, and accepting responsibility for a change that affects customers. In the workflow above, the engineer appears twice, and both moments are decisions rather than inspections.
What the human sees at those moments matters as much as where they sit in the workflow. A reviewer handed a raw diff and a model's confident summary is in a weak position, because the summary frames how the diff gets read. Give the same reviewer the original specification, the results of each check, and the review agent's list of behavioral differences, and the position is much stronger. The aim of orchestration, done well, is to give people better evidence at the moments when they are asked to decide.
The open questions concern cost, measurement, and design
Taken together, these principles amount to an architectural philosophy. Models are treated as capable but fallible components, agents are given narrow roles, deterministic checks sit between them, the work is structured so that failures are visible and located, and people remain at the decisions that carry consequence. I have described the philosophy here instead of any particular implementation because I think the reasoning stands on its own, and because the ideas are more useful if readers can test them against their own systems. I expect other teams to reach similar conclusions from different directions, since the problems that lead there are common.
Several questions remain unsettled, and I do not want to understate them. Orchestration adds cost and latency, and for many tasks a single well-prompted model is the right tool, so the architecture should match the stakes. Future models may absorb some of what external structure provides today through better self-checking or larger working context, though I would not expect them to replace independent checks, since independence is the point of those checks. The field also lacks good ways to measure the reliability of a whole workflow, as opposed to the accuracy of a single model on a benchmark [5][6].
The practical recommendation I would offer is to start from the failure rather than from the model. For any task an organization wants AI to handle, the first step is to identify where an error would be most costly and least visible. The workflow should then be designed so that point is covered by something independent of the model that made the error, whether a test, a rule, a second agent with a narrow mandate, or a person with the right evidence in front of them. In the Northwind example, that point was the half-cent case. Every serious task has its own version of it, and the systems that prove dependable will be the ones built to find it.
References
Lei Huang et al., A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM Trans. on Info. Sys. (2024), arXiv:2311.05232, https://doi.org/10.1145/3703155. https://arxiv.org/pdf/2311.05232
Jie Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet (Int'l Conf. on Learning Representations 2024), arXiv:2310.01798. https://arxiv.org/abs/2310.01798
Nelson F. Liu et al., Lost in the Middle: How Language Models Use Long Contexts, 12 Transactions of the Ass'n for Computational Linguistics 157 (2024). https://aclanthology.org/2024.tacl-1.9/
Mert Cemri et al., Why Do Multi-Agent LLM Systems Fail? (NeurIPS 2025 Datasets & Benchmarks Track), arXiv:2503.13657v3 (Oct. 26, 2025). https://arxiv.org/pdf/2503.13657
Sayash Kapoor et al., AI Agents That Matter, arXiv:2407.01502 (July 1, 2024). https://arxiv.org/abs/2407.01502
Shunyu Yao et al., τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains, arXiv:2406.12045 (June 17, 2024). https://arxiv.org/abs/2406.12045
