AI agent orchestration is production ready when eight things are true: agents execute in isolated workspaces, every change passes automated verification, a human approves at defined checkpoints, cost is attributed per unit of merged work, agent instructions are versioned files, the orchestrator is not bound to one model vendor, data residency is known, and a failed run has a defined recovery path. A pilot needs none of these and still looks impressive. Production needs all eight, because each one is a failure mode that only appears at volume. This article is the checklist, what a failing answer looks like, and how to test each item against a system you already run.
The checklist
| # | Requirement | The question to ask | A failing answer |
|---|---|---|---|
| 1 | Workspace isolation | Where does each agent write? | "The repository checkout" |
| 2 | Automated verification | What must pass before a human looks? | "The reviewer runs the tests" |
| 3 | Human checkpoints | Where exactly does a person decide? | "Reviewers look at the PRs" |
| 4 | Cost attribution | What did the last merged change cost? | "We see the monthly API bill" |
| 5 | Versioned instructions | Where do the agent's instructions live? | "In a prompt someone pasted" |
| 6 | Vendor portability | What breaks if the model is deprecated? | "We would rewrite the prompts" |
| 7 | Data residency | Which parties hold a copy of the code? | "The vendor handles that" |
| 8 | Recovery | What happens when a run fails halfway? | "Someone notices and cleans up" |
The order matters. Items 1 to 3 are correctness: without them, volume produces defects faster than review catches them. Items 4 to 6 are sustainability: without them, the system works but cannot be reasoned about or moved. Items 7 and 8 are the ones a security review and an on-call engineer respectively will raise, usually after the rollout.
1. Workspace isolation
Two agents editing one checkout will interleave writes, and the resulting diff belongs to neither plan. The isolation unit should be a git worktree: a separate working directory with its own branch and index, sharing the object store with the main checkout. Creating one is a checkout, not a clone, so it costs a fraction of a second rather than the time to re-download history.
# One directory and one branch per plan.
git worktree add ../work/plan-4812 -b plan/4812
git -C ../work/plan-4812 status --short
git worktree remove ../work/plan-4812
Containers are the stronger boundary when the agent runs untrusted code, and Docker's security model sets out what that boundary actually guarantees. For a trusted agent operating on your own repository, a worktree is usually the right trade: isolation per plan without a container start on the critical path. Git worktrees for parallel AI agents covers the failure modes, including the shared-hooks and submodule cases that catch teams out.
Test it: start two plans that touch the same file and confirm both produce clean, separate diffs.
2. Automated verification before a human looks
A reviewer reading a diff that does not compile is a reviewer whose time you wasted. Tests, lint, type check, build, and a scope check against the plan should all run in the agent's worktree and attach their output to the change. This is continuous integration applied per plan rather than per branch.
Two gates are specific to agent output. A scope check compares files touched against files the plan named, because an agent asked to fix a null check will sometimes also reformat the file and rename a variable. A security scan looks for the categories in the OWASP Top 10 for LLM Applications: insecure output handling, supply-chain additions, leaked credentials. Verification gates for AI-generated code sets out which gates should block and which should only inform.
Test it: ask what percentage of agent changes reach a human with failing tests. If nobody knows, the gate is not a gate.
3. Human checkpoints at exactly two points
Zero checkpoints means an agent merges to main and errors are found by other people. Ten checkpoints means approving each tool call, which makes the human the slowest component and trains reviewers to approve by reflex.
Two checkpoints put judgement where it changes the outcome. The plan gate is where a misunderstanding is cheapest to fix, because no code exists and a plan is a page of text. The diff gate is where correctness is confirmed against verification output. What is a software factory describes the full sequence.
Test it: name the two moments a person decides. If the answer is a range of activities rather than two points, there is no checkpoint, only supervision.
4. Cost attributed per merged change
The monthly API bill tells you nothing actionable. The number to track is cost per merged pull request: total spend in the period, including plans that were denied or abandoned, divided by pull requests merged. Denied work is part of the cost of accepted work.
Read it alongside denial rate, the share of pull requests closed without merging. Either number alone can be improved by making the other worse, which is why the DORA metrics have always been reported as a pair. Measuring AI coding agent throughput defines each metric and what a bad reading means.
Test it: ask for cost per merged PR for last month. If it can only be computed by hand from an invoice, it is not being managed.
5. Instructions as versioned files
Agent behaviour that lives in a prompt someone pasted into a chat window cannot be reviewed, diffed, or rolled back. Instructions belong in the repository, for the same reason configuration does in the twelve-factor app: a change becomes a diff in a pull request that a senior engineer can reject.
// Instructions and memory are files the orchestrator reads, not
// strings compiled into the binary. A convention learned on Monday
// is applied by every agent, and every developer, on Tuesday.
interface Promptware {
program: string; // Program.md - the stage's instructions
memory: string[]; // Memory/ - learnings about this codebase
tools: string[]; // Tools/ - scoped permissions
logs: string[]; // Logs/ - append-only execution history
}
The risk is drift: an agent that can edit its own instructions can also make them worse, for example by adding "retry failing tests up to three times" after one flaky run. Versioning is the control, because the revision is a diff a reviewer sees. Promptware: agents that improve their own instructions covers both that risk and memory bloat.
Test it: run git log on the directory holding your agent instructions. An empty result means the instructions are not under management.
6. Portability across models and agents
Models are deprecated on the vendor's schedule, not yours. An orchestrator that hard-codes one provider turns a deprecation notice into a migration project. The layer worth owning is the workflow: stages, gates, checkpoints, and memory. The execution engine underneath should be swappable per plan, and tool access should go through a published interface such as the Model Context Protocol rather than a bespoke integration per agent.
Portability also lets you route by cost. A triage step does not need a frontier model; an architectural plan might. You can only make that trade if changing the model is configuration rather than a rewrite.
Test it: switch the model on one plan and run it. If that requires a code change, you have a dependency, not a configuration.
7. Known data residency
Every copy of your repository outside your control is one you must inventory, contract for, and eventually delete. A hosted agent clones the repository into the vendor's environment, which puts at least two processors on your source: the agent vendor and the model provider. A local-first orchestrator has one, and only for the portions of code that appear in prompts.
This is not a security argument so much as a scope argument: it determines how many data processing agreements a GDPR review has to cover, and whether EU residency is a provider setting or a vendor negotiation. Local-first AI development works through the questions a security team will ask.
Test it: list every party that holds a copy of your code during one agent run. If the list is longer than you expected, the review will find the same thing.
8. A defined recovery path
At volume, runs fail: a network timeout mid-execution, an agent that loops, a verification step that never returns. The orchestrator needs an answer for each, and the answer must not be "someone notices."
- Failed verification. The work returns to the agent with the actual failing output attached, not a summary, and reruns in the same worktree, up to a retry cap.
- Abandoned run. The worktree and its branch are removed, so a stalled plan does not leave a directory that a later run trips over.
- Runaway cost or loop. A per-plan token and time budget that stops execution rather than one that reports after the fact.
- Partial state. Because each plan owns its worktree and branch, a failed run is discarded by deleting one directory. Nothing was written anywhere shared.
Test it: kill an agent process halfway through a plan. What is left on disk, and does the next plan care?
How Ivy Tendril answers the eight
Ivy Tendril is a local-first desktop application for macOS, Windows, and Linux that runs coding agents from plan to reviewed pull request. Worktree-per-plan (1), tests, lint and diff inspection attached to every change with CI import on Pro (2), the two checkpoints (3), cost per plan and per job (4), promptware units as versioned files (5), any CLI agent and any model chosen per plan (6), code and logs that never leave your machine (7), and retry with full failing output plus worktree cleanup (8).
curl -sSf https://cdn.ivy.app/install-tendril.sh | sh
On Windows, use irm https://cdn.ivy.app/install-tendril.ps1 | iex. Tendril is free and source-available under the Functional Source License; team features, on-prem hosting, SSO, and CI verification imports are on the Pro and Enterprise plans.
Frequently asked questions
Which of the eight should a team fix first?
Isolation, then verification, then checkpoints. Cost and memory problems are annoying; two agents writing to one checkout produces a defect nobody can attribute, which is worse and harder to diagnose.
Is this checklist specific to coding agents?
The failure modes are. Isolation, scope checks, and diff review exist because the output is a change to a shared codebase. An agent that only reads, or that writes to a store it owns, has a different list.
How many agents can run in parallel before this matters?
Two. One agent in one checkout needs none of this. The moment a second runs concurrently, isolation and cost attribution stop being optional, and review capacity becomes the constraint on throughput rather than execution speed.
Does more parallelism keep increasing throughput?
No. Review is serial, so by Amdahl's law the serial fraction sets the ceiling: adding agents past the point where reviewers saturate raises cost and denial rate without raising merges.