Long-running agent harnesses
Agent readiness is a measure of how many deterministic validation loops are present inside a codebase.
The Same Agent in Two Codebases
A capable autonomous agent can look brilliant in one customer's environment and disappointing in another's. The model has not changed, and the task may look similar on paper. What differs is the environment the agent works inside. The surrounding codebase, not only the agent, determines how much an agent can be trusted to do without supervision.
Keep one question in mind as you read: when an agent produces work, what is there to check that work against? The answer governs how much independence an organization can give the agent. This article follows one agent change through a codebase, then asks what happens when that codebase has fewer or more checks.
What Counts as a Validation Loop
A validation loop is a cycle in which work is produced, checked automatically, and judged against a standard. The check runs without a person deciding in the moment whether the work looks acceptable.
Deterministic means that the same input to the check yields the same verdict every time. That repeatability is what allows an agent to rely on the result. A check that sometimes passes and sometimes fails for no clear reason gives the agent nothing stable to push against.
The source material names four kinds of checks that make a codebase dense: linters, type checkers, tests, and security scans. Each examines code automatically and returns a verdict that does not depend on anyone's opinion at the time of the check. Taken together, these checks form the validation loops that give agent readiness its meaning.
Readiness as a Degree, Not a Yes or No
Agent readiness is a measure of how many deterministic validation loops are present inside a codebase. The key word is measure. Readiness is not a yes or no property of a team or a tool. It is a degree, describing how much of the codebase is equipped with checks that run automatically and return a definite result. The more of these checks a codebase contains, the more readiness it has.
Validation density is the presence of those loops across the codebase. Density is about coverage. A codebase with a few excellent checks in one corner has low density, because most of the code is not covered. A codebase where many parts of the code are covered by linters, type checkers, tests, and security scans has high density. Density describes where the checks are, not only how good any single check is.
Following One Change Through the Loops
Imagine an agent has just produced a change to the codebase. The source describes a path that such a change takes through a dense set of loops. Each loop returns a definite verdict, and the change only moves forward when the checks pass. The diagram below shows that path. The source does not fix a single required order for the four check types, so the order shown is one illustrative arrangement. What matters is that each stage can return a verdict that stops the change from moving on.
Read the diagram as a sequence of gates. The change enters a loop, the loop returns a verdict, and only a passing verdict lets it continue. A failing verdict on any stage sends the change back rather than letting it compound into later work. Where a loop is missing, the path has a gap: nothing on that stretch of the codebase can push back on a bad change. This is the mechanism behind the claim that validation density limits autonomy.
Why Density Sets the Autonomy Ceiling
The source connects validation density directly to how much autonomy an organization can grant. The quality of long-running agent output is directly proportional to how well that work can be validated. In plain terms, the more you can check what an agent produced, the better the output you can expect from a long run, and the more freely you can let it run.
Autonomy is not granted because the agent is trusted in the abstract. It is granted in proportion to the checks that can catch a bad step before it compounds. Without validation, an organization has no basis for letting the agent go further, so it must keep a human close to every step.
Low Density Versus High Density
The following scenario is an original teaching example, not taken from the source. Consider the same agent receiving the same task in two codebases. In the first, only one module has tests and a type checker, and most of the code has no automated check at all. In the second, linters, type checkers, tests, and security scans cover most of the code. The comparison below summarizes how the agent's work and the human role differ between them.
Predicting the agent's behavior in each environment
The agent makes a change that alters how a function handles an empty input. In the low-density codebase, that function sits outside any check. In the high-density codebase, a test covers that function. What happens to the change in each environment?
- 1Identify the loops that reach the change
In the low-density codebase, no deterministic check covers the function, so no loop can return a verdict on this change. In the high-density codebase, the test covers the function and can return a verdict.
- 2Trace the change through the available checks
In the low-density codebase the change moves forward without a verdict, so the error is found later, perhaps by a person or in production. In the high-density codebase the test runs on the change. If the empty-input behavior breaks the expected result, the verdict is failing and the change is sent back.
- 3Compare the required human involvement
In the low-density codebase, a reviewer has to watch this kind of change closely, since nothing automated can push back. In the high-density codebase, the reviewer's attention can go to what the checks cannot judge, because the covered behavior already has a definite verdict.
The same agent and task produce a silent, unverified change in the low-density codebase and a change that is either accepted or returned with a definite failing verdict in the high-density codebase. The organization can grant more autonomy in the second environment because it can see the outcome of the work.
Mistakes When Reading Readiness
Common mistake Treating readiness as a yes or no property
What it looks likeConcluding that a codebase is agent-ready because it has a test suite.
Why it failsReadiness is a degree that depends on how much of the codebase the checks cover, not on whether any checks exist.
Do this insteadAsk how much of the code is covered by deterministic checks, and where the uncovered regions are.
Common mistake Judging density by the quality of one check
What it looks likePraising an excellent test suite that covers only one module.
Why it failsDensity is about coverage across the codebase. A few excellent checks in one corner still leave most of the code without a verdict.
Do this insteadCount coverage across the codebase, not the strength of a single check.
Common mistake Assuming a capable agent deserves high autonomy by default
What it looks likeLetting a strong agent run unsupervised in a sparsely validated codebase.
Why it failsValidation density sets the ceiling on safe autonomy. Capability does not replace the ability to check the work.
Common mistake Treating checks that sometimes pass and sometimes fail as validation
Why it failsThe source requires a deterministic check. A check whose verdict varies for no clear reason gives the agent nothing stable to push against.
Do this insteadMake sure each check returns the same verdict for the same input.
Predict the Agent's Behavior
A customer's codebase has a type checker that covers about a fifth of the modules and no other automated checks. An agent is asked to refactor a function in an uncovered module. Before reading the answer, predict: which loops can return a verdict on this change, and how much human review should the organization expect to need? Then compare your prediction with the two environments above and explain your reasoning in terms of coverage and deterministic verdicts.
Hints
- Ask whether the uncovered module is reached by any deterministic check at all.
- Consider what happens to a change that no loop can return a verdict on.
- Connect the answer to the idea that autonomy is granted in proportion to the checks that can catch a bad step.
Raising Validation Density for a Customer
The source draws a practical conclusion for the forward-deployed engineer. Part of every engagement is raising the customer's validation density, not only building the system the customer asked for. The evals written together with the customer are the beginning of that customer's agent readiness.
Agent readiness is also the ceiling on how much autonomy an engineer can responsibly hand over. Expanding autonomy faster than the customer's validation loops can support creates a gap between what the agent is allowed to do and what anyone can verify. Closing that gap is part of the engagement's work.
- Start from the evals written with the customer and identify which parts of the codebase they reach.
- Find the uncovered regions, where a change would meet no deterministic verdict.
- Add or extend checks of the four kinds the source names: linters, type checkers, tests, and security scans, so that more of the code returns a definite result.
- Confirm that each new check is deterministic, returning the same verdict for the same input.
- Widen the autonomy granted to the agent only as the checks in a region can catch a bad step before it compounds.
Key Takeaways
- Agent readiness is a measure of how many deterministic validation loops are present inside a codebase, and it is a degree rather than a yes or no property.
- A validation loop produces work, checks it automatically with a deterministic verdict, and judges it against a standard; the source names linters, type checkers, tests, and security scans as the checks that build density.
- Validation density sets the ceiling on how much autonomy an organization can safely grant, because autonomy follows the checks that can catch a bad step before it compounds.
- The same agent behaves differently in a low-density and a high-density codebase: gaps leave errors to be found late and demand close human review, while coverage returns definite verdicts early.
- A forward-deployed engineer raises a customer's validation density as part of every engagement, starting from the evals written with the customer and widening autonomy only as the loops can support it.