Skip to content

From Building It to Cutting It in Half: My Agent Harness Postmortem

This post is part of the AI Agents series and follows Building an Agent Delivery Harness from Zero. More precisely, it is the teardown report.

The short version: in the runs I reviewed, repeated context reloads and broad verification added substantial cost, while the intermediate handoffs produced no attributable incremental findings. The reduced Agent Delivery Harness keeps one delivery context, deterministic workflow gates, and one independent final review.

Not long ago, I wrote about a harness built around conductor, generator, and reviewer roles, a RED/GREEN loop, and more than two hundred sensors. The code is public in agent-delivery-harness. Before that first article had time to settle, I cut the instruction set each agent had to read by half and removed most of the role handoffs.

The old design was not a mistake. It worked for the models and problems I had at the time. Then the conditions changed.

Why the old design worked

I started working seriously on harness design in February and March. The main references were Anthropic's pattern for long-running agents, including the initializer/coding-agent split, feature lists, and incremental progress; the Guides and Sensors framework published on Martin Fowler's site; and OpenAI's approach to treating the repository as the system of record. The harness I built from those ideas worked very well.

This was during the Opus 4.5 and 4.6 period. In my projects, agents working inside the harness performed much better than agents working without it. Requirements lived in an issue, preflight checks ran before implementation, roles were separated, and sensors checked each step. One project stands out: issues took roughly 30 minutes each during an unattended overnight run, and the results were ready to merge. My conclusion at the time was straightforward. The model could write code, but the harness made it safe to let the model deliver.

The role changes also had a purpose. Work moved between conductor, generator, and reviewer subagents so that each stage started with a clean context. Longer contexts tended to drift, and residue from earlier reasoning could distort later decisions. Paying for isolation seemed worthwhile.

Recalculating the cost after a model change

With Fable 5 and GPT-5.6, issues of similar scope began taking two hours or more instead of 30 minutes. That gap cannot be attributed directly to either the models or the harness. The models, instructions, sensors, and gates all changed during the same period. It was still enough of a warning to stop and inspect the full workflow.

Fortunately, the harness already produced detailed traces and logs. I compared its lifecycle trace, per-issue execution records, and GitHub Copilot's native session logs over three evenings. Two major cost centers emerged.

The first was accumulated constraint. The harness had grown to more than three thousand lines of instructions, more than two hundred sensors, and over a dozen gates. Most rules were scars from earlier incidents. New rules kept arriving; old ones rarely left. I started calling the result harness debt. Every rule added a recurring comprehension and compliance cost to every agent and every issue.

The incentives created by those rules mattered even more. The instructions said that GREEN should run only relevant sensors, yet both Claude and GPT models repeatedly ran the full suite after a change. They needed evidence for the next gate, and the most conservative evidence was a complete run. The same verification anxiety appeared across model families. A prohibition written in prose could not overcome the incentive built into the workflow.

The second cost center was role switching. In this set of issues, a newer model could carry an issue through one context without an observable decline at the existing quality gates. Each handoff still created another subagent, reloaded more than three thousand lines of instructions, and added a median model delay of five to seven seconds per step across thousands of steps. The traces showed model processing accounting for 62 to 73 percent of wall-clock time. One overnight run created 101 subagents and accumulated 169 million subagent tokens. Those figures do not attribute all delay to role switching, but they do show how large the multiplier had become.

More importantly, the intermediate handoff protocols and evidence rituals produced no attributable incremental findings in these runs. Deterministic boundary scripts and the final independent review caught the actual problems. The scripts stopped an agent from skipping the PR and closing an issue directly. The reviewer found a crash bug, an injection risk, and an 844-line overbuilt test. The retained review is itself a role separation, but it has a specific purpose, independent judgment, and observed findings. The trace reconstructs which gate found each problem first; it is not a defect detector by itself.

A practical comparison

A postmortem was not enough, so I ran a practical comparison in the same repository with the same quality gates: behavioral tests, mutation verification, green CI, and confirmation that the PR had actually merged.

  • The old sequential harness averaged 2.4 hours and roughly 28 million subagent tokens per issue.
  • The reduced harness used one delivery agent from start to finish, ran scoped sensors during implementation, and invoked one independent reviewer before the PR. Issues took 35 to 75 minutes, without relaxing the existing quality gates.

This was not a controlled benchmark. The issues in the two groups were not identical, the sample was small, and the new version changed role switching, instruction volume, and sensor scope at the same time. The improvement cannot be assigned to any one change. The narrower conclusion is still useful: in this set of production work, the reduced harness delivered much faster, and the existing gates did not observe a quality regression.

Timing came from issue execution records and lifecycle traces. Subagent counts, token totals, and model time came from GitHub Copilot's native session logs. I attributed a problem to the first gate that actually stopped it. The public repository includes the trace schema and product-quality rubric, but the raw session logs contain project material and will not be published.

How the reduced harness works

The new version keeps the delivery process but gives one agent responsibility for the full issue. I still use Stage 0 through Stage 4 to compare it with the earlier design. In code, four hard gates control the workflow: start, scoped sensors, independent review, and ship.

flowchart TD
  A["GitHub issue"] --> B["Stage 0: preflight + worktree"]
  B --> C["Stage 1: plan + human-input gate + feature list"]
  C --> D["Stage 2: one agent, one context<br/>TDD + scoped sensors"]
  D --> E{"All features pass?"}
  E -- "No" --> D
  E -- "Yes" --> F["Stage 3: fresh-context review"]
  F --> G{"Revision required?"}
  G -- "Yes" --> D
  G -- "No" --> H["Stage 4: approve + PR + CI + merge + finish"]

Stage 0: prepare the right workspace

start-issue.sh runs environment and identity preflight checks, verifies GitHub authentication and required project gates, then creates an issue-specific branch and worktree. Its job is simple: do not let an agent start writing code in the wrong directory, under the wrong account, or in a broken environment.

Stage 1: plan before decomposing the work

The same agent reads the GitHub issue and comments, writes a plan to the local issue-tracking directory, and lists decisions that require human input. It cannot write feature_list.json until those questions are resolved. Once the direction is confirmed, the agent usually decomposes the issue into two to five features. Each feature has an observable acceptance condition and a regression sensor. Work that crosses a real runtime boundary also gets an end-to-end sensor.

Stage 2: one agent owns delivery

This is the largest change. The old workflow repeatedly handed work among tester, implementer, and conductor roles. The new workflow keeps one feature at a time with the same agent in the same context. TDD remains: write a failing test, make the smallest implementation change, then run GREEN. During the loop, the harness runs declared sensors plus scoped sensors affected by the diff. The model no longer decides whether to run the full suite. After a feature passes, the agent commits, pushes, and moves to the next feature.

I removed the handoffs and evidence forms around every RED, implementation, and GREEN cycle. I did not remove test discipline. Setting passes:true still requires executable verification, while the final independent review judges test quality.

Stage 3: change context once

After every feature passes, the harness runs the complete pre-review sensor set and creates the only subagent in the delivery lifecycle. This reviewer has not seen the implementation process. It receives the issue, acceptance criteria, and complete branch diff, then looks for failure modes that the tests may have missed. It may add a minimal test or fixture to demonstrate a problem, but it cannot modify production code.

If the verdict is NEEDS_REVISION, the delivery agent fixes the issue and the reviewer rechecks only the changed feature. I kept this role separation because it produced findings in the trace and because its independence serves a clear purpose.

Stage 4: finish with deterministic boundaries

The harness runs pre-PR sensors and binds the review approval to the current HEAD. create-pr.sh synchronizes the latest main, verifies the gates, pushes, and opens the PR. merge-pr.sh checks GitHub again and merges only after CI is green. Finally, finish-issue.sh confirms that the PR is in the MERGED state, records the outcome and execution cost, and removes the worktree.

The model does not get to interpret these states. Scripts and live GitHub data determine whether review happened, CI passed, and the PR merged.

The warning signs were already there

Cognition's June 2025 article Don't Build Multi-Agents describes a failure mode close to what I observed: coordination cost can outrun the benefit when several agents hold partial context and make conflicting decisions. It does not reject every subagent. It argues for keeping decisions on one thread while using subagents for narrow questions that do not split decision ownership. The idea was not new. I had simply reached the point where I could see it in my own traces.

Anthropic's description of Fable 5 emphasizes minimal oversight, fewer turns, and less correction. OpenAI's GPT-5.6 release similarly describes fewer model round trips and less guidance on tool-heavy tasks. Neither company tells users to remove a harness. The relevant signal is narrower: as models require less external guidance, prompts, handoffs, and guardrails designed for earlier models need to justify their recurring cost again.

This does not make multi-agent design a mistake. The same OpenAI report shows gains from multiple agents on work that can be explored in parallel. My workflow repeatedly rebuilt context around one issue without receiving that parallelism in return.

The sources I used in February and March were not wrong. They described the models and problems of that period. A few months later, the way models completed the work had changed while the harness had stayed in place.

Independent review has a different evidence base. Research has observed that LLM evaluators can favor their own outputs, while self-correction without external feedback can be ineffective or even harmful. These papers do not prove that a fresh-context reviewer always beats self-critique. They support a more conservative engineering choice: the context that produced the diff should not be its only judge. In my traces, independent review also found problems that the intermediate roles missed, so it remains the one intentional role separation.

What I removed and what I kept

I used the same principle for subtraction that I had used for construction: inspect the evidence. For each component, I considered its execution cost, the problems it had stopped, the consequence of failure, and whether another control already covered the same risk. Zero observed findings trigger scrutiny; they do not prove that a gate has no value.

Four hard boundaries remain: preflight, scoped sensors, one independent review, and merge/closeout evidence. Each has a defined failure mode and an observable result. None depends on the model declaring its own success.

I removed the rest: conductor/generator role files, the handoff payload protocol, the four-part quality ritual and five-dimension self-check around every GREEN, the red-first evidence chain (48 logs and 181 features, with zero defects attributed to it), duplicate log streams, and more than fifty fake sensors that used grep to check whether a document contained a sentence.

The final accounting: 12,500 lines removed; instructions loaded by each agent reduced from 3,333 lines to 1,630; sensors reduced from 223 to 170; and model contexts per issue reduced from more than a dozen to two, one delivery context and one review context.

Four principles I am keeping

The rewrite clarified how responsibility should be divided inside the harness:

  • Keep one decision chain with one agent. Planning, implementation, and repair benefit from shared context. Create a subagent only when work can actually proceed in parallel, needs a different information source, or benefits from independent judgment.
  • Let the model make judgments and let scripts enforce boundaries. The agent can decide how to test and repair. Tools determine whether gates passed, CI is green, and the PR merged. When the workflow encourages the wrong behavior, change the tools and process before adding more prose.
  • Measure cost, yield, and risk for every gate. A rule that once prevented an incident does not deserve permanent status, and a gate with no recent findings is not automatically disposable. Consider time, tokens, observed findings, failure severity, and alternative controls. A cheap deterministic boundary can remain valuable even when it rarely fires.
  • Treat tracing as part of the harness. Without execution records, this rewrite would have been based on intuition. Time, tokens, role switches, and first-detecting-gate attribution make it possible to remeasure the design after a model change instead of carrying forward assumptions from the previous generation.

This revision does not turn single-agent delivery into a new doctrine. The division of responsibility is concrete: one context delivers, deterministic gates verify workflow state, and one fresh context performs the final review. After the next model change, I will measure those three parts again and decide what still earns its place.