Harness research
Harness-of-Harness (HoH)
Harness-of-Harness wraps an existing coding-agent harness in a persistent planning, development and independent-testing loop so the software artifact and its evidence can improve across many iterations. It is a research framework, not an interoperability protocol, UHP extension or released HarnessRouter integration.
Status first
Section titled “Status first”Harness-of-Harness (HoH) is a research framework for long-horizon autonomous software development introduced in arXiv preprint 2609.01481, submitted on 1 September 2026. The project builds on existing coding-agent harnesses rather than defining a new agent protocol. Its public experiments use Codex with GPT-5.5 (high), OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3 across GameCraft-Bench, FrontierSWE and ProgramBench.
The current public repository and project page expose the paper, benchmark results, demos and development trajectories. The repository explicitly says HoH-lite, a lightweight implementation of the core workflow, is coming soon. This guide therefore treats HoH as published research and architecture evidence, not as a generally released software package or independently replicated production system.
What Harness-of-Harness actually does
Section titled “What Harness-of-Harness actually does”HoH keeps an evolving software project and an evidence record across repeated iterations while coordinating three stable responsibilities:
- Project Planner — reads the requirements and prior evidence, then writes a prioritized development document without modifying the artifact.
- Developer — implements that development document in the shared project workspace while preserving already verified behavior.
- QA Tester — independently inspects and tests the updated artifact, recording verified behavior and remaining gaps while staying read-only with respect to the artifact.
The resulting artifact state and evidence state become the input to the next iteration. The paper describes this as a planning–coding–testing loop with small verifiable increments, independent evaluation, structured outputs, progressive disclosure of deliverables/tools/skills and versioned project history.
The architectural distinction is important: HoH does not require the base coding-agent harness to become a new protocol endpoint. It repeatedly uses an existing harness as the execution substrate while the project evolves from loop to loop.
Requirements / PRD │ ▼Project Planner │ development plan ▼Developer on an existing coding-agent harness │ updated artifact ▼Independent QA Tester │ evidence + unresolved gaps └───────────────┐ │ next iteration ▼ Project PlannerPublished benchmark evidence
Section titled “Published benchmark evidence”The project reports Vanilla versus HoH@1–3 results for three harness/model pairs. The table below reproduces only the published Vanilla and HoH@3 endpoints so the direction and scale of the measured change stay visible without turning the guide into a leaderboard.
| Harness + model | Metric | Vanilla | HoH@3 | Reported absolute gain |
|---|---|---|---|---|
| Codex + GPT-5.5 (high) | GameCraft-Bench Overall | 49.58 | 71.52 | +21.93 |
| Codex + GPT-5.5 (high) | FrontierSWE Dominance | 44% | 71% | +27 pp |
| Codex + GPT-5.5 (high) | ProgramBench Pass Rate | 60.41 | 66.50 | +6.09 |
| OpenCode + DeepSeek-V4-Pro | GameCraft-Bench Overall | 26.90 | 48.98 | +22.08 |
| OpenCode + DeepSeek-V4-Pro | FrontierSWE Dominance | 25% | 44% | +19 pp |
| OpenCode + DeepSeek-V4-Pro | ProgramBench Pass Rate | 45.27 | 57.56 | +12.29 |
| Pi + MiniMax-M3 | GameCraft-Bench Overall | 42.16 | 58.78 | +16.62 |
| Pi + MiniMax-M3 | FrontierSWE Dominance | 35% | 64% | +29 pp |
| Pi + MiniMax-M3 | ProgramBench Pass Rate | 35.83 | 52.68 | +16.85 |
Across all reported benchmark settings, the paper summarizes an average relative gain of 52.25% and a maximum relative gain of 82.86% after three iterations. Those are author-reported aggregate results for this experimental design, not universal performance guarantees for the underlying harnesses.
Multi-day case study
Section titled “Multi-day case study”The project also reports Fusepoint, a single-player narrative first-person shooter built from an empty workspace containing only a product requirements document. The project page says HoH ran for more than 70 iterations without a human in the loop and produced a human-playable game with implemented core mechanics, storyline, visuals and audio.
This case is significant because the unit of autonomy is not one coding turn or one benchmark patch. HoH attempts to preserve verified progress and evidence over many repeated development cycles, making long-horizon artifact continuity itself part of the harness-level architecture.
The public repository links the Fusepoint trajectory so the development history can be inspected. The project also reports another game, Mournlight, at more than 80 loops, but the paper’s principal public multi-day example is Fusepoint.
HoH vs harness self-evolution
Section titled “HoH vs harness self-evolution”The name can be misleading. HoH is not primarily a system for rewriting the underlying coding-agent harness. The paper positions it around continual improvement of the software artifact while operating on existing harnesses. Planner, developer and tester roles, runtime conditions and the base harness/model pairing are held stable within an experimental run while the artifact and evidence evolve.
That makes HoH different from work whose objective is to optimize prompts, tools, skills or execution logic inside the harness itself. The “harness-of-harness” layer is an orchestration envelope around repeated harness executions, not evidence that the base harness has become self-modifying.
HoH, UHP and FrontierHarness answer different questions
Section titled “HoH, UHP and FrontierHarness answer different questions”| Layer | Primary question | Unit being controlled or measured |
|---|---|---|
| Harness-of-Harness | How can repeated harness executions improve one software artifact across many autonomous iterations? | Planner/developer/QA loop, project artifact and evidence state |
| UHP | How can a client execute a selected complete harness through one interoperable HTTP contract? | Client ↔ UHP server ↔ harness task/session execution |
| FrontierHarness Eval | How differently do frozen harness/model/runtime configurations perform on the same benchmark tasks? | Harness + model + runtime configuration |
These layers can be complementary without being interchangeable. HoH provides a long-horizon orchestration strategy; UHP provides an external execution contract; FrontierHarness provides comparative evaluation evidence. None of the three automatically establishes the other two.
Why it matters for harness engineering
Section titled “Why it matters for harness engineering”HoH adds a useful architectural pattern to the emerging harness-engineering vocabulary: continual artifact improvement through evidence-grounded repeated harness execution. The key design choices are not simply “run the coding agent longer.” The framework explicitly separates planning, implementation and independent QA, carries evidence forward, constrains work to bounded increments, and preserves versioned history so later loops can distinguish verified capability from unresolved work.
For interoperability work, the lesson is narrower. A protocol such as UHP can standardize how a harness is addressed and executed, but long-horizon project orchestration still needs its own policy for iteration boundaries, evidence retention, role separation, acceptance criteria and recovery. HoH is one research design for that layer; it does not alter the UHP specification.
Evidence limits
Section titled “Evidence limits”- The currently exposed primary artifact is an arXiv preprint plus project/repository materials; this guide does not treat it as a standards-track document or certification.
- The authors state that HoH-lite is not yet publicly released, so complete implementation-level reproducibility is not available from a packaged HoH runtime at this cutoff.
- The published benchmark covers three harness/model pairs and three software-development benchmarks; it does not establish behavior across arbitrary harnesses, models or domains.
- The benchmark and case-study results are author-published. No independent replication was verified in this research pass.
- The reported gains measure final artifact quality under the HoH experimental setup. They do not measure UHP conformance, protocol interoperability or native UHP adoption.
Related pages
Section titled “Related pages”Read harness composition and subagent delegation for child-agent and cross-harness runtime topologies, FrontierHarness Eval for controlled harness benchmarking, and UHP conformance for protocol-contract verification. The individual Codex, OpenCode and Pi pages track current integration/upstream evidence separately from the versions and models used in the HoH paper.