Code as Agent Harness

Today’s paper introduces the concept of “code as agent harness,” framing code as the central execution, verification, and stateful medium for artificial intelligence agent systems. Historically, large language models generated code merely as a final output target. However, as agentic systems tackle increasingly complex and long-horizon tasks, code is shifting to serve as an infrastructure layer that enables reasoning, physical or digital actions, and environmental modeling. The paper presents a comprehensive survey and taxonomy to systematically organize this paradigm, highlighting how structured code-based loops improve the reliability and safety of autonomous systems.

Overview

The core idea behind this framework is to position code as the foundational harness that wraps around a large language model, transforming it from a stateless text generator into a functional and verifiable agent. Instead of relying solely on natural language, the agent interacts with its environment by generating executable programs, receiving runtime feedback, and updating its state. The overall architecture is structured across three connected layers: the harness interface, harness mechanisms, and scaling the harness. This structure ensures that decisions made by the model are grounded in executable actions and checked by deterministic feedback loops.

The first layer, the harness interface, establishes how code connects the model to reasoning, action, and environment modeling. For reasoning, code externalizes intermediate thoughts into programs that can be run and verified by interpreters or symbolic solvers, separating high-level logic from raw computation. For acting, code serves as the API or control policy that translates high-level plans into physical or digital commands. For environment modeling, code represents the state of the world through test files, simulators, and execution logs, giving the agent an inspectable world to interact with.

The second layer focuses on harness mechanisms, which sustain the agent over long-horizon tasks. These mechanisms include planning (decomposing and structuring trajectories), memory (storing working context, past experiences, and repository structures), and tool use (interacting with APIs and sandboxes). A crucial component is the control loop, which uses a Plan-Execute-Verify cycle. This cycle ensures that before any program state transition occurs, it is thoroughly tested in sandboxed environments and verified against pre-defined constraints. Additionally, the framework outlines how “evolution agents” can optimize the harness itself by analyzing execution logs to suggest improvements to prompts, tools, or workflows.

The third layer, scaling the harness, extends this framework to multi-agent systems. When multiple agents collaborate, they share a common codebase and coordinate through specialized roles such as planners, coders, testers, and reviewers. The agents synchronize their activities through shared blackboards or repository structures, resolving conflicts and verifying progress collectively. This collaborative structure prevents a single agent from being overwhelmed by too much context while allowing specialized modules to handle different parts of the software engineering pipeline.

Takeaways

In this paper, a thorough systematic review of existing literature up to 2026 is conducted, showcasing the practical utility of code as an agent harness across diverse application domains. These domains include repository-level coding assistants, OS and browser automation, embodied robotics, and scientific discovery.

The analysis demonstrates that delegating computation and action to code-grounded environments substantially increases task success rates compared to pure natural-language methods. By analyzing structures like the Plan-Execute-Verify loop, the paper shows how real-world environments benefit from deterministic sensors (such as compilers and unit tests) to catch and fix errors automatically. Furthermore, the evaluation outlines critical open challenges for the field, including the need for better evaluation benchmarks, safety-critical human oversight, and the management of semantic conflicts in multi-agent shared states.

The paper presents a unified roadmap for AI agents by framing code not just as a product of machine learning, but as the primary runtime substrate for execution, verification, and state preservation. By organizing this perspective into interfaces, mechanisms, and scaling strategies, the paper establishes a clear engineering discipline for building more reliable, stateful, and verifiable agentic systems.

For more information please consult the full paper.

Congrats to the authors for their work!

Ning, Xuying, et al. “Code as Agent Harness: Toward Executable, Verifiable, and Stateful Agent Systems.” arXiv preprint arXiv:2605.18747v1, 2026.