LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks

Overview

A task solved end to end, the results, and one command to run it.

Core Idea

Keep the goal fixed; ground every decomposition in audited facts.

A long-horizon task is ultimately solved as a series of subtasks, yet the right decomposition cannot be fixed in advance: the next subtask depends on what has actually happened in the environment so far. LongHorizon-Harness therefore replaces the single growing session with a loop over three separated roles, so the task is decomposed dynamically and every decision is grounded in independently verified facts rather than self-reports.

Compounding errors

An early mistake distorts every choice made after it, and the agent drifts away from the goal it started with.

Context rot

As the history grows, what matters gets harder to retrieve, and performance falls off sharply once context use crosses a threshold.

Task-state loss

No accurate record of what is done, what was produced, and what the environment actually contains, so progress cannot be resumed.

Once an unverified premise enters the record, every later decision inherits it.

Existing harnesses never separate progress assessment from execution: a subagent reports a summary of its own work, and review is an optional action rather than a mandatory gate. Unverified premises accumulate and lead to repeated attempts, goal drift, or premature termination. An audit gate removes that failure at its root.

Two panels side by side. Left: a single growing session coils back on itself, judging its own progress, and drifts. Right: a manager re-plans the next subtask from audited facts, a fresh-context executor performs it, and a read-only auditor certifies what actually changed. — **Audited state transitions.** Instead of one continuously growing session that judges its own progress, a *manager* re-plans the next subtask from audited facts, a fresh-context *executor* performs it, and a read-only *auditor* certifies what actually changed in the environment. Audit reports are the only cross-round memory.

Left panel · today

One session, judging itself

Execution, task state and completion assessment share a single context that only grows. The agent's own summary is the record, so an unverified claim is indistinguishable from a fact and every later step inherits it. What the drawing shows as a coil is exactly that: motion without a fixed reference to check against.

Right panel · ours

Three roles, one audited record

The task state lives outside execution and is written only by the auditor, from the environment. The manager re-plans against that record, the executor works one subtask in a fresh context and discards it, and each round leaves behind an audit report rather than a transcript. Any backbone can fill any of the three roles.

P1 · Dynamic decomposition, fixed goal

Every round, the manager reads what has been verified as done, which constraints remain active, and what is still missing, then derives the next subtask with its target, dependencies, boundaries, and acceptance criteria.

P2 · Audit-grounded progress

Only a read-only auditor can update the record. It inspects the real environment after each round and never sees the executor’s reasoning or self-assessment, so a fact is completed only when an audit report cites it.

P3 · Round-local execution

A fresh-context executor performs the single current subtask and discards its context afterwards. Only compact audit reports cross rounds, so dense step-level observations never reach long-term coordination.

P4 · Reset without amnesia

An execution error is either exposed by the next audit or never enters the record, so it stays confined to its round. The audit report separates what remains trustworthy from what must be repaired.

Method

Three structurally isolated roles, one audited state machine.

LongHorizon-Harness preserves the native agent loop of existing systems. A lightweight AgentAdapter lets Claude Code, Codex CLI, OpenClaw and Hermes Agent serve as interchangeable backends for all three roles, spanning models such as Claude Opus, GPT and Qwen. Each role has its own budget: 1800s per round for the executor, 300s for the manager and auditor, under a cap of 25 MEA rounds per task.

Overview of LongHorizon-Harness: the manager emits a subtask contract, the executor performs the state-changing action in a fresh context, and the auditor appends a read-only report to the chain.

MEA Loop

Round i: manage, execute, audit.

Each round applies three operators in sequence: state transition (manage), state-changing action (execute), and state capture (audit). The auditor’s report from round i feeds the manager’s decision in round i+1, which in turn shapes the executor’s context in that same round.

Manage sees every report, never the environment

Manager reads the record

Takes the original task, the current task state, and every audit report gathered so far.

Apply the verified findings

Confirmed changes add or update requirement, artifact and fact records; anything unresolved stays pending, blocked or untrusted. Nothing is marked completed without audit evidence.

Emit one contract

Picks an unresolved objective reachable from the current state and bounds it: goal, acceptance criteria, boundary constraints, and the prior audit evidence relevant to doing and checking it.

Execute the only writer, fresh context each round

Executor acts, then forgets

Starts from a fresh context holding only what this round supplies, changes the environment, and reports what it did. Its raw trajectory is then discarded.

Audit read-only, independent of the executor

Auditor captures the state

Inspects the environment through read-only tools against the contract's criteria. The executor's report may guide where to look, but cannot establish completion.

Continue, or stop

The manager returns done when the audited state satisfies the task, blocked when nothing feasible advances it, ask when it needs user input, otherwise execute with the next contract, up to 25 rounds.

Benchmarks

Three long-horizon benchmarks, three axes of difficulty.

We evaluate LongHorizon-Harness on WeaveBench, OSWorld 2.0, and Terminal-Bench 2.1, covering cross-interface coordination, long-horizon state management under realistic professional complexity, and pure CLI competence. Our runs use Qwen 3.7-Plus as the primary backbone and Claude Opus 4.7 for the backbone-generality study.

WeaveBench weavebench.github.io ↗ OSWorld 2.0 osworld-v2.xlang.ai ↗ Terminal-Bench 2.1 tbench.ai ↗

Scale 114 tasks across 8 domains 108 tasks, median 1.6 h for a human Hard CLI tasks, 3 runs each

Interface GUI and CLI in one trajectory Desktop GUI, 1920×1080 CLI only — no visual perception

Scoring Trajectory-aware agentic judge; PassRate counts tasks scoring ≥ 0.8 Native env.evaluate(); binary and partial Per-task pass/fail, averaged over runs

Environment Containerized VM, frozen snapshot, restricted network Official osworld-v2-2026.06.24 Docker VM Harbor on the Docker backend

What it tests Whether evidence gathered in one interface survives into another Whether task state survives an hour-long professional workflow The state-management layer alone, with GUI routing removed

Results

Consistent gains across benchmarks and backbones.

With the same backbone and the same execution backend, LongHorizon-Harness lifts WeaveBench PassRate from 51.8% to 80.7%, OSWorld 2.0 binary completion by 3.0×, and Terminal-Bench 2.1 success from 69.7% to 77.2%. The gains carry over from Qwen 3.7-Plus to Claude Opus 4.7.

Table 1 · WeaveBench results (114 tasks).

Agent	Harness	PR ↑	Overall ↑	DSK	DOC	GAM	WEB	DAV	OPS	SPA	DES
Claude Opus 4.7	Claude Code	41.2	0.532	55.6	47.1	23.5	53.3	23.1	50.0	33.3	40.0
Claude Opus 4.7	OpenClaw	35.1	0.482	55.6	29.4	23.5	66.7	15.4	41.7	16.7	20.0
Claude Opus 4.7	Hermes Agent	28.1	0.516	33.3	47.1	11.8	26.7	30.8	50.0	8.3	10.0
Claude Opus 4.7	Codex CLI	13.2	0.378	16.7	11.8	11.8	6.7	7.7	25.0	16.7	10.0
GPT-5.5	Codex CLI	35.1	0.499	38.9	29.4	23.5	53.3	15.4	50.0	58.3	10.0
GPT-5.5	OpenClaw	33.3	0.466	38.9	35.3	35.3	21.4	23.1	38.5	33.3	40.0
GPT-5.5	Hermes Agent	31.6	0.466	55.6	29.4	35.3	40.0	7.7	25.0	25.0	20.0
GPT-5.5	Claude Code	14.9	0.299	33.3	11.8	11.8	0.0	15.4	16.7	25.0	0.0
GPT-5.4	OpenClaw	22.8	0.465	55.6	35.3	5.9	0.0	23.1	23.1	8.3	20.0
GPT-5.3-codex	OpenClaw	18.4	0.456	33.3	23.5	29.4	0.0	7.7	16.7	8.3	20.0
GPT-5.2-codex	OpenClaw	6.1	0.321	5.6	11.8	0.0	0.0	15.4	16.7	0.0	0.0
GPT-5.1-codex	OpenClaw	1.8	0.226	0.0	5.9	0.0	0.0	7.7	0.0	0.0	0.0
Gemini 3.1 pro	OpenClaw	1.8	0.223	0.0	0.0	0.0	0.0	0.0	8.3	8.3	0.0
Qwen3.5-397B-A17B	OpenClaw	0.9	0.318	0.0	0.0	0.0	0.0	0.0	8.3	0.0	0.0
Qwen3-VL-8B-Think	OpenClaw	0.9	0.092	0.0	0.0	0.0	0.0	8.3	0.0	0.0	0.0
GUI-Owl-1.5-32B	OpenClaw	0.0	0.065	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Qwen 3.7-Plus	Claude Code	51.8	0.702	83.3	76.5	29.4	46.7	53.8	66.7	16.7	20.0
Qwen 3.7-Plus	LongHorizon-Harness (ours)	80.7	0.835	88.9	100.0	58.8	73.3	84.6	91.7	66.7	80.0
Change vs. the same backbone on bare Claude Code		+28.9	+0.133	+5.6	+23.5	+29.4	+26.6	+30.8	+25.0	+50.0	+60.0

Gray rows are official results reported by WeaveBench (best thinking mode per backbone); PR denotes PassRate (%), Overall the mean per-task score. All eight domains improve and DOC reaches 100.0. Our runs execute the agent with root privileges inside the task VM, whereas the official baselines run under a regular user account, so the gray rows are reference points rather than strictly matched comparisons.

Terminal-Bench 2.1 leaderboard: LongHorizon-Harness with Codex and GPT-5.6 Luna reaches 83.1%, and with Claude Code and Qwen 3.7-Plus reaches 77.2%, up from the 69.7% baseline. — Terminal-Bench 2.1 official leaderboard LongHorizon-Harness retains Claude Code as its executor backend and improves success rate from 69.7% to 77.2%. Values marked with ∗ are externally reported metrics.

Table 2 · OSWorld 2.0 (108 desktop workflows).

Agent	Harness / Mode	Binary ↑	Partial ↑
Claude Opus 4.8	Batched actions	20.6	54.8
Claude Opus 4.7	Batched actions	18.2	48.9
GPT-5.5	Batched actions	13.0	49.5
Claude Opus 4.8	Single action	18.5	49.3
Claude Opus 4.7	Single action	13.9	49.1
Claude Sonnet 4.6	Single action	8.3	41.5
MiniMax M3	Single action	4.6	22.3
Kimi 2.6	Single action	4.6	22.1
Qwen 3.7-Plus	Single action	2.8	21.5
Qwen 3.7-Plus	LongHorizon-Harness	8.3	35.2

Gray rows denote official results reported by OSWorld 2.0 under batched- and single-action settings, while the black row denotes LongHorizon-Harness with Qwen 3.7-Plus in the hybrid setting. Binary is the percentage of tasks whose final benchmark score equals 1, and Partial is the mean benchmark score over all 108 tasks. Bold marks the best official result.

Table 3 · OSWorld 2.0 Opus 4.7 subset (34 tasks).

Agent	Harness / Mode	Binary ↑	Partial ↑
Claude Opus 4.7	Single action	20.6	55.8
Claude Opus 4.7	LongHorizon-Harness	35.3	66.9

Swapping only the harness lifts binary completion by +14.7 points and partial score by +11.1, so the gains are a property of the framework rather than of one backbone. The backbone still decides the quality of each round’s actions, while the harness decides how reliably verified outcomes survive between rounds. That is why the two compound instead of substituting for each other.

WeaveBench PassRate 51.880.7 +28.9
OSWorld 2.0 Binary completion 2.88.3 3.0×
Terminal-Bench 2.1 Success rate 69.777.2 +7.5

Two scatter panels of OSWorld 2.0 binary completion and partial score against output tokens per task. A blue star marks LongHorizon-Harness with Qwen 3.7-Plus, reached by a dashed arrow from the single-action baseline. — Cost–performance frontier on OSWorld 2.0 Binary completion (left) and partial score (right) versus average output tokens per task. Colored curves show the official results under different reasoning-effort settings, with marker size indicating the reasoning effort. The blue star denotes LongHorizon-Harness with Qwen 3.7-Plus, and the dashed arrow shows its improvement over the official single-action Qwen 3.7-Plus.

Cost & Attribution

Coordination is cheap; token cost tracks the backbone, not the architecture.

The manager accounts for only 2.8% (WeaveBench), 2.0% (OSWorld 2.0), and 8.1% (Terminal-Bench 2.1) of total tokens. Auditing is the larger investment at 19.4%, 24.8%, and 38.1%. And overall cost is not uniformly higher: on Terminal-Bench 2.1 the harness consumes 24% fewer tokens than the baseline while gaining 7.5 points.

Stacked bars of tokens per task on WeaveBench, OSWorld 2.0 and Terminal-Bench 2.1, split into manager, executor and auditor shares against the baseline agent. — Where the tokens go Average tokens per task consumed by the manager, executor, and auditor in LongHorizon-Harness, compared with the corresponding baseline. Segment labels report the fraction consumed by each role. All LH-Harness roles use Qwen 3.7-Plus. WeaveBench and Terminal-Bench 2.1 report total tokens, whereas OSWorld 2.0 reports output tokens because the official results provide only output-token statistics.

−33% Overhead is not intrinsic

On the 17 Games tasks the same framework inverts sign by backbone: Opus 4.7 spends 33% fewer tokens (16.5M → 11.1M) while scoring higher, whereas Qwen 3.7-Plus spends 3.2× more (10.7M → 34.3M). A stronger backbone satisfies a contract in fewer audit–replan rounds.

0.733vs0.680 Capability is a system property

On those same tasks Qwen 3.7-Plus under the harness reaches 0.733, above the 0.680 that Opus 4.7 reaches on bare Claude Code. A harness cannot add primitive capability, but it decides how much of it survives to the end of the task.

Three dumbbell panels comparing baseline and LongHorizon-Harness across WeaveBench domains, OSWorld 2.0 capability tags and Terminal-Bench 2.1 categories, with the point change listed at the right of each panel. — Where the gains come from LongHorizon-Harness (blue) is compared with the same-model baseline (gray) across WeaveBench domains (left), OSWorld 2.0 capability tags (middle), and Terminal-Bench 2.1 categories (right). The rightmost column of each panel reports the absolute performance change in points. Blue connectors indicate improvements, and red connectors indicate regressions. Parenthesized values report the number of OSWorld tasks or Terminal-Bench trajectories in each category.

Insights

Four recurring patterns in paired trajectories.

We compare paired trajectories in which the Claude Code baseline and LongHorizon-Harness execute the same task using the same Qwen 3.7-Plus model. These cases examine what persists between rounds under the MEA loop.

Recovering from a stalled interaction.

The baseline does notice the failed GUI interaction, but the observation stays buried in its growing execution history, so it retries the same interaction for 400+ steps. The harness writes the stall and the unresolved evidence gaps into the task state, so recovery resumes from the latest audited state rather than from the failed trajectory.

Wireshark’s “Decode As” dialog stops responding. The baseline degenerates into 400+ steps of repeated clicking; the auditor rewrites the same difficulty into concrete evidence gaps, and the next round targets only the unmet items.

WEB_task_16 storyboard: the baseline loops on an unresponsive Wireshark dialog while LongHorizon-Harness rewrites the failure into evidence gaps. — `WEB_task_16` · WebRTC simulcast layer audit · **0.59** → **0.92**

Auditing apparent completion.

The executor stops at a visually plausible result that misses the specification: the baseline edits the document XML directly, and the task requires the LibreOffice workflow, so it scores 0.00. The auditor re-parses the XML instead, keeping a plausible but non-compliant claim out of the task state.

The baseline advances the document’s appearance through direct ODT XML edits and treats “looks right” as done; the judge scores the shortcut 0.00. Re-parsing the XML turns completion into an independently checkable claim.

DOC_task_2 storyboard: the baseline edits ODT XML directly and treats looks-right as done, while the auditor re-parses the XML. — `DOC_task_2` · Heading style normalization · **0.00** → **0.89**

Preserving pre-repair evidence.

Part of the required evidence belongs to the state before the repair. The baseline modifies the spreadsheet before that sequence is complete, leaving an inconsistent before-and-after record. Recording the gap as a pending requirement changes the order of work: the original state is documented before the repair may touch it.

The baseline does fix the spreadsheet, but the direct-XML repair destroys the original error scene, so its “before” screenshots show post-fix state. The harness locks the pre-repair window first, then audits all nine screenshots for consistency.

DOC_task_4 storyboard: the baseline repair erases the pre-fix forensic context; the harness locks the evidence window down first. — `DOC_task_4` · Calc VLOOKUP repair · **0.45** → **0.87**

Continuing from verified progress.

Both systems can perform the core optimization; they differ in maintaining the rest. The baseline improves the page, but the same session must also drive DevTools, track deliverables and judge its own progress, and the budget runs out. Keeping finished work outside the execution history lets fresh executors spend context on what remains, while the manager maintains continuity across the whole workflow.

The baseline is fully capable of the optimization, yet the same session must also keep operating DevTools and tracking what is left. Under the harness the optimization persists in the task state, and fresh executors collect the remaining evidence.

WEB_task_10 storyboard: the baseline lands the core optimization but burns its budget in DevTools; the harness phases the work across roles. — `WEB_task_10` · Lighthouse performance drill · **0.53** → **0.85**

BibTeX

Cite this work.

If you use LongHorizon-Harness in your research, please cite the project.

@article{longhorizonharness2026,
  title={LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks},
  author={Ziyu Ma and Hailang Huang and Shun Zou and Yong Wang and Shidong Yang and Yiming Hu and Fei Wei and XiangXiang Chu},
  journal={arXiv preprint arXiv:2608.01964},
  year   = {2026},
  url    = {https://arxiv.org/abs/2608.01964}
}