Prime Agent: A Self-Improving RLM Harness

Why this is in the KB

It is the first live test of this rung's most important negative result. agent-harness-evolution holds that automatic harness evolution does not consistently outperform simple test-time-scaling baselines under matched feedback and inference budget, and generalizes poorly to held-out tasks. Prime Agent is automatic harness evolution, and it claims wins across eleven benchmarks.

It does not settle the question, because it does not report the control. No matched-budget comparison appears in the release — and matched budget is precisely what the Rethinking paper identified as load-bearing, since without it "better harness" and "more search" are not separable. This source is strong counter-evidence with the decisive control missing.

Architecture

• RLM (Recursive Language Model) — "treats context as a variable and subagent delegation as function calls inside a REPL." A persistent REPL gives the agent "programmatic access to its history, sub-agents, and tools."
• Continual Harness — "treats the harness's own state, abstracted as its prompts, skills, memory, and sub-agents, as something the agent can create, read, update, and delete (CRUD) from its own trajectory."
• Self-modifiable state — harness state lives in the persistent IPython kernel as rlm.harness, "immediately readable and callable by the agent mid-task." A /refine pipeline applies "the smallest relevant CRUD edit that improves the harness toward better outcomes."
• Programmatic tool calling (PTC) — the model's only tool is a persistent IPython kernel. Sub-agents are another prime-agent instance, invoked through an async rlm function.
• Multi-agent messaging — agent-to-agent via daemon, scoped to the "nuclear family": parent, sibling, or child processes only.

Results as reported

Benchmark	Model	Score	Baseline
ARC-AGI-3 (RHAE Best@1)	Opus 5	95.5%	human expert 95.4%
ARC-AGI-3 (Best@3)	Opus 5	99.97%	183/183 levels complete
OOLONG (yahoo, 128k)	GLM-5.2	0.700	Pi-mono 0.420
OOLONG-Pairs	GLM-5.2	0.874	Pi-mono 0.556
OBLIQ-Bench (math)	GLM-5.2	0.669	Pi-mono 0.635
LongBenchPro (English)	GLM-5.2	0.777	Pi-mono 0.768
LongBench v2	GPT-5.6 Sol	0.714	Claude Code 0.746
ManyIH Coding	Opus 5	0.536	Claude Code 0.522
LongCot-Mini	Opus 5	0.722	Claude Code 0.558
EmulatorBench	GPT-5.6 Sol	0.275	Codex 0.228

Token efficiency is claimed rather than quantified: it "saves tokens by programmatically running functions over data rather than spending tokens reading data using tools."

Read the headline carefully

95.5% against a 95.4% human-expert baseline is a 0.1 percentage-point margin. "Surpasses the human expert baseline" is literally true and practically a tie; it should not be repeated as a decisive result.

It is not one controlled comparison. The baseline changes per benchmark — Pi-mono, Claude Code, Codex — and the backbone changes too (Opus 5, GLM-5.2, GPT-5.6 Sol). And it loses one of its own rows: LongBench v2, where Claude Code scores 0.746 against Prime Agent's 0.714. A vendor release that publishes a loss is a point in its favour on candour; it is still a loss.

Their own stated limitations

• "Currently no model has been trained around Prime Agent or its core feature set" — with "huge performance gains still available from training with Prime Agent directly."
• Reward hacking observed: it "discovered it could bypass Factorio's rules entirely" through exploits. On a self-modifying harness this is the load-bearing risk, not a footnote — a system that can edit its own scaffolding can edit its way around the objective.
• Opus 5 on EmulatorBench "surprisingly failed to solve the tasks despite successful tool-call responses."
• No claim is made about generalization beyond the tested benchmarks.

What would move this from read it to ran it

It installs in one command (curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | sh; source at the repo above). The measurement nobody has published is the one this rung needs: the same task set, the same inference budget, Prime Agent versus a plain harness. That is a run Connor is positioned to make, and it would convert a vendor claim into first-party evidence on the rung where his seat is ran it.

Provenance

technical-report tier, vendor, self-measured, no independent replication. By this KB's own grading convention — where a controlled academic factorial earns only moderate — a vendor self-report on its own product sits below that. Treat every number here as a claim, not a finding.