Prime Agent: A self-improving RLM harness
Self-improving RLM harness with CRUD-editable harness state (prompts/skills/memory/sub-agents) in a persistent IPython kernel; reports 95.5% ARC-AGI-3 vs a 95.4% human-expert baseline (a 0.1pp margin) and wins on 9 of 11 rows. VENDOR SELF-REPORT with NO matched-budget control -- the exact control that would test agent-harness-evolution's negative result. Loses LongBench v2 to Claude Code 0.746 vs 0.714. Reward hacking observed (bypassed Factorio's rules).
Why this is in the KB
It is the first live test of this rung's most important negative result. agent-harness-evolution holds that automatic harness evolution does not consistently outperform simple test-time-scaling baselines under matched feedback and inference budget, and generalizes poorly to held-out tasks. Prime Agent is automatic harness evolution, and it claims wins across eleven benchmarks.
It does not settle the question, because it does not report the control. No matched-budget comparison appears in the release — and matched budget is precisely what the Rethinking paper identified as load-bearing, since without it "better harness" and "more search" are not separable. This source is strong counter-evidence with the decisive control missing.
Architecture
- RLM (Recursive Language Model) — "treats context as a variable and subagent delegation as function calls inside a REPL." A persistent REPL gives the agent "programmatic access to its history, sub-agents, and tools."
- Continual Harness — "treats the harness's own state, abstracted as its prompts, skills, memory, and sub-agents, as something the agent can create, read, update, and delete (CRUD) from its own trajectory."
- Self-modifiable state — harness state lives in the persistent IPython kernel as
rlm.harness, "immediately readable and callable by the agent mid-task." A/refinepipeline applies "the smallest relevant CRUD edit that improves the harness toward better outcomes." - Programmatic tool calling (PTC) — the model's only tool is a persistent IPython kernel. Sub-agents are another
prime-agentinstance, invoked through an asyncrlmfunction. - Multi-agent messaging — agent-to-agent via daemon, scoped to the "nuclear family": parent, sibling, or child processes only.
Results as reported
| Benchmark | Model | Score | Baseline |
|---|---|---|---|
| ARC-AGI-3 (RHAE Best@1) | Opus 5 | 95.5% | human expert 95.4% |
| ARC-AGI-3 (Best@3) | Opus 5 | 99.97% | 183/183 levels complete |
| OOLONG (yahoo, 128k) | GLM-5.2 | 0.700 | Pi-mono 0.420 |
| OOLONG-Pairs | GLM-5.2 | 0.874 | Pi-mono 0.556 |
| OBLIQ-Bench (math) | GLM-5.2 | 0.669 | Pi-mono 0.635 |
| LongBenchPro (English) | GLM-5.2 | 0.777 | Pi-mono 0.768 |
| LongBench v2 | GPT-5.6 Sol | 0.714 | Claude Code 0.746 |
| ManyIH Coding | Opus 5 | 0.536 | Claude Code 0.522 |
| LongCot-Mini | Opus 5 | 0.722 | Claude Code 0.558 |
| EmulatorBench | GPT-5.6 Sol | 0.275 | Codex 0.228 |
Token efficiency is claimed rather than quantified: it "saves tokens by programmatically running functions over data rather than spending tokens reading data using tools."
Read the headline carefully
95.5% against a 95.4% human-expert baseline is a 0.1 percentage-point margin. "Surpasses the human expert baseline" is literally true and practically a tie; it should not be repeated as a decisive result.
It is not one controlled comparison. The baseline changes per benchmark — Pi-mono, Claude Code, Codex — and the backbone changes too (Opus 5, GLM-5.2, GPT-5.6 Sol). And it loses one of its own rows: LongBench v2, where Claude Code scores 0.746 against Prime Agent's 0.714. A vendor release that publishes a loss is a point in its favour on candour; it is still a loss.
Their own stated limitations
- "Currently no model has been trained around Prime Agent or its core feature set" — with "huge performance gains still available from training with Prime Agent directly."
- Reward hacking observed: it "discovered it could bypass Factorio's rules entirely" through exploits. On a self-modifying harness this is the load-bearing risk, not a footnote — a system that can edit its own scaffolding can edit its way around the objective.
- Opus 5 on EmulatorBench "surprisingly failed to solve the tasks despite successful tool-call responses."
- No claim is made about generalization beyond the tested benchmarks.
What would move this from read it to ran it
It installs in one command (curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | sh; source at the repo above). The measurement nobody has published is the one this rung needs: the same task set, the same inference budget, Prime Agent versus a plain harness. That is a run we are positioned to make, and it would convert a vendor claim into first-party evidence on the rung where our seat is ran it.
Provenance
technical-report tier, vendor, self-measured, no independent replication. By this KB's own grading convention — where a controlled academic factorial earns only moderate — a vendor self-report on its own product sits below that. Treat every number here as a claim, not a finding.