Harnesses & Agent Systems
The system around the model — the rung that turns a token price into a TASK price. In scope: scaffolding and orchestration, tool use, context and memory management, retries and recovery, agent evaluation and benchmarks, failure modes at step three, agent security and the exploitation surface, and cost-per-completed-task. The load-bearing finding this topic already holds: for long-horizon agentic work the harness is often a stronger performance determinant than the model. Connor's seat here is `ran it` — he runs desks, subagents and workflows daily, which is the standing the teaching doctrine requires for a teaching-level claim.
Agentic Reasoning for Large Language Models
Three-layer framework for agentic reasoning: foundational, self-evolving, multi-agent
From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review
Unified taxonomy of ~60 benchmarks, agent framework comparison, collaboration protocols survey
Agentic Tool Use in Large Language Models
Unified evolutionary framework for LLM tool use: prompting, supervised, RL paradigms
Agentic AI Security & Autonomous Red-Teaming
Red-teaming framework for agentic AI: permission escalation, hallucination, orchestration flaws, memory manipulation, supply chain
Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers
Write-manage-read taxonomy, 5 mechanism families, three-dimensional taxonomy for agent memory
A-MEM: Agentic Memory for LLM Agents
Agentic memory with Zettelkasten-inspired note construction, dynamic linking, memory evolution
Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities
Large-scale systematic taxonomy of LLM agent exploitation triggers across 12 attack dimensions, identifying goal reframing as the sole reliable trigger while ruling out nine others, with GPT-4.1 achieving complete immunity across 1,850 trials.
Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw
First real-world safety evaluation of a deployed personal AI agent (OpenClaw), introducing the CIK taxonomy and showing that poisoning any single dimension raises attack success rate from 24.6% to 64–74%.
The Amazing Agent Race: Strong Tool Users, Weak Navigators
DAG-structured benchmark of 1,400 Wikipedia navigation tasks revealing that current best agents achieve only 37.2% accuracy with navigation errors dominating (27–52% of failures), exposing compositional reasoning as the primary frontier bottleneck.
Model-First Reasoning LLM Agents: Reducing Hallucinations through Explicit Problem Modeling
Two-phase reasoning: LLMs construct explicit problem models before generating solutions. Reduces constraint violations vs CoT and ReAct across five planning domains.
Rethinking the Evaluation of Harness Evolution for Agents
Automatic harness evolution for LLM agents does not consistently outperform simple test-time-scaling baselines under matched feedback/inference budget, and generalizes poorly to held-out tasks (Terminal-Bench 2.1, GPT-5.4 + Claude Opus 4.6) — a methodological check on 'scaffold gains' claims in the agentic-harness literature.
Stop Comparing LLM Agents Without Disclosing the Harness
Formalizes the Binding Constraint Thesis — for long-horizon agentic tasks across comparable-capability models the execution harness is often a stronger performance determinant than the model — and measures harness-induced variance at 7.80x model-induced variance (18.48 vs 2.37 pp^2, 6/9 ranking reversals) in a 3-model x 3-harness SWE-bench Verified experiment; proposes an ETCSOVG Harness Card disclosure standard + variance-decomposition protocol. Directly on the MenFem 'edge is the harness' thesis.
Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
Diagnostic benchmark (106 sandboxed offline tasks, 5,194 execution trajectories) that isolates configuration-level harness effects from model capability by fixing task/budget/eval and varying only the harness across model backends; finds substantial variation in completion, quality, efficiency, and failure behavior, and names execution-alignment decoupling as the dominant failure class. Empirical complement to Stop-Comparing.
Measuring Harness-Induced Belief Divergence in Multi-Step LLM Agents
Shows the harness changes an agent's multi-step beliefs (progress, risk, recoverability, constraints, failure mode, uncertainty, future success, repair cost, next action) even with task/env/model fixed; introduces a belief-rollout diagnostic, a cross-harness belief-divergence metric split into arrival (interface) + growth (horizon) terms, and BIWM (no-training trajectory alignment). Terminal success often preserved while decision-driving beliefs diverge. Freshest of the July harness cluster.
AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
Gemini-powered evolutionary coding agent; 0.7% Google compute recovery; math breakthroughs