Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents

Abstract (verbatim)

"Large language models deployed as autonomous agents for enterprise workflows face a key challenge: verbose tool responses from enterprise systems can cause context overflow, stale-state errors, and high inference cost. We study this problem in automated expense itemization in Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools. We evaluate four GPT-5 configurations on a 50-task hotel expense benchmark: no user model, full conversation history, context pruned to the last 5 tool call/response pairs, and pruning with automated summarization. Results are averaged across 5 independent runs, with the user model held constant for the context-engineering comparison. The no-user-model baseline achieves only 8.0% complete itemization. Full-context retention improves completion to 71.0%, but consumes 1,480,996 tokens and 14.56 hours per benchmark. Pruning to the last 5 tool calls improves completion to 79.0% while reducing token use to 535,274 and runtime to 5.39 hours. Adding summarization achieves the best result: 91.6% complete itemization and 99.64% average amount itemized, with 553,374 tokens and 5.79 hours. We further report confidence intervals, effect-size analysis, sensitivity over pruning and summary windows, failure analysis, results across five expense types grouped into three categories, and cross-model evidence with Claude Sonnet 4.5. These results show that, for this class of enterprise tool-use workflow, selective retention of recent tool interactions plus compact summarization can improve both reliability and efficiency compared with full-history retention."

Key contributions

1. The counter-intuitive result stated cleanly: in long-horizon tool use, keeping everything is both the expensive option and the less reliable one. The usual framing — context is a cost you pay for accuracy — inverts here.
2. A named mechanism, not a leaderboard. Verbose enterprise tool responses cause context overflow and, critically, stale-state errors — the agent acts on a tool response that a later call has already superseded. Pruning removes the stale state along with the tokens, which is why reliability rises rather than merely holding.
3. Summarisation recovers what pruning loses. Pruning alone (79.0%) beats full history (71.0%); pruning plus a compact summary (91.6%) beats both at essentially the same token count as pruning alone (553k vs 535k, +3.4%). The summary is nearly free and worth 12.6 points.
4. Cross-model evidence with Claude Sonnet 4.5, so the finding is not a GPT-5 artefact.

Methodology

• Task: automated expense itemization in Microsoft Dynamics 365 Finance and Operations, driven through Model Context Protocol tools.
• Benchmark: 50 hotel-expense tasks; results averaged over 5 independent runs.
• Four configurations, user model held constant across the three context arms:
  1. no user model (ablation baseline),
  2. full conversation history,
  3. context pruned to the last 5 tool call/response pairs,
  4. pruning + automated summarisation.
• Also reported: confidence intervals, effect sizes, sensitivity over pruning and summary window sizes, failure analysis, and a breakdown across five expense types in three categories.

Results

Derived: full history costs 2.68x the tokens of the best configuration and lands 20.6 points worse on completion. Runtime is 2.5x. There is no axis on which full retention wins.

Limitations

• One workflow, one vendor system. Dynamics 365 hotel-expense itemization is a single enterprise task shape; the abstract claims generality only "for this class of enterprise tool-use workflow." Whether the inversion holds for research, coding or open-ended agent work is not shown.
• 50 tasks × 5 runs is small, though confidence intervals and effect sizes are reported.
• The "last 5 tool calls" window is tuned by sensitivity analysis on this same benchmark — the optimal window is workflow-specific, not a constant to copy.
• No explicit limitations section on the abs page; the above are read off the scope of the design.

How it bears on MenFem

• ai T1 — "agentic reasoning will consolidate around a standard stack." If context policy is worth 20 completion points and 2.68x the token bill, context management is a harness responsibility, not a prompt-authoring habit — and that is a reason stacks converge.
• Pairs with the Harness Effect paper (same rung, ingested same day): that one prices orchestration in dollars per task; this one names one of the six mechanisms — cache/context discipline — and measures it alone.
• Teaching moment: "more context, worse agent" is a complete counter-intuitive unit with a mechanism attached (context rot and stale state under long horizons). It survives the fifth-grader test: the agent gets confused by its own old notes, and the notes cost money.