Reinforcement Learning for Agents

Reinforcement learning represents the third and most advanced paradigm for teaching LLMs to use tools and act as agents. Where prompting relies on frozen models and supervised learning internalizes patterns from examples, RL optimizes agent behavior through reward signals — enabling autonomous adaptation in dynamic, unpredictable environments.

Hu et al. document three RL sub-approaches: strategic tool selection (learning when and which tool to invoke), multi-turn reasoning optimization (improving over extended interaction sequences), and multimodal RL frameworks (extending reward-driven learning to vision-language-action settings).

Wei et al. frame RL as the "post-training reasoning" paradigm — it updates model parameters rather than relying on in-context learning. This makes it more robust and efficient at deployment time, but requires significant training infrastructure and careful reward design to avoid reward hacking.

Key Claims

• RL enables autonomous adaptation in dynamic environments — Unlike prompting or SFT, RL agents can improve behavior from interaction signals. Evidence: strong (Agentic Tool Use in LLMs)
• RL is the "post-training reasoning" paradigm — Updates model parameters, making capabilities persistent rather than context-dependent. Evidence: strong (Agentic Reasoning for LLMs)
• Credit assignment in long tool chains is unsolved — RL struggles to determine which actions in a multi-step sequence led to success or failure. Evidence: strong (Agentic Tool Use in LLMs)

Open Questions

• How to solve credit assignment in lengthy tool chains?
• How to prevent reward hacking in open-ended agent environments?
• Can RL-trained agents maintain alignment while optimizing for task completion?
• How to make RL training sample-efficient enough for practical agent development?

Related Concepts

• Agentic Reasoning — RL is the post-training reasoning paradigm
• LLM Tool Use — RL is the third paradigm for learning tool policies
• Agent Evaluation Benchmarks — Interactive benchmarks measure RL agent performance

Backlinks

Pages that reference this concept:

• Google DeepMind