LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation

Abstract

LoopFormer introduces a looped Transformer that supports budget-conditioned reasoning through variable computational depth. A looped Transformer repeatedly applies the same transformer layers — iterating in weight-space rather than producing explicit chain-of-thought tokens. Prior looped models fixed the iteration count at training time, preventing flexible adaptation to different inference budgets. LoopFormer's central contribution is a shortcut-consistency training scheme that aligns the trajectories of different loop lengths, so a single trained model can run with elastic depth at inference and degrade gracefully under tight compute.

Key Claims

• Looped Transformers have an inductive bias toward latent reasoning — iterating layers internally is a form of in-weights deliberation, parallel to (but distinct from) token-level CoT.
• Elastic depth solves the fixed-iteration limitation. The model adjusts loop count at inference based on a compute budget, instead of being locked to the training-time depth.
• Shortcut modulation makes this stable. Each loop is conditioned on the current time and step size, so representations "evolve consistently across trajectories of varying length" — different-length runs converge to compatible states.
• Latent vs. explicit reasoning. The looping mechanism mirrors CoT's step-by-step refinement but operates within model weights rather than as an explicit token sequence — relevant to the broader "reasoning is latent, not the chain of thought" debate.

Results

• Reports robust performance on language-modeling and reasoning benchmarks even under aggressive compute constraints, while scaling gracefully with additional budget. (Specific benchmark numbers are not given in the abstract; venue: ICLR 2026.)

Significance

LoopFormer is part of a 2026 cluster of looped / recursive / latent-reasoning architectures (alongside SpiralFormer, SpiralThinker, MeSH, and memory-efficient looped transformers) arguing that reasoning depth should be an adjustable inference-time axis in weight-space — not something paid for purely in CoT tokens. Combined with the latent-reasoning critique (arXiv 2604.15726), it points at a research direction where deliberation is decoupled from visible token output, with direct implications for interpretability and CoT-based safety auditing.

Source: arXiv:2602.11451 — LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation, Jeddi, Ciccone & Taati, 11 February 2026 (ICLR 2026).