Groq

Type: Organization (AI accelerator vendor)

Groq appears on this rung as the counterexample that breaks aggregate benchmarking. In a peer-reviewed conference evaluation (HPAI4S'26, co-located with IEEE IPDPS 2026) measuring prefill and decode separately on a common model, GroqRack achieves significantly lower TPOT (time per output token) during decode, while GPUs consistently win prefill.

That inversion is the finding. Prefill is compute-bound and decode is memory-bound — they are different problems, and a single tokens-per-second figure averages across an inversion rather than describing it.

Key contributions to this rung

• Demonstrates that accelerator advantage is phase-dependent, not global (Prefill/Decode-Aware Evaluation)
• Provides the empirical basis for treating prefill and decode as separately procurable (Prefill/Decode-Aware Evaluation)

The caveat that limits it

Batching is not currently supported on that platform. Since the rung's own owned number — the 44x cost spread — is produced entirely by batching, a decode-phase win measured without batching cannot be carried across to a cost claim. The comparison is real and the inversion is real; the economic conclusion does not follow yet.

Mentioned in

• Prefill/Decode Disaggregation — the empirical case for splitting the phases
• Frontier — cited under phase-aggregated benchmarking