ROSA: A Robotics Foundation Model Serving System for Robot Factories

Abstract

Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing systems are largely shaped by a single-robot, single-model assumption: inference is treated as an edge-computing problem handled by an on-robot or dedicated nearby GPU, and the serving objective is to minimize the latency of a single action model. In this paper, we propose ROSA, an RFM serving system for robot factories designed around three key principles. First, ROSA adopts shared GPU-pool serving, allowing a fleet of robots to access powerful server-class GPUs over the network in order to improve inference performance, battery duration, and GPU utilization. Second, ROSA provides a robotics-aware programming abstraction and system design that supports multi-model pipelines, per-task performance requirements, and failure handling. Third, ROSA uses factory-objective-driven scheduling to maximize SLO-qualified factory productivity rather than minimizing individual request latency. We implement ROSA on top of Ray Serve for distributed orchestration, with vLLM, PyTorch, and JAX as model-serving backends, and evaluate it on both real robots and synthetic large-scale workloads. The results show that ROSA improves factory productivity by up to 12.06x over conventional dedicated serving systems.

Key Contributions

• Reframes robot inference serving from a per-robot edge-computing problem to a fleet-level, shared-GPU-pool serving problem — the same "disaggregate compute from the physical unit" logic that transformed data-center LLM serving, now applied to physical robot fleets.
• A robotics-aware programming abstraction supporting multi-model pipelines (a robot task may chain several models — perception, planning, action-generation), per-task performance/latency requirements, and failure handling — none of which fit cleanly into generic LLM-serving abstractions.
• Factory-objective scheduling: optimizes for aggregate SLO-qualified factory productivity rather than minimizing any single robot's request latency — a scheduling objective specific to multi-robot industrial deployment rather than single-user inference.
• A working implementation atop Ray Serve (orchestration) with vLLM, PyTorch, and JAX as interchangeable serving backends, evaluated on both physical robots and synthetic large-scale fleet workloads.

Methodology

ROSA is built on Ray Serve for distributed orchestration, using vLLM, PyTorch, and JAX as pluggable model-serving backends. It pools server-class GPUs and serves multiple robots' inference requests from that shared pool over the network rather than dedicating a GPU per robot. The system's scheduler is factory-objective-driven, meaning it optimizes for overall throughput/productivity subject to per-task service-level objectives (SLOs), rather than treating every request as latency-critical in isolation. Evaluation spans real physical robots and synthetic workloads modeling large robot-fleet scale.

Results

• Up to 12.06x improvement in factory productivity compared to conventional dedicated (per-robot/edge) serving systems.
• Reported gains span inference performance, robot battery duration (less onboard compute draw), and GPU utilization — i.e., the shared-pool model improves multiple resource dimensions simultaneously, not just raw throughput.

Limitations

Not explicitly detailed in the fetched abstract; the approach's real-world value depends on factory network reliability/latency between robots and the shared GPU pool, a dependency the abstract does not quantify (e.g., failure modes if network connectivity to the pool is interrupted are not characterized here).

Full Content

ROSA matters to the embodied-AI-as-compute-demand thesis in a very direct way: it is the clearest evidence yet that robotics foundation models are becoming large/expensive enough that inference serving infrastructure — not just training compute — is now a first-class systems-research problem, mirroring the LLM-serving literature (vLLM, disaggregated prefill/decode, etc.) one step removed. The author list (Shuran Song — Stanford robotics; Yashraj Narang, Jason Clemons, Hugo Hadfield, Alperen Degirmenci — NVIDIA; Christos Kozyrakis — Stanford systems) signals this is an academic-systems paper with direct NVIDIA involvement, not vendor product marketing. For MenFem's broader inference-economics coverage, this is a concrete instance of the "compute demand moves downstream, from training runs to physical deployment fleets" pattern — every robot in a ROSA-style factory becomes a persistent inference client rather than a one-time training data point.

Source: ROSA: A Robotics Foundation Model Serving System for Robot Factories by Wenqi Jiang et al.