Astera Labs, Inc. announced Orion: a modular rack-scale reference design with production-grade connectivity that allows customers to choose their own GPU and CPU. To be offered through Astera Labs’ ODM and OEM partners, the Orion production-grade reference designs are expected to help enable neocloud and enterprise customers to deploy AI infrastructure faster and run it more efficiently, turning racks built on open standards into token-efficient AI factories.
For neoclouds and enterprises to economically serve inference at scale, they must secure sufficient accelerator and CPU capacity while delivering predictable, low-latency performance across increasingly complex AI infrastructure. But because they must commit to that capacity long before they know which accelerator will be available or affordable for their workloads, tying early investment to a single platform carries unnecessary cost and risk. At the same time, ODMs and OEMs are forced to repeatedly solve complex integration, validation, and software challenges for every new accelerator platform—slowing deployments when inference capacity is in critical demand.
Orion addresses these challenges with a production-grade reference architecture that decouples rack-scale designs from accelerator and CPU selection. Built on open standards to support industry-leading inference accelerators and CPU ecosystems, Orion is designed to enable operators to deploy infrastructure once and scale inference workloads across the most cost-effective and readily available compute resources—via PCIe plug-in cards—without redesigning racks as technology and market dynamics evolve. At the core of Orion is a high-performance, standards-based, scale-up fabric engineered to connect up to 160 accelerators per pod and deliver predictable, low-latency communication and expanded memory capacity across racks and pods. Orion’s optimized PCIe connectivity and composable infrastructure are additionally designed to help accelerators extract more tokens-per-second across AI inference workloads. Orion is delivered as a complete modular rack-scale reference design—including system configurations for the compute tray, fabric tray, memory tray, link board, and compute board—all integrated with COSMOS Rack Manager and COSMOS Fabric Manager orchestration software for management, fleet-wide telemetry, diagnostics, and AI infrastructure intelligence.
Orion rack reference and system designs are on track for availability through Astera Labs’ OEM and ODM partners beginning in Q1 2027. Astera Labs will showcase a powered Orion rack—featuring interchangeable, pre-validated compute trays with different CPU and accelerator configurations at the 2026 Open Compute Project (OCP) Global Summit, taking place October 12-15 in San Jose, California.
For more information, visit asteralabs.com.
The post Astera Labs unveils modular Orion AI rack architecture appeared first on Engineering.com.