Meta details Prometheus, its gigawatt-scale supercluster in Ohio, which will have ~1M GPUs at full scale and was used to train its Muse foundational models

Your coding assistant capabilities can improve as Meta's new AI models trained on this supercluster are released.

Meta has detailed its Prometheus supercluster, an AI infrastructure project in New Albany, Ohio. The first phase of this gigawatt-scale facility is complete and has already been used to train the company's Muse foundational models. Prometheus is designed to eventually house approximately 1 million GPUs and employs advanced technologies like closed-loop liquid cooling for efficiency. The supercluster utilizes multiple generations of NVIDIA GPUs, including the GB200 NVL72 (Codename: Catalina) and the upcoming GB300 NVL72 (Codename: Clemente). Meta has achieved high stability during training runs, with over 85% goodput for pre-training and over 90% for reinforcement learning, and a mean time between interruptions (MTBI) exceeding 47,000 GPU hours, now improved to over 54,000 GPU hours. The infrastructure includes a dedicated, high-bandwidth, low-latency "Back-End" network designed for direct GPU-to-GPU communication to support large-scale AI training workloads.

Meta's Prometheus supercluster represents a significant investment in AI infrastructure, signaling a commitment to building massive, custom-built hardware for frontier model development. The scale, reaching up to 1 million GPUs, and the integration of advanced cooling and networking technologies suggest a strategy focused on maximizing training efficiency and stability for the most demanding AI workloads. The deployment of multiple generations of NVIDIA GPUs, including early adoption of high-performance interconnects, indicates a drive to stay at the forefront of hardware capabilities for AI.

This scale of infrastructure impacts the entire AI ecosystem by increasing the computational power available for training complex models. It suggests a future where larger and more sophisticated AI models become accessible, potentially accelerating advancements in areas like agentic tasks and generative AI. Other cloud providers and hardware manufacturers will likely respond by increasing their own infrastructure investments and pushing innovation in GPU technology and data center design to compete with Meta's capabilities.