Static

Remember the frontier AI pause? Chip companies don’t either

First reported by The Verge ·

The signal ●○○○ Compiled by AI from The Verge, the single source so far
Why you might care

Your cloud AI costs may increase as falling token prices are offset by rising GPU rental prices.

What happened

Despite a brief summer pause prompted by security concerns around advanced AI models, the demand for AI chips has surged, defying expectations of a slowdown. OpenAI, which initiated the pause, resumed its largest training runs within days, while Anthropic released upgraded models. This activity has fueled record growth for chipmakers like Nvidia and AMD. Nvidia reported a blockbuster quarter with data center revenue up 110% year-over-year, driven by its new Vera Rubin NVL72 system, which is seeing rapid adoption by major cloud providers and AI labs. AMD also achieved its largest quarter ever, with data center sales doubling, supported by early customers using its Helios systems. While a significant slowdown in AI training could take months to impact earnings due to existing contracts, orders for chip manufacturers may reflect changes sooner. However, the growth in inference, the process of running existing AI models, is expected to offset any decline in training demand, potentially increasing the overall need for compute infrastructure.

What it means

The market is reallocating spend away from frontier model training toward inference, a shift that nonetheless increases demand for AI accelerators. While frontier model families represented a record 60% of business token usage in early November, intense competition among AI providers is driving down token prices, even as GPU rental prices climb. This dynamic means that companies like Nvidia, despite the growing need for their hardware, may face pressure on profit margins as AI labs increasingly opt for custom silicon to optimize their inference workloads.

The increasing reliance on custom AI chips, predicted to surpass data center GPUs by 2028, signals a significant long-term challenge to established chip giants. While Nvidia and AMD are positioning their powerful chips for both training and inference, the economic imperative for AI companies to serve millions of users cheaply is driving a move towards bespoke solutions. This trend suggests that future growth for chipmakers might depend less on selling general-purpose GPUs and more on supplying components for custom designs or maintaining leadership in the custom chip manufacturing ecosystem.

AI-written summary. May contain errors.