The AI chip race is shifting to inference, not training

A panel of executives from DeepInfra, Abacus Semiconductor, MemVerge and Positron explores why the competitive focus in AI chips is moving from training to inference—the process of running deployed models. The discussion covers the economics of serving AI queries, including cost per token, memory bandwidth constraints, and the specialized chip designs emerging to handle these workloads.

The speakers also address manufacturing bottlenecks, memory requirements for AI agents, and the cloud infrastructure needed to scale inference. The conversation bridges the gap between prototype development and real-world deployment, highlighting how inference efficiency is becoming the limiting factor for AI companies trying to operate profitably at scale.