The second day of the Hot Chips conference underscored a shift in artificial intelligence infrastructure: networking, not raw compute, is now the primary constraint on system performance as AI models expand in size and complexity.
Citi analysts highlighted that while early AI development focused on faster graphics processing units (GPUs) and larger accelerators, the ability to move data efficiently between chips, servers, racks and data centers has emerged as the greater challenge. Models approaching trillions of parameters, larger context windows and autonomous agent workloads generate exponentially more data traffic, overwhelming traditional interconnects.
The bottleneck is evident in the industry’s response. Nvidia is promoting its "AI factories" concept, a data center architecture designed around optimized data flows linking compute, memory, networking, storage and security, rather than the traditional server-centric model. The approach reflects a broader industry pivot toward integrated systems where bandwidth and latency, not just processing power, determine overall performance.
Memory is increasingly part of the networking challenge. Samsung, XCENA and Cerebras presented technologies aimed at keeping data closer to where it is processed, reducing the volume that must traverse congested interconnects. Every byte retained locally lowers communication overhead, improving efficiency in large-scale deployments.
Industry participants are also rethinking physical-layer infrastructure. Lumentum, Coherent, Marvell Technology and Astera Labs are among companies developing optical and electrical components to support higher data rates and lower latency across data center networks. Arista Networks and Broadcom are supplying high-speed switches and silicon to meet rising demand for low-latency connectivity.
Citi analysts emphasized the strategic implications in a recent note: "The future AI winner is not necessarily the company with the fastest processor, but the company that can move data most efficiently throughout the entire system." The observation underscores how AI performance is increasingly tied to the quality of the underlying networking stack, from chip-to-chip links to data center-scale fabrics.
The trend has drawn attention from major AI developers. Meta Platforms and Alphabet’s Google, among others, are reportedly evaluating next-generation networking solutions to support their expanding model training and inference workloads. Samsung’s participation in the conference signals its push into AI-optimized memory and interconnect technologies, while Cerebras continues to advocate for wafer-scale integration to minimize data movement.
