Skip to header Skip to main navigation Skip to main content Skip to footer
Computernik Networks
Remain at the cutting edge of technology

Main navigation

  • Home

Power and Cooling Limits Push 100,000-Accelerator AI Clusters Into Scale-Across Networking

A Google data center building in Eemshaven, Netherlands, photographed from outside the facility.

The largest artificial intelligence clusters no longer fit comfortably inside a single building, and the network is becoming the reason. On November 20 at Supercomputing 2026, known as SC26, researchers from The Ohio State University described how power, cooling and space limits are pushing AI compute beyond one data center, according to reporting by Tom's Hardware. The emerging answer is a third networking domain called scale-across, which connects buildings, campuses and even regions toward one performance target.

Modern AI clusters typically hold more than 100,000 accelerators and potentially millions of CPU endpoints. Accelerator performance is also growing faster than communication capability, so a faster GPU does not automatically deliver a proportional gain in application performance. Instead, network efficiency increasingly decides how well a cluster uses the compute it has already bought, which is why interconnect design now sits alongside chip design in capacity planning.

The first two domains are familiar. Scale-up fabrics such as Nvidia NVLink, AMD Infinity Fabric, Intel UPI and the emerging UALink tie accelerators inside one tightly coupled system. Scale-out technologies such as Ethernet, InfiniBand, HPE Slingshot and Ultra Ethernet connect racks within one data center. Scale-across extends that high-performance connectivity between sites, allowing operators to combine resources from multiple locations rather than forcing every accelerator onto a site that may not have the power, cooling or physical space to support it.

The transition has already begun. Google introduced its Virgo network architecture earlier in 2026 for AI workloads that exceed one data center's space and power limits, treating an entire campus of facilities as one system with specialized networks for accelerators, servers and data centers. Cisco and Marvell are also developing scale-across technology, including high-capacity routers, switching chips and optical interconnects for distributed clusters. No capacity commitments, shipment schedules or pricing were reported for those efforts, so buyers should treat them as direction rather than a purchasable plan.

The caveats are structural, and the researchers were clear about them. Long-distance physics does not disappear: even optical networks cannot eliminate propagation delay, and congestion, packet loss and synchronization overhead reduce effective performance. Connecting two large GPU clusters with fast links does not automatically make them behave as one tightly integrated system.

Training feels the problem first. Accelerators constantly exchange model parameters, gradients and intermediate results through collective communications, and a delay affecting individual accelerators can drag down the whole cluster. Mitigation is moving into the network itself, with modern network interface cards performing RDMA without continuous CPU involvement, and with fabric offloads and libraries such as libfabric, MPI and UCX shaping real behavior.

The industry already ships 800 Gb/s and 1.6 Tb/s Ethernet, yet peak transfer rates alone do not describe a system. Topology, congestion management, protocols and software cooperation matter just as much. Demand for bandwidth between sites is rising too: IDC survey data published by Cisco, drawn from 293 organizations, found that 91 percent expect between-data-center bandwidth to grow by 11 percent or more in the next year, and 36 percent anticipate growth above 51 percent.

For teams planning AI capacity, the lesson is practical. Campus-scale systems are now a legitimate design target, and power, cooling and network paths need to be weighed together, not port speed alone. A faster link does not guarantee faster training, and cross-site delay and congestion belong in the plan before a multi-site design is signed.

Photo: A Google data center in Eemshaven, Netherlands, photographed by Ymnes, Wikimedia Commons, CC BY-SA 4.0.

Networking
Data Centers
Artificial Intelligence
Cloud Computing
Enterprise IT

Visit Red River New Mexico

Visit Our Vacation Website, Red River New Mexico

Copyright © 2026 Rocky Mountain Madman LLC - All rights reserved

Developed and Designed by Rocky Mountain Madman LLC