NVIDIA Made a CPU.. I’m Holding It. - Grace CPU/Hopper SuperChip @ Computex 2023

NVIDIA showcased the Grace CPU/Hopper SuperChip at Computex 2023, revealing a data center oriented design that centers around ARM-based processing combined with high bandwidth interconnects. The display module at Gigabyte features a Grace SuperChip with 72 ARM cores per CPU and NVLink chip-to-chip interconnect, enabling a 144 core count per node and the possibility to deploy four such modules in a single 2U server chassis for a total of 576 cores. The presentation emphasizes ARM advantages like power efficiency while acknowledging the need for software and ecosystem adaptation for ARM in traditional PC markets, and highlights how data-center customers, including big cloud providers, may migrate to ARM-based architectures to optimize performance per watt. A notable pairing discussed is Grace Hopper, which combines a Grace CPU with an H100 Hopper GPU via NVLink, delivering extremely high CPU-GPU bandwidth and substantial memory on-die through both LPDDR5X for the CPU and HBM3 for the GPU. NVIDIA positions this stack as a path to massive scale, mentioning potential memory and bandwidth advantages that could support multi-GPU deployments and large AI/data-center workloads, while also noting power and cost considerations in accelerated environments. The segment concludes with a nod to traditional GPU configurations and faster networking components like the ConnectX-7 and BlueField-3, underscoring NVIDIA’s strategy to offer both ARM-based accelerators and classic PCIe/HGX options for different customers, and teasing the broader ecosystem around AI, HPC, and data-center acceleration. The video provides concrete figures such as 900 gigabytes per second of CPU-GPU interconnect bandwidth between Grace and Hopper modules, versus 64 GB/s peak for a GPU on a standard PCIe Gen5 lane, and up to 4 TB/s memory bandwidth on the GPU side with HBM3. It emphasizes that HBM memory remains a premium resource, limiting capacity to 96 GB on the H100 in this setup, while still allowing a practical, transparent memory access model via NVLink. The presenter also notes that ARM’s software and ecosystem requirements are non-trivial barriers in consumer markets, but in data centers where developers often optimize software for their own needs, the ARM-based Grace architecture could offer compelling economics and performance for large-scale AI and analytics tasks. The narrative also reflects a broader industry shift toward disaggregated and tightly integrated accelerators, where memory proximity, high bandwidth interconnects, and specialized networking components contribute to overall system efficiency and throughput. Overall, the coverage frames the Grace and Grace Hopper combination as a bold, high-end entry into the data-center race for efficiency and scale, while acknowledging practical considerations like software support, cost, and the ongoing coexistence with traditional x86 configurations. The host commentary also hints at future implications for software licensing and per-core pricing in cloud environments, suggesting that offloading tasks to specialized network and compute resources could reshape licensing models and revenue strategies in large-scale deployments. The piece also teases future iterations and the potential for even denser, more capable GPU/CPU pairings, signaling NVIDIA’s intent to maintain leadership in AI workloads and HPC infrastructure through a hybrid ARM-GPU strategy and advanced interconnects.

Topics · technology · data_center · ai · cpu · gpu · arm · hpc · servers