Enlarge / NVIDIA’s GH200 “Grace Hopper” AI superchip.
Nvidia
Early final week at COMPUTEX, Nvidia introduced that its new GH200 Grace Hopper “Superchip”—a mixture CPU and GPU particularly created for large-scale AI purposes—has entered full manufacturing. It’s a beast. It has 528 GPU tensor cores, helps as much as 480GB of CPU RAM and 96GB of GPU RAM, and boasts a GPU reminiscence bandwidth of as much as 4TB per second.
We’ve beforehand coated the Nvidia H100 Hopper chip, which is at the moment Nvidia’s strongest information middle GPU. It powers AI fashions like OpenAI’s ChatGPT, and it marked a big improve over 2020’s A100 chip, which powered the primary spherical of coaching runs for most of the news-making generative AI chatbots and picture mills we’re speaking about immediately.
Sooner GPUs roughly translate into extra highly effective generative AI fashions as a result of they will run extra matrix multiplications in parallel (and do it quicker), which is important for immediately’s synthetic neural networks to operate.
The GH200 takes that “Hopper” basis and combines it with Nvidia’s “Grace” CPU platform (each named after pc pioneer Grace Hopper), rolling it into one chip via Nvidia’s NVLink chip-to-chip (C2C) interconnect expertise. Nvidia expects the mix to dramatically speed up AI and machine-learning purposes in each coaching (making a mannequin) and inference (working it).
Commercial
“Generative AI is quickly remodeling companies, unlocking new alternatives and accelerating discovery in healthcare, finance, enterprise companies and plenty of extra industries,” stated Ian Buck, vice chairman of accelerated computing at Nvidia, in a press launch. “With Grace Hopper Superchips in full manufacturing, producers worldwide will quickly present the accelerated infrastructure enterprises wanted to construct and deploy generative AI purposes that leverage their distinctive proprietary information.”
In line with the corporate, key options of the GH200 embody a brand new 900GB/s coherent (shared) reminiscence interface, which is seven occasions quicker than PCIe Gen5. The GH200 additionally provides 30 occasions larger mixture system reminiscence bandwidth to the GPU in comparison with the aforementioned Nvidia DGX A100. Moreover, the GH200 can run all Nvidia software program platforms, together with the Nvidia HPC SDK, Nvidia AI, and Nvidia Omniverse.
Notably, Nvidia additionally introduced that will probably be constructing this combo CPU/GPU chip into a brand new supercomputer known as the DGX GH200, which may make the most of the mixed energy of 256 GH200 chips to carry out as a single GPU, offering 1 exaflop of efficiency and 144 terabytes of shared reminiscence, almost 500 occasions extra reminiscence than the previous-generation Nvidia DGX A100.
The DGX GH200 will likely be able to coaching big next-generation AI fashions (GPT-6, anybody?) for generative language purposes, recommender methods, and information analytics. Nvidia didn’t announce pricing for the GH200, however in accordance with Anandtech, a single DGX GH200 pc is “simply going to price someplace within the low 8 digits.”
General, it’s cheap to say that because of continued {hardware} developments from distributors like Nvidia and Cerebras, high-end cloud AI fashions will possible proceed to turn out to be extra succesful over time, processing extra information and doing it a lot quicker than earlier than. Let’s simply hope they don’t argue with tech journalists.