Huang Renxun has upgraded his "equipment" again. On November 14th, a reporter from Daily Economic News learned from NVIDIA that on November 13th local time, NVIDIA announced the launch of NVIDIA HGX H200 (AI chip model, hereinafter referred to as "H200"). It is reported that H200 is the first to use HBM3e GPU (memory, faster and larger than before), further accelerating generative AI and large language models, while promoting scientific computing for HPC (high-performance computing) workloads. It can provide 141GB of display memory with a transmission speed of 4.8 TB/s, nearly doubling the capacity and bandwidth compared to the previous generation architecture of NVIDIA A100.
In the view of Ian Buck, Vice President of NVIDIA's Ultra Large Scale and High Performance Computing, in order to create intelligence through generative AI and HPC applications, it is necessary to use large, fast GPU graphics memory to process massive amounts of data quickly and efficiently. When H200 is used in conjunction with NVIDIA Grace CPUs using ultra fast NVLink C2C interconnect technology, it forms the GH200 Grace Hopper superchip with HBM3e - a computing module designed specifically for large-scale HPC and AI applications.
From the perspective of specifications, H200 will provide options for four and eight way H200 server motherboards, which are compatible with the hardware and software of the HGX H100 system; It can also be used for the NVIDIA GH200 Grace Hopper superchip using HBM3e released in August this year. These configurations enable H200 to be deployed in various data centers, including local, cloud, hybrid cloud, and edge; It can provide the highest performance for various application workloads, including LLM training and inference for super large models with parameters above 175 billion.
In terms of architecture, compared to the previous generation, the NVIDIA Hopper architecture has achieved a performance leap, such as nearly doubling the inference speed on a 70 billion parameter LLM - Llama 2 compared to the H100 (NVIDIA AI chip).
According to NVIDIA, H200 will be available through global system manufacturers and cloud service providers starting in the second quarter of 2024; Server manufacturers and cloud service providers are also expected to start offering systems equipped with H200 at the same time.