The highly anticipated GTC developer conference of AI chip giant Nvidia is about to be held, and the global trend of AI computing power is receiving attention.
As UK chip architecture company Arm continues to focus on the server market and recently updated its product roadmap for the Arm Neoverse series of server processors, two new Arm Neoverse computing subsystems (CSS) based on the all-new third-generation Neoverse IP have been launched. The outside world will also have a glimpse of the next generation of AI "super chips" that integrate CPUs and GPUs, and whether Nvidia will follow suit will also be closely monitored.
Neoverse is a server processor brand launched by Arm in 2018 for the data center market. Under Arm's planning, Neoverse's N series, V series, and E series each have their own positioning. For example, the V series emphasizes performance first and is used in the high-end server market. The previous generation Neoverse V2 was used in Nvidia's AI chip design.
Last March, Nvidia launched its first "Grace Hopper" GH200 superchip that combines CPU and GPU packaging. "Grace" refers to Nvidia's data center Arm CPU series released in April 2021, while "Hopper" refers to Nvidia's latest architecture GPU production model H100.
A chip industry investor told Interface News that Nvidia's Grace Hopper chip combines CPUs with top AI training products (GPUs) to create a "super chip" and jointly build a complete AI solution.
GH200 can be used for AI training and inference, and Nvidia significantly improves data transmission efficiency between CPUs and GPUs by packaging one CPU and one H100 GPU into a single chip. In November of the same year, Nvidia upgraded the GH200 again, upgrading the 96GB capacity HBM3 memory equipped on the GPU in the GH200 to 144GB HBM3e, significantly improving data transmission efficiency once again.
In the process of Nvidia seizing the AI wave with its GPU products, Arm also benefits from Nvidia's strong position in AI computing, which means that the data center market may adopt more processors based on Arm technology.
Mohammed Awad, General Manager of Arm's Infrastructure Business Unit, explained to Interface News that Nvidia's previously launched Grace Hopper Superchip has redesigned the system architecture. In the past, data centers used a single CPU to manage multiple GPUs, while Grace Hopper chips have been transformed to correspond to only one GPU per CPU. "More CPUs mean memory consistency, which ultimately greatly improves GPU utilization."
Arm stated that as the industry's demand for AI computing power gradually shifts from training to inference, CPU inference will be a key component of generative AI computing applications.
But not all AI processing will be performed on the CPU. Dermot O'Driscoll, Vice President of Product Solutions for Arm Infrastructure Business Unit, cited Grace Hopper as an example, stating that Nvidia's important innovation in this chip lies in memory capacity and shared memory mode. This tightly coupled CPU design, coupled with the configuration of AI accelerators, is very beneficial for the current popular large parameter language models and other AI applications.
In order to make custom chips faster and reduce design difficulty, Arm launched Arm Neoverse CSS last year. In Neoverse CSS, Arm configures, optimizes, and verifies the complete computing subsystem, and configures it for various computing cases. Partners focus on software tuning, customization acceleration, and other work, which can also accelerate product launch time and reduce engineering costs.
Dermot O'Driscoll pointed out that Neoverse CSS is a product launched specifically to help customers quickly build general-purpose computing chips on the Arm CPU platform. It can provide all the interfaces that customers need to choose the accelerator that couples itself. This method can provide both CPU and AI accelerators when needed, achieving the best of both worlds.
Nvidia has always played down its competitive edge with Intel and AMD for its self-developed Arm architecture Grace CPU.
Huang Renxun once told Interface News reporters in 2021 that the vast majority of data centers will continue to use existing x86 CPUs, while Grace will mainly be used in large data intensive sub markets in the computing field and will not have a "game changing" impact on existing CPU manufacturers.
However, the market landscape has changed. In the data center market, Arm is gradually gaining a foothold and posing a challenge to the giants Intel and AMD.
According to a report by market research firm Counterpoint, Arm architecture servers earned over $1 billion in revenue in the data center market for the first time in 2022, with AWS self-developed chips accounting for 3.16% of the market share and Ampere accounting for 1.52%. With the deployment of Microsoft's self-developed Arm chip in 2023 and the shipment of Grace Hopper, it is expected that Arm's market share in the server market will continue to rise.