Recently, an article entitled "Nvidia H100 GPU overseas rental price dropped to $2/hour" spread rapidly in China, and the market also discussed the topics such as "whether the foam of computing power has begun to burst" and "H100 computing power is no longer popular".
Previously, Featherless, an AI inference service provider in the United States AI co-founder Eugene Cheah wrote that he has recently received frequent advertising emails from computing power rental companies, stating that the rental price of a single Nvidia H100 GPU has dropped to about $2/hour, or even lower, nearly halving the market average price of around $5/hour in 2023.
Last year, Nvidia predicted that the price of GPU at $4 per hour would remain unchanged for four years, but it went down in less than a year and a half.
Eugene Cheah stated in the article that large and medium-sized AI modeling companies such as AWS, Meta, and Google have extracted the value of computing power through long-term leasing. At present, unless it is a company that wants to establish a large intelligent computing cluster, do not purchase the brand new H100. "Renting computing power" is a more cost-effective choice.
The market generally agrees with the trend of H100's overseas price reduction, but believes that "$2/hour" or even "$1/hour" is mainly due to individual startup computing rental companies such as Lambda Labs promoting to attract customers, and cannot reflect the average market price level.
When browsing the official website of the Amazon Web Services (AWS) cloud service platform, Interface News reporters found that according to different conditions of short-term and long-term leasing, the latest price of the H100, calculated based on 8 GPUs per server, has two different levels: $12/hour (for long-term leasing contracts) and $5/hour (for long-term leasing contracts). The price of similar products from another cloud vendor, Google Cloud, is also around $10.
A domestic industry insider engaged in AI computing rental business told Interface News reporters that the logic behind the overseas H100 price reduction is easy to understand - with Nvidia's new GPU products H200 and B200 starting to be launched this year, the new products have stronger performance, and the average cost of H100 computing power is relatively lower. The original old products naturally need to be reduced in price, and the difference lies only in the magnitude and speed of the price reduction. According to his understanding, a price range of $5 to $8 per hour better represents the current price level of mainstream overseas platforms and is also in line with Nvidia's previously predicted product price trend.
After Nvidia's new products started to be launched and supplied recently, the market response remains enthusiastic.
The CEO of the company, Huang Renxun, revealed at a seminar this month that the B200 GPU has recently started mass production and delivery, and is favored by customers. All Blackwell architecture GPU orders for the next 12 months have been sold out, and any new customers will need to wait until 2025 to receive the product.
The situation in China is different from overseas because Nvidia's high-end graphics cards are banned, making it difficult to get new products and taking a different path. The above-mentioned person believes that overseas price reductions have almost no impact on China. At present, the biggest problem in the domestic computing rental market is still the supply-demand imbalance. "Domestic computing resources are extremely scattered, and most of the time sellers cannot find buyers, and buyers cannot find sellers
The reason for this is that the total supply of computing power resources in China is currently limited, making it impossible to achieve on-demand allocation.
According to Interface News reporters, in addition to AI GPU H100 and A100, there are also Nvidia's consumer GPU product 4090 and domestic AI computing power from different manufacturers used for training AI models in China.
At the same time, domestic companies engaged in computing power leasing are mixed, and there is a lack of unified standards for product services and prices. There are few companies like AWS and Google Cloud overseas that can provide standardized leasing services to customers.
Several market insiders have also told Interface News reporters that there have been fluctuations in server prices for domestic computing resource leasing this year. A H100 server, with a market price of around 120000 yuan per year at the beginning of the year, is now priced at approximately 70000 yuan.
The CEO of a technology company that has participated in the construction of an intelligent computing center by a local government in China mentioned that because the computing resources held by Internet giants such as ByteDance, Ali, and Tencent are mainly used by their own big models, few of them can provide leasing services to the public market. The vast majority of vendors engaged in computing power leasing in the market now sell server hardware and cannot provide standard services and unified pricing like cloud computing vendors did in the past.
Most of these computing power rental service providers hoarded a certain amount of AI server spot due to the surge in computing power demand last year, and then speculated on computing power hardware as' futures'. In order to ensure hardware cost recovery, they rarely have the flexibility to provide services based on hourly pricing. Many orders have to be rented for one year or even longer, which is a considerable cost. "This CEO believes that the main impact of price cuts in the domestic market is on these 'speculators', whose hardware assets are depreciating.
According to two sales personnel of AI servers, the current small number of H100 servers circulating through non-public channels in China have a spot price of around 2.4 million to 2.5 million yuan per unit, which has decreased compared to the selling price of nearly 3 million yuan last year.
In the opinion of the CEOs of the above technology companies, it is too early to predict the "bursting of the foam of computing power" through the price fluctuation of H100 alone.
In terms of supply, compared to overseas computing giants such as Meta, Microsoft, and Tesla, which already have hundreds of thousands of H100 GPUs and are still increasing their purchases, the total amount of computing power in China is limited, and various regions are still accelerating investment in building intelligent computing centers. The government's investment direction for computing power construction this year still advocates for "moderately ahead of schedule" to increase supply.
From a demand perspective, whether it is AI model training or inference, as well as supporting traditional enterprises to explore business transformation through AI, advanced computing resources have always been a "hot commodity" in the market.
There are still very few customers in the market who have the resources and strength to build computing power centers. The large number of customers we have contacted this year are extremely eager for affordable, stable, and on-demand computing power, "said the CEO.