On March 19th, Nvidia released Project GR00T, a universal basic model for humanoid robots, aimed at further advancing its breakthroughs in robotics and embodied intelligence.
The GR00T driven robot will be able to understand natural language and mimic actions by observing human behavior. It can quickly learn and coordinate various skills, and has better flexibility to adapt to the real world and interact with it.
"Developing a basic model for universal humanoid robots is one of the most exciting topics in the field of AI today. Robot technology leaders from around the world are gathering various enabling technologies to achieve breakthroughs in the field of artificial universal robots," said Huang Renxun, founder and CEO of NVIDIA.
In his keynote speech at the GTC conference, he demonstrated through video how several such robots complete various tasks.
This includes robotic arms used for medical and logistics handling and sorting, as well as humanoid household robots that can do household chores such as juice making. Some robots can also mimic human drumming and learn human dance movements from videos for simulation. Based on the big language model, some humanoid robots can also understand human verbal commands and respond.
After showcasing the video, Huang Renxun and a row of robots driven and trained by NVIDIA chips and services behind him greeted the audience, creating a sci-fi atmosphere.
Huang Renxun's keynote speech at the GTC conference
Nvidia has also released a new humanoid robot computer, Jetson Thor, based on the NVIDIA Thor system level chip (SoC), and has made significant upgrades to the NVIDIA Isaac robot platform, including generative AI basic models and simulation tools, as well as AI workflow infrastructure.
This SoC includes a next-generation GPU with a transformer engine, which adopts the NVIDIA Blackwell architecture and can provide 800 trillion 8-bit floating-point AI performance per second to run multi-modal generative AI models such as GR00T. With integrated functional security processors, high-performance CPU clusters, and 100GB Ethernet bandwidth, design and integration work are greatly simplified.
Training embodied intelligence models requires massive amounts of real and synthetic data. The new Isaac Lab is a lightweight application with GPU acceleration and performance optimization, built on Isaac Sim, specifically designed to run thousands of parallel simulations for robot learning.
Many robots undergo extensive training in Isaac Lab and learn how to complete tasks. Orange and Green, small robots produced by Disney that use related NVIDIA chips, also arrived at the scene. Huang Renxun introduced that they learned to walk on the Isaac platform.
Disney robot Green who arrived at the speech venue
Nvidia is developing a comprehensive AI platform for humanoid robotics companies such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, and more.
"The next generation of robots is likely to be humanoid robots, as we have more imitation training data to provide to robots with similar structures. Humanoid robots can be more efficiently deployed in human designed workstations, manufacturing, and logistics processes," said Huang Renxun.
With the intensification of population aging and the continuous increase in labor costs, the humanoid robot industry is facing development opportunities. According to a report by GGII (Institute of Advanced Industry Research), the global humanoid robot market is expected to exceed $2 billion by 2026.
At this point in time, robots explode, and generative AI is the biggest pusher behind it. Deepu Talla, vice president of Nvidia Robotics and edge computing, said at CES this year that with the Large Language Model (LLM), robots can more naturally understand and respond to human commands; With generative AI, robots can achieve unprecedented perception and adaptability. Goldman Sachs previously stated that humanoid robots will become another AI landing scenario after intelligent driving of electric vehicles.
"The combination of AI and robot hardware is indeed a highly imaginative thing, and it is one of the few investment trends in 2023," Xia Zhijin, managing partner of Xiangfeng Investment, told Interface News.
Last December, Tesla released the second generation of Optimus robots, and Musk stated that some Optimus robots are likely to be delivered next year. In January of this year, the highly skilled Stanford robot Mobile ALOHA became popular throughout the internet. In March, Figure 01 released a demonstration video of the robot supported by the OpenAI big model, which was able to have smooth conversations with humans and analyze and execute verbal commands.
Nvidia also followed this trend and announced the establishment of the General Agent Research Laboratory (Gear) in February, with the aim of building embodied agent basic models in both virtual and real worlds, focusing on four areas: multimodal basic models, research on general-purpose robots, basic agents in virtual worlds, and simulation and simulation data.
At this year's GTC, Agility Robotics, Boston Dynamics, Disney, and Google DeepMind will showcase 25 robots and related products, including humanoid robots, robotic arms, robot vision systems, and high-precision 3D scanning systems.
At the same time, there have been multiple conferences related to robots, covering topics such as robot manipulation, navigation, perception, computation, simulation/synthesis data generation, and more.
Industry voices believe that 2024 may become the first year of mass production of humanoid robots. The explosion of large models does indeed provide more new opportunities for humanoid robots, but the prerequisite is that old problems such as technological bottlenecks and high costs no longer become new challenges.
Zhang Wei, founder of Zhuji Power, told Interface News that most robots have only reached a state of "walking two steps on flat ground", making it still difficult for them to help us carry things smoothly.
At the NVIDIA media communication meeting, Deepu Tella mentioned that the development of humanoid robots is still in its early stages, and the first batch of cases used will mainly focus on labor shortage in the manufacturing industry. However, with the resolution of some common problems, humanoid robots will be more widely used in the service industry, hospitals, and even households.