According to Tianyancha's intellectual property information, on March 15th, Taobao (China) Software Co., Ltd. applied for a patent for "Digital Human based Video Generation and Interaction Methods, Devices, Storage Media, and Program Products".
The present embodiment of the application performs text to speech processing based on the user's voice features and emotional labels, as well as speech to expression processing based on the mapping relationship between the user's voice features and expression coefficients, and renders a digital human model based on voice signals and expression coefficients to obtain video data of the digital human model. This patent accurately simulates the voice characteristics of users, ensuring that the voice output of digital humans not only sounds natural, but also realizes personalized driving of digital humans, improving the realism of digital humans in terms of sound and dynamic image, thereby improving the user experience, enhancing the interactivity, realism, and immersion of digital humans.