In the bustling labs of Hohai University in east China’s Jiangsu Province, Professor Liu Xiaofeng and his dedicated research team have spent their summer break pushing the boundaries of robotics. Their focus? Developing humanoid robots capable of expressing highly realistic and nuanced facial expressions.
Driven by the goal of enhancing emotional interaction between humans and robots, Liu’s team has introduced a groundbreaking algorithm that enables humanoid robots to generate authentic facial expressions. This innovation addresses one of the most challenging aspects of robotics: conveying the intricate and natural expressions characteristic of human emotion.
At the 26th annual meeting of the China Association of Science and Technology on July 2, research on emotionally intelligent digital humans and robots was highlighted as the foremost cutting-edge scientific issue for 2024. On the same day, Liu’s team published their findings in the esteemed international journal IEEE Transactions on Robotics, detailing their new approach to action unit (AU)-driven facial expression synthesis.
“Humanoid robots often struggle to display the complex facial expressions that humans naturally exhibit, which can hinder user engagement,” Liu explained. “To overcome this challenge, we’ve developed a comprehensive two-stage methodology that empowers our autonomous affective robots to exhibit rich and natural facial expressions.”
In the first stage, the team’s method generates nuanced robotic facial expression images guided by action units—a set of facial muscle movements that represent basic expressions. The second stage involves implementing these expressions on the robot, which has been designed with multiple degrees of freedom for facial movements, allowing it to embody fine-grained expressions.
Co-author and researcher Ni Rongrong from Changzhou University noted that while “digital humans” and “virtual anchors” can generate a variety of real-time expressions in virtual settings, replicating this on physical humanoid robots presents unique challenges. “Humanoid robots have specific constraints, such as the size and number of motors,” Ni said. “For instance, the robot we previously used had only nine micro motors beneath its facial surface, far fewer than the number of muscles in a human face.”
To address this limitation, the team ingeniously divided the nine motors into 17 action units, enabling richer expressions and smoother transitions through coordinated movements. This approach allows the robot to mimic human facial expressions more closely despite hardware constraints.
Looking ahead, Liu and his team plan to expand the number of facial action units and further enhance the robot’s ability to autonomously produce delicate expressions. They envision a future where emotionally intelligent humanoid robots equipped with high emotional and intellectual capabilities become integral in various settings.
“As the emotional interaction capabilities of humanoid robots continue to advance, these robots will find widespread use in nursing homes, kindergartens, special education schools, and beyond,” Liu remarked. “They will not only assist or replace humans in completing certain tasks but also bring additional emotional value.”
Reference(s):
China develops humanoid robots with realistic facial expression
cgtn.com
