Open Access
ARTICLE
Negative Emotions Sensitive Humanoid Robot with Attention-Enhanced Facial Expression Recognition Network
1 College of IoT Engineering, Hohai University, Changzhou, 213100, China
2 Faculty of Computer Science & Information Technology, Universiti Malaya, Kuala Lumpur, 50603, Malaysia
* Corresponding Author: Xiaofeng Liu. Email:
Intelligent Automation & Soft Computing 2022, 34(1), 149-164. https://doi.org/10.32604/iasc.2022.026813
Received 05 January 2022; Accepted 12 February 2022; Issue published 15 April 2022
Abstract
Lonely older adults and persons restricted in movements are apt to cause negative emotions, which is harmful to their mental health. A humanoid robot with audiovisual interactions is presented, which can correspondingly output positive facial expressions to relieve human's negative facial expressions. The negative emotions are identified through an attention-enhanced facial expression recognition (FER) network. The network is firstly trained on MMEW macro-and micro-expression databases to discover expression-related features. Then, macro-expression recognition tasks are performed by fine-tuning the trained models on several benchmarking FER databases, including CK+ and Oulu-CASIA. A transformer network is introduced to process the sequential features engineered by the FER network and output a final stable control order. This order is used to control the robot's facial motor units to generate different expressions, e.g., a smile expression. Evaluations on benchmarking databases verify that the proposed method can precisely recognize facial expressions. The joint modulation with the humanoid robot proves that the robot can respond effectively to the user's negative emotions.Keywords
Cite This Article
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.