Open Access

ARTICLE

Negative Emotions Sensitive Humanoid Robot with Attention-Enhanced Facial Expression Recognition Network

by Rongrong Ni¹, Xiaofeng Liu^1,*, Yizhou Chen¹, Xu Zhou¹, Huili Cai¹, Loo Chu Kiong²

1 College of IoT Engineering, Hohai University, Changzhou, 213100, China
2 Faculty of Computer Science & Information Technology, Universiti Malaya, Kuala Lumpur, 50603, Malaysia

* Corresponding Author: Xiaofeng Liu. Email:

Intelligent Automation & Soft Computing 2022, 34(1), 149-164. https://doi.org/10.32604/iasc.2022.026813

Received 05 January 2022; Accepted 12 February 2022; Issue published 15 April 2022

Abstract

Lonely older adults and persons restricted in movements are apt to cause negative emotions, which is harmful to their mental health. A humanoid robot with audiovisual interactions is presented, which can correspondingly output positive facial expressions to relieve human's negative facial expressions. The negative emotions are identified through an attention-enhanced facial expression recognition (FER) network. The network is firstly trained on MMEW macro-and micro-expression databases to discover expression-related features. Then, macro-expression recognition tasks are performed by fine-tuning the trained models on several benchmarking FER databases, including CK+ and Oulu-CASIA. A transformer network is introduced to process the sequential features engineered by the FER network and output a final stable control order. This order is used to control the robot's facial motor units to generate different expressions, e.g., a smile expression. Evaluations on benchmarking databases verify that the proposed method can precisely recognize facial expressions. The joint modulation with the humanoid robot proves that the robot can respond effectively to the user's negative emotions.

---

Keywords

---