Open Access

ARTICLE

Deep Facial Emotion Recognition Using Local Features Based on Facial Landmarks for Security System

Youngeun An, Jimin Lee, EunSang Bak^*, Sungbum Pan^*

IT Research Institute, Chosun University, Gwang-Ju, 61452, Korea

* Corresponding Authors: Eunsang Bak. Email: ; Sungbum Pan. Email:

(This article belongs to the Special Issue: Advances in Information Security Application)

Computers, Materials & Continua 2023, 76(2), 1817-1832. https://doi.org/10.32604/cmc.2023.039460

Received 31 January 2023; Accepted 23 May 2023; Issue published 30 August 2023

Abstract

Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces. Most conventional methods for emotion recognition using facial expressions use the entire facial image to extract features and then recognize specific emotions through a pre-trained model. In contrast, this paper proposes a novel feature vector extraction method using the Euclidean distance between the landmarks changing their positions according to facial expressions, especially around the eyes, eyebrows, nose, and mouth. Then, we apply a new classifier using an ensemble network to increase emotion recognition accuracy. The emotion recognition performance was compared with the conventional algorithms using public databases. The results indicated that the proposed method achieved higher accuracy than the traditional based on facial expressions for emotion recognition. In particular, our experiments with the FER2013 database show that our proposed method is robust to lighting conditions and backgrounds, with an average of 25% higher performance than previous studies. Consequently, the proposed method is expected to recognize facial expressions, especially fear and anger, to help prevent severe accidents by detecting security-related or dangerous actions in advance.

---

Keywords

---