Open Access

ARTICLE

Engagement Detection Based on Analyzing Micro Body Gestures Using 3D CNN

Shoroog Khenkar^1,*, Salma Kammoun Jarraya^1,2
1 Department of Computer Science, King Abdul-Aziz University, Jeddah, Saudi Arabia
2 MIRACL-Laboratory, Sfax, Tunisia
* Corresponding Author: Shoroog Khenkar. Email:
(This article belongs to this Special Issue: Machine Learning Applications in Medical, Finance, Education and Cyber Security)

Computers, Materials & Continua 2022, 70(2), 2655-2677. https://doi.org/10.32604/cmc.2022.019152

Received 04 April 2021; Accepted 23 June 2021; Issue published 27 September 2021

Abstract

This paper proposes a novel, efficient and affordable approach to detect the students’ engagement levels in an e-learning environment by using webcams. Our method analyzes spatiotemporal features of e-learners’ micro body gestures, which will be mapped to emotions and appropriate engagement states. The proposed engagement detection model uses a three-dimensional convolutional neural network to analyze both temporal and spatial information across video frames. We follow a transfer learning approach by using the C3D model that was trained on the Sports-1M dataset. The adopted C3D model was used based on two different approaches; as a feature extractor with linear classifiers and a classifier after applying fine-tuning to the pre-trained model. Our model was tested and its performance was evaluated and compared to the existing models. It proved its effectiveness and superiority over the other existing methods with an accuracy of 94%. The results of this work will contribute to the development of smart and interactive e-learning systems with adaptive responses based on users’ engagement levels.

---

Keywords

Micro body gestures; engagement detection; 3D CNN; transfer learning; e-learning; spatiotemporal features

---