Open Access

ARTICLE

Deep Bimodal Fusion Approach for Apparent Personality Analysis

Saman Riaz¹, Ali Arshad², Shahab S. Band^3,*, Amir Mosavi⁴

1 Department of Computer Science, National University of Technology, Islamabad, 44000, Pakistan
2 Department of Computer Science, Institute of Space Technology, Islamabad, 44000, Pakistan
3 Future Technology Research Center, National Yunlin University of Science and Technology, Douliu, 64002, Yunlin, Taiwan
4 Faculty of Civil Engineering, Technische Universitat Dresden, Dresden, 01069, Germany

* Corresponding Author: Shahab S. Band. Email:

Computers, Materials & Continua 2023, 75(1), 2301-2312. https://doi.org/10.32604/cmc.2023.028333

Received 08 February 2022; Accepted 31 March 2022; Issue published 06 February 2023

Abstract

Personality distinguishes individuals’ patterns of feeling, thinking, and behaving. Predicting personality from small video series is an exciting research area in computer vision. The majority of the existing research concludes preliminary results to get immense knowledge from visual and Audio (sound) modality. To overcome the deficiency, we proposed the Deep Bimodal Fusion (DBF) approach to predict five traits of personality-agreeableness, extraversion, openness, conscientiousness and neuroticism. In the proposed framework, regarding visual modality, the modified convolution neural networks (CNN), more specifically Descriptor Aggregator Model (DAN) are used to attain significant visual modality. The proposed model extracts audio representations for greater efficiency to construct the long short-term memory (LSTM) for the audio modality. Moreover, employing modality-based neural networks allows this framework to independently determine the traits before combining them with weighted fusion to achieve a conclusive prediction of the given traits. The proposed approach attains the optimal mean accuracy score, which is 0.9183. It is achieved based on the average of five personality traits and is thus better than previously proposed frameworks.

---

Keywords

---