Open Access

ARTICLE

Mental Illness Disorder Diagnosis Using Emotion Variation Detection from Continuous English Speech

by S. Lalitha¹, Deepa Gupta^2,*, Mohammed Zakariah³, Yousef Ajami Alotaibi³

1 Department of Electronics & Communication Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Bengaluru, India
2 Department of Computer Science & Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Bengaluru, India
3 Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Saudi Arabia

* Corresponding Author: Deepa Gupta. Email:

(This article belongs to the Special Issue: Recent Trends in Machine Intelligence respected to Medical Field Applications)

Computers, Materials & Continua 2021, 69(3), 3217-3238. https://doi.org/10.32604/cmc.2021.018406

Received 07 March 2021; Accepted 09 April 2021; Issue published 24 August 2021

Abstract

Automatic recognition of human emotions in a continuous dialog model remains challenging where a speaker’s utterance includes several sentences that may not always carry a single emotion. Limited work with standalone speech emotion recognition (SER) systems proposed for continuous speech only has been reported. In the recent decade, various effective SER systems have been proposed for discrete speech, i.e., short speech phrases. It would be more helpful if these systems could also recognize emotions from continuous speech. However, if these systems are applied directly to test emotions from continuous speech, emotion recognition performance would not be similar to that achieved for discrete speech due to the mismatch between training data (from training speech) and testing data (from continuous speech). The problem may possibly be resolved if an existing SER system for discrete speech is enhanced. Thus, in this work the author’s existing effective SER system for multilingual and mixed-lingual discrete speech is enhanced by enriching the cepstral speech feature set with bi-spectral speech features and a unique functional set of Mel frequency cepstral coefficient features derived from a sine filter bank. Data augmentation is applied to combat skewness of the SER system toward certain emotions. Classification using random forest is performed. This enhanced SER system is used to predict emotions from continuous speech with a uniform segmentation method. Due to data scarcity, several audio samples of discrete speech from the SAVEE database that has recordings in a universal language, i.e., English, are concatenated resulting in multi-emotional speech samples. Anger, fear, sad, and neutral emotions, which are vital during the initial investigation of mentally disordered individuals, are selected to build six categories of multi-emotional samples. Experimental results demonstrate the suitability of the proposed method for recognizing emotions from continuous speech as well as from discrete speech.

---

Keywords

---