Open Access

ARTICLE

A New Intelligent Approach for Deaf/Dumb People based on Deep Learning

by Haitham Elwahsh^1,*, Ahmed Elkhouly¹, Emad Abouel Nasr², Ali K. Kamrani³, Engy El-shafeiy⁴

1 Computer Science Department, Faculty of Computers and Information, Kafrelsheikh University, Kafrelsheikh, Egypt
2 Industrial Engineering Department, College of Engineering, King Saud University, Riyadh 11421, Saudi Arabia
3 Industrial Engineering Department, College of Engineering, University of Houston, Houston, TX 77204-4008, USA
4 Department of Computer Science, Faculty of Computers and Artificial Intelligence, University of Sadat City, Sadat City, Egypt

* Corresponding Author: Haitham Elwahsh. Email:

Computers, Materials & Continua 2022, 72(3), 6045-6060. https://doi.org/10.32604/cmc.2022.026309

Received 22 December 2021; Accepted 23 February 2022; Issue published 21 April 2022

Abstract

People who are deaf or have difficulty speaking use sign language, which consists of hand gestures with particular motions that symbolize the “language” they are communicating. A gesture in a sign language is a particular movement of the hands with a specific shape from the fingers and whole hand. In this paper, we present an Intelligent for Deaf/Dumb People approach in real time based on Deep Learning using Gloves (IDLG). The approach IDLG offers scientific contributions based deep-learning, a multi-mode command techniques, real-time, and effective use, and high accuracy rates. For this purpose, smart gloves working in real time were designed. The data obtained from the gloves was processed using deep-learning-based approaches and classified multi-mode commands that allow dumb people to speak with regular people via their smart phone. Internally, the glove has five flex sensors and an accelerometer using to achieve Low-Cost Control System. The flex sensor generates a proportional change in resistance for each individual move. The processing of these hand gestures is in Atmega32A Microcontroller which is an advance version of the microcontroller and the lab view software. IDLG compares the input signal to memory-stored specified voltage values. The performance of the IDLG approach was verified on a dataset created using different hand gestures from 20 different people. In the test using the IDLG approach on 10,000 data points, process time performance of milliseconds was achieved with 97% accuracy.

---

Keywords

---