Open Access

ARTICLE

Alphabet-Level Indian Sign Language Translation to Text Using Hybrid-AO Thresholding with CNN

by Seema Sabharwal^1,2,*, Priti Singla¹

1 Department of Computer Science and Engineering, Baba Mastnath University, Rohtak, 124001, India
2 Department of Computer Science, Government Post Graduate College for Women, Panchkula, 134109, India

* Corresponding Author: Seema Sabharwal. Email:

(This article belongs to the Special Issue: Deep Learning for Image Video Restoration and Compression)

Intelligent Automation & Soft Computing 2023, 37(3), 2567-2582. https://doi.org/10.32604/iasc.2023.035497

Received 23 August 2022; Accepted 13 January 2023; Issue published 11 September 2023

Abstract

Sign language is used as a communication medium in the field of trade, defence, and in deaf-mute communities worldwide. Over the last few decades, research in the domain of translation of sign language has grown and become more challenging. This necessitates the development of a Sign Language Translation System (SLTS) to provide effective communication in different research domains. In this paper, novel Hybrid Adaptive Gaussian Thresholding with Otsu Algorithm (Hybrid-AO) for image segmentation is proposed for the translation of alphabet-level Indian Sign Language (ISLTS) with a 5-layer Convolution Neural Network (CNN). The focus of this paper is to analyze various image segmentation (Canny Edge Detection, Simple Thresholding, and Hybrid-AO), pooling approaches (Max, Average, and Global Average Pooling), and activation functions (ReLU, Leaky ReLU, and ELU). 5-layer CNN with Max pooling, Leaky ReLU activation function, and Hybrid-AO (5MXLR-HAO) have outperformed other frameworks. An open-access dataset of ISL alphabets with approx. 31 K images of 26 classes have been used to train and test the model. The proposed framework has been developed for translating alphabet-level Indian Sign Language into text. The proposed framework attains 98.95% training accuracy, 98.05% validation accuracy, and 0.0721 training loss and 0.1021 validation loss and the performance of the proposed system outperforms other existing systems.

---

Keywords

---