Real-time sign language recognition using parallel multi-scale CNN to enhance inclusive education for deaf and hard of hearing students.
Saved in:
| Title: | Real-time sign language recognition using parallel multi-scale CNN to enhance inclusive education for deaf and hard of hearing students. |
|---|---|
| Authors: | El-Marzouki, Naoufal1 (AUTHOR) naoufal_elmarzouki@um5.ac.ma, Lasri, Imane1 (AUTHOR) imane_lasri@um5.ac.ma, Riadsolh, Anouar1 (AUTHOR) a.riadsolh@um5r.ac.ma, Elbelkacemi, Mourad1 (AUTHOR) mourad_prof@yahoo.fr |
| Source: | Multimedia Tools & Applications. Sep2025, Vol. 84 Issue 31, p38741-38760. 20p. |
| Subjects: | Sign language, Deaf students, Machine learning, Convolutional neural networks, Higher education, Inclusive education, Real-time computing |
| Abstract: | Inclusive education, a fundamental right for all individuals, presents unique challenges in higher education, particularly when it comes to students with disabilities. Our research focuses on rectifying the gaps in catering to the needs of deaf university students by introducing real-time sign language recognition through a parallel multi-scale CNN. We conducted model testing with five different optimizers, including Adam, Nadam, Adadelta, RMSprop, and Stochastic Gradient Descent (SGD). Notably, the best test accuracy of 99.70% was achieved using SGD on the Sign MNIST dataset, which comprises 7200 images categorized into 24 alphabet classes (excluding 'J' and 'Z'). The parallel multi-scale CNN outperformed the conventional CNN model, which achieved an accuracy of 96%. Our approach outperformed other state-of-the-art methods, demonstrating superior performance based on accuracy, precision, recall, and F1-score. The model was tested in real-time scenarios with Moroccan deaf students pursuing degrees in electronics, computer science, and robotics at Mohammed V University in Rabat. This testing underscores the model's adaptability and its potential to improve inclusivity in higher education. [ABSTRACT FROM AUTHOR] |
| Copyright of Multimedia Tools & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Inclusive education, a fundamental right for all individuals, presents unique challenges in higher education, particularly when it comes to students with disabilities. Our research focuses on rectifying the gaps in catering to the needs of deaf university students by introducing real-time sign language recognition through a parallel multi-scale CNN. We conducted model testing with five different optimizers, including Adam, Nadam, Adadelta, RMSprop, and Stochastic Gradient Descent (SGD). Notably, the best test accuracy of 99.70% was achieved using SGD on the Sign MNIST dataset, which comprises 7200 images categorized into 24 alphabet classes (excluding 'J' and 'Z'). The parallel multi-scale CNN outperformed the conventional CNN model, which achieved an accuracy of 96%. Our approach outperformed other state-of-the-art methods, demonstrating superior performance based on accuracy, precision, recall, and F1-score. The model was tested in real-time scenarios with Moroccan deaf students pursuing degrees in electronics, computer science, and robotics at Mohammed V University in Rabat. This testing underscores the model's adaptability and its potential to improve inclusivity in higher education. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 13807501 |
| DOI: | 10.1007/s11042-025-20692-7 |
