Sign Language Recognition Using Deep Learning

Sign language is an essential means of communication for individuals who are speech- or hearing-impaired. It uses structured hand gestures, movements, and facial expressions to convey meaning. However, most people are not familiar with sign language, which creates a significant communication barrier between speech-impaired individuals and the rest of society.
With advancements in Artificial Intelligence and Deep Learning, it has become possible to automatically recognize hand gestures and translate them into understandable text or speech. Sign language recognition systems aim to bridge this communication gap by converting hand gestures into natural language in real time.
This project focuses on developing a real-time sign language recognition system using deep learning techniques. The system captures live hand gestures using a smartphone or webcam and converts them into corresponding text and speech outputs. By enabling real-time processing, the system supports faster and more intuitive communication.
Traditional gesture recognition systems rely on static images or offline video datasets, which are ineffective for real-time gesture interpretation. These systems often fail to capture the temporal dynamics of gestures. To overcome these challenges, the proposed system integrates Media Pipe for real-time hand landmark detection and a LSTM hybrid deep learning model for gesture classification. The LSTM models temporal dependencies across gesture sequences.

Sign Language Recognition, Deep Learning, Artificial Intelligence, Computer Vision, Hand Gesture Recognition, Convolutional Neural Networks (CNN).

1. TensorFlow, “TensorFlow Lite Documentation,” Google AI Platform, 2024. [Online]. Available: https://www.tensorflow.org/lite [Google Scholar] [Crossref]

2. M. U. Rehman, F. Ahmed, M. A. Khan, et al., “Dynamic hand gesture recognition using 3D-CNN and LSTM networks,” Computers, Materials & Continua, vol. 72, no. 3, pp. 5901–5918, 2022 Available: https://doi.org/10.32604/cmc.2022.019586 [Google Scholar] [Crossref]

3. A. Khan, M. Sayed, et al., “Hand gesture recognition using keypoint landmarks with machine learning,” in Proc. IEEE Int. Conf. Computing and Comm. Systems, 2021. [Online]. Available: https://ieeexplore.ieee.org/document/9562108 [Google Scholar] [Crossref]

4. F. Zhang, V. Bazarevsky, A. Vakunov, et al., “MediaPipe Hands: On-device real-time hand tracking,” arXiv preprint arXiv:2006.10214, 2020. [Online]. Available: https://arxiv.org/abs/2006.10214 [Google Scholar] [Crossref]

5. A. Gunduz, N. Kose, and G. Rigoll, “Real-time hand gesture detection and classification using CNNs,” arXiv preprint arXiv:1901.10323, 2019. [Online]. Available: https://arxiv.org/abs/1901.10323 [Google Scholar] [Crossref]

6. P. Molchanov, X. Yang, S. Gupta, et al., “Online detection and classification of dynamic hand gestures with recurrent 3D-CNN,” in Proc. IEEE CVPR, 2016 Available: https://openaccess.thecvf.com/content_cvpr_2016/papers/Molchanov_Online_Detection_and_CVPR_2016_paper.pdf [Google Scholar] [Crossref]

• The Role of Artificial Intelligence in Revolutionizing Library Services in Nairobi: Ethical Implications and Future Trends in User Interaction
• ESPYREAL: A Mobile Based Multi-Currency Identifier for Visually Impaired Individuals Using Convolutional Neural Network
• Comparative Analysis of AI-Driven IoT-Based Smart Agriculture Platforms with Blockchain-Enabled Marketplaces
• AI-Based Dish Recommender System for Reducing Fruit Waste through Spoilage Detection and Ripeness Assessment
• SEA-TALK: An AI-Powered Voice Translator and Southeast Asian Dialects Recognition