Ai-Based Real-Time Filipino Sign Language Recognition and Translation System with Context-Aware Natural Language Processing

The Filipino Sign Language (FSL) serves as the official sign language of the Deaf community in the Philippines, as recognized under Republic Act 11106. Despite this recognition, communication barriers persist in educational institutions, healthcare facilities, workplaces, and other settings where Deaf and hearing individuals interact, largely due to the limited understanding of FSL among the hearing majority. Existing sign language recognition applications predominantly target other sign languages such as American Sign Language and are constrained to isolated gesture recognition, failing to address the continuous and dynamic nature of Filipino Sign Language communication. This study developed an AI-Based Real-Time Filipino Sign Language Recognition and Translation System with Context-Aware Natural Language Processing, implemented as an Android mobile application. The system employs MediaPipe Holistic for coordinate-based 3D landmark extraction from hand and body pose, and a Long Short-Term Memory (LSTM) neural network for sequential gesture pattern recognition. A dataset of 20 FSL vocabulary items was collected from seven Filipino Sign Language signers, and a four-technique data augmentation pipeline including Gaussian noise, time shifting, scale variation, and hand mirroring was applied to improve model generalization across different signing styles and dominant hand preferences. The trained model was converted to TensorFlow Lite format for efficient on-device inference without requiring server-side computation or internet connectivity. The system achieved an overall classification accuracy of 97% on the evaluated vocabulary, demonstrating the viability of real-time FSL gesture recognition on consumer-grade mobile hardware. The study adopted a developmental research design with an Agile software development approach to facilitate iterative refinement throughout the development process. The results confirm that the proposed MediaPipe-LSTM architecture provides a practical and accessible foundation for bridging the communication gap between the Filipino Deaf community and the hearing population.

Filipino Sign Language, Artificial Intelligence, MediaPipe, LSTM, Real-Time Translation

1. Abalos, D. M. M., Dacsil, J. C. B., & Raquedan, A. A. L. (2024). SignTalk: An assistive bi-directional real-time sign language mobile application using MediaPipe, CNN, and GAN. IC-ITECHS, 5(1), 217–225. [Google Scholar] [Crossref]

2. Abd Al-Latief, S. T., Yussof, S., Ahmad, A., & Khadim, S. (2024). Deep learning for sign language recognition: A comparative review. Journal of Smart Internet of Things (JSIoT), 2024(01),77–116. [Google Scholar] [Crossref]

3. Ahmed, S., Rahman, M. S., Kaiser, M. S., & Hosen, A. S. (2025). Advancing personalized and inclusive education for students with disability through artificial intelligence: Perspectives, challenges, and opportunities. Digital, 5(2), Article 11. [Google Scholar] [Crossref]

4. Alit, N. A., Ellias, M. S., & Ahmad, A. D. (2025). Technology application in teaching and learning for hearing impaired students: A recent systematic review. International Journal of Education, Psychology and Counseling, 10(58), 541–561. [Google Scholar] [Crossref]

5. Al-Qurishi, M., Khalid, T., & Souissi, R. (2021). Deep learning for sign language recognition: Current techniques, benchmarks, and open issues. IEEE Access, 9, 126917–126951. [Google Scholar] [Crossref]

6. Amangeldy, N., et al. (2023). Continuous sign language recognition and its translation into intonation-colored speech. Sensors (MDPI). [Google Scholar] [Crossref]

7. Ardales. (n.d.). Towards a memory-efficient Filipino sign language recognizer. In DLSU Student Research Workshop Proceedings. [Google Scholar] [Crossref]

8. Ben Haj Amor, A., El Ghoul, O., & Jemni, M. (2023). Sign language recognition using the electromyographic signal: A systematic literature review. Sensors, 23(19), 8343. [Google Scholar] [Crossref]

9. Camgoz, N. C., Hadfield, S., Koller, O., Ney, H., & Bowden, R. (2020). Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). [Google Scholar] [Crossref]

10. Cayme, J. (2024). Gesture recognition of Filipino sign language using CNN-LSTM. Knowledge (MDPI). [Google Scholar] [Crossref]

11. Córdova-Esparza, D. M., Terven, J., & Romero-González, J. A. (2025, October). Sign language recognition and translation with large multimodal models: A systematic review. In Advances in Soft Computing: 24th Mexican International Conference on Artificial Intelligence (MICAI 2025), Proceedings, Part II (p. 272). Springer Nature. [Google Scholar] [Crossref]

12. Dagde, R., Thakre, S., Chopade, S., Rokde, L., & Kakani, V. (2025). A hand sign recognition based signal system for mute people using machine learning. MethodsX, Article 103670. [Google Scholar] [Crossref]

13. De La Salle University. (2022). Implementation of a Filipino sign language alphabet recognition system on Raspberry Pi. In DLSU Research Congress Proceedings. [Google Scholar] [Crossref]

14. Dela Cruz, M., & Navarro, L. (2024). Utilization of Filipino sign language–sight words intervention (FSL-SWI). Research Publish Journal. [Google Scholar] [Crossref]

15. Dostal, H. M., Scott, J. A., Chappell, M. D., & Black, C. (2025). A scoping review of literacy interventions using signed languages for school-age deaf students. Behavioral Sciences, 15(8), 1104. [Google Scholar] [Crossref]

16. Galván-Ruiz, J., Travieso-González, C. M., Tejera-Fettmilch, A., Piñán-Roescher, A., Esteban-Hernández, L., & Domínguez-Quintana, L. (2020). Perspective and evolution of gesture recognition for sign language: A review. Sensors, 20(12), 3571. [Google Scholar] [Crossref]

17. Hu, Y., et al. (2024). CorrNet+: Sign language recognition and translation via spatial-temporal correlation. arXiv. [Google Scholar] [Crossref]

18. Ibrahim, N. B., Zayed, H. H., & Selim, M. M. (2020). Advances, challenges and opportunities in continuous sign language recognition. Journal of Engineering and Applied Sciences, 15(5), 1205–1227. [Google Scholar] [Crossref]

19. Jetomo, A., & De Lara, M. (2025). Filipino sign language alphabet recognition using a persistent homology approach. PeerJ Computer Science. [Google Scholar] [Crossref]

20. Kalsh, E. A., & Garewal, N. S. (2013). Sign language recognition system. International Journal of Computational Engineering Research, 3(6), 15–21. [Google Scholar] [Crossref]

21. Li, X., & Meng, Q. (2022). Multi-view spatial temporal network for continuous sign language recognition. arXiv. [Google Scholar] [Crossref]

22. Mukushev, A., et al. (2022). FluentSigners-50: A signer-independent benchmark for sign language recognition. [Google Scholar] [Crossref]

23. Najib, F. M. (2025). Sign language interpretation using machine learning and artificial intelligence. Neural Computing and Applications, 37(2), 841–857. [Google Scholar] [Crossref]

24. Papadimitriou, S., et al. (2023). Multimodal locally enhanced transformer for continuous sign language recognition. In Proceedings of Interspeech 2023. [Google Scholar] [Crossref]

25. Papastratis, I., Chatzikonstantinou, C., Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2021). Artificial intelligence technologies for sign language. Sensors, 21(17), 5843. [Google Scholar] [Crossref]

26. Repal, P., Pore, N., Pasale, A., Thengal, R., Ambule, S., & Deval, N. D. (2024). Real-time sign language translator using machine learning. Journal of Artificial Intelligence, Machine Learning and Neural Network, 4(4), 22–30. [Google Scholar] [Crossref]

27. Saeed, Z. R., Zainol, Z. B., Zaidan, B. B., & Alamoodi, A. H. (2022). A systematic review on systems-based sensory gloves for sign language pattern recognition: An update from 2017 to 2022. IEEE Access, 10, 123358–123377. [Google Scholar] [Crossref]

28. Santos, R., & Villanueva, P. (2024). An assistive bi-directional real-time sign language mobile application. In IC-ITECHS Conference Proceedings. [Google Scholar] [Crossref]

29. Teran Quezada, R., et al. (2024). Sign-to-text translation from Panamanian sign language to Spanish in continuous capture mode with deep neural networks. Applied Sciences (MDPI). [Google Scholar] [Crossref]

30. Tipan, L. K., Abalos, A. M., Bondoc, A. E., To, J. J., Rivera, J. P., Laguna, A. F., & Tighe, E. (2024, December). TinyFSL: Tiny machine learning for Filipino sign language. In Proceedings of the 38th Pacific Asia Conference on Language, Information and Computation (pp. 1504–1513). [Google Scholar] [Crossref]

31. Toshpulatov, M., Lee, W., Jun, J., & Lee, S. (2025). Deep learning pathways for automatic sign language processing. Pattern Recognition, Article 111475. [Google Scholar] [Crossref]

32. Ugale, M., Rodrigues, O., Shinde, A., Desle, K., & Yadav, S. (2023, May). A review on sign language recognition using CNN. In International Conference on Advanced Machine Intelligence and Data Analytics (pp. 251–259). Springer. [Google Scholar] [Crossref]

33. Wadhawan, A., & Kumar, P. (2021). Sign language recognition systems: A decade systematic literature review. Archives of Computational Methods in Engineering, 28(3). [Google Scholar] [Crossref]

34. WiKA. (2024). WiKA vision-based sign language recognition. MSU-GenSan Journal. [Google Scholar] [Crossref]

35. Yenisari, E., & Yavuz, S. (2025). Deep learning based sign language recognition using efficient multi-feature attention mechanisms. IEEE Access. [Google Scholar] [Crossref]

36. Yin, Z., et al. (2020). Better sign language translation with STMC transformer. In Proceedings of COLING 2020. [Google Scholar] [Crossref]

37. Zhang, F., et al. (2020). MediaPipe hands: On-device real-time hand tracking. arXiv. [Google Scholar] [Crossref]

• Smart Iot Device for Weather And Health
• Merlarchive: A Web-Based Academic Hub for UDM With AI-Powered Natural Language Processing
• Enhanced Social Network Security System: Integrating Biometric Authentication for Improved User Verification and Privacy Protection
• Smart Budget Allocation in Public Policy: A Data-Driven Approach for Equitable Resource Distribution
• Decision Support System for Faculty Selection, Promotion, and Reclassification Using Predictive Analytics