Enhancing Yoruba Text Autocompletion with an Attention-Augmented Recurrent Neural Network

The digital divide in Natural Language Processing (NLP) is particularly pronounced for low-resource, morphologically complex languages like Yoruba. This paper addresses the challenge of developing an effective text autocompletion system for Yoruba, a language characterized by its tonal diacritics and agglutinative structure, which are poorly handled by conventional models. A character-level Recurrent Neural Network (RNN) architecture enhanced with a multi-head attention mechanism to overcome the limitations of standard RNNs in capturing long-range contextual dependencies was proposed. A curated dataset of 4,431 Yoruba words was used for training and evaluation. The proposed RNN + Attention model was rigorously evaluated against a baseline RNN, demonstrating a significant 82.5% improvement in model confidence, achieving a perplexity of 2.21 compared to the baseline's 12.67. The model also achieved perfect Top-K accuracy and Mean Reciprocal Rank, indicating its high precision in ranking correct suggestions. The results conclusively show that integrating an attention mechanism is a pivotal architectural enhancement for sequence prediction tasks in Yoruba, leading to a robust and contextually aware autocompletion system. This work provides a validated framework for building efficient NLP tools for low-resource languages.

Natural Language Processing, Text Autocompletion, Low-Resource Languages, Yoruba, Recurrent Neural Network, Attention Mechanism

1. Aditi, M. (2019). Understanding RNN and LSTM. What is Neural Network? | by Aditi Mittal | Medium. https://aditi mittal.medium.com/understanding-rnn-and-lstm-f7cdf6dfc14e [Google Scholar] [Crossref]

2. Adeyemi, Q. (2022). New Yoruba Data. Kaggle. https://www.kaggle.com/datasets/adeyemiquadri1/new-yoruba-data [Google Scholar] [Crossref]

3. Adeniyi, K. (2020). Lexicalisation of tonal downstep in Yoruba. Canadian Journal of Linguistics/Revue canadienne de linguistique, 65(4), 642-669. https://doi.org/10.1017/cnj.2020.22 [Google Scholar] [Crossref]

4. Akindele, O., Jesujoba, O., Aderonke, S., & David, I. (2024). YAD: Leveraging T5 for Improved Automatic Diacritization of Yoruba Text. AfricaNLP workshop at ICLR 2024. http://arxiv.org/abs/2412.20218v1 [Google Scholar] [Crossref]

5. Akinola, O., Adeyemi, A., & Alabi, J. (2021). Character-level modeling of agglutinative morphology: A case study of Yoruba. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (pp. 78-89). https://doi.org/10.18653/v1/2021.emnlp-main.7 [Google Scholar] [Crossref]

6. Al-Anzi, F. S., & Shalini, S. T. B. (2024). Revealing the next word and character in Arabic: An effective blend of long Short-Term memory networks and ARABERT. Applied Sciences, 14(22), 10498. https://doi.org/10.3390/app142210498 [Google Scholar] [Crossref]

7. Asahiah, F. O., Odejobi, O. A., & Adagunodo, E. R. (2017). Restoring tone marks in standard Yorùbá electronic text: Improved model. Computer Science, 18(3), 301-315. https://doi.org/10.7494/csci.2017.18.3.2128 [Google Scholar] [Crossref]

8. Atharv, P., Pushkar, P., Sumi, P., & Aniruddha, S. (2023). Next Word Prediction using Recurrent Neural Networks. International Journal of Progressive Research in Engineering Management and Science, 3(2), 123-132. https://doi.org/10.58257/IJPREMS32232 [Google Scholar] [Crossref]

9. Hifny, Y. (2018). Hybrid LSTM/MaxEnt Networks for Arabic Syntactic Diacritics Restoration. IEEE Signal Processing Letters, 25(10), 1515–1519. https://doi.org/10.1109/LSP.2018.2865098 [Google Scholar] [Crossref]

10. Kumar, A., & Gupta, R. (2021). Natural Language Processing for Low-Resource Languages: Current Trends and Future Directions. Artificial Intelligence Review, 54*(1), 1-23. https://doi.org/10.1007/s10462-020-09934-2 [Google Scholar] [Crossref]

11. Ogheneruemu, O., Adeyemi, M., & Olatunji, S. (2023). A syllable-based LSTM network for comprehensive diacritic restoration in Yorùbá text. Journal of African Language Studies, 3(1), 45-60. https://doi.org/10.1016/j.jals.2023.01.004 [Google Scholar] [Crossref]

12. Rayhan, A., & Kinzler, R. (2023). Natural Language Processing: Transforming How Machines Understand Human Language. RG, 2(2), 34900-99200. http://doi.org/10.13140/RG.2.2.34900.99200 [Google Scholar] [Crossref]

13. Singh, A., & Patel, M. (2022). Advances in N-gram Language Modeling for Speech Recognition. Journal of Speech Technology, 25(1), 15-30. https://doi.org/10.1007/s10772-021-09882-4 [Google Scholar] [Crossref]

14. Ugwu, C. C., Oyewole, A. R., Popoola, O. S., Adetunmbi, A. O., & Elebute, A. (2024). A Part of Speech Tagger for Yoruba Language Text using Deep Neural Network. Franklin Open, 5, 100185. https://doi.org/10.1016/j.fraope.2024.100185 [Google Scholar] [Crossref]

15. VanamaYaswanth, Ajay Kumar, Neelesh Kumar Jain, Nilesh Kumar Patel (2023). Leveraging Bi-Directional LSTM for Robust lyrics Generation in Telugu: Methodology and Improvements. Tuijin Jishu/Journal of Propulsion Technology, 44(3), 1908–1913. https://doi.org/10.52783/tjjpt.v44.i3.618 [Google Scholar] [Crossref]

16. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30 (NIPS 2017). https://doi.org/10.48550/arXiv.1706.03762 [Google Scholar] [Crossref]

17. Yang, H., & Zhang, X. (2023). N-gram-Based Autocomplete: A Study on User Behavior and Predictive Accuracy. Computers in Human Behavior, 141, 107572. https://doi.org/10.1016/j.chb.2023.107572 [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet