Evaluating the Effectiveness of a Japanese Language Learning Management System with an AI-Powered Tutor Utilizing LLaMA 3.1 8B

This research aims to develop a web-based platform that integrates artificial intelligence to make Japanese language learning more accessible and interactive. Designed for travelers, overseas workers, and self-learners, Japanese Learning Management system with Artificial Intelligence Powered Tutor utilizing Llama-3.1-8B provides JLPT N5–N4 lessons with an AI tutor that offers pronunciation feedback and captions displaying Japanese scripts with romaji (the Romanized representation of Japanese words) and English translations. The system was developed using Flask, Alpine.js, Tailwind CSS, and MySQL, while the AI feature was trained with Hugging Face Transformers and PyTorch for voice recognition. To achieve these objectives, the system was developed using the Agile methodology following the Scrum framework, ensuring iterative design, testing, and integration of features. Evaluation results from 219 students, three IT professionals, and one Japanese language instructor showed high satisfaction, with overall ratings of 3.85, 4.61, and 5.00 respectively. These results confirm that the system effectively enhances pronunciation, comprehension, and learner engagement through AI integration. Future improvements include expanding datasets, training the AI on stronger hardware, refining usability, and adding higher JLPT levels and other languages to make the platform more versatile and scalable. This study presents the development and evaluation of a Japanese Language Learning Management System (LMS) integrated with an AI-powered tutor utilizing LLaMA 3.1 8B, designed to support learners preparing for JLPT N5–N4 levels. The system combines structured lessons with interactive features, including real-time pronunciation feedback, voice recognition, and captioned scripts displaying Japanese characters with romaji and English translations. Built using Flask, Alpine.js, Tailwind CSS, and MySQL, the AI component was fine-tuned with Hugging Face Transformers and PyTorch for speech processing. A quantitative descriptive design guided the evaluation, involving 219 students, three IT professionals, and one certified Japanese language instructor. Assessment instruments were based on ISO/IEC 25010 software quality standards and language learning objectives. Results indicate high user satisfaction: end users rated the system 3.85 (Agree), IT professionals scored 4.61 (Agree), and the instructor gave a perfect 5.00 (Strongly Agree). Findings confirm that the platform effectively enhances pronunciation, comprehension, and learner engagement while meeting international software quality benchmarks. Future improvements include expanding datasets, optimizing AI performance, and adding higher JLPT levels and multilingual support to ensure scalability and broader applicability.

Japanese language, learning management system, AI tutor

1. JLPT Official Statistics (2024). JLPT in Charts: Changes in number of applicants (1984–2024). Available: https://www.jlpt.jp/e/statistics/ [Google Scholar] [Crossref]

2. Hetz, G. (2025, May 6; updated Nov 27). Jargonic sets new standards for Japanese ASR. aiOla.ai. [Google Scholar] [Crossref]

3. Bajo, M., Fukukawa, H., Morita, R., & Ogasawara, Y. (2024, Dec 14). Efficient adaptation of multilingual models for Japanese ASR. arXiv:2412.10705v1. [Google Scholar] [Crossref]

4. Alotaibi, N. S. A. (2024). The impact of AI and LMS integration on the future of higher education: Opportunities, challenges, and strategies for transformation. Sustainability, 16(23), 10357. https://doi.org/10.3390/su162310357 [Google Scholar] [Crossref]

5. Song, J. Y., Pycha, A., & Culleton, T. (2022). Interactions between voice-activated AI assistants and human speakers and their implications for second language acquisition. Frontiers in Communication, 7, 995475. https://doi.org/10.3389/fcomm.2022.995475 [Google Scholar] [Crossref]

6. Meta (2024, July 22). Introducing LLaMA 3.1: Multilingual LLM with 128K context window. Meta AI Blog. [Google Scholar] [Crossref]

7. Hugging Face (2024, July 23). LLaMA 3.1 8B Instruct Model Card. Hugging Face Hub. [Google Scholar] [Crossref]

8. Bayly Castaneda, K., Ramirez Montoya, M.-S., & Morita Alexander, A. (2024, Aug 7). Crafting personalized learning paths with AI for lifelong learning: A systematic literature review. Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1424386 [Google Scholar] [Crossref]

9. Sharma, S., Mittal, P., Kumar, M., & Bhardwaj, V. (2025, Apr 4). The role of large language models in personalized learning: A systematic review of educational impact. Discover Sustainability, 6, 243. https://doi.org/10.1007/s43621-025-01094-z [Google Scholar] [Crossref]

10. Ngo, T. T.-N., Chen, H. H.-J., & Lai, K. K.-W. (2023). The effectiveness of automatic speech recognition in ESL/EFL pronunciation: A meta-analysis. ReCALL, 36(1), 4–21. https://doi.org/10.1017/S0958344023000113 [Google Scholar] [Crossref]

11. Alharthi, S. M. (2024). Siri as an interactive pronunciation coach: Its impact on EFL learners. Cogent Education, 11(1), 2304245. https://doi.org/10.1080/2331186X.2024.2304245 [Google Scholar] [Crossref]

12. Purani, A. (2025, Jul 22). AI Integration in LMS Platforms: Personalized Learning Paths. SevenSquareTech. [Google Scholar] [Crossref]

13. Qian, C. (2025, Nov 19). The application of artificial intelligence technology in Japanese language education and its impact on learning effectiveness. Proceedings of SPIE, 13940, MLNNCS 2025, 139401O. https://doi.org/10.1117/12.3090376 [Google Scholar] [Crossref]

14. Kaleci, D. (2025). Integration and application of artificial intelligence tools in the Moodle platform: A theoretical exploration. Journal of Educational Technology & Online Learning, 8(1), 100–111. [Google Scholar] [Crossref]

15. Department of Education (2025, Feb 20). DepEd launches AI center for education. Available: https://www.deped.gov.ph/2025/02/20/deped-launches-ai-center-for-education/ [Google Scholar] [Crossref]

16. Department of Education (2025, Apr 29). DO 013, s. 2025 – Establishment of the Center for Artificial Intelligence Research for the Department of Education. Available: https://www.deped.gov.ph/2025/04/29/april-29-2025-do-013-s-2025-establishment-of-the-center-for-artificial-intelligence-research-for-the-department-of-education/ [Google Scholar] [Crossref]

17. Department of Education (2025, Dec 19). AI research initiative ni Angara, nagbibigay ng data-driven, efficient reforms sa DepEd. Available: https://www.deped.gov.ph/2025/12/19/ai-research-initiative-ni-angara-nagbibigay-ng-data-driven-efficient-reforms-sa-deped/ [Google Scholar] [Crossref]

18. DOST-ASTI (n.d.). iTANONG: A Natural Language Interface to Databases for Filipinos. Available: https://asti.dost.gov.ph/projects/itanong/ [Google Scholar] [Crossref]

19. Rest of World (2024, Feb 19). The Philippines is creating a ChatGPT rival that speaks Filipino and Taglish. Available: https://restofworld.org/2024/filipino-ai-chatbot-launches-2025-local-languages/ [Google Scholar] [Crossref]

20. University of Southeastern Philippines (n.d.). SePhi: USeP Chatbot AI Teaser. Available: https://www.youtube.com/watch?v=GcmnanHkmYU [Google Scholar] [Crossref]

• The Impact of Ownership Structure on Dividend Payout Policy of Listed Plantation Companies in Sri Lanka
• Urban Sustainability in North-East India: A Study through the lens of NER-SDG index
• Performance Assessment of Predictive Forecasting Techniques for Enhancing Hospital Supply Chain Efficiency in Healthcare Logistics
• The Fractured Self in Julian Barnes' Postmodern Fiction: Identity Crisis and Deflation in Metroland and the Sense of an Ending
• Impact of Flood on the Employment, Labour Productivity and Migration of Agricultural Labour in North Bihar