An Explainable and Adaptive Multimodal Framework for Real-Time Digital Education Quality Evaluation

Digital education quality evaluation increasingly relies on multimodal, AI-assisted systems, yet the literature remains fragmented: models that are explainable are rarely real-time, models that are real-time are rarely multimodal or adaptive, and models that fuse multiple modalities typically trade away transparency for accuracy. This study addresses that gap by extending the MAC-HASA model into EAM-HASA (Explainable Adaptive Multimodal-HASA), a neurosymbolic, multi-objective framework designed to unify explainable reasoning, cross-modal fusion, and continuous real-time adaptation within a single architecture for evaluating digital education quality. The framework couples cross-modal attention fusion of motion, audio/text, and interaction data with a post-hoc SHAP-based explanation layer and a hierarchical reinforcement-learning policy agent that retunes evaluation weights as instructional conditions change. As a proof-of-concept, the predictive component was trained and validated on a public multimodal physical-education teaching dataset (1,000 session records from 200 students) using an XGBoost classifier benchmarked against MAC-HASA and a Transformer baseline. The classifier achieved 92.0% accuracy and an 89.5% macro F1-score in predicting teaching-effectiveness category, and SHAP attribution showed that engagement and instructional-clarity indicators, rather than raw motion-sensor signals, were the dominant drivers of the evaluation outcome. These results support the feasibility of the predictive core and the pedagogical relevance of its explanations, while the full explainable, adaptive, governance-aware architecture is formalized algorithmically and proposed for empirical validation on multimodal, multi-institutional data in future work.

Computer Science

1. Akoka, J., Comyn-Wattiau, I., Prat, N., & Storey, V. C. (2023). Knowledge contributions in design science research: Paths of knowledge types. Decision Support Systems, 166, Article 113898. https://doi.org/10.1016/j.dss.2022.113898 [Google Scholar] [Crossref]

2. Almubarak, A., Alhalabi, W., Albidewi, I., & Alharbi, E. (2025). An AI-powered framework for assessing teacher performance in classroom interactions: A deep learning approach. Frontiers in Artificial Intelligence, 8, Article 1553051. https://doi.org/10.3389/frai.2025.1553051 [Google Scholar] [Crossref]

3. Braun, V., & Clarke, V. (2022). Thematic analysis: A practical guide. SAGE Publications. [Google Scholar] [Crossref]

4. Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023). A review of trustworthy and explainable artificial intelligence (XAI). IEEE Access, 11, 78994–79015. https://doi.org/10.1109/ACCESS.2023.3294569 [Google Scholar] [Crossref]

5. Chango, W., Lara, J. A., Cerezo, R., & Romero, C. (2022). A review on data fusion in multimodal learning analytics and educational data mining. WIREs Data Mining and Knowledge Discovery, 12(4), Article e1458. https://doi.org/10.1002/widm.1458 [Google Scholar] [Crossref]

6. Che, L., Guo, P., Isleem, H. F., & Wang, Z. (2025). The necessity of multimodal feedback for learning effective pedagogical policies with reinforcement learning. Scientific Reports, 15, Article 29892. https://doi.org/10.1038/s41598-025-29892-5 [Google Scholar] [Crossref]

7. Chen, M.-Y., et al. (2025). A realtime classroom assessment system for analysis of students' evaluation of teaching through a deep learning and emotional contagion mechanism. International Journal of Interactive Multimedia and Artificial Intelligence. https://www.ijimai.org/index.php/ijimai/article/view/859 [Google Scholar] [Crossref]

8. Hasib, K. M., Rahman, F., Hasnat, R., & Alam, M. G. R. (2022). A machine learning and explainable AI approach for predicting secondary school student performance. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC). [Google Scholar] [Crossref]

9. Jang, Y., Choi, S., Jung, H., & Kim, H. (2022). Practical early prediction of students' performance using machine learning and explainable AI. Education and Information Technologies, 27(9), 12855–12889. [Google Scholar] [Crossref]

10. Khosravi, H., Buckingham Shum, S., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, Article 100074. https://doi.org/10.1016/j.caeai.2022.100074 [Google Scholar] [Crossref]

11. Liao, C.-H., & Wu, J.-Y. (2022). Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance. Computers & Education, 190, Article 104599. https://doi.org/10.1016/j.compedu.2022.104599 [Google Scholar] [Crossref]

12. Liu, H. (2025). Personalized learning support system for special education: A real-time feedback mechanism based on deep reinforcement learning. Frontiers in Psychology, 16, Article 1658698. https://doi.org/10.3389/fpsyg.2025.1658698 [Google Scholar] [Crossref]

13. Liu, T., Wang, C., Chang, L., & Gu, T. (2022). Predicting high-risk students using learning behavior. Mathematics, 10(14), Article 2483. [Google Scholar] [Crossref]

14. Luo, H. (2023). Editorial: Advances in multimodal learning: Pedagogies, technologies, and analytics. Frontiers in Psychology, 14, Article 1286092. https://doi.org/10.3389/fpsyg.2023.1286092 [Google Scholar] [Crossref]

15. Mousa, K., Zhang, Z. L., Sumarliah, E., & Hamdan, I. K. A. (2024). The impact of cloud computing adoption on firm performance among SMEs in Palestine: A machine learning approach. International Journal of Intelligent Information Technologies, 20(1), 1–24. https://doi.org/10.4018/IJIIT.338715 [Google Scholar] [Crossref]

16. Nakkach, C., Zrelli, A., & Ezzedine, T. (2023). Long-term energy forecasting system based on LSTM and deep extreme machine learning. Intelligent Automation and Soft Computing, 37(1), 545–560. https://doi.org/10.32604/iasc.2023.036385 [Google Scholar] [Crossref]

17. Narayanan, E., & Muthukumar, B. (2022). A machine learning framework for providing data integrity and confidentiality for sensitive data cloud applications. International Journal of System Assurance Engineering and Management. https://doi.org/10.1007/s13198-022-01741-y [Google Scholar] [Crossref]

18. Pan, Y., Wu, J., Ju, R., Zhou, Z., Gu, J., Zeng, S., Yuan, L., & Li, M. (2022). A multimodal framework for automated teaching quality assessment of one-to-many online instruction videos. In Proceedings of the 26th International Conference on Pattern Recognition (ICPR) (pp. 1777–1783). IEEE. https://doi.org/10.1109/ICPR56361.2022.9956185 [Google Scholar] [Crossref]

19. Schoormann, T., Möller, F., Chandra Kruse, L., & Otto, B. (2024). BAUSTEIN—A design tool for configuring and representing design research. Information Systems Journal, 34(6), 1871–1901. https://doi.org/10.1111/isj.12516 [Google Scholar] [Crossref]

20. Selamat, A., Ibrahim, R., Alrashidi, M., & Fujita, H. (2024). Social recommender system based on CNN incorporating tagging and contextual features. Journal of Cases on Information Technology, 26(1), 1–20. https://doi.org/10.4018/JCIT.335524 [Google Scholar] [Crossref]

21. Smit, N., van Dijk, M., de Bot, K., & Lowie, W. (2022). The complex dynamics of adaptive teaching: Observing teacher-student interaction in the language classroom. International Review of Applied Linguistics in Language Teaching, 60(1), 23–40. https://doi.org/10.1515/iral-2018-0138 [Google Scholar] [Crossref]

22. Trabelsi, Z., Alnajjar, F., Parambil, M. M. A., Gochoo, M., & Ali, L. (2023). Real-time attention monitoring system for classroom: A deep learning approach for student's behavior recognition. Big Data and Cognitive Computing, 7(1), Article 48. https://doi.org/10.3390/bdcc7010048 [Google Scholar] [Crossref]

23. Tuunanen, T., Winter, R., & vom Brocke, J. (2024). Dealing with complexity in design science research: A methodology using design echelons. MIS Quarterly, 48(2), 427–458. https://doi.org/10.25300/MISQ/2023/16700 [Google Scholar] [Crossref]

24. Xu, W., Mahmud, R., & Hoo, W. L. (2024). Literature review: Are automated essay scoring systems competent in real-life education scenarios? IEEE Access, 12, 77639–77657. https://doi.org/10.1109/ACCESS.2024.3399163 [Google Scholar] [Crossref]

25. Yan, L., Echeverria, V., Jin, Y., Fernandez-Nieto, G., Zhao, L., Li, X., Alfredo, R., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Evidence-based multimodal learning analytics for feedback and reflection in collaborative learning. British Journal of Educational Technology, 55(5), 1900–1925. https://doi.org/10.1111/bjet.13496 [Google Scholar] [Crossref]

• Assessment of the Role of Artificial Intelligence in Repositioning TVET for Economic Development in Nigeria
• Teachers’ Use of Assure Model Instructional Design on Learners’ Problem Solving Efficacy in Secondary Schools in Bungoma County, Kenya
• “E-Booksan Ang Kaalaman”: Development, Validation, and Utilization of Electronic Book in Academic Performance of Grade 9 Students in Social Studies
• Analyzing EFL University Students’ Academic Speaking Skills Through Self-Recorded Video Presentation
• Major Findings of The Study on Total Quality Management in Teachers’ Education Institutions (TEIs) In Assam – An Evaluative Study