AISTY: An Explainable AI-Driven Vision-Based Adaptive Learning System for Children with Autism Spectrum Disorder

The educational needs of children with autism spectrum disorder (ASD) require individualized approaches because they face major difficulties with communication and social skills and learning processes. The diverse requirements of children with ASD remain unmet by conventional educational approaches because these methods fail to provide suitable flexibility and expandability. The research develops Adaptive Interface System for Tracking Engagement (AISTY) as an explainable AI-based learning module which solves the current educational needs of children with ASD. The system AISTY uses computer vision to track behavioural data through real-time analysis while delivering interactive learning content that adjusts to student needs. The system uses a custom Convolutional Neural Network (CNN) model to analyse facial expressions for engagement and emotional state classification (Happy, Sad, Neutral, Surprise, Fear, Angry, Disgust) with 78% accuracy while operating at 14-16 frames per second on typical computer equipment. The system contains three learning modules (Mathematics, Sing-Along, and Mini-Games) which modify their content according to the child's current state. The system includes an explainable AI (XAI) dashboard that uses SHAP-inspired visualizations to show caregivers exactly what elements influence the system's adaptation decisions. The usability assessment with parents and a special education teacher validated the system's operational capabilities and user-friendly interface and confirmed its worth. The research shows AISTY offers a flexible autism education system which proves that explainable adaptive AI can be effectively used in special needs education to boost student participation and academic success.

autism spectrum disorder (ASD), Artificial Intelligence

1. Martínez-González, A.E., Cervin, M. and Piqueras, J.A., (2022). Relationships between emotion regulation, social communication and repetitive behaviors in autism spectrum disorder. Journal of autism and developmental disorders, 52(10), pp.4519-4527. [Google Scholar] [Crossref]

2. World Health Organization: WHO. (2023), November 15). Autism. https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders [Google Scholar] [Crossref]

3. Jiar, Y.K., (2014). Factors Associated with Diagnosis of Autism Spectrum Disorder (ASD) under the Age of 24 Months in Malaysia. Sains Humanika, 2(1). [Google Scholar] [Crossref]

4. Lei, J., & Ventola, P. (2017). Pivotal response treatment for autism spectrum disorder: current perspectives. Neuropsychiatric Disease and Treatment, Volume 13, 1613–1626. [Google Scholar] [Crossref]

5. Lindroth, H., Nalaie, K., Raghu, R., Ayala, I.N., Busch, C., Bhattacharyya, A., Moreno Franco, P., Diedrich, D.A., Pickering, B.W. and Herasevich, V., (2024). Applied artificial intelligence in healthcare: a review of computer vision technology application in hospital settings. Journal of Imaging, 10(4), p.81. [Google Scholar] [Crossref]

6. Ahmmad, J., Al-Dayel, O.A., Khan, M.A. and Mahmood, T., (2025). AI-assisted technology optimization in disability support systems using fuzzy rough MABAC decision-making. Scientific Reports, 15(1), p.18335. [Google Scholar] [Crossref]

7. Hulsen, T., (2023). Explainable artificial intelligence (XAI): concepts and challenges in healthcare. Ai, 4(3), pp.652-666. [Google Scholar] [Crossref]

8. Gitimoghaddam, M., Chichkine, N., McArthur, L., Sangha, S.S. and Symington, V., (2022). Applied behavior analysis in children and youth with autism spectrum disorders: a scoping review. Perspectives on behavior science, 45(3), pp.521-557 [Google Scholar] [Crossref]

9. Schopler, E. and Van Bourgondien, M.E., (2020). Treatment and Education of Autistic and Related Communication Children. In Autistic adults at Bittersweet Farms (pp. 85-94). Routledge. [Google Scholar] [Crossref]

10. Yu, L. and Zhu, X., (2018). Effectiveness of a SCERTS model-based intervention for children with autism spectrum disorder (ASD) in Hong Kong: A pilot study. Journal of Autism and Developmental Disorders, 48(11), pp.3794-3807. [Google Scholar] [Crossref]

11. Bone, C., Simmonds-Buckley, M., Thwaites, R., Sandford, D., Merzhvynska, M., Rubel, J., Deisenhofer, A., Lutz, W., & Delgadillo, J. (2021). Dynamic prediction of psychological treatment outcomes: development and validation of a prediction model using routinely collected symptom data. The Lancet Digital Health, 3(4), e231–e240. https://doi.org/10.1016/s2589-7500(21)00018-2 [Google Scholar] [Crossref]

12. Cai, Y., Li, X., & Li, J. (2023). Emotion recognition using different sensors, emotion models, methods and datasets: A Comprehensive review. Sensors, 23(5), 2455. https://doi.org/10.3390/s23052455 [Google Scholar] [Crossref]

13. Goodwin, M.S., Intille, S.S., Albinali, F. and Velicer, W.F., (2011). Automated detection of stereotypical motor movements. Journal of autism and developmental disorders, 41(6), pp.770-782. 1-12. [Google Scholar] [Crossref]

14. Seyderhelm, A.J. and Blackmore, K., 2021. Quantifying in-game task difficulty using real-time cognitive load. [Google Scholar] [Crossref]

• The Role of Artificial Intelligence in Revolutionizing Library Services in Nairobi: Ethical Implications and Future Trends in User Interaction
• ESPYREAL: A Mobile Based Multi-Currency Identifier for Visually Impaired Individuals Using Convolutional Neural Network
• Comparative Analysis of AI-Driven IoT-Based Smart Agriculture Platforms with Blockchain-Enabled Marketplaces
• AI-Based Dish Recommender System for Reducing Fruit Waste through Spoilage Detection and Ripeness Assessment
• SEA-TALK: An AI-Powered Voice Translator and Southeast Asian Dialects Recognition