AI-Powered Mobile Application ‘Play Home’ for Autism Education

AI-Powered Mobile Application ‘Play Home’ for Autism Education

Haslinah Mohd Nasir¹, Noor Mohd Ariff Brahin², Suraya Zainuddin³, Ida Syafiza Md Isa⁴, Maizatul Amirah Azhar⁵, Nurul Syazwani binti Rohizat⁶, Norezmi Jamal^7*

^1,2,3,4,5,6Faculty of Electronics & Computer Technology & Engineering, University Technical Malaysia Melaka, Jalan Hang Tuah Jaya, Melaka, 76100, Malaysia

⁷Faculty of Engineering Technology, University Tun Hussein Onn Malaysia, Pagoh Higher Education Hub, 84600, Pagoh, Johor, Malaysi

^*Corresponding author

DOI: https://dx.doi.org/10.47772/IJRISS.2025.9010179

Received: 18 August 2025; Accepted: 23 August 2025; Published: 20 September 2025

ABSTRACT

This project describes the development of an AI-powered mobile application ‘Play Home’ which was designed as a teaching aid for children with autism. To enhance the learning process, the application employ machine learning (ML) techniques for image classification and object recognition. The AI models were trained using Google’s Teachable Machine platform which simplified and improved the model construction process. Meanwhile, the application ‘Play Home’ was built using MIT App Inventor which offers a low-code environment ideal for rapid development and deployment. ‘Play Home’ consists of five modules, each designed to help autistic children develop their cognitive and sensory skills. These modules improve attention span, recognition abilities, and overall participation through dynamic and engaging tasks. The integration of AI enables ‘Play Home’ to respond in real-time, making the learning experience more personalized and adaptable. The ‘Play Home’ includes a real-time database for progress monitoring, and performance metrics. This allows caregivers and educators to monitor learning outcomes and tailor content to individual students’ needs. Initial testing shows that the AI-powered application ‘Play Home’ works well, and has an appealing, instructive interface that caters to the special needs of children with autism. The use of interactive elements and responsive feedback has shown potential in making learning more enjoyable and accessible for the target group especially for autistic children. Future research will concentrate on real-world validation by working with autism centers to evaluate the educational impact and usability in real-world scenarios. The findings will guide further enhancements, ensuring the application meets the educational and developmental needs of children with autism.

Keywords— AI-powered mobile application; Machine Learning; autism education; Teachable Machine; MIT App Inventor; Interactive learning

INTRODUCTION

Autism Spectrum Disorder (ASD) is a complex neurodevelopmental condition marked by persistent challenges in social communication, restricted interests, and repetitive behaviors. The severity and manifestation of these characteristics can vary greatly, resulting to a broad spectrum of needs and abilities among individuals with autism. Early diagnosis and intervention are crucial for better developmental outcomes, as appropriate educational and behavioral strategies have been shown to significantly enhance cognitive, social, and communication skills in children with ASD [1], [2].

In Malaysia, the number of school-age children with autism has increased dramatically in recent years, from 6.34 per 1,000 in 2018 to 9.29 per 1,000 in 2022 [3]. This growth emphasizes the critical need for innovative and accessible educational resources or tools to support children on the autism spectrum. Conventional teaching methods often fall short in addressing the diverse and personalized learning needs of autistic children [4]. As a result, autism education must employ a variety of strategies, including structured teaching, visual assistance, and individualized education plans (IEPs) that are specifically suited to the unique cognitive and behavioral profiles of each learner [5].

Recent advancements in Artificial Intelligence (AI) have created new opportunities for personalized and adaptive learning experiences. AI-based educational applications are increasingly being explored as tools for promoting cognitive and communication development in children with and without ASD [6], [7]. These systems use machine learning (ML) algorithms to assess user behavior and adapt learning content dynamically, ensuring that educational interventions are appropriate for the learner’s pace, strengths, and challenges [8]. For example, AI can adjust the complexity, delivery style, and feedback mechanisms of a lesson to maximize user engagement and retention [9], [10].

AI-powered tools can also help with language and communication development by utilizing natural language processing (NLP) approaches. Speech recognition, language modeling, and real-time feedback mechanisms have showed promise in improving verbal and nonverbal communication in children with ASD [11], [12]. Furthermore, AI-driven applications can facilitate collaborative learning by allowing engagement with peers, educators, and therapists through virtual settings or gamified tasks, boosting social connection and cooperative behavior [13], [14].

Several studies have shown the effectiveness of AI-assisted interventions in autism education. For instance, Habibi et al. (2025) discovered that AI-based tailored learning environments significantly improved the attention and learning outcomes of children with ASD [15]. Similarly, Chandrakant (2023) highlighted the potential of AI in delivering personalized feedback and promoting social engagement through gamified learning modules which produce efficient and delightful educational experiences for students [16].

Given this context, there is a growing need for intelligent mobile applications that not only deliver educational content but also adapt to the specific learning profiles of autistic children. This paper presents the development of an AI-powered mobile application as known as ‘Play Home’ that supports autism education through image classification, object detection, and real-time feedback. The application integrates with a cloud database to track user performance and adapt the content accordingly. Developed using Teachable Machine and MIT App Inventor, the tool aims to offer a highly interactive and user-centered experience, ultimately contributing to more inclusive and effective educational practices for children with ASD.An easy way to comply with the conference paper formatting requirements is to use this document as a template and simply type your text into it.

METHODOLOGY

This project adopted a structured approach to develop an AI-powered application targeted for supporting autism education called ‘Play Home’. The project was divided into four main stages: Module design, AI model training, Mobile application development and Data integration with cloud-based monitoring as shown in Fig. 1.

Fig. 1 Project block diagram

Stage 1: Module Design

The initial stage involved designing learning modules that are both fun and educational. These modules were carefully planned to focus on specific skills that children with autism often need help with such as attention span, emotional recognition, and physical coordination. rowing from best practices in autism education [17]. Five interactive modules were created as presented in Table 1.

TABLE1 interactive modules with description

Every module was designed to be simple, colorful, and visually clear, known to reduce cognitive overload and enhance engagement for neuro-diverse learners so that children can stay focused and feel encouraged throughout the learning process [18], [19].

Stage 2: AI Model Training

The second stage involved the application of machine learning to power the interactive capabilities of the modules. Google’s Teachable Machine was used as the training platform due to its accessibility and effectiveness for non-expert users [20].

Prior for training purposes, many sample images were collected and used to teach the AI what different objects, shapes, emotions, or movements look like. Once trained, these models were exported and prepared to work smoothly within the app. Care was taken to ensure the models gave fast and accurate feedback, even when lighting or camera angles varied.

The models were trained to recognize shapes, facial emotions, and body poses using convolutional neural networks (CNNs), employing transfer learning methods to improve accuracy and performance in Google’s Teachable Machine platform. Anupriya et al. and Varghese et al. found that AI-based classification systems have shown promising results in enhancing interaction for users with ASD [21], [22]. AI model integration has become an effective tool for allowing personalized, adaptive learning in assistive educational systems [23], [24].

Stage 3: Mobile Application Development

In the third stage, the trained AI models were integrated into a mobile app using MIT App Inventor. This platform allows visual programming via drag-and-drop blocks, making it ideal for designing user interfaces and implementing logic [25]. The app was built to be intuitive and user-friendly, with features including real-time camera input, visual feedback, and progress monitoring. This stage ensure that the app would be accessible and enjoyable for children with autism, aligning with findings by Somerton as well as Muchagata and Ferreira, who found that apps designed with clear visuals and few distractions improve learning outcomes in ASD users [26], [27].

The app included features such as user registration, module selection, interactive camera-based tasks, and real-time feedback. The user interface was designed with accessibility in mind with large icons, vivid images, and voice instructions to support children with limited literacy or verbal skills. This design approach supports the principles of Universal Design for Learning (UDL), which emphasize the necessity of making educational experiences flexible and inclusive [28], [29].

The app includes a clean and interactive interface that’s easy for children to navigate. In addition, camera-based interaction with trained AI models, friendly audio and visual responses when children complete tasks, as well as progress tracking to help parents or educators to see how children are doing.

The app ‘Play Home’ was designed with children in mind, with large buttons, clear instructions and positive reward for each completed activity.

Stage 4: Cloud-based Integration

The final stage of development involved integrating a cloud-based system that used Firebase to store and manage user data. This enabled the seamless storage and retrieval of user data, such as module scores, completion status, and timestamps. The database integration enabled caregivers and educators to monitor user performance over time and gain insights into each child’s progress. The application provides not only real-time engagement but also supports evidence-based monitoring for long-term developmental tracking [30]. The integration of such a database aligns with broader efforts in digital health and education to promote remote monitoring and personalized learning pathways.

RESULTS & ANALYSIS

The development and evaluation of the ‘Play Home’ application followed a four-stage process involving module design, artificial intelligence (AI) model training, mobile application development, and cloud-based data integration. Each stage is discussed below, with attention to technical implementation, user interaction, and implications for autism education.

The five modules developed for the ‘Play Home’ application was specifically tailored to address cognitive, motor, and emotional skill development in children with Autism Spectrum Disorder (ASD). These are shown in Table 2.

TABLE2 Play home interfaces

No.	Interface	Description
1		Login authentication
2		Menu Selection for five module: Module 1: Catch the cat Module 2: Match number with picture Module 3: Shape Detection Module 4: Pose Detector Module 5: Emotion Face
3		Module 1: Catch the cat interface
4		Module 2: Match Number with Pictures interface
5		Module 3: Shape detection interface
6		Module 4: Pose detection interface
7		Module 5: Emotion Face interface

AI functionality was integrated into the modules through image classification and pose estimation models trained using Google’s Teachable Machine. The shape and object detection models performed with high accuracy across varied lighting and user conditions. The emotion detection model reliably identified basic emotional states such as “happy,” but performance was lower for more nuanced expressions like “surprise” and “anger.” These findings align with existing literature highlighting the limitations of AI in detecting subtle emotional cues, particularly in child users who may not express emotions consistently.

Pose estimation, a more complex task, showed mixed results. While the model detected Level 1 poses over 90% accuracy, it declined 7% by Level 3. This drop indicates either excessive complexity in the needed poses or limitations in the training dataset, emphasizing the need for improved real-time visual feedback and enhanced training diversity. Nonetheless, incorporating pose recognition remains a novel and interesting approach to fostering body awareness in children with ASD.

‘Play Home’ was designed with accessibility and convenience of use as primary goals. The interface features included large, clearly labeled icons, minimal text, vibrant visuals, and friendly auditory cues. Children navigated the app freely, following intuitive screens that guided them from registration to module completion.

Initial testing proved that integration between the app and the AI models are well functional. Real-time feedback was successfully delivered with low latency using on-device camera. Performance was consistent across Android devices. However, there were some issues with camera calibration, particularly in modules that required exact posture alignment, suggesting that environmental variability such as lighting or background clutter can clutter AI responsiveness.

The high user performance in simpler modules and drop-off in complex ones highlight the need of adaptive interfaces that respond to a user’s ability level. Adaptive scaffolding, wherein difficulty dynamically adjusts to a child’s performance, may improve sustained engagement and learning outcomes.

The integration of Firebase as shown in Fig. 2 as the cloud backend enabled real-time user data storage and retrieval. Key data includes module completion status, timestamped performance scores, and login activity. The Analysis of these datasets provided useful insights into user behavior patterns.

Fig. 2 Firebase Data Interface

Firebase data presents that users who frequently revisited introductory modules such as Catch the Cat and Match Number with Pictures, while less time was spent on more cognitively difficult modules. These patterns highlight the value of data-driven personalization in autism education applications. By tracking learning progress longitudinally, the system provides caregivers and educators with meaningful metrics to inform individualized education plans (IEPs). The analysis through the database is shown in Table 3.

TABLE3 Analysis for each module

No.	Module	Analysis
1	Catch the Cat
2	Match Number with Pictures
3	Shape Detector
4	Pose Detector
5	Emotion Face

Furthermore, the availability of performance analytics enables stakeholders to assess module effectiveness and identify areas for content revision. Such evidence-based practice is increasingly recognized as essential in special needs education technology research.

The results of this study confirm that an AI-powered mobile application, when designed with user-centered and developmentally appropriate principles, can effectively support learning in children with autism. The integration of AI allows for interactive and personalized experiences that traditional tools may lack. While image and object detection functionalities were robust, further development is needed for complex emotion recognition and movement-based tasks.

Importantly, the app’s use of Firebase for progress monitoring creates new opportunities for caregiver involvement and remote support. With further refinement, this application could be implemented in home, therapy, and classroom environments as a scalable solution to individualized autism education.

CONCLUSION

This study has demonstrated the successful design, development, and preliminary evaluation of Play Home, an AI-powered mobile application aimed at enhancing educational experiences for children with autism spectrum disorder (ASD). The application integrates five interactive modules which each targeting specific cognitive, behavioral, and emotional skills through real-time AI functionalities such as image classification, pose detection, and facial emotion recognition.

The use of Google’s Teachable Machine enabled the training of lightweight, responsive AI models suitable for mobile devices, while MIT App Inventor provided a user-friendly development environment for constructing a child-centric interface. Furthermore, integration with Firebase allowed for effective real-time data tracking, supporting continuous performance monitoring and learning analytics.

Evaluation results indicate that the app is functional, accessible, and engaging for its intended users. Modules focusing on basic cognitive tasks (e.g., number matching and object detection) showed high user success rates, while more complex tasks such as emotion and pose recognition highlighted areas for further improvement in both AI training depth and user guidance.

Overall, the project demonstrates how artificial intelligence, when applied thoughtfully within a pedagogical framework, can be a powerful tool in personalized autism education. The Play Home app represents a step toward more inclusive, adaptive, and data-informed learning technologies for children on the autism spectrum.

ACKNOWLEDGMENT

We would like to thank University Technical Malaysia (UTeM) for preparing research workplace in Faculty of Electronics & Computer Technology & Engineering. Not forgotten to the participant that involved during the project testing.

REFERENCES

1. H. Hodges, C. Fealko, and N. Soares, “Autism spectrum disorder: Definition, epidemiology, causes, and clinical evaluation,” Feb. 01, 2020, AME Publishing Company. doi: 10.21037/tp.2019.09.09.
2. P. Rajabi, A. S. Noori, and J. Sargolzaei, “Autism spectrum disorder and various mechanisms behind it,” Dec. 01, 2024, Elsevier Inc. doi: 10.1016/j.pbb.2024.173887.
3. S. Niza Shair et al., “Prevalence of Autism Spectrum Disorder Among School-Age Children in Malaysia: Analysis by Age and States.”
4. Y. Wang, “A Comparative Study on the Effectiveness of Traditional and Modern Teaching Methods,” in Proceedings of the 2022 5th International Conference on Humanities Education and Social Sciences (ICHESS 2022), Atlantis Press SARL, 2022, pp. 270–277. doi: 10.2991/978-2-494069-89-3_32.
5. L. Petersson-Bloom and M. Holmqvist, “Strategies in supporting inclusive education for autistic students—A systematic review of qualitative research results,” 2022, SAGE Publications Ltd. doi: 10.1177/23969415221123429.
6. C. Collazos, G. E. Constain, and F. Dalipi, “Cutting-edge communication and learning assistive technologies for disabled children: An artificial intelligence perspective.” [Online]. Available: http://aima.cs.berkeley.edu/
7. K.-J. Laak and J. Aru, “AI and personalized learning: bridging the gap with modern educational goals,” Apr. 2024, [Online]. Available: http://arxiv.org/abs/2404.02798
8. I. Gligorea, M. Cioca, R. Oancea, A. T. Gorski, H. Gorski, and P. Tudorache, “Adaptive Learning Using Artificial Intelligence in e-Learning: A Literature Review,” Dec. 01, 2023, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/educsci13121216.
9. H. Mohd Nasir et al., “AI Educational Mobile App Using Deep Learning Approach,” International Journal on Informatics Visualization, vol. 7, no. 3, pp. 952–958, 2023, doi: 10.30630/joiv.7.3.1247.
10. H. Yaseen, A. S. Mohammad, N. Ashal, H. Abusaimeh, A. Ali, and A. A. A. Sharabati, “The Impact of Adaptive Learning Technologies, Personalized Feedback, and Interactive AI Tools on Student Engagement: The Moderating Role of Digital Literacy,” Sustainability (Switzerland), vol. 17, no. 3, Feb. 2025, doi: 10.3390/su17031133.
11. D. Kristiawan, K. Bashar, and D. A. Pradana, “Artificial Intelligence in English Language Learning: A Systematic Review of AI Tools, Applications, and Pedagogical Outcomes,” The Art of Teaching English as a Foreign Language (TATEFL), vol. 5, no. 2, pp. 207–218, Nov. 2024, doi: 10.36663/tatefl.v5i2.912.
12. D. Minoli and B. Occhiogrosso, “Current and Evolving Applications to Speech Processing,” in AI Applications to Communications and Information Technologies, IEEE, 2023, pp. 117–171. doi: 10.1002/9781394190034.ch3.
13. W. Strielkowski, V. Grebennikova, A. Lisovskiy, G. Rakhimova, and T. Vasileva, “AI-driven adaptive learning for sustainable educational transformation,” Apr. 01, 2024, John Wiley and Sons Ltd. doi: 10.1002/sd.3221.
14. F. Ouyang and L. Zhang, “AI-driven learning analytics applications and tools in computer-supported collaborative learning: A systematic review,” Aug. 01, 2024, Elsevier Ltd. doi: 10.1016/j.edurev.2024.100616.
15. F. Habibi, S. Sedaghatshoar, T. Attar, M. Shokoohi, A. Kiani, and A. N. Malek, “Revolutionizing education and therapy for students with autism spectrum disorder: a scoping review of AI-driven tools, technologies, and ethical implications,” AI and Ethics, vol. 5, no. 3, pp. 2055–2070, Jun. 2025, doi: 10.1007/s43681-024-00608-1.
16. Soni Maitrik Chandrakant, “Gamified learning and NLP: Enhancing student engagement through AI-driven interactive education models,” International Journal of Science and Research Archive, vol. 9, no. 1, pp. 813–824, Jun. 2023, doi: 10.30574/ijsra.2023.9.1.0496.
17. Y. Bolourian, A. Losh, N. Hamsho, A. Eisenhower, and J. Blacher, “General Education Teachers’ Perceptions of Autism, Inclusive Practices, and Relationship Building Strategies,” J Autism Dev Disord, vol. 52, no. 9, pp. 3977–3990, Sep. 2022, doi: 10.1007/s10803-021-05266-4.
18. K. G. Ranaweera, J. Praneetha, W. P. S. De Silva, K. R. B. N. Kempitiya, I. Gamage, and T. Thilakarthna, “Personalized and Interactive Learning Platform for Students with Autism Spectrum Disorder,” in ICAC 2023 – 5th International Conference on Advancements in Computing: Technological Innovation for a Sustainable Economy, Proceedings, Institute of Electrical and Electronics Engineers Inc., 2023, pp. 828–833. doi: 10.1109/ICAC60630.2023.10417404.
19. A. Tulshan and N. Raul, “Krisha: An Interactive Mobile Application for Autism Children,” in International Conference on Advances in Computing and Data Sciences, Springer, Singapore, 2019, pp. 207–218. doi: 10.1007/978-981-13-9942-8_20.
20. H. Kacorri, “Teachable machines for accessibility,” ACM SIGACCESS Accessibility and Computing, no. 119, pp. 10–18, Nov. 2017, doi: 10.1145/3167902.3167904.
21. Anupriya, A. Singh, J. S. Jadon, S. R. Nayak, and R. Kumar, “Conversation AI for Verbal Children with ASD to Facilitate Social Interaction,” in Proceedings of 3rd International Conference on Smart Computing and Cyber Security, P. K. Pattnaik, M. Sain, and A. A. Al-Absi, Eds., Singapore: Springer Nature Singapore, 2024, pp. 295–304.
22. E. B. Varghese, M. Qaraqe, and D. Al-Thani, “Utilizing Artificial Intelligence to Support Autism Research,” in The Impact of Artificial Intelligence on Societies: Understanding Attitude Formation Towards AI, C. Montag and R. Ali, Eds., Cham: Springer Nature Switzerland, 2025, pp. 87–108. doi: 10.1007/978-3-031-70355-3_7.
23. L. Y. Tan, S. Hu, D. J. Yeo, and K. H. Cheong, “Artificial intelligence-enabled adaptive learning platforms: A review,” Dec. 01, 2025, Elsevier B.V. doi: 10.1016/j.caeai.2025.100429.
24. A. Kumar and S. Srivastava, “The Analysis of AI Based Algorithms in Personalized and Adaptive E-learning Experiences,” in 2nd IEEE International Conference on IoT, Communication and Automation Technology, ICICAT 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 11–16. doi: 10.1109/ICICAT62666.2024.10923335.
25. E. W. Patton, M. Tissenbaum, and F. Harunani, “MIT App Inventor: Objectives, Design, and Development,” in Computational Thinking Education, S.-C. Kong and H. Abelson, Eds., Singapore: Springer Singapore, 2019, pp. 31–49. doi: 10.1007/978-981-13-6528-7_3.
26. M. Somerton, “Developing an educational app for students with autism,” Front Educ (Lausanne), vol. 7, Nov. 2022, doi: 10.3389/feduc.2022.998694.
27. J. Muchagata and A. Ferreira, “Visual schedule: A mobile application for autistic children – Preliminary study,” in ICEIS 2019 – Proceedings of the 21st International Conference on Enterprise Information Systems, SciTePress, 2019, pp. 452–459. doi: 10.5220/0007732804520459.
28. Q. I. Almeqdad, A. M. Alodat, M. F. Alquraan, M. A. Mohaidat, and A. K. Al-Makhzoomy, “The effectiveness of universal design for learning: A systematic review of the literature and meta-analysis,” 2023, Taylor and Francis Ltd. doi: 10.1080/2331186X.2023.2218191.
29. Mohamamad Hasan, Afaque Nadeem Khan, and Tabassum Fatima, “Universal Design for Learning (UDL): A Paradigm Shift in Inclusive Education,” Journal of Education and Practice, vol. 14, no. 25, pp. 1–4, Sep. 2023, doi: 10.7176/jep/14-25-01.
30. R. Kumar, N. Thakur, A. Saeed, and C. Jaiswal, “Enhancing Data Analytics Using AI-Driven Approaches in Cloud Computing Environments,” Software Engineering, vol. 11, no. 2, pp. 11–18, Oct. 2024, doi: 10.5923/j.se.20241102.01