Enabling Learning of Programming through Educational Chatbot

Enabling Learning of Programming through Educational Chatbot

Nantha Kumar Subramaniam

Open University Malaysia (OUM)

DOI: https://doi.org/10.51584/IJRIAS.2024.912042

Received: 11 December 2024; Accepted: 16 December 2024; Published: 17 January 2025

ABSTRACT

This research investigates the integration of AI-powered chatbots as transformative tools to enhance the teaching and learning of programming, specifically targeting Java programming in an open and distance learning (ODL) environment. The chatbot was conceptualized and developed using principles from constructivist, problem-based, and active learning theories to support a learner-centric approach. Designed to offer multimodal content, real-time feedback, and interactive problem-solving exercises, the chatbot aims to reduce cognitive load and facilitate a deeper understanding of complex programming concepts. The implementation involved deploying the chatbot in an online Java programming course over one semester, engaging 32 ODL learners in a blended learning mode. Learners utilized the chatbot for interactive, one-on-one tutoring sessions, benefiting from multimodal materials such as animations, visualizations, and interactive exercises. These resources supported learners in addressing challenging concepts and completing programming tasks, enhancing their engagement and comprehension. Evaluation was conducted through a structured survey using a 4-point Likert scale, capturing learner perceptions on the chatbot’s effectiveness. The findings revealed high levels of satisfaction, with learners highlighting the chatbot’s ability to provide  learning support and its unique learning experience. The survey further underscored the chatbot’s role in alleviating cognitive load and bridging the gap between abstract programming theories and their practical applications. The study demonstrates the potential of AI-powered chatbots to transform programming education.

Keywords: artificial intelligence, chatbot, programming education

INTRODUCTION

The integration of chatbots into educational settings has garnered significant attention in recent years, with growing evidence supporting their potential to revolutionize the learning experience across various educational contexts. Chatbot typically is a software that simulates human conversation through  the  text-based chats. Some of the chatbots have both text chats and  voice commands  [1]. [1]  has defined chatbots as a tool that is  designed in order to simulate the  human behavior via the conversation or dialogues by using a convincing artificial colleague.  Intelligent chatbots with a strong pedagogical elements can be used to bring teaching and learning closer to the  learners in a more effective manner. This can be done by converting the commonplace lectures  into  the  sequence  of running messages in the form of  uniform  chat conversation [2].

LITERATURE REVIEW

Over the years, extensive research has explored the application of chatbot technology in education, highlighting its versatility in addressing diverse educational needs. One of the primary areas where chatbots have shown great promise is in responding to student queries. Studies by [3], [4] and [5] have demonstrated that chatbots can efficiently handle a wide range of student inquiries, providing instant answers and freeing up educators to focus on more complex instructional tasks. This capability not only improves the efficiency of student-teacher interactions but also ensures that students have access to the information they need at any time, fostering a more supportive learning environment.

In addition to addressing student queries, chatbots have proven to be effective in enhancing the understanding of complex subjects, particularly in technical fields such as programming. Researches by [6], [7], and [8] have shown that chatbots can be instrumental in breaking down difficult programming concepts into more manageable and comprehensible pieces. By offering interactive, step-by-step explanations and real-time feedback, chatbots help learners grasp intricate ideas more effectively, thereby improving their overall comprehension and retention of the material. This is particularly valuable in disciplines where the cognitive load can be high, and learners benefit from immediate clarification and guidance.

Furthermore, chatbots are increasingly being utilized to assess student performance, providing educators with valuable insights into learners’ progress. Studies by [9] and [10] have investigated how chatbots can be used to administer quizzes, track student progress, and provide personalized feedback. This automated assessment process not only saves time for educators but also allows for more frequent and consistent evaluation of student performance, enabling timely interventions when needed. The ability to offer immediate, personalized feedback helps students stay on track with their learning objectives and can lead to improved academic outcomes.

The applications of chatbots in education extend far beyond these areas, encompassing a broad spectrum of activities that include teaching, learning, administration, assessment, advisory services, and research and development. In teaching, chatbots can act as virtual tutors, guiding students through lessons and offering additional resources to reinforce learning. In learning, they can personalize educational experiences by adapting content to suit individual learning styles and paces, ensuring that each student receives the support they need to succeed. In administrative roles, chatbots can handle tasks such as scheduling, reminders, and providing information, thus improving the efficiency and accessibility of educational services.  Moreover, chatbots are playing an increasingly important role in advisory services, where they can offer guidance on academic pathways, career choices, and other educational decisions. By providing personalized advice based on a student’s academic history and interests, chatbots help students make informed decisions about their education and future careers. In research and development, chatbots are being explored for their potential to support collaborative learning environments, facilitate peer-to-peer interactions, and contribute to the creation of adaptive learning systems that respond to the evolving needs of learners.

The growing body of research on chatbots in education underscores their potential to transform the educational landscape by enhancing the learning experience, improving administrative efficiency, and supporting a wide range of educational activities. As educational institutions continue to explore the capabilities of chatbot technology, it is likely that their role will expand, offering even greater benefits to students, educators, and administrators alike. The versatility and effectiveness of chatbots make them a valuable addition to the educational toolkit, with the potential to drive significant improvements in both the quality and accessibility of education.

However, despite the many advantages chatbots bring to education, their application remains somewhat limited, especially in technical subjects such as programming. Programming education involves intricate problem-solving and logic that require deep comprehension of syntax and semantics, making it difficult for current chatbot technologies to fully meet student needs in this domain. As [11] observe, there are still many questions regarding the potential and limitations of using chatbots for learning programming languages. This highlights the necessity for further research and development to enhance the effectiveness of chatbots in handling more sophisticated educational tasks.

Purpose

The general aim of this study is to enhance students’  learning experiences through  a chatbot by designing an AI-powered chatbot. The specific objectives are given below:

• To develop an AI-powered chatbot to support learning of programming; and
• To evaluate the effectiveness of student learning via the chatbot

METHODOLOGY

This study employs a structured approach to design, implement, and evaluate the effectiveness of AI-powered chatbot in a programming education context. The methodology consists of three main phases: chatbot design and development, implementation, and evaluation. Each phase is elaborated in detail in the following sections.

Chatbot Design & Development

The author had designed and developed instructional-based an intelligent chatbot to support the learning of Java programming course delivered as an online course.  The chatbot developed in this study provides a platform to the  learners in order to comprehend the concept covered by the chatbot while conversing  with the  automated tutor in an active manner  by resolving the programming problem put forward by the chatbot. The chatbot facilitates the learners by  engaging them  in the “one-to-one” learning session with the ultimate aim of solving  the given programming problem successfully. The interface of the chatbot and some of its   main features are shown in Figure 1.

Figure 1 The interface of the chatbot

Figure 2 shows the architecture of the chatbot with various diverse components including the intelligence components.

Figure 2 Chatbot system’s architecture

The chatbot send their messages or replies to the learners via the automated tutor. These replies or messages can be in any of the following forms depending on the nature of question and at the which episode the discussion is:

• Plain text
• QR codes
• Link to multi-modal content
• Link to the interactive exercise

Depending on the type of message and the depth of the discussion, any one of the  pattern-matching algorithm, machine learning approach or  retrieval-based model is used by the chatbot in order to formulate responses  for its replies.

While interacting with the chatbot, students will be asked to go to the work area section available in the chatbot. The work area is  used by the chatbot to present the incomplete algorithms or program codes, provide visualizations about  the problem and clarify the program’s underlying principles  through visuals as shown in  the Work Area of Figure 1. Thanks to animations and visualization provided by the chatbot, the students will quickly get their  head around the concepts of the subject-matter. Context-sensitive hints  and  instruction are provided by the chatbot  in order to propel and guide the learners  towards the next steps of the discussion.

It is important to note that programming can’t be learnt simply through dialogues and conversations. Thus, the chatbot also  support multi-modal content  in order to explain the  difficult Java programming concepts in a more effective manner for easy understanding. Mental load of the students can be reduced through multi-modal content as difficult concepts can be explained and visualized easily in the chatbot.

The links and QR-code that has multi-modal  learning elements are pushed to the students at the appropriate time as replies from the chatbot  as shown in  Figure 3a and Figure 3b. All the multi-modal learning materials inside the chatbot are the highly interactive content.

As a follow-up initiative to support the learning via the chatbot, the interactive exercises  inside the chatbot provides the appropriate suggestions or  hints  to the students  to solve the  programming problem presented by the chatbot based on the solution path taken by the students (see Figure 4).

Figure 3a Multi-modal learning content in the chatbot

Figure 3b Another example of multi-modal learning content in o the chatbot

Figure 4 Interactive exercises inside the chatbot that will appear at the end of the lesson

The development of the chatbot was guided through three educational theories, namely constructivism, problem-based learning and active learning.

Implementation

The chatbot was deployed in an online Java programming course over one semester.  The students (n=32), who were open and distance learners, participated in this course, which was delivered in a blended mode consisting of face-to-face tutorials, self-managed learning, and online learning components. In the study, the chatbot was used to support their online learning and was made available to them in the second week of the semester.

Evaluation

The evaluation of the chatbot’s effectiveness was conducted at the  of the semester  through a structured survey using a 4-point Likert scale, which captured students’ perceptions across multiple dimensions as indicated in Figure 5. A significant 70% of the students enrolled in the Java programming course participated, providing valuable insights into the chatbot’s performance.

Figure 5 Perceptions about the chatbot by the students

The survey results revealed overwhelmingly positive feedback, with students expressing satisfaction with the chatbot’s ability to simplify their understanding of the subject matter. The findings align with those of [12], who observed that students generally favor chatbots in educational contexts, viewing them as essential tools for communication and information sharing. The chatbot’s unique features, including multimodal content delivery, interactive exercises, and context-sensitive hints, not only tackled learning challenges effectively but also alleviated cognitive load by presenting information in clear and easily digestible formats. These features allowed students to engage with the material more effectively, fostering a deeper understanding and improving their overall learning experience. For example, the use of visualizations and step-by-step guidance enabled students to tackle complex problems with confidence, bridging the gap between abstract concepts and practical application.

While the relatively small sample size limits the generalizability of the findings, the insights gained indicate that the chatbot has been effective within its current scope. Moreover, these findings underscore the chatbot’s potential to transform programming education by providing an interactive, engaging, and learner-centered learning experience. This approach not only supports students in overcoming learning challenges but also opens new avenues for innovation in educational technology.  Its ability to address common learning barriers and provide real-time support underscores its value as a critical educational tool.

DISCUSSION AND CONCLUSION

Chatbots have emerged as a transformative tool in programming education, addressing many of the challenges associated with teaching technical subjects. Programming requires a deep understanding of syntax, semantics, and algorithmic thinking, often necessitating iterative learning and immediate feedback. Chatbots, with their interactive and adaptive capabilities, provide a personalized and engaging solution to these demands.

By offering real-time assistance and tailored feedback, chatbots act as virtual tutors, enabling learners to identify and correct coding errors effectively. They also support the visualization of abstract programming concepts, such as object-oriented principles and data structures, through multimodal content like animations and interactive diagrams. This approach enhances comprehension and reduces cognitive load, particularly for novice learners.

The scaffolded problem-solving enabled by chatbot aligns with constructivist and problem-based learning theories, guiding students through incremental challenges and promoting active engagement. Additionally, chatbots facilitate independent learning by being available anytime, empowering students to learn at their own pace without the need for immediate instructor intervention. Gamified elements further motivate learners by transforming traditional exercises into dynamic, goal-oriented activities.

Despite these advantages, challenges persist. Developing chatbots capable of handling the nuanced and context-sensitive nature of programming tasks requires sophisticated natural language processing models and extensive training data. Ensuring the accuracy and reliability of chatbot feedback is critical to avoid reinforcing misconceptions. Moreover, scalability and integration with existing educational technologies remain areas for further development.

The study conducted demonstrated the efficacy of chatbots in a Java programming course, with students reporting significant improvements in understanding complex concepts of programming. The positive reception highlights the potential of chatbots to bridge the gap between theoretical knowledge and practical application, making learning more accessible and enjoyable.

In conclusion, educational chatbots represent an innovative approach to enhancing programming education. Their ability to provide interactivity, immediate feedback, and personalized learning experiences positions them as a central tool for modern education. As chatbot technology evolves, addressing current limitations, it holds the promise of revolutionizing technical education by offering scalable, cost-effective, and engaging learning solutions for diverse learners. This underscores the importance of continued research and development to fully realize their potential in programming and other technical subjects.

Future Work

As part of future work, the author is currently working to enhance the accuracy of chatbot responses formulated through the retrieval-based method. The author also plans to integrate generative AI, specifically GPT-4, through the Application Programming Interface (API) into the chatbot to enhance its ability to generate more nuanced and contextually relevant responses.

REFERENCES

1. Frankenfield, J. (2018). Chatbot. Investopedia. Retrieved from https://www.investopedia.com/terms/c/chatbot.asp
2. Singh, R. (2018, May 2). AI and chatbots in education: What does the future hold?. Chatbots Magazine. Retrieved from https://chatbotsmagazine.com/ai-and-chatbots-in-education-what-does-the-futurehold-9772f5c13960
3. Clarizia, F., Colace, F., Lombardi, M., Pascale, F., & Santaniello, D. (2018). Chatbot: An education support system for students. In International Symposium on Cyberspace Safety and Security (pp. 291–302). Springer. https://doi.org/10.1007/978-3-319-94141-9_28
4. Ranoliya, B. R., Raghuwanshi, N., & Singh, S. (2017). Chatbot for university-related FAQs. In Proceedings of the 2017 International Conference on Advances in Computing, Communications, and Informatics (ICACCI) (pp. 1525-1530). IEEE. https://doi.org/10.1109/ICACCI.2017.8126057
5. Sinha, S., Basak, S., Dey, Y., & Mondal, A. (2020). An educational chatbot for answering queries. In Emerging Technology in Modelling and Graphics (pp. 89-100). Springer. https://doi.org/10.1007/978-3-030-37919-4_10
6. Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033
7. Pham, X. L., Pham, T., Nguyen, Q. M., Nguyen, T. H., & Cao, T. T. H. (2018). Chatbot as an intelligent personal assistant for mobile language learning. In Proceedings of the 2018 2nd International Conference on Education and E-Learning (pp. 16-21). https://doi.org/10.1145/3291085.3291091
8. Zhao, J., Song, T., & Sun, Y. (2020). Apihelper: Helping junior android programmers learn API usage. IAENG International Journal of Computer Science, 47(1), 1-9.
9. Benotti, L., Martínez, M. C., & Schapachnik, F. (2017). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192. https://doi.org/10.1109/TLT.2017.2762718
10. Durall, E., & Kapros, E. (2020). Co-design for a competency self-assessment chatbot and survey in science education. In International Conference on Human-Computer Interaction (pp. 133-144). Springer. https://doi.org/10.1007/978-3-030-50454-1_11
11. Daud, S., Teo, N., & Zain, N. (2020). e-JAVA Chatbot for learning programming language: A post-pandemic alternative virtual tutor. International Journal of Emerging Trends in Engineering Research. https://doi.org/10.30534/ijeter/2020/67872020
12. Salloum, S. A., Almarzouqi, A., Aburayya, A., & Alfaisal, R. (2024). Adoption of chatbots for university students. In A. Al-Marzouqi, S. A. Salloum, M. Al-Saidat, A. Aburayya, & B. Gupta (Eds.), Artificial intelligence in education: The power and dangers of ChatGPT in the classroom (Studies in Big Data, Vol. 144, pp. 203-218). Springer. https://doi.org/10.1007/978-3-031-52280-2_15