Effectiveness of Basic to Advanced Computational Thinking Program for Secondary Students: Evaluation Study in Melaka, Malaysia

Effectiveness of Basic to Advanced Computational Thinking Program for Secondary Students: Evaluation Study in Melaka, Malaysia

Zuraini Othman^1*, Mashanum Othman¹, Khadijah Wan Mohd Ghazali¹, Masitah Seikh Omar²

¹Center for Advanced Computing Technology (C-ACT),Faculty Technology Maklumat dan Komunikasi (FTMK), University Technical Malaysia Melaka

²Sekolah Menengah i-Musleh, Melaka, Malaysia

^*Corresponding author

DOI: https://dx.doi.org/10.47772/IJRISS.2025.909000107

Received: 24 August 2025; Accepted: 30 August 2025; Published: 30 September 2025

ABSTRACT

Computational thinking has emerged as a critical 21st-century skill, yet many secondary students in Malaysia have not been formally exposed to structured computational thinking education. This study evaluates the effectiveness of a “Basic to Advanced Computational Thinking for Secondary Students” program implemented in Melaka, Malaysia. To assess the impact of a four-day computational thinking program on secondary students’ knowledge, skills, and confidence in computational thinking concepts. A pre-post evaluation design was employed using the validated form provided by Research, Innovation, Commercialisation, and Entrepreneurship (RICE) University Technical Malaysia Melaka (UTeM) assessment instrument. Forty-eight secondary students (Forms 1-3) participated in the program, which covered introduction to computational thinking which includes Microbit, Scratch, artificial intelligence (AI), and robotics. Participants completed self-assessment questionnaires before and after the program using a 5-point Likert scale. The program demonstrated significant effectiveness across all measured competencies. Overall mean scores improved from 2.45 to 3.71 (51.3% improvement). The highest improvements were observed in knowledge acquisition (60.9%) and program relevance perception (53.2%). Female students (n=9) and male students (n=20) both showed substantial improvements, with the program successfully engaging diverse learners across different form levels. The structured computational thinking program effectively enhanced secondary students’ competencies in computational thinking. The findings support the integration of comprehensive computational thinking curricula in Malaysian secondary education to prepare students for digital literacy demands.

Keywords- Computational thinking, secondary education, STEM education, digital literacy, program evaluation, Malaysia

INTRODUCTION

Computational thinking represents a fundamental skill set that enables individuals to solve complex problems systematically using computer science principles (Wing, 2006). As digital technologies increasingly permeate all aspects of society, the ability to think computationally has become essential for students’ future academic and career success (Grover & Pea, 2013). The concept encompasses four core components: decomposition, pattern recognition, abstraction, and algorithmic thinking (Brennan & Resnick, 2012).

In Malaysia, the integration of computational thinking into secondary education remains limited despite recognition of its importance in the Malaysian Education Blueprint 2013-2025 (Ministry of Education Malaysia, 2013). Many students enter higher education with insufficient exposure to structured computational thinking experiences, creating gaps in digital literacy and problem-solving capabilities (Ahmad et al., 2020). This challenge is particularly pronounced at the secondary level, where foundational computational thinking skills should be established.

Research has demonstrated the effectiveness of structured computational thinking programs in improving students’ problem-solving abilities and STEM engagement (Yadav et al., 2014; Angeli et al., 2016). However, limited studies have examined the effectiveness of comprehensive computational thinking programs specifically designed for Malaysian secondary students. This study addresses this gap by evaluating a multi-level program that progresses from basic concepts to advanced applications.

The “Basic to Advanced Computational Thinking for Secondary Students” program was designed to provide systematic exposure to computational thinking concepts through hands-on activities, coding experiences, and robotics applications. The program’s effectiveness was evaluated using a pre-post assessment design to measure changes in students’ perceived competencies across multiple dimensions.

Related Work

Computational thinking has gained recognition as a fundamental literacy alongside reading, writing, and arithmetic (Wing, 2006). Cuny et al. (2010) and Cansu et al. (2019) define computational thinking as the thought processes involved in formulating problems and their solutions so that solutions can be effectively carried out by information-processing agents. This definition emphasizes the transferable nature of computational thinking skills beyond computer science contexts.

Research has consistently demonstrated positive outcomes from computational thinking interventions. A systematic review by Tang et al. (2020) found that computational thinking education significantly improved students’ problem-solving skills, logical thinking, and creativity. Similarly, Shute et al. (2017) reported that students who participated in computational thinking programs showed enhanced performance in mathematics and science subjects.

The secondary education period is critical for developing computational thinking skills as students possess the cognitive maturity to engage with abstract concepts while remaining receptive to new learning approaches (Piaget, 1977). Kafai & Burke (2014) argued that early adolescence represents an optimal window for introducing structured computational thinking experiences that can influence future academic and career trajectories.

In the Malaysian context, studies have highlighted the need for enhanced computational thinking education at the secondary level. Rahman et al. (2021) found that Malaysian secondary students demonstrated below-average computational thinking skills compared to international benchmarks, emphasizing the urgency for targeted interventions.

Effective computational thinking programs typically incorporate multiple pedagogical approaches, including unplugged activities, visual programming languages, and robotics applications (Bell et al., 2009). The progression from basic concepts to advanced applications allows students to build confidence and competency systematically (Resnick et al., 2009).

METHODOLOGY

This section outlines the comprehensive methodological approach employed to evaluate the effectiveness of the computational thinking program. The methodology encompasses the research design framework, participant selection criteria, detailed program structure, assessment instruments used, and data collection and analysis procedures to ensure rigorous evaluation of program outcomes.

A. Research Design

This study employed a pre-post evaluation design to assess the effectiveness of the computational thinking program. The design allows for comparison of participants’ self-reported competencies before and after the intervention, providing evidence of program impact (Campbell & Stanley, 1963).

B. Participants

A total of 48 secondary students participated in the computational thinking program; however, the evaluation study focused on 30 students who completed both pre- and post-program assessments from schools in Melaka, Malaysia. Figure 1 illustrates the distribution of participants across different form levels and gender distribution. The participant demographics comprised students from three form levels: Form 1 with 22 students (11 male, 11 female), Form 2 with 11 students (5 male, 6 female), and Form 3 with 15 students (10 male, 5 female). The overall gender distribution consisted of 26 male students and 22 female students with all participants aged between 13-15 years. This demographic composition provided a representative sample across different secondary education levels, ensuring the program’s effectiveness could be evaluated across diverse grade levels and gender groups within the Malaysian secondary education context.

Figure 1. Distribution of survey respondents by form level and gender (N=30 survey respondents from 48 total program participants: Form 1=22, Form 2=11, Form 3=15).

This demographic diversity ensures that the program’s effectiveness can be evaluated across different educational levels and gender groups, providing robust evidence for its applicability in Malaysian secondary education contexts.

C. Program Description

The “Basic to Advanced Computational Thinking for Secondary Students” programme was conducted over four days, focussing on specific components for Form 1, Form 2, and Form 3:

Form 1: Introduction to Computational Thinking and Techniques of Problem-Solving using Scratch

Form 2: Microbit Coding and AI

Form 3: Introductions to robotics using the Lego EV3 robot.

Day 1 and Day 2 participants learnt theory and hands-on practical. Day 3 focuses on project development, with a related SDG chosen, and Day 4 is for presentations for all groups. All sessions were conducted in computer lab at Faculty of Information & Communication Technology (FTMK) providing access to appropriate technology resources and learning environments. The program implementation is illustrated in Figure 2, which shows participants actively engaged in various computational thinking activities alongside speakers and facilitators, demonstrating the hands-on and interactive nature of the learning experience.

Figure 2. Some pictures of the participants, speakers and facilitators during the program.

D. Assessment Instrument

The study utilized the validated RICE UTeM assessment instrument (Form C: Program Effectiveness Feedback for Industry & Community Programs). This instrument employs a 5-point Likert scale (1 = Strongly Disagree, 2 = Disagree, 3 = Uncertain, 4 = Agree, 5 = Strongly Agree) to measure participants’ self-reported competencies across multiple dimensions:

1. Knowledge Domain: Program relevance, subject knowledge, positive perception, knowledge acquisition
2. Skills Domain: Technical skills, skill application, skill transfer, skill adaptation
3. Application Domain: Independent application, economic benefits, community welfare enhancement.

Figure 3 displays the hard copy version of the RICE UTeM feedback form, though participants in this study completed the assessment online for more efficient data collection and analysis.

Figure 3. A hard copy of the RICE UTeM feedback form to be filled in by participants. However, for this program, feedback is filled in online.

E. Data Collection and Analysis

Participants completed the assessment questionnaire immediately before program commencement and upon program completion. Data analysis included:

1. Descriptive statistics for demographic characteristics
2. Pre-post mean comparisons for each competency dimension
3. Percentage improvement calculations
4. Overall program effectiveness assessment

Statistical analysis was conducted using appropriate measures to ensure data integrity and meaningful interpretation of results. Of the 48 students who participated in the program, 30 students (62.5% response rate) completed both pre- and post-program assessments and were included in the final analysis. The remaining 18 students either did not complete both assessments or were absent during data collection periods.

RESULTS

This section presents the comprehensive findings from the pre-post evaluation of the computational thinking program. The results are organized into demographic analysis of participants, detailed competency improvements across all measured dimensions, and key findings that demonstrate the program’s effectiveness in enhancing students’ computational thinking capabilities.

A. Demographic Profile

While the program successfully engaged 48 secondary students across diverse grade levels, this evaluation focuses on the 30 students who completed both pre- and post-assessment surveys, representing a 62.5% response rate with balanced gender participation. The majority of participants (96.7%) were within the target age range of 13-15 years, confirming appropriate participant selection for the program objectives.

B. Competency Improvements

The program demonstrated significant effectiveness across all measured competencies. Figure 4 provides a visual comparison of pre- and post-program competency scores, while Figure 5 highlights the percentage improvement in each area. Table 1 presents the detailed numerical results, and Figure 6 illustrates the consistent progression pattern across all competencies.

Figure 4. Comparison of mean competency scores before and after the computational thinking program. All competency areas showed substantial improvement (n=30).

Figure 5. Percentage improvement in each competency area following program completion. Subject Knowledge showed the highest improvement (60.9%), while Skill Transfer showed the lowest (41.2%).

TABLE I. PRE-POST COMPETENCY ASSESSMENT RESULTS

Figure 6. Line graph illustrating the consistent upward progression across all competency measures. The parallel upward trend demonstrates the program’s comprehensive effectiveness across different skill domains.

C. Key Findings

Knowledge Domain Improvements 

The most substantial improvement was observed in subject knowledge acquisition (60.9% improvement), indicating that participants significantly enhanced their understanding of computational thinking concepts. Program relevance perception also showed remarkable improvement (53.2%), suggesting that students recognized the value and applicability of computational thinking skills.

Skills Domain Development 

Technical skills development showed significant improvement (50.7%), demonstrating the effectiveness of hands-on programming activities with Microbit and Scratch. Skill adaptation capabilities improved by 55.6%, indicating that students developed flexibility in applying computational thinking approaches to varied contexts.

Application Domain Enhancement 

The application domain showed consistent improvements across all measures, with community welfare enhancement (57.6%) and economic benefits perception (57.4%) showing the highest gains. These results suggest that students developed an understanding of computational thinking’s broader societal implications.

Competency Domain Analysis 

Figure 7 provides a comprehensive analysis of the three main competency domains (Knowledge, Skills, and Application). The radar chart visualization clearly demonstrates that all three domains experienced substantial and balanced improvements, with the Knowledge domain showing the largest absolute gains from 2.59 to 3.91 (51.0% improvement), followed by the Application domain from 2.28 to 3.58 (57.0% improvement), and the Skills domain from 2.44 to 3.60 (47.5% improvement).

Figure 7. Radar chart comparing pre- and post-program competency levels across three main domains: Knowledge, Skills, and Application. All domains showed substantial improvement, with Knowledge domain showing the largest absolute gain.

Overall Program Effectiveness

The overall mean improvement of 51.3% (from 2.45 to 3.71) represents a substantial enhancement in participants’ self-reported computational thinking competencies. This improvement magnitude suggests that the program successfully achieved its educational objectives across all measured dimensions.

DISCUSSION

This section provides a comprehensive analysis and interpretation of the research findings, examining the program’s effectiveness within theoretical and practical contexts. The discussion explores pedagogical implications, cultural considerations specific to the Malaysian educational environment, and acknowledges study limitations while suggesting directions for future research.

Program Effectiveness

The results demonstrate that the program was highly effective in enhancing participants’ computational thinking competencies. The 51.3% overall improvement aligns with findings from similar international studies (Chen et al., 2017; Korkmaz et al., 2017), confirming the transferability of computational thinking education approaches across different cultural contexts.

The particularly strong improvement in subject knowledge (60.9%) suggests that the program’s structured progression from basic concepts to advanced applications was pedagogically sound. As illustrated in Figure 5, this represents the highest percentage improvement among all competency areas, followed closely by community welfare enhancement (57.6%) and economic benefits perception (57.4%). This finding supports the theoretical framework proposed by Brennan & Resnick (2012) regarding the importance of systematic skill development in computational thinking education.

The consistent improvement pattern shown in Figure 6 demonstrates that the program’s multi-faceted approach was effective across all competency dimensions, with no areas showing decline or stagnation. This comprehensive effectiveness indicates that the four-day program structure successfully integrated theoretical understanding with practical application.

Pedagogical Implications

The program’s effectiveness across diverse competency dimensions indicates that computational thinking education benefits from multi-modal approaches. The integration of unplugged activities, visual programming, hardware programming, and project-based learning appears to have created comprehensive learning experiences that engaged different learning preferences and cognitive styles.

The substantial improvement in skill adaptation (55.6%) and independent application (56.2%) suggests that participants developed transferable skills rather than context-specific knowledge. This finding is particularly significant for educational policy, as it supports the inclusion of computational thinking in core curricula rather than as isolated computer science courses (Yadav et al., 2014).

The balanced improvements across the three competency domains, as visualized in Figure 7, demonstrate that the program successfully addressed both cognitive and affective learning outcomes. The Knowledge domain improvements reflect enhanced understanding of computational concepts, while the Skills and Application domain improvements indicate practical competency development and real-world connection-making abilities.

Cultural and Contextual Considerations

The program’s success in the Melaka, Malaysian secondary education context demonstrates that computational thinking education can be effectively implemented across diverse cultural settings. As shown in Figure 1, the demographic distribution shows balanced participation, and the results demonstrate balanced improvements across male and female participants, suggesting that the program design successfully addressed potential gender disparities in STEM engagement (Master et al., 2017). The demographic analysis reveals that while male students comprised 66.7% of participants, both genders showed substantial improvements across all competency areas.

The strong improvement in community welfare perception (57.6%) indicates that Malaysian students readily connected computational thinking skills to broader societal benefits, reflecting cultural values that emphasize collective well-being and community contribution. This finding suggests that computational thinking programs in Malaysia should emphasize social applications and community impact to maximize student engagement and relevance.

Limitations

Several limitations should be acknowledged in interpreting these results. The study employed a single-group pre-post design without a control group, limiting causal inferences about program effectiveness. The response rate of 62.5% (30 out of 48 program participants) for complete pre-post assessments, while acceptable for educational research, may introduce potential non-response bias that should be considered when interpreting results. The reliance on self-reported competency measures may introduce response bias, particularly in post-program assessments where participants may feel obligated to report improvements.

The relatively small sample size and regional focus (Melaka) limit the generalizability of findings to broader Malaysian secondary student populations. While Figure 1 shows diverse participation across form levels, the demographic distribution may not be representative of all Malaysian secondary schools. Additionally, the immediate post-program assessment does not capture longer-term retention or application of computational thinking skills.

CONCLUSION

This study provides evidence that structured computational thinking programs can significantly enhance secondary students’ competencies in computational thinking concepts and applications. The “Basic to Advanced Computational Thinking for Secondary Students” program demonstrated effectiveness across knowledge, skills, and application domains, with an overall improvement of 51.3%.

As illustrated throughout Figures 1-7, the program achieved comprehensive improvements across all measured competency areas, with particularly notable gains in subject knowledge acquisition (60.9%), community welfare enhancement (57.6%), and economic benefits perception (57.4%). The consistent improvement patterns observed across different form levels and genders demonstrate the program’s broad applicability and effectiveness.

The findings support the integration of comprehensive computational thinking curricula in Malaysian secondary education as a means of enhancing students’ digital literacy and problem-solving capabilities. The program’s success across diverse participant demographics suggests broad applicability within the Malaysian educational context, with potential for scaling to reach larger student populations.

Future studies will address the limitations in this study through controlled experimental designs, objective measures of efficacy, larger and more diverse samples, and longitudinal follow-up assessments to determine the sustainability of the observed improvements.

ACKNOWLEDGEMENT

The author would like to thank the Centre of Research and Innovation Management of University Technical Malaysia Melaka (UTeM) for sponsoring the publication fees under the Tabung Penerbitan CRIM UTeM.

REFERENCES

1. Ahmad, S., Prasetyo, Z. K., & Nusantara, T. (2020). Computational thinking integration in mathematics learning for secondary school students in Malaysia. International Journal of Instruction, 13(4), 739-752.
2. Angeli, C., Voogt, J., Fluck, A., Webb, M., Cox, M., Malyn-Smith, J., & Zagami, J. (2016). A K-6 computational thinking curriculum framework: Implications for teacher knowledge. Educational Technology & Society, 19(3), 47-57.
3. Bell, T., Alexander, J., Freeman, I., & Grimley, M. (2009). Computer science unplugged: School students doing real computing without computers. New Zealand Journal of Applied Computing and Information Technology, 13(1), 20-29.
4. Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American Educational Research Association (pp. 1-25). American Educational Research Association.
5. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Rand McNally.
6. Cansu, F. K., & Cansu, S. K. (2019). An Overview of Computational Thinking. International Journal of Computer Science Education in Schools, 3(1), 17–30. https://doi.org/10.21585/ijcses.v3i1.53.
7. Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162-175.
8. Cuny, J., Snyder, L., & Wing, J. M. (2010). Demystifying computational thinking for non-computer scientists. Unpublished manuscript. Carnegie Mellon University.
9. Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38-43.
10. Kafai, Y. B., & Burke, Q. (2014). Connected code: Why children need to learn programming. MIT Press.
11. Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558-569.
12. Master, A., Cheryan, S., Moscatelli, A., & Meltzoff, A. N. (2017). Programming experience promotes higher STEM motivation among first-grade girls. Journal of Experimental Child Psychology, 160, 92-106.
13. Ministry of Education Malaysia. (2013). Malaysia Education Blueprint 2013-2025: Preschool to post-secondary education. Ministry of Education Malaysia.
14. Piaget, J. (1977). The development of thought: Equilibration of cognitive structures. Viking Press.
15. Rahman, M. M., Ghazali, M., & Ismail, Z. (2021). Computational thinking skills among Malaysian secondary school students: A comparative study. Education and Information Technologies, 26(2), 2303-2318.
16. Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., … & Kafai, Y. (2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60-67.
17. Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142-158.
18. Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798.
19. Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33-35.
20. Yadav, A., Hong, H., & Stephenson, C. (2014). Computational thinking for all: Pedagogical approaches to embedding 21st century problem solving in K-12 classrooms. TechTrends, 60(6), 565-568.