Nominal Group Technique for Identifying Challenges in Data Measurement Within Fundamental of Computer Science Curriculum

Nominal Group Technique for Identifying Challenges in Data Measurement Within Fundamental of Computer Science Curriculum

Selamat. S.S.M, Adnan. N.H

National University of Malaysia

 DOI: https://dx.doi.org/10.47772/IJRISS.2024.8080200

Received: 01 August 2024; Accepted: 09 August 2024; Published: 12 September 2024

ABSTRACT

The fast progress of Information Technology (IT) requires modifications to educational methods, particularly in secondary education where Science, Technology, English and Mathematics (STEM) subjects are taught. The implementation of STEM approaches that requires hands-on and critical thinking leads to the necessity of improving innovative technology especially in the Fundamentals of Computer Science course. However, few studies have focused the issues arising that may lead to the inability to convey data measurement concepts effectively within the curriculum and classroom setting. To discover the issues and challenges, a Nominal Group Technique was utilised to uncover the key issues related to data measurements. Five experts that have educational backgrounds related to the subject was chosen to gather information and views about the arising issue and findings discovered 11 key issues. After conducting online sessions with five experts in Computer Science Education, we identified many significant obstacles. These concerns include struggles with applying formulas for file sizes, difficulty with unit conversions, and the necessity for more effective visual aids. The findings also indicated lack of interactive learning materials that aligns with the current Fundamental of Computer Science curriculum, therefore limiting the resources at hand needed to convey information effectively and in an engaging way. This study highlights the need for developing a better learning material and solution to make complex concepts easier to understand. With the findings at hand, future research can investigate and develop more effective and interactive applications that are in line with the curriculum and new digital era.

Keywords: Nominal Group Technique, Data Measurement, Fundamental of Computer Science, STEM education.

INTRODUCTION

The rapid evolution of information technology has influenced the need for updating educational practices and curriculum design. Its influence on the Internet and several technological innovations has caused changes in sectors like social, economic, and educational contexts [1]. It has shifted the educational approach to more interactive technology-integrated teaching methods, with an emphasis on critical thinking and problem-solving skills among students [2]. Revisioning the curriculum using technology related theories was therefore found to be of vital importance in enhancing students’ digital competence and practices [3]. [4] states that the digital age and the rapid growth in information technology has called for an overhaul of educational content so that it fits the new age.

Technology-based and more interactive methods of teaching and learning are becoming increasingly necessary in this advanced era of secondary education that is concerned with introductory courses such as Fundamentals of Computer Science that form the basis of the advanced IT concept study for the students. One of the subjects in the Science, Technology, Engineering, and Mathematics (STEM) subjects is Fundamentals of Computer Science which is a very significant tool in making the students assessed as those who have the skills and knowledge necessary to survive in the technology-oriented world. Computer Science is a crucial subject in the senior secondary school curriculum aimed at equipping students with computer knowledge and its applications in today’s ICT-driven economy [5]. The need to introduce and reinforce computer science basics in every grade of elementary, middle school, and high school is underlined to ensure learners are ready for the advanced syllabus [6].

Despite its significance, the effectiveness of the Fundamentals of Computer Science teaching and learning approaches, specifically in data measurement concepts, remains underexplored. Previous research related to the Fundamentals of Computer Science has only concentrated on topics such as algorithms and data representation. [7] pursued the study to teach algorithms through the means of multimedia technology, thus boosting the engagement of learners and increasing their understanding of information technology and multimedia in the computer science course. [8] carried out a study that addressed the student’s problem with data representation by using multimedia as the main part of the taught material. This way, students might learn the material more engagingly and effectively via active participation. In this regard, both researchers ([7],[9]) highlighted the need for visualizing and using multimedia as the main tactics to guarantee that students are engaged and understand computer science.

Even though a variety of studies have been conducted on fundamental computer science themes such as algorithms and data representation, the spotlight has not been thrown on data measurement concepts as such. This represents a knowledge gap, hence a need for further research and development of targeted educational tools and approaches to address the challenges and enhance the learning experience related to data measurement concepts.

Therefore, this study aims to fill in the gap in the data measurement curriculum of Fundamental of Computer Science through the Nominal Group Technique (NGT) that entails expert opinions on the barriers and learning issues linked to data measurement topics within the Fundamental of Computer Science curriculum. Drawing on the findings of educational experts, this undertaking will aim at pinpointing and tackling the problems faced by people that affect the pupils’ grasp and involvement in concepts related to data measurement in computer science.

LITERATURE REVIEW

A. Importance of STEM Education

The creation of a robust Science, Technology, Engineering, and Mathematics (STEM) education system in Malaysia is crucial for preparing a skilled workforce capable of excelling in future industries, as emphasized in various research studies. The Malaysian Ministry of Education has acknowledged the importance of enhancing the quality and innovation in STEM education to align with global advancements in science and technology [10]. The guidelines for STEM education in Malaysia outline seven criteria used in designing learning modules to foster a comprehensive and engaging learning experience for students. These criteria include student involvement in inquiry and exploration, encouraging hands-on and investigative activities; student involvement in productive teamwork, promoting collaborative learning and teamwork skills; understanding and application of STEM content, ensuring that students can comprehend and apply STEM concepts effectively; opportunities for improvement, allowing students to refine their answers or products through iterative processes; application of design process skills, involving students in practical design and problem-solving; variety in solutions and justifications, encouraging diverse solutions and requiring students to justify their approaches; and sensitivity to real-world issues, enhancing students’ awareness and sensitivity to real-world problems and issues [11],[12]).

The Fundamentals of Computer Science is one of the subjects within the STEM framework that addresses key topics such as computational thinking, data representation, algorithms, and coding. This subject aligns with the broader objectives of STEM education by developing students’ technical skills and problem-solving abilities, which are essential for future technological advancements.

By prioritizing cognitive development in STEM education and career pathways, Malaysia aims to ensure continuous support for the nation’s economic development with a competent workforce [13]. To strengthen Malaysia’s position in the Global Innovation Index and advance the objectives of the National Science, Technology, and Innovation Policy (NSTIP) 2021-2030, educational administrators are encouraged to promote teachers’ engagement in innovative work behavior activities [14]. This focus on innovative work behavior activities aims to leverage teachers’ competencies and strengths to cultivate a technologically proficient society in line with the country’s aspirations for technological advancement.

The Malaysia Education Blueprint 2013-2025 emphasizes the need for STEM education to produce a workforce proficient in technology and information fields. However, student engagement in STEM subjects remains a significant issue due to ineffective and unattractive learning tools. The complexity of these subjects requires innovative educational tools that can make learning more interactive and accessible. There is also a pressing need to ensure that the content of educational modules is valid and aligns with national educational goals, which can be achieved through expert evaluations and the incorporation of visualization and multimedia elements [15].

B. Fundamentals of Computer Science

Fundamentals of Computer Science have been incorporated into STEM education in schools since 2017. The purpose of the Fundamental of Computer Science subject is to equip students with knowledge and computational thinking skills essential for understanding basic Computer Science concepts, including coding and algorithms. STEM education encourages students to be creative, innovative, dynamic, and ethical in their approach to Information and Communication Technology (ICT) [16].The objectives of the subject are to compile, analyse, and present data or ideas logically and systematically, identify, detect, and correct errors in algorithms and programs using logical and computational thinking, solve complex problems through computational thinking with computer-based solutions and apply computer knowledge and skills in an ethical, practical, and responsible manner. The Fundamentals of Computer Science curriculum for Form 1 students covers four essential topics: basic concepts of computational thinking, data representation, algorithms, and coding instructions. The data representation and data measurement section include instructions on unit conversions and the various formats used for audio and images. It comprises methods for determining the sizes of audio and image files and identifying different types of bit depth and sample sizes. Teachers are provided with a standard curriculum document and textbook as teaching aids to guide their instruction. To gain better understanding of Computer Science and programming concepts, teachers and students’ involvement play a crucial role. Apart from the teacher and students’ involvement, the choice of learning materials are crucial.[17]

However, research on the effectiveness of the materials provided to teach this subject in Malaysia remains limited. Therefore, this study focuses on evaluating the issues related to the fundamentals of computer science subject. The current learning approach is predominantly instructor-centered, emphasizing direct instruction over student engagement. With rapid advancements in educational technology, instructor-centered methods are becoming less relevant. The integration of Information and Communication Technology (ICT) in teaching and learning processes is becoming increasingly crucial [18]. Instead, student-centered classrooms and interactive teaching methods are increasingly preferred as it not only activates students’ activities but also direct them toward communication and discussion, promoting engagement among students and between students and teachers [19]. To align with these trends, the teaching approach for teaching Fundamentals of Computer Science must be updated. Given that Fundamental of Computer Science is a STEM subject, integrating STEM guidelines into the learning strategy is essential. This research will focus on identifying issues related to the topic of data measurement in data representation as a guideline to create solutions that can overcome the identified challenges.

C. Teaching Aids

The landscape of education has undergone significant changes, particularly with the integration of technology into the teaching process. Traditional teaching methods have often struggled to keep students engaged, especially in subjects requiring the understanding of complex and abstract concepts. Research indicates that conventional lecture-based teaching methods can sometimes fail to captivate learners and provide an enjoyable language acquisition experience [20]. For instance, the use of conventional methods to teach topics like algorithms and the evolution of the Internet has proven to be insufficient in maintaining student interest and ensuring effective learning. Studies have highlighted the importance of utilizing multimedia elements such as animations, graphics, audio, video, and interactive tools to diversify teaching methods and enhance student engagement ([7],[9]).

Engagement is a critical factor in education, and the lack of it can lead to lower retention rates and reduce interest in the subject matter. The complexity of certain subjects further worsens this issue, making it challenging for students to grasp intricate concepts through traditional approaches alone. However, there has been a growing recognition of the need for innovative solutions, such as augmented reality applications, which can provide interactive and immersive learning experiences. These tools not only help in visualizing complex topics but also make abstract concepts more tangible and understandable [9].

In Malaysia educational setting, the challenges that arises related to the need of bridging the digital divide and ensuring equitable access to quality education for all students, those who are in the rural area. The traditional methods of teaching computer science topics, such as data measurement, often fail to engage students and make the learning process a boring one and less effective. Previous studies have recognized the importance of incorporating technology into teaching to enhance learning outcomes such as the development of the mobile application to teach algorithms has shown significant success in improving student engagement and learning experiences by integrating multimedia elements and offering flexible learning environments [7] Given these challenges, there is a pressing need to investigate issues related to the fundamentals of computer science subjects, particularly focusing on data measurement. By addressing these specific challenges, a more effective and engaging solution can be developed to improve teaching methods and learning outcomes in this area.

Research Aim

This study aims to explore the main issues related to the difficulties in understanding the data measurement topic in a computer science course. Therefore, this study focuses on two main objectives:

1. Identify the issues related to the inability to understand data measurement concepts among students based on experts’ opinions.
2. Validate the item proposed related to the issues based on the consensus and opinion of experts.

METHODOLOGY

The Nominal Group Technique (NGT) technique is the main research tool in this study. Five professionals in the field of Computer Science Education participated in the study. Researchers conducted the NGT sessions online using Google Meet as it was not practical to carry out the session in person due to geographical issues. A group discussion was conducted with a duration of one and a half hours. To gain the views and issues of data measurement topics within the Fundamentals of Computer Science curriculum based on expert opinion, the researcher assembled experts and used the brainstorming session based on the NGT technique. The researcher employed the NGT approach to execute a targeted computation at the session’s conclusion, resulting in data that addressed the study’s objectives.

A.    Sampling

The study involved 5 experts who specialize in computer science education. All the experts are chosen to have at least 5 years of experience related to computer science education and have experience in teaching the data measurement topic within the Fundamental of the Computer Science curriculum

Table 1: Experts were chosen from the computer education field

The number of experts specified is adequate to guarantee that the sample utilized is specialized and does not need a substantial quantity. The Nominal Group Technique (NGT) can be applied with either a small cohort or a big group ([21], (29]). For example, a study by [22] applied 45 nurse educators through a virtual workshop while another study by [23] only involved four study groups that consisted of seven participants to gather public patient views on artificial intelligence in healthcare. Alternatively, it can be subdivided into smaller clusters to enhance efficient communication based on the requirements of the research. [24] suggests that the sample size should range from 7 to 10 individuals, however, [25] suggest a range of 5 to 9 is enough for implementing the NGT process.

B. Nominal Group Technique Steps

NGT, or Nominal Group Technique, is a systematic approach used to establish a consensus among a group of individuals over a certain subject. The notion was initially developed as a method for engaging in social planning scenarios, as defined by [26]. These situations encompass exploratory research, public engagement, the involvement of multidisciplinary experts, and the evaluation of suggestions. Since then, it has been employed in other group settings, including empirical studies in the field of social science. ([27], [28], [29], [ 30]) are among the researchers who have employed it in the field of education. However, this approach is commonly used in medical sectors.

The Nominal Group Technique (NGT) is a carefully organized procedure that unfolds in four discrete stages. In the first stage, participants have to generate spontaneous thoughts in response to a question asked and these thoughts are shared and listed in a round-robin fashion, without engaging in any discussion. This process allows participants to engage equally and provide their opinions, ensuring a fair and inclusive process [31]. In the third stage, each concept is explained, and similar ideas are categorized. Finally, participants cast their votes to establish the order of importance for the ideas generated. To promote genuine outcomes and active participation, participants cast their votes anonymously to prioritize ideas [32]. This method allows for fair and unbiased decision-making by ensuring that each participant’s input is considered equally without the influence of personal biases or external pressures [32]. NGT ensures lasting documentation of the group’s procedure and results by transcribing all recommendations and authorized modifications onto flipchart sheets. Presenting these documents is a commendable approach to educating individuals who were absent for all or a portion of a gathering, and it also enables groups to resume their discussions from the point they ended in the preceding meeting ([29], [33]). For this research, the researcher followed the NGT steps and employed NGT-Plus software to list down issues regarding data measurement topics according to experts’ opinions and compute the votes.

FINDINGS

Using the nominal group technique, a discussion was held via Google Meet with five experts to brainstorm the issues regarding data measurement topics in the fundamentals of computer science subject. A set of questions were given to the expert for them to read and were asked to provide their opinion. During the discussion, each expert took turns providing their opinions regarding the issues in understanding the data measurement topic. Each idea was listed on the NGT-Plus Software. After that, each issue listed was discussed thoroughly and categorized. The results of the discussion listed 11 crucial issues.

Fig 1: Issues In Understanding Data Measurement Topic

Experts stated that many of these students have problems using formulas to calculate the file size of audio and images. Sometimes, concepts are just not clearly understood, and instruction with usually existing resources cannot bridge the gap between theory and application. They also agreed that students faced challenges in doing unit conversions accurately. These challenges often arise from lack of understanding of the underlying principles and inadequate practice opportunity. In this respect, effective instructional strategies are called for in trying to solve these challenges to improve students’ proficiency in unit conversions. Another concern raised was the need for visual aids in understanding the complex concepts of measurement data. According to experts, the problem usually faced by students is relating such abstract terminologies with their practical use and application. They said that the visual aids would provide concrete examples and illustrations, thereby making these concepts more inclusive and easier to relate to. A few other factors that had added to this impact on the students’ ability to engage and learn the complex concepts were the lack of interactive resources, including apps, software, educational games, etc. Moreover, the problem worsens due to the high cost of development and learning to design innovative technology, in addition to the misalignment with the national curriculum, which provides impetus for seeking more affordable interactive tools that align with the national curriculum. In the final consideration, a balanced approach should be maintained between the complexity and simplicity of learning materials. Those with extreme complexity can lead to withdrawal, and those that are extremely simple are sure to cause procrastination in student interest and motivation. Effective instructional design should strive to create materials that are appropriately challenging and engaging.

To validate the findings and reach a consensus on the next steps, the NGT approach using NGT-Plus software was used. The results are as follows.

Table 2: NGT-Plus Analysis

The findings depict the aggregate agreement and assessment ratings for the model. This analysis has determined that all concentrations of model constructs are within the optimal range. The results of these investigations indicate that the proportion should currently be above 70%. All the items are above 70% expert consensus, according to the study of expert approval data. Findings from studies such as ([29],[34], [35]) provide support for this idea. Researchers can thus conclude that the model’s essential components are practical and well-received by the intended users. Instead of going through the tedious rounds of expert judgment that the Delphi method requires, time can be saved by using the modified NGT strategy.

Based on the analysis using the NGT approach, highlighted several critical issues in teaching data measurement concepts in computer science. Experts identified key areas of concern, including students’ difficulties with applying formulas for calculating audio and image file sizes, challenges with unit conversions, and the need for effective visual aids. Additionally, the lack of interactive educational materials and the misalignment of some resources with the national curriculum were significant issues.

DISCUSSION AND FUTURE DIRECTIONS

Based on expert consensus, the main issues regarding understanding the concepts of data measurements revolve around students’ difficulties with applying formulas for calculating audio and image file sizes, challenges with unit conversions, and the need for effective visual aids. Additionally, the lack of interactive educational materials and the misalignment of some resources with the national curriculum were significant issues.

Therefore, based on these findings, there is a need to develop interactive learning materials that can enhance understanding related to data measurement topics, hence proving the need to develop more learning materials that fit the current education setting in this new digital era. The application of interactive learning materials, such as augmented reality and interactive multimedia, are essential tools for enhancing students’ understanding of complex concepts. The incorporation of various interactive elements such as videos, animations, and simulations help students visualize abstract concepts effectively, thus leading to improved comprehension ([36], [37] [38][ 39]). Through the use and development of interactive learning materials, educational content will not only be delivered effectively, but also foster critical thinking skills, autonomy, and collaboration among students ([40],[41]). These development and use of interactive learning materials can cater to different learning styles, provide immediate feedback, and offer opportunities for personalized learning experiences [42]. Moreover, interactive learning materials have been found to enhance student motivation, improve retention of information, and increase overall learning effectiveness ([42],[43]).

Additionally, findings also show the need for visual representation to help improve learning outcomes. This finding is consistent with [44] that states visual aids, including pictures, models, charts, and videos, contribute to making learning more realistic, accurate, and dynamic, thereby aiding in information retention and enhancing the memorability of lessons. The implementation of visual representation also encourages the learning process especially in understanding complex topics. A study conducted by [45], discovered that the integration video increased the eased teaching and learning Hypertext Markup Languages in classroom setting. Besides that, the integration of animation in a programming course can increase students’ skill, understanding and learning outcome [46] Therefore, to suit the learning materials according to the latest technology, visual learning materials should be developed so that students can access them anytime, anywhere, hence the need to create a mobile learning application. Visual learners, in particular, benefit from online platforms offering a variety of visual aids, interactivity, and flexibility, leading to more effective learning experiences [47].

Another issue is related to the alignment of learning materials with the current curriculum of Fundamental of Computer Science, highlighting the need for improvements regarding learning materials that are more current and aligned with the curriculum. Therefore, future research on the development and usability of interactive learning materials should consider the curriculum and the involvement of experts’ opinions and consensus in the design and development phase. By involving end users, the usability of the product will be validated, and a more effective learning material can be developed.

Current solutions that address the issues for Fundamentals of the Computer Science curriculum in Malaysia through interactive learning materials are studies conducted by [7] and [15] that focus on the algorithm topic through the development of a mobile application. Another research by [8], also addresses the lack of interactive learning material related to data representation through the development of courseware. However, very few have focused on the issues and challenges related to data measurement topic. Hence, this research provides a clear understanding of the issues in relation to this topic. By understanding the current issues and challenges of understanding data measurement topic, effective ideas and solutions can be designed. Therefore, to address these challenges effectively, future work should concentrate on developing comprehensive and engaging interactive learning materials that incorporates visual, interactive and multimedia elements specifically designed to enhance understanding and proficiency in this topic, thereby providing learners with a more immersive and supportive educational experience that can cater to the needs of STEM education.

CONCLUSIONS

By using the nominal group technique, researchers were able to identify several key issues related to the data measurement topic. Among the issues are the difficulty of applying formulas correctly, changing unit conversions and the lack of interactive educational learning materials. The recent materials that are in the market or provided have found to be insufficient and lack interactive elements and not aligned with the current curriculum of Fundamental of Computer Science in Malaysia. Therefore, there is a crucial need for more creations, designs and development of interactive learning materials that can effectively help to assist students and educators, teaching and learning process. Through the design and development of interactive learning materials, students are engaged, motivated and understand better by providing a more accessible and relatable learning material for complex topics. This finding will help address the current issues and assist in developing better learning materials that are more relatable to the current technologies nowadays, align with the STEM approach, hence potentially, improve students understanding of the data measurement concept.

ACKNOWLEDGMENT

This research is part of research grant GG-2024-004. All the writers of this publication have collectively contributed to the development of this research, conducting interviews with participants, and writing the text. The final version of the work has been thoroughly examined and endorsed by all authors.

REFERENCES

1. Vidal, I. M. G. (2022). Architectures of contemporary digital platforms in education: analysis of exclusion processes. Universal Access in the Information Society, 22(4), 1445-1453. https://doi.org/10.1007/s10209-022-00887-7
2. Nuraeni, N. and Walahe, D. (2023). The evolution of science education. Journal of Training, Education, Science and Technology, 25-30. https://doi.org/10.51629/jtest.v1i1.172
3. Lyngdorf, N., Bertel, L., & Lindsay, E. (2022). A matrix for making sense of digital competences in formal engineering education curricula. https://doi.org/10.5821/conference-9788412322262.1262
4. Firdaus, F. and Abdulkarim, A. (2022). The life skills and careers of citizens in the digital age of pancasila and citizenship education curriculum content. Advances in Social Science, Education and Humanities Research. https://doi.org/10.2991/assehr.k.220108.019
5. Ngozi, E.J. (2023). Availability and utilization of the projecting multimedia for computer education in senior secondary schools in Nnewi Metropolis, Anambra State, Nigeria, West Africa. World J. Adv. Res. Rev. 2023, 18, 159–166, https://doi.org/10.30574/wjarr.2023.18.2.0966.
6. Hurlburt, G. (2023). The importance of educating computational educators. Computer, 56(3), 131-134. https://doi.org/10.1109/mc.2023.3235097
7. Shaharom, A. S., & Abdul Rahman, M. H. (2021). Keberkesanan Aplikasi Mudah Alih ‘Algoritma bersama Algo’ bagi Pembelajaran Topik Algoritma dalam Subjek Asas Sains Komputer. Journal of Engineering, Technology & Applied Science, 1-10.
8. Putera, T.N.T & Rosli, A.N. (2021). Pembangunan koswer interaktif Asas Sains Komputer sistem nombor perduaan menggunakan animasi jari. Journal of ICT in Education, 8(3), 60- 75. https://doi.org/10.37134/jictie.vol8.sp.1.6.202
9. Hashim, S., Abdul Rahman, K. A., Khamis, N., Shukor, U. H., Che Lah, N. H., & Zulkifli, N. N. (2023). The Design and Development of Augmented Reality (AR) Application for Internet Evolution Learning Topics. International Journal of Interactive Mobile Technologies (iJIM), 17(05). https://doi.org/10.3991/ijim.v17i05.36483
10. Idris, R., & Bacotang, J. (2023). Exploring STEM Education Trends in Malaysia: Building aTalent Pool for Industrial Revolution 4.0 and Society 5.0. International Journal of Academic Research In Progressive Education and Development, 12(2), 381–393. (6)
11. Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, (1), Article 11. https://doi.org/10.1186/s40594-016-0046-z
12. Ministry of Education Malaysia. (2016). Implementation Guide for Science, Technology, Engineering, and Mathematics (STEM) in Teaching and Learning. Retrieved from http://smksyedsira.edu.my/wp-content/uploads/2016/12/7.-Panduan-PelaksanaanSTEM-Dalam-PP.pdf
13. Idris, R., Govindasamy, P., Nachiappan, S., & Bacotang, J. (2023). Exploring the impact of cognitive factors on learning, motivation and career in malaysia’s stem education. International Journal of Academic Research in Business and Social Sciences, 13(6). https://doi.org/10.6007/ijarbss/v13-i6/17227
14. Daud, S. R., Rahim, A., Abdullah, M. S., Sehat, N. S., Sarkam, S. F., Abas, N., & Suhaime, I. L. (2024). A conceptual framework for the individual factors fostering the innovative work behavior of stem teachers. Information Management and Business Review, 16(1(I)), 252-261. https://doi.org/10.22610/imbr.v16i1(i).3709
15. Abdul Rahman, M., Zakaria, M. S., & Din, R. (2021). Design learning module fundamental of computer science: Evaluation of content validity. International Journal of Research in Education and Science (IJRES), 7(4), 1104-1116. https://doi.org/10.46328/ijres.2346
16. Curriculum Development Division, Ministry of Education Malaysia. (2016). Secondary School Standard Curriculum (KSSM) Fundamentals of Computer Science. Malaysia.
17. Mukhtar, S., & Adnan, N. (2022). Satu Kajian Kes: Analisis Tahap Kelemahan Dan Kekuatan Dalam Pengajaran Dan Pembelajaran Bagi Kursus Pengaturcaraan Web. Jurnal Dunia Pendidikan, 4(3), 216-233. Retrieved from https://myjms.mohe.gov.my/index.php/jdpd/article/view/19557
18. Jaiswal, P. (2020). Integrating educational technologies to augment learners’ academic achievements. International Journal of Emerging Technologies in Learning (iJET), 15(02), 145. https://doi.org/10.3991/ijet.v15i02.11809
19. Rybchynska, A. (2023). Development of communication skills through the introduction of interactive teaching methods in english lessons. Scientific Bulletin of Mukachevo State University Series Pedagogy and Psychology, 9(2). https://doi.org/10.52534/msu-pp2.2023.09
20. Ningrum, N. K., Harisa, A. B., & Umaroh, L. (2024). Learning through play: utilizing board games to enhance english vocabulary for early students. Techno Creative, 1(2), 117. https://doi.org/10.62411/tcv.v1i2.1784
21. Rees, S., Cooklin, A., Duncan, C., Matharu, M., Naghdi, S., Underwood, M., & Mistry, H. (2024). Research priorities for randomised controlled trials in chronic migraine preventive medication: a stakeholder consensus workshop. NIHR Open Research, 4, 16. https://doi.org/10.3310/nihropenres.13548.1
22. Vardaman, S. A., Logan, L., Davis, S. P., Sciarra, E., Doria, J. B., Baker, J., & Bajwa, M. (2024). Addressing the shortage of academic nurse educators: recommendations for educational institutions based on nominal group technique research. Nursing Education Perspectives, 45(4), 201-207. https://doi.org/10.1097/01.nep.0000000000001264
23. Musbahi, O., Syed, L., Feuvre, P. L., Cobb, J., & Jones, G. (2021). Public patient views of artificial intelligence in healthcare: a nominal group technique study. Digital Health, 7, 205520762110636. https://doi.org/10.1177/20552076211063682
24. Horton, J. N. (1980). Nominal group technique: A method of decision-making by committee. Anaesthesia, 35(8), 811-814. https://doi.org/10.1111/j.1365-2044.1980.tb03924.x
25. Delbecq, A. L., & Van de Ven, A. H. (1971). A group process model for problem identification and program planning. The Journal of Applied Behavioral Science, 7(4), 466-492. https://doi.org/10.1177/002188637100700404
26. Delbecq, A. L., Van De Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes. Glenview IL: Scott Foresman and Company.
27. Azzouzi, A. E., Kaddari, F., & Elachqar, A. (2023). Physics mathematization assessment: the nominal group technique as a context to investigate student understanding. International Journal of Evaluation and Research in Education (IJERE), 12(3), 1206. https://doi.org/10.11591/ijere.v12i3.25271
28. Shamsuddin, F. and Abdul Razak, A. Z. (2023). Development of a model for data-driven decision making: critical skills for school leaders. Malaysian Journal of Social Sciences and Humanities (MJSSH), 8(12), e002614. https://doi.org/10.47405/mjssh.v8i12.2614
29. Mustapha, R., Ibrahim, N., Mahmud, M., Jaafar, A. B., Ahmad, W. A. W., & Mohamad, N. H. (2022). Brainstorming the Students Mental Health after Covid-19 Outbreak and How to Curb from Islamic Perspectives: Nominal Group Technique Analysis Approach. International Journal of Academic Research in Business and Social Sciences, 12(2), 90–99. (6).
30. Ibrahim, R., Norman, H., Nordin, N., Zaini, H., Hamdan, F., Aziz, R., & Nabilah, F. (2023). MOOC Design Phase: Nominal Group Technique in Determining the MOOC Elements Using Relative Importance Index (RII). International Journal of Academic Research in Business and Social Sciences, 13(12), 3853–3869
31. Maguire, T., Garvey, L., Ryan, J., Olasoji, M., & Willetts, G. (2022). Using the nominal group technique to determine a nursing framework for a forensic mental health service: a discussion paper. International Journal of Mental Health Nursing, 31(4), 1030-1038. https://doi.org/10.1111/inm.13023
32. Young, R., Sage, K., Broom, D., Broomfield, K., Church, G., & Smith, C. (2021). Using nominal group technique to advance power-assisted exercise equipment for people with stroke. Research Involvement and Engagement, 7(1). https://doi.org/10.1186/s40900-021-00311-z
33. Fox, W. M. (1989). The improved nominal group technique (INGT). Journal of Management Development, 8(1), 20-27. https://doi.org/10.1108/eum0000000001331
34. Paulina, Bueno, Garcia, Reyes., K, Butcher., Nikki, Cotterill., Marcus, J., Drake., Amy, Gimson., L., Gogola., Emily, J., Henderson., Alyson, L, Huntley., Jonathan, L., Rees., E., Strong., Chih, M., Wong., Andrew, P., Skyrme-Jones., Shoba, Dawson. (2022). Implications of Cardiovascular Disease for Assessment and Treatment of Nocturia in Primary Care: Systematic Review and Nominal Group Technique Consensus. European urology focus, doi: 10.1016/j.euf.2021.12.014
35. Pham, Q., Hearn, J., Bender, J. L., Berlin, A., Brown, I., Bryant‐Lukosius, D., & Cafazzo, J. A. (2021). Virtual care for prostate cancer survivorship: protocol for an evaluation of a nurse-led algorithm-enhanced virtual clinic implemented at five cancer centres across canada. BMJ Open, 11(4), e045806. https://doi.org/10.1136/bmjopen-2020-045806
36. Choudhary, Y. (2023). Augmented reality in education. International Journal for Research in Applied Science and Engineering Technology, 11(11), 1830-1836. https://doi.org/10.22214/ijraset.2023.56965
37. Yanti, S. H., Yennita, Y., & Syahril, S. (2022). Improving the understanding of students’ learning concepts on optical equipment materials through interactive learning media. Journal of Educational Sciences, 6(3), 459. https://doi.org/10.31258/jes.6.3.p.459-471
38. Meiriska, A., Adnan, A., & Junda, M. (2024). Expert response to the development of interactive video as teaching media on cell material. JPBIO (Jurnal Pendidikan Biologi), 9(1), 85-97. https://doi.org/10.31932/jpbio.v9i1.3384
39. Kiflee, C. S., Hassan, S. A., Maaluot, N., Yusof, R., & Khalid, F. (2020). Analisis Keberkesanan Kaedah Multimedia Dalam Pengajaran dan Pembelajaran Terhadap Pelajar Pintar dan Berbakat. Jurnal Personalia Pelajar, 23(2).
40. Wea, D., Sri Pudjiarti, E., & Kristian Sarang, R. (2023). Assessing the quality of learning materials in a learning management system (lms): its impact on learning outcomes. Interciencia. https://doi.org/10.59671/gc23o
41. Rizqiani, D. A. and Yuliani, S. (2023). Developing critical literacy-based instructional reading materials for teaching efl reading classes. Pedagogy: Journal of English Language Teaching, 11(2), 124. https://doi.org/10.32332/joelt.v11i2.7758
42. Azizah, S., Widjanarko, M., Darmanto, E., & Pratama, H. (2022). Interactive learning media 2d educational game to improve learning effectiveness in kindergarten students. ICCCM Journal of Social Sciences and Humanities, 1(1), 23-38. https://doi.org/10.53797/icccmjssh.v1i1.4.2022
43. Budiarto, F. and Jazuli, A. (2021). Interactive learning multimedia improving learning motivation elementary school students. Proceedings of the 1st International Conference on Social Sciences, ICONESS 2021, 19 July 2021, Purwokerto, Central Java, Indon. https://doi.org/10.4108/eai.19-7-2021.2312497
44. Hamad, J. I. (2023). The impacts of visual aids in promoting the learning processes in schools in pakistan. African Journal of Education and Practice, 9(1), 51-60. https://doi.org/10.47604/ajep.1912
45. Nur Amanina, H., & Nor Hafizah, A. (2020). Implementasi video pengajaran dalam pembelajaran HTML melalui massive open online courses (MOOCs). Jurnal Dunia Pendidikan, 2(2), 205-212. https://doi.org/10.1234/jdp.2020.12345
46. Anuar, N. A., & Adnan, N. H. (2021). Kesan penggunaan video animasi dalam kursus pengaturcaraan Java bagi pelajar IPTA. Jurnal Dunia Pendidikan, 3(2), 84-97. http://myjms.mohe.gov.my/index.php/jdpd
47. Gogiashvili, S. and Demetrashvili, A. (2024). The impact of online learning on visual learners in the heis for sustainable development and well-being. Journal of Development Studies. https://doi.org/10.52340/jds.2022.03.03.02