Programme Outcome Measurement through Triangulated Evidence: Direct Attainments and Indirect Assessment in a Data Communication & Network Course

Programme Outcome Measurement through Triangulated Evidence: Direct Attainments and Indirect Assessment in a Data Communication & Network Course

Zatul Iffah Abd Latiff^*, Nor Affida M. Zin, Fadila Mohd Atan, Nur Asfahani Ismail, Zahari Abu Bakar

Faculty of Electrical Engineering, Universiti Teknologi MARA, Johor Branch, Pasir Gudang Campus, Masai 81750, Malaysia

^*Corresponding Author

DOI: https://dx.doi.org/10.47772/IJRISS.2025.909000783

Received: 29 September 2025; Accepted: 05 October 2025; Published: 30 October 2025

ABSTRACT

This study examines Programme Outcome (PO) measurement in the ETC574: Data Communication & Network course under the Bachelor of Technology (Digitalization and Automation) programme at UiTM Pasir Gudang, adopting a triangulated approach to provide robust evidence of outcome achievement. Direct assessment of Course Outcomes (CO) showed that all targeted outcomes exceeded the University Key Performance Indicator (KPI) of 65 %, with CO1 mapped to PO2 (Problem Analysis), attaining 70.15%, CO2 mapped to PO5 (Modern Tool Usage) attaining 77.5%, and CO3 mapped to PO10 (Communication), attaining 79.33%. Indirect assessment through Student Feedback Online (SuFO), based on 27 respondents from two classes, recorded an overall satisfaction score of 95.86%, with strong ratings for lecturer professionalism, teaching clarity, and student confidence. Entrance–Exit Survey (EES) results further confirmed significant learning gains, with entry self-assessments averaging 1.2 and exit ratings averaging 4.6 on a five-point scale, reflecting an average improvement of 3.4 points across ten skill domains including networking concepts, simulation tools, and communication skills. The triangulation of direct attainment, student feedback, and self-reported learning gain demonstrates that ETC574 course effectively supports the achievement of PO while highlighting areas for enhancement, such as reinforcing conceptual understanding at the physical/network layer and addressing infrastructure-related concerns. This study offers a course-level model of PO measurement that strengthens accreditation readiness and contributes to continuous improvement in engineering technology education.

Keywords: Programme Outcomes, Outcome-Based Education, Direct Assessment, Indirect Assessment, Student Feedback, Entrance–Exit Survey

INTRODUCTION

Outcome-Based Education (OBE) is the guiding framework for engineering and technology programmes worldwide, requiring institutions to demonstrate that their graduates achieve specific, measurable PO. In Malaysia, the Engineering Technology Accreditation Council (ETAC) mandates systematic measurement of PO to ensure alignment with the Sydney Accord and international standards (Liew et al., 2021). The measurement of PO is typically operationalized through CO, which represent specific subject-level achievements that are mapped to broader programme-level outcomes. The dual-level measurement of CO and PO provides a structured mechanism for monitoring student performance (Ahmed Ghaly, 2020). CO attainment (%) reflects achievement within a course, while PO attainment (%) represents how that course contributes to programme-wide outcomes. Together, they provide both course-level diagnostics and accreditation-level evidence.

This paper presents a case study of ETC574: Data Communication & Network, a Semester 4 course in the Bachelor of Engineering Technology (Digitalization & Automation) programme at Universiti Teknologi MARA (UiTM) Pasir Gudang. The course is mapped to three POPs which are CO2 (Problem Analysis), PO5 (Modern Tool Usage), and PO10 (Communication).The objectives of this study are to (1) Measure CO attainment percentages for ETC574 using direct assessments, (2) Translate CO attainment into corresponding PO attainment percentages and (3) Validate direct evidence with indirect measures, namely Student Feedback Online (SuFO) and Entrance–Exit Surveys (EES). By integrating CO%, PO% and indirect evidence, this study demonstrates a holistic approach to PO measurement, strengthening the reliability of outcome reporting and providing accreditation-ready evidence of teaching and learning effectiveness (Namratha et al., 2023).

LITERATURE REVIEW

OBE is now a globally adopted approach in engineering education, with increasing emphasis on not only achieving measurable PO, but also on validating them through a combination of direct and indirect evidence. To ensure that POs are attained effectively, several researchers have proposed and implemented various methods to assess both COs and their alignment with PO.

(Lavanya & Murthy, 2022) outlines a systematic methodology for assessing CO and PO attainment through a combination of direct assessments, such as exams and assignments, guided by Bloom’s Taxonomy. The study emphasizes the use of threshold levels to measure transformation in student learning, a method that mirrors the KPI approach adopted in the present study. (Kumar et al., 2021) present a case study applying CO–PO mapping and attainment analysis in a Big Data Analytics course. Their methodology includes using internal assessments, course feedback, and end-of-course surveys in demonstrating how triangulated evidence can provide a more comprehensive view of student achievement. The findings support the integration of multiple assessment components to improve course planning and evaluation.

Automated tools are also becoming more common in PO assessment. (Saad & Haque, 2020) describe an Excel-based model used for automating the direct assessment of outcomes. Their work supports the efficiency and scalability of digital assessment dashboards, aligning with recommendations for future enhancements mentioned in this study. Another relevant study by (Namratha et al., 2023) explores the use of Alternate Assessment Tools (AAT), including simulation software and peer reviews, to improve the attainment of specific POs in a network security course. The study found that interactive and hands-on activities significantly improved both technical understanding and communication skills, closely reflecting the approach used in the ETC574 course analyzed here.

Together, these studies affirm the effectiveness of a triangulated assessment model incorporating direct performance data, student feedback, and self-assessment in providing robust and accreditation-ready evidence of PO attainment. They also highlight best practices and tools that can be replicated across courses and institutions to ensure continuous quality improvement.

METHODOLOGY

This study was conducted within the Bachelor of Engineering Technology (Digitalization & Automation) programme at Universiti Teknologi MARA (UiTM), Johor, Pasir Gudang campus. The selected subject was ETC574 – Data Communication & Network, a Semester 4 core course designed to introduce students to the principles of data transmission, networking layers, Internet protocols, and the use of simulation tools for network analysis. In addition to technical competencies, the course emphasizes communication skills through written reports and oral presentations. A total of 27 students were enrolled in the March–August 2025 semester.

The course was structured around three Course Outcomes (COs): (1) the ability to analyze systematically issues in the physical and network layers, (2) the ability to construct network architectures and protocols using modern simulation tools, and (3) the ability to demonstrate written and verbal communication skills in presenting network designs and analyses. Each CO was mapped to a corresponding PO, ensuring alignment between subject-level outcomes and programme-level graduate attributes (Lavanya & Murthy, 2022).Table 1 presents the mapping between COs and POs.

To measure these outcomes, a Course Assessment Plan (CAP) was implemented, distributing assessment components across quizzes, laboratory work, projects, presentations, and tests. Each component was mapped to a CO, which in turn contributed to a specific PO. For example, CO1 (linked to PO2) was assessed mainly through test items, while CO2 (linked to PO5) was evaluated through laboratory sessions, assignments, and project work. CO3 (linked to PO10) was assessed through presentations and written documentation. Rubric-based evaluation methods were used to ensure consistency and objectivity in grading, in alignment with micro-level CO–PO mapping models used in other engineering programmes (Karthikeyan et al., 2021). The KPI for attainment was set at 65%, meaning that the total average mark of assessment that link to each PO should be at least 65%. The mapping of COs to assessment components is summarized in Table 2.

Table 1: CO–PO Mapping

Table 2 : Assessment Distribution Across COs and POs

In addition to direct assessments, indirect measurement instruments were employed to provide complementary evidence. The first was the  SuFO survey, administered at the end of the semester, which evaluated students’ perceptions of course delivery, assessment methods, lecturer professionalism, and infrastructure. The survey used a 4-point Likert scale (1 = Strongly Disagree, 4 = Strongly Agree). A total of 27 students completed the SuFO survey for this course. Indirect evaluations such as SuFO have been shown to offer reliable indicators of programme effectiveness (Ahmed Ghaly, 2020).

The second instrument was the  EES, which measured students’ perceived confidence in their networking knowledge and skills at the beginning and end of the semester. The survey included ten items aligned to the COs and their corresponding POs, such as describing basic networking concepts, configuring IP addressing, using simulation tools, and communicating technical content. Students rated their confidence on a 5-point Likert scale (1 = Very Low Confidence, 5 = Very High Confidence). Twenty-seven students completed both entrance and exit surveys, providing paired data to evaluate learning gain.

Data analysis followed a structured process. CO attainment percentages were first computed from the weighted performance of students across assessment components. These were then translated into PO attainment percentages according to the CO–PO mapping, such that each PO directly inherited the attainment level of its mapped CO. For instance, CO1 attainment of 70.15% represented the attainment of PO2. SuFO results were averaged across questions and reported as percentages for each section, benchmarked against the institutional performance scale (≥90% = Excellent, 80–89% = Very Good, 70–79% = Good, 60–69% = Average, <60% = Weak). EES results were analyzed by computing average scores for entry and exit stages, with the difference used to quantify learning gain. Finally, results from all three sources (CO%, PO%, SuFO, EES) were triangulated to provide a comprehensive evaluation of programme outcome achievement (Namratha et al., 2023).

RESULT & DISCUSSION

Direct Attainment of COs and Pos

The attainment results for the three Course Outcomes (COs) in ETC574: Data Communication & Network were computed from weighted student performance across assessments. Each CO mapped directly to a single Programme Outcome (PO), enabling direct translation of attainment values. The results are shown in Table 3 and represented in Figure 1.

All three COs exceeded the KPI threshold of 65%, with the highest performance observed in CO3 (79.33%), linked to communication (PO10). The lowest, CO1 (70.15%), still surpassed the KPI, though slightly weaker than the other outcomes, reflecting that students found analytical concepts in physical and network layers more challenging than practical or communication tasks.

Table 3: CO and PO Attainment Results

Figure 1. CO and PO Attainment Results

As shown in Table 3 and Figure 1, CO1-PO2 (Problem Analysis) recorded the lowest attainment (70.15%) but still surpassed the benchmark, indicating that students were able to analyze networking issues at the physical and network layers. However, the relatively lower score suggests that topics such as IP addressing and routing require additional emphasis in future iterations, possibly through extended tutorials or problem-based learning activities. CO2-PO5 (Modern Tool Usage) and CO3-PO10 (Communication) achieved higher levels of attainment, at 77.53% and 79.33% respectively, reflecting the effectiveness of practical laboratory sessions, simulator-based exercises, and structured communication assessments (Namratha et al., 2023). These findings are consistent with earlier studies that highlight the positive impact of hands-on simulation activities on learning outcomes in networking courses (Ahmed Ghaly, 2020).

Student Feedback Online (SuFO)

The SuFO survey collected student perceptions of knowledge gain, lecturer professionalism, teaching effectiveness and infrastructure. Table 4 summarizes the results, while Figure 2 visualize the Summary of Feedback represented by the spiderweb.

Table 4: Student Feedback Online (SuFO) Summary

Figure 2. Summary of Student Feedback Online (SuFO)

The Student Feedback Online (SuFO) responses from two classes, yielding a total of 27 respondents, revealed consistently high levels of student satisfaction. The combined averages across all dimensions indicated excellent performance, with overall course impression scoring 96.6%, lecturer professionalism 96 %, teaching and learning activities 95.4%, and infrastructure 94.4%. The grand average total was 95.86% percent, which falls into the “Excellent” category. Students particularly valued the course’s contribution to their knowledge and confidence, as well as the lecturer’s professionalism and clarity in content delivery. The only slightly lower rating was related to infrastructure, suggesting an area for continuous enhancement. These findings align with research supporting the use of indirect tools like SuFO to validate PO achievement through student experience and perception (Lavanya & Murthy, 2022).

The indirect assessment through SuFO further strengthens the validity of the direct results. With a combined satisfaction score of 95.86%, students affirmed the relevance of the course content, the effectiveness of teaching strategies, and the lecturer’s professionalism. Importantly, the consistently high ratings across all dimensions demonstrate that students perceived the course as well-structured and supportive of their learning, with infrastructure noted as the only area with slightly lower satisfaction, yet still within the “Excellent” category.

Entrance–Exit Survey (EES)

The EES captured students’ self-perceived confidence in achieving learning outcomes at the beginning (entry) and end (exit) of the semester. The results summarized in Table 5 and represented in bar comparative bar chart as shown in Figure 3 that demonstrate substantial learning gains across all ten items. The combined analysis of Entrance–Exit Surveys (EES) from both classes further validated the strong learning gain achieved in this course. Entry self-assessments averaged between 1.14 and 1.28 across ten skill domains, indicating low initial confidence and prior knowledge. At the end of the semester, exit ratings increased significantly, averaging between 4.46 and 4.74 across the same domains. For example, students’ ability to describe basic networking concepts improved from 1.22 at entry to 4.71 at exit, while competence in simulation tools improved from 1.22 to 4.66. Similarly, communication and presentation skills rose from 1.14 to 4.62. These results confirm meaningful learning progression in both technical and communication domains, aligning with evidence that student self-assessments can be a reliable indicator of skill development when triangulated with other data sources (Lavanya & Murthy, 2022).

Table 5: Entrance–Exit Survey (EES) Results

The most striking evidence comes from the combined EES data, which highlights dramatic improvements in student confidence and skill mastery across all ten measured domains. Entry scores around 1.2 confirm the students’ limited prior knowledge of networking, whereas exit scores near 4.6–4.7 demonstrate substantial gains in both technical and communication skills. The improvement patterns were consistent across cognitive domains (concepts, protocols, simulation tools) and soft skills (documentation, presentation), reflecting a well-rounded achievement of PO.

Figure 3. Entrance-Exit Survey Results

Triangulated Evidence

The integration of direct and indirect assessments provides strong triangulated evidence of PO achievement. Direct CO attainment confirms that students performed above the KPI in measurable assessments such as tests, labs, and case study. SuFO results validate that students were highly satisfied with the teaching and learning process, highlighting the course’s impact on knowledge acquisition and confidence building. The EES results provide further confirmation, showing significant self-reported improvement across all technical and communication skills.

This multi-layered approach aligns with literature advocating for triangulated data to ensure both validity and reliability in OBE environments (Tanjong et al., 2020). This triangulated approach strengthens the credibility of the findings by demonstrating consistency across multiple sources of evidence. The alignment of CO attainment, student satisfaction, and learning gain ensures that the course not only meets accreditation requirements but also promotes authentic student development. Such comprehensive evidence also allows programme leaders to pinpoint areas for enhancement, such as providing more scaffolding for complex concepts (CO1) and addressing infrastructure concerns highlighted in SuFO.

CONCLUSION

This study presented a comprehensive measurement of POs in the ETC574: Data Communication & Network course within the BTech programme. Using a triangulated approach that incorporated direct assessment (CO & PO attainment), indirect student feedback (SuFO), and student-perceived learning gain (EES), the results provide robust evidence of outcome achievement.

Direct CO attainment showed that all targeted POs, Problem Analysis (PO2), Modern Tool Usage (PO5), and Communication (PO10) were achieved above the University KPI threshold. Indirect feedback through SuFO indicated very high levels of student satisfaction, particularly with the lecturer’s professionalism and clarity of teaching, while the EES results highlighted significant growth in student confidence and competence across technical and communication domains. Together, these findings affirm that the course not only achieved its intended outcomes but also supported students’ overall academic and professional development.

The triangulation of multiple data sources strengthens the credibility of the results and ensures alignment with accreditation requirements under outcome-based education. At the same time, the evidence highlights areas for improvement, such as reinforcing conceptual understanding at the physical and network layer (CO1) and addressing infrastructure-related concerns raised in the SuFO. Future work could extend this analysis across multiple semesters, include employer or industry feedback, and integrate digital assessment dashboards for real-time monitoring of attainment.

Overall, the study demonstrates that ETC574: Data Communication & Network effectively contributes to the attainment of PO and provides a model of evidence-based evaluation that can be replicated across other courses within the programme.

REFERENCES

1. Ahmed Ghaly, S. M. (2020). Indirect Evaluation of Program Educational Objectives and Student Outcomes for Engineering Programs: A Case Study. Engineering, Technology and Applied Science Research. https://doi.org/10.48084/etasr.3751
2. Karthikeyan, P., Abirami, A. M., & Thangavel, M. (2021). A micro-level assessment methodology for attaining programme outcomes through undergraduate engineering projects. Journal of Engineering Education Transformations. https://doi.org/10.16920/jeet/2021/v34i0/157128
3. Kumar, K. S. A., Worku, B., Hababa, S. M., Balakrishna, R., & Prasad, A. Y. (2021). Outcome-based education: A case study on course outcomes, program outcomes and attainment for big data analytics course. Journal of Engineering Education Transformations. https://doi.org/10.16920/jeet/2021/v35i2/22072
4. Lavanya, C., & Murthy, J. N. (2022). Assessment and Attainment of Course Outcomes and Program Outcomes. Journal of Engineering Education Transformations. https://doi.org/10.16920/jeet/2022/v35i4/22109
5. Liew, C. P., Puteh, M., Mohammad, S., Omar, A. A., & Kiew, P. L. (2021). Review of engineering programme outcome assessment models. In European Journal of Engineering Education. https://doi.org/10.1080/03043797.2020.1852533
6. Namratha, M., Selva Kumar, S., & Sainath, K. (2023). Enhancement of Program Outcomes for Cryptography and Network Security course using Alternate Assessment Tool: An approach towards outcome-based education. Journal of Engineering Education Transformations. https://doi.org/10.16920/jeet/2023/v36i4/23121
7. Saad, K., & Haque, A. (2020). A Systematic Automation of Direct Assessment of Outcomes Attainment in Outcome Based Education. 2020 IEEE Region 10 Symposium, TENSYMP 2020. https://doi.org/10.1109/TENSYMP50017.2020.9230636
8. Tanjong, S. J., Razali, N. T., Andrew-Munot, M., Jong, R. P., Junaidi, E., & Jamali, A. (2020). Analyses on Programme Outcomes Measurements for Continuous Quality Improvement of an Undergraduate Engineering Programme. Journal of Southwest Jiaotong University. https://doi.org/10.35741/issn.0258-2724.55.6.6