Content Validity of an Instrument to Assess Innovation Competencies in Engineering Education

Innovation competency has become an essential requirement for engineering graduates as they navigate increasingly complex and technologically advanced environments. This study aims to develop and validate an instrument to assess innovation competencies among engineering students in Malaysian technical and vocational institutions. The construction of the instrument was guided by grounded theory insights obtained from structured interviews with engineering educators, together with a synthesis of current literature and ABET-aligned competency frameworks. This process established three major dimensions knowledge, skills, and personality which formed the basis for an initial pool of 53 items. The content validity of the instrument was evaluated by a panel of five experts using the Content Validity Index (CVI) and the modified Kappa statistic. Experts rated each item for relevance using a four-point scale. The results demonstrated strong overall agreement, with most items achieving I-CVI values between 0.80 and 1.00 and Kappa coefficients indicating good to excellent concordance. Items falling below acceptable thresholds were removed or refined based on expert judgment. As a result, 42 items were retained, representing essential components of innovation competency across the three dimensions. The findings confirm that the instrument possesses strong content validity and is suitable for further psychometric evaluation. This validated item set offers educators, curriculum designers, and policymakers a structured and evidence-based tool for assessing innovation readiness among engineering students. The instrument has potential applications in evaluating program effectiveness, identifying student competency gaps, and supporting targeted instructional improvements. Future research should incorporate pilot testing, reliability analysis, exploratory and confirmatory factor analysis, and multi-institutional validation to strengthen the instrument’s robustness, generalizability, and practical utility.

innovation competency, content validity index, Kappa statistic, engineering education

1. Akdur, D. (2021). Skills gaps in the industry: Opinions of embedded software practitioners. ACM Transactions on Embedded Computing Systems (TECS), 20(5), 1-39. [Google Scholar] [Crossref]

2. Almanasreh, E., Moles, R. J., & Chen, T. F. (2022). A practical approach to the assessment and quantification of content validity. In Contemporary research methods in pharmacy and health services (pp. 583-599): Elsevier. [Google Scholar] [Crossref]

3. Chong, J., Mokshein, S. E., & Mustapha, R. (2021). A content validity study for vocational teachers’ assessment literacy instrument (VoTAL). International Journal of Academic Research in Business and Social Sciences, 11(4), 868-883. [Google Scholar] [Crossref]

4. de Campos, D. B., de Resende, L. M. M., & Fagundes, A. B. (2020). The importance of soft skills for the engineering. Creative Education, 11(8), 1504-1520. [Google Scholar] [Crossref]

5. Ealangov, S. (2023). Cabaran dan Strategi Menghadapi Perubahan Kurikulum dalam Kalangan Pensyarah Bidang Agroteknologi Kolej Komuniti: Challenges and Strategies in Curriculum Change among Agrotechnology Lecturers in Community College. Online Journal for TVET Practitioners, 8(2), 23-36. [Google Scholar] [Crossref]

6. Gorlewicz, J. L., & Jayaram, S. (2020). Instilling curiosity, connections, and creating value in entrepreneurial minded engineering: Concepts for a course sequence in dynamics and controls. Entrepreneurship Education and Pedagogy, 3(1), 60-85. [Google Scholar] [Crossref]

7. Hirudayaraj, M., Baker, R., Baker, F., & Eastman, M. (2021). Soft skills for entry-level engineers: What employers want. Education Sciences, 11(10), 641. [Google Scholar] [Crossref]

8. Indarta, Y., Ranuharja, F., & Dewi, I. P. (2023). Measuring Validity of Interactive Presentation Media Using Content Validity Index (CVI). Paper presented at the 9th International Conference on Technical and Vocational Education and Training (ICTVET 2022). [Google Scholar] [Crossref]

9. Jamaludin, T. S. S., Nurumal, M. S., Ahmad, N., Muhammad, S. A. N., & Chan, C. M. (2021). Development and evaluating content validity of clinical skill analysis index tools. Open Access Macedonian Journal of Medical Sciences, 9(T5), 6-12. [Google Scholar] [Crossref]

10. Kipli, M., & Khairani, A. Z. (2020). Content Validity Index: An application of validating CIPP instrument for programme evaluation. Int Multidiscip Res J, 2(4), 31-40. [Google Scholar] [Crossref]

11. Madadizadeh, F., & Bahariniya, S. (2023). Tutorial on how to calculating content validity of scales in medical research. Perioperative Care and Operating Room Management, 31, 100315. [Google Scholar] [Crossref]

12. Munir, F. (2022). More than technical experts: Engineering professionals’ perspectives on the role of soft skills in their practice. Industry and Higher Education, 36(3), 294-305. [Google Scholar] [Crossref]

13. Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health, 29(5), 489-497. [Google Scholar] [Crossref]

14. Qadir, J., Yau, K.-L. A., Imran, M. A., & Al-Fuqaha, A. (2020). Engineering education, moving into 2020s: Essential competencies for effective 21st century electrical & computer engineers. Paper presented at the 2020 IEEE Frontiers in Education Conference (FIE). [Google Scholar] [Crossref]

15. Reich, P. D., Marchesi, P., Hamilton, P. L., Austin, A. W., Dehne, C., Munson, J., . . . Klimek, J. F. (2020). The Synergistic Classroom: Interdisciplinary Teaching in the Small College Setting: Rutgers University Press. [Google Scholar] [Crossref]

16. Rovida, E., & Zafferri, G. (2022). The importance of soft skills in engineering and engineering education: Springer. [Google Scholar] [Crossref]

17. Roy, M., & Roy, A. (2021). The rise of interdisciplinarity in engineering education in the era of industry 4.0: implications for management practice. IEEE Engineering Management Review, 49(3), 56-70. [Google Scholar] [Crossref]

18. Shrotryia, V. K., & Dhanda, U. (2019). Content validity of assessment instrument for employee engagement. SAGE Open, 9(1), 2158244018821751. [Google Scholar] [Crossref]

19. Spoto, A., Nucci, M., Prunetti, E., & Vicovaro, M. (2023). Improving content validity evaluation of assessment instruments through formal content validity analysis. Psychological methods. [Google Scholar] [Crossref]

20. Suyuti, S. (2024). The Importance of Creativity and Innovation in Education: How to Prepare Students for the 21st Century Workforce. Education Studies and Teaching Journal (EDUTECH), 1(1), 80-92. [Google Scholar] [Crossref]

21. Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J.-M., Morisseau, T., Bourgeois-Bougrine, S., . . . Mourey, F. (2023). Creativity, critical thinking, communication, and collaboration: assessment, certification, and promotion of 21st century skills for the future of work and education. Journal of Intelligence, 11(3), 54. [Google Scholar] [Crossref]

22. Tutar, H., Şahin, M., & Sarkhanov, T. (2024). Problem areas of determining the sample size in qualitative research: a model proposal. Qualitative Research Journal, 24(3), 315-336. [Google Scholar] [Crossref]

23. Vasileiou, K., Barnett, J., Thorpe, S., & Young, T. (2018). Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC medical research methodology, 18(1), 148. [Google Scholar] [Crossref]

24. Walter, M., & Mondal, P. (2019). A rapidly assessed wetland stress index (RAWSI) using Landsat 8 and Sentinel-1 radar data. Remote Sensing, 11(21), 2549. [Google Scholar] [Crossref]

25. Zamanzadeh, V., Rassouli, M., Abbaszadeh, A., Majd, H. A., Nikanfar, A., & Ghahramanian, A. (2014). Details of content validity and objectifying it in instrument development. Nursing Practice Today, 1(3), 163-171. [Google Scholar] [Crossref]

• LeafQuest: A Mobile-Based Augmented Reality for Plant Placement, Discovery, and Growth
• Participatory Ergonomic Intervention Approach on Musculoskeletal Disorder (MSD) in Construction Sectors: A Systematic Review
• Integrating GIS into Traffic Incident Management: A Web-Based System
• RideSmart: A Personalized Motorcycle Product Recommendation System Using TF-IDF and Descriptive Analytics for Javidson Motorshop
• Educational Technology Course Design in Pre-Service Teachers Education: A Bibliometric Review of the Research Landscape