Applying CIPP Model for Syllabus Evaluation: Assessing the MA in ELT Syllabus of a Private University in Bangladesh

Applying CIPP Model for Syllabus Evaluation: Assessing the MA in ELT Syllabus of a Private University in Bangladesh

Md. Ziaul Karim

Associate Professor (Englsih), Department of Languages, Faculty of Agriculture Bangladesh Agricultural University, Mymensingh-2202

DOI: https://dx.doi.org/10.47772/IJRISS.2025.9020114

Received: 31 January 2025; Accepted: 05 February 2025; Published: 05 March 2025

ABSTRACT

The graduate and post-graduate degrees in universities need to have contemporary courses that will impact the students’ lives and make them prepare for the realm of future career opportunities. That’s why; the curriculum and syllabus should see some moderations, deletions and incorporations as per the glocal needs. The CIPP (Context, Input, Process and Product) Model proposed by Stufflebeam (2007) is an assessment strategy to evaluate the degree programs in universities. In this article, it is shown how this model is adopted to prepare an evaluation report of the MA in ELT syllabus of a private university in Bangladesh. It aims to focus on improving the syllabus as per the present needs of the stakeholders. It also presents the existing syllabus and prescribes the areas of improvement through a narrative enquiry method. It further investigates the policy reforms at the departmental level, finds out the challenges to overcome, gives guidelines to upgrade and overall predicts the future prospects of the degree. It is expected to assist the policy makers of the university, members of the curriculum committee, teachers, students and other stakeholders to understand the present realities in order to frame a better syllabus and make the degree impactful as per the requirement of the present context in Bangladesh and abroad.

Keywords: CIPP model, syllabus evaluation, assessing university degrees, program upgradation

INTRODUCTION

The CIPP model by Stufflebeam has been tailored to cope up with the present context (a private university in Dhaka, Bangladesh). This model seeks to improve and achieve accountability in educational programming through a “learning-by-doing” approach (Zhang et al., 2011). According to Stufflebeam and Skinshield (2007), this is a program evaluation model developed in the 1960s by Daniel Stufflebeam and colleagues. CIPP is an acronym for context, input, process and product. The components of this evaluation model checklist are – contractual agreement, context evaluation (assesses needs, assets, and problems within a defined environment), input evaluation (assesses competing strategies and the work plans and budgets), process evaluation (monitors, documents, and assesses program activities), product evaluation (impact, effectiveness and sustainability evaluations) and two optional evaluations (transportability evaluation and metaevaluation). Stufflebeam and Shinkfield (2007) claim that “Evaluation’s most important purpose is not to prove, but to improve” (p.331). This model provides a holistic view by combining both types of evaluation – summative and formative.

This article describes the context of the study, rationale of the evaluation, research questions, methodology, the existing programme structure, findings based on the CIPP model, and some shortcomings before drawing the conclusion.

Context of the Study

The MA in English Language Teaching curricula (adopted in 2014-2015, still existing) has been selected for evaluation from a renowned private university where three-hour long classes usually take place on Fridays and Saturdays (weekly holidays) from 4:30 pm. There are total 30-35 students in all semesters and more than 500 others already completed the degree. An academic year was split into trimesters: Spring, Summer and Fall but since 2022, only two semesters were available– Spring and Fall, according to the directions of the University Grants Commission (UGC). There were five teachers (2 Lecturers, 2 Assistant Professors and the department Head who was an Associate Professor) in the English Department. Three teachers were from literature background while two were from language stream. Therefore, only two teachers took all the MA classes though sometimes adjunct faculty members were hired. These two teachers were also involved in the MA Curricula Design Committee of the department. A verbal agreement was taken between the evaluator (who is the researcher himself) and three stakeholders (2 teachers involved in the MA Curricula Design Committee and only 1 student who completed the degree).

The Structure of the Existing Programme Programme Duration:

Admission Requirement: An applicant seeking admission into a graduate program must have completed a bachelor’s degree with a minimum GPA of 2.0 or second division. (Extra credit requirements are applicable for students with a pass course or a three year bachelor degree). Admission committee decides how all applications would be assessed. In this process, certain standards were not met as the committee could use their power and whim in student selection.

Full-time/Part-time Students: A graduate student must register for at least 6 credits to be considered full-time. Students are allowed to register for a maximum of 12 credits in a semester.

MA in English Language Teaching (ELT) Programme Structure

List of Courses

Core:  33 credits

Optional:  3 credits

Additional Optional Courses (for Non-English four-year Honours and BA pass graduates): 12 Credits

Additional Optional Courses (only for BA pass graduates): 12 Credits

Rationale of the Evaluation

This evaluation was done as part of the coursework under the course EDEL609: Language Teacher Education and Development offered in the second semester of the PhD enrolment in the Language Education Department at Kathmandu University, Nepal. It was a study to judge how practically CIPP model could be applied for program evaluation. It not only enhance the researcher’s own insights but also provided other stakeholders related to this programme with a report that would be an eye opening call for updating the syllabus to survive in the competitive private education sector of Bangladesh. Furthermore, it would add insights to those teachers and policy makers who would involve themselves in program evaluation and upgradation as this is a common yearly or bi-yearly practice in many contexts in the world.

Research Questions

According to Borges and Hartung (2007), CIPP evaluation assists a decision-maker to answer four basic questions based on the four aspects in its name. Hence, the following recommended questions were applied for this evaluation:

• What needs to be done?  (context)
• How should it be done?  (input)
• Is it being done as planned? (process)
• Did the programme work successfully? (product)

METHODOLOGY

This is a qualitative study that involves narrative enquiry as its core methodology, while being guided by the prescribed CIPP model. Only two teachers involved in designing the curriculum and conducting all the classes, were interviewed in addition to a student who obtained this degree and was pursuing a second Master’s degree in an international university in Japan. The interpretive paradigm was applied to see the multiple realities involved in this evaluation. This evaluation was expected to uphold trustworthiness. According to Guba et. al. (1994), trustworthiness could be maintained through four criteria – credibility, dependability, confirmability and transferability. Moreover, proper ethical rules were maintained while collecting data. Implementation of ethical considerations usually depends on four principles namely Privacy, Accuracy, Property and Accessibility under the acronym PAPA (Sidgwick, 1907; Slote, 1985) which the researcher strictly maintained when dealing with the research participants during data collection.

FINDINGS

The CIPP model evaluation of the above-mentioned programme was conducted and the findings are very easily categorized into four segments that are shown below (the image is collected from Google) –

Context Evaluation Findings

As per the opinions of two teachers and the student, it was found that students needed to enrol for optional courses initially but they were mostly advised to take the core courses due to teacher shortage. Theories and lectures (didactic) were overpowering interactive and practical (Socratic/ dialectic) sessions in class. Only two full-time teachers had been teaching all the courses for the last 4 years. Sometimes adjunct faculty members were hired before that time. Ideally a teacher shouldn’t take more than 4 courses per semester but s/he was assigned 6 courses, which the teachers considered as overloads for them and they were not paid for these extra loads of classes. This load actually was imposed after the COVID-19 pandemic started. The course contents were not connected with the real-life problems/events and there was no performance/project-based testing, just cognition or memory based sit-down summative written tests were more weighted (50 marks) than formative tests as part of continuous assessments (attendance, quiz, assignment and midterm – 50 marks).

No upgradation of the syllabus was made until 2022 since 2014. A handbook, published in 2015, mentioned to provide service to all students who wanted to pursue that degree irrespective of their discipline or age only when they had a bachelor’s degree. Anyone (a student or a professional from any field in addition to an English teacher) could enroll in that program. Professional development was the ultimate aim but the degree should have been for the pre-service and in-service teachers only. This looked like supporting all types of professional development but actually the courses were related to English language education. Out of 15 current students, 3 were teachers; 1 was an ICT professional; the rest were regular students of English. In the past, businesswomen, secretaries, army personnel also completed this degree. For revising the syllabus, the internal curriculum committee members didn’t get any incentive but the experts from outside used to get a lump-sum amount.

Input Evaluation Findings

Under UGC’s directives, OBLT (Outcome Based Language Teaching)–based ELT was implemented since Spring 2023. Hence, internal teachers updated their syllabus according to the OBLT guidelines without market assessment or talking with the outside stakeholders to get it approved by the UGC in a hurry. The research participants opined that teaching practicum, school visits, practical class taking, class observations, attending conferences and idea sharing in one dish parties should be prioritized. As there was no MoU (memorandum of understanding) with schools and colleges, they didn’t want to allow students to take demo classes. However, personal networking worked in this case but it was sometimes difficult for some students. Students had to pay tuition fees 4000 taka per credit but there were waivers and scholarships for meritorious students. This MA program was mostly an ornamental program where during every Spring trimester, the highest number of students (from 8 to 10) got admitted but usually one or two students took enrolment in other two trimesters (Summer and Fall).

Process Evaluation Findings

Guidelines from National Qualification Framework of 2021 (Part-B, Tertiary Education) stipulated the following programmme structure to be implemented from Spring 2023:

Based on the above structure, there would be three types of MA programmes and that is why, the internal teachers were working accordingly. Classrooms needed surround sound systems but  all classrooms didn’t have the sound systems. The participants felt that ELT Management course should be optional, not a core course. Co-curricular activities like arranging seminar, school/ college visit needed to be arranged and negotiated. For example, the student participant gave an example from the Japanese university (where she was pursuing her 2^nd master degree) that she had 3 Teaching Practicum courses (1 credit each) in 3 semesters. Teachers there demonstrated how to take classes and arrange school visits. Students shared their reflective writings after visiting the schools and making teaching demonstrations.

Product Evaluation Findings

Teaching Practicum and Dissertation writing were very important and hence, more credits needed to be allocated for these courses. Internship programs (4 types, based on the backgrounds of students – primary, secondary, higher secondary and tertiary) must be incorporated due to the UGC directives. Admission tests were required to filter the students’ capability. Every two years, a budget should be allocated to evaluate the syllabus through revising the courses, deleting the obsolete ones, synthesizing the overlapping and incorporating the trending courses. The student participant mentioned the names of some courses such as Blended Learning in a Digital Age, Teaching and Learning Online, ELE for a Sustainable Development, English for Employability, Language Teacher Education and Development and Equality, Diversity and Inclusion in ELE, which needed to be incorporated in the syllabus. The university should recruit more efficient teachers who could teach and design syllabus.

Regarding the effectiveness of the programme and its sustainability, let’s look at a similar programme run by the most reputed private university (the 1^st established private university in Bangladesh) in Dhaka. Master of Arts (MA) programs in English were divided into three specializations: TESOL, linguistics, and literature. Course fee was 6,500 taka per credit. The English Department offered a number of foreign language courses for the students of the university and organized a variety of activities such as the Spectrum Seminars (seminar series), Discourse I and II (lecture series), and language workshops to raise language awareness and generate research interest in language and related issues among the students as well as the faculty members. That shows how varieties and diversities should be nurtured to build an image and reputation among the stakeholders.

Shortcomings of the CIPP Model

The CIPP model, however, had some shortcomings. It seemed difficult to implement and expensive to maintain. The researcher had to cut short the checklist to adapt to his small evaluation procedure. There was a lot of overlappings in the checklist which made the evaluator’s job a bit complex. Alkin (2004) outlined the demerits of this model as follows:

• Evaluator may not deal with issues that go against the concerns of the decision maker
• If there is not decisive leadership, this model would be ineffective
• Top management is given preference – so, it may become unfair and undemocratic

Driscoll et. al. (1998) argued that the model could be said to blur the line between evaluation and other investigative processes such as needs assessment.

CONCLUSION

This study is a micro level research work to apply the CIPP model to evaluate an MA programme. Here, transportability evaluation and metaevaluation are not considered as they are time consuming in nature. However, this model guided the interview and data collection procedure in step by step progression. Evaluation based on a model like this is very essential every two years for any existing programme in a university. It will provide a step by step guide to the programme evaluators and make the degree programmes effective and up-to-date to keep pace with the global educational trends.

REFERENCES

1. Alkin, M. C. (2004).Evaluation roots: Tracking theorists’ views and influences. Thousand Oaks, CA: Sage.
2. BNFQ. (2021). Bangladesh National Qualification Framework, Part-B: Higher Education
3. Borges, N., &Hartung, P. (2007).Service-learning in medical education: Project description and evaluation. International Journal of Teaching & Learning in Higher Education, 19(1), 1–7.
4. Driscoll, A., Gelmon, S., Holland, B., Kerrigan, S., Spring, A., Grosvold, K., & Longley, M. (1998). Assessing the impact of service-learning: A workbook of strategies and methods (2nd ed.).Portland State University.
5. Guba E. G., Lincoln, Y. S., & Denzin, N. K. (1994). Handbook of qualitative research. Thousand Oaks, CA: Sage, pp. 105– 17.
6. Sidgwick, H. (1907). The Methods of Ethics. 7th ed. London: Macmillan.
7. Slote, M. (1985). Common-Sense Morality andConsequentialism. London: Routledge & Kegan Paul.
8. Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, & applications. San Francisco, CA: Jossey-Bass.

APPENDIX

CIPP evaluation model checklist to assess the MA in ELT curricula at a private university

The CIPP Model is tailored to cope up with the context (a private university in Bangladesh) and to assess the curricula based on the evaluator’s (Md. Ziaul Karim) and stakeholders’ (2 Teachers & 1 student) feedback.

Contractual Agreement

Context Evaluation

(Context evaluation assesses needs, assets, and problems within a defined environment)

Input Evaluation

(Assesses competing strategies and the work plans and budgets of the selected approach)

Process Evaluation

(Process evaluations monitor, document, and assess program activities)

Impact Evaluation

(Assesses a program’s reach to the target audience.)

Effectiveness Evaluation

(Documents and assesses the quality and significance of outcomes)

Sustainability Evaluation

(Assesses the extent to which a program’s contributions are institutionalized successfully and continued over time.)