Assessing and Monitoring Online Learning in the Midst of Covid-19 Pandemic in Olivarez College Tagaytay: A Sequential Mixed Method Approach

Assessing and Monitoring Online Learning in the Midst of Covid-19 Pandemic in Olivarez College Tagaytay: A Sequential Mixed Method Approach

Ann Rachel Y. Cuadra

Olivarez College Tagaytay Tagaytay City, Philippines

DOI: https://dx.doi.org/10.47772/IJRISS.2024.8120027

Received: 27 June 2024; Received: 15 July 2024; Accepted: 20 July 2024; Published: 28 December 2024

ABSTRACT

Education has altered tremendously due to the Covid-19 epidemic, with the emergence of online learning, where teaching is done remotely and on digital platforms. The assessment and monitoring of online learning were brought up to be one of the things that must be considered in the changes brought about by the pandemic. As a result, the researcher utilized the Exploratory Sequential Mixed Methods Research Design to explore the experiences of the Strand Coordinators, teachers, and students in assessing and monitoring online learning and the challenges that emerged through observation, interview, and survey. The study was carried out in Olivarez College Tagaytay Senior High School Department, conducting Focus group discussions with the three groups to obtain necessary data. Based on the results of the qualitative phase of the study, three themes were identified: (1) Creative Integration of online assessment strategies, (2) Increasing Engagement in the Virtual Classroom, and (3) Time-on-Task in achieving essential learning competencies. The category utilization of open source platforms, which got the highest mean score among the four categories in challenges encountered, must be given attention. The result leads to an enhancement of the assessment framework for the Senior High School Department that will help the teachers cope with the situations that they are facing.

Keywords: Education, assessment, monitoring, online learning, exploratory mixed method research design, Philippines

INTRODUCTION

Under the extraordinary circumstances of the COVID-19 pandemic, and as part of the social distancing measures implemented to prevent the transmission of the coronavirus SARS-CoV-2, all educational institutions in practically every country across the world declared a suspension of in-campus activities (Engzell et al., 2020). Our education system is confronted with an unprecedented scenario, which the government must address by attempting to maintain quality education despite the fact that students and teachers are unable to collaborate in the classroom where teaching no longer takes place in the classroom, but at home. Learning should never cease, and because the pandemic altered our way of life, the Department of Education (DepEd) made changes to the basic education curriculum to reflect the current scenario caused by the Coronavirus (COVID-19).

A Learning Continuity Plan had been devised by the Department of Education (DepEd) (LCP). Face-to-face learning has been completely substituted by distance learning via various learning management systems and platforms, and all educators are putting in considerable effort to continue teaching and assessing their courses (Trucco & Palma, 2020).

According to the UNICEF East Asia and Pacific Regional Office, also known as UNICEF EAPRO (2020), distance learning programs need students to have a high level of autonomy in terms of self-learning and self-motivation. Learning autonomy is vital not only in distant learning but also in traditional classrooms; nevertheless, many students, particularly younger children, may not have established enough autonomy for self-learning. Teachers in distance learning programs lack the tools they need to assess students and provide feedback and formative coaching. Informal assessment may be more problematic for online teachers due to the lack of face-to-face contact, according to Oncu & Cakir (2011). The remote character of Web-based techniques renders many observational and participatory assessments challenging that could hinder the students’ learning (Hannafin et al., 2003).

Teachers have a significant role in this pandemic especially in assessing and monitoring online learning, having the assurance that the students can achieve the same learning outcomes during the face to face classes (Lapsley, 2008). New regulations for distance learning, whether online or not, necessitate rapid changes and transition in teachers’ daily activities, obligations, and accountability. These changes are inevitable as the time goes by. According to the theory of Blending with Purpose: The Multimodal Model by Picciano (2009), in particular, recognizes that teachers and instructional designers should seek to use multiple approaches, including face-to-face methods and online technologies that includes content, dialectics or questioning, reflection, collaborative learning, and evaluation to address the learning needs of a wide range of students, because learners represent different generations, personality types, and learning styles. That is why individual and group feedback and support may require different ways depending on the grade and topic area. (UNICEF, 2020)

Students nowadays have access to different sites at home like Google, which could help them answer their assessments quickly and make it difficult for the teacher to monitor their learning. According to the ranking analysis made by the Similar Web (2021), Google.com is the most popular website in the Philippines as of October 2021. Users spend an average of undefined minutes on the website, and they view an average of undefined pages every visit. The total number of website views divided by the number of visitors yields pages per visit, a popular engagement statistic. Google.com has an undefined bounce rate, which means that undefined visitors leave after seeing only one page.

It is easy for the students to look for answers online, and it does not show enough evidence to prove the quality of their learning. Seeing the needs of the students in learning, especially in these times of pandemic, the researcher chose this topic to know the issues that teachers, students, and administration are encountering to ensure the students’ learning. As a teacher and Strand Coordinator, the researcher has a passion for improving online assessments and monitoring to ensure quality in the teaching and learning process. Through this research, the academic community were aware of the encountered issues and adopted the best practices in assessing and monitoring online learning to ensure quality in the teaching and learning process amidst the pandemic.

Given the situation wherein teachers and students are still in the process of adjustment in this pandemic, this study aims to examine the teachers’ and students’ experiences in the assessment and monitoring of online learning in the new normal and what challenges emerge through observation and interview. This study emphasized the necessity of developing a quality assurance mechanism for assessing and monitoring online learning that includes relevant quantitative and qualitative indicators to address the educational, not just technical, strengths and limitations of technology-dependent learning.

METHODS

An exploratory sequential mixed method research (MMR) design was used in order to broadly explore the assessment and monitoring of online learning among the Senior High School Teachers of Olivarez College Tagaytay. As a result, three rounds of analyses are carried out in this design: after the primary qualitative phase, after the secondary quantitative phase, and finally, after the integration phase, which joins the two strands of data and expands the initial qualitative exploratory discoveries (Creswell & Clark, 2011).

Figure 1 Exploratory Sequential Mixed Methods Research Design

The semi-structured interview was utilized to collect data for the qualitative portion of this study. The interview outline served primarily as a guide; the researcher encouraged interviewees to ask their own questions while asking questions, and modified the interview technique and content to the interview’s individual circumstances. The researcher used three (3) themes and six (6) sub themes from the qualitative phase to build the survey questionnaire, which was used to determine dimensions and items in the questionnaire. The content validity and reliability of the research instrument was also tested. The Cronbach Alpha was used to determine whether the instrument is consistent, accurate, and dependable in terms of producing consistent results with the same set of people under the same conditions.

In phase one, Qualitative, the study was conducted in Olivarez College Tagaytay as the selected participating private school in the District of Tagaytay City. Purposeful sampling was used to select the participants of this study. Strand Coordinators, Senior High School Faculty, Senior High School students were the participants of this study. The sample size of qualitative study is calculated based on the principle of data saturation until no new information is available. While, for the quantitative part of the study, the entire population of Strand Coordinators, faculty members, and students were used. Universal Sampling technique was utilized in this study

RESULTS

The researcher slices down the result into two parts as narratives and tables. Primarily, the description of narrated responses of the three groups of participants, and transcribed them based on the interview questions using open coding, and constant comparative method. Second, the survey questionnaire responses of the three groups of respondents in numerical form accompanied by statistical analysis attempts to answer the research questions. Hence, determining the core categories, shaping the emerging themes, mixing of the filtered responses from both research approaches, and sorting until the end product were of this data analysis.

An exploration of assessment of learning and monitoring systems in the new normal was made possible by conducting Focus Group Discussions (FGDs) in Olivarez College Tagaytay, where online learning of the Senior High School Department was present. Three (3) groups of participants became part of this method. This consists of three (3) Strand Coordinators, four (4) Senior High School Teachers, and four (4) Senior High School Students. The researcher facilitated the three (3) FGDs after achieving data saturation. The findings were translated in English and back translated. The characteristics of respondents were described in the Population and Sampling Section (Chapter 2) prior to the matrix emerged.

Table 1 Matrix of Emerging Themes on the Experiences in the Assessment and Monitoring of Online Learning in the Midst of COVID 19 Pandemic

The matrix of emerging themes and subthemes given in table 1 gave rise to the experiences of three (3) groups of participants in the assessment and monitoring of online learning in the midst of Covid- 19 pandemic. The following themes are identified which represent the indicators on the extent of implementation of online assessment of learning and monitoring systems in the midst of Covid-19 pandemic in Olivarez College Tagaytay. The discussion of each theme will be presented below.

For phase two, Quantitative, the researcher used the qualitative themes in conjunction with the theoretical framework to identify the major sections and scales of the instrument. Table 4 aligns the theoretical framework, qualitative findings, and instrument dimensions. The qualitative themes emerged from FGD aligned with the five (5) components of blending with Pedagogical Purpose Model: content, dialectics or questionnaires, reflection, collaborative learning, and evaluation.

Table 2 Aligning the Theoretical Framework, Qualitative Themes and Instrument Dimension

Table 2 presents the scale construction considered in this study as a rating scale. Therefore confidence ratings in the form of a Likert scale were affixed to each initial statement. Response choices in rating the extent of implementation of online assessment of learning and monitoring systems in the new normal fell across a continuum ranging as 1- Not Implemented, 2- Slightly Implemented, 3- Implemented, and 4- Fully Implemented. While in rating the challenges encountered by the participants to the online assessment of learning and monitoring system in the new normal fell across a continuum ranging as 1- Not Serious Issue, 2- Fairly Serious Issue, 3- Moderately Serious Issue, and 4- Very Serious Issue.

Table 3 Challenges Encountered by the Participants to the Online Assessment of Learning and Monitoring System in the New Normal

Utilization of online open source platforms is a moderately Serious Issue (Table 3). Specifically, the indicator under this category with the most concern was the difficulty in understanding some parts of online application. Among the indicators rated, the learning assessments are not up- to-date and not compatible with the situation at hand is the most challenging. Hence, the above data showing the highest rating of student and Strand Coordinator groups may imply that being technology-oriented is essential to the online learning assessments and experiences in the new normal. Thus, the institution may provide other seminars and training for the teachers as implementers of assessment in online learning. According to the study of Hoffman et al. (2020), students and teachers must be prepared for the academic and social learning that will be part of their educational journey, whether they are taking a single online class or a fully virtual class. While some students may have years of online learning experience, others may need to become acquainted with the methods, tools, expectations, and campus community—and all will benefit from an orientation to their new academic community. It might be a serious issue if the students and teachers are not oriented before the implementation of the transition.

Table 4 Extent of Implementation of Online Assessment of Learning and Monitoring System in the New Normal

Utilization of online open source platforms as lowest indicator and verbally interpreted as Moderately Implemented (Table 4). Specifically, the indicators under this category that must be given focus was the consideration of the user’s feedback on the utilization of online open source platforms in assessing learning. Generally, the extent of implementation of online assessment of learning and monitoring in the new normal was rated as Fully Implemented. However, the above data, showing the lowest rating by the Strand Coordinator group may indicate that devising a feedback tool is essential to the teaching and learning process especially in monitoring learning. Hearing students’ comments and suggestions may help the teachers and Strand Coordinators in improving the learning materials and assessments of online learning. This could also help the teachers to monitor the progress of the learners. Hattie (2009) stated that feedback has a significant impact on student progress. Teaching and learning can be synchronized and strong when teachers seek or at least are open to what students know, understand, where they make mistakes, when they have misconceptions, and when they are not engaged. This may indicate that consideration of the users’ feedback on the utilization of online open source platforms is important among the three groups of respondents. Teachers receive and provide feedback, which makes learning more evident.

Table 5 Difference in the Challenges Encountered during the Implementation of Online Assessment and Monitoring

The null hypothesis is rejected, there is a significant difference on the ratings given by the three groups of respondents to the online assessment of learning and monitoring systems in the new normal (Table 5). This may imply that how each group rated the extent of implementation of assessment of online learning and monitoring systems in the new normal are different from each other. Students, as the one who are being assessed, teachers as implementers of learning, and Strand Coordinators as the one who supervised and ensure the achievement of learning outcomes have different perceptions when it comes to the success in online learning.

Table 6 Difference in the Challenges Encountered during the Implementation of Online Assessment and Monitoring

The null hypothesis is accepted, there is no significant difference between the rating of the three groups of respondents towards the challenges encountered in online assessment of learning and monitoring systems in the new normal. This may imply that among the three groups of respondents, each could relate to the same challenges encountered during the implementation.

DISCUSSION

The participants described the online learning and monitoring system assessment in the new normal as extremely challenging to grasp. Higher demands were placed on students, instructors, and the senior high school department’s strand coordinators because of the COVID-19 pandemic. These individuals had to make significant adjustments to e-learning. According to the research participants, implementing these different changes has presented challenges for all those involved in education.

The challenges that were experienced by the groups taking part in this study differ among themselves. Participants identified all criteria related to learning evaluation and monitoring in the new normal must be addressed through statistical analysis. Although it is paramount to focus on other identified hurdles in this sector during adoption, for instance, open source platforms’ mean score surpassed other means with significance among the four categories, some cases cause more attention than others.

An example of this is the utilization of internet source platforms, which poses problems for learners, notably understanding various bits of online applications. Teachers and learners can tackle this challenge by offering regular training sessions and seminars or workshops together. This aims to realize effective learning results in virtual classrooms. Feedback and progress monitoring issues, which are equally important, achieved the lowest scores on the extent of implementation. These areas also deserve consideration to ensure a holistic improvement of the online learning environment.

Although considered implementing online learning assessment fully during the COVID-19 pandemic, schools need to pay more attention to using open-source platforms. It is important for one to consider the feedback from end-users about the use of open-source platforms in evaluating learning online. Devising a feedback tool is essential to teaching and learning, particularly in monitoring learning, given that it involves the students’ feedback and consistency in progress monitoring. Curriculum designers may improve the learning materials and assessments of online learning by hearing students’ comments and suggestions, like aiding teachers to monitor learners’ progress.

The three groups of participants gave different ratings for the online evaluation of the learning to monitor the system under the new normal. The researchers rejected the null hypothesis because they found a significant difference between these ratings. However, the null hypothesis stands in relation to the problems, showing that there is no significant distinction among the three group respondents’ ratings concerning this front.

CONCLUSION

In conclusion, this study mentions various challenges that the participants experienced. The statistical analysis revealed that every indicator or difficulty that the participants faced when evaluating learning and monitoring systems in the new normal needed to be attended to and given careful consideration, particularly when it came to the use of open-source platforms, which received the highest mean score out of the four categories.
In general, the degree of online monitoring and learning assessment in the new normal was graded as Fully Implemented. Nonetheless, the signal under the lowest category of open-source platform use required attention, which was the evaluation of customer feedback regarding the use of online open-source platforms.

With this, an enhancement to the assessment tool utilized by the Olivarez College Tagaytay will be proposed to address the users’ feedback on the utilization of the online open source platform in assessing learning as it was rated the lowest among the four indicators under the lowest category, utilization of open source platform. The said enhancement will also be available and applicable to modular learning as part of the different learning modalities of the institution.

Talking more about the use of open-source platforms would make sure that other important results are noticed by the general impact of the study it will enhance. To refine the framework through which this assessment is carried out and prove its usefulness in myriad educational settings, researchers should conduct further study to test it. Researchers need to take proactive steps to improve the assessment framework and verify its efficacy in various educational environments. This can be achieved by conducting a follow-up study to analyze and validate its practical application and benefits.

Output of the Study

The Propose Enhance Assessment Framework aims to address the need for consideration of students’ feedback and consistency in progress monitoring and providing timely feedback to the learners. The mentioned OCT Assessment Framework is being utilized in the College Department as a monitoring tool and basis for continuous quality improvement to check the progress of the students, assess learners aligned to the learning outcomes, and give the right feedback to address the needs of the learners. The proposed enhancement is to make it institutionalized and be adapted by the Senior High School Department online learners, and can also be contextualized to the modular learners. However, the students’ feedback part of the assessment will be improved and will make it more accessible to the learners and teachers whether online or modular learning.

STEP 1 : Planning the Assessment
STEP 2 : Developing / Using Appropriate Assessment Tools
STEP 3 : Collecting Detail Data
STEP 4 : Calculating the Result of Achievement
STEP 5 : Analyzing the Result
STEP 6 : Propose Improvements
STEP 7 : Documentation

The framework’s objective is to ensure that there are mechanisms, procedures and processes in place to ensure that the desired quality, however defined and measured, is delivered. In this framework, the following aspects were taken into consideration: First, the performance indicators that were set at the start while planning for the subject, i.e., the competencies under each outcome, that each student should demonstrate at the end of the semester; Second, the assessment methods which should be appropriately selected to measure the performance; Third, the standards that indicate the quality of the product, i.e., level of student performance. This aspect also indicates the presence, extent, and effectiveness of the implementation of the learning plan which was required in order to deliver a certain level of performance. Moreover, the framework is anchored to the Outcome- Based Education. OBE – is an Education System that requires the Learning Outcomes to be made explicit and visible, aligned Assessment to the intended learning outcomes, aligned Delivery (Learning Activities & Environments) to the intended learning outcomes, and provide necessary infrastructures and support systems for the above to happen. Subject teachers must be able to continuously use this framework to make sure that they monitor the progress of their students, give the right assessments, and achieve the needed learning outcomes/ competencies at the same time.

ACKNOWLEDGMENT

I would like to express my deepest gratitude for the untiring support and effort of the following persons who shared their valuable and substantial support behind the improvement of this study:

Foremost, the researcher would like to thank the loving Creator, God Almighty for the wisdom and strength to undertake the thesis. Without His blessings, this achievement would not have been possible,

Dr. Jeanrizza A. Dela Cruz, my thesis adviser, for her unceasing support and guidance and passionately sharing her precious time and commitment during the development of this paper,

Dr. Elmer De Leon, for sharing his immense knowledge, insightful comments and remarks during the revision of this master’s thesis,

Likewise, the author expresses appreciation to the associates of the Oral Committee, Dr. Gemma Rojales, Dr. Crisanto Limson and Dr. Bernadeth Limson, for sharing their expertise in the improvement of this research study,

Ms. Clarence A. Castillo, Olivarez College Tagaytay SHS Principal, for allowing to conduct the research in the department.

To the Strand Coordinators, Faculty Members and SHS Learners as study participants who have readily contributed and shared their experiences in this paper, without your contribution, this paper would not materialize.

Lastly, the researcher humbly and faithfully expresses his gratitude and joy to the LORD JESUS CHRIST, our God who gave him light through the Holy Spirit, with all the strength, knowledge and wisdom, courage and determination to surpass all the hardships.

REFERENCES

1. Altheide, D., & Schneider, C. (2013). Qualitative media analysis (2).
2. Ambrose, S., Bridges, M., Lovett, M., DiPietro, M., & Norman, M. (2010). How learning works: 7 research-based principles for smart teaching.
3. Bennett, R. E. (2011). Formative assessment: a critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25. https://doi.org/10.1080/0969594x.2010.513678
4. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. doi:10.1191/1478088706qp063oa/.
5. Camacho, D. J., & Legare, J. M. (2016). Shifting gears in the classroom-movement toward personalized learning and competency-based education. The Journal of Competency-Based Education, 1(4), 151–156. https://doi.org/10.1002/cbe2.1032
6. Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry & research design: Choosing among five approaches (4).
7. Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. Handbook of qualitative research, 105–117.
8. Han, F., & Ellis, R. A. (2019). Identifying consistent patterns of quality learning discussions in blended learning. The Internet & Higher Education, 40, 12-19.
9. Goodman, J., Schlossberg, N. K., & Anderson, M. L. (2006). Counseling adults in transition: Linking practice with theory. (3)
10. Asia-Pacific Economic Corporation. (2019, December). APEC Quality Assurance of Online Learning Toolkit. SOM Steering Committee on Economic and Technical Cooperation (SCE), Human Resources Development (HRDWG).