Validity and Reliability of Senior Secondary School Class 2 Unified Promotion Examination in Mathematics in Ondo State, Nigeria

Joseph Oluwatayo Osakuade

Department of Guidance and Counselling, Faculty of Education, Adekunle Ajasin University, Akungba-Akoko, Ondo State, Nigeria

DOI: https://doi.org/10.51244/IJRSI.2024.1102004

Received: 07 January 2024; Revised 19 January 2024; Accepted: 24 January 2024; Published: 28 February 2024

ABSTRACT

Despite the importance of Mathematics to sustainable national development, there has been a repeated poor performance of secondary school students in the subject in Ondo state, Nigeria. The assumption of this researcher is that the psychometric properties of Unified Promotion Examination (UPE) used to promote these students to certificate class were not properly ascertained. This study sought to ascertain the validity and reliability of the UPE Mathematics objective items between 2017 and 2018. Two research questions were raised and two hypotheses formulated. The study was anchored on descriptive survey research design. The population consisted of all public senior secondary school class two (SSS2) students for 2017 and 2018. Nine hundred (900) SSS2 students constituted the sample. Multistage and stratified random sampling techniques were used to choose the sample. The research instrument that was used for data collection was the Optical Magnetic Reader (OMR) sheets. Data collected were analyzed using both descriptive and inferential statistics. Findings showed that: (i) out of 21 Mathematics topics in SSS2 curriculum, there were variations in the number of questions set in 14 (66.7%) topics for 2017 and 11(52.4%) topics for 2018 compared to the number of questions expected to be set. The trend of these variations was the same for the two years (Chi-sq cal = 6.52, Chi-sq cri = 31.41, P>0.05). (ii) the reliability coefficients (r=0.678, and r =0.765) of Ondo State UPE Mathematics objective items for the two years, although moderately good but there was a significant difference in the reliability coefficients (Z-cal = 2.84, Z-cri = 1.96). Based on the findings of this study, it can be concluded that the UPE Mathematics objective items lacks content validity, although reliable, but not comparable over time. It was recommended among others that the psychometric properties of the Ondo State UPE in Mathematics should be ensured before final administration.

Keywords: Reliability, validity, unified promotion examination, mathematics. SSS2

INTRODUCTION

Mathematics is a veritable tool in the understanding of science and technology. Hassan in Odual (2013) observed that mathematics as a subject is universally acknowledged as indispensable to self-reliance and sustainable national development because of its perceived functional utility. Hassan further stressed that any nation seeking to develop a strong level of science and technology must pay proper attention to the teaching and learning of Mathematics. In view of the importance attached to the technological development of any nation, the Nigerian government has not only made Mathematics a compulsory subject in the curriculum of primary and secondary schools (FRN, 2004) but also a pre requisite to the study of Engineering and Science related courses in tertiary institutions (JAMB Brochure, 2018). Despite the importance of Mathematics to national development, a cursory observation shows that there has been a repeated low level of achievement or poor performance of secondary school students in Mathematics in Nigeria and Ondo state in particular. The trend of students’ performance in WASSCE in Ondo state between 2010 and 2018 is shown in Table 1.

Table 1. Ondo State Secondary School Statistics of Performance of Candidates in Mathematics in May/June Wassce 2010-2018

(Source: Ondo state Ministry of Education, Planning, Research & Statistics Department as cited in Osakuade & Ani, 2020)

From Table 1, the performance of students in WASSCE in Mathematics in Ondo state between 2010 and 2018 is far below average in most cases. The percentage of the students that obtained credits (C6-A1) in Mathematics in the stipulated time ranges between 9.5 and 38.7.

Many factors have been identified by previous researchers to be responsible for the poor performance of students in Mathematics. Some of these factors are: shortage of qualified teachers to teach the subject (Ajayi, 2004); lack of interest and commitment  on the part of Mathematics teachers to their profession (Olarewaju, 2009); gender and age factor (Ebere, 2006); parental educational background (Adeogun, 2009); deficiencies in teachers’ preparations (Ogunleye, 2008); inadequate provision of instructional resources that could facilitate Mathematics teaching and learning (Olagunju, 2010); students’ negative attitudes to Mathematics subject (King’aru, 2014) and so on.

Nature of the test items and the examinees characteristics could also result in students’ low performance in Mathematics (Ashikha, 2010). Effective teaching and learning cannot be said to take place in the absence of adequate testing. Bandele (2006) opined that all educational endeavours stand on the three practical concepts of test, measurement and evaluation. The essence of testing in educational process is to generate scores on students’ performance with the intention of making vital decisions on the students (Alonge, 2004). Decisions taken on students would go a long way in determining the future of students as it could mar or make their future. Good quality test can yield accurate data for accurate decisions on the students as poor quality test can yield inaccurate data which can result into wrong decisions on students.

For a test to be of good quality, there are certain psychometric properties it must possessed (Kolawole, 2006). These properties according to Kolawole are validity, reliability and practicality or usability. A test is said to be valid if it measures accurately what purports to measure, nothing but what it purports to measure and all it purports to measure. Lennon in Adebule and Ayodele (2007) defined validity of a test as the extent to which a test does the job for which it is used. Anikweze (2018) is of the view that it is very important for teachers to ascertain the validity of tests given to students because of the following reasons:

1. For accurate and reliable prediction of students’ future success
2. Valid test can serve as a reliable reference points for the promotion of learners to higher class
3. Judgments based on the results from valid tests cannot be contradicted
4. For the objective grading of learners
5. To confirm if desired changes have taken place from the teachers’ instruction
6. To provide a reliable basis for the comparison of teachers’ efforts

Reliability of a test refers to the ability of the test to measure consistently what it purports to measure. Anastasi in Adebule and Ayodele (2007) defined reliability as the consistency of scores obtained by the same person when re-examined with the same test or different sets of equivalent test items or under other varying conditions. Anikweze (2018) also justifies the need for teachers to ascertain the reliability of tests giving to students because of the following reasons:

1. Test with established reliability index makes testing meaningful and dependable
2. The reliability of a test attests to the teachers’ honesty in evaluation
3. Only reliable test produces test scores that can usually discriminate learners’ abilities
4. It is tests with proven reliability that can perform the important function of motivating student learning
5. Teachers can depend on reliable test outcomes as basis for realizing prediction of learners’ future attainments
6. Only reliable teacher-made tests can effectively prepare students for more serious external examinations.

Reliability is a part of validity, reliability is necessary but not sufficient condition for validity. Therefore, it is possible to have highly reliable tests that are useless. For a test to be useful, it must be reliable and valid (Dubois & Stanley, 1979).

For any test to be worthy of being administered to the students, it must possess the three psychometric properties of validity, reliability and practicality. A valid and reliable test should be capable of predicting the future performance of students.

The Ondo State Unified Promotion Examination (UPE) for senior secondary school class 2 students was introduced in 2012/2013 academic session as an intervention measure to reduce the poor performance of students in public examinations, most especially in West African Senior School Certificate Examination (WASSCE) (Oloda, 2012). The essence of the examination is to select credible and competent students to be promoted to SSS3 and register for SSCE. Despite this giant stride of the government in improving the quality of education, the poor performance of students, most especially in Mathematics continues unabated.

The abysmal performance as indicated in (Tables 1) has been a matter of great concern to stake holders in education sector. Stakeholders are worried about the outcome of the SSCE result because of its social and economic implications to the students, family and the society at large. SSCE results affect the society in terms of social welfare, economy, politics, culture, religious value, educational advancement, science and technology, now and distant time to come.

The assumption of this researcher is that the Unified Promotion Examination Mathematics items are not of good quality. It seems the state ministry of education saddled with the responsibility of conducting UPE did not carry out the psychometric properties of their test items. If the psychometric properties of the tests items are poor it is likely to affect students’ achievement negatively. Perhaps this might be one of the reasons why Ondo State UPE is having poor prediction of SSCE. Against this background information, the general question that come to mind is:

1. are the Ondo State UPE Mathematics items of good quality?

Purpose of the Study

The main purpose of this study is to ascertain and compare the quality of the Ondo State Unified Promotion Examination Mathematics objective items from 2017 to 2018. Specifically, the study seeks to ascertain and compare:

1. the validity; and
2. reliability; of the Ondo State UPE Mathematics objective items between 2017 and 2018.

Research Questions

The following research questions are raised to guide the study.

1. What is the content validity of Ondo State UPE Mathematics objective items from 2017 to 2018?
2. How reliable is the Ondo State UPE Mathematics objective items from 2017 to 2018?

Research Hypotheses

The following hypotheses are generated to find solutions to the problem of this study.

1. There is no significance difference in the number of questions set on each topic of the UPE Mathematics objective items for 2017 and 2018
2. There is no significant difference in the reliability coefficients of the UPE Mathematics objective items from 2017 to 2018.

LITERATURE REVIEW

A test is said to be valid if it measures accurately what it is intended to measure. Validity of a test according to Obilor and Obubere (2020) is the extent to which a test measures what it sets out to measure and nothing else. The validity of a test therefore must relate to the purpose for which it is designed. A test designed to measure mathematics achievement for JSS1 may not be valid to measure mathematics achievement for JSS2. The four main types of validity are: face validity, content validity, criterion-related validity (which could be predictive validity or concurrent validity), and construct validity (Ezeh, 2013). Content validity is the most appropriate form of validity for any achievement test (Ezeh, 2013) as is the measure of extent to which the measure both the subject matter, content and instructional objectives designed for a given course.

Some of the factors that affect the validity of a test are: content coverage in terms of sampling and weighing, wording of items, difficulty level, fairness of the test, test-wiseness, reliability level, validity of instructions, use of inappropriate items (Nkemakolam, 1997). If adequate care is taken to address these factors, high validity would be ensured.

Reliability of a test refers to its ability to ensure consistency whatever it sets out to measure. Reliability according to Eze and Onah (2005) is the degree of consistency of two or more measures of the same thing. Reliability therefore is the degree to which a given measurement procedure will yield the same description of that phenomenon if that measurement is repeated severally. Reliability coefficient denotes the extent of reliability of a test. If the reliability coefficient is high, it is an indication that the test will be reliable, but if the reliability coefficient is low, it shows that the test will not be reliable. There are four major methods commonly used in literature to estimate the reliability coefficient of a test. These four methods are: Test-retest, equivalent form, split-half and Kuder-Richardson’s 20/21 formula (Nkemakolam, 1997).

Since a test can be consistent in measuring the wrong thing, then a test can have high reliability with very low validity. Since virtually all factors affecting validity also affect reliability, then a test that is highly valid can have a very reliable degrees of reliability guarantee.

METHODOLOGY

Ex-post facto research design was used for this study. The key features of ex-post factor research design according to Faleye and Rasheed (2020) are the existence of a control or comparison group, the use of intact groups and non-manipulated data. This design is closely connected to this study because it allows analysis to be performed on existing data. In this case, the responses of students to Mathematics objective items in Mathematics of the UPE between 2017 and 2018, constituted the data for this study. In this case, the responses of students to Mathematics multiple–choice objective items were accessed from the existing data base.

The population of the study consisted of all the entire public senior secondary school class two (SSS2) students in Ondo State between 2017 and 2018. Ondo State has eighteen Local Government area Councils with 304 public senior secondary schools (Ondo State Ministry of Education, Research and Statistics Division, 2020)

The sample for the study comprised nine hundred (900) SSS2 students that sat for UPE Mathematics in 2017 and 2018 in Ondo State (i.e 450 sample for 2017 and 450 sample for 2018).The sample was selected from the entire population using multistage and stratified random sampling techniques. Ondo state has three (3) senatorial districts with six (6) Local Government Area Councils in each of the senatorial districts. First stage involved random selection of 1 Local Government Area Council from each of the 3 senatorial districts. This resulted into 3 Local Government Area Councils. The second stage was the random selection of five (5) schools from each of the three Local Government Area Councils using LGA as a stratum. This resulted into 15 schools. The third stage was the random selection of thirty (30) students from each of the 15 selected schools. This amounted to 450 students. Thus, the total of 450 students were selected from each of 2017 and 2018 sets, making a total of 900 sample.

The research instruments used for data collection were the 2017 and 2018 Ondo State UPE Mathematics question papers and Optical Mark Recorder (OMR) sheets of all the students that wrote the Mathematics examinations in the selected schools.

The multiple choice Mathematics questions had already been administered on SSS2 students in their respective schools under the supervision of the representatives of Ondo State Ministry of Education and appointed school supervisors in each school. The demographic data of each of the students were extracted from the OMR sheets to ensure proper coding for computer analysis.

Data obtained were analyzed using both descriptive and inferential statistics such as Chi-square statistics and fisher Z test transformation.

RESULTS

Research Questions 1: What is the content validity of Ondo State UPE Mathematics objective  items from 2017 to 2018?

In order to answer this research question, topics taught in Mathematics for 2017 and 2018 were outlined with the number of week(s) used to teach each of them as extracted from the schools’ scheme of work. The number of questions expected to be set on each of the topics were also listed against the number of questions actually set using table of specification. Details of this analysis is as shown in table2

Table 2. Content Specification of Test Items in Mathematics for Upe 2017 And 2018

From table 2, for the 21 topics meant for UPE Mathematics as specified in the curriculum, in 2017, questions set were below expected number of questions to be set in 5 topics, questions set were above the expected number of questions to be set in 9 topics, questions set were exactly the same with the expected number of questions to be set in just 8 topics.  In 2018, questions actually set were below expected number of questions to be set in 4 topics, number of questions set were above the expected number of questions to be set in 7 topics, number of questions set were exactly the same with the expected number of questions to be set in 10 topics. This implies that out of the 21 Mathematics topics, there were variations in the number of questions set in 14 (66.7%) compared to the number of questions expected to be set in 2017. Also, there were variations in the number of questions set in 11 (52.4%) compared to the number of questions expected to be set in 2018. It can be concluded therefore that the number of questions set for 2017 and 2018 UPE Mathematics were not adequate in relation to the topics (contents) specified in the syllabus. Also, the number of questions set for Mathematics in the two years were not comparable.

Research Question 2.: How reliable is the Ondo State UPE Mathematics objective items from 2017 to 2018?

In order to answer this research question, the reliability coefficients of Ondo State UPE Mathematics objective items from 2017 to 2018 were computed using Kuder Richarson 20 formula and compared.  The result is as shown in table 3.

Table 3. Reliability Coefficients of Ondo State Upme Objective Items For 2017 And 2018

From table 3, the reliability coefficient of Mathematics for 2017 is .675 while that of 2018 is. 765. We can therefore conclude that the reliability coefficients of Ondo State UPME objective items for 2017 and 2018 were moderately good and comparable

Hypothesis 1: There is no significance difference in the number of questions set on each topic of the UPE Mathematics objective items for 2017 and 2018

In order to test this hypothesis, topics taught in Mathematics for 2017 and 2018 were outlined with the number of week(s) used to teach each of them as extracted from the schools’ scheme of work. The number questions actually set for each year were extracted and subjected to chi-square statistics. Details of this analysis is as shown in table 4

Table 4. Chi-Square Table Showing Difference in The Number of Question Sets for Upme Objective Items In 2017 And 2018

From Table 4, The value of Chi-square calculated is 6.52 and the Chi-square cri value is 31.41. since the Chi-square value is less than Chi-square table value, the hypothesis earlier stated that there is no significance difference in the number of questions set per each topic for 2017 and 2018 is hereby upheld at 0.05 level of significance (Chi-sq cal = 6.52, Chi-sq cri = 31.41, P>0.05).  This shows that the number of questions set on each topic for each year are comparable.

Hypothesis Two: There is no significant difference in the reliability coefficients of the UPE Mathematics objective items from 2017 to 2018.

In order to test this hypothesis, the reliability coefficients of the UPE Mathematics objective items of 2017 and 2018 were computed using Kuder Richardson-20 formula. The reliability coefficients obtained for Mathematics for two years were further subjected to fisher Z test statistics to see if significant difference exists in the reliability coefficients of Mathematics for the two years. The result is as shown in table 5

Table 5: Fisher’s Z Test Showing Significant Difference in The Reliability Coefficients of The Upe Mathematics In 2017 And 2018.

From table 5, the value of Z-calculated is greater than the Z-critical value at 0.05 level of significance. The hypothesis earlier stated that there is no significant difference in the reliability coefficients of the UPE Mathematics objective items for 2017 and 2018 is hereby rejected. It showed that there was a significant difference in the reliability coefficients of the UPE Mathematics objective items for 2017 and 2018 (Z-cal = 2.84, Z-cri = 1.96)

DISCUSSION OF FINDINGS

Findings from Research question 1 showed that for the period of two years (2017 to 2018), out of the 21 Mathematics topics for SS2 Mathematics curriculum, there were variations in the number of questions set in 19 (90.5%) compared to the number of questions expected to be set for the UPE Mathematics. It can be concluded therefore that the number of questions actually set for 2017 and 2018 UPE Mathematics were not adequate in relation to the topics (contents) specified in the syllabus.

Finding from hypothesis 1 further showed the trend of these variations was the same for the two years. This shows that the UPE Mathematics lack content validity. as some topics in the periods under review were over emphasized, under emphasized and even ignored. finding is in line with the findings of Oguguo et al (2020) in which some topics in WASSCE multiple choice questions in Financial Accounting from 2016-2018 were over emphasized, under emphasized and even ignored. The number of questions set in both studies did not correspond to the expected number of questions to be set. The plausible reason for this might be that the table of specification was not strictly adhered to

Finding from research question 2 showed that the reliability coefficients of Ondo State UPE Mathematics objective items for 2017 and 2018 are (r=0.678, and r =0.765) respectively. The reliability coefficients although moderately good and comparable. Hypothesis two further revealed that there was a significant difference in the reliability coefficients for the two years (Z-cal = 2.84, Z-cri = 1.96). This study is in line with the earlier study conducted by Chidozle and Okara (2021) who investigated the reliability of Senior Secondary Certificate Examination multiple choice Physic questions from 2016-2018 in Rivers State in which the findings revealed that the internal consistency reliability coefficients of the items of SSCE in Physics question of WAEC between 2016-208 (r= 0.910, 0.860, 0.882) although acceptable but were not comparable

CONCLUSION

Arising from the findings of this study, it can be concluded that the UPE Mathematics lacks content validity. Although meet up with the expected standards or acceptable level of reliability coefficients, but not comparable over time. High reliability of a test cannot guarantee the validity of a test. A test might be reliable but not valid, but if a test is valid, it is reliable.

RECOMMENDATIONS

Based on the findings and conclusion of this study, the following recommendations are made:

1. the psychometric properties of the Ondo State UPE in Mathematics should be ensured before final administration.
2. People that are saddled with the responsibility of conducting this examination in the Ministry of Education should be properly trained from time to time.

ACKNOWLEDGEMENT

The researcher wishes to acknowledge the officials of examination unit, Ondo State Ministry of Education, Akure and also all authors whose articles were made use of in this article.

REFERENCES

1. Adebule, S. O. & Ayodele, C. S. (2007). The reliability and validity of a student anxiety rating scale in mathematics for schools in Nigeria. Pakistan Journal of Social Sciences, 4(1), 78-81.
2. Adeogun, O. B. (2009). The role of socio-economic Variables on students’ achievement in senior secondary school chemistry in Ogun student Nigeria. Journal of Science Teachers Association of Nigeria, 3(1). 20-24.
3. Ajayi N. (2004) A reflection on the Nigeria experience: Education for Today. Journal of the Federal Ministry of Education 3(1),9-12.
4. Alonge, M.F. (2004). Measurement and evaluation in education and psychology. Adedayo printing (Nigeria) Limited, Iworoko, Ado-Ekiti, Ekiti State, Nigeria.
5. Anikweze, C.A. (2018). Measurement and evaluation for teacher education. Shiloh press associates, Onitsha, Anambra State, Nigeria.
6. Bandele, S.O. (2006). Test, measurement and evaluation: The educational tripod. 17^thinaugural lecture of University of Ado-Ekiti, Ekiti State, Nigeria, held Thursday, 9^th March, 2006.
7. Dubois, N. F., & Stanley, R. K. (1979). Educational Psychology and Instruction  Decisions USA, Dorsey press Ltd.
8. Ebere L. (2006). Breaking Gender Barriers on Achievement in STM using Hands, on minds Science: implication for supply of resources science Teacher Association of Nigeria proceeding 47^th annual Conference Education in Nigeria, A research project submitted to FAWE, Nairobi Kenya.
9. Eze, C. O. & Onah, P. C. (2005). Measurement evaluation in education. Enugu: Computer Edge Publishers
10. Ezeh, D. N. (2013). Reliability and validation of tests In B. G. Nworgu (Ed), Educational measurement and evaluation: Theory and practice (pp. 123-135). Nsukka: University Trust Publishers
11. Federal republic of Nigeria, (2004). National policy on Education, 4^th Edition Lagos NERDC press.
12. Kolawole, E.B. (2006). Principles of test construction and administration. Bolabay publications, Ikeja, Lagos Nigeria.
13. Nkemakolam, E. O. (1997). Measurement in education an introduction. Barloz Publishers, Inc. Benin, Nigeria.
14. Obilor, E. I. & Obubere, M. (2020). Development and validation of social studies cognitive achievement test for junior secondary schools in Rivers State. International Academic Journal of Educational Research, 5(3), 1-20.
15. Odual, N. N. (2013). Relationship between mathematical ability and achievement in mathematics among female secondary school students in Bayelsa State, Nigeria. Social and Behavioral Sciences, 106(2013), 2230-2240.
16. Olagunju, A.M. (2000a). Enhancing girls Participation in science in S.O. Ayodele Education Teaching strategies for Nigeria Secondary schools, 187-2004.
17. Olarewaju A.O. (2009) Evaluation of the teaching of integrated science in Lagos and Ondo State Junior secondary school science teacher association of Nigeria 4^th Annual conference proceeding, Heinemann Education books (Nigeria) Plc.
18. Oloda, F.S.S. (2021). Location differential item functioning of test items of 2015 joint senior secondary II Mathematics promotion examination in Ondo State. European Journal of Education Studies, 8(2), 492-503.
19. Oguguo, B.C.E., Agah, J.J., Ene, C.U., Acholonu, V.N., Azubuike, R.N., Okeke, M.A., & Agbo, L.P. (2020), Content validity of West African Examination Council Finacial Accounting questions. International Journal of Learning, Teaching, and Educational Research, 19(6), 161-178.
20. Ossai, A.G., & Nwalado, E.N. (2017). Quality in higher education in Nigeria: perception of global challenges. Nigeria Academic Forum, 25(1), 1596-3306.
21. Osakuade, J. O. & Ani, M. I. (2020). Attitude of Nigerian undergraduates towards statistics and their perceived competence in the use of computer statistical packages for data analysis, Prestige Journal of Education, 3(1), 29-39.