Content Validity of Questionnaire on the Influence of Housing Affordability Factors on the Well-Being of the B40 Group Using the Content Validity Ratio (CVR)

Content Validity of Questionnaire on the Influence of Housing Affordability Factors on the Well-Being of the B40 Group Using the Content Validity Ratio (CVR)

Fatimah, A. W.¹, Kamaruddin M, K, A. ², Lukman, Z. M³., T. M. Zukri⁴, Norashida, S. R⁵., Azizah, I.⁶

^1.3.4,5, Faculty of Applied Social Science, Universiti Sultan Zainal Abidin, Kuala Nerus, 20300, Terengganu, Malaysia

²East Coast Environmental Institute (ESERI), Universiti Sultan Zainal Abidin, Kampus Gong Badak, 21300 Terenggany, Malaysia.

 ⁶Faculty of Built Environment and Surveying University Technology Malaysia

DOI: https://dx.doi.org/10.47772/IJRISS.2023.701070

Received: 07 September 2023; Revised: 26 September 2023; Accepted: 02 October 2023; Published: 03 November 2023

ABSTRACT

This article discusses content validity to evaluate research questionnaires. All items are built based on the adaptation of several management theories and models and literature studies obtained from inside and outside the country. This study aims to examine the content validity of a questionnaire on the influence of housing affordability factors on the well-being of life for the B40 group on the East Coast using the Content Validity Ratio (CVR). Quantitative measurement methods through Content Validity Ratio (CVR) and Content Validity Index (CVI) have been used to assess whether the items need to be retained or dropped. Twelve experts were selected to evaluate and confirm the content of the questionnaire. This questionnaire involves things with key factors. The study results show that this questionnaire has good content validity and proves that this questionnaire has excellent potential to be used to identify the influence of housing affordability factors on the well-being of life for the B40 group in peninsular Malaysia using the content validity ratio (CVR).

Keywords: Content Validity Ratio (CVR), Content Validity Index (CVI), Housing Affordability Level, Mental Health, and Well-Being.

INTRODUCTION

The rapid growth in Terengganu and economic stability convinces investors to expand their business, especially in Kuala Terengganu. Parallel to this economic development, house prices in the market are increasing occasionally. This situation causes people to be unable to buy and own their own houses, especially low-income B40 [4] [16] [25]. This study focuses on the level of home ownership ability for B40 in Kuala Terengganu. A questionnaire was built based on the adaptation of several management theories and models as well as literature studies obtained from inside and outside the country to see the level of housing affordability, mental health, and well-being for the B40 group on the East Coast of Peninsular Malaysia. Content validity is done to confirm the accuracy of the questionnaire items to measure what is to be measured and measure the content of a field of study [24]. Content validity also examines how much the instrument’s content aligns with the measurement scale [6]. The instrument’s validity should also be based on an exploratory effort against literature studies on certain variables to be studied and confirmed through expert validity review [17]. For that, the expert assessment panel’s view on the variable to be measured is very necessary, especially in providing input for improving the instrument [6] [14].

Through expert validity, comments and suggestions are useful in modifying and further strengthening the questionnaire [15]. Validity generally means the ability to measure what it is intended to measure and is one of the essential aspects of an instrument. Without satisfactory validity, the characteristics of an instrument will be compromised even if the instrument has excellent reliability. The validation process will ensure the instrument has defensible, accurate, appropriate, meaningful, and valuable properties. Therefore, the validation process must be done accurately to develop a valid tool. Validity is generally divided into several categories with different purposes and goals: face, content, criterion, and construct. Local and foreign researchers have widely used the CVR method as an initial step in the instrument preparation process.

LITERATURE REVIEW

The questionnaire on the housing affordability, mental health, and well-being level for the B40 group on the East Coast of the Peninsula uses the content validity ratio (CVR) and the content validity index (CVI). Quantitative analysis of the content validity of this research questionnaire was carried out through the Lawshe Model [21], which uses the technique of determining the content validity ratio (CVR) and the content validity for each item (CVI).

The appropriateness of using the CVR and CVI techniques is believed to be beneficial in helping the researcher filter items empirically on the instrument with quantitative procedures to ensure that each item truly represents the content of the construct domain, then decide whether to retain or drop the item. The strengths of the CVR method are more transparent and directed, user-friendly, simple computer calculations, the availability of a critical cut-off determination table and the emphasis on the issue of expert agreement up to the level of items considered “very important” or essential [17]. The content validity technique through CVR is described as a classical measurement but very practical from the aspect of time and cost, in addition to being simple and fast [19]. Therefore, the technique of determining the content validity of CVR instruments is very popular in most studies [8] [5] [19] [15] [10] [27] [19]. The procedure for carrying out CVR analysis requires a panel of experts to be appointed to assess in deciding whether a measurement item is very important to maintain based on the theoretical operationalisation of the construct.

The more the item scale represents the domain of a concept being measured, the higher the content validity [24]. The procedure begins with determining and selecting a panel of experts who have recognised expertise in the variables being studied [5]. To facilitate evaluation, the use of a three-point scale for each item, namely (1) Essential (very important), (2) Useful but not essential (helpful but not important) and (3) Not necessary (not necessary) [20]. Content validity was determined based on the professional judgment of a panel of experts by asking for their views and comments on the compatibility of constructs, items, and measurement scales on the instrument. If more than the number of experts involved evaluate the item as very important, then the item is considered to have met content validity [20].

The CVR value is from -1 to +1, where a value close to +1 indicates that experts agree that the item is essential in content validity. A CVR value less than zero (CVR) indicates that half of the panel of experts believe the measurement items meet content validity. If more than the number of experts involved evaluate the item as very important, then the item is considered to have met content validity [20]. However, the minimum CVR value for the total study validity expert panel of 12 experts must be at the CVR Critical value of 0.669 [5]. Furthermore, for the determination of the minimum value of the content validity index (CVI) for each item, it is set not less than 0.78 [23], and the CVI value for the newly developed instrument is ≥ 0.80 [11]. Therefore, the formula for determining the value of CVR and CVI is explained below.

Fig, 1 CVR and CVI formula

METHODOLOGY

Experts are people who have expertise and skills in a particular field. The expert’s function is to examine each item seriously before deciding whether to eliminate the item that has been suggested. Therefore, content validity analysis of the Content Validity Ratio (CVR) and Instrument Content Validity Index (CVI) using the Lawshe Model [20] was carried out involving the consent of 12 experts, as in Table 1.1. In this regard, several opinions determine the number of experts. According to [20], expert panels of at least four people in UniSZA were suggested to carry out content validity using CVR.

Whereas [1] stated that eight to 16 expert panels are needed. For [2], the expert panel should consist of two to 20 people. In this regard, a total of 12 experts were involved in conducting a review of the validity of the study content. Next, after the draft questionnaire was reviewed and improved, the instrument was given to a panel of experts for evaluation [19]. The determination of the CVR cut-off point value in this study is based on the determination that the minimum value of Critical Values for Lawshe’s CVR corresponds to the total validity expert panel of 12 people, at a value of 0.667 [20]. Meanwhile, the minimum indicator of the Content Validity Index (CVI) for each item is set at a value not less than 0.78 [15] [17] and preferably at a value greater than 0.80 [12].

RESULTS AND DISCUSSION

Table 1.1 List of Study Expert Panel

In this study, the number of items for the questionnaire was 68, confirmed in content based on the evaluation of 12 experts using the CVR and CVI methods based on the proposed options [20]. The results analysis of the study through CVR and CVI found that the range of each finding showed the value of CVR and CVI for the items of the ability level factor was (0.50 to 1.00) and (0.92 to 1.00). For the e-housing need factor items, the CVR value (0.50 to 1.00) while for the CVI item (0.92 to 1.00). For the demographic factor items CVR value (0.50) and CVI value (0.75). Meanwhile, the economic factor items CVR value (0.83) and CVI value (0.92).

Furthermore, the environmental factor items CVR value (0.83) while the CVI value is (0.92). For social factors, the CVR value is (0.83-1.00) while the CVI value is (0.92-1.00). Next, the CVR value for the housing market factor, the CVR value is (0.83) while the CVI is (0.92). For the mental health factor, the value of CVR (0.50-1.00) while CVI (0.75-1.00).  Finally, the life well-being factor for the CVR value (0.33-0.50) while the CVI value (0.67-0.75). Therefore, this finding confirms that the instrument used in this study has met the criteria of content validity [21]. For that, the details of the analysis of CVR and CVI findings for each item are as in Table 1.2, Table 1.3, Table 1.4, and Table 1.5. Table 1.6, Table 1.7, Table 1.8, and Table 1.9.

Table 1.2: Values of CVR and CVI Factors of Ability Level

Based on Table 1.2, each item of ability level factor is accepted and some items are reassessed. The first item for the ability level factor for high cost of living is accepted, the second item i.e. the house price/rent is too high is accepted, the 3rd item high monthly payment of more than 30% of income is also accepted, item 4 i.e. low income or salary modified through CVR and CVI evaluation, item 5 is it is difficult to obtain financial financing facilities needs to be modified and item 6 is provision of 10% housing deposit needs to be modified, item 7 is unaffordable loan eligibility is accepted, item 8 is suitable location (urban /rural) is accepted and item 9 is employment (permanent/ contract etc.) is accepted

Table 1.3: CVR and CVI Values of Housing Need Factors

Based on table 1.3, each item of housing need level factors is accepted and some items are reassessed. The first item for the lifestyle and infrastructure offered by the developer factors is accepted, the second item i.e. number of house units offered is accepted, the 3rd item security aspects affect house prices (gated community) is re-evaluated, item 4 i.e. The number of bedrooms/bathrooms affects the price of the house is accepted, item 5 is design and construction or finishing materials used also accepted, item 6 is basic amenities are far from the location is accepted and item 7 is the location of the housing estate is not suitable (disturbing physical and mental health) is accepted.

Table 1.4: CVR and CVI Values of Demographic Factors

Based on table 1.4, each item of demographic factors is accepted and some items are re-evaluated.  The first item for type of job not comparable to income is reassessed, the second item i.e. lack of work skills is reassessed, the 3rd item rural-urban migration population factor is reassessed, item 4 i.e. household increase is reassessed, item 5 is purchasing power for limited housing applications also reassessed, item 6 is low income must have reassessed, item 7 is high household costs and expenses is reassessed and item 8 education is not in line with income is reassessed.

Table 1.5: Value of CVR and CVI of Economic Factors

Based on table 1.5, each item of economics factors is accepted. The first item for the funding application process is difficult is accepted. the second item i.e. house prices are rising is accepted, the 3rd item high cost of housing (stamp duty, lawyer’s fees) is accepted, item 4 i.e. economic instability affects home purchases is accepted, item 5 instalment too high for a suitable and desirable house (low purchasing power) is accepted. item 6 is the housing supply offered is unbalanced is accepted. item 7 is difficult financing process and provision of deposit money is accepted. Item 8 is high interest rates and mortgages also accepted.  item 9 is long payback period and also accepted. The last one for this factor is item 10, high cost of living and health also accepted.

Table 1.6: Value of CVR and CVI of Environmental Factors

Based on table 1.6, each item of environmental factor is accepted. The first item is the location of the residence is far from the city. Second item is the size of the small residential property cannot accommodate a large family is accepted. The third item is the quality of housing offered affects house prices also accepted. Item 4 i.e. services and comfort of the neighbourhood is accepted. Item 5 the location of the residence is far from the workplace also accepted, item 6 is type of residence offered is accepted, item 7 is payment of high management charges also accepted.

Table 1.7: CVR and CVI values of Social Factors

Based on table 1.7, each item of social factors item

is accepted.  The first item is housing far from social facilities (surau, mosque or hall) is accepted. The second item management charge on the home environment also accepted.  The 3^rd item high health and medical costs is accepted. Item 4 i.e.  the quality of neighbourhood relationships, e.g. social relationships is accepted. Item 5 is The neighbourhood offered is comfortable and safe is accepted. Item 6 is High maintenance costs because there is always vandalism and social symptoms such as drugs, theft, etc. is accepted.

Table 1.8: CVR and CVI Values of Housing Market

Based on table 1.8, each item of housing market is accepted.  The first item for policy and political changes in housing is accepted. The second item speculation in the housing market is accepted. The 3^rd item speculation in the housing market is accepted, item 4 i.e. home purchase by foreigners is accepted., item 5 is housing policies vary between states is accepted. Item 6 is Low and medium-cost houses are not enough being accepted. Item 7, affordable housing prices are high and unaffordable is accepted. The item 8 is The housing application process is difficult.

Table 1.9: CVR Value of Mental Health Factors

Based on table 1.9, each item of Mental Health Factors is accepted and reassessed. The first item stress with the constraints of financial provision is accepted, the second item i.e. disrupted social relationships (depression etc.) is accepted, the third item is emotions are easily disturbed; for example, is quick to anger and the item is reassessed. Item 4 is physical and mental health is disturbed and need to reassessed. Item 5 is dealing with debt (house, car etc.) and also this item need to reassessed. Lastly is item 6 worrying about physical diseases such as diabetes, high blood pressure, etc. causes health costs to also increase need to reassessed.

Table 1.10: CVR Value of the Well-Being Factor

Based on Table 1.10, each item of Well-Being Factor is reassessed. The first item for the Well-Being factor is harmonious family relationships and need to reassessed. The second item i.e. stable family economy is reassessed, the 3rd item good family health is reassessed, item 4 i.e. family safety is guaranteed through CVR and CVI is reassessed, item 5 is family and community also have good cooperation needs to be modified and item 6 is the family has strong religious beliefs also need reassessed, item 7 is comfortable housing and a good environment and also reassessed.

Next, based on the feedback of the expert panel detailed in Table 1.11, the process of modification and refinement of the questionnaire was carried out, which involved the appropriateness of items, modification of statements, modification of sentence order, correction of technical errors such as spelling and language, modification of the measurement scale to produce meaningful measurement as well as sorting items, especially for items that are identified as overlapping with the same meaning. For that, the details of the analysis of the research findings for each item are as in Table 1.2, Table 1.3, Table 1.4, a   Table 1.5. Table 1.6, Table 1.7, Table 1.8, Table 1.9, and Table 1.10

Table 1.11: Analysis of Expert Panel Responses to Study Instruments

CONCLUSIONS

Determination of CVR and CVI values because of the evaluation of 12 experts, improvements and modifications as suggested by the 12-expert panel were also carried out. Therefore, the respective items of the questionnaire for the ability level factor (9 items), housing need factor (7 items), demographic factor (8 items), economic factor (10 items), environmental factor (7 items), social factors (6 items), housing market factors (8 items), mental health factors (6 items), and the last well-being of life (7 items) were maintained after the improvement and modification of the questionnaire. Articles were reviewed after final agreement with the panel and supervisor. This includes improvements, modifications, and language corrections in line with what has been discussed in the framework. The item review process takes about a month to complete. After that, it is submitted to the supervisory committee to analyse the proposed modification to ensure that it aligns with the objectives in this field of study. Therefore, the conclusion is that as many as 68 items have been successful in conducting this study. Further, these items will be refined based on the expert panel’s advice before being further examined in the pilot study instrument.

ACKNOWLEDGEMENTS

thanks to the expert panel for this study and UniSZA for cooperation and complete support.

REFERENCES

1. Allahyari, T., Hassanzadeh, R. N., Khosravi, Y., & Zayeri, F. (2011). Development and evaluation of a new questionnaire for rating cognitive failures at work.
2. Ali, N. A., Tretiakov, A., & Whiddett, D. (2014). A content validity study for a knowledge management systems success model in healthcare. JITTA: Journal of Information Technology Theory and Application, 15(2), 21.
3. Alwayi, Z. M., Embong, R., & Hashim, H. A. (2021). Perancangan Pengajaran Guru dalam Pembelajaran dan Pemudahcaraan: Satu Kajian Rintis [Teacher Teaching Planning in Learning and Facilitating: A Pilot Study]. Asian Journal of Civilizational Studies (AJOCS), 3(1), 11-18.
4. Aziz, A., Anwar, M. M., Abdo, H. G., Almohamad, H., Al Dughairi, A. A., & AlMutiry, M. (2023). Proximity to Neighborhood Services and Property Values in Urban Area: An Evaluation through the Hedonic Pricing Model, Land, 12(4), 1-12.
5. Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s content validity ratio: revisiting the original calculation methods. Measurement and evaluation in counselling and development, 47(1), 79-86.
6. Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. (2018). Introduction to research in education. Cengage Learning.
7. Amatan, M. A., Han, C. G. K., & Pang, V. (2021). Kesahan Kandungan Soal Selidik Faktor Konteks, input dan Proses Terhadap Penerimaan Pelaksanaan Elemen Pendidikan STEM Dalam Pengajaran dan Pembelajaran Guru Menggunakan Nisbah Kesahan Kandungan (CVR): Content Validity for Context, Input and Process Questionnaire on the Acceptance of the Implementation of STEM Education Elements in Teacher Teaching and Learning using Content Validity Ratio (CVR). International Journal of Advanced Research in Future Ready Learning and Education, 23(1), 10-22.
8. Aziz, A. A., Yusof, Z. M., Mokhtar, U. A., & Jambari, D. I. (2018). A conceptual model for adopting electronic document and records management systems in the Malaysian public sector. International Journal on Advanced Science, Engineering, and Information Technology, 8(4), 1191-1197.
9. Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s content validity ratio: revisiting the original calculation methods. Measurement and evaluation in counselling and development, 47(1), 79-86.
10. Ahmad, S., & Silong, A. D. (1996). Keberkesanan penilaian prestasi kerja di kalangan kakitangan perkeranian. Pertanika Journal Social Sciences & Humanities, 4(1), 55-63.
11. Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approach. Sage publications.
12. Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied nursing research, 5(4), 194-197.
13. Effendi, M., Matore, E. M., Idris, H., Rahman, N. A., & Khairani, A. Z. (2017). Kesahan kandungan pakar instrumen IKBAR bagi pengukuran AQ menggunakan Nisbah Kesahan Kandungan.
14. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate education research (Vol. 7, p. 429). New York: McGraw-Hill.
15. Ghazali, N., Nordin, M. S., Hashim, S., & Hussein, S. (2017, October). Measuring content validity: Students’ self-efficacy and meaningful learning in massive open online course (MOOC) scale. In International Conference on Education in Muslim Society (ICEMS 2017) (pp. 128-133). Atlantis Press.
16. Hassan, M. M., Ahmad, N., & Hashim, A. H. (2021). A review on housing affordability in Malaysia: are we doing fine. Malaysian Journal of Consumer and Family Economics, 26(2021), 181-206.
17. Jackson, S. E., Joshi, A., & Erhardt, N. L. (2003). Recent research on team and organisational diversity: SWOT analysis and implications. Journal of Management, 29(6), 801-830.
18. Johnston, P., & Wilkinson, K. (2009, November). Enhancing validity of critical tasks selected for college and university program portfolios. In National Forum of Teacher Education Journal (Vol. 19, No. 3, pp. 1-6).
19. Lewis, B. R., Templeton, G. F., & Byrd, T. A. (2005). A methodology for construct development in MIS research. European Journal of Information Systems, 14(4), 388-400.
20. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575.
21. Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382-386.
22. Lindell, M. K., & Brandt, C. J. (1999). Assessing interrater agreement on the job relevance of a test: A comparison of CVI, T, rWG (J)}, and r* WG (J)} indexes. Journal of Applied Psychology, 84(4), 640.
23. Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in nursing & health, 30(4), 459-467.
24. Sekaran, U., & Bougie, R. (2016). Research methods for business: A skill building approach. New York: john wiley & sons.
25. Mariadas, P. A., Abdullah, H., & Abdullah, N. (2019). Factors influencing the first home purchase decision of middle-income earners (M40) in Selangor, Malaysia. e-BANGI, 16(1), 1-11.
26. Norashida S, R.., Norshahira, O., & Lukman, Z. M. (2021). Content Validity of Drug Addiction Recovery Test Instruments Using Content Validity Ratio (CVR) Method. Journal of Drug Delivery and Therapeutics, 11(4), 24-29.
27. Surip, N. A., Razak, K. A., & Tamuri, A. H. (2019). Kesahan dan kebolehpercayaan Instrumen Pengurusan Pendidikan Dakwah Di Sekolah. Journal of Islamic, 4(19), 108-118.
28. Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A. R. (2015). Design and implementation content validity study: developing an instrument for measuring patient-centered communication. Journal of Caring Sciences, 4(2),