Judgement in Measurement and Analysis

True measuring scales behave in the same way as the real-life things that they are measuring. They also permit an estimate of the level of error in making a measurement, through calibration. Where these two characteristics are not present, then a purported measurement is not a true one. Numbering is not the same as measuring. Errors in measurement also propagate in calculations, but without a true measure we cannot tell how. There is no technical or statistical solution to this. When judging the trustworthiness of the findings from a piece of research, the quality of the measurement used is an important criterion. Without this knowledge, research cannot be trusted. Therefore, analyses and the trust placed in them must be based on appropriate judgement.

Data quality, error propagation, fake measures, inferential statistics

1. Berka, K (1983) Measurement: its concepts, theories and problems, London: Reidel [Google Scholar] [Crossref]

2. Cumming, G. (2014) The new statistics: why and how, Psychological Science, 25, 1, 7-2, https://doi.org/10.1177/09567976135049669 [Google Scholar] [Crossref]

3. De Vrieze, J. (2019) What science reporters should know about meta-analyses, What science reporters should know about meta-analyses before covering them | by Jop de Vrieze | Medium [Google Scholar] [Crossref]

4. Deeks J., Dinnes J., D'Amico R., Sowden A., Sakarovitch C., Song F. et al. (2003) Evaluating non- randomised intervention studies, Technology Assessment, 7, 27, iii–x, 1–173, doi: 10.3310/hta7270 [Google Scholar] [Crossref]

5. Farrington, D., Gottfredson, D., Sherman, L. and Walsh, B. (2002) Evidence-based crime prevention, London: Routledge [Google Scholar] [Crossref]

6. Fielding, J. and Gilbert, N. (2000) Understanding social statistics, London: Sage Gorard, S. (2006) Towards a judgement-based statistical analysis, British Journal of Sociology of Education, 27, 1, 67-80, https://doi.org/10.1080/01425690500376663 [Google Scholar] [Crossref]

7. Gorard, S. (2010a) Measuring is more than assigning numbers, pp.389-408 in Walford, G., Tucker, E. and Viswanathan, M. (Eds.) Sage Handbook of Measurement, Los Angeles: SAGE [Google Scholar] [Crossref]

8. Gorard, S. (2010b) Serious doubts about school effectiveness, British Educational Research Journal, 36, 5, 735-766, https://doi.org/10.1080/01411920903144251 [Google Scholar] [Crossref]

10. Gorard, S. (2013) The propagation of errors in experimental data analysis: a comparison of pre- and post- test designs, International Journal of Research and Method in Education, 36, 4, 372-385, http://dx.doi.org/10.1080/1743727X.2012.741117 [Google Scholar] [Crossref]

11. Gorard, S. (2020) Handling missing data in numeric analyses, International Journal of Social Research Methods, 23, 6, 651-660, https://www.tandfonline.com/doi/full/10.1080/13645579.2020.1729974 [Google Scholar] [Crossref]

12. Gorard, S. (2021) How to make sense of statistics: Everything you need to know about using numbers in social science, London: SAGE [Google Scholar] [Crossref]

13. Gorard, S. (2024) Judging the relative trustworthiness of research results: how to do it and why it matters, Review of Education, 12, 1, https://doi.org/10.1002/rev3.3448 [Google Scholar] [Crossref]

14. Gorard, S. (2025) Mixing methods is still wrong, in Morrison, K. and See, BH (Eds) Handbook of Mixed Methods, Sage [Google Scholar] [Crossref]

15. Gorard, S. and Chen, W. (2025) What is the evidence on research-informed education?, Chapter 2, pp.55- 76 in Wyse, D., Baumfield, V., Mockler, N and Reardon, M. (Eds.) The BERA/SAGE Handbook of Research-Informed Education Practice and Policy [Google Scholar] [Crossref]

16. Gorard, S. and See, BH. (2009) The impact of SES on participation and attainment in science, Studies in Science Education, 45, 1, 93-129, https://doi.org/10.1080/03057260802681821 [Google Scholar] [Crossref]

17. Nickerson, R. (2000) Null hypothesis significance testing: a review of an old and continuing controversy, Psychological Methods, 5, 2, 241-301, 10.1037/1082-989x.5.2.241 [Google Scholar] [Crossref]

18. Nunnally. J. (1975) Psychometric theory 25 years ago and now, Educational Researcher, 4, 7, 7-21 [Google Scholar] [Crossref]

19. Prandy, K. (2002) Measuring quantities: the qualitative foundation of quantity, Building Research Capacity, 2, 2-3 [Google Scholar] [Crossref]

20. Rozeboom, W. (1960) The fallacy of the null hypothesis significance test, Psychological Bulletin, 57, 416-428 [Google Scholar] [Crossref]

21. Smith, E., Gorard, S., Morris, R., Perry, T. and Pilgrim-Brown, J. (2025) Then and now: Twenty years of Education research methods use in the UK, British Educational Research Journal, 51, 1, 2347-2400, Does school matter for children's cognitive and non‐cognitive learning? Findings from a natural experiment in Pakistan and India - Siddiqui - British Educational Research Journal - Wiley Online Library [Google Scholar] [Crossref]

• The Impact of Ownership Structure on Dividend Payout Policy of Listed Plantation Companies in Sri Lanka
• Urban Sustainability in North-East India: A Study through the lens of NER-SDG index
• Performance Assessment of Predictive Forecasting Techniques for Enhancing Hospital Supply Chain Efficiency in Healthcare Logistics
• The Fractured Self in Julian Barnes' Postmodern Fiction: Identity Crisis and Deflation in Metroland and the Sense of an Ending
• Impact of Flood on the Employment, Labour Productivity and Migration of Agricultural Labour in North Bihar