Designing A Web-Based Questionnaire with Incentive Verification and Behavioural Screening Mechanisms to Improve Survey Data Quality

Web-based survey platforms such as Google Forms and SurveyMonkey have become widely adopted in academic and applied research due to their efficiency, accessibility, and low operational cost. However, open online survey environments introduce challenges related to respondent inattentiveness, satisficing behaviours, and data authenticity, which can compromise the reliability and validity of collected data. To address these issues, this study aims to design, develop, and evaluate a web-based questionnaire framework that integrates intervention techniques to enhance response quality. Specifically, the study identifies three practical intervention mechanisms—straightlining detection, consistency or logic checks, and attention check questions—and embeds them within an incentive-based survey system that securely collects respondent banking details for token distribution. The methodology comprises four phases: identification of intervention techniques through literature review; system design and proof of concept; data collection using the developed platform; and usability evaluation conducted with 11 survey instrument developers. Usability was assessed across five dimensions: interface usability, navigation, clarity, satisfaction, and overall experience. Results indicated strong usability performance, with average scores exceeding 80% across all dimensions. Clarity of instructions achieved the highest score (87%), while efficiency scored 80%, suggesting minor areas for optimization. The findings demonstrate that embedded intervention techniques do not detract from user experience while supporting attentive participation. Future work should focus on validating the effectiveness of these intervention techniques in detecting low-quality responses across diverse respondent populations and survey contexts, as well as exploring automation of real-time response filtering to further enhance data integrity.

Web-based, interventions, questionnaire, quality responses, survey

1. Andrade, C. (2020). The limitations of online surveys. Indian Journal of Psychological Medicine, 42(6), 575–576. https://doi.org/10.1177/0253717620957496 [Google Scholar] [Crossref]

2. Aust, F., Diedenhofen, B., Ullrich, S., & Musch, J. (2023). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 55(1), 123–141. [Google Scholar] [Crossref]

3. https://doi.org/10.3758/s13428-012-0265-2 [Google Scholar] [Crossref]

4. Bloy, L., Resheff, Y., Kluger, A., & Malovicki-Yaffe, N. (2025). Identifying careless survey respondents through machine learning using responses to a gibberish scale. Advances in Methods and Practices in Psychological Science, 8(4). https://doi.org/10.1177/25152459251378420 [Google Scholar] [Crossref]

5. Calderon Vriesema, C., & Gehlbach, H. (2021). Assessing survey satisficing: The impact of unmotivated questionnaire responding on data quality. Educational Researcher, 50(9), 607–617. https://doi.org/10.3102/0013189X211040054 [Google Scholar] [Crossref]

6. Caven, I., Yang, Z., Saragosa, M., Lunsky, Y., Cameron, J., Newman, K., Bookey-Bassett, S., Hahn-Goldberg, S., & Okrainec, K. (2025). So you want to conduct an online survey? Strategies for identifying and eliminating fraudulent responses. International Journal of Integrated Care, 25(S1), Article 242. https://doi.org/10.5334/ijic.NACIC24142 [Google Scholar] [Crossref]

7. Clement, S. L., Severin-Nielsen, M. K., & Shamshiri-Petersen, D. (2023). Satisficing behaviour in web surveys: Results from a comparison of web and paper mode across four national survey experiments. Survey Methods: Insights from the Field, 1. https://doi.org/10.13094/SMIF-2023-00007 [Google Scholar] [Crossref]

8. Goldammer, P., Annen, H., Stöckli, P. L., & Jonas, K. (2020). Careless responding in questionnaire measures: Detection, impact, and remedies. The Leadership Quarterly, 31(4), Article 101384. https://doi.org/10.1016/j.leaqua.2020.101384 [Google Scholar] [Crossref]

9. Höhne, J. K., Revilla, M., & Schlosser, S. (2020). Motion instructions in surveys: Compliance, acceleration, and response quality. International Journal of Market Research, 62(1), 43–57. https://doi.org/10.1177/1470785319887889 [Google Scholar] [Crossref]

10. Huang, J. L., Liu, M., & Bowling, N. A. (2015). Insufficient effort responding: Examining an insidious confound in survey data. Journal of Applied Psychology, 100(3), 828–845. [Google Scholar] [Crossref]

11. https://doi.org/10.1037/a0038510 [Google Scholar] [Crossref]

12. International Organization for Standardization. (2022). Information security, cybersecurity and privacy protection—Information security management systems—Requirements (ISO/IEC Standard No. 27001:2022). https://www.iso.org/standard/82875.html [Google Scholar] [Crossref]

13. Jin, K. Y., & Chiu, M. M. (2024). Modeling insufficient effort responses in mixed-worded scales. Behavior Research Methods, 56(3), 2260–2272. https://doi.org/10.3758/s13428-023-02146-w [Google Scholar] [Crossref]

14. Johnson, M. S., Adams, V. M., & Byrne, J. (2024). Addressing fraudulent responses in online surveys: Insights from a web-based participatory mapping study. People and Nature, 6(1), 147–164. https://doi.org/10.1002/pan3.10557 [Google Scholar] [Crossref]

15. Kane, J. V., Velez, Y. R., & Barabas, J. (2023). Analyze the attentive and bypass bias: Mock vignette checks in survey experiments. Political Science Research and Methods, 11(2), 293–310. [Google Scholar] [Crossref]

16. https://doi.org/10.1017/psrm.2023.3 [Google Scholar] [Crossref]

17. Ladini, R. (2022). Assessing general attentiveness to online panel surveys: The use of instructional manipulation checks. International Journal of Social Research Methodology, 25(2), 233–246. https://doi.org/10.1080/13645579.2021.1877948 [Google Scholar] [Crossref]

18. Lorincz, J., Barišić, K., & Vlahović, V. (2026). Usability testing and the System Usability Scale effectiveness assessment on different sensing devices of prototype and live web system counterpart. Sensors, 26(2), 679. https://doi.org/10.3390/s26020679 [Google Scholar] [Crossref]

19. Muszyński, M. (2023). Attention checks and how to use them: Review and practical recommendations. Ask: Research and Methods, 32, 3–38. https://doi.org/10.18061/ask.v32i1.0001 [Google Scholar] [Crossref]

20. Muszynski, M., & Jabkowski, P. (2025). Comparing response behaviours between face-to-face and self-completion modes [Conference presentation]. City St Georges, ESS and NatCen Webinar Series. https://natcen.ac.uk/events/comparing-response-behaviours-between-face-face-and-self-completion-modes [Google Scholar] [Crossref]

21. Ozaki, K. (2024). Detecting inattentive respondents by machine learning: A generic technique that substitutes for the directed questions scale and compensates for its shortcomings. Behavior Research Methods, 56(7), 7059–7078. https://doi.org/10.3758/s13428-024-02407-2 [Google Scholar] [Crossref]

22. Raynes, S., & Marlar, J. (2024). Data quality issues with opt-in panels: Part 2. Gallup News. https://news.gallup.com/opinion/methodology/654494/data-quality-issues-opt-panels-part-two.aspx [Google Scholar] [Crossref]

23. Reuning, K., & Plutzer, E. (2020). Valid vs. invalid straightlining: The complex relationship between straightlining and data quality. Survey Research Methods, 14(5), 439–459. [Google Scholar] [Crossref]

24. https://doi.org/10.18148/srm/2020.v14i5.7641 [Google Scholar] [Crossref]

25. Revilla, M., & Höhne, J. K. (2020). Comparing the participation of millennials and older age cohorts in the cross-national online survey panel and the German Internet Panel. Survey Research Methods, 14(5), 499–513. https://doi.org/10.18148/srm/2020.v14i5.7619 [Google Scholar] [Crossref]

26. Shamon, H., & Berning, C. C. (2020). Attention check items and instructions in online surveys with incentivized and non-incentivized samples: Boon or bane for data quality? Survey Research Methods, 14(1), 55–77. https://doi.org/10.18148/srm/2020.v14i1.7374 [Google Scholar] [Crossref]

27. Silber, H., Roßmann, J., & Gummer, T. (2022). The issue of noncompliance in attention check questions: False positives in instructed response items. Field Methods, 34(4), 346–360. [Google Scholar] [Crossref]

28. https://doi.org/10.1177/1525822X221115830 [Google Scholar] [Crossref]

29. Stanley, M., Roycroft, J., Amaya, A., Dever, J. A., & Srivastav, A. (2020). The effectiveness of incentives on completion rates, data quality, and nonresponse bias in a probability-based internet panel survey. Field Methods, 32, 159–179. https://doi.org/10.1177/1525822x20901802 [Google Scholar] [Crossref]

30. Sturgis, P., & Brunton-Smith, I. (2023). Personality and survey satisficing. Public Opinion Quarterly, 87(3), 689–710. https://doi.org/10.1093/poq/nfad029 [Google Scholar] [Crossref]

31. Ward, M. K., & Meade, A. W. (2023). Dealing with careless responding in survey data: Prevention, identification, and recommended best practices. Annual Review of Psychology, 74, 577–596. https://doi.org/10.1146/annurev-psych-040422-045007 [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet