When Six Respondents Make the Model: Extreme-Response Clusters and the Fragility of PLS-SEM Results in Tourism Technology Adoption Research
Authors
Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka, , Malaysia (Malaysia)
Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka, , Malaysia (Malaysia)
Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka, , Malaysia (Malaysia)
Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka, , Malaysia (Malaysia)
Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka, , Malaysia (Malaysia)
Article Information
DOI: 10.47772/IJRISS.2026.1017PSY0073
Subject Category: Psychology
Volume/Issue: 10/17 | Page No: 947-960
Publication Timeline
Submitted: 2026-10-21
Accepted: 2026-10-26
Published: 2026-11-16
Abstract
Tourism technology adoption studies rely heavily on Likert-scale surveys analyzed with partial least squares structural equation modelling (PLS-SEM). Such data are often strongly skewed towards agreement. This study shows how a very small cluster of uniformly negative respondents can manufacture the reliability, validity, structural, predictive and configurational results that researchers report, and how the standard diagnostics fail to detect it. Survey data from 304 Malaysian respondents on the adoption of a solar-powered tourist boat (the ePenambang, Kuala Terengganu) were analyzed with an extended Technology Acceptance Model. Every analysis was run with and without six respondents who disagreed with all 35 items. A dose-response analysis, bootstrap composition analysis, permutation null models and a Monte Carlo simulation (2,000 replications) were used to separate substantive signal from the influence of the extreme cluster. With all 304 cases, the model met every conventional criterion: α = 0.81–0.91, all ten paths significant and R² = 0.85 for intention. Without the six cases (2% of the sample), α fell to between −0.60 and 0.61, nine of ten paths became non-significant and R² fell to 0.24. Not one of 1,000 bootstrap resamples excluded all six cases. Necessary condition effect sizes fell from 0.46–0.72 to 0–0.11, a multigroup difference by education vanished, and fsQCA solutions were reproduced by a randomly permuted outcome. In simulation, adding six all-disagree cases to pure noise produced R² = 0.83, α = 0.84 and HTMT above 0.90 for every construct pair. Standard careless-response indices did not flag the six cases; a multivariate outlier screen did. Researchers, reviewers and editors in tourism should require sensitivity analyses that report results with and without uniformly extreme respondents, report within-cluster correlations, and treat very high R² and reliability in ceiling-affected data as a warning rather than a strength. A ten-point reporting checklist is provided. To our knowledge, this is the first study to document, with real tourism data and simulation, how a small extreme-response cluster propagates through the full modern PLS-SEM toolkit, including NCA, fsQCA, MICOM/MGA, IPMA and PLSpredict.
Keywords
PLS-SEM; influential observations; careless responding; ceiling effects; necessary condition analysis; fsQCA; robustness; sustainable tourism technology
Downloads
References
1. Aguinis, H., Gottfredson, R. K., & Joo, H. (2013). Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16(2), 270–301. [Google Scholar] [Crossref]
2. https://doi.org/10.1177/1094428112470848 [Google Scholar] [Crossref]
3. Ali, F., Rasoolimanesh, S. M., Sarstedt, M., Ringle, C. M., & Ryu, K. (2018). An assessment of the use of partial least squares structural equation modeling (PLS-SEM) in hospitality research. International Journal of Contemporary Hospitality Management, 30(1), 514–538. https://doi.org/10.1108/IJCHM-10-2016-0568 [Google Scholar] [Crossref]
4. Azmi, F. R., Abdullah, A., Musa, H., & Wan Mahmood, W. H. (2020). Perception of food manufacturers towards adoption of halal food supply chain in Malaysia. Journal of Islamic Marketing, 11(3), 571–589. https://doi.org/10.1108/JIMA-12-2018-0236 [Google Scholar] [Crossref]
5. Baumgartner, H., & Steenkamp, J.-B. E. M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143–156. [Google Scholar] [Crossref]
6. https://doi.org/10.1509/jmkr.38.2.143.18840 [Google Scholar] [Crossref]
7. Becker, J.-M., Cheah, J.-H., Gholamzade, R., Ringle, C. M., & Sarstedt, M. (2023). PLS-SEM's most wanted guidance. International Journal of Contemporary Hospitality Management, 35(1), 321–346. https://doi.org/10.1108/IJCHM-04-2022-0474 [Google Scholar] [Crossref]
8. Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19. [Google Scholar] [Crossref]
9. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. [Google Scholar] [Crossref]
10. do Valle, P. O., & Assaker, G. (2016). Using partial least squares structural equation modeling in tourism research: A review of past research and recommendations for future applications. Journal of Travel Research, 55(6), 695–708. https://doi.org/10.1177/0047287515569779 [Google Scholar] [Crossref]
11. Dul, J. (2016). Necessary condition analysis (NCA): Logic and methodology of "necessary but not sufficient" causality. Organizational Research Methods, 19(1), 10–52. [Google Scholar] [Crossref]
12. https://doi.org/10.1177/1094428115584005 [Google Scholar] [Crossref]
13. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall. [Google Scholar] [Crossref]
14. Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2–24. [Google Scholar] [Crossref]
15. Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. [Google Scholar] [Crossref]
16. Henseler, J., Ringle, C. M., & Sarstedt, M. (2016). Testing measurement invariance of composites using partial least squares. International Marketing Review, 33(3), 405–431. [Google Scholar] [Crossref]
17. Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal of e-Collaboration, 11(4), 1–10. [Google Scholar] [Crossref]
18. Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. [Google Scholar] [Crossref]
19. Musa, H., Abdullah, A. R., Othman, M. N., Rashid, N., & Azmi, F. R. (2025). Rethinking the technology acceptance model: A structural analysis of sustainable technology adoption in Malaysian tourism. International Journal of Research and Innovation in Social Science, 9(9), 6125–6139. [Google Scholar] [Crossref]
20. https://doi.org/10.47772/IJRISS.2025.909000500 [Google Scholar] [Crossref]
21. Musa, H., Li, S. C. H., Abas, Z. A., & Mohamad, N. (2016). Adoption factor of mobile marketing: The case of small medium enterprises in Malaysia. International Review of Management and Marketing, 6(7S), 112–115. [Google Scholar] [Crossref]
22. Mustafa, S. K., Musa, H., Abdullah, A. R., & Azmi, F. R. (2017). The influences of consumer preference characteristics towards green product. International Journal of Human and Technology Interaction, 1(1), 43–50. [Google Scholar] [Crossref]
23. Pappas, I. O., & Woodside, A. G. (2021). Fuzzy-set qualitative comparative analysis (fsQCA): Guidelines for research practice in information systems and marketing. International Journal of Information Management, 58, 102310. https://doi.org/10.1016/j.ijinfomgt.2021.102310 [Google Scholar] [Crossref]
24. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. [Google Scholar] [Crossref]
25. Rasoolimanesh, S. M., Ringle, C. M., Sarstedt, M., & Olya, H. (2021). The combined use of symmetric and asymmetric approaches: Partial least squares-structural equation modeling and fuzzy-set qualitative comparative analysis. International Journal of Contemporary Hospitality Management, 33(5), 1571–1592. https://doi.org/10.1108/IJCHM-10-2020-1164 [Google Scholar] [Crossref]
26. Richter, N. F., Schubring, S., Hauff, S., Ringle, C. M., & Sarstedt, M. (2020). When predictors of outcomes are necessary: Guidelines for the combined use of PLS-SEM and NCA. Industrial Management & Data Systems, 120(12), 2243–2267. https://doi.org/10.1108/IMDS-11-2019-0638 [Google Scholar] [Crossref]
27. Ringle, C. M., & Sarstedt, M. (2016). Gain more insight from your PLS-SEM results: The importance-performance map analysis. Industrial Management & Data Systems, 116(9), 1865–1886. [Google Scholar] [Crossref]
28. https://doi.org/10.1108/IMDS-10-2015-0449 [Google Scholar] [Crossref]
29. Rönkkö, M., McIntosh, C. N., Antonakis, J., & Edwards, J. R. (2016). Partial least squares path modeling: Time for some serious second thoughts. Journal of Operations Management, 47–48, 9–27. [Google Scholar] [Crossref]
30. https://doi.org/10.1016/j.jom.2016.05.002 [Google Scholar] [Crossref]
31. Safri, N., & Musa, H. (2022). The adoption factors of low loss microwave transmission glass among green buildings in Malaysia. Journal of Positive School Psychology, 6(3), 4878–4897. [Google Scholar] [Crossref]
32. Sarstedt, M., Ringle, C. M., Cheah, J.-H., Ting, H., Moisescu, O. I., & Radomir, L. (2020). Structural model robustness checks in PLS-SEM. Tourism Economics, 26(4), 531–554. [Google Scholar] [Crossref]
33. https://doi.org/10.1177/1354816618823921 [Google Scholar] [Crossref]
34. Shmueli, G., Sarstedt, M., Hair, J. F., Cheah, J.-H., Ting, H., Vaithilingam, S., & Ringle, C. M. (2019). Predictive model assessment in PLS-SEM: Guidelines for using PLSpredict. European Journal of Marketing, 53(11), 2322–2347. [Google Scholar] [Crossref]
35. Ward, M. K., & Meade, A. W. (2023). Dealing with careless responding in survey data: Prevention, identification, and recommended best practices. Annual Review of Psychology, 74, 577–596. [Google Scholar] [Crossref]
36. https://doi.org/10.1146/annurev-psych-040422-045007 [Google Scholar] [Crossref]
Metrics
Views & Downloads
Similar Articles
- A Comparative Study of Adjustment Level, Parental Income and Academic Achievement of Adolescent Girls
- Increase in STIs among Adolescents in Masvingo District
- A Correlation between SDG 4 (Quality Education) and Emotional Disposition of Teacher Educators in Hyderabad District
- Ear Piercing Behaviour and Self-Esteem among Male University Students in Nigeria
- A Study of Religiosity and Psychological Well-Being