Using Large Language Models to Generate Feedback for Educational Assessments: A Comparative Exploratory Study

Feedback in online environments can be an essential learning tool. However, prior research indicates that poorly designed feedback, particularly when punitive or generic, can undermine student motivation and self-perception (Schartel, 2012). In educational assessment, high-quality feedback can provide learners with information about their performance and guidance for subsequent learning. Hattie and Timperley (2007) conceptualized effective feedback around three key questions: (a) Where am I going? (b) How am I doing? and (c) Where do I go next? From this perspective, useful feedback does more than indicate whether a response is correct; it helps learners understand the nature of their performance and identify appropriate next steps.
In digital assessment environments, feedback can be tailored to learners' responses and performance. Research suggests that such formative, adaptive feedback can support learning, particularly when it is specific, supportive, and prompts learners to reconsider their performance (Narciss et al., 2014; Shute, 2008). However, manually developing individualized feedback requires substantial time and resources (Leibold & Schwarz, 2015).
Multiple-choice assessments offer a useful context for response-specific feedback. Distractors may be designed to represent common errors or misconceptions, allowing feedback to address reasoning plausibly associated with a selected response (Glassey & Bälter, 2020). Response-contingent feedback can provide information beyond simple verification of whether an answer is correct or incorrect by addressing features of the learner's specific response (Shute, 2008). Feedback is also an important component of using multiple-choice assessment for formative learning purposes (Butler, 2018). However, selecting an incorrect response alone does not necessarily reveal the reasoning process that produced it. Consequently, explanations attributed to a particular distractor should be treated cautiously unless the underlying misconception has been independently established. This distinction matters especially when Large Language Models (LLMs) are asked to infer plausible errors associated with distractors.

Recent advances in large language models (LLMs) have created new opportunities to generate individualized assessment feedback

1. Adult Skills Assessment Program. (n.d.). What is ASAP? University of Massachusetts Amherst. https://websites.umass.edu/asap/what-is-the-asap [Google Scholar] [Crossref]

2. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., . . . Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. [Google Scholar] [Crossref]

3. Bulut, O., Beiting-Parrish, M., Casabianca, J. M., Slater, S. C., Jiao, H., Song, D., Ormerod, C. M., Fabiyi, D. G., Ivan, R., Walsh, C., Rios, O., Wilson, J., Yildirim-Erbasli, S. N., Wongvorachan, T., Liu, J. X., Tan, B., & Morilova, P. (2024). The rise of artificial intelligence in educational measurement: Opportunities and ethical challenges. Chinese/English Journal of Educational Measurement and Evaluation, 5(3), Article 3. https://www.ce-jeme.org/journal/vol5/iss3/3/?utm [Google Scholar] [Crossref]

4. Butler, A. C. (2018). Multiple-choice testing in education: Are the best practices for assessment also good for learning? Journal of Applied Research in Memory and Cognition, 7(3), 323–331. [Google Scholar] [Crossref]

5. Dai, W., Tsai, Y. S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., & Chen, G. (2024). Assessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence, 7, Article 100299. https://doi.org/10.1016/j.caeai.2024.100299 [Google Scholar] [Crossref]

6. Federiakin, D., Molerov, D., Zlatkin-Troitschanskaia, O., & Maur, A. (2024). Prompt engineering as a new 21st-century skill. Frontiers in Education, 9, Article 1366434. [Google Scholar] [Crossref]

7. Glassey, R., & Bälter, O. (2020). Put the students to work: Generating questions with constructive feedback. In 2020 IEEE Frontiers in Education Conference (FIE) (pp. 1–8). IEEE. [Google Scholar] [Crossref]

8. Glassey, R., & Bälter, O. (2022). Quality agreement on learner-sourced multiple-choice questions. In Proceedings of the Workshop on Learnersourcing: Student-Generated Content @ Scale 2022 (pp. 45–49). CEUR-WS. [Google Scholar] [Crossref]

9. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487 [Google Scholar] [Crossref]

10. Leibold, N., & Schwarz, L. M. (2015). The art of giving online feedback. The Journal of Effective Teaching, 15(1), 34–46. https://files.eric.ed.gov/fulltext/EJ1060438.pdf [Google Scholar] [Crossref]

11. Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education, 71, 56–76. 10.1016/j.compedu.2013.09.011 [Google Scholar] [Crossref]

12. Schartel, S. A. (2012). Giving feedback—An integral part of education. Best Practice & Research Clinical Anaesthesiology, 26(1), 77–87. 10.1016/j.bpa.2012.02.003 [Google Scholar] [Crossref]

13. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. DOI: 10.3102/0034654307313795 [Google Scholar] [Crossref]

14. Sireci, S. G., Suárez-Álvarez, J., Zenisky, A. L., & Oliveri, M. E. (2024). Evolving educational testing to meet students' needs: Design-in-real-time assessment. Educational Measurement: Issues and Practice, 43(4), 112–118. [Google Scholar] [Crossref]

15. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824–24837. [Google Scholar] [Crossref]

• Assessment of the Role of Artificial Intelligence in Repositioning TVET for Economic Development in Nigeria
• Teachers’ Use of Assure Model Instructional Design on Learners’ Problem Solving Efficacy in Secondary Schools in Bungoma County, Kenya
• “E-Booksan Ang Kaalaman”: Development, Validation, and Utilization of Electronic Book in Academic Performance of Grade 9 Students in Social Studies
• Analyzing EFL University Students’ Academic Speaking Skills Through Self-Recorded Video Presentation
• Major Findings of The Study on Total Quality Management in Teachers’ Education Institutions (TEIs) In Assam – An Evaluative Study