The Effect of Generative AI (ChatGPT) on EFL Learners’ Academic Writing Performance: A Preliminary Meta-Analysis

Generative AI (GenAI), and ChatGPT in particular, is changing how academic writing is taught to learners of English as a foreign language (EFL). Even so, the individual studies and experiments conducted so far report results that conflict with one another, which makes the question worth examining.
Aim. Our study aimed to find out the result of the generative artificial intelligence effect on writing achievement of English language learners quantifying the effect and examine the sources of heterogeneity based on the observed differences.
Methods. To address the research objectives, we implemented a PRISMA 2020-compliant search protocol across Web of Science (n = 99) and ERIC (n = 200). The search incorporated targeted keywords covering topics such as AI writing tools (e.g., ChatGPT, Gemini), metacognitive and personalized learning, EFL academic writing skills, and higher/engineering education. Following a three-phase screening process and full-text assessment, nine eligible empirical studies (N = 624) with between-group post-test comparisons were selected. Standardized effect sizes (Hedges’) were extracted and pooled using a restricted maximum likelihood (REML) random-effects meta-analysis.
Nine studies (N=624) were included. The effect was large and positive: g=0.92, 95% CI [0.55, 1.28], p<.001; heterogeneity was high (I²=79%). Sensitivity analyses were stable (g=0.83 with the AWE tool removed; g=0.88–0.95 across leave-one-out runs). One study (Ekmekçi & Karabulut, 2025) reported a negative effect, where teacher instruction outperformed AI. Secondary constructs (metacognition, learner autonomy) also showed a positive trend. We conclude that GenAI tends to produce a meaningful average improvement in EFL writing performance, but the outcome depends heavily on the conditions of use and on teacher guidance. Because the number of studies is small, the results should be read as preliminary.

generative AI, ChatGPT, EFL academic writing, meta-analysis, effect size, PRISMA

1. Apriani, E., Daulay, S. H., Aprilia, F., Marzuki, A. G., Warsah, I., Supardan, D., & Muthmainnah. (2025). A mixed-method study on the effectiveness of using ChatGPT in academic writing and students’ perceived experiences. Journal of Language and Education, 11(1), 26–45. https://doi.org/10.17323/jle.2025.17913 [Google Scholar] [Crossref]

2. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Wiley. [Google Scholar] [Crossref]

3. DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7(3), 177–188. https://doi.org/10.1016/0197-2456(86)90046-2 [Google Scholar] [Crossref]

4. Ekmekçi, E., & Karabulut, E. (2025). Chatbot-driven writing practice: A boon or a bust for EFL learners? GIST Education and Learning Research Journal, (31), 45–64. [Google Scholar] [Crossref]

5. Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544 [Google Scholar] [Crossref]

6. Hao, H., & Razali, A. B. (2025). Impact of ChatGPT on English academic writing ability and engagement of Chinese EFL undergraduates. International Journal of Instruction, 18(2), 323–346. https://doi.org/10.29333/iji.2025.18218a [Google Scholar] [Crossref]

7. Hao, H., Razali, A. B., & Zuo, R. (2026). Exploring the impact of integrated AWE and generative AI feedback on Chinese EFL undergraduates’ higher-order thinking in argumentative writing. SAGE Open, 16(1). https://doi.org/10.1177/21582440251413884 [Google Scholar] [Crossref]

8. Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107 [Google Scholar] [Crossref]

9. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186 [Google Scholar] [Crossref]

10. Li, C., & Long, J. (2025). Comparing AI-assisted and traditional teaching in college English: Pedagogical benefits and learning behaviors. Information, 16(10), Article 895. https://doi.org/10.3390/info16100895 [Google Scholar] [Crossref]

11. Mekheimer, M. (2025). Generative AI-assisted feedback and EFL writing: A study on proficiency, revision frequency and writing quality. Discover Education, 4, Article 170. https://doi.org/10.1007/s44217-025-00602-7 [Google Scholar] [Crossref]

12. Namaziandost, E. (2025). Integrating flipped learning in AI-enhanced language learning: Mapping the effects on metacognitive awareness, writing development, and foreign language learning boredom. Computers and Education: Artificial Intelligence, 9, Article 100446. https://doi.org/10.1016/j.caeai.2025.100446 [Google Scholar] [Crossref]

13. Nguyen, Q. N., & Doan, D. T. H. (2025). ChatGPT’s effects on EFL learners’ autonomy and self-efficacy in self-regulated learning contexts. The JALT CALL Journal, 21(2), Article 102968. [Google Scholar] [Crossref]

14. Ozan, O., & Azap, S. (2026). Employing ChatGPT to improve high school students’ writing skills by providing feedback on topic-specific writing tasks. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2026.2613219 [Google Scholar] [Crossref]

15. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 [Google Scholar] [Crossref]

16. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03 [Google Scholar] [Crossref]

17. Vo, T. L., & Ho, T. M. P. (2026). A quasi-experimental study on enhancing EFL learners’ grammar precision in writing skills under supported-ChatGPT approach at a university in a remote region of Vietnam. Arab World English Journal, 361–375. https://doi.org/10.24093/awej/A3.23 [Google Scholar] [Crossref]

• Assessment of the Role of Artificial Intelligence in Repositioning TVET for Economic Development in Nigeria
• Teachers’ Use of Assure Model Instructional Design on Learners’ Problem Solving Efficacy in Secondary Schools in Bungoma County, Kenya
• “E-Booksan Ang Kaalaman”: Development, Validation, and Utilization of Electronic Book in Academic Performance of Grade 9 Students in Social Studies
• Analyzing EFL University Students’ Academic Speaking Skills Through Self-Recorded Video Presentation
• Major Findings of The Study on Total Quality Management in Teachers’ Education Institutions (TEIs) In Assam – An Evaluative Study