Comparing Learners’ Perceptions of AI-Enabled Writing Feedback and Human Teacher Feedback in IELTS Writing Preparation
Main Article Content
Abstract
This study investigated International English Language Testing System (IELTS) Writing learners’ perceptions of AI-enabled writing feedback and human teacher feedback in relation to perceived writing improvement, IELTS Writing areas, and trust. Using an explanatory sequential mixed-methods design, questionnaire data were collected from 130 learners at a private IELTS teaching center in Ho Chi Minh City, Vietnam, and follow-up interviews were conducted with eight purposively selected participants. The questionnaire used a paired-source structure, enabling learners to evaluate AI feedback and human teacher feedback across parallel IELTS Writing dimensions. Quantitative data were analyzed in RStudio through reliability analysis, descriptive statistics, paired-samples t-tests, and Cohen’s dz, while interview data were analyzed in QualCoder using hybrid deductive–inductive thematic analysis. The findings showed no significant difference between AI feedback and human teacher feedback in overall perceived writing improvement. However, AI feedback was rated higher for grammar correction, vocabulary improvement, and revision speed, whereas human teacher feedback was rated higher for coherence and cohesion, task response, idea development, and trust. The integrated findings indicate differentiated, task-sensitive perceptions of the two feedback sources and perceived complementarity in how some learners combined them, although the study did not evaluate objective writing outcomes.
Keywords
AI feedback, automated writing evaluation, human teacher feedback, IELTS Writing, L2 writing feedback, trust in AI
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315-1325. https://doi.org/10.1080/02602938.2018.1463354
Chenail, R. J. (2012). Conducting qualitative data analysis: Reading line-by-line, but analyzing by meaningful qualitative units. The Qualitative Report, 17(1), 266-269. https://doi.org/10.46743/2160-3715/2012.1817
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE.
Crosthwaite, P., & Sun, S. (2026). Generative AI and L2 written feedback studies: A scoping review. RELC Journal, 57(1), 207-219. https://doi.org/10.1177/00336882251386530
Curtain, C. (n.d.). QualCoder [Computer software]. GitHub. https://github.com/ccbogel/QualCoder
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008
Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods, 5(1), 80-92. https://doi.org/10.1177/160940690600500107
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—Principles and practices. Health Services Research, 48(6 Pt 2), 2134-2156. https://doi.org/10.1111/1475-6773.12117
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487
Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39(2), 83-101. https://doi.org/10.1017/S0261444806003399
IELTS. (2023, May 3). IELTS Writing band descriptors and key assessment criteria. https://ielts.org/news-and-insights/ielts-writing-band-descriptors-and-key-assessment-criteria
Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20. https://doi.org/10.1177/1525822X05282260
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53-71. https://doi.org/10.1207/S15327566IJCE0401_04
Karatay, Y., & Karatay, L. (2024). Automated writing evaluation use in second language classrooms: A research synthesis. System, 123, Article 103332. https://doi.org/10.1016/j.system.2024.103332
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, Article 863. https://doi.org/10.3389/fpsyg.2013.00863
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50-80. https://doi.org/10.1518/hfes.46.1.50_30392
Link, S., Mehrzad, M., & Rahimi, M. (2022). Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning, 35(4), 605-634. https://doi.org/10.1080/09588221.2020.1743323
Lu, Q., Zhu, X., Zhu, S., & Yao, Y. (2024). Effects of writing feedback literacies on feedback engagement and writing performance: A cross-linguistic perspective. Assessing Writing, 62, Article 100889. https://doi.org/10.1016/j.asw.2024.100889
Mao, Z., Lee, I., & Li, S. (2024). Written corrective feedback in second language writing: A synthesis of naturalistic classroom studies. Language Teaching, 57(4), 449-477. https://doi.org/10.1017/S0261444823000393
Miao, F., & Holmes, W. (2023). Guidance for generative AI in education and research. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000386693
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1-13. https://doi.org/10.1177/1609406917733847
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Pearson, W. S. (2022). Student engagement with teacher written feedback on rehearsal essays undertaken in preparation for IELTS. SAGE Open, 12(1). https://doi.org/10.1177/21582440221079842
Posit team. (2025). RStudio: Integrated development environment for R [Computer software]. Posit Software, PBC. https://posit.co/
R Core Team. (2025). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
Ranalli, J. (2021). L2 student engagement with automated feedback on writing: Potential for learning and issues of trust. Journal of Second Language Writing, 52, Article 100816. https://doi.org/10.1016/j.jslw.2021.100816
Wei, P., Wang, X., & Dong, H. (2023). The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: A randomized controlled trial. Frontiers in Psychology, 14, Article 1249991. https://doi.org/10.3389/fpsyg.2023.1249991
Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28(11), 13943-13967. https://doi.org/10.1007/s10639-023-11742-4
Yoon, S.-Y., Miszoglad, E., & Pierce, L. R. (2023). Evaluation of ChatGPT feedback on ELL writers’ coherence and cohesion [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2310.06505
Zhang, T., & Mao, Z. (2023). Exploring the development of student feedback literacy in the second language writing classroom. Assessing Writing, 55, Article 100697. https://doi.org/10.1016/j.asw.2023.100697
Zhang, Z. V., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90-102. https://doi.org/10.1016/j.asw.2018.02.004
Zhang, Z., & Hyland, K. (2025). The role of digital literacy in student engagement with automated writing evaluation (AWE) feedback on second language writing. Computer Assisted Language Learning, 38(5-6), 1060-1085. https://doi.org/10.1080/09588221.2023.2256815