Views: 393
Arab World English Journal (AWEJ) Special Issue on CALL Vol. 17, No.12. July 2026 Pp. 237-256
Department of English Language, College of Language Sciences,
King Saud University
Riyadh, Saudi Arabia
This study explores the nature of automated scoring and feedback provided by an artificial intelligence-powered tool, the Virtual Writing Tutor. It also assessed the quality of the feedback specifically for non-native writers of IELTS Writing Task 1 essays. The study therefore contributes to the general research literature devoted to evaluating writing aids, which as yet has paid scant attention to this particular tool, and relatively little to the actual quality of feedback on writing for specific tasks. Eight students participated in this convergent mixed-method study, conducted during a 30-hour IELTS preparation program. A quantitative student-based perspective came from a 15 item, 5-point Likert scale response, questionnaire administered to students concerning their subjective evaluation of the Tutor’s feedback. A qualitative expert perspective came from researcher review of the automated scoring and feedback provided on the participants’ IELTS Writing Task 1 practice essays. The findings showed that the Tutor provided feedback relevant to all four of the IELTS writing assessment criteria, albeit the feedback was not presented straightforwardly in the IELTS categories. The software often used indicators, rather than more direct measures, and so could be regarded as overlooking important details that a human evaluator would notice. Additionally, the Tutor provided wrong information in some specific lexical and grammatical correction feedback. Students however judged that the Tutor was easy to use and (surprisingly) significantly useful in helping them improve grammatical accuracy, but they did not find it definitely effective in terms of improving spelling, content, vocabulary, and variety of sentence structures.
Alsalem, M. S. (2026). Evaluating the Automated Scoring and Feedback provided by the Virtual Writing Tutor on IELTS Writing Task 1 Essays: An Exploratory Study. Arab World English Journal (AWEJ) Special Issue on CALLVol.17. (12).237-256
Al Badi, A., Osman, M., & Al-Mekhlafi, A. (2020). The impact of Virtual Writing Tutor on writing skills and attitudes of Omani college students. Journal of Education and Development, 4(3), 101-116. https://doi.org/10.20849/jed.v4i3.828
Almegren, A., Mahdi, H. S., Hazaea, A. N., Ali, J. K., & Almegren, R. M. (2024). Evaluating the quality of AI feedback: A comparative study of AI and human essay grading. Innovations in Education and Teaching International, 1–16. https://doi.org/10.1080/14703297.2024.2437122
Altamimi, O. (2014). Dynamic Written Corrective Feedback revisited: A critique and an improved model. Arab World English Journal, 5(3), 180-189. https://awej.org/wp-content/uploads/2021/01/14-1-1.pdf
Atigh, R. & Khabbazi, S. (2021). The evaluation of ESP Textbooks based on students’ perspectives. International Journal of Linguistics, Literature and Translation, 4(5), 56-62. DOI: 10.32996/ijllt
Behzad, S., Kashefi, O., & Somasundaran, S. (2024). Assessing online writing feedback resources: Generative AI vs. good samaritans. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1638–1644. Torino, Italia: ELRA and ICCL.
Bringman-Rodenbarger L., & Hortsch M. (2020). How students choose E-learning resources: The importance of ease, familiarity, and convenience. FASEB Bioadvances, 6(5), 286-295. doi: 10.1096/fba.2019-00094.
Calhoun, C., Sahay, S., & Wilson, M. (2025). Instructional design evaluation. LibreText Social Sciences. https://socialsci.libretexts.org/Bookshelves/Education_and_Professional_Development/Design_for_Learning
Chan, S., Lo, N. & Wong, A. (2025). Leveraging generative AI for enhancing university-level English writing: Comparative insights on automated feedback and student engagement. Cogent Education, 12(1), 2440182. DOI: 10.1080/2331186X.2024.2440182.
Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20, 57.
Hattie, J. & Timperley, H. (2008). The Power of Feedback. Review of Educational Research, 77(1). https://doi.org/10.3102/003465430298487
IELTS (2025). Understanding IELTS scoring. https://ielts.org/organisations/ ielts-for-organisations/understanding-ielts-scoring
Jamshed, M., Ahmed, A.S.M.M., Sarfaraj, M., & Warda, W.U. (2024). The impact of ChatGPT on English language learners’ writing skills: An assessment of AI feedback on Mobile. International Journal of Interactive Mobile Technologies (iJIM), 18(19), 18–36. DOI:10.3991/ijim.v18i19.50361
Kaharuddin, A., D., Mardiana., L. I., Arafah, B., & Suryadi, R. (2024). Defining the role of Artificial Intelligence in improving English writing skills among Indonesian students. Journal of Language Teaching and Research, 15(2), 568-678.
Mohammed, S., & Khalid, M. (2025). Under the world of AI-generated feedback on writing: Mirroring motivation, foreign language peace of mind, trait emotional intelligence, and writing development. Language Testing in Asia. 15. 10.1186/s40468-025-00343-2.
Morrison, G. R., Ross, S. J., Morrison, J. R., & Kalman, H. K. (2019). Designing effective instruction. Wiley.
Nasution, R., Debora, M., Saragih, B., Gultom. J., & Lubis, S. (2023). Virtual Writing Tutor (VWT) on students’ writing skill improvement at writing class, Asian TESOL Journal, 3(1), 15-18. DOI:10.35307/asiantj.v3i1.42
Petter, S., DeLone, W. & McLean, E. (2008). Measuring information systems success: Models, dimensions, measures, and interrelationships. European Journal of Information Systems 17, 236–263 (2008). https://doi.org/10.1057/ejis.2008.15
Polakova, P., & Ivenz, P. (2024). The impact of ChatGPT feedback on the development of EFL students’ writing skills. Cogent Education, 11(1). https://doi.org/10.1080/2331186X.2024.2410101
Schmidt, R. (2010). Attention, awareness, and individual differences in language learning. In W. M. Chan, S. Chi, K. N. Cin, J. Istanto, M. Nagami, J. W. Sew, T. Suthiwan, & I. Walker, Proceedings of CLaSIC 2010, Singapore, December 2-4 (pp. 721-737). Singapore: National University of Singapore, Centre for Language Studies.
Venkatesh, V., Thong, J. & Xu, X. (2016). Unified Theory of Acceptance and Use of Technology: A synthesis and the road ahead. Journal of the Association for Information Systems, 17 (5), 328-376.
Ventayen, R. J. M., & Orlanda-Ventayen, C. C. (2018). Graduate students’ perspective on the usability of Grammarly® in one ASEAN state university. Asian ESP Journal, 14(7.2). http://www.scopus.com/inward/record.url?eid=2-s2.0 85061485593&partnerID=MN8TOARS
Viantika, S., & Dangin, D. (2024). Improving students’ writing skill using Virtual Writing Tutor: Automatic corrective feedback. EDULIA: English Education, Linguistic and Art Journal, 4(2), 138–151. https://doi.org/10.31539/edulia.v4i2.8890
Virtual Writing Tutor (2025). https://virtualwritingtutor.com/
Wei P, Wang X, & Dong H. (2023), The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: A randomized controlled trial. Frontiers in Psychology, 14, 1249991. doi: 10.3389/fpsyg.2023.1249991.
Zakaria, N., Hashim, H., & Jamaludin, K. (2025). Mobile technology for enhanced ESL writing education: A systematic review of benefits and pedagogical innovations. Arab World English Journal (AWEJ) Special Issue on CALL, 11, 210-222. https://dx.doi.org/10.24093/awej/call11.13
Zein, T., Sinar, S., & Deliana D. (2025). Using Virtual Writing Tutor in teaching writing to EFL learners. Jurnal Ilmiah Ilmu Terapan Universitas Jambi (JIITUJ), 9(2). DOI: 10.22437/jiituj.v9i2.36052
Zhang, Z.V. & Hyland, K. (2022). Fostering student engagement with feedback: An integrated approach. Assessing Writing, 51, 100586. https://doi.org/10.1016/j.asw.2021.100586.
Zhao, H., Huang, J. & Whipple, P. (2026), Reliability of ChatGPT’s scoring and actionability of its feedback in IELTS writing preparation classrooms: Teacher and student acceptance in China. AERA Open. June 2026. https://doi.org/10.1177/23328584261448
Dr. Mashael Alsalem is an assistant professor at the Faculty of Language Sciences, King Saud University. She has been teaching English for about 22 years and training on IELTS for four years. Her main research interests include AI in language teaching and learning, Discourse Analysis, and language teacher cognition. ORCID ID: https://orcid.org/0009-0004-7067-1085