Arab World English Journal (AWEJ) Special Issue on CALL Number 9. July 2023                     Pp.3- 17

Full Paper PDF

 ChatGPT’s Capabilities in Spotting and Analyzing Writing Errors Experienced by EFL

Jeehaan Algaraady
English Department., Faculty of Education, Taiz University, Taiz, Yemen 

Mohammad Mahyoob
Languages & Translation Dept. Taibah University, Madina, Saudi Arabia
Technical Community College -Taiz, Yemen
Corresponding Author:


Received: 04/02/ 2023             Accepted:07/03/2023              Published:07/24/2023


The recent Large Language Models (LLMs) use advanced algorithms to identify areas where sentence structure and word choice can be improved and to detect grammar, syntax, and spelling mistakes in sentences. This study aimed to investigate the effectiveness of the Chat Generative Pre-trained Transformer (ChatGPT) in detecting English as a foreign language (EFL) learners’ writing errors compared to human instructors. This study examines the ChatGPT as a recent and advanced LLM in analyzing and processing EFL learners’ writing issues. This paper provides valuable insights into the potential benefits and challenges of integrating Artificial Intelligence (AI) into EFL writing education. Our results revealed that ChatGPT successfully identified most surface-level errors but could not detect writing errors related to deep structure and pragmatics. Conversely, human teachers could spot most of these issues. These findings suggest that while ChatGPT can be a valuable tool in identifying surface-level errors, it cannot replace human instructors’ expertise and nuanced understanding in detecting errors related to the more complex aspects of writing. The writing error types (data) are statistically analyzed. The descriptive analysis displays valuable insights into the reliability of the data and its potential implications, where the F-score, which measures the statistical model accuracy, is found to be 1.5. In the meantime, the p-value score, which shows the probability of obtaining results as extreme as the detected data, is calculated to be 0.23. The results suggest that the collected data is statistically significant, and further analysis may yield valuable insights.
Keywords: Artificial Intelligence, ChatGPT, EFL writing, EFL learners, LLMs, error analysis

Cite as: Algaraady, J., & Mahyoob, M.(2023). ChatGPT’s Capabilities in Spotting and Analyzing Writing Errors Experienced by EFL Learners.  Arab World English Journal (AWEJ) Special Issue on CALL (9)3-17.


Abdalkader, S. M. A. (2022). Using Artificial Intelligence to improve Writing Fluency for The

Preparatory Stage Students in Distinguished Governmental Language Schools. Egyptian 

     Journal of Educational Sciences2(2), 39-70.

Al-Garaady, J., & Mahyoob, M. (2021). Social network communication: Emojis and EFL learners’ writing issues. TESOL International16(3.1).

Algaraady, J., & Alrahaili, M. (2022). An Investigation of Parental Perspectives on the

Efficiency of Online Schooling in Primary Schools During the COVID-19

Outbreak. International Journal of Information and Communication Technology Education

     (IJICTE)18(1), 1-18.

Baskara, F. R. (2023). Integrating ChatGPT into EFL writing instruction: Benefits and

challenges. International Journal of Education and Learning5(1), 44-55.

Bishop, L. (2023). A computer wrote this paper: What Chatgpt means for education, research,

and writing. Research, and Writing (January 26, 2023).

Bowman, Emma (December 19, 2022). A new AI chatbot might do your homework for you. But it’s
still not an A+ student
NPRArchived from the original on January 20, 2023. Retrieved December 19, 2022.

Baskara, R. (2023). Integrating ChatGPT into EFL Writing Instruction: Benefits and

Challenges. International Journal of Education and Learning5(1).

Chawla, Raveen (December 26, 2022). What is ChatGPT? History, Features, Uses, Benefits,    Drawbacks 2023.
Archived from the original on January 7, 2023. Retrieved December 27, 2022.

Chen, T. (2016). Technology-supported peer feedback in ESL/EFL writing classes: A research

synthesis. Computer Assisted Language Learning29(2), 365-397.

Cho, K., & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: A web-

based reciprocal peer review system. Computers & Education48(3), 409-426.

Corder, S.P. 1981. Error Analysis and Interlanguage. Oxford: Oxford University Press.

Ellis, R. 2002. Second Language Acquisition. Oxford: Oxford University Press

Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based

writing assistant’s impact on English language learners. Computers and Education: Artificial

      Intelligence3, 100055.

Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation.   

     Contemporary Educational Technology 15(2).

Han, J., Yoo, H., Kim, Y., Myung, J., Kim, M., Lim, H., … & Oh, A. (2023). RECIPE: How to

Integrate ChatGPT into EFL Writing Education. arXiv preprint arXiv:2305.11583.

Holmes, W., Bialik, M., & Fadel, C. (2023). Artificial intelligence in education. Globethics


Hong, W. C. H. (2023). The impact of ChatGPT on foreign language teaching and learning:

opportunities in education and research. Journal of Educational Technology and


Karim, A., Mohammed A.R., Ismail, SAMM, Shahed, F.H., Rahman, M.M., & Haque, M.H.

  1. Error Analysis in EFL Writing Classroom. International Journal of English

      Linguistics, Vol. 8 (4): 122-138.

Lin, V., Liu, G. Z., & Chen, N. S. (2022). The effects of an augmented-reality ubiquitous writing

application: A comparative pilot project for enhancing EFL writing instruction. Computer

    Assisted Language Learning35(5-6), 989-1030

Lu, X. (2019). An empirical study on the artificial intelligence writing evaluation system in

China CET. Big data7(2), 121-129.

Mahyoob, M. (2021). Online learning effectiveness during the COVID-19 pandemic: A case

study of Saudi universities. International Journal of Information and Communication

      Technology Education (IJICTE)17(4), 1-14.

Park, J. (2019). An AI-based English grammar checker vs. human raters in evaluating EFL

learners’ writing. Multimedia-Assisted Language Learning22(1), 112-131.

Park, J. (2019).’ Implications of AI-based grammar checker in EFL learning and testing: Korean

high school students’ writing. The Korea English Language Testing Association14(1), 11-39.

Pedro, F., Subosa, M., Rivas, A., & Valverde, P. (2019). Artificial intelligence in education:

Challenges and opportunities for sustainable development. Paris: UNESCO.

Perkins, M. (2023). Academic Integrity considerations of AI Large Language Models in the post-

pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning

     Practice20(2), 07.

Shi, H & Aryadoust, V. (2023) A systematic review of automated writing evaluation

systems.” Education and Information Technologies 28. (1): 771-795.

Stevenson, M., & Phakiti, A. (2019). Automated feedback and second language

writing. Feedback in second language writing: Contexts and issues, 125-142.

Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing

classrooms. Assessing Writing57, 100752.
Tate, T. P., Doroudi, S., Ritchie, D., Xu, Y., & uci, m. w. (2023, January 10). Educational

Research and AI-Generated Writing: Confronting the Coming Tsunami.

Wang, Z. (2022). Computer-assisted EFL writing and evaluations based on artificial intelligence:

a case from a college reading and writing course. Library Hi Tech40(1), 80-97.

Wu, H., Wang, W., Wan, Y., Jiao, W., & Lyu, M. (2023). Chatgpt or grammarly? evaluating

chatgpt on grammatical error correction benchmark. arXiv preprint arXiv:2303.13648.

Zhai, N., & Ma, X. (2023). The Effectiveness of Automated Writing Evaluation on Writing

Quality: A Meta-Analysis. Journal of Educational Computing Research61(4), 875–


Yan, D. (2023). How ChatGPT’s automatic text generation impact on learners in a L2 writing

practicum: an exploratory investigation. ⟨hal-04037687⟩

Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory

investigation. Education and Information Technologies, 1-25.

Received: 04/02/ 2023   
Accepted: 07/03/2023 
Published: 07/24/2023 

Dr. Jeehaan Algaraady is an Assistant Professor at the English Department, Faculty of Education, Taiz University, Yemen. She received her Ph.D. in Computational Linguistics. Her research interests include Computational Linguistics, Theoretical Linguistics, ELT, and e-learning. ORCID:

 Dr. Mohammad Mahyoob is an Associate Professor of Computational linguistics at the Department of Languages & Translation, Al-Ula Branch, Taibah University, Madina, Saudi Arabia. He is also an Associate Professor of Computational linguistics at Technical Community College -Taiz, Yemen. He holds his Ph.D. in Computational Linguistics. His research interests include Computational linguistics, Theoretical, and Applied Linguistics, ELE, e-learning and digital media studies. ORCID: