Evaluating The Efficacy of A Large Language Model in Scaffolding Research Report Writing for EFL Learners

  • Authors

    • Taj Mohammad Associate Professor, English Skills Department, Preparatory Year, Najran University, Najran, Saudi Arabia
    • Mohd Nazim Associate Professor, Department of English, College of Languages and Translation, Najran University, Saudi Arabia
    • Ali Abbas Falah Alzubi Assistant Professor of Applied Linguistics, Department of English, College of Languages and Translation, Najran University, Saudi Arabia
    • Soada Idris Khan Teacher and trainer, Sanabel Al Noor International School, Najran, Saudi Arabia
    https://doi.org/10.14419/g2apsg21

    Received date: October 13, 2025

    Accepted date: October 22, 2025

    Published date: November 2, 2025

  • ChatGPT; EFL writing; Efficacy; Large Language Models; Research report writing skills

  • Abstract

    This study examines how the Generative Pre-trained Transformer (GPT) architecture can assist English as a Foreign Language (EFL) learners in writing research reports. In a quasi-experimental design, 60 undergraduates were divided into two groups: a control group and an experimental group, where the latter utilized ChatGPT as a writing assistant. The intervention tested the model's performance in generating and improving text in five areas: organization and structure, research and evidence, analysis and critical thinking, writing quality and presentation, and content and relevance. Writing skills were measured with a 20-item achievement test before and after the intervention. Additionally, 15 participants took part in semi-structured interviews to provide qualitative feedback. The groups had similar skills before the intervention (p > 0.05). Afterward, the experimental group improved in all research report writing skills, particularly in language accuracy, organization, and content relevance. Participants praised ChatGPT for its user-friendly interface, time-saving features, and ability to improve grammar, vocabulary, and report structure. However, some raised concerns about plagiarism and reliance associated with the tool. These findings revealed that large language models (LLMs) can greatly improve the structure and language of student reports. Still, human oversight is needed to ensure critical thinking and factual accuracy.

  • References

    1. Pratama, Y. D. (2020). The investigation of using Grammarly as an online grammar checker in the process of writing. English Ideas: Journal of Eng-lish Language Education, 1(1), 46–54.
    2. Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learn-ing for EFL students. Frontiers in psychology, 14, 1260843.‏
    3. Grabe, W. (2001). Notes toward a theory of second language writing. In T. Silva & P. K. Matsuda (Eds.), On second language writing (pp. 39-57). Lawrence Erlbaum Associates.
    4. Matsuda, P. K. (2019). The myth of linguistic homogeneity in U.S. college composition. In Landmark essays on ESL writing (pp. 65-84). Routledge.
    5. Campbell, C. (2019). Teaching second language writing: Interacting with text. Routledge.
    6. Mohsen, M. A. (2022). Computer-mediated corrective feedback to improve L2 writing skills: A meta-analysis. Journal of Educational Computing Research, 60(5), 1253-1276.‏
    7. Han, J., Yoo, H., Kim, Y., Myung, J., Kim, M., Lim, H., Kim, J., Lee, T. Y., Hong, H., & Ahn, S. (2023). RECIPE: How to integrate ChatGPT into EFL writing education. In Proceedings of the 10th ACM Conference on Learning @ Scale (pp. 416–420). ACM. https://doi.org/10.1145/3573051.3596200
    8. Hyland, K. (2009). Academic discourse: English in a global context. Continuum.
    9. Nation, P. (2007). The four strands. International Journal of Innovation in Language Learning and Teaching, 1(1), 2–13. https://doi.org/10.2167/illt039.0
    10. Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or refor-mation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790
    11. Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868
    12. Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies. https://doi.org/10.1007/s10639-023-11742-4
    13. Perkins, M. (2023). Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching and Learning Practice, 20(2), 07. https://doi.org/10.53761/1.20.02.07
    14. Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Jour-nal of Applied Learning and Teaching, 6(1), 1–9. https://doi.org/10.37074/jalt.2023.6.1.17
    15. Meniado, J. C. (2023). The impact of ChatGPT on English language teaching, learning, and assessment: A rapid review of literature. Arab World English Journal, 14(4), 3–18. https://doi.org/10.24093/awej/vol14no4.1
    16. Imran, M., & Almusharraf, N. (2023). Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature. Contemporary Educational Technology, 15(4), ep464. https://doi.org/10.30935/cedtech/13605
    17. Darwin, Rusdin, D., Mukminatien, N., Suryati, N., Laksmi, E. D., & Marzuki. (2024). Critical thinking in the AI era: An exploration of EFL stu-dents’ perceptions, benefits, and limitations. Cogent Education, 11(1), 2290342.‏
    18. Vargas-Murillo, A. R., de la Asuncion, I. N. M., & de Jesús Guevara Soto, F. (2023). Challenges and opportunities of AI-assisted learning: A sys-tematic literature review on the impact of ChatGPT usage in higher education. International Journal of Learning, Teaching and Educational Re-search, 22(7), 122–135. https://doi.org/10.26803/ijlter.22.7.7
    19. Chapelle, C., & Chapelle, C. A. (2001). Computer applications in second language acquisition. Cambridge university press.‏
    20. Xiao, Y., Li, D., & Guo, K. (2025). Generative AI-powered non-player characters in digital storyline-based learning: an innovative approach to EFL writing. Innovation in Language Learning and Teaching, 1-15.‏
    21. Hampel, R., & Stickler, U. (2005). New skills for new classrooms: Training tutors to teach languages online. Computer Assisted Language Learning, 18(4), 311–326.
    22. Warschauer, M. (1996). Computer-assisted language learning: An introduction. In S. Fotos (Ed.), Multimedia language teaching (pp. 3-20). Logos International.
    23. Cheong, W., & Hong, H. (2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Jour-nal of Educational Technology and Innovation, 37–45.
    24. Domenech, J. (2023). ChatGPT in the classroom: Friend or foe? In Proceedings of the 9th International Conference on Higher Education Advances (HEAd’23) (pp. 339–347). Universitat Politècnica de València. https://doi.org/10.4995/HEAd23.2023.16179
    25. Chen, H. (2024). Educational applications of ChatGPT: Ethical challenges and countermeasures. English Language Teaching and Linguistics Stud-ies, 6(3), 100–110. https://doi.org/10.22158/eltls.v6n3p100
    26. Ulla, M. B., Perales, W. F., & Tarrayo, V. N. (2021). Integrating Internet-based applications in English language teaching: Teacher practices in a Thai university. Issues in Educational Research, 31(1), 283–301.
    27. Ulla, M. B., & Perales, W. F. (2021). Facebook as an integrated online learning support application for English language teaching: A case study in a Thai university. THAITESOL Journal, 34(2), 1–24.
    28. Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Edu-cation: Artificial Intelligence, 2, 100017. https://doi.org/10.1016/j.caeai.2021.100017
    29. Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
    30. Chen, X., & Xu, L. (2024, December). Effectiveness of ChatGPT in education: a meta-analysis. In 2024 5th International Conference on Infor-mation Science and Education (ICISE-IE) (pp. 428-431). IEEE.‏
    31. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Ad-vances in Neural Information Processing Systems (pp. 5998–6008).
    32. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). https://doi.org/10.1145/3442188.3445922
    33. Cooper, K. (2021, March 25). *OpenAI GPT-3: Everything you need to know*. Springboard. https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/
    34. Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based writing assistant’s impact on English language learners. Computers and Education: Artificial Intelligence, 3, 100055. https://doi.org/10.1016/j.caeai.2022.100055
    35. Welborn, A. (2023, March 7). ChatGPT and fake citations. Duke University Libraries Blogs. https://blogs.library.duke.edu/scholcomm/2023/03/07/chatgpt-and-fake-citations/
    36. Özçelik, N., & Yangın Ekşi, G. (2024). Cultivating writing skills: the role of ChatGPT as a learning assistant—a case study. Smart Learning Envi-ronments, 11(1), 10.‏
    37. Abdullayeva, M., & Musayeva, Z. M. (2023). The impact of ChatGPT on students’ writing skills: An exploration of AI-assisted writing tools. In-ternational Conference on Educational Research and Innovation, 4, 61–66.
    38. Nazari, N., Shabbir, M. S., & Setiawan, R. (2021). Application of artificial intelligence powered digital writing assistant in higher education: Ran-domized controlled trial. Heliyon, 7(5), e07014.
    39. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
    40. Chui, H. C. (2023). ChatGPT as a tool for developing paraphrasing skills among ESL learners. Journal of Creative Practices in Language Learning and Teaching (CPLT), 11(2), 85-105.‏
    41. Andargie, A., Amogne, D., & Tefera, E. (2025). Effects of project-based learning on EFL learners’ writing performance. PLoS ONE, 20(1), e0317518. https://doi.org/10.1371/journal.pone.0317518
    42. Wei, P., Wang, X., & Dong, H. (2023). The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: A randomized controlled trial. Frontiers in Psychology, 14, 1249991. https://doi.org/10.3389/fpsyg.2023.1249991
    43. Alkamel, M. A. A., & Alwagieh, N. A. S. (2024). Utilizing an adaptable artificial intelligence writing tool (ChatGPT) to enhance academic writing skills among Yemeni university EFL students. Social Sciences & Humanities Open, 10, 101095. https://doi.org/10.1016/j.ssaho.2024.101095
    44. Behforouz, B., & Al Ghaithi, A. (2024). Grammar gains: Transforming EFL learning with ChatGPT. Educational Process: International Journal, 13(4), 25–41. https://doi.org/10.22521/edupij.2024.134.2
    45. Adeshola, I., & Adepoju, A. P. (2024). The opportunities and challenges of ChatGPT in education. Interactive Learning Environments, 32(10), 6159-6172.‏
    46. Amin, M. Y. M. (2023). AI and ChatGPT in language teaching: Enhancing EFL classroom support and transforming assessment techniques. Inter-national Journal of Higher Education Pedagogies, 4(4), 1–15. https://doi.org/10.33422/ijhep.v4i4.602
    47. Rasul, T., Nair, S., Kalendra, D., Robin, M., de Oliveira Santini, F., Ladeira, W. J., Heathcote, L., & Sun, M. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6(1), 41–56. https://doi.org/10.37074/jalt.2023.6.1.4
    48. Odeh, R. (2005). Examining the difficulty coefficient in educational assessments. Journal of Educational Research, 98(4), 200–210. https://doi.org/10.1080/00220670509597512

  • Downloads

  • How to Cite

    Mohammad , T. ., Nazim , M. ., Alzubi, A. A. F., & Khan , S. I. . (2025). Evaluating The Efficacy of A Large Language Model in Scaffolding Research Report Writing for EFL Learners. International Journal of Basic and Applied Sciences, 14(7), 33-45. https://doi.org/10.14419/g2apsg21