Teacher- Versus AI-Generated (Poe Application) Corrective Feedback and Language Learners’ Writing Anxiety, Complexity, Fluency, and Accuracy
DOI:
https://doi.org/10.19173/irrodl.v25i3.7646Keywords:
artificial intelligence, AI, corrective feedback, writing anxiety, fluency, accuracyAbstract
This study examines the effects of corrective feedback (CF) on language learners’ writing anxiety, writing complexity, fluency, and accuracy, and compares the effectiveness of feedback from human teachers with an AI-driven application called Poe. The study included three intact classes, each with 25 language learners. Using a quasi-experimental design with pretest and posttest measures, one class received feedback from the teacher, one from the Poe application, and the third received no response to their writing. Data were generated though tests and a writing anxiety scale developed for the study. Data analysis, conducted using one-way ANOVA tests, revealed significant effects of teacher and AI-generated feedback on learners’ writing anxiety, accuracy, and fluency. Interestingly, the group that received AI-generated feedback performed better than the group that received teacher feedback or no AI support. Additionally, learners in the AI-generated feedback group experienced a more significant reduction in writing anxiety than their peers. These results highlight the remarkable impact of AI-generated CF on improving writing outcomes and alleviating anxiety in undergraduate language learners at East China University of Political Science and Law . The study demonstrates the benefits of integrating AI applications into language learning contexts, particularly by promoting a supportive environment for students to develop writing skills. Educators, researchers, and developers can use these findings to inform pedagogical practices and technological interventions to optimize the language learning experience in primary school settings. This research highlights the effectiveness of AI-driven applications in language teaching. It highlights the importance of considering learners’ psychological well-being, particularly anxiety levels, when developing effective language learning interventions.
References
Benavides, L. M. C., Tamayo Arias, J. A., Arango Serna, M. D., Branch Bedoya, J. W., & Burgos, D. (2020). Digital transformation in higher education institutions: A systematic literature review. Sensors, 20(11), Article 3291. https://doi.org/10.3390/s20113291
Benson, S., & DeKeyser, R. (2018). Effects of written corrective feedback and language aptitude on verb tense accuracy. Language Teaching Research, 23(6), 702–726. https://doi.org/10.1177/1362168818770921
Biber, D., Nekrasova, T., & Horn, B. (2011). The effectiveness of feedback for L1-English and L2-writing development: A meta-analysis. ETS Research Report Series, 2011(1), i–99.https://doi.org/10.1002/j.2333-8504.2011.tb02241.x
Bonilla López, M., Van Steendam, E., Speelman, D., & Buyse, K. (2018). The differential effects of comprehensive feedback forms in the second language writing class. Language Learning, 68(3), 813–850. https://doi.org/10.1111/lang.12295
Bozkurt, A., Karadeniz, K., Baneres, D., Guerrero-Roldán, A. E., & Rodríguez, M. E. (2021). Artificial intelligence and reflections from educational landscape: A review of AI studies in half a century. Sustainability, 13(2), Article 800. https://doi.org/10.3390/su13020800
Bozkurt, A., Xiao, J., Lambert, S., Pazurek, A., Crompton, H., Koseoglu, S., Farrow, R., Bond, M., Nerantzi, C., Honeychurch, S., Bali, M., Dron, J., Mir, K., Stewart, B., Costello, E., Mason, J., Stracke, C. M., Romero-Hall, E., Koutropoulos, A., … Jandrić, P. (2023). Speculative futures on ChatGPT and generative artificial intelligence (AI): A collective reflection from the educational landscape. Asian Journal of Distance Education, 18(1), 53–130. https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/709
Cao, L. (2023). Trans-AI/DS: Transformative, transdisciplinary and translational artificial intelligence and data science. International Journal of Data Science and Analytics,15(1), 119–132. https://doi.org/10.1007/s41060-023-00383-y
Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1(2020), Article 100002. https://doi.org/10.1016/j.caeai.2020.100002
Chu, H.-C., Hwang, G.-H., Tu, Y.-F., & Yang, K.-H. (2022). Roles and research trends of artificial intelligence in higher education: A systematic review of the top 50 most-cited articles. Australasian Journal of Educational Technology, 38(3), 22–42. https://ajet.org.au/index.php/AJET/article/view/7526
Cox, A. M. (2021). Exploring the impact of artificial intelligence and robots on higher education through literature-based design fictions. International Journal of Educational Technology in Higher Education, 18(2021), Article 3. https://doi.org/10.1186/s41239-020-00237-8
DeKeyser, R. (2007). Skill acquisition theory. In B. VanPatten & J. Williams (Eds.), Theories in second language acquisition: An introduction (2nd ed., pp. 94–112). Routledge.
Diebold, G. (2023, January 17). Higher education will have to adapt to generative AI—And that’s a good thing. Center for Data Innovation. https://datainnovation.org/2023/01/higher-education-will-have-to-adapt-to-generative-ai-and-thats-a-good-thing/
Dogan, M. E., Goru Dogan, T., & Bozkurt, A. (2023). The use of artificial intelligence (AI) in online learning and distance education processes: A systematic review of empirical studies. Applied Sciences, 13(5), Article 3056. https://doi.org/10.3390/app13053056
Ellis, R. (2009a). A typology of written corrective feedback types. ELT Journal, 63(2), 97–107. https://doi.org/10.1093/elt/ccn023
Ellis, R. (2009b). Corrective feedback and teacher development. L2 Journal: An electronic refereed journal for foreign and second language educators, 1(1), 2–18. https://doi.org/10.5070/l2.v1i1.9054
Ellis, R. (2009c). Task-based language teaching: Sorting out the misunderstandings. International Journal of Applied Linguistics, 19(3), 221–246. https://doi.org/10.1111/j.1473-4192.2009.00231.x
Evans, N. W., Hartshorn, K. J., Cox, T. L., & Martin de Jel, T. (2014). Measuring written linguistic accuracy with weighted clause ratios: A question of validity. Journal of Second Language Writing, 24(1), 33–50. https://doi.org/10.1016/j.jslw.2014.02.005
Goksel, N., & Bozkurt, A. (2019). Artificial intelligence in education: Current insights and future perspectives. In S. Sisman-Ugur & G. Kurubacak (Eds.), Handbook of research on learning in the age of transhumanism (pp. 224–236). IGI Global. https://doi.org/10.4018/978-1-5225-8431-5.ch014
Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of Second Language Writing, 30(1), 31–44. https://www.doi.org/10.1016/j.jslw.2015.08.002
Hartshorn, K. J., & Evans, N. W. (2015). The effects of dynamic written corrective feedback: A 30-week study. Journal of Response to Writing, 1(2), 6–34.
Humble, N., & Mozelius, P. (2022). The threat, hype, and promise of artificial intelligence in education. Discover Artificial Intelligence, 2(2022), Article 22. https://doi.org/10.1007/s44163-022-00039-z
Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39(2), 83–101. https://doi.org/10.1017/S0261444806003399
Kang, E., & Han, Z. (2015). The efficacy of written corrective feedback in improving L2 written accuracy: A meta-analysis. The Modern Language Journal, 99(1), 1–18. https://doi.org/10.1111/modl.12189
Karim, K., & Nassaji, H. (2018). The revision and transfer effects of direct and indirect comprehensive corrective feedback on ESL students’ writing. Language Teaching Research, 24(4), 519–539. https://doi.org/10.1177/1362168818802469
Karim, K., & Nassaji, H. (2019). The effects of written corrective feedback: A critical synthesis of past and present research. Instructed Second Language Acquisition, 3(1), 28–52. https://doi.org/10.1558/isla.37949
Kurzer, K. (2018). Dynamic written corrective feedback in developmental multilingual writing classes. TESOL Quarterly, 52(1), 5–33. https://doi.org/10.1002/tesq.366
Kurzweil, R. (2014). The singularity is near. In R. L. Sandler (Ed.), Ethics and emerging technologies (pp. 393–406). Palgrave Macmillan UK. https://doi.org/10.1057/9781137349088_26
Leeman, J. (2010). Feedback in L2 learning: Responding to errors during practice. In R. DeKeyser (Ed.), Practice in a second language: Perspectives from applied linguistics and cognitive psychology (pp. 111–138). Cambridge University Press. https://doi.org/10.1017/cbo9780511667275.007
Li, S., & Vuono, A. (2019). Twenty-five years of research on oral and written corrective feedback in System. System, 84(1), 93–109. https://doi.org/10.1016/j.system.2019.05.006
Lim, S. C., & Renandya, W. A. (2020). Efficacy of written corrective feedback in writing instruction: A meta-analysis. TESL-EJ, 24(3), 1–26. https://tesl-ej.org/wordpress/issues/volume24/ej95/ej95a3/
Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30(1), 66–81. https://doi.org/10.1016/j.jslw.2015.08.011
Liu, Y., & Huang, J. (2020). The quality assurance of a national English writing assessment: Policy implications for quality improvement. Studies in Educational Evaluation, 67(2), Article 100941. https://doi.org/10.1016/j.stueduc.2020.100941
Long, M. H. (1980). Input, interaction, and second language acquisition. University of California.
Luo, Y., & Liu, Y. (2017). Comparison between peer feedback and automated feedback in college English writing: A case study. Open Journal of Modern Linguistics, 7(04), 197–215. https://doi.org/10.4236/ojml.2017.74015
Luan, H., Geczy, P., Lai, H., Gobert, J., Yang, S. J. H., Ogata, H., Baltes, J., Guerra, R., Li, P., & Tsai, C.-C. (2020). Challenges and future directions of big data and artificial intelligence in education. Frontiers in Psychology, 11, Article 580820. https://doi.org/10.3389/fpsyg.2020.580820
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2(2021), Article 100041. https://doi.org/10.1016/j.caeai.2021.100041
Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6), 7893–7925. https://doi.org/10.1007/s10639-022-10925-9
Polio, C. (2012). The relevance of second language acquisition theory to the written error correction debate. Journal of Second Language Writing, 21(4), 375–389. https://doi.org/10.1016/j.jslw.2012.09.004
Polio, C., & Yoon, H. J. (2018). The reliability and validity of automated tools for examining variation in syntactic complexity across genres. International Journal of Applied Linguistics (United Kingdom), 28(1), 165–188. https://doi.org/10.1111/ijal.12200
Riazantseva, A. (2012). Outcome measure of L2 writing as a mediator of the effects of corrective feedback on students’ ability to write accurately. System, 40(3), 421–430. https://doi.org/10.1016/j.system.2012.07.005
Robinson, P. (2011). Second language task complexity, the cognition hypothesis, language learning, and performance. In P. Robinson (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 3–38). John Benjamins.
Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of L2 grammar: A meta-analysis of the research. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 133–164). John Benjamins.
Schmidt, R. (2012). Attention, awareness, and individual differences in language learning. Perspectives on Individual Characteristics and Foreign Language Education, 6(27), 27-49.
Selwyn, N., Hillman, T., Bergviken-Rensfeldt, A., & Perrotta, C. (2023). Making sense of the digital automation of education. Postdigital Science and Education, 5(1), 1–14. https://doi.org/10.1007/s42438-022-00362-9
Sharma, R. C., Kawachi, P., & Bozkurt, A. (2019). The landscape of artificial intelligence in open, online and distance education: Promises and concerns. Asian Journal of Distance Education, 14(2), 1–2. https://doi.org/10.5281/zenodo.3730631
Sia, P. F. D., & Cheung, Y. L. (2017). Written corrective feedback in writing instruction: A qualitative synthesis of recent research. Issues in Language Studies, 6(1), 61–80. https://www.ils.unimas.my/images/pdf/v6n1/ILS_Vol6No1_Sia.pdf
Stefanou, C., & Révész, A. (2015). Direct written corrective feedback, learner differences, and the acquisition of second language article use for generic and specific plural reference. The Modern Language Journal, 99(2), 263–282. https://doi.org/10.1111/modl.12212
Storch, N. (2009). The impact of studying in a second language (L2) medium university on the development of L2 writing. Journal of Second Language Writing, 18(2), 103–118. https://api.semanticscholar.org/CorpusID:62520708
Tang, K.-Y., Chang, C.-Y., & Hwang, G.-J. (2021). Trends in artificial intelligence-supported e-learning: A systematic review and co-citation network analysis (1998–2019). Interactive Learning Environments, 31(4), 2134–2152. https://doi.org/10.1080/10494820.2021.1875001 Thi, K, N., & Nikolov, M. (2021). Feedback treatments, writing tasks, and accuracy measures: A critical review of research on written corrective feedback. TESL-E, 25(3), 1-25. https://www.tesl-ej.org/pdf/ej99/a16.pdf
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), Article 15. https://doi.org/10.1186/s40561-023-00237-x
Truscott, J. (2010). Further thoughts on Anthony Bruton’s critique of the correction debate. System, 38(4), 626–633. https://doi.org/10.1016/j.system.2010.10.003
Truscott, J., & Hsu, A. Y.-P. (2008). Error correction, revision, and learning. Journal of Second Language Writing, 17(4), 292–305. https://doi.org/10.1016/j.jslw.2008.05.003
Winterson, J. (2022). 12 Bytes: How artificial intelligence will change the way we live and love. Penguin Random House.
Wolfe-Quintero, K. (1998). The connection between verbs and argument structures: Native speaker production of the double object dative. Applied Psycholinguistics, 19(2), 225–257. https://doi.org/10.1017/S0142716400010055
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), Article 39. https://doi.org/10.1186/s41239-019-0171-0
Zhang, T. (2021). The effect of highly focused versus mid-focused written corrective feedback on EFL learners’ explicit and implicit knowledge development. System, 99(2), Article 102493. https://doi.org/10.1016/j.system.2021.102493
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
