Does AI Simplification of Authentic Blog Texts Improve Reading Comprehension, Inferencing, and Anxiety? A One-Shot Intervention in Turkish EFL Context
DOI:
https://doi.org/10.19173/irrodl.v25i3.7779Keywords:
artificial intelligence, ChatGPT, simplification, reading, language teachingAbstract
This experimental study investigates the impact of ChatGPT-simplified authentic texts on university students’ reading comprehension, inferencing, and reading anxiety levels. A within-subjects design was employed, and 105 undergraduate English as a foreign language (EFL) students engaged in both original and ChatGPT-simplified text readings, serving as their own controls. The findings reveal a significant improvement in reading comprehension scores and inferencing scores following ChatGPT intervention. However, no significant change in reading anxiety levels was observed. Results suggest that ChatGPT simplification positively influences reading comprehension and inferencing, but its impact on reading anxiety remains inconclusive. This research contributes to literature on the use of artificial intelligence (AI) in education and sheds light on ChatGPT’s potential to influence language learning experiences within higher education contexts. The study highlights the practical application of ChatGPT as a tool for helping students engage in authentic text readings by making text more comprehensible. Based on the findings, several multifaceted implications that extend to various stakeholders in the field of language education are provided.
References
AbdulHussein, F. R. (2014). Investigating EFL college teachers’ and learners’ attitudes toward using authentic reading materials in Misan. Procedia—Social and Behavioral Sciences, 136, 330–343. https://doi.org/10.1016/j.sbspro.2014.05.338
Bayat, N., & Çetinkaya, G. (2020). The relationship between inference skills and reading comprehension. Education and Science, 45(203), 177–190. https://doi.org/10.15390/EB.2020.8782
Bin-Hady, W. R. A., Al-Kadi, A., Hazaea, A., & Ali, J. K. M. (2023). Exploring the dimensions of ChatGPT in English language learning: A global perspective. Library Hi Tech. Advance online publication. https://doi.org/10.1108/LHT-05-2023-0200
Brown, H., Lee, K., Mireshghallah, F., Shokri, R., & Tramèr, F. (2022, June 21–24). What does it mean for a language model to preserve privacy? In FaccT ’22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 2280–2292). Association for Computing Machinery. https://doi.org/10.1145/3531146.3534642
Cain, K., Oakhill, J. V., Barnes, M. A., & Bryant, P. E. (2001). Comprehension skill, inference-making ability, and their relation to knowledge. Memory & Cognition, 29(6), 850–859. https://doi.org/10.3758/BF03196414
Chen, H.-R., & Hsu, W.-C. (2022). Do flipped learning and adaptive instruction improve student learning outcome? A case study of a computer programming course in Taiwan. Frontiers in Psychology, 12, Article 768183. https://doi.org/10.3389/fpsyg.2021.768183
Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, Article 100002. https://doi.org/10.1016/j.caeai.2020.100002
Chen, Y., Chen, Y., & Heffernan, N. (2020). Personalized math tutoring with a conversational agent (arXiv preprint: arXiv:2012.12121).
Chow, B. W.-Y., Mo, J., & Dong, Y. (2021). Roles of reading anxiety and working memory in reading comprehension in English as a second language. Learning and Individual Differences, 92, Article 102092. https://doi.org/10.1016/j.lindif.2021.102092.
Corpus of Contemporary American English (COCA). (n.d.). Retrieved January 27, 2024, from https://www.english-corpora.org/coca/
Crossley, S. A., McCarthy, P. M., Louwerse, M. M., & McNamara, D. S. (2007). A linguistic analysis of simplified and authentic texts. The Modern Language Journal, 91, 15–30. https://doi.org/10.1111/j.1540-4781.2007.00507.x
Crum, A. J., Salovey, P., & Achor, S. (2013). Rethinking stress: The role of mindsets in determining the stress response. Journal of Personality and Social Psychology, 104(4), 716–733. https://doi.org/10.1037/a0031201DeBacker, T. K., Heddy, B. C., Kershen, J. L., Crowson, H. M., Looney, K., & Goldman, J. A. (2018). Effects of a one-shot growth mindset intervention on beliefs about intelligence and achievement goals. Educational Psychology, 38(6), 711–733. https://doi.org/10.1080/01443410.2018.1426833
Dewi, P. H. R., & Pramerta, G. P. A. (2021). Correlation between anxiety and reading comprehension: A study in a secondary school. Academic Journal on English Studies, 1(2), 152–161. https://e-journal.unmas.ac.id/index.php/ajoes/article/view/4606
Dida, H. A., Chakravarthy, D. S. K., & Rabbi, F. (2023). ChatGPT and big data: Enhancing text-to-speech conversion. Mesopotamian Journal of Big Data, 2023, 33–37. https://doi.org/10.58496/MJBD/2023/005
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A.M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M.A., Al-Busaidi, A.S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., Carter, L., & Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
FeedSpot. (2024, May 20). 100 best life blogs and websites in 2024. Lifestyle Bloggers Database. https://lifestyle.feedspot.com/life_blogs/
Gashti, Y. B. (2018). The effect of authentic and simplified literary texts on the reading comprehension of Iranian advanced EFL learners. Iranian Journal of English for Academic Purposes, 7(2), 32–44. https://e-journal.unmas.ac.id/index.php/ajoes/article/view/4606
Graesser, A. C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101(3), 371–395. https://doi.org/10.1037/0033-295X.101.3.371
Haastrup, K. (2008). Lexical inferencing procedures in two languages. In D. Albrechtsen, K. Haastrup, & B. Henriksen (Eds.), Vocabulary and writing in a first and second language: Processes and development (pp. 67–111). Palgrave Macmillan. https://doi.org/10.1057/9780230593404_3
Holmes, W., Bialik, M., & Fadel, C. (2023). Artificial intelligence in education. In C. Stückelberger & P. Duggal (Eds.), Data ethics: Building trust: How digital technologies can serve humanity (pp. 621–653). Globethics Publications. https://doi.org/10.58863/20.500.12424/4276068
Hong, W. C. H. (2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Journal of Educational Technology and Innovation, 5(1), 37–45. https://jeti.thewsu.org/index.php/cieti/article/view/103
Hu, H. C. M., & Nassaji, H. (2014). Lexical inferencing strategies: The case of successful versus less successful inferencers. System, 45, 27–38. https://doi.org/10.1016/j.system.2014.04.004
Hwang, G. J., Xie, H., Wah, B. W., Gasevic, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, Article 100001. https://doi.org/10.1016/j.caeai.2020.100001
Jalongo, M. R., & Hirsh, R. A. (2010). Understanding reading anxiety: New insights from neuroscience. Early Childhood Education Journal, 37(6), 431–435. https://doi.org/10.1007/s10643-010-0381-5
Jeon, J., Lee, S., & Choe, H. (2023). Beyond ChatGPT: A conceptual framework and systematic review of speech-recognition chatbots for language learning. Computers & Education, 206, Article 104898. https://doi.org/10.1016/j.compedu.2023.104898
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274. https://doi.org/10.1016/j.lindif.2023.102274
Keren, G. (2014). Between- or within-subjects design: A methodological dilemma. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Vol. 1. Methodological issues (pp. 257–272). Psychology Press. https://doi.org/10.4324/9781315799582-10
Kim, S., Park, J., & Lee, H. (2019). Automated essay scoring using a deep learning model. Journal of Educational Technology Development and Exchange, 2(1), 1–17.
Kispal, A. (2008). Effective teaching of inference skills for reading: Literature review (Research report DCSFRR031). National Foundation for Educational Research. https://eric.ed.gov/?id=ED501868
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868
Laufer, B. (2020). Lexical coverages, inference unknown words and reading comprehension: How are they related? TESOL Quarterly, 54(4), 1076–1085. https://doi.org/10.1002/tesq.3004
Lottridge, S. M., Nicewander, W. A., & Mitzel, H. C. (2011). A comparison of paper and online tests using a within-subjects design and propensity score matching study. Multivariate Behavioral Research, 46(3), 544–566. https://doi.org/10.1080/00273171.2011.569408
Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26–29. https://doi.org/10.1108/LHTN-01-2023-0009
Meymeh, M. H., Rashtchi, M., & Mohseni, A. (2019). Anxiety effect: A case of text modification and the effect of high and low anxiety levels on medical students’ comprehension performance. Journal of Paramedical Sciences, 10(1), 20–26. https://doi.org/10.22037/jps.v10i1.24111
Mhlanga, D. (2023, February 11). Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. SSRN. https://doi.org/10.2139/ssrn.4354422
Mikami, H. (2019). Reading anxiety scales: Do they measure the same construct? Reading in a Foreign Language, 31(2), 249–268. https://eric.ed.gov/?id=EJ1232213
Oakhill, J., & Cain, K. (2007). Issues of causality in children’s reading comprehension. In D. S. McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and technologies (pp. 47–72). Lawrence Erlbaum Associates Publishers.
Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, Article 100020. https://doi.org/10.1016/j.caeai.2021.100020
Pang, G., Shen, C., Cao, L., Hengel, A. V. D. (2021). Deep learning for anomaly detection: A review. ACM Computing Surveys, 54(2), 1–38. https://doi.org/10.1145/3439950
Pavlik, J. V. (2023). Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education. Journalism & Mass Communication Educator, 78(1), 84–93. https://doi.org/10.1177/10776958221149577
Pogla, M. (2023, April 24). ChatGPT: Optimizing language models for dialogue. AutoGPT. https://autogpt.net/chatgpt-optimizing-language-models-for-dialogue/
Rospigliosi, P. A. (2023). Artificial intelligence in teaching and learning: What questions should we ask of ChatGPT? Interactive Learning Environments, 31(1), 1–3. https://doi.org/10.1080/10494820.2023.2180191
Saito, Y., Garza, T. J., & Horwitz, E. K. (1999). Foreign language reading anxiety. Modern Language Journal, 83(2), 202–218. https://doi.org/10.1111/0026-7902.00016
Soma, R., Mukminin, A., & Noprival, A. M. (2015). Toward a better preparation of student teachers’ reading skill: The SQ3R strategy with authentic and simplified texts on reading literacy and vocabulary mastery. Journal of Education and Learning, 9(2), 125–134. https://doi.org/10.11591/edulearn.v9i2.1527
Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning and Teaching, 6(1), 31–40. https://doi.org/10.37074/jalt.2023.6.1.17
Susnjak, T., & Maddigan, P. (2023). Forecasting patient flows with pandemic induced concept drift using explainable machine learning. EPJ Data Science, 12, Article 11. https://doi.org/10.1140/epjds/s13688-023-00387-5
Taecharungroj, V. (2023). What can ChatGPT do? Analyzing early reactions to the innovative AI chatbot on Twitter. Big Data and Cognitive Computing, 7(1), Article 35. https://doi.org/10.3390/bdcc7010035
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, Article 15. https://doi.org/10.1186/s40561-023-00237-x
Tomlinson, B. (2004). Materials development in language teaching (2nd ed.). Cambridge University Press. https://doi.org/10.1017/9781139042789
Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331(6023), 1447–1451. https://doi.org/10.1126/science.1198364
Wesche, P. M. B., & Paribakht, P. T. S. (2009). Lexical inferencing in a first and second language: Cross-linguistic dimensions. Multilingual Matters. https://doi.org/10.21832/9781847692245
Yang, S. J., Ogata, H., Matsui, T., & Chen, N. S. (2021). Human-centered artificial intelligence in education: Seeing the invisible through the visible. Computers and Education: Artificial Intelligence, 2, Article 100008. https://doi.org/10.1016/j.caeai.2021.100008
Yi, Y., Cho, S., & Jang, J. (2022). Methodological innovations in examining digital literacies in applied linguistics research. TESOL Quarterly, 56(3), 1052–1062. https://doi.org/10.1002/tesq.3140
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
