“To Use or Not to Use?” A Mixed-Methods Study on the Determinants of EFL College Learners’ Behavioral Intention to Use AI in the Distributed Learning Context

Hanwei Wu; Yunsong Wang; Yongliang Wang

doi:10.19173/irrodl.v25i3.7708

Authors

Hanwei Wu School of Foreign Studies, Hunan Normal University, Changsha, China https://orcid.org/0000-0001-7247-5484
Yunsong Wang School of Foreign Languages and Cultures, Nanjing Normal University, Nanjing, China https://orcid.org/0009-0004-8918-7613
Yongliang Wang School of Foreign Studies, North China University of Water Resources and Electric Power, Zhengzhou, China https://orcid.org/0000-0002-4672-8481

DOI:

https://doi.org/10.19173/irrodl.v25i3.7708

Keywords:

artificial intelligence, AI, EFL college learner, behavioral intention, distributed learning

Abstract

Artificial intelligence (AI) offers new possibilities for English as a foreign language (EFL) learners to enhance their learning outcomes, provided that they have access to AI applications. However, little is written about the factors that influence their intention to use AI in distributed EFL learning contexts. This mixed-methods study, based on the technology acceptance model (TAM), examined the determinants of behavioral intention to use AI among 464 Chinese EFL college learners. As to quantitative data, a structural equation modelling (SEM) approach using IBM SPSS Amos (Version 24) produced some important findings. First, it was revealed that perceived ease of use significantly and positively predicts perceived usefulness and attitude toward AI. Second, attitude toward AI significantly and positively predicts behavioral intention to use AI. However, contrary to the TAM assumptions, perceived usefulness does not significantly predict either attitude toward AI or behavioral intention to use AI. Third, mediation analyses suggest that perceived ease of use has a significant and positive impact on students’ behavioral intention to use AI through their attitude toward AI, rather than through perceived usefulness. As to qualitative data, semi-structured interviews with 15 learners, analyzed by the software MAXQDA 2022, provide a nuanced understanding of the statistical patterns. This study also discusses the theoretical and pedagogical implications and suggests directions for future research.

References

An, X., Chai, C. S., Li, Y., Zhou, Y., Shen, X., Zheng, C., & Chen, M. (2022). Modeling English teachers’ behavioral intention to use artificial intelligence in middle schools. Education and Information Technologies, 28, 5187–5208. https://doi.org/10.1007/s10639-022-11286-z

An, X., Chai, C. S., Li, Y., Zhou, Y., & Yang, B. (2023). Modeling students’ perceptions of artificial intelligence assisted language learning. Computer Assisted Language Learning, 1–22. https://doi.org/10.1080/09588221.2023.2246519

Ayanwale, M. A., Sanusi, I. T., Adelana, O. P., Aruleba, K. D., & Oyelere, S. S. (2022). Teachers’ readiness and intention to teach artificial intelligence in schools. Computers and Education: Artificial Intelligence, 3, Article 100099. https://www.doi.org/10.1016/j.caeai.2022.100099

Ayedoun, E., Hayashi, Y., & Seta, K. (2019). Adding communicative and affective strategies to an embodied conversational agent to enhance second language learners’ willingness to communicate. International Journal of Artificial Intelligence in Education, 29(1), 29–57. https://doi.org/10.1007/s40593-018-0171-6

Barrot, J. S. (2022). Integrating technology into ESL/EFL writing through Grammarly. RELC Journal, 53(3), 764–768. https://doi.org/10.1177/0033688220966632

Bearman, M., Ryan, J., & Ajjawi, R. (2023). Discourses of artificial intelligence in higher education: A critical literature review. Higher Education, 86(2), 369–385. https://doi.org/10.1007/s10734-022-00937-2

Betal, A. (2023). Enhancing second language acquisition through artificial intelligence (AI): Current insights and future directions. Journal for Research Scholars and Professionals of English Language Teaching, 7(39). https://doi.org/10.54850/jrspelt.7.39.003

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Castañeda, L., & Selwyn, N. (2018). More than tools? Making sense of the ongoing digitizations of higher education. International Journal of Educational Technology in Higher Education, 15, Article 22. https://doi.org/10.1186/s41239-018-0109-y

Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A. (2018). Artificial intelligence trends in education: A narrative overview. Procedia Computer Science, 136, 16–24. https://doi.org/10.1016/j.procs.2018.08.233

Chen, H., & Pan, J. (2022). Computer or human: A comparative study of automated evaluation scoring and instructors’ feedback on Chinese college students’ English writing. Asian-Pacific Journal of Second and Foreign Language Education, 7, Article 34. https://doi.org/10.1186/s40862-022-00171-4

Chen, X., Xie, H., & Hwang, G.-J. (2020). A multi-perspective study on artificial intelligence in education: Grants, conferences, journals, software tools, institutions, and researchers. Computers and Education: Artificial Intelligence, 1, Article 100005. https://doi.org/10.1016/j.caeai.2020.100005

Creswell, J. W. (2014). A concise introduction to mixed methods research. SAGE Publications.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008

Derakhshan, A., Wang, Y., Wang, Y., & Ortega-Martín, J. L. (2023). Towards innovative research approaches to investigating the role of emotional variables in promoting language teachers’ and learners’ mental health. International Journal of Mental Health Promotion, 25(7), 823–832. https://doi.org/10.32604/ijmhp.2023.029877

Divekar, R. R., Drozdal, J., Chabot, S., Zhou, Y., Su, H., Chen, Y., Zhu, H., Hendler, J. A., & Braasch, J. (2022). Foreign language acquisition via artificial intelligence and extended reality: Design and evaluation. Computer Assisted Language Learning, 35(9), 2332–2360. https://doi.org/10.1080/09588221.2021.1879162

Dizon, G. (2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language, Learning and Technology, 24(1), 16–26. https://doi.org/10125/44705

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312

Gado, S., Kempen, R., Lingelbach, K., & Bipp, T. (2022). Artificial intelligence in psychology: How can we enable psychology students to accept and use artificial intelligence? Psychology Learning & Teaching, 21(1), 37–56. https://doi.org/10.1177/14757257211037149

Gao, Q., Yan, Z., Zhao, C., Pan, Y., & Mo, L. (2014). To ban or not to ban: Differences in mobile phone policies at elementary, middle, and high schools. Computers in Human Behavior, 38, 25–32. https://doi.org/10.1016/j.chb.2014.05.011

Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression based approach. Guilford Press.

Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Huang, F., & Teo, T. (2020). Influence of teacher-perceived organisational culture and school policy on Chinese teachers’ intention to use technology: An extension of technology acceptance model. Educational Technology Research and Development, 68(3), 1547–1567. https://doi.org/10.1007/s11423-019-09722-y

Huang, H.-M., & Liaw, S.-S. (2005). Exploring users’ attitudes and intentions toward the web as a survey tool. Computers in Human Behavior, 21(5), 729–743. https://doi.org/10.1016/j.chb.2004.02.020

Janbi, N., Katib, I., & Mehmood, R. (2023). Distributed artificial intelligence: Taxonomy, review, framework, and reference architecture. Intelligent Systems With Applications, 18, Article 200231. https://doi.org/10.1016/j.iswa.2023.200231

Jiang, R. (2022). How does artificial intelligence empower EFL teaching and learning nowadays? A review on artificial intelligence in the EFL context. Frontiers in Psychology, 13, Article 1049401. https://doi.org/10.3389/fpsyg.2022.1049401

Kelly, S., Kaye, S.-A., & Oviedo-Trespalacios, O. (2023). What factors contribute to the acceptance of artificial intelligence? A systematic review. Telematics and Informatics, 77, Article 101925. https://doi.org/10.1016/j.tele.2022.101925

Klotz, A. C., Swider, B. W., & Kwon, S. H. (2023). Back-translation practices in organizational research: Avoiding loss in translation. Journal of Applied Psychology, 108(5), 699–727. https://doi.org/10.1037/apl0001050

Klimova, B., Pikhart, M., Benites, A. D., Lehr, C., & Sanchez-Stockhammer, C. (2023). Neural machine translation in foreign language teaching and learning: A systematic review. Education and Information Technologies, 28(1), 663–682. https://doi.org/10.1007/s10639-022-11194-2

Kline, R. B. (2016). Principles and practice of structural equation modeling (6th ed.). Guilford Press.

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023a). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023b). Exploring generative artificial intelligence preparedness among university language instructors: A case study. Computers and Education: Artificial Intelligence, 5, Article 100156. https://doi.org/10.1016/j.caeai.2023.100156

Kuddus, K. (2022). Artificial intelligence in language learning: Practices and prospects. In A. Mire, S. Malik, & A. K. Tyagi (Eds.), Advanced analytics and deep learning models (pp. 3–18). Wiley. https://doi.org/10.1002/9781119792437.ch1

Li, K. (2023). Determinants of college students’ actual use of AI-based systems: An extension of the technology acceptance model. Sustainability, 15(6), Article 5221. https://www.mdpi.com/2071-1050/15/6/5221

Liu, G., & Ma, C. (2024). Measuring EFL learners’ use of ChatGPT in informal digital learning of English based on the technology acceptance model. Innovation in Language Learning and Teaching, 18(2), 125–138. https://doi.org/10.1080/17501229.2023.2240316

Liu, G. L., & Wang, Y. (2024). Modeling EFL teachers’ intention to integrate informal digital learning of English (IDLE) into the classroom using the theory of planned behavior. System, 120, Article 103193. https://doi.org/10.1016/j.system.2023.103193

Muftah, M., Al-Inbari, F. A. Y., Al-Wasy, B. Q., & Mahdi, H. S. (2023). The role of automated corrective feedback in improving EFL learners’ mastery of the writing aspects. Psycholinguistics, 34(2), 82–109. https://doi.org/10.31470/2309-1797-2023-34-2-82-109

Namaziandost, E., Hashemifardnia, A., Bilyalova, A. A., Fuster-Guillén, D., Palacios Garay, J. P., Diep, L. T. N., Ismail, H., Sundeeva, L. A., Hibana, & Rivera-Lozada, O. (2021). The effect of WeChat-based online instruction on EFL learners’ vocabulary knowledge. Education Research International, 2021, Article 8825450. https://doi.org/10.1155/2021/8825450

Namaziandost, E., Razmi, M. H., Atabekova, A., Shoustikova, T., & Kussanova, B. H. (2021). An account of Iranian intermediate EFL learners’ vocabulary retention and recall through spaced and massed distribution instructions. Journal of Education, 203(2), 275–284. https://doi.org/10.1177/00220574211031949

Noar, S. M. (2003). The role of structural equation modeling in scale development. Structural Equation Modeling: A Multidisciplinary Journal, 10(4), 622–647. https://doi.org/10.1207/S15328007SEM1004_8

Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, Article 100020. https://doi.org/10.1016/j.caeai.2021.100020

Rezai, A. (2023). Investigating the association of informal digital learning of English with EFL learners’ intercultural competence and willingness to communicate: A SEM study. BMC Psychology, 11, Article 314(2023). https://doi.org/10.1186/s40359-023-01365-2

Schepman, A., & Rodway, P. (2022). The General Attitudes towards Artificial Intelligence Scale (GAAIS): Confirmatory validation and associations with personality, corporate distrust, and general trust. International Journal of Human-Computer Interaction, 39(13), 2724–2741. https://doi.org/10.1080/10447318.2022.2085400

Selwyn, N. (2016). Is technology good for education? Polity Press.

Shortt, M., Tilak, S., Kuznetcova, I., Martens, B., & Akinkuolie, B. (2023). Gamification in mobile-assisted language learning: A systematic review of Duolingo literature from public release of 2012 to early 2020. Computer Assisted Language Learning, 36(3), 517–554. https://doi.org/10.1080/09588221.2021.1933540

Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7(4), 422–445.

Siyam, N. (2019). Factors impacting special education teachers’ acceptance and actual use of technology. Education and Information Technologies, 24(3), 2035–2057. https://doi.org/10.1007/s10639-018-09859-y

Teo, T., Huang, F., & Hoi, C. K. W. (2017). Explicating the influences that explain intention to use technology among English teachers in China. Interactive Learning Environments, 26(4), 460–475. https://doi.org/10.1080/10494820.2017.1341940

Ulla, M. B., Perales, W. F., & Busbus, S. O. (2023). ‘To generate or stop generating response’: Exploring EFL teachers’ perspectives on ChatGPT in English language teaching in Thailand. Learning: Research and Practice, 9(2), 168–182. https://doi.org/10.1080/23735082.2023.2257252

Ursavaş, Ö. F. (2022). Technology acceptance model: History, theory, and application. In Ö. F. Ursavaş (Ed.), Conducting technology acceptance research in education: Theory, models, implementation, and analysis (pp. 57–91). Springer International Publishing. https://doi.org/10.1007/978-3-031-10846-4_4

Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273–315. https://doi.org/10.1111/j.1540-5915.2008.00192.x

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540

Wang, Y., Yu, L., & Yu, Z. (2022). An extended CCtalk technology acceptance model in EFL education. Education and Information Technologies, 27, 6621–6640. https://doi.org/10.1007/s10639-022-10909-9

Wang, Y. L., Wang, Y. X., Pan, Z. W., & Ortega-Martín, J. L. (2023). The predicting role of EFL students’ achievement emotions and technological self-efficacy in their technology acceptance. The Asia-Pacific Education Researcher, 2023. https://doi.org/10.1007/s40299-023-00750-0

Yang, H., Gao, C., & Shen, H.-z. (2023). Learner interaction with, and response to, AI-programmed automated writing evaluation feedback in EFL writing: An exploratory study. Education and Information Technologies, 29, 3837–3858. https://doi.org/10.1007/s10639-023-11991-3

Zhang, K., & Aslan, A. B. (2021). AI technologies for education: Recent research and future directions. Computers and Education: Artificial Intelligence, 2, Article 100025. https://doi.org/10.1016/j.caeai.2021.100025

Zhi, R., Wang, Y., & Wang, Y. (2023). The role of emotional intelligence and self-efficacy in EFL teachers’ technology adoption. The Asia-Pacific Education Researcher, 2023. https://doi.org/10.1007/s40299-023-00782-6