Video Lectures With AI-Generated Instructors: Low Video Engagement, Same Performance as Human Instructors
DOI:
https://doi.org/10.19173/irrodl.v25i3.7815Keywords:
generative AI, human instructor, AI-generated instructor, video lecture, video engagementAbstract
Via AI video generators, it is possible to create educational videos with humanistic instructors by simply providing a script. The characteristics of video types and features of instructors in videos impact video engagement and, consequently, performance. This study aimed to compare the impact of human instructors and AI-generated instructors in video lectures on video engagement and academic performance. Additionally, the study aimed to examine students’ opinions on both types of videos. Convergent-parallel approach mixed method was used in this study. A total of 108 undergraduate students participated: 48 in the experimental group, 52 in the control group, and eight in the focus group. While the experimental group (AI-generated instructor) and control group (human instructor) watched 10 minutes of two videos each in two weeks, the students in the focus group watched both types of videos with human and AI-generated instructors. Data were collected through the Video Engagement Scale (VES) after the experimental process, and the Academic Performance Test as a pretest and posttest was administered in both groups. The findings of the experimental part revealed that learners’ video engagement was higher in the course with the human instructor compared to the course with the AI-generated instructor. However, the instructor type did not have a significant effect on academic performance. The results based on the qualitative part showed that students thought the AI-generated instructor caused distraction, discomfort, and disconnectedness. However, when the video lesson topic was interesting or when students focused on the video with the intention of learning, these feelings could be ignored. In conclusion, even in today’s conditions, there is no difference in performance between human and AI-generated instructors. As AI technology continues to develop, the difference in engagement is expected to disappear, and AI-generated instructors could be used effectively in video lectures.
References
Alemdag, E. (2022). Effects of instructor-present videos on learning, cognitive load, motivation, and social presence: A meta-analysis. Education and Information Technologies, 27(9), 12713–12742. https://doi.org/10.1007/s10639-022-11154-w
Beege, M., Schroeder, N. L., Heidig, S., Rey, G. D., & Schneider, S. (2023). The instructor presence effect and its moderators in instructional video: A series of meta-analyses. Educational Research Review, Article 100564. https://doi.org/10.1016/j.edurev.2023.100564
Bozkurt, A. (2023). Unleashing the potential of generative AI, conversational agents and chatbots in educational praxis: A systematic review and bibliometric analysis of GenAI in education. Open Praxis, 15(4), 261–270. https://doi.org/10.55982/openpraxis.15.4.609
Büyüköztürk, Ş. (1998). Analysis of covariance (A comparative study with analysis of variance). Ankara University Journal of Faculty of Educational Sciences (JFES), 31(1), 93–100.
Cambridge Dictionary. (n.d.). Uncanny valley. In Cambridge online dictionary. Retrieved May 3, 2024, from https://dictionary.cambridge.org/dictionary/english/uncanny-valley.
Chen, H.-T. M., & Thomas, M. (2020). Effects of lecture video styles on engagement and learning. Educational Technology Research and Development, 68(5), 2147–2164. https://eric.ed.gov/?id=EJ1269149
Chiou, E. K., Schroeder, N. L., & Craig, S. D. (2020). How we trust, perceive, and learn from virtual humans: The influence of voice quality. Computers & Education, 146, Article 103756. https://doi.org/10.1016/j.compedu.2019.103756
Creswell, J. W., & Clark, V. P. (2011). Mixed methods research. SAGE Publications.
Crook, C., & Schofield, L. (2017). The video lecture. The Internet and Higher Education, 34, 56–64. https://doi.org/10.1016/j.iheduc.2017.05.003
Dai, L., Jung, M. M., Postma, M., & Louwerse, M. M. (2022). A systematic review of pedagogical agent research: Similarities, differences and unexplored aspects. Computers & Education, Article 104607. https://doi.org/10.1016/j.compedu.2022.104607
Daniels, D., & Lee, J. S. (2022). The impact of avatar teachers on student learning and engagement in a virtual learning environment for online STEM courses. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies. Novel technological environments. HCII 2022. Lecture Notes in Computer Science, Vol. 13329 (pp. 158–175). Springer International Publishing. https://doi.org/10.1007/978-3-031-05675-8_13
Dao, X.-Q., Le, N.-B., & Nguyen, T.-M.-T. (2021). AI-powered MOOCs: Video lecture generation. In X. Jiang (Chair), IVSP ’21: Proceedings of the 2021 3rd International Conference on Image, Video and Signal Processing (pp. 95–102). Association for Computing Machinery. https://doi.org/10.1145/3459212.3459227
Davis, R. O., Vincent, J., & Park, T. (2019). Reconsidering the voice principle with non-native language speakers. Computers & Education, 140, Article 103605. https://doi.org/10.1016/j.compedu.2019.103605
Deryakulu, D., Sancar, R., & Ursavaş, Ö. F. (2019). Adaptation, validity and reliability study of the video engagement scale. The Journal of Educational Technology Theory and Practice, 9(1), 154–168. https://doi.org/10.17943/etku.439097
Ferre, A. G. (2023, November 27). Videos con inteligencia artificial: mejores herramientas para impactor. [Videos with artificial intelligence: better tools to impact]. Vidext Technologies. https://blog.vidext.io/videos-con-inteligencia-artificial-mejores-herramientas-para-impactar
Fiorella, L., & Mayer, R. E. (2018). What works and doesn’t work with instructional video. Computers in Human Behavior, 89, 465–470. https://doi.org/10.1016/j.chb.2018.07.015
García-Peñalvo, F. J., & Vázquez-Ingelmo, A. (2023). What do we mean by GenAI? A systematic mapping of the evolution, trends, and techniques involved in Generative AI. International Journal of Interactive Multimedia and Artificial Intelligence, 8(4), 7–16. DOI:10.9781/ijimai.2023.07.006
Grassini, S. (2023). Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Education Sciences, 13(7), Article 692. https://doi.org/10.3390/educsci13070692
Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In M. Sahami (Chair), L@S ’14: Proceedings of the first ACM Conference on Learning@Scale (pp. 41–50). Association for Computing Machinery. https://doi.org/10.1145/2556325.2566239
Henderson, M. L., & Schroeder, N. L. (2021). A systematic review of instructor presence in instructional videos: Effects on learning and affect. Computers and Education Open, 2, Article 100059. https://doi.org/10.1016/j.caeo.2021.100059
Hew, K. F. (2018). Unpacking the strategies of ten highly rated MOOCs: Implications for engaging students in large online courses. Teachers College Record, 120(1), 1–40. https://eric.ed.gov/?id=EJ1162815
Higgins, D., Zhan, Y., Cowan, B. R., & McDonnell, R. (2023, August). Investigating the effect of visual realism on empathic responses to emotionally expressive virtual humans. In A. Chapiro, A. Robb, F. Durupinar, Q. Sun, & L. Buck (Eds.), SAP ’23: ACM Symposium on Applied Perception 2023 (pp. 1–7). Association for Computing Machinery. https://doi.org/10.1145/3605495.3605799
Horovitz, T., & Mayer, R. E. (2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, Article 106724. https://doi.org/10.1016/j.chb.2021.106724
Hu, L., & Wang, Y. (2023). The predicting role of EFL teachers’ immediacy behaviors in students’ willingness to communicate and academic engagement. BMC Psychology, 11, Article 318. https://doi.org/10.1186/s40359-023-01378-x
Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Sage Publications.
Lambert, J., & Stevens, M. (2023, September 9). ChatGPT and generative AI technology: A mixed bag of concerns and new opportunities. Computers in the Schools, 1–25. https://doi.org/10.1080/07380569.2023.2256710
Leiker, D., Gyllen, A. R., Eldesouky, I., & Cukurova, M. (2023). Generative AI for learning: Investigating the potential of learning videos with synthetic virtual instructors. In N. Wang, G. Rebolledo-Mendez, N. Matsuda, O. C. Santos, & V. Dimitrova (Eds.), Artificial Intelligence in Education: 24th International Conference proceedings. Posters and late breaking results, workshops and tutorials, industry and innovation tracks, practitioners, doctoral consortium and blue sky. (pp. 523–529). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-36336-8_81
Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), Article 410. https://doi.org/10.3390/educsci13040410
Manasrah, A., Masoud, M., & Jaradat, Y. (2021). Short videos, or long videos? A study on the ideal video length in online learning. In K. M. Jaber (Ed.), 2021 International Conference on Information Technology (ICIT) (pp. 366–370). IEEE. https://www.doi.org/10.1109/ICIT52682.2021.9491115
Marín Orozco, D. (2024). Aplicación de inteligencias artificiales en los procesos de diseño y creación en la malla curricular del programa de Diseño Visual [Application of artificial intelligence in design and creation processes in the curriculum of the visual design program] [Doctoral dissertation, University of Caldas]. Universidad de Caldas Repositorio Institucional. https://repositorio.ucaldas.edu.co/handle/ucaldas/19777
Miller, E. J., Foo, Y. Z., Mewton, P., & Dawel, A. (2023). How do people respond to computer-generated versus human faces? A systematic review and meta-analyses. Computers in Human Behavior Reports, Article 100283. https://doi.org/10.1016/j.chbr.2023.100283
Nightingale, S. J., & Farid, H. (2022). AI-synthesized faces are indistinguishable from real faces and more trustworthy. Proceedings of the National Academy of Sciences, 119(8), Article e2120481119. https://doi.org/10.1073/pnas.2120481119
Ozan, O., & Ozarslan, Y. (2016). Video lecture watching behaviors of learners in online courses. Educational Media International, 53(1), 27–41. https://doi.org/10.1080/09523987.2016.1189255
Pan, Z., Wang, Y., & Derakhshan, A. (2023). Unpacking Chinese EFL students’ academic engagement and psychological well-being: The roles of language teachers’ affective scaffolding. Journal of Psycholinguistic Research, 52(5), 1799–1819. https://doi.org/10.1007/s10936-023-09974-z
Pellas, N. (2023). The influence of sociodemographic factors on students’ attitudes toward AI-generated video content creation. Smart Learning Environments, 10(1), Article 57. https://doi.org/10.1186/s40561-023-00276-4
Pelletier, K., Robert, J., Muscanell, N., McCormack, M., Reeves, J., Arbino, N., & Grajek, S. (with Birdwell, T., Liu, D., Mandernach, J., Moore, A., Porcaro, A., Rutledge, R., & Zimmern, J.). (2023). EDUCAUSE horizon report, teaching and learning edition. EDUCAUSE. https://library.educause.edu/resources/2023/5/2023-educause-horizon-report-teaching-and-learning-edition
Roulston, K. (2010). Reflective interviewing: A guide to theory and practice. Sage Publications. https://doi.org/10.4135/9781446288009
Seo, K., Dodson, S., Harandi, N. M., Roberson, N., Fels, S., & Roll, I. (2021). Active learning with online video: The impact of learning context on engagement. Computers & Education, 165, Article 104132. https://doi.org/10.1016/j.compedu.2021.104132
Shoufan, A. (2019). What motivates university students to like or dislike an educational online video? A sentimental framework. Computers & Education, 134, 132–144. https://doi.org/10.1016/j.compedu.2019.02.008
Soffer, T., & Cohen, A. (2019). Students’ engagement characteristics predict success and completion of online courses. Journal of Computer Assisted Learning, 35(3), 378–389. https://doi.org/10.1111/jcal.12340
Urhan, S., & Kocadere, S. A. (2024). The effect of video lecture types on the computational problem-solving performances of students. Educational Technology & Society, 27(1), 117–133. https://doi.org/10.30191/ETS.202401_27(1).RP08
Vallis, C., Wilson, S., Gozman, D., & Buchanan, J. (2023, June 6). Student perceptions of AI-generated avatars in teaching business ethics: We might not be impressed. Postdigital Science and Education, 1–19. https://doi.org/10.1007/s42438-023-00407-7
Verma, N., Getenet, S., Dann, C., & Shaik, T. (2023). Characteristics of engaging teaching videos in higher education: A systematic literature review of teachers’ behaviours and movements in video conferencing. Research and Practice in Technology Enhanced Learning, 18, Article 040. https://doi.org/10.58459/rptel.2023.18040
Visser, L. N. C., Hillen, M. A., Verdam, M. G. E., Bol, N., de Haes, H. C. J. M., & Smets, E. M. A. (2016). Assessing engagement while viewing video vignettes; Validation of the Video Engagement Scale (VES). Patient Education and Counseling, 99(2), 227–235. https://doi.org/10.1016/j.pec.2015.08.029
Wang, Y., & Kruk, M. (2024). Modeling the interaction between teacher credibility, teacher confirmation, and English major students’ academic engagement: A sequential mixed-methods approach. Studies in Second Language Learning and Teaching, 1–31. https://doi.org/10.14746/ssllt.38418
Yıldırım, A., & Şimşek, H. (2006). Qualitative research methods in social sciences. Seçkin Press.
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
