A Systematic Review of Questionnaire-Based Quantitative Research on MOOCs
DOI:
https://doi.org/10.19173/irrodl.v22i2.5208Keywords:
MOOC, factors-goals graph (F-G graph), questionnaire-based survey, quantitative analysis, research topicsAbstract
Massive open online courses (MOOCs) have attracted much interest from educational researchers and practitioners around the world. There has been an increase in empirical studies about MOOCs in recent years, most of which used questionnaire surveys and quantitative methods to collect and analyze data. This study explored the research topics and paradigms of questionnaire-based quantitative research on MOOCs by reviewing 126 articles available in the Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) databases from January 2015 to August 2020. This comprehensive overview showed that: (a) the top three MOOC research topics were the factors influencing learners’ performance, dropout rates and continuance intention to use MOOCs, and assessing MOOCs; (b) for these three topics, many studies designed questionnaires by adding new factors or adjustments to extant theoretical models or survey instruments; and (c) most researchers used descriptive statistics to analyze data, followed by the structural equation model, and reliability and validity analysis. This study elaborated on the relationship of research topics and key factors in the research models by building factors-goals (F-G) graphs. Finally, we proposed some directions and recommendations for future research on MOOCs.
References
Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl & J. Beckmann (Eds.), Action control: From cognition to behavior (pp. 11–39). Springer.
Albert, S., Mercedes, G. S., & Terry, A. (2015). Meta-Analysis of the research about MOOC during 2013-2014. Educacion Xx1, 18(2). https://doi.org/10.5944/educxx1.14808
Ang, K. L. M., Ge, F. L., & Seng, K. P. (2020). Big educational data & analytics: Survey, architecture and challenges. IEEE Access, 8, 116392–116414. https://doi.org/10.1109/ACCESS.2020.2994561
Babori, A., Abdelkarim, Z., & Fassi, H. F. (2019). Research on MOOCs in major referred journals: The role and place of content. International Review of Research in Open and Distributed Learning, 20(3). https://doi.org/10.19173/irrodl.v20i4.4385
Blum-Smith, S., Yurkofsky, M. M., & Brennan, K. (2021). Stepping back and stepping in: Facilitating learner-centered experiences in MOOCs. Computers & Education, 160, 104042. https://doi.org/10.1016/j.compedu.2020.104042
Botero, G. G., Questier, F., Cincinnato, S., He, T., & Zhu, C. (2018). Acceptance and usage of mobile assisted language learning by higher education students. Journal of Computing in Higher Education, 5, 1–26. https://doi.org/10.1007/s12528-018-9177-1
Bozkurt, A., Akgun-Ozbek, E., & Zawacki-Richter, O. (2017). Trends and patterns in massive open online courses: Review and content analysis of research on MOOCs (2008–2015). International Review of Research in Open and Distributed Learning, 18(5), 118–147. http://doi.org/10.19173/irrodl.v18i5.3080
Cagiltay, N. E., Cagiltay, K., & Celik, B. (2020). An analysis of course characteristics, learner characteristics, and certification rates in MITx MOOCs. International Review of Research in Open and Distributed Learning, 21(3), 121–139. http://doi.org/10.19173/irrodl.v21i3.4698
Carlos, A. H., Iria, E. A., Mar, P. S., Carlos, D. K., & Carmen, F. P. (2017). Understanding learners’ motivation and learning strategies in MOOCs. International Review of Research in Open and Distributed Learning, 18(3), 119–137. http://doi.org/10.19173/irrodl.v18i3.2996
Castano-Munoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education, 29(1), 28–46. http://doi.org/10.1007/s12528-016-9123-z
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-339. http://doi.org/10.2307/249008
de Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32(3), 218–231. http://doi.org/10.1111/jcal.12130
Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627–626. http://doi.org/10.1037/0033-2909.125.6.627
Ding, Y., & Zhao, T. (2020). Emotions, engagement, and self-perceived achievement in a small private online course. Journal of Computer Assisted Learning, 36(4), 449–457. https://doi.org/10.1111/jcal.12410
Donitsa-Schmidt, S., & Topaz, B. (2018). Massive open online courses as a knowledge base for teachers. Journal of Education for Teaching, 44(5), 608–620. https://doi.org/10.1080/02607476.2018.1516350
Durksen, T. L., Chu, M. W., Ahmad, Z. F., Radil, A. I., & Daniels, L. M. (2016). Motivation in a MOOC: A probabilistic analysis of online learners’ basic psychological needs. Social Psychology of Education, 19(2), 241–260. https://doi.org/10.1007/s11218-015-9331-9
Fang, J. W., Hwang, G. J., & Chang, C. Y. (2019). Advancement and the foci of investigation of MOOCs and open online courses for language learning: A review of journal publications from 2009 to 2018. Interactive Learning Environments, 28, 1–19. http://doi.org/10.1080/10494820.2019.1703011
Farhan, W., Razmak, J., Demers, S., & Laflamme, S. (2019). E-learning systems versus instructional communication tools: Developing and testing a new e-learning user interface from the perspectives of teachers and students. Technology in Society, 59, 101192. http://doi.org/10.1016/j.techsoc.2019.101192
Fryer, L. K., & Bovee, H. N. (2018). Staying motivated to e-learn: Person- and variable-centred perspectives on the longitudinal risks and support. Computers & Education, 120, 227–240. http://doi.org/10.1016/j.compedu.2018.01.006
Gallagher, S. E., & Savage, T. (2016). Comparing learner community behavior in multiple presentations of a massive open online course. Journal of Computing in Higher Education, 28(3), 358–369. http://doi.org/10.1007/s12528-016-9124-y
Gan, T. (2018). Construction of security system of flipped classroom based on MOOC in teaching quality control. Educational Sciences: Theory & Practice, 18(6), 2707–2717. http://doi.org/10.12738/estp.2018.6.170
Geng, S., Niu, B., Feng, Y. Y., & Huang, M. J. (2020). Understanding the focal points and sentiment of learners in MOOC reviews: A machine learning and SC-LIWC-based approach. British Journal of Educational Technology, 51(5), 1785–1803. http://doi.org/10.1111/bjet.12999
Goodhue, D. L., Klein, B. D., & March, S. T. (2000). User evaluations of IS as surrogates for objective performance. Information and Management, 38(2), 87–101. http://doi.org/10.1016/S0378-7206(00)00057-4
Hone, K. S., & El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98, 157–168. http://doi.org/10.1016/j.compedu.2016.03.016
Hung, C. Y., Sun, J. C. Y., & Liu, J. Y. (2019). Effects of flipped classrooms integrated with MOOCs and game-based learning on the learning motivation and outcomes of students from different backgrounds. Interactive Learning Environments, 27(8), 1028–1046. http://doi.org/10.1080/10494820.2018.1481103
Jansen, R. S., van Leeuwen, A., Janssen, J., Kester, L., & Kalz, M. (2017). Validation of the self-regulated online learning questionnaire. Journal of Computing in Higher Education, 29(1), 6–27. http://doi.org/10.1007/s12528-016-9125-x
Jo, D. H. (2018). Exploring the determinants of MOOCs continuance intention. Ksii Transactions on Internet and Information Systems, 12(8), 3992–4005. http://doi.org/10.3837/tiis.2018.08.024
Joo, Y. J., So, H. J., & Kim, N. H. (2018). Examination of relationships among students’ self-determination, technology acceptance, satisfaction, and continuance intention to use k-MOOCs. Computers & Education, 122, 260–272. http://doi.org/10.1016/j.compedu.2018.01.003
Jung, E., Kim, D., Yoon, M., Park, S. H., & Oakley, B. (2019). The influence of instructional design on learner control, sense of achievement, and perceived effectiveness in a supersize MOOC course. Computers & Education, 128, 377–388. https://doi.org/10.1016/j.compedu.2018.10.001
Jung, Y., & Lee, J. (2018). Learning engagement and persistence in massive open online courses (MOOCs). Computers & Education, 122, 9–22. https://doi.org/10.1016/j.compedu.2018.02.013
Kahan, T., Soffer, T., & Nachmias, R. (2017). Types of participant behavior in a massive open online course. International Review of Research in Open and Distributed Learning, 18(6). https://doi.org/10.19173/irrodl.v18i6.3087
Khan, I. U., Hameed, Z., Yu, Y., Islam, T., & Khan, S. U. (2018). Predicting the acceptance of MOOCs in a developing country: Application of task-technology fit model, social motivation, and self-determination theory. Telematics and Informatics, 35(4), 964–978. https://doi.org/10.1016/j.tele.2017.09.009
Kizilcec, R. F., Perez-Sanagustin, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in massive open online courses. Computers & Education, 104, 18–33. https://doi.org/10.1016/j.compedu.2016.10.001
Kormos, J., & Nijakowska, J. (2017). Inclusive practices in teaching students with dyslexia: Second language teachers’ concerns, attitudes and self-efficacy beliefs on a massive open online learning course. Teaching & Teacher Education, 68, 30–41. https://doi.org/10.1016/j.tate.2017.08.005
Kovanovic, V., Joksimovic, S., Poquet, O., Hennis, T., & Gasevic, D. (2017). Exploring communities of inquiry in massive open online courses. Computers & Education, 119, 44–58. https://doi.org/10.1016/j.compedu.2017.11.010
Lee, D., Watson, S. L., & Watson, W. R. (2020). The relationships between self-efficacy, task value, and self-regulated learning strategies in massive open online courses. International Review of Research in Open and Distributed Learning, 21(1), 23–39. https://doi.org/10.19173/irrodl.v20i5.4389
Li, B., Wang, X., & Tan, S. C. (2018). What makes MOOC users persist in completing MOOCs? A perspective from network externalities and human factors. Computers in Human Behavior, 85, 385–395. https://doi.org/10.1016/j.chb.2018.04.028
Lowenthal, P., Snelson, C., & Perkins, R. (2018). Teaching massive, open, online, courses (MOOCs): Tales from the front line. International Review of Research in Open and Distributed Learning, 19(3), 1–19. https://doi.org/10.19173/irrodl.v19i3.3505
Luik, P., Suviste, R., Lepp, M., Palts, T., Tonisson, E., Sade, M., & Papli, K. (2019). What motivates enrolment in programming MOOCs? British Journal of Educational Technology, 50(1), 153–165. https://doi.org/10.1111/bjet.12600
Maldonado-Mahauad, J., Perez-Sanagustin, M., Kizilcec, R. F., Morales, N., & Munoz-Gama, J. (2018). Mining theory-based patterns from big data: Identifying self-regulated learning strategies in massive open online courses. Computers in Human Behavior, 80, 179–196. https://doi.org/10.1016/j.chb.2017.11.011
Marta-Lazo, C., Frau-Meigs, D., & Osuna-Acedo, S. (2019). A collaborative digital pedagogy experience in the tMOOC “step by step.” Australasian Journal of Educational Technology, 35(5), 111–127. https://doi.org/10.14742/ajet.4215
Martin Nunez, J. L., Tovar Caro, E., & Hilera Gonzalez, J. R. (2017). From higher education to open education: Challenges in the transformation of an online traditional course. IEEE Transactions on Education, 60(2), 134–142. https://doi.org/10.1109/TE.2016.2607693
Martinez-Lopez, R., Yot, C., Tuovila, I., & Perera-Rodriguez, V. H. (2017). Online self-regulated learning questionnaire in a Russian MOOC. Computers in Human Behavior, 75, 966–974. https://doi.org/10.1016/j.chb.2017.06.015
Meinert, E., Alturkistani, A., Brindley, D., Carter, A., Wells, G., & Car, J. (2018). Protocol for a mixed-methods evaluation of a massive open online course on real world evidence. BMJ Open, 8(8), e025188. https://doi.org/10.1136/bmjopen-2018-025188
Montes-Rodriguez, R., Martinez-Rodriguez, J. B., & Ocana-Fernandez, A. (2019). Case study as a research method for analyzing MOOCs: Presence and characteristics of those case studies in the main scientific databases. International Review of Research in Open and Distributed Learning, 20(3), 59–79. https://doi.org/10.19173/irrodl.v20i4.4299
Peral, J., Maté, A., & Marco, M. (2017). Application of data mining techniques to identify relevant key performance indicators. Computer Standards & Interfaces, 50, 55–64. https://doi.org/10.1016/j.csi.2016.09.009
Pintrich, P. R., Smith, D. A. F., Duncan, T., & Mckeachie, W. J. (1991). A manual for the use of the motivated strategies for learning questionnaire (MSLQ). University of Michigan.
Rasheed, R. A., Kamsin, A., Abdullah, N. A., Zakari, A., & Haruna, K. (2019). A systematic mapping study of the empirical MOOC literature. IEEE Access, 7, 124809–124827. https://doi.org/10.1109/ACCESS.2019.2938561
Robinson, R. (2016). Delivering a medical school elective with massive open online course (MOOC) technology. Peerj, 4(1), e2343. https://doi.org/10.7717/peerj.2343
Ruiz-Palmero, J., Lopez-Alvarez, D., Sanchez-Rivas, E., & Sanchez-Rodriguez, J. (2019). An analysis of the profiles and the opinion of students enrolled on xMOOCs at the University of Malaga. Sustainability, 11(24). https://doi.org/10.3390/su11246910
Sablina, S., Kapliy, N., Trusevich, A., & Kostikova, S. (2018). How MOOC-takers estimate learning success: Retrospective reflection of perceived benefits. International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3768
Sanchez-Gordon, S., & Lujan-Mora, S. (2017). Research challenges in accessible MOOCs: A systematic literature review 2008–2016. Universal Access in the Information Society, 17(4), 775–789. http://doi.org/10.1007/s10209-017-0531-2
Sari, A. R., Bonk, C. J., & Zhu, M. (2020). MOOC instructor designs and challenges: What can be learned from existing MOOCs in Indonesia and Malaysia? Asia Pacific Education Review, 21(1), 143–166. https://doi.org/10.1007/s12564-019-09618-9
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460–475. https://doi.org/10.1006/ceps.1994.1033
Seddon, P. (1997). A respecification and extension of the Delone and McLean model of IS success. Information Systems Research, 8, 240–253. https://doi.org/10.1287/isre.8.3.240
Shahzad, F., Xiu, G., Khan, I., Shahbaz, M., & Abbas, A. (2020). The moderating role of intrinsic motivation in cloud computing adoption in online education in a developing country: A structural equation model. Asia Pacific Education Review, 21(1), 121–141. https://doi.org/10.1007/s12564-019-09611-2
Shao, Z. (2018). Examining the impact mechanism of social psychological motivations on individuals’ continuance intention of MOOCs. Internet Research, 28(1), 232–250. https://doi.org/10.1108/IntR-11-2016-0335
Shirky, C. (2013, July 8). MOOCs and economic reality. Chronicle of Higher Education, 59(42), B2.
Sneddon J., Barlow G., Bradley S., Brink A., Chandy S. J., & Nathwani D. (2018). Development and impact of a massive open online course (MOOC) for antimicrobial stewardship. Journal of Antimicrobial Chemotherapy, 73(4), 1091–1097. https://doi.org/10.1093/jac/dkx493
Soffer, T., & Cohen, A. (2015). Implementation of Tel Aviv University MOOCs in academic curriculum: A pilot study. International Review of Research in Open and Distributed Learning, 16(1), 80–97. https://doi.org/10.19173/irrodl.v16i1.2031
Sun, Y., Ni, L., Zhao, Y., Shen, X.-L., & Wang, N. (2019). Understanding students’ engagement in MOOCs: An integration of self-determination theory and theory of relationship quality. British Journal of Educational Technology, 50(6), 3156–3174. https://doi.org/10.1111/bjet.12724
Tao, D., Fu, P., Wang, Y., Zhang, T., & Qu, X. (2019). Key characteristics in designing massive open online courses (MOOCs) for user acceptance: An application of the extended technology acceptance model. Interactive Learning Environments, 44, 1–14. https://doi.org/10.1080/10494820.2019.1695214
Teo, T., & Dai, H. M. (2019). The role of time in the acceptance of MOOCs among Chinese university students. Interactive Learning Environments, 1–14. https://doi.org/10.1080/10494820.2019.1674889
Teresa Garcia-Alvarez, M., Novo-Corti, I., & Varela-Candamio, L. (2018). The effects of social networks on the assessment of virtual learning environments: A study for social sciences degrees. Telematics and Informatics, 35(4), 1005–1017. https://doi.org/10.1016/j.tele.2017.09.013
Tsai, Y. H., Lin, C. H., Hong, J. C., & Tai, K. H. (2018). The effects of metacognition on online learning interest and continuance to learn with MOOCs. Computers & Education, 121, 18–29. https://doi.org/10.1016/j.compedu.2018.02.011
Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. International Review of Research in Open and Distributed Learning, 17(2). https://doi.org/10.19173/irrodl.v17i2.2448
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. https://doi.org/10.2307/30036540
Wang, Y., & Baker, R. (2018). Grit and intention: Why do learners complete MOOCs? International Review of Research in Open and Distributed Learning, 19(3), 20-42. https://doi.org/10.19173/irrodl.v19i3.3393
Warr, P., & Downing, J. (2000). Learning strategies, learning anxiety and knowledge acquisition. British Journal of Psychology, 91(3), 311–333. https://doi.org/10.1348/000712600161853
Watson, S. L., Watson, W. R., & Tay, L. (2018). The development and validation of the attitudinal learning inventory (ALI): A measure of attitudinal learning and instruction. Educational Technology Research and Development, 66(6), 1601–1617. https://doi.org/10.1007/s11423-018-9625-7
Watson, W. R., Kim, W., & Watson, S. L. (2016). Learning outcomes of a MOOC designed for attitudinal change: A case study of an animal behavior and welfare MOOC. Computers & Education, 96, 83–93. https://doi.org/10.1016/j.compedu.2016.01.013
Watted, A., & Barak, M. (2018). Motivating factors of MOOC completers: Comparing between university-affiliated students and general participants. The Internet and Higher Education, 37, 11–20. https://doi.org/10.1016/j.iheduc.2017.12.001
Wu, B., & Chen, X. (2017). Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Computers in Human Behavior, 67, 221–232. https://doi.org/10.1016/j.chb.2016.10.028
Yang, H. H., & Su, C. H. (2017). Learner behaviour in a MOOC practice-oriented course: An empirical study integrating TAM and TPB. International Review of Research in Open and Distributed Learning, 18(5), 35–63. https://doi.org/10.19173/irrodl.v18i5.2991
Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development, 65(5), 1195–1214. https://doi.org/10.1007/s11423-017-9513-6
Zhang, J. (2016). Can MOOCs be interesting to students? An experimental investigation from regulatory focus perspective. Computers & Education, 95, 340–351. https://doi.org/10.1016/j.compedu.2016.02.003
Zhang, M., Yin, S., Luo, M., & Yan, W. (2016). Learner control, user characteristics, platform difference, and their role in adoption intention for MOOC learning in China. Australasian Journal of Educational Technology, 33(1), 114–133. https://doi.org/10.14742/ajet.2722
Zhao, Y., Wang, A., & Sun, Y. (2020). Technological environment, virtual experience, and MOOC continuance: A stimulus-organism-response perspective. Computers & Education, 144, 103721. https://doi.org/10.1016/j.compedu.2019.103721
Zhou, J. (2017). Exploring the factors affecting learners’ continuance intention of MOOCs for online collaborative learning: An extended ECM perspective. Australasian Journal of Educational Technology, 33(5), 123–135. http://doi.org/10.14742/ajet.2914
Zhou, M. (2016). Chinese university students’ acceptance of MOOCs: A self-determination perspective. Computers & Education, 92–93, 194–203. https://doi.org/10.1016/j.compedu.2015.10.012
Zhu, M., Sari, A., & Lee, M. M. (2018). A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). The Internet and Higher Education, 37. https://doi.org/10.1016/j.iheduc.2018.01.002
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
