Development of the Online Course Overload Indicator and the Student Mental Fatigue Survey
DOI:
https://doi.org/10.19173/irrodl.v23i4.6223Keywords:
mental fatigue, cognitive overload, online learning, online course design, student support, online course developmentAbstract
The purpose of this study is to develop and examine the psychometric properties of the Online Course Overload Indicator (OCOI) and the Student Mental Fatigue Survey (SMFS). The OCOI was designed to measure students’ perceptions of cognitive overload in online courses. The SMFS was used to assess students’ perceptions of mental fatigue while taking online courses. An exploratory factor analysis was conducted on a sample of 378 undergraduate students from various institutions offering online courses across the United States. Results of a factor and reliability analyses confirmed that the instruments are valid and reliable measures of students’ perceived mental fatigue and overload from online course elements. The analysis supported the model that students’ perceptions of overload in online courses consist of four constructs—information relevance, information overload, course design, and facilitation—in addition to the one-factor structure of the SMFS, which consists of the student mental fatigue construct.
References
Ackerman, P. L. (2011). Cognitive fatigue: Multidisciplinary perspectives on current research and future applications. American Psychological Association. https://doi.org/10.1037/12343-000
Ackerman, P. L., Kanfer, R., Shapiro, S. W., Newton, S., & Beier, M. E. (2010). Cognitive fatigue during testing: An examination of trait, time-on-task, and strategy influences. Human Performance, 23(5), 381–402. https://doi.org/10.1080/08959285.2010.517720
Al Ma’mari, Q., Sharour, L. A., Al Omari, O. (2020). Fatigue, burnout, work environment, workload and perceived patient safety culture among critical care nurses. British Journal of Nursing, 29(1), 28–34. https://doi.org/10.12968/BJON.2020.29.1.28
Alleyne Bayne, G. (2016, March 25). An exploratory study of university students’ mental fatigue in online courses [Poster presentation]. Texas Tech Univeristy 15th Annual Graduate Student Research Poster Competition, Lubbock TX.
Ayres, P. (2006). Using subjective measures to detect variations of intrinsic cognitive load within problems. Learning and Instruction, 16(5), 389–400. https://doi.org/10.1016/j.learninstruc.2006.09.001
Baker, C. (2010). The impact of instructor immediacy and presence for online student affective learning, cognition, and motivation. Journal of Educators Online, 7(1), 1–30. https://doi.org/doi:10.9743/JEO.2010.1.2
Bakker, R. J., van de Putte, E. M., Kuis, W., & Sinnema, G. (2009). Risk factors for persistent fatigue with significant school absence in children and adolescents. Pediatrics, 124(1), e89–e95. https://doi.org/10.1542/peds.2008-1260
Balkin, T. J., & Wesensten, N. J. (2011). Differentiation of sleepiness and mental fatigue effects. In P. L. Ackerman (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications (pp. 47–66). American Psychological Association. https://doi.org/10.1037/12343-002
Barnard, L., & Paton, V. O. (2007, November). Distance learning survey of Texas Tech University’s fall 2006 distance and off-campus students. https://www.depts.ttu.edu/opa/docs/Barnard_Paton_2007_Distance_Learning_Survey_TTU_F2006.pdf
Bartlett, M. S. (1951). The effect of standardization on a χ2 approximation in factor analysis. Biometrika, 38(3/4), 337–344. https://doi.org/10.2307/2332580
Beiske, A. G., & Svensson, E. (2010). Fatigue in Parkinson’s disease: A short update. Acta Neurologica Scandinavica, 122(s190), 78–81. https://doi.org/10.1111/j.1600-0404.2010.01381.x
Boksem, M. A. S., Meijman, T. F., & Lorist, M. M. (2005). Effects of mental fatigue on attention: An ERP study. Cognitive Brain Research, 25(1), 107–116. https://doi.org/10.1016/j.cogbrainres.2005.04.011
Boksem, M. A. S., & Tops, M. (2008). Mental fatigue: Costs and benefits. Brain Research Reviews, 59(1), 125–139. https://doi.org/10.1016/j.brainresrev.2008.07.001
Bolliger, D. U., & Inan, F. A. (2012). Development and validation of the Online Student Connectedness Survey (OSCI). The International Review of Research in Open and Distributed Learning, 13(3), 41-65. https://doi.org/10.19173/irrodl.v13i3.1171
Carpenter, S. (2018). Ten steps in scale development and reporting: A guide for researchers. Communication Methods and Measures, 12(1), 25–44. https://doi.org/10.1080/19312458.2017.1396583
Chalder, T., Berelowitz, G., Pawlikowska, T., Watts, L., Wessely, S., Wright, D., & Wallace, E. P. (1993). Development of a fatigue scale. Journal of Psychosomatic Research, 37(2), 147–153. https://doi.org/10.1016/0022-3999(93)90081-P
Chen, C. Y., Pedersen, S., & Murphy, K. L. (2011). Learners’ perceived information overload in online learning via computer-mediated communication. Research in Learning Technology, 19(2), 101–116. https://doi.org/10.1080/21567069.2011.586678
Clark, R., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (4th ed.). John Wiley & Sons. https://doi.org/10.1002/9781119239086
Clark, R., Nguyen, F., & Sweller, J. (2006). Efficiency in learning: Evidence-based guidelines to manage cognitive load. John Wiley & Sons.
Comrey, A. L., & Lee, H. (1992). A first course in factor analysis (2nd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9781315827506
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 10(7), 1–9. https://doi.org/10.7275/jyj1-4868
Csathó, Á., van der Linden, D., Hernádi, I., Buzás, P., & Kalmár, G. (2012). Effects of mental fatigue on the capacity limits of visual attention. Journal of Cognitive Psychology, 24(5), 511–524. https://doi.org/10.1080/20445911.2012.658039
DeLuca, J. (Ed.). (2005). Fatigue as a window to the brain. MIT press. https://doi.org/10.7551/mitpress/2967.001.0001
DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Sage Publications.
Edwards, J. R., & Cooper, C. L. (2013). The person-environment fit approach to stress: Recurring problems and some suggested solutions. In C. L. Cooper (Ed.), From stress to wellbeing: Vol. 1. Theory and research on occupational stress and wellbeing (pp. 91–108). Palgrave Macmillan. https://doi.org/10.1057/9781137310651_5
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Sage Publications. https://uk.sagepub.com/en-gb/eur/discovering-statistics-using-ibm-spss-statistics/book257672
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6
Guo, P. J., Kim, J., & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of MOOC videos. L@S ’14: Proceedings of the First ACM Conference on Learning @ Scale Conference (pp. 41–50). Association for Computing Machinery. https://doi.org/10.1145/2556325.2566239
Guramatunhu, P. (2015). The gender shift in enrollment patterns in higher education: A case study of a school administration program. Advancing Women in Leadership, 35, 120–133. https://doi.org/10.21423/awlj-v35.a137
Hafezi, S., Zare, H., Najafi Mehri, S., & Mahmoodi, H. (2010). The Multidimensional Fatigue Inventory validation and fatigue assessment in Iranian distance education students. Proceedings of the 4th International Conference on Distance Learning and Education (ICDLE) (pp. 195–198) Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDLE.2010.5606006
Harris, S. M., Larrier, Y. I., & Castano-Bishop, M. (2011). Development of the Student Expectations of Online Learning Survey (SEOLS): A pilot study. Online Journal of Distance Learning Administration, 14(4). https://eric.ed.gov/?id=EJ960578
Herlambang, M. B., Taatgen, N. A., & Cnossen, F. (2019). The role of motivation as a factor in mental fatigue. Human Factors, 61(7), 1171–1185. https://doi.org/10.1177/0018720819828569
Hinkin, T. R., Tracey, J. B., & Enz, C. A. (1997). Scale construction: Developing reliable and valid measurement instruments. Journal of Hospitality and Tourism Research, 21(1), 100–120. https://doi.org/10.1177/109634809702100108
Hockey, R. (2013). The psychology of fatigue: Work, effort and control. Cambridge University Press. https://doi.org/10.1017/CBO9781139015394
Holmes, C. M., & Reid, C. (2017). A comparison study of on-campus and online learning outcomes for a research methods course. The Journal of Counselor Preparation and Supervision, 9(2), 1–24. https://doi.org/10.7729/92.1182
Jensen, J. L., Berry, D. A., & Kummer, T. A. (2013). Investigating the effects of exam length on performance and cognitive fatigue. PLoS One, 8(8), e70270. https://doi.org/10.1371/journal.pone.0070270
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20(1), 141–151. https://doi.org/10.1177/001316446002000116
Keller, J. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructional design theories and models: An overview of their current status (pp. 383–434). Lawrence Erlbaum. https://doi.org/10.4324/9780203824283
Lambert, M., Sattler, S., & Paton, V. O. (2009, June). Distance learning survey of Texas Tech University’s fall 2008 distance and off-campus students. https://www.depts.ttu.edu/opa/docs/Lambert_Sattler_Paton_2009_Distance_Learning_Survey_TTU_F2008.pdf
Lee, A., Son, S. M., & Kim, K. K. (2016). Information and communication technology overload and social networking service fatigue: A stress perspective. Computers in Human Behavior, 55, 51–61. https://doi.org/10.1016/j.chb.2015.08.011
Matsunaga, M. (2010). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological Research, 3(1), 97–110. https://doi.org/10.21500/20112084.854
Mayadas, F., Miller, G. E., & Sener, J. (2015, July). Updated e-learning definitions. Online Learning Consortium. http://onlinelearningconsortium.org/updated-e-learning-definitions/
Mertler, C. A., & Vannatta, R. A. (2016). Advanced and multivariate statistical methods: Practical application and interpretation (6th ed.). Routledge. https://doi.org/10.4324/9781315266978
Miller, G. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343–352. https://doi.org/10.1037/0033-295x.101.2.343
Mizuno, K., Tanaka, M., Fukuda, S., Yamano, E., Shigihara, Y., Imai-Matsumura, K., & Watanabe, Y. (2011). Low visual information-processing speed and attention are predictors of fatigue in elementary and junior high school students. Behavioral and Brain Functions, 7(1), 1–7. https://doi.org/10.1186/1744-9081-7-20
Mota, D. D. C. F., & Pimenta, C. A. M. (2006). Self-report instruments for fatigue assessment: A systematic review. Research and Theory for Nursing Practice: An International Journal, 20(1), 49–78. https://doi.org/10.1891/rtnp.20.1.49
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
O’connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396–402. https:/doi.org/10.3758/BF03200807
O’Sullivan, P. B., Hunt, S. K., & Lippert, L. R. (2004). Mediated immediacy: A language of affiliation in a technological age. Journal of Language and Social Psychology, 23(4), 464–490. https://doi.org/10.1177/0261927X04269588
Palmer, L. K. (2013). The relationship between stress, fatigue, and cognitive functioning. College Student Journal, 47(2), 312–325. https://www.ingentaconnect.com/content/prin/csj/2013/00000047/00000002/art00007
Plass, J. L., Moreno, R., & Brunken, R. (Eds.). (2010). Cognitive load theory. Cambridge University Press. https://doi.org/10.1017/CBO9780511844744
Plukaard, S., Huizinga, M., Krabbendam, L., & Jolles, J. (2015). Cognitive flexibility in healthy students is affected by fatigue: An experimental study. Learning and Individual Differences, 38, 18–25. https://doi.org/10.1016/j.lindif.2015.01.003
Roach, V., & Lemasters, L. (2006). Satisfaction with online learning: A comparative descriptive study. Journal of Interactive Online Learning, 5(3), 317–332. https://files.eric.ed.gov/fulltext/ED447310.pdf
Roberson, R. (2013, September). Helping students find relevance. Psychology Teacher Network, 23(2), 18–20.
Roelle, J., Lehmkuhl, N., Beyer, M., & Berthold, K. (2015). The role of specificity, targeted learning activities, and prior knowledge for the effects of relevance instructions. Journal of Educational Psychology, 107(3), 705–723. https://doi.org/10.1037/edu0000010
Sarkar, S., & Parnin, C. (2017). Characterizing and predicting mental fatigue during programming tasks. Proceedings: 2017 IEEE/ACM 2nd International Workshop on Emotion Awareness in Software Engineering (pp. 32–37). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/SEmotion.2017.2
Slavin, R. E. (2007). Educational research in an age of accountability. Allyn & Bacon.
Sweller, J. (2008). Human cognitive architecture. In J. M. Spector, M. D. Merrill, J. Van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 369–381). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203880869
Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/A:1022193728205
Tugtekin, U. (2022). Development and validation of an instrument for online learning fatigue in higher education. In G. Durak & S. Çankaya (Eds.), Handbook of research on managing and designing online courses in synchronous and asychronous environments (pp. 566–586). IGI Global. https://doi.org/10.4018/978-1-7998-8701-0.ch028
van der Linden, D. (2011). The urge to stop: The cognitive and biological nature of acute mental fatigue. In P. L. Ackerman (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications. (pp. 149–164). American Psychological Association. https://doi.org/10.1037/12343-007
Vercoulen, J. H. M. M., Swanink, C. M., Fennis, J. F., Galama, J. M., van der Meer, J. W., & Bleijenberg, G. (1994). Dimensional assessment of chronic fatigue syndrome. Journal of Psychosomatic Research, 38(5), 383–392. https://doi.org/10.1016/0022-3999(94)90099-x
Wanstreet, C. E. (2006). Interaction in online learning environments: A review of the literature. Quarterly Review of Distance Education, 7(4), 399–411. https://www.learntechlib.org/p/106711/
Wilcox, R. R. (2013). Introduction to robust estimation and hypothesis testing (3rd ed.). Elsevier. https://doi.org/10.1016/C2010-0-67044-1
Worthington, R., & Whittaker, T. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
