Design Matters: Development and Validation of the Online Course Design Elements (OCDE) Instrument
DOI:
https://doi.org/10.19173/irrodl.v22i2.5187Keywords:
online course design, design elements, instrument validation, confirmatory factor analysis, structural equation modelAbstract
Course design is critical to online student engagement and retention. This study focused on the development and validation of an online course design elements (OCDE) instrument with 38 Likert-type scale items in five subscales: (a) overview, (b) content presentation, (c) interaction and communication, (d) assessment and evaluation, and (e) learner support. The validation process included implementation with 222 online instructors and instructional designers in higher education. Three models were evaluated which included a one-factor model, five-factor model, and higher-order model. The five-factor and higher-order models aligned with the development of the OCDE. The frequency of use of OCDE items was rated above the mean 4.0 except for two items on collaboration and self-assessment. The overall OCDE score was related to self-reported levels of expertise but not with years of experience. The findings have implications for the use of this instrument with online instructors and instructional designers in the design of online courses.
References
Allen, I. E., & Seaman, J. (2017). Digital learning compass: Distance education enrollment report 2017. https://onlinelearningsurvey.com/reports/digtiallearningcompassenrollment2017.pdf
Anderson, K. (2017). Have we reached an inflection point in online collaboration? From e-mail to social networks, online collaboration has evolved fast—as have users. Research Information, 92, 24.
Baldwin, S., Ching, Y-H., & Hsu, Y-C. (2018). Online course design in higher education: A review of national and statewide evaluation instruments. TechTrends, 62(1), 46–57. https://doi.org/10.1007/s11528-017-0215-z
Blackboard. (2012). Blackboard exemplary course program rubric. https://www.blackboard.com/resources/are-your-courses-exemplary
Bozarth, J., Chapman, D. D., & LaMonica, L. (2004). Preparing for distance learning: Designing an online student orientation course. Journal of Educational Technology & Society, 7(1), 87–106. https://www.jstor.org/stable/jeductechsoci.7.1.87
Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005
California Community Colleges Online Education Initiative. (2016). Course design rubric. http://cvc.edu/wp-content/uploads/2016/11/OEI_CourseDesignRubric Nov2016-3.pdf
California State University. (2015). QOLT evaluation rubric. https://cal.sdsu.edu/_resources/docs/QOLT%20Instrument.pdfCapdeferro, N., & Romero, M. (2012). Are online learners frustrated with collaborative learning experiences? International Review of Research in Open and Distributed Learning, 13(2), 26–44. https://doi.org/10.19173/irrodl.v13i2.1127
Castle, S. R., & McGuire, C. J. (2010). An analysis of student self-assessment of online, blended, and face-to-face learning environments: Implications for sustainable education delivery. International Education Studies, 3(3), 36–40. https://doi.org/10.5539/ies.v3n3p36
Chen, S.-J. (2007). Instructional design strategies for intensive online courses: An objectivist-constructivist blended approach. Journal of Interactive Online Learning, 6(1), 72–86. http://www.ncolr.org/jiol/issues/pdf/6.1.6.pdf
Czerkawski, B. C., & Lyman, E. W. III. (2016). An instructional design framework for fostering student engagement in online learning environments. TechTrends, 60(6), 532–539. https://doi.org/10.1007/s11528-016-0110-z
Dell, C. A., Dell, T. F., & Blackwell, T. L. (2015). Applying universal design for learning in online courses: Pedagogical and practical considerations. Journal of Educators Online, 12(2), 166–192. https://doi.org/10.9743/jeo.2015.2.1
Dick, W. (1996). The Dick and Carey model: Will it survive the decade? Educational Technology Research and Development, 44(3), 55–63. https://doi.org/10.1007/BF02300425
Dietz-Uhler, B., Fisher, A., & Han, A. (2007). Designing online courses to promote student retention. Journal of Educational Technology Systems, 36(1), 105–112. https://doi.org/10.2190/ET.36.1.g
Fathema, N., Shannon, D., & Ross, M. (2015). Expanding the technology acceptance model (TAM) to examine faculty use of learning management systems (LMSs) in higher education institutions. Journal of Online Learning & Teaching, 11(2), 210–232. https://jolt.merlot.org/Vol11no2/Fathema_0615.pdf
Gaytan, J., & McEwen, B. C. (2007). Effective online instructional and assessment strategies. American Journal of Distance Education, 21(3), 117–132. https://doi.org/10.1080/08923640701341653
Graf, S., Liu, T.-C., & Kinshuk. (2010). Analysis of learners’ navigational behaviour and their learning styles in an online course. Journal of Computer Assisted Learning, 26(2), 116–131. https://doi.org/10.1111/j.1365-2729.2009.00336.x
Han, I., & Shin, W. S. (2016). The use of a mobile learning management system and academic achievement of online students. Computers & Education, 102, 79–89. https://doi.org/10.1016/j.compedu.2016.07.003
Illinois Online Network. (2015). Quality online course initiative (QOCI) rubric. University of Illinois. https://www.uis.edu/ion/resources/qoci/
Jaggars, S. S., & Xu, D. (2016). How do online course design features influence student performance? Computers & Education, 95, 270–284. https://doi.org/10.1016/j.compedu.2016.01.014
Jones, K. R. (2013). Developing and implementing a mandatory online student orientation. Journal of Asynchronous Learning Networks, 17(1), 43–45. https://doi.org/10.24059/olj.v17i1.312
Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.
Ko, S., & Rossen, S. (2017). Teaching online: A practical guide (4th ed.). Routledge.
Kumar, S., Martin, F., Budhrani, K., & Ritzhaupt, A. (2019). Award-winning faculty online teaching practices: Elements of award-winning courses. Online Learning, 23(4), 160– http://dx.doi.org/10.24059/olj.v23i4.2077
Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas, R., Whitley, E.A., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15–30. https://doi.org/10.1111/j.1365-2729.2011.00458.x
Le Maistre, C. (1998). What is an expert instructional designer? Evidence of expert performance during formative evaluation. Educational Technology Research and Development, 46(3), 21–36. https://doi.org/10.1007/BF02299759
Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7
Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202.
Liu, X., Magjuka, R. J., Bonk, C. J., & Lee, S.-H. (2007). Does sense of community matter? An examination of participants’ perceptions of building learning communities in online courses. Quarterly Review of Distance Education, 8(1), 9–24.
Luo, N., Zhang, M., & Qi, D. (2017). Effects of different interactions on students’ sense of community in e-learning environment. Computers & Education, 115, 153–160. https://doi.org/10.1016/j.compedu.2017.08.006
Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205–222. http://dx.doi.org/10.24059/olj.v22i1.1092
Martin, F., Wang, C., & Sadaf, A. (2018). Student perception of helpfulness of facilitation strategies that enhance instructor presence, connectedness, engagement and learning in online courses. The Internet and Higher Education, 37, 52–65. https://doi.org/10.1016/j.iheduc.2018.01.003
Martin, F., Ritzhaupt, A., Kumar, S., & Budhrani, K. (2019). Award-winning faculty online teaching practices: Course design, assessment and evaluation, and facilitation. The Internet and Higher Education, 42, 34–43. https://doi.org/10.1016/j.iheduc.2019.04.001
Moore, M. G. (1989). Three types of interaction [Editorial]. American Journal of Distance Education, 3(2), 1–7. https://doi.org/10.1080/08923648909526659
Muthén, L. K., & Muthén, B. O. (2012). Mplus (Version 7.11) [Computer software]. Mplus. https://www.statmodel.com/
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Online Learning Consortium. (2016). OSCQR course design review. https://s3.amazonaws.com/scorecard-private-uploads/OSCQR+version+3.1.pdf
Perez, R. S., Johnson, J. F., & Emery, C. D. (1995). Instructional design expertise: A cognitive model of design. Instructional Science, 23(5–6), 321–349. https://doi.org/10.1007/BF00896877
Pimmer, C., Mateescu, M., & Grohbiel, U. (2016). Mobile and ubiquitous learning in higher education settings. A systematic review of empirical studies. Computers in Human Behavior, 63, 490–501. https://doi.org/10.1016/j.chb.2016.05.057
Price, J. M., Whitlatch, J., Maier, C. J., Burdi, M., & Peacock, J. (2016). Improving online teaching by using established best classroom teaching practices. Journal of Continuing Education in Nursing, 47(5), 222–227. https://doi.org/10.3928/00220124-20160419-08
Quality Matters. (2020). Specific review standards from the QM Higher Education Rubric (6th ed.). https://www.qualitymatters.org/sites/default/files/PDFs/StandardsfromtheQMHigherEducationRubric.pdf
Rennstich, J. K. (2019). Creative online collaboration: A special challenge for co-creation. Education and Information Technologies, 24(2), 1835–1836. https://doi.org/10.1007/s10639-019-09875-6
Salmon, G. (2013). E-tivities: The key to active online learning. Routledge.
Shackelford, J. L., & Maxwell, M. (2012). Sense of community in graduate online education: Contribution of learner to learner interaction. The International Review of Research in Open and Distributed Learning, 13(4), 228–249. https://doi.org/10.19173/irrodl.v13i4.1339
Shanteau, J. (1992). Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes, 53(2), 252–266. https://doi.org/10.1016/0749-5978(92)90064-E
Ssekakubo, G., Suleman, H., & Marsden, G. (2013). Designing mobile LMS interfaces: Learners’ expectations and experiences. Interactive Technology and Smart Education, 10(2), 147–167. https://doi.org/10.1108/ITSE-12-2012-0031
Stavredes, T., & Herder, T. (2014). A guide to online course design: Strategies for student success. Jossey Bass.
Stevens, D. D., & Levi, A. J. (2013). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning (2nd ed.). Stylus.
Swan, K. (2001). Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Education, 22(2), 306–331. https://doi.org/10.1080/0158791010220208
Vai, M., & Sosulski, K. (2016). Essentials of online course design: A standards-based guide (2nd ed.). Routledge.
Waterhouse, S., & Rogers, R. O. (2004). The importance of policies in e-learning instruction. EDUCAUSE Quarterly, 27(3), 28–39.
Young, S. (2006). Student views of effective online teaching in higher education. American Journal of Distance Education, 20(2), 65–77. https://doi.org/10.1207/s15389286ajde2002_2
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
