Editorial, Special Issue: Outcomes of Openness: Empirical Reports on the Implementation of OER by John Hilton III is licensed under a Creative Commons Attribution 4.0 International License.
Volume 18, Number 4
John Hilton III
Brigham Young University
Open Educational Resources have been used in classrooms for more than a decade, and yet the research surrounding OER is still in its infancy. This special issue contains 16 empirical studies that address different aspects of the implementation of OER centered on the COUP framework. Bliss, Robinson, Hilton, and Wiley (2013) provide the COUP framework as a model for studying the impact of Open Educational Resources (OER). COUP stands for:
Cost - What are the financial impacts for students and other stakeholders of OER adoption?
Outcomes - How does OER influence student academic performance?
Usage - In what ways do faculty members, as well as learners use OER?
Perceptions - What do faculty and students think about, and feel toward, OER?
Six years ago, Ehlers (2011) identified a need for research to determine the efficacy of OER. My recollection of this time period is that some believed that OER were inferior to commercial materials and that student learning would decline if OER were adopted. Conversely, some argued that OER would dramatically improve student learning as students gained access to learning resources. This special issue addresses the significant need for much more research on the question of OER efficacy. Consider for example just two aspects of the COUP framework, "Outcomes" and "Perceptions." Hilton (2016) identified only 16 studies that had examined these two areas. Since that time additional peer-reviewed studies have been published, but only a handful (Pitt, 2015; Wiley, Williams, DeMarte, & Hilton, 2016; Delimont, Turtle, Bennett, Adhikari, & Lindshield, 2016; Hilton, Fischer, Wiley, & Williams, 2016; Illowsky, Hilton, Whiting, & Ackerman, 2016; Croteau, 2017; Ozdemir & Hendricks, 2017). Given the millions of dollars and equally significant amounts of time that have gone into OER creation and adoption, much more research is needed on the efficacy and perceptions of these resources. Even outside of outcomes and perceptions, the cost savings of OER is far from a settled issue, and the usage aspect of OER is the least studied of all. The current special issue helps to address this gap in the literature by providing 16 articles, each of which focuses on empirical aspects of one of more aspects of the COUP framework. The articles in this issue are arranged by the COUP framework, acknowledging that many of the articles address multiple aspects of it.
Annand and Jensen describe an OER implementation project that has resulted in significant, ongoing savings at a Canadian university. Based on these results, they identify a significant barrier to greater use of OER in higher education: a misalignment of financial incentives among faculty, students, and educational institutions. Changes to government policies are proposed to rectify this.
Goodier attempts to examine claims regarding the amount of money spent on textbooks and saved by OER in a K-12 education setting in South Africa. One of the valuable findings from this study is the complexity of quantifying these numbers that we might think would be straightforward to identify. This study highlights important information gaps that need to be filled.
Lashley, Cummings-Sauls, Bennett, and Lindshield provide a compelling Note from the Field that illustrates a unique approach to cost problems with OER. Their work at Kansas State University indicates that in some instances, student leaders are willing to subsidize the institutional efforts to mainstream OER. Their approach offers one possible way to sustainably provide ongoing incentives to faculty who use OER.
Winitzky-Stephens and Pickavance examine a large-scale OER adoption across 37 different courses across several different general education subjects. Over a four-year period of time, 26,538 students enrolled in non-OER versions of these courses while 7,588 students enrolled in the OER versions. The multilevel models used by the authors found no significant difference between courses using OER and traditional textbooks for continuing students, and a small benefit for new students.
Chiorescu explores OER adoption in a college algebra class and found that by adopting an open textbook, each enrolled student could save $86.00. Depending on which semesters were compared, students were either as likely or more likely to pass the class when OER was used; moreover, significantly fewer students withdrew when OER were implemented.
Wiley, Tonks, Weston, and Webb study the use of student-created OER as a means to improving student performance. Students created tutorial videos, chapter summaries, and review games for a particular topic, some of which were placed into later versions of the course. The average grade on student assignments rose significantly as more student-created OER were added to the course.
Kelly and Rutherford compare the academic results of seventh graders studying mathematics. All students received regular instruction, and for supplemental instruction (30 minutes a day, over four weeks) were placed into groups that: (1) used Khan Academy videos, (2) received other math instruction, or (3) studied English Language Arts. At the conclusion of the study, all students were given a mathematics test; there were no significant differences between the three groups.
Grewe and Davis 's Note from the Field indicates a positive relationship between the use of OER and student academic achievement in an online history course. She studies 146 students who enrolled in the Fall of 2013 and Spring of 2014; these courses had an identical instructional design (but different content and different instructors). In examining the final course grades of 146 students, Grewe and Davis demonstrates that OER adoption can be associated with higher student performance.
Hendricks, Reinsberg, and Rieger examine the use of OER in a physics course at the University of British Columbia. 143 students completed surveys about the OER; students reported using the open textbook with the same frequency as their traditional textbooks. Moreover, 93% of respondents said it was the same or better than textbooks in other courses. The researchers also found that in one year of OER adoption students saved approximately C$85,000 and that student final exam scores and grade distributions remained the same after OER adoption as they had been previously.
Sheu and Shih provide context and results for the OCW project at National Taiwan University (NTU). They find that nearly two-thirds of visitors to the site are life-long learners in informal educational settings. Visits have increased from 800 daily visits (2012) to 8,000 (2016). Additional statistics indicate successful uses of the NTU-OCW initiative.
Jung, Bauer, and Heaps report on the perceptions of faculty members who used OpenStax textbooks. This study adds significant value by focusing not simply on "OER in general," but on a specific set of OER. They survey 137 faculty and find that only 16% report taking more time preparing to teach a course using OpenStax textbooks, and that of these, 78% felt that the benefits of OpenStax were worth the additional time. In terms of quality; 81% thought that OpenStax textbooks have same or higher quality as commercial textbooks. Thematic analysis revealed faculty perceptions of what constitutes high-quality open textbooks.
Fischer, Ernst, and Mason examine the rich corpus of the reviews hosted by the Open Textbook Network (OTN). The OTN's Open Textbook Library has been an extremely beneficial way to identify open textbooks and includes a very useful feature of allowing faculty members at participating universities to review open textbooks. This study examines 416 reviews of 121 textbooks and finds that reviewers generally gave open textbooks high ratings (a median of 4.5/5 overall rating). Moreover, there were variances between textbooks, indicating that they vary in quality, but that these variances can be identified and quantified.
Cooney studies three sections of a Health Psychology course that replaced a traditional textbook with OER. Students were surveyed and interviewed regarding their perceptions of the OER; findings indicate that students were generally positive about the OER. Just over 80% of 67 students surveyed rated the OER as being better than a traditional textbook, with an additional 16% saying it was of the same quality.
Vojtech and Grissett explore a novel approach to student perceptions by examining how students perceive hypothetical faculty members who use open textbooks. They find that students rated faculty who assign an open textbook to be kinder, as well as more encouraging and creative. Although the study was designed to have open textbooks be the only difference between the hypothetical professors that students rated, only 14% students attributed their belief that the professor who used OER was kinder, more creative, etc. to the costs of textbooks.
Jhangiani and Jhangiani survey 320 post-secondary students in British Columbia who were enrolled in courses that used an open textbook. They find that a majority of students report not purchasing at least one of their required textbooks and that a substantial portion of students report taking fewer courses and/or dropping courses based on textbook costs. Students favorably rated open textbooks, with 96% of survey participants stating that they were at or above average.
Kwak examines the use of a specific set of OER in South Korea through means of an ethnographic perspective to explore the experiences of teachers and students. More than 90% of the students perceived OER as beneficial to their studies and that teachers were motivated to improve or alter their pedagogy because of the existence of OER. While this may be a benefit, Kwak also highlights associated challenges associated with the relation between OER and teacher authority.
The research in this special issue presents a compelling case for the accelerated adoption of OER. This research indicates that the use of OER saves significant amounts of money, students perform as well or better when using OER, students use OER like other traditional learning resources, and both students and faculty overwhelmingly perceive OER to be as good or better than traditional textbooks. These results need to be communicated to faculty members, administrators, policy makers, and others responsible for ensuring that students are treated fairly and receive the best educational value for their money.
In commending this special edition to the reader I offer two specific calls to the research community interested in topics related to OER. First, while the research around OER is increasingly robust, more work is needed to clearly establish the value of OER. Additional rigorous studies in a variety of contexts, employing varied methodologies are needed.
Second, some aspects relating to openness have been almost completely ignored in terms of research. For example, open pedagogy is frequently touted at conferences, yet little research has been done on its efficacy, how teachers/students perceive it, and so forth. This puts us in a similar situation as that framed by Ehlers (2011). Will widespread adoption of open pedagogy spark dramatic improvements in learning? Those who study this question need to carefully consider what they mean by open pedagogy, an increasingly contested term, and the metrics they use when determining whether open pedagogy leads to increased learning outcomes. In what ways would we expect open pedagogy to make a difference in student learning? Much of the OER efficacy research conducted to date focuses on traditional metrics like GPA, completion rates, enrollment intensity and so forth. These might be appropriate for determining the influence of open pedagogy, but other measures may be more appropriate. For example, open pedagogy could conceivably lead to increases in student creativity, enthusiasm, or satisfaction.
Proponents of any new intervention (including those related to OER) need to be careful that their rhetoric does not get too far ahead of research. I hope that future special editions of this and other journals will continue to examine the empirical outcomes of OER adoption, open pedagogy, and other important topics related to openness.
Bliss, T., Robinson, T. J., Hilton, J., & Wiley, D. (2013). An OER COUP: College teacher and student perceptions of Open Educational Resources. Journal of Interactive Media in Education, 2013(1), 1-25. doi: http://doi.org/10.5334/2013-04
Croteau, E. (2017). Measures of student success with textbook transformations: The Affordable Learning Georgia Initiative. Open Praxis, 9(1), 93-108.
Delimont, N., Turtle, E. C., Bennett, A., Adhikari, K., & Lindshield, B. L. (2016). University students and faculty have positive perceptions of open/alternative resources and their utilization in a textbook replacement initiative. Research in Learning Technology, 24.
Ehlers, U.-D. (2011). Extending the territory: From open educational resources to open educational practices. Journal of Open, Flexible and Distance, 15(2), 1-10.
Hilton, J. (2016). Open educational resources and college textbook choices: A review of research on efficacy and perceptions. Educational Technology Research and Development, 64(4), pp. 573-590.
Hilton, J. Fischer, L., Wiley, D., & Williams, L. (2016). Maintaining momentum toward graduation: OER and the course throughput rate. International Review of Research on Distance and Open Learning, 17(6), 1-10.
Illowsky, B. S., Hilton III, J., Whiting, J., & Ackerman, J. D. (2016). Examining student perception of an open statistics book. Open Praxis, 8(3), 265-276.
Ozdemir, O., & Hendricks, C. (2017). Instructor and student experiences with open textbooks, from the California open online library for education (Cool4Ed). Journal of Computing in Higher Education, 29(1), 98-113.
Pitt, R. (2015). Mainstreaming open textbooks: Educator perspectives on the impact of openstax college open textbooks. The International Review of Research in Open and Distributed Learning, 16(4).
Wiley, D., Williams, L., DeMarte, D., & Hilton, J. (2016). The Tidewater Z-Degree and the INTRO Model for Sustaining OER Adoption. Education Policy Analysis Archives, 24(41), pp.1-12.
Editorial, Special Issue: Outcomes of Openness: Empirical Reports on the Implementation of OER by John Hilton III is licensed under a Creative Commons Attribution 4.0 International License.