September – 2015

OER Quality and Adaptation in K-12: Comparing Teacher Evaluations of Copyright-Restricted, Open, and Open/Adapted Textbooks

Royce Kimmons
Brigham Young University, USA

Volume 16, Number 5

Abstract

Conducted in conjunction with an institute on open textbook adaptation, this study compares textbook evaluations from practicing K-12 classroom teachers (n = 30) on three different types of textbooks utilized in their contexts: copyright-restricted, open, and open/adapted. Copyright-restricted textbooks consisted of those textbooks already in use by the teachers in their classrooms prior to the institute, open textbooks included alternatives from CK-12 and OpenStax, and open/adapted consisted of open textbooks that the teachers devoted time to adapting to their individual needs. Results indicate that open/adapted textbooks were evaluated as having the highest quality and that open textbooks were of higher quality than copyright-restricted textbooks. Though some factors of quality might be influenced by cost differences (e.g., timeliness and the ability to adopt updated textbooks), results reveal that open and open/adapted textbooks may do a better job of meeting the needs of K-12 teachers in a variety of ways that may not be captured through traditional approaches to quality assurance. This study marks an early step in exploring the quality of K-12 open educational resources (OER) and the use of practicing teachers as authentic evaluators of textbooks for their local contexts.

Keywords: open education; OER; K-12; open textbooks; professional development; OER quality; quality assurance

Introduction

Open educational resources (OER) are “any type of educational materials that are in the public domain or introduced with an open license” (UNESCO, n.d., para. 1) or are “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others” (William and Flora Hewlett Foundation, n.d., para. 2). As such, OER differ from traditional copyright-restricted educational materials in that “anyone can legally and freely copy, use, adapt and re-share them” (UNESCO, n.d., para. 1) provided that those who use these materials adhere to specific guidelines of use that may vary from resource to resource (e.g., attributing the author, non-commercial redistribution). OER have gained much attention in recent years as a means to increase access to quality educational materials, especially in underserved parts of the world (Atkins, Brown, & Hammond, 2007; Baraniuk, 2007; D’Antoni, 2008; Mtebe & Raisamo, 2014; Prasad & Usagawa, 2014), to drive down educational costs (Bliss, Hilton, Wiley, & Thanos, 2013; Wiley & Green, 2012; Wiley, Hilton, Ellington, & Hall, 2012), and to support valuable collaborative practices between educators and students (Brown & Adler, 2008; Petrides, Jimes, Middleton-Detzner, Walling, & Weiss, 2011). Furthermore, though adoption of OER began largely in higher education, we are now seeing its influences trickle down to K-12 institutions and public education in the U.S. (Bliss & Patrick, 2013; K12 OER Collaborative, 2014; Usdan & Gottheimer, 2012).

Like any educational materials, OER come in a variety of forms and are shared via a variety of media. OER repositories like the Digital Public Library of America (http://dp.la), OER Commons (http://oercommons.org), Merlot (http://merlot.org), and many others have been created and curated to connect educators to digital OER in a meaningful manner via hosting, rating, categorizing, tagging, and otherwise sharing these resources (Atenas & Havemann, 2013). In a more traditional manner, however, OER have also been explored as replacements for pre-digital media in the form of OER textbooks, wherein the common textbook structure is used as a means for providing OER to schools and higher education institutions in a familiar manner (Baker, Thierstein, Fletcher, Kaur, & Emmons, 2009). Such textbooks can be freely shared and distributed digitally as e-books and may also be freely printed as hardcopy books without licensing fees or seeking author permission (Hilton & Laman, 2012). Projects such as OpenStax College (n.d.) and CK-12 (n.d.) provide OER in this textbook format, and because these textbooks are openly licensed, educators and students are free to download and utilize them as-is or to adapt them to their individual needs. This approach to content licensing and curation is of special interest to those interested in online and blended learning, since OER may be freely distributed via electronic means, and OER adoption for online instruction can further drive down cost in this medium while simultaneously improving access for students (e.g., no logins are necessary to access content) and ease of implementation by designers (e.g., content may be readily incorporated into any online instructional system without additional permissions).

Though the relative low cost of OER is undeniably tempting to educators, students, and those financing education, cost is not the only factor driving curricular adoptions. For instance, a recent survey of higher education faculty found that the cost of a textbook is of relatively low importance to faculty members when selecting which materials to adopt as compared to other content quality factors, such as “proven efficacy” or trustworthiness of the creator (Allen & Seaman, 2014; cf. Clements & Pawlowski, 2011). Despite the fact that higher education students may view open textbooks as a comparative or even better alternative to copyright-restricted textbooks (Bliss, Hilton, Wiley, & Thanos, 2013), their instructors and others making curricular decisions must sign off on the content that is being provided. In this situation, the traditional publication mechanism, which relies upon peer review and authorship by content experts, is used as a means for verifying quality, and since OER does not generally undergo a similar process, educational institutions may perceive them as lacking a necessary stamp of approval that may not overcome the cost incentive (cf. Prasad & Usagawa, 2014).

Quality OER undoubtedly exist, and research into crowd-generated content has been promising in showing that even platforms as open as Wikipedia can produce materials that live up to reasonable standards of quality, at least in terms of traditional measures such as content accuracy (Giles, 2005). On the other hand, poor-quality OER also undoubtedly exist, and though stakeholders recognize that quality control is an essential priority for OER adoption (D’Antoni, 2008), it is currently unclear how to best ensure quality in lieu of a traditional publication process, and the perceived lack of quality that results from this situation marks a major barrier for OER adoption (de Hart, Chetty, & Archer, 2015; Kimmons, in press). Furthermore, though some valuable studies have begun establishing student outcome measures to determine the effects of particular OER use on student achievement (Robinson, Fischer, Wiley, & Hilton, 2014; Wiley, Hilton, Ellington, & Hall, 2012; cf. Walker, 2007), these studies need to be supplemented with research that considers the quality of the resources themselves in order to support decision-making and to control for implementation factors that might influence results.

Some OER undergo peer review (e.g., OpenStax) and are authored by content experts (e.g., CK-12), but such examples are not the norm, and generally speaking, “[OER] quality control is in the hands of learners and teachers … instead of instructional [or content] experts” (Dinevski, 2008, p. 121), and quality assurance in OER is seen as “primarily the responsibility of … education institutions” (UNESCO & COL, 2011, p. 13). This represents a paradigmatic shift in quality control processes, wherein evaluations of quality must be conducted by the educators and institutions that will be using the products and must also rely upon the suitability of the products themselves rather than upon the processes (e.g., peer review) that produced them (cf. Wiley, 2013). Though some work has been done to begin systematizing the evaluation of OER repositories (e.g., Atenas & Havemann, 2013; Atenas, Havemann, & Priego, 2014; OECD, 2007), research on local evaluations of OER by educators and educational institutions is a largely uncharted and daunting territory that requires intense contextualization and a knowledge of how such institutions operate (Clements & Pawlowski, 2011; Pawlowski, 2007).

Fledgling approaches to quality assurance are especially problematic for K-12 institutions, for whom cost is a much greater incentive in driving a shift to OER, and for whom time constraints for evaluation might be more severe than for university faculty (Kimmons, in press). Though problematic, this shift may also represent a much-needed, positive opportunity for K-12 institutions to shape the quality control process in a manner that will be beneficial for teachers and students. The reason for this possibility is that traditional quality assurance mechanisms may not effectively respond to the authentic realities of K-12 institutions, and the quality of a resource in K-12 institutions involves much more than mere content accuracy and proper formatting (Carey & Hanley, 2007).

Consider the factors of cost and content accuracy and how this differs between K-12 and higher education institutions. The cost of a resource in higher education may not be correlated to accuracy, because students may be directed to purchase new versions of the assigned textbook each semester, and instructors may guide students to purchase these resources without consideration of cost (Allen & Seaman, 2014). This means that university students will generally be expected to have up-to-date materials even if these materials are expensive. In public K-12 institutions in the U.S., on the other hand, textbook purchasing works very differently. K-12 institutions purchase textbooks on behalf of their students and operate on textbook adoption cycles that are dependent upon budget planning and availability of resources. Textbooks are purchased for a lengthy (sometimes non-predetermined) cycle, which in many schools may extend to ten years or longer (FCC, 2012). This means that K-12 teachers may be required to teach out of textbooks that are outdated or inaccurate for several years due to the high cost of replacing the resource. This situation is further exacerbated when states adopt new standards, such as the Common Core State Standards (CCSSI, n.d.) or Next Generation Science Standards (n.d.), which require large-scale curricular redesigns. Schools that cannot afford to replace all of their textbooks simultaneously in response to such shifts are thereby required to teach out of materials that no longer address mandated standards. Thus, in K-12, quality is a function of cost, because cost drives the ability of teachers to gain access to updated materials that are aligned to current standards.

Since OER are freely available, however, they can be adopted at any time and can thereby accelerate or altogether bypass the traditional adoption cycle. This allows teachers to acquire up-to-date, accurate, standards-aligned materials in a timely manner, but peer review and traditional quality assurance measures ignore this factor. Resource quality in K-12 is additionally influenced by a number of other factors that may differ from higher education, including alignment with standards, content conciseness, content reading level, content engagement, media accessibility, and supplementary materials, such as learning activities, rubrics, and lesson plans (Carey & Hanley, 2007; Mtebe & Raisamo, 2014). The idea of the textbook as an information source, on the other hand, seems to be highly influenced by higher education, and for this reason textbooks tend to be treated as content sources that are created and reviewed by content experts rather than educators. Most textbooks used either at K-12 or higher education levels are written by college professors or other professionals with PhDs in the content area, not classroom teachers or pedagogues. This means that though the content may be accurate and up-to-date when it is written, there may be a variety of limitations in its design, including the following:

1. It may not be written at a reading level appropriate for classroom students;
2. It may not be organized in a manner that aligns with state standards or with scaffolded learning in the subject area;
3. It may not include supplementary resources that can be useful for students and teachers, such as meaningful practices, quizzes, lesson plans, classroom activities, rubrics, and so forth;
4. It may not be written in a manner that allows teachers to easily integrate the material into diverse classrooms in response to cultural, community, or geographic considerations or to individual student needs (Kimmons, in press).

For these reasons, a paradigmatic shift in quality assurance for K-12 textbooks is not only feasible but may be long overdue, and OER represents an opportunity to make this shift. Quality cannot exist without an intended context, and the quality of OER, as with any educational resource, is connected with the contexts in which they will be used (Clements & Pawlowski, 2011). Because individual educators are the best experts on their local contexts, it seems to make sense for these educators to be the evaluators of the resources for their contexts. This study seeks to explore how a group of practicing educators evaluate existing textbooks from CK-12 and OpenStax in comparison to their currently-used, copyright-restricted alternatives.

Breaking away from the limitations of the copyright-restricted paradigm, however, this study also attempts to take the next step to consider how the process of adaptation, rather than mere adoption, of OER helps to improve the quality of these resources for practicing educators. Like copyright-restricted materials, OER may be adopted without any form of adaptation (Duncan, 2009; Hilton & Wiley, 2012; Prasad & Usagawa, 2014), but they can also be remixed, reorganized, revised, updated, reworded, and otherwise adjusted in response to educator needs (Tonks, Weston, Wiley, & Barbour, 2013). In a perfect world, we can envision K-12 teachers being the content experts in their classrooms and having the resources and flexibility necessary to adopt, create, revise, and adapt educational materials to their students’ needs. However, within the existing paradigm, teachers are not only typically required to utilize curricula that they have no voice in choosing, but because of the copyright-restricted nature of the materials, they also cannot update, adjust, reword, or otherwise alter the materials that they have been given. Teachers must make their students malleable to the content rather than make the content malleable to their students.

This reveals a significant difficulty with current approaches to teaching within a copyright-restricted content paradigm that is less than ideal and that restricts teachers from making full use of the limited resources that they have. However, OER may offer a means by which teachers and K-12 institutions can gain the ability to more meaningfully differentiate instruction for their students and to share valuable classroom resources with those beyond their local contexts. Thus, in this study we will also guide teachers in the adaptation of their own textbooks to meet their diverse classroom needs and use this as a basis of comparison to determine the relative quality of adapted OER to non-adapted OER and to copyright-restricted alternatives.

Methods

This study attempts to determine whether open educational resources (OER) and specifically open textbooks can be as high quality as copyright-restricted alternatives for K-12 educators and also to determine whether the ability to adapt these textbooks to local or classroom needs makes any difference in their quality. We use practicing K-12 middle- and high-school teachers in this study as evaluators in place of traditional content-expert peer reviewers. We do this for a number of reasons, but primarily because we recognize that teachers are the only professionals qualified to recognize the needs of their students and the realities that they face in their classrooms. We also recognize that teachers should be content experts and should be treated as such. In this process, we also hope to expand the conversation about content quality so that it takes into consideration aspects of quality beyond mere accuracy of information that are essential for educational materials to actually be useful in authentic classrooms. The driving research question for this study is: How do teacher evaluations of textbook quality compare between copyright-restricted, open, and open/adapted textbooks?

Toward this end, this study makes three comparisons between teacher evaluations of copyright-restricted, open, and open/adapted textbooks as follows: 1) open vs. copyright-restricted (O-C), 2) open/adapted vs. open (A-O), and 3) open/adapted vs. copyright-restricted (A-C). By comparing these evaluations, we hope to provide evidence to guide decision-making in the adoption, adaptation, or rejection of open textbooks for K-12 institutions.

Context

This study was conducted at a public research university in the northwestern United States. Some teachers may not have the expertise, time, or resources necessary to develop their own open textbooks due to local realities and lack of training (cf. Petrides, Jimes, Middleton-Detzner, Walling, & Weiss, 2011; Unwin et al., 2010), and professional development and engaging in professional learning communities with other educators toward this goal can help (DuFour, 2004; Kimmons, 2014; Wilson-Strydom, 2009). For these reasons, this study took place in conjunction with a three-day institute focused on helping participants create and adapt open textbooks. This approach has been found to be successful through previous research (Kimmons, 2014). The institute was announced via email to teachers throughout the target state, and applications were accepted from any practicing teacher in the state who taught in one of the targeted institute grade level and subject areas. These included: U.S. History (8th grade), Algebra 2 (10-11th), and all Middle and High School Sciences. In total, thirty (n = 30) teachers participated, serving as content creators and evaluators.

During these institutes, participating teachers learned about OER-related topics like copyright, Creative Commons (n.d.), and public domain and were also given time and guidance to evaluate, adapt, and create assigned resources. As part of participation, the institute paid some teacher costs including lodging, professional development credits, and food for the duration of the institute and also offered to provide printed classroom sets of textbooks created through the institute for participants’ ongoing classroom use. Teachers applied to attend the institute by filling out a short survey wherein they documented their experience in the classroom and the subject area and grade levels wherein they would be interested and qualified for participating in the creation of an open textbook. Prior to attending this institute, many participants had very limited background on the concept of OER, and several participants arrived and stated that they did not fully understand what they would be doing in the institute but that it sounded interesting, revealing that participants were drawn to the idea of creating a textbook but were unclear how this would be possible within the allotted timeframe.

Participating teachers were also encouraged to bring a copy of their current copyright-restricted textbooks that they use in their classroom so that they could be used for comparison and evaluation purposes. Early in the first day of the institute, participants were organized into subject area and grade level groups of between three and six people and were instructed to have a discussion about their existing copyright-restricted textbooks, taking notes on what their textbooks did well and what they did not do well. These conversations lasted around 45 minutes, and collaborative documents were created to guide ongoing conversations. Because participants came from different schools, some had not been using the same textbooks and some had been using copyright-restricted e-books. Upon concluding these discussions, participants individually completed an evaluation of their copyright-restricted textbooks using the textbook evaluation survey form. Copyright-restricted textbooks included teacher-provided examples from prominent publishers such as Glencoe/McGraw-Hill, Houghton Mifflin Harcourt, Discovery Education, Prentice Hall, and others.

Participants were then provided with a link to one or more online exemplary open textbooks for their grade level and subject area from either CK-12 or OpenStax. Participants were then given 45 minutes to an hour to go through the contents of the open textbook as a group, adding to their previous collaborative document and taking notes on its strengths and limitations. Again, participants were instructed to individually evaluate this exemplary textbook utilizing the textbook evaluation survey.

The institute progressed with lecture and discussion about OER, laying groundwork understanding on issues of copyright, public domain, Creative Commons, and so forth. Participants were then guided in a step-by-step manner in selecting an existing open textbook as a foundational text. These textbooks were then iteratively adapted to their needs by aligning them with standards, updating content, adding additional multimedia resources, and so forth. The entirety of the second day as well as significant portions of the first and third days were devoted to this process of adaptation. Near the end of the institute, participants shared their products with the larger group and went through troubleshooting processes to determine how they would share their created resources with their students and utilize them in their classrooms either as an electronic resource or as a printed textbook. When completed, all printed textbooks produced from this institute cost between $3 and $15 USD per copy without benefits of bulk or educational pricing.

Upon completion of their open/adapted textbooks, participants were again directed to complete the textbook evaluation survey for the open/adapted textbook and also to complete an evaluation of the institute itself as a learning experience. Thus, data sources for this study consisted of survey results from three separate institutes with three data collection conditions for each institute, representing each textbook type: copyright-restricted (C), open (O), and open/adapted (A).

Textbook Evaluation Survey

Items in the textbook evaluation survey were constructed out of a previous research study by the author with a similar participant group, wherein participants identified potentials and barriers of OER adoption (Kimmons, in press). Participants were provided with the following instructions: “Please rate the textbook {textbook name here} on each of the following items.” Items included the following:

1. Accuracy: Accuracy of Content (e.g., accurate information)
2. Aesthetics: Aesthetics (e.g., cover, color usage)
3. Alignment: Standards Alignment
4. Conciseness: Conciseness (e.g., unnecessary content is removed)
5. Formatting: Formatting of Text (e.g., font size, spacing)
6. Media: Quality of Images and other Media
7. Readability: Readability of Content (e.g., grade level appropriateness)
8. Resources: Connections to other Meaningful Resources (e.g., Internet resources)
9. Supplements: Supplementary Materials (e.g., teacher's guide, equation references)
10. Timeliness: Timeliness of Content (e.g., up-to-date information)

Initially, the survey only included nine items, but after conducting the survey with the first group, it became clear that alignment should be included as an evaluation item, based upon feedback from participants. For this reason, results for the alignment item have fewer responses than the other items. All survey items were averaged together to create an overall evaluation of each textbook, and the evaluation survey was determined to be reliable with a very high Cronbach’s Alpha of .91 between the individual evaluation items.

Delimitations

One argument against this evaluation approach may be that because participants brought their current, copyright-restricted textbooks with them (some of which were several years old), it may not be fair to compare these to potentially newer open textbooks, because one might be older than the other. However, as discussed above, this is a reality in K-12 settings where the quality of materials is influenced by cost and affordability. If the goal of this study was to inform one-time adoption of such textbooks, then it might make sense to compare only new copyright-restricted textbooks with new open textbooks. However, our goal is to understand this phenomenon as it actually exists in schools, wherein new copyright-restricted materials are not a viable option in an on-going way (i.e. between adoption cycles). If a school has the funds to start a new cycle, then up-to-date texts of both types could be meaningfully compared, but once this adoption is made, a timer begins until another copyright-restricted adoption may be made again, and within this countdown, schools will perpetually face the real opportunity costs for having adopted copyright-restricted textbooks in contrast to open textbooks that might be more regularly updated. Thus, given the realities of school adoptions, comparing OER with what teachers currently have instead of what they could have in an unrealistic alternate reality seems to make more experimental sense. This decision may influence results of the O-C and A-C comparisons, but not the A-O comparison.

Furthermore, this study focuses upon a very specific set of open textbooks created by CK-12 and OpenStax for a limited number of subject areas and grade levels. As such, results should not be extrapolated to apply to all OER irrespective of source or subject. Rather, these specific sources, subject areas, and grade levels were selected specifically because they provided a number of widely-used, professionally-created resources that seemed to be promising grounds for exploring the issue of quality.

Results

Results of this study revealed differences in all three comparisons, as textbook evaluations descriptively improved on every item from copyright-restricted to open and again from open to open/adapted (cf. Fig. 1).

Fig. 1. Graphical results of textbook evaluations across all three type conditions.

Overall, the open textbooks were rated 22% higher than the copyright-restricted textbooks, the open/adapted textbooks were rated 16% higher than the open textbooks, and the open/adapted textbooks were rated 38% higher than the copyright-restricted textbooks (cf. Table 1).

To determine whether evaluation differences were statistically significant for the sample size, a series of one-way analysis of variance (ANOVA) tests of textbook type (independent) effects on evaluation items (dependents) yielded significant differences in all factors (cf. Table 2).

Given the significance of the overall tests, Bonferonni post hoc testing was utilized to determine differences between individual groups. The A-C comparison significantly differed in all factors, while the C-O and A-O comparisons differed on six of ten factors (cf. Table 3).

Qualitative feedback regarding the experience of adaptation was also gathered from participants in order to add depth to the quantitative results provided above. Some statements provided by the participants that illustrated their feelings upon completing the institute included the following:

• “Outstanding … extremely valuable. This [institute] has opened the door to a world of opportunity” (P1).
• “This was an extremely useful experience. I can't wait to start using this open [textbook] in my classroom” (P2).
• “This was AWESOME!! The product we created will be immensely useful and has given me a great tool for all of my students to use. The idea that all of the information is in one place will make my life a ton easier. I am ecstatic” (P3).
• “So useful. My district is interested in moving this direction due to its cost-effectiveness. I was mostly concerned with losing out on the copywrite-restricted resources that I already use in my class, but now I see how I can use both” (P4).
• “It's beyond easy” (P5).
• “This [institute] was way more engaging than most classes I've taken for teaching. I also feel like it's the best product I've ever came out with” (P6).
• “This … was [an institute] that I will actually apply almost daily in my class” (P7).
• “I am very motivated to use the [textbook] I created in my classroom next year and to share what I learned with my colleagues” (P8).
• “Please do whatever you can to keep these [institutes] going!” (P9).
• “This stuff will help teachers everywhere!!” (P10).

Discussion and Implications

This study provides two major findings for K-12 practitioners, researchers, and decision-makers. First, it is clear from the results that both the open textbooks and the open/adapted textbooks were evaluated by K-12 teachers as being superior to their copyright-restricted alternatives. Though part of this difference may be attributable to the relative ages of the resources and benefits emerging directly from cost that allows OER to be adopted outside of traditional textbooks cycles, results indicate that improved quality is also significant in other factors that have nothing to do with the age of the resource, including conciseness, readability, and additional resources. This means that the benefit of open textbooks extends beyond just cost and factors correlated to cost but seems also to be influenced by factors related to openness itself and the paradigmatic shift that open textbooks are permitting within schools, wherein teachers can take a more active role in resource evaluation, adoption, and modification. This means that though cost is certainly a major benefit of OER that influences other aspects of resource quality, such as timeliness, benefits of OER do not stop there. Rather, open textbooks are allowing for the creation of new resources that are better serving the needs and expectations of K-12 teachers.

Conclusion

This study has sought to establish how the quality of open textbooks and open/adapted textbooks can compare to copyright-restricted alternatives. In this study, open textbooks included pre-made, openly-licensed textbooks made available through an online non-profit, such as CK-12 or OpenStax, while open/adapted textbooks represented teacher remixes of open textbooks. Results of the study indicate that in the subject areas and grade levels presented, open textbooks were of higher quality than copyright-restricted textbooks, and open/adapted textbooks were of higher quality than open textbooks. These results mark an early step in guiding K-12 institutions in a shift toward open adoption and adaptation and provides evidence that open alternatives can be viable. This study has also demonstrated how a number of K-12 OER quality assurance factors may be included in teacher-based evaluations that may improve traditional quality assurance processes beyond content-expert peer-review, thereby ensuring that quality becomes a more concrete measure and reflects actual usefulness for classroom teachers.

References

Allen, I. E., & Seaman, J. (2014). Opening the curriculum: Open educational resources in U.S. higher education. Retrieved from http://www.onlinelearningsurvey.com/reports/openingthecurriculum2014.pdf

Atenas, J., Havemann, L., & Priego, E. (2014). Opening teaching landscapes: The importance of quality assurance in the delivery of open educational resources. Open Praxis, 6(1), 29–43.

Atenas, J., & Havemann, L. (2013). Quality assurance in the open: an evaluation of OER repositories. International Journal for Innovation and Quality in Learning, 1(2), 22-34.

Atkins, D., Brown, J. S., & Hammond, A. (2007). A review of the open educational resources (OER) movement: Achievements, challenges, and new opportunities (Report to The William and Flora Hewlett Foundation). Retrieved from http://www.hewlett.org/uploads/files/ReviewoftheOERMovement.pdf

Baker, J., Thierstein, J., Fletcher, K., Kaur, M., & Emmons, J. (2009). Open textbook proof-of-concept via Connexions. The International Review of Open and Distributed Learning, 10(5).

Baraniuk, R. G. (2007). Challenges and opportunities for the open education movement: A Connexions case study. In T. Iiyoshi & M. S. V. Kumar (Eds.) Opening Up Education: The Collective Advancement of Education through Open Technology, Open Content, and Open Knowledge, 229-246.

Bliss, T. J., Hilton III, J., Wiley, D., & Thanos, K. (2013). The cost and quality of online open textbooks: Perceptions of community college faculty and students. First Monday, 18, 1–7.

Bliss, T., & Patrick, S. (2013). OER state policy in K–12 education. iNACOL. Retrieved from http://www.inacol.org/resources/publications/inacol-reports/

Brown, J. S., & Adler, R. P. (2008). Minds on fire: Open education, the long tail, and learning 2.0. Educause Review, 43(1), 16-20.

Carey, T. & Hanley, G. L. (2007). Extending the impact of open educational resources through alignment with pedagogical content knowledge and institutional strategy: Lessons learned from the MERLOT community experience. In T. Iiyoshi & M. S. V. Kumar (Eds.) Opening up education: The collective advancement of education through open technology, open content, and open knowledge, 181-196.

CK-12. (n.d.). Mission | CK-12 foundation. CK-12. Retrieved from http://www.ck12.org/about/mission/

Clements, K. I., & Pawlowski, J. M. (2011). User-oriented quality for OER: understanding teachers' views on re-use, quality, and trust. Journal of Computer Assisted Learning, 28(1), 4-14.

Common Core State Standards Initiative. (n.d.). About the standards. Common Core State Standards Initiative: Preparing America’s Students for College & Career. Retrieved from http://www.corestandards.org/about-the-standards/

Creative Commons. (n.d.). About the licenses. Creative Commons. Retrieved from http://creativecommons.org/licenses/

D'Antoni, S. (2008). Open educational resources: The way forward. UNESCO and International Institute for Educational Planning.

de Hart, K., Chetty, Y., & Archer, E. (2015). Uptake of OER by staff in distance education in South Africa. The International Review of Research in Open and Distributed Learning, 16(2).

Dinevski, D. (2008). Open educational resources and lifelong learning. ITI 2008 - 30th International Conference on Information Technology Interfaces (pp. 117–122). IEEE. doi:10.1109/ITI.2008.4588393

DuFour, R. (2004). What is a professional learning community? Educational Leadership, 61(8), 6-11.

Duncan, S. M. (2009). Patterns of learning object reuse in the Connexions repository. All Graduate Theses and Dissertations. Paper 423. Retrieved from http://digitalcommons.usu.edu/etd/423

Federal Communications Commission. (2012). FCC chairman Genachowski joins Secretary of Education Duncan to unveil new “digital textbook playbook,” a roadmap for educators to accelerate the transition to digital textbooks. Retrieved from http://transition.fcc.gov/Daily_Releases/Daily_Business/2012/db0201/DOC-312244A1.pdf

Giles, J. (2005). Internet encyclopedias go head to head. Nature, 438, 900-901.

Hilton, J., & Laman, C. (2012). One college’s use of an open psychology textbook. Open Learning, 27(3), 265–272.

Hilton III, J., & Wiley, D. A. Examining the reuse of open textbooks. The International Review of Research in Open and Distributed Learning, 13(2).

K12 OER Collaborative. (2014). Request for proposals. K12 OER Collaborative. Retrieved from http://k12oercollaborative.org/rfp/

Kimmons, R. (2014). Developing open education literacies with practicing K-12 teachers. The International Review of Research in Open and Distributed Learning, 15(6).

Kimmons, R. (in press). Expansive openness in teacher practice. Teachers College Record, 119(1) (planned).

Mtebe, J. S., & Raisamo, R. (2014). Challenges and instructors’ intention to adopt and use open educational resources in higher education in Tanzania. The International Review of Research in Open and Distributed Learning, 15(1).

Next Generation Science Standards. (n.d.). About the standards development process. Next Generation Science Standards. Retrieved from http://www.nextgenscience.org/about-standards-development-process

OpenStax College. (n.d.). About us. OpenStax College. Retrieved from https://openstaxcollege.org/about

Organisation for Economic Co-Operation and Development (2007). Giving knowledge for free: The emergence of open educational resources. Retrieved from http://www.oecd.org/edu/ceri/38654317.pdf

Pawlowski, J.M. (2007). The quality adaptation model: adaptation and adoption of the quality standard ISO/IEC 19796-1 for learning, education, and training. Educational Technology & Society 10, 3–16.

Petrides, L., Jimes, C., Middleton-Detzner C., Walling J., & Weiss S. (2011). Open textbook adoption and use: Implications for teachers and learners. Open Learning: The Journal of Open, Distance and e-Learning. 26(1), 39-49.

Prasad, D., & Usagawa, T. (2014). Towards development of OER derived custom-built open textbooks: A baseline survey of university teachers at the University of the South Pacific. The International Review of Research in Open and Distributed Learning, 15(4).

Robinson, T. J., Fischer, L., Wiley, D., & Hilton, J. (2014). The impact of open textbooks on secondary science learning outcomes. Educational Researcher, 43(7).

Tonks, D., Weston, S., Wiley, D., & Barbour, M. (2013). “Opening” a new kind of school: The story of the Open High School of Utah. The International Review of Research in Open and Distance Learning, 14(1), 255-271. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1345/2419

United Nations Educational, Scientific and Cultural Organization. (n.d.). What are open educational resources (OER)? UNESCO. Retrieved from http://www.unesco.org/new/en/communication-and-information/access-to-knowledge/open-educational-resources/what-are-open-educational-resources-oers/

United Nations Educational, Scientific and Cultural Organization, & Commonwealth of Learning. (2011). Guidelines for open educational resources (OER) in higher education. Paris & Vancouver: UNESCO and COL.

Unwin, T., Kleessen, B., Hollow, D., Williams, J., Oloo, L. M., Alwala, J., ... Muianga, X. (2010). Digital learning management systems in Africa: Myths and realities. Open Learning: The Journal of Open and Distance Learning, 25(1), 5–23. doi:10.1080/02680510903482033

Usdan, J., & Gottheimer, J. (2012, February 3). FCC chairman: Digital textbooks to all students in five years [Web log post]. Federal Communications Commission. Retrieved from http://www.fcc.gov/blog/fcc-chairman-digital-textbooks-all-students-five-years

Walker, E. (2007). Evaluating the results of open education. In T. Iiyoshi & M. S. V. Kumar (Eds.) Opening up education: The collective advancement of education through open technology, open content, and open knowledge, 77-88.

William and Flora Hewlett Foundation, The. (n.d.). Open educational resources. The William and Flora Hewlett Foundation. Retrieved from http://www.hewlett.org/programs/education/open-educational-resources

Wiley, D. (2013). On quality and OER [Web log post]. iterating toward openness. Retrieved from http://opencontent.org/blog/archives/2947

Wiley, D., & Green, C. (2012). Why openness in education? In D. Oblinger (Ed.), Game changers: Education and information technologies, 81-89. Educause.

Wiley, D., Hilton, J., Ellington, S., & Hall, T. (2012). A preliminary examination of the cost savings and learning impacts of using open textbooks in middle and high school science classes. The International Review of Research in Open and Distance Learning, 13(3), 262-276.

Wilson-Strydom, M. (2009). The potential of open educational resources (Report for OER Africa). South African Institute for Distance Education. Retrieved from http://www.oerafrica.org/understandingoer/UnderstandingOER/ResourceDetails/tabid/1424/mctl/Details/id/36389/Default.aspx

© Kimmons