International Review of Research in Open and Distributed Learning

Volume 21, Number 2

April - 2020

Open Textbooks: Quality and Relevance for Postsecondary Study in The Bahamas

Edward Bethel
University of The Bahamas

Abstract

Open educational resources (OER), are openly licensed text, media, and other digital and analog assets that are useful for teaching, learning, and research. Recent research has shown that in courses where open textbooks are assigned, students perform as well as or better than students in similar courses with commercially licensed textbooks. Despite cost savings and demonstrated effectiveness, perceptions about quality, relevance, ease of access, and other concerns persist. The objectives of this study were twofold: (a) to develop a practical and reusable measure for evaluating open textbook quality in terms of pedagogy, openness, accessibility, and relevance; and (b) to use the measure to rate the quality and relevance of open textbooks for use in higher education in The Bahamas. The study confirmed the viability of the quality measure as a practical tool to assess open resources and found that the open textbooks studied were accessible and well matched to course content, but of varying quality. More study is needed to explore ways to increase faculty adoption, use, adaption and production of OER.

Keywords: open educational resources, textbooks, access, quality, relevance

Introduction

Textbooks play an important role in student course outcomes. As the cost of university and college textbooks continues to rise, however, students increasingly cannot afford to purchase them. Increasing costs can lead students to purchase fewer texts and hence do poorly in courses, take fewer courses, or drop out of programmes of study altogether (Florida Virtual Campus, 2016). While textbook costs are a burden to all students, poor students are particularly impacted as they are least able to shoulder high educational costs. In effect, increasing costs present an additional educational barrier. Simply put, when costs increase, access is reduced.

While the population of The Bahamas is spread over 700 islands and cays, most residents live in the nation’s capital city, Nassau, on the island of New Providence. Access to education in general, and higher education in particular, is challenging for residents on the Family Islands - the name given to the 30 or so populated islands other than New Providence. Even when Family Island residents can afford expensive texts, mail delivery is limited, making it more difficult for them to access learning materials. Open educational resources (OER) and open textbooks are promising alternative to commercial textbooks, and availability of OER is expanding (Wiley, Bliss, & McEwen, 2012). OER and open textbooks released in digital format seem especially suited to the islands of The Bahamas as Internet and mobile data connectivity is widespread, even in some of the more remote settlements.

Availability is not a sufficient condition for adopting open textbooks. Several factors may inhibit OER adoption including (a) relevance (Albright, 2005); (b) findability (Larson & Murray, 2008); (c) existing teaching practices (Hattaka, 2009); and (d) quality (Albright, 2005; D’Antoni 2006). Importantly, texts must be or must be perceived to be of a certain quality, to be considered as replacements for existing texts (D’Antoni, 2006). Despite cost savings, perceptions about quality, ease of use, ease of access, and match to course may discourage faculty from making the effort to find, review, and use open textbooks. At present, these explanations for the low adoption rate are speculative, as no formal study of OER and open textbook use in The Bahamas has been attempted to date.

This study focused on the question of open textbook quality. Specifically, the intention was to design and build an evaluation tool that would be easy to use and adapt to assess quality in a variety of different types of textbooks, regardless of content. This study made two unique contributions. First, it demonstrated the use of a new and easily reproducible approach for evaluating open textbooks quality on five dimensions: (a) pedagogy, (b) openness of license, (c) cost, (d) available media, and (e) relevance. Second, this study used this approach to evaluate potential OER texts for use in higher education courses in The Bahamas.

Objectives

The objectives of this study were twofold: (a) to develop a practical and reusable measure for evaluating open textbook quality in terms of pedagogy, openness, accessibility, and relevance; and (b) to use the measure to rate the quality and relevance of open textbooks for use in higher education in The Bahamas. Specifically, the study attempted to answer the following questions:

• How well did open textbooks score on a simple scale of pedagogical quality?
• How open were the textbooks in terms of their licenses?
• How accessible were the open textbooks in terms of cost?
• How accessible were the open textbooks in terms of the media delivery formats?
• How relevant were the open textbooks in terms of the degree to which they match course content at postsecondary institutions in The Bahamas?

Theoretical Framework

Textbooks can play an important role in student course outcomes. With students paying as much as $1,200 annually on average for books and supplies in the US (College Board, 2015), increasingly, students cannot afford to purchase textbooks and hence purchase fewer texts, take fewer courses, or may even drop out altogether (Florida Virtual Campus, 2016). Few local texts are available to Bahamian higher education students, and the cost of importing texts means that costs in The Bahamas are even higher. The University of The Bahamas website estimates typical annual costs of books and supplies for undergraduates at B$1,500 (University of The Bahamas, 2017). Textbook costs are a burden and can serve as a barrier. Added to which, off-site students, such as those studying online or at Family Island learning centres, are further disadvantaged, because in addition to high costs, they have to wait for books to be mailed to them.

Open educational resources would seem to be a viable and affordable alternative to traditional, commercially licensed textbooks. In addition to the cost savings of OER in general, electronic OER are particularly accessible and downloadable via the Internet, immediately lowering barriers to access for students in Nassau and the Family Islands alike. Despite over a decade in research and development of OER, increasing availability and ease of locating OER, and despite The Bahamas being one of the few countries in the Caribbean with a strategy for information and communications technology in education that promotes OER (Hoosen, 2012; Ministry of Education, 2017), OER use in The Bahamas seems a promise rather than an actuality (George, 2015). The reasons for slow adoption in The Bahamas have not been researched, but in other locations, issues have included: (a) questions of quality; (b) concerns about content match; (c) lack of awareness of OER; (d) difficulty in locating appropriate OER; (e) lack of recognition for faculty efforts to adopt, adapt, and reuse OER; and (f) complicated textbook approval processes (Allen & Seaman, 2016; Hilton, 2016; Mishra, 2017). Specifically, Wiley et al. (2012) identified several challenges to the adoption and reuse of OER: (a) discovery, (b) concerns about quality, (c) localization to meet local needs, and (d) lack of skills to adapt and remix content. In this study, open textbook quality was explored on a variety of levels.

Alsagoff (2016) identified four approaches to assessing OER and open textbook quality: (a) faculty (peer) reviews, where college and university faculty review the content against a standard set of criteria, similar to how they would review a manuscript for a peer-reviewed journal; (b) faculty ratings, where faculty rate their perception of quality after having used the content for a course; (c) student ratings, where students rate their perception of quality after using the text in a course; and (d) efficacy research where researchers compare the outcomes of students using openly licensed resources versus commercially licensed resources for different sections of the same course. While criteria-based evaluation rates textbooks prior to use, the other three approaches rely on ratings after textbook use. Examples of studies referencing faculty and student ratings consistently reported positive ratings of open textbooks by faculty and students (Bliss, Hilton, Wiley, & Thanos 2013; Jung, Bauer, & Heaps 2017; Vander Waal Mills, Gucinski, & Vander Waal 2019). Similarly, in a review of efficacy research on OER and pen Textbooks, Hilton (2016) found that OER were at least as effective as licensed materials at promoting student achievement.

This study needed a criteria-based framework to rate textbooks prior to use, as the intention was to determine textbook quality in order to encourage adoption and use. Harvey and Green (1993) identified five dimensions of quality in higher education, three of which are most applicable to OER—transformative learning, fitness for purpose, and cost efficiency. Expanding on these, Kawachi (2014) developed the four-part TIPS framework for quality assurance of Open Educational Resources, version 2.0 (TIPS 2.0), as a tool to guide faculty in creating and authoring their own OER. TIPS is an acronym for the four categories of quality the framework is designed to evaluate: (a) teaching and learning processes, (b) information and material content, (c) presentation product and format, and (d) technical systems and technology. TIPS 2.0 revised version 1.0, incorporating feedback from OER experts, online OER forums, and teacher-practitioners (Kawachi, 2014). In total, TIPS 2.0 identified 38 quality criteria around the themes of pedagogy, learner-centeredness, openness, accessibility, relevance, adaptability, and content quality (Kawachi, 2014).

Although TIPS 2.0 provided a comprehensive quality framework for evaluating OER and open textbooks, its length and reliance on content expert knowledge was not well suited to this study. For this study, a more practical two-part assessment instrument, derived from the TIPS framework, was developed to evaluate textbook quality on five dimensions—pedagogy, openness of license, cost, available media, and relevance.

Pedagogy

The pedagogical quality of each text was rated using five items adapted from TIPS:

1. The use of learning objectives to signal the intent and purpose of the learning content. Clearly articulated learning objectives can boost learning in multiple ways including: (a) helping students differentiate types of knowledge; (b) ensuring students focus on the appropriate content and skills; (c) scaffolding independent, self-directed learning; and (d) enhancing meta-cognitive skills (Eberley Center, 2019).
2. The use of unit learning activities to engage and stimulate the learner. Effective learning activities can deepen learning by encouraging active as opposed to passive learning (Fink, 2003).
3. The use of diagrams and charts to enrich the content and facilitate retention. Decades of research has demonstrated that students learn better from text and images together than from images alone (Mayer, 2008; Nilson, 2010).
4. The use of practice exercises to deepen learning. Practice exercises give students the opportunity to apply the knowledge and skills they have learned in different contexts (Fink, 2003).
5. The use of pre- and post-assessments to deepen learning and help learners monitor their progress. Self-testing before, during, and after learning is not only an effective way to monitor progress, but also an effective tool for learning itself (Rohrer & Pashler, 2010).

Openness

Openness of license was rated according to a scale shown in Figure 1. Licenses are ranked inversely according to the limits on sharing of the resource—the more the limits, the lower the ranking (Green, 2014).

Figure 1. Openness scale. From “Open Education: The Moral, Business and Policy Case for OER,” by C. Green, 2014 (https://www.slideshare.net/cgreen/updated-keynote-slides-october-2014). CC BY.

From the openness scale, three main principles of open licensing are used to differentiate the degree of openness of textbook licenses: (a) whether the resources can be reused at all (the basic principle of open licensing); (b) whether the resources can be re-used for commercial use or not; and (c) whether the resources can be repurposed and adapted (the principle of derivation). As Green (2014) demonstrated, the litmus test distinguishing OER from non-OER shareable resources is whether the license allows for repurposing and derivation. Neither of the NoDerivative (ND) licenses qualify as OER. Because all textbooks evaluated in this study were located in OER repositories, they were expected to be openly licensed in one form or another.

Accessibility: Cost

This study opened with reference to the increasing costs of commercially licensed textbooks and the impact those costs have on the accessibility of higher education. From an economic and social justice point of view, OER textbooks represent an improvement over commercially licensed texts (Hodgkinson-Williams & Trotter, 2018; Wiley, 2017). Affordability must therefore be one of the key criteria by which the accessibility of textbooks is judged. Lower cost texts were rated more highly. Because all textbooks evaluated in this study were located in OER repositories, they were expected to be either completely free or free in some format.

Accessibility: Media

There is disagreement in the literature about whether OER must be electronic or otherwise. Electronic distribution facilitates sharing, re-use, and repurposing. Moreover, electronic formats can be edited to enable accessibility for visually impaired learners (Butcher & Moore, 2015). HTML and PDF formats are particularly portable as they can be accessed on any electronic device. On the other hand, at least one definition of OER suggests that they must still be available in suitable formats (presumably print) to facilitate access for those learners who still do not have access to the Internet (Cape Town Open Education Declaration, 2007). Accessibility is also rated according to the number of media formats in which the OER are published.

Relevance: Match

Relevance was rated by comparing textbooks to local higher education course content. Course outlines were subdivided into topics and then topics were subdivided into the specific learning objectives. Each learning objective was compared with the open resource to determine how completely, if at all, each learning objective was covered in the text.

Methodology

Textbooks from two open textbook repositories were identified for evaluation: OpenStax CNX (https://cnx.org/) and BCcampus OpenEd (https://open.bccampus.ca/). Senior undergraduate students were recruited as research assistant coders (at the time of the study, the higher education institution of the study had few graduate students, none of whom were available for the study). The research assistants participated in two training sessions, one on the process of coding for each of the research instruments. Included in both sessions were brief introductions to OER in general and the use of open textbooks specifically. For each instrument, two coders were anonymously assigned to evaluate each textbook. Scores were compared for consistency. Where coders disagreed, these ratings were re-evaluated to arrive at a mutually acceptable coding. All coding was done online using document sharing facilities of Google Drive.

Instrument One (Pedagogy and Accessibility)

See Appendix A for a copy of the complete form for evaluating textbook pedagogy and accessibility.

Pedagogy. Two coders rated each text on a variety of quality items suggested by the TIPS framework: (a) use of learning objectives, (b) unit activities, (c) diagrams and charts, (d) practice exercises, and (e) unit pre-assessments and post-assessments. Item ratings were totaled to give an overall quality score for each textbook. Textbooks with significant disagreements were noted and re-evaluated by a third coder. Coders could add optional comments as well.

License. Textbooks were coded according to their license. The licenses were ranked in order of openness from more to less open according to the scale illustrated in Figure 1, namely:

• PD or CC0: Public Domain
• CC BY: Attribution
• CC BY-SA: Attribution ShareAlike
• CC BY-NC: Attribution NonCommercial
• CC BY-NC-SA: Attribution NonCommercial ShareAlike
• CC BY-ND: Attribution NoDerivatives
• CC BY-NC-ND: Attribution NonCommercial NoDerivatives
• ©: Copyright - all rights reserved

Texts with CC BY-ND, CC BY-NC-ND, or © were not considered to be OER.

Cost. The lowest-cost version of each textbook was coded on a scale of 0 to 5 with 0=$0 (free), 1=$0.01 to $24.99, 2=$25.00 to $49.99, 3=$50.00 to $74.99, 4=$75.00 to $99.99, and 5=$100 or greater.

Media. Coders recorded each of the formats in which the resource was available: print, HTML, PDF, ePub, iBook, Kindle, and other. Accessibility was ranked according to the number of different formats available.

Instrument Two: Relevance

See Appendix B for a copy of the complete form for evaluating textbook relevance.

Course match. Textbooks were identified that might be suitable for specific courses at a local higher education institution. Course outlines were used to detail specific topics, subtopics and learning objectives to be taught. Textbooks were then evaluated on how completely they covered each learning objective from the course outlines using a three-point scale (2=completely, 1=partially, 0=not at all). Ratings totals were given as a percentage of the highest possible score. Coder totals were compared and where significant differences were noted, textbooks were re-evaluated by a third coder.

Data Sources

Textbooks from OpenStax CNX and BCcampus OpenEd were identified that targeted higher education academic courses that were likely to have content matching to local higher education courses. Technical books were not included.

Results

Textbook Identification

In the two repositories, a total of 71 college textbooks were identified that were likely to be a good match with university or college courses. Some titles were cross-listed, resulting in some degree of duplication. In all, 30 texts were listed in both repositories, meaning that a total of 41 unique textbooks were identified. Table 1 shows how these textbooks were classified by discipline.

Table 1

Distribution of OER Textbooks by Discipline

Pedagogical Quality

Scores for pedagogical quality ranged from 1 to 9 out of a possible 9, with modal scores of 4 and 7 and a mean of 5.4 (SD 2.2; see Figure 2). Overall scores revealed only part of the picture, however. When individual item scores were compared, a clear pattern emerged. Mean item scores expressed as a percentage of the maximum possible score were ranked from highest to lowest: (a) stated learning objectives (78%); (b) use of diagrams, charts, and other visuals (68%); (c) use of practice activities (66%); (d) use of learning activities (61%); (e) use of post-assessments (48%); and (f) use of pre-assessments (7%). Of note, three items: use of visuals, practice, and learning activities scored on all measure levels (i.e., extensive use, limited use and no use). For these three items, when extensive and limited use were collapsed into the binary use versus no use categories, each approached 90% or higher, indicating almost universal use of these three instructional strategies. It was clear that a significant proportion of the textbooks assessed adhered to a fairly consistent model of textbook construction that included the use of stated learning objectives, visuals, learning activities, and practice activities. The use of unit assessments was rare, particularly in terms of pre-assessments.

Figure 2. Distribution of pedagogical quality scores.

License

Approximately three quarters (31 out of 41) of the texts were licensed under the CC BY, one of the most open of the Creative Common licenses. Under this license, faculty and students can freely access, share, reuse, and re-write content provided attribution is given to the original author (see Figure 3). As expected, given that all textbooks were located in open repositories, all of the licenses qualified as OER under the openness score, even if some licenses were not as open as the CC BY.

Figure 3. Distribution of textbooks by license.

Cost

Of the 41 textbooks evaluated, 40 were free and the remaining one cost between $25 and $49.99, though some texts added additional charges if students wanted to access printed copies. Nonetheless, all but one of the texts were free and the one that was not free, was still less expensive than commercial textbooks. Again, as expected, almost all the textbooks were completely free or free in some format.

Media

All textbooks were available in multiple media formats, with 31 of 41 available in three or more formats. All textbooks were available electronically, either as PDF downloads or HTML or both (see Figure 4). Specialized electronic formats (i,e., ePub, iBook, Kindle) were somewhat less common.

Figure 4. Textbook distribution by the number of available media formats per title.

Course Match

From the total sample, 17 textbooks were selected to be evaluated against 14 different courses. Courses were matched against as many as three different texts. Similarly, the same text could be matched against more than one course. In total, 29 unique pairings of textbooks and courses were evaluated. Two coders rated each text using the form shown in Appendix B. Where there were disagreements, the author re-coded the text to resolve the disagreement; re-coding successfully resolved all the disagreements. The degree of match for each of the courses is shown in Table 2.

Table 2

Rate of OER Textbook Match to Postsecondary Courses at a Postsecondary Institution in The Bahamas

In general, the textbooks were well-matched to the courses. The mean degree of match was 74%, but this measure of central tendency may be misleading as two texts were particularly poorly matched to their respective courses (6% and 14%, respectively.) In general, textbook content was well-aligned with courses at the tertiary institution being studied, as demonstrated by the distribution of textbooks by degree of match (Figure 5.) The modal category is 80 to 89% match and a majority of textbook-course pairings (22 out of 29) fell within the top three categories.

Figure 5. Distribution of textbook-course pairings rated by degree of match.

Discussion

This study set out to explore the quality of potential open textbooks on a variety of levels. In particular, open textbooks were evaluated on measures of pedagogical quality, openness, cost, available media, and local relevance.

Pedagogical quality varied across the spectrum. Scores clustered around two modes, namely 4 and 7, indicating that texts exhibited consistent differences in instructional design. As noted earlier, almost all texts scored highly on certain key items: (a) learning objectives, (b) diagrams and charts, (c) learning activities, and (d) practice exercises. Those texts scoring higher made more consistent use of post-assessments. Few texts employed pre-assessments. This heuristic approach of evaluating texts using instructional design principles needs to be balanced by comparing quality ratings to faculty and student ratings of open textbooks as in other studies (e.g., Jung, Bauer, & Heaps, 2017; Vander Waal Mills et al., 2019) and to the results of efficacy studies (e.g., Hilton, 2016).

Open licensing of texts was expected, as all texts were selected from a database of open resources. That there was variety at all in the types of licenses is perhaps of more interest. This may reflect a difference in approach between the repositories. On the one hand, OpenStax CNX explicitly encourages sharing, reuse, remixing and author attribution (OpenStax CNX, 2019). This means that most of the content in their repository was licensed under the CC BY license. BCcampus OpenEd on the other hand, takes a broader approach to Open Textbooks and advocates on behalf of CC licensing in general (BCcampus, n.d.). Resources stored in their repository are likely to have a wider variety of licenses than those in OpenStax CNX.

Unsurprisingly, all but one of the texts were free in some format or another (although some charged for hard copies). Even the text that was not free, cost in the $25 to $49.99 range, still considerably more affordable than commercial texts typically costing between $100 and $300 each.

As noted earlier, all texts were available electronically and in multiple formats. This is not surprising as it costs very little to migrate documents between different electronic formats. By publishing under an open license to begin with, OER publishers demonstrated a commitment to accessibility by publishing in as many formats as possible. Perhaps a more interesting question is why all OER publishers do not distribute in as many formats as possible?

Given the uniqueness of postsecondary courses, one would normally expect much variation in the degree to which texts matched courses. The high average degree of course match in the dataset (mean 74% and modal grouping 80-89%) was somewhat of a surprise. One explanation is that both databases had a large proportion of texts for introductory general courses that would not rely heavily on local content, for example mathematics, natural science, and business studies texts. This high degree of course match might be the strongest factor that could influence faculty to evaluate and potentially adopt texts for their courses.

Significance and Conclusion

This study set out to create a practical and easy-to-use tool to assess the quality of open textbooks and use that tool to evaluate the suitability of those textbooks for higher education courses in The Bahamas. The quality assessment tool proved fit for purpose but could be strengthened by comparison with other measures of quality as described below. Comparison of quality scores with content expert ratings would give further insight into the effectiveness of this quality measure. Variation in measures of pedagogical quality may reflect different approaches to textbook design, particularly if authors are viewing texts as tools to encourage self-directed learning on the one hand or instructor-led learning on the other.

Unsurprisingly, when the assessment measure was used to evaluate texts in two OER repositories, all texts scored highly on openness and cost. Both of these measures seem more suited to deciding whether a resource qualifies as OER rather than evaluating the quality of the resource. On the other hand, Wiley (2017) argued that because education is the sharing of public goods of knowledge, skills, and attitudes, openness of learning content is a fundamental and core value. Copyright law and commercial licensing can have the effect of restricting public goods and turning them into private goods. In this interpretation, the first measure of quality must be openness, as this measures how well the resource is aligned with educational core values (Wiley, 2017).

Educational costs are increasing and as a result, students have difficulty financing higher education. Expensive, commercially licensed textbooks present a significant cost barrier, notably for the traditionally underserved students whether in the Family Islands of The Bahamas, or any other remote settlement worldwide. Open textbooks can significantly reduce barriers and facilitate alternative delivery mechanisms such as online learning for all students. Faculty adoption of open textbooks and OER is slow, often because there is a distrust of the quality of open textbooks. This study investigated the quality of open texts and found them to score highly on several quality measures.

This study can also be useful in raising awareness of suitable open textbooks for higher education use, in that it addresses several common barriers to use and adoption. Potential suitable textbooks were found to be readily available, inexpensive or free, and to match well with local higher education course content. Moreover, as most of them are licensed under the CC BY license, faculty are free to use, excerpt, adopt, and adapt these materials for their instructional needs.

The study is also useful because it demonstrates a new, easily reproducible method for evaluating the suitability of OER and open textbooks for adoption in courses. Next steps might include:

• public campaigns explaining the findings to increase awareness of OER by faculty and students;
• an exploration of faculty awareness of OER and their use;
• an exploration of faculty capacity to use, reuse, adapt, and create OER;
• capacity development workshops and training on how to find, use, reuse, and create OER;
• an exploration of how faculty presently select textbooks for courses;
• validation of the quality tool used in this study by:
  • faculty measures of quality, satisfaction, and attitudes;
  • students’ measures of satisfaction and attitudes;
  • comparing quality scores for open texts with comparable commercial texts; and
  • comparing quality scores for open texts with student outcomes; and
• an evaluation of student outcomes in courses using open texts compared to students in similar courses using commercial texts.

This study and the next steps discussed above have focused primarily on use and reuse of OER and open textbooks by faculty in their courses. The longer-term goal must be the creation of sustainable infrastructure to embed the use and production of OER and open textbooks in higher education ecosystems. Changes in policy and practice around knowledge creation and sharing are needed to foster a mindset change toward openness in education.

References

Albright, P. (2005). Internet discussion forum: Open educational resources, open content for higher education. Final forum report. UNESCO International Institute for Educational Planning. Retrieved from https://docs.iiep.unesco.org/I009621.pdf

Allen, I. E., & Seaman, J. (2016, July). Opening the textbook: Educational resources in U.S. higher education, 2015-16 : Babson Survey Research Group. Retrieved from https://www.onlinelearningsurvey.com/reports/openingthetextbook2016.pdf

Alsagoff, Z. A. (2016). Is the quality of open textbooks equal to that of commercial textbooks? Retrieved from http://discourse.col.org/t/is-the-quality-of-open-textbooks-equal-to-that-of-commercial-textbooks/139

BCcampus. (n.d.). Advocating for open education Retrieved from https://open.bccampus.ca/advocate-for-open-education/advocating-for-open-education/

Bliss, T. J., Hilton, J., III, Wiley, D., & Thanos, K. (2013). The cost and quality of online open textbooks: Perceptions of community college faculty and students. First Monday, 18(1). doi: 10.5210/fm.v18i1.3972

Butcher, N., & Moore, M. (2015). Understanding open educational resources. Vancouver, BC: Commonwealth of Learning. Retrieved from http://oasis.col.org/bitstream/handle/11599/1013/2015_Butcher_Moore_Understanding-OER.pdf?sequence=1&isAllowed=y

College Board. (2015). Trends in college pricing 2015. Retrieved from http://trends.collegeboard.org/sites/default/files/2015-trends-college-pricing-final-508.pdf

Commonwealth of Learning. (2012). Fostering governmental support for OER internationally (FGSOERI): Caribbean. Vancouver, BC: Commonwealth of Learning and UNESCO. Retrieved from http://oasis.col.org/bitstream/handle/11599/704/FGSOERI_Caribbean-Summary-Report_1201.pdf?sequence=1&isAllowed=y

Cape Town Open Education Declaration. (2007, September 15). Retrieved from https://www.capetowndeclaration.org/read-the-declaration

D’Antoni, S. (2006, June 13). Open educational resources: Deliberations of a community of interest (Keynote address). The International Council for Open and Distance Education Standing Conference of Presidents, Rectors, and Chief Executive Officers, Lillehammer, Norway. Retrieved from https://hewlett.org/wp-content/uploads/2016/08/OER_DeliberationsofaCommunityofInterest.pdf

Eberly Center. (2019). The educational value of course-level learning objectives/outcomes. Retrieved from https://www.cmu.edu/teaching/resources/Teaching/CourseDesign/Objectives/CourseLearningObjectivesValue.pdf

Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass.

Fischer, L., Ernst, D., & Mason, S. L. (2017). Rating the quality of open textbooks: How reviewer and text characteristics predict ratings. The International Review of Research in Open and Distributed Learning, 18(4), 142-154. doi: 10.19173/irrodl.v18i4.2985

Florida Virtual Campus. (2016). 2016 Florida student textbook survey. Tallahassee, FL: Florida Virtual Campus. Retrieved from https://florida.theorangegrove.org/og/items/3a65c507-2510-42d7-814c-ffdefd394b6c/1/

George, N. (2015). Technology-Enabled learning in the Commonwealth Caribbean countries: A baseline study. Vancouver, BC: Commonwealth of Learning. Retrieved from http://oasis.col.org/handle/11599/1210

Green, C. (2014). Open education: The moral, business and policy case for OER. Retrieved from https://www.slideshare.net/cgreen/updated-keynote-slides-october-2014

Harvey, L., & Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education, 18, 9-34. doi: 10.1080/0260293930180102

Hatakka, M. (2009). Build it and they will come? Inhibiting factors for reuse of open content in developing countries. The Electronic Journal of Information Systems in Developing Countries, 37(1), 1-16. doi: 10.1002/j.1681-4835.2009.tb00260.x

Hilton, J., III. (2016). Open educational resources and college textbook choices: A review of research on efficacy and perceptions. Educational Technology Research and Development, 1(18), 573-590. doi: 10.1007/s11423-016-9434-9

Hodgkinson-Williams, C. A., & Trotter, H. (2018). A social justice framework for understanding open educational resources and practices in the global south. Journal of Learning for Development, 5(3), 204-224. Retrieved from https://jl4d.org/index.php/ejl4d/article/view/312

Hoosen, S. (2012). Survey on governments’ open educational resources (OER) policies. Vancouver, BC: Commonwealth of Learning. Retrieved from http://oasis.col.org/handle/11599/291

Jung, E., Bauer, C., & Heaps, A. (2017). Higher education faculty perceptions of open textbook adoption. The International Review of Research in Open and Distributed Learning, 18(4), 123-141. doi: 10.19173/irrodl.v18i4.3120

Kawachi, P. (2014). Quality assurance guidelines for open educational resources: TIPS framework. New Delhi, India: The Commonwealth Educational Media Centre for Asia. Retrieved from http://oasis.col.org/handle/11599/562

Larson, R. C., & Murray, E. (2008) Open educational resources for blended learning in high schools: Overcoming impediments in developing countries. Journal for Asynchronous Learning Networks, 12, 85-103. Retrieved from https://www.learntechlib.org/p/104082/

Mayer, R. E. (2008). Applying the science of learning: Evidence-based principles for the design of multimedia instruction. American Psychologist, 63(8), 760-769. doi: 10.1037/0003-066x.63.8.760

Ministry of Education. (2017). ICT in education strategic plan. Unpublished policy document.

Mishra, S. (2017). Promoting use and contribution of open educational resources. New Delhi, India: Commonwealth Educational Media Centre for Asia. Retrieved from http://oasis.col.org/bitstream/handle/11599/2659/2017_Mishra_Promoting-Use-Contribution-of-OER.pdf?sequence=1&isAllowed=y

Nilson, L. B. (2010). Teaching at its best: A research-based resource for college instructors (3rd ed.). San Francisco, CA: Jossey-Bass.

OpenStax CNX. (2019). About us. Retrieved from https://cnx.org/about

Rohrer, D., & Pashler, H. (2010). Recent research on human learning challenges conventional instructional strategies. Educational Researcher, 39(5), 406-412. doi: 10.3102/0013189X10374770

University of The Bahamas. (2017). University of The Bahamas: Tuition and fees. Retrieved from http://www.ub.edu.bs/admissions/tuition-fees/

Vander Waal Mills, K. E., Gucinski, M., & Vander Waal, K. (2019). Implementation of open textbooks in community and technical college biology courses: The good, the bad, and the data. CBE—Life Sciences Education, 18(3). doi: 10.1187%2Fcbe.19-01-0022

Wiley, D. (2017). The evolving economics of educational materials and open educational resources: Toward closer alignment with the core values of education. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (4th ed.; pp. 314-322). New York, NY: Pearson Education.

Wiley, D., Bliss, T. J., & McEwen, M. (2014). Open educational resources: A review of the literature. In Handbook of research on educational communications and technology (pp. 781-789). New York, NY: Springer. doi: 10.1007/978-1-4614-3185-5_63

Appendix A

Textbook Evaluation Form 1 (Pedagogy and Accessibility)

Appendix B

Textbook Evaluation Form 2 (Relevance)

One form to be completed for each course. List texts to be applied to that course as Textbook 1, Textbook 2, Textbook 3, and so on.

Open Textbooks: Quality and Relevance for Postsecondary Study in The Bahamas by Edward Bethel is licensed under a Creative Commons Attribution 4.0 International License.