Abstract

The field of open education and research on the topic has notably expanded since the introduction of the term 20 years ago. Given these developments, a framework to structure research inquiry is necessary to ground and organize findings in open education. We propose the SCOPE framework for open education research: social justice, cost, outcomes, perceptions, and engagement. In this article, we explain how this framework emphasizes the need for social justice at the forefront of open education research. In addition, we incorporate existing theories in social justice, motivation, cognition, pedagogy, and engagement into each of the components to propose theoretical connections to future open education research. We suggest areas in which future research is needed. Finally, we conclude with suggestions as to how the SCOPE framework may be useful when connecting open education to open science and open scholarship as well as a call for considering intersectionality and critical methods in quantitative research (QuantCrit) in future inquiry.

Keywords: engagement, open education, motivation, research framework, social justice

Introduction

Open education is the term used for developing and using resources, materials, and pedagogies which are freely accessible and shareable with others through open and flexible licensing. After the term open educational resources (OER) was coined in 2002 to describe openly licensed teaching and learning materials, research to examine the efficacy of these resources followed (Bliss & Smith, 2017). This burgeoning field of inquiry prompted the need for a framework to provide organization and cohesion to the research findings. The COUP framework, consisting of cost, outcomes, use, and perceptions of OER, addressed that need (Bliss et al., 2013). It is an important contribution to the field, but the concepts of inclusion and innate human right to knowledge essential to the open education movement (Biswas-Diener & Jhangiani, 2017; Deimann & Farrow, 2013) are not explicitly emphasized in the COUP research framework.

Open education research has shifted from primarily focusing on OER adoption to examining the novel teaching and learning techniques afforded by open licensing (open pedagogy; DeRosa & Robinson, 2017), which could affect student engagement. Open pedagogy, also known as OER-enabled pedagogy (Wiley & Hilton, 2018) and open educational practices (Tietjen & Asino, 2021), provides opportunities for students to “add value to the world” (Van Allen & Katz, 2019, p. 312). This value is added by creating, editing, revising, and sharing their work under a Creative Commons license (e.g., photographs, videos, websites, Wikipedia articles; Van Allen & Katz, 2019). This is based on constructivist pedagogy, which is “based on the idea that knowledge is constructed during experience, such as real-world problem solving, and by reflecting on those experiences” (Santos, 2018, p. 173).

Because of these advancements in the open education field, we propose the development and expansion of the COUP framework: the SCOPE framework (see Table 1). The SCOPE framework comprises social justice, cost, outcomes, perspectives, and engagement. In this discussion, we describe each component of the SCOPE framework and how it may be used to develop future research questions and purposes. We begin with background on the COUP framework.

Table 1

SCOPE Framework Concepts, Definitions, and Examples

COUP Framework

In 2013, the COUP framework was proposed to organize research on the efficacy of adopting open educational resources in the contexts of costs, outcomes, use, and perceptions (Bliss et al., 2013). This framework has been a useful tool for grounding inquiry into OER for many empirical studies (e.g., Nusbaum & Cuttler, 2020; Tlili et al., 2022) as well as reviews of research findings (e.g., Clinton, 2019; Hilton, 2016, 2020). The researchers who developed the COUP framework provided helpful suggestions for methods of inquiry for testing the impact of OER adoption (Open Education Group, 2023). For example, cost includes financial expenses with textbooks and other course materials, tuition revenue due to changes in enrollment patterns, and student enrollment intensity (number of credits per term; Open Education Group, 2023). In this way, the COUP framework has been particularly useful in assessing differences between using traditionally copyrighted materials compared to open educational resources. However, one of the authors of the COUP framework and member of the Open Education Group has provided valid critique of open education research stating that “a stronger theoretical framework, including a hypothesized explanatory mechanism, is required for comparative research to provide useful insights” (Wiley, 2021, p. 412). In other words, in order to understand and facilitate the potential for open education to foster learning, we, as a field, need theoretically-grounded inquiry into why and how open educational resources (and open education in general) relate to learning (Wiley, 2021). The SCOPE framework is intended to expand upon the COUP framework to include theoretical frameworks for open education research. In addition, we explicitly update the COUP framework to include the dimensions of social justice and engagement.

Social Justice

The SCOPE framework leads off with social justice because commitment to inclusion and equity is fundamental to the ethos of open education (Bali et al., 2020; Croft & Brown, 2020; Lambert, 2018) and education in general (Ladson-Billings, 2014). Education equity is impossible without the centering of justice. Social justice is “... an ideal condition in which all members of a society have the same rights, protections, opportunities, obligations, and social benefits. Implicit in this concept is the notion that historical inequalities should be acknowledged and remedied through specific measures” (Barker, 2003, p. 405). As the field of open education continues to expand and identify more benefits of OER beyond cost savings to include practices and pedagogies that attempt to decolonize the curriculum, it’s important to acknowledge the scholarship of several educational theorists who worked to break down barriers of white supremacist educational systems. An emancipatory approach to education, heavily represented in the works of Paulo Freire, bell hooks, Ira Shore, Henry Giroux, and Peter McLaren, focuses on the idea that a just and democratic society relies on the reimagining or negotiating of the curriculum and a negotiated curriculum allows students to assume more significant and shared responsibility in the classroom (Nouri & Sajjadi, 2014)—a concept that open education has more recently built upon with respect to open pedagogy. For example, Black feminist scholar bell hooks challenged educators to confront and transgress traditional boundaries of power and authority so that classrooms can truly become democratic spaces that support all students in the daily work towards freedom (hooks, 2003). This demonstrates how both open pedagogy and Black feminist pedagogy seek to challenge the role of both educators and students as well the utility of assignments (Roberts-Crews, 2022). As Omolade (1993) argued, Black feminist pedagogy advises educators to be more like guides or “consultants” in the education process, rather than controllers. Open pedagogy stems from the same impulse. Seiferle-Valencia (2020) argued how current efforts to create a more socially just open education is not entirely new and reflects a Black feminist pedagogy, among many other critical pedagogies. Black feminist pedagogical rhetoric is not only relevant to open education, but essential to conversations about open values, practices, and pedagogy.

Social justice principles particularly relevant to open education include redistributive, recognitive, and representational justice (Lambert, 2018). Redistributive justice is ensuring access to resources to those who would typically encounter barriers (Rawls, 1971). With redistributive justice, OER removes financial barriers to learning materials. Recognitive justice is “recognition and respect for cultural and gender differences” (Lambert 2018, p. 227; see also Fraser, 1995). When faculty are designing and editing OER, recognitive justice is a critical consideration—is the content diverse, and are students able to see themselves in the materials? Representational justice involves hearing the voices and experiences of groups whose identities have been historically underserved in society (Fraser, 1995; Lambert, 2018). In open education, representational justice may manifest by actively inviting (but not requiring) all students to contribute their own voices to assignments that are then openly licensed and shared beyond the confines of the classroom. Indeed, students reported more opportunities for representational justice with open pedagogy assignments compared to traditional assignments (Clinton-Lisell & Gwozdz, 2023). Many commercial and OER textbooks still have limited coverage and representation of historically marginalized groups (Apple & Christian-Smith, 2017; Brandle, 2020), and both recognitive and representational justice can be leveraged to provide students with more diverse and representative learning materials.

It’s important to note the application of these social justice principles will vary widely depending on the educational and structural context. This is particularly critical to consider as there remains an overrepresentation of the Global North in academic knowledge production (Collyer, 2018). There are structural and systemic inequities unique to the Global South. For example, examining the redistributive principle more closely, an application of this principle would give thought to the differing levels of participation in open education in the Global South due to infrastructure challenges. These specific challenges include, as Hodgkinson-Williams and Trotter (2018) explained, “adequate buildings for instruction, uninterrupted power supply, functional technological equipment, affordable and stable connectivity and access to requisite educational materials” (p. 207). Furthermore, a social justice lens considers some of the origins of the present infrastructure challenges in the Global South, such as colonialism and imperialism, instead of positioning the inequities as innate to these populations. We welcome a response and critique of the SCOPE framework in the context of the Global South.

Learners with disabilities will likely experience open education differently as a function of the systems in place that perpetuate inaccessibility. As part of redistributive justice, open education, both in terms of resources and pedagogies, needs to be accessible across functional diversity needs. Unfortunately, most OER do not meet accessibility guidelines and accessibility does not tend to be a priority when considering adopting OER (Azadbakht et al., 2021; Schultz & Azadbakht, 2021).

Finally, a commitment to social justice in open education research and practice means engaging in critical self-reflection and avoiding anti-essentialism in the interpretation of research findings. A part of this critical self-reflection is asking whether the researcher is conducting research about a community versus conducting research for or with a community. This critical self-reflection also involves examining their role with respect to power and privilege in the research process. Researchers with a social justice orientation understand how broader systems and social situations affect their participants differently depending on their context and background. Not all students, for example, will be affected by open education in the same way. A commitment to social justice also means revisiting previous open education conceptualizations and having potentially uncomfortable conversations. Wiley’s (2014) 5Rs framework provided a helpful foundation to the conceptualization of OER, which argues a resource is generally considered open if the user is able to retain, reuse, redistribute, remix, and/or revise for their specific learning situation. A social justice application of the 5Rs would ask questions such as, “In what situation would altering OER content create or hinder liberation for marginalized communities as a whole?” and “Who has the power and access to edit, retain, redistribute and/or remix OER and how do we create networks and resources for distribution equity?” (Adams & Dannick, 2022; Hollich, 2022).

Cost

Cost refers to the loss assumed to be either due to or avoided by open education (Bliss et al., 2013). In the COUP framework, cost in open education has traditionally focused on the financial cost savings of OER, which are available in digital formats without access fees, as compared to commercial materials, which tend to be very expensive for students (e.g., Farrow et al., 2020). Another conceptualization of cost considers institutional savings through lower course withdrawal rates with courses using OER, specifically open textbooks, compared to commercial textbooks. OER may be linked to less time searching for less expensive alternatives to commercial textbooks, which indicates a lower time cost for students (Katz, 2019). The cost of faculty time and effort to adopt OER has also been examined as a barrier to OER adoption (Belikov & Bodily, 2016; Thomas & Bernhardt, 2018). These aspects of costs traditionally examined are important when considering how to ease the burden of postsecondary education to support student success through OER adoption.

As the field has grown to expand its focus to open pedagogy, K-12, and informal learning, the broader conceptualizations of costs must also be considered in the SCOPE framework. Expectancy-value-cost theory may be informative in this regard (Barron & Hulleman, 2015). According to the expectancy-value-cost theory, one’s motivation for a task is the product of their expectancy to successfully complete the task and the value of that task subtracted by the cost of that task. In this theory, cost includes the aforementioned components of finances and time as well as emotions, opportunities (loss of alternatives to the task), and effort (Flake et al., 2015). Inquiry in this regard could explore the emotions of open education, particularly as students encounter novel pedagogies that incorporate open licensing. For example, students may find it anxiety provoking to consider openly licensing their assignments for others to use.

Cost has been further conceptualized to include the processing demands on cognitive resources (e.g., working memory) known as cognitive load (Feldon et al., 2019). Cognitive load theory is helpful to consider when designing both OER and open pedagogy assignments. OER design features may increase cognitive load and subsequently learning would be less efficient as there are fewer cognitive resources to attend to the content (Paas & Merriënboer, 2020; Sweller, 2020). Importantly, open pedagogy assignments may cause undue burden on cognitive load if the instructions are confusing to students and collaboration with peers (considered a component of open pedagogy; Hegarty, 2015) is cumbersome (Janssen et al., 2010; Janssen & Kirschner, 2020). Clear assessment criteria may lessen cognitive load and improve learning (Krebs et al., 2022). Furthermore, assignment transparency in open pedagogy may be helpful for reducing unnecessary cognitive load. Assignment transparency is communicating the purpose, steps, and resources for completing an assignment in the assignment directions (Winkelmes et al., 2019). Logically, if students’ cognitive resources are not focused on trying to understand how to do the assignment, there would be more cognitive resources available to perform well on the assignment.

Lastly, a less considered, but extremely important cost is the social and political costs of marginalized scholars engaging in open work, including BIPOC and LGBTQ+ scholars. For example, Finley et al. (2018) discussed the particular vulnerability that faculty of color face in writing or researching race-related issues or talking about such topics in the classroom. Furthermore, there is increased political hostility toward race-conscious and LGBTQ-inclusive education (Kaerwer & Prichett, 2023). Open education advocates, scholars, and practitioners need to be mindful of the differing risks when asking marginalized faculty to consider “open” as a practice in their research and teaching as institutional support and protection for these communities to fully participate is often lacking. Furthermore, open education practitioners need to apply a more critical conscious lens in order to consider what kind of support and networks are necessary to truly protect and advance the open work of scholars from vulnerable populations.

Outcomes

Outcomes refer to the presumed effects due to open education with the focus generally on students’ academic outcomes (Bliss et al., 2013). A critical concern when the open education movement began was whether OER would be as effective as commercial resources regarding academic outcomes. Based on three syntheses of research findings, students’ grades are similar whether using OER textbooks or commercial textbooks (Clinton & Khan, 2019; Hilton, 2016, 2020). These findings have been replicated in studies since these syntheses (e.g., Clinton-Lisell, 2023; Fialkowski et al., 2020; Howard & Whitmore, 2020; Nusbaum et al., 2020; Samson et al., 2021; Wynants & Dennis, 2022). Course enrollment intensity, which is the number of credits in which students enroll in a given term, has also been considered as an outcome of OER adoption. The logic behind examining course enrollment intensity is that the reduced financial burden of OER relative to commercial resources enables students to afford more credits per term. Indeed, there is evidence that OER adoption is positively associated with the number of credits enrolled in during a term (Griffiths et al., 2022). However, more examination of how open educational resources relate to outcomes for groups historically underserved in education is needed. Furthermore, more inquiry into K-12 is needed given lack of exploration in these areas (Blomgren & McPherson, 2018; Otto et al., 2021).

One critical area for inquiry in terms of students’ academic outcomes is open pedagogy in which students are creators and editors of resources for others to use (Clinton-Lisell, 2021). Thus far, one study has been published in which student learning outcomes in a course with an open pedagogy assignment were compared to outcomes in a course without an open pedagogy assignment (Tillinghast et al., 2020). This study found comparable learning outcomes in the two course sections. Further work with similar designs and additional student populations and disciplinary content is needed to comprehensively address how open pedagogy relates to student learning outcomes.

Outcomes should be considered for faculty and academic staff who adopt OER, create OER, and/or incorporate open pedagogy into their teaching. Many institutions have had initiatives to provide financial incentives and training for faculty to adopt or create OER (McGowan, 2020). Some faculty have reported experiencing an impact and transformation in their teaching practices as a result of adopting OER (Pitt, 2015). Moreover, faculty felt using OER in their classroom provided them with more autonomy to their pedagogy and claimed that adopting OER helped in accommodating diverse learners and developing a critical consciousness (Pitt, 2015). While faculty tend to positively view incentives to adopt OER, they are hesitant about creation of OER (Todorinova & Wilkinson, 2020). This may be due to the time and effort involved in OER development—particularly for faculty seeking promotion and/or tenure (Blankstein, 2022). Consideration of how to incorporate open education work to support faculty outcomes in terms of retention, promotion, and tenure may be critical to expanding the use and creation of OER and open pedagogy in higher education.

Perceptions

Perceptions refer to impressions and opinions of open education (Bliss et al., 2013). Similar to outcomes, perceptions of quality have been a longstanding concern about open education. The conclusions of reviews of students have typically indicated similar or better perceptions of OER quality compared to commercial textbooks (Clinton, 2019; Hilton, 2016, 2020). When considering academics’ perceptions, concerns about quality may be seen as a barrier to OER adoption (Adil et al., 2022), although some faculty perceive benefits in terms of flexible licensing allowing modifications (Lantrip & Ray, 2021). The majority of studies on open pedagogy have involved student perceptions with findings generally indicating positive experiences with the technique (Clinton-Lisell, 2021). However, continued inquiry into perceptions of open education, particularly grounded in theories (e.g., Werth & Williams, 2021), is valuable considering the evolving nature of both online resources in education (many previous studies compared online OER to hard-copy commercial books) and open pedagogy.

There is evidence from controlled experiments that students perceive faculty who use OER more positively than faculty who use commercial resources. In these studies, students read hypothetical examples of professors and were randomly assigned to read a version of the professor who used an OER or commercial resources (Nusbaum & Cuttler, 2020; Vojtech & Grissett, 2017). These studies provided valuable insights (Curby et al., 2020; Fan et al., 2019), but studies examining actual student evaluations of teaching should be conducted and published. Student evaluations of teaching are an important aspect of many institution’s criteria for faculty retention, promotion, and tenure.

Engagement

We propose the term engagement to refer to fully participating and being actively involved in open education. Tillinghast and colleagues (2020) have proposed adding engagement into the COUP framework (i.e., the COUPE framework) given the importance of engagement for students to succeed. In this way, engagement may be used to expand on “use” in the COUP framework to consider how deeply students were involved in an aspect of open education.

Theoretically, engagement may be categorized as behavioral, affective (emotional), cognitive, or agentic (Fredricks et al., 2004; Reeve & Tseng, 2011). Behavioral engagement is focusing and persevering in a task (Fredricks et al., 2004). In open education, behavioral engagement has been examined through the time spent reading or viewing materials (e.g., Kim et al., 2020) as well as using resources such as study aids (Clinton, 2018; Jhangiani et al., 2018). Learning analytics (examining the data students produce during digital activities) may be useful to examine behavioral engagement (Clow, 2013; Kew & Tasir, 2022; Kim et al., 2020). Affective engagement involves emotions that may facilitate learning, such as excitement (Reeve, 2012). This may be a particularly useful area to examine in open pedagogy to better understand how students emotionally engage with these novel approaches. Cognitive engagement involves using strategies to optimize learning (Reeve & Jang, 2022). There has been some inquiry into OER using self-reports of learning strategies (Clinton, 2018; Jhangiani et al., 2018. It would be helpful to examine what strategies students engage in during open pedagogy assignments as well. Agentic engagement is contributing to the learning activity in an intentional and productive manner (Reeve & Jang, 2022). This category of engagement is clearly relevant to open pedagogy in which students are knowledge creators.

Future Inquiry Needed Across the SCOPE Framework

Across these five proposed components of the SCOPE framework, there are areas of open education that need examination due to critical or emerging importance. A rapidly growing concern is online homework systems for which students need to purchase access codes. Faculty use of these online homework systems has increased dramatically since the pandemic shutdown of March 2020, perhaps due to commercial publishers temporarily having these homework systems available without financial costs during the rapid shift to virtual learning (Nagle & Vitez, 2021). It is currently not well understood how these commercial online homework systems compare to open education, and the SCOPE framework may be useful to guide inquiry into this area. Inclusive access is also an area of emerging importance and concern. Inclusive access involves having the cost of commercial materials directly billed as part of tuition or fees, provided the cost is less than market rates (Thomas et al., 2022). This is intended to lower the financial burden of course materials for college students. However, these materials do not have the flexibility and rights to retain that are afforded through open licensing used in OER and would likely disproportionately harm students enrolled in courses and disciplines that historically have lower cost thresholds for materials. There has been some initial inquiry into inclusive access and student academic outcomes (see Spica, 2023), but direct comparisons with open education (both resources and pedagogy) are needed. Finally, artificial intelligence has rapidly become a critical area of need for research in all of education (Hwang & Chien, 2022; Ouyang et al., 2022), including open education.

There is a need for future quantitative research studies in open education to embrace a QuantCrit perspective. QuantCrit (shortened version of quantitative critical race theory) is a methodological approach in which quantitative data are collected and statistically analyzed through the lens of critical race theory (Gillborn et al., 2017). A key principle in this is that, rather than being objective, quantitative data are collected and analyzed in a socially constructed manner (Gillborn et al., 2017; Sablan, 2019). This is practiced through data analysis that mindfully considers race and other socially constructed groups of peoples historically marginalized by society (Castillo & Gillborn, 2022; Crawford, 2019). Moreover, limitations of research findings for those who have been historically underserved by educational systems must be acknowledged explicitly (Young & Young, 2022). Open education research data and materials should be available whenever possible for others to consider through their own analyses and interpretations (Grahe et al., 2020; Young & Young, 2022). This need for research transparency coincides with the open scholarship and science movements. Imperative in this work is holding responsible systems that created inequities in interpretations of research findings (Gillborn et al., 2017; Young & Young, 2022).

Conclusions

We acknowledge that there is inherent overlap in the SCOPE framework components and do not recommend attempting to silo these issues without considering their relations to each other. This is particularly important when considering how social justice issues may lead to varying experiences in cost, outcomes, perceptions, and engagement depending on identities and the intersectionality of these identities. Developed by Crenshaw (1991), intersectionality refers to how several forms of oppressions come together and compound given an individual’s multiple identities. Rather than compartmentalizing or neatly separating identities, Kouzoukas (2019) stated the framework examines the joining of the identities and how systems of oppressions converge at the intersections (e.g., examining the racialized AND gendered realities of Black women, rather than examining this solely from a racial or gendered perspective). The effect of compounding systems of oppression creates differing experiences for marginalized people doing open education work. For example, the cost of time for the responsibilities of open education may be experienced differently by women of color (Jordan, 2022) who are often disproportionately tasked with service work in educational systems (Domingo et al., 2022; Rideau, 2021). In addition, perceptions of open education intended to enhance recognitive or representational justice, particularly for those from groups historically underserved in higher education, should be considered (see Nusbaum, 2020, for an example).

OER and open pedagogy were the focus of research areas presented in this article. However, open education overlaps both philosophically and in practice with open assessment, open data, open access publications, open scholarship and science, and massive online open courses. There has been insufficient cross-collaboration across these components (Weller, 2020, 2022); however, the SCOPE framework could provide a means of connecting various aspects of open. For example, both OER and open-access publishing involve removing barriers to knowledge.

The SCOPE framework is proposed to both structure and guide future inquiry into open education. Importantly, existing theories of social justice, motivation, cognition, and engagement are incorporated into the SCOPE framework to suggest foundations for future research. This is important given that open education research has been criticized for lacking depth in its theoretical inquiry (Wiley, 2021). We acknowledge that open education is an evolving field and, subsequently, its frameworks, such as the SCOPE, need to be flexible and evolve with it. Our intention with this proposed framework is to deepen the theoretical grounding of open education research while placing social justice at the forefront.

References

Adams, K., & Dannick, S. (2022). Repairing the curriculum: Using OER to fill gaps. In C. J. Ivory & A. Pashia (Eds.), Using open educational resources to promote social justice (pp. 23-40). Association of College and Research Libraries. https://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/booksanddigitalresources/digital/9780838936771_OA.pdf

Adil, H. M., Ali, S., Sultan, M., Ashiq, M., & Rafiq, M. (2022). Open education resources’ benefits and challenges in the academic world: A systematic review. Global Knowledge, Memory and Communication. Advance online publication. https://doi.org/10.1108/GKMC-02-2022-0049

Apple, M., & Christian-Smith, L. (1991). The politics of the textbook (1^st ed.). Routledge. https://doi.org/10.4324/9781315021089

Azadbakht, E., Schultz, T., & Arellano, J. (2021). Not open for all: Accessibility of open textbooks. Insights: The UKSG Journal, 34(1), Article 24. https://doi.org/10.1629/uksg.557

Bali, M., Cronin, C., & Jhangiani, R. S. (2020). Framing open educational practices from a social justice perspective. Journal of Interactive Media in Education, 2020(1), Article 10. https://doi.org/10.5334/jime.565

Barker, R. (2003). The social work dictionary (5th ed.). NASW Press.

Barron, K. E., & Hulleman, C. S. (2015). Expectancy-value-cost model of motivation. In J. S. Eccles & K. Salmela-Aro (Eds.), International encyclopedia of social and behavioral sciences: Motivational psychology (pp. 261-271). Elsevier.

Belikov, O. M., & Bodily, R. (2016). Incentives and barriers to OER adoption: A qualitative analysis of faculty perceptions.  Open Praxis, 8(3), 235-246. https://doi.org/10.5944/openpraxis.8.3.308

Blankstein, M. (2022). Ithaka S & R US Faculty Survey 2021. Copyright, Fair Use, Scholarly Communication, Etc. University of Nebraska, Lincoln. Article 225. https://digitalcommons.unl.edu/scholcom/225

Bliss, T. J., Robinson, T. J., Hilton, J., & Wiley, D. A. (2013). An OER COUP: College teacher and student perceptions of open educational resources. Journal of Interactive Media in Education, 2013(1), Article 4. https://doi.org/10.5334/2013-04

Blomgren, C., & McPherson, I. (2018). Scoping the nascent: An analysis of K-12 OER research 2012-2017. Open Praxis, 10(4), 359-375. https://www.learntechlib.org/p/207176/

Brandle, S. (2020). It’s (not) in the reading: American government textbooks’ limited representation of historically marginalized groups. PS: Political Science & Politics, 53(4), 734-740. https://doi.org/10.1017/S1049096520000797

 Castillo, W., & Gillborn. D. (2022). How to “QuantCrit:” practices and questions for education data researchers and users. (EdWorkingPaper: 22-546). Annenberg Institute at Brown University. https://doi.org/10.26300/v5kh-dd65

Clinton, V. (2018). Savings without sacrifices: A case study of open-source textbook adoption. Open Learning: The Journal of Distance and Open Learning, 33(3), 177-189. https://doi.org/10.1080/02680513.2018.1486184

Clinton, V. (2019). Cost, outcomes, use, and perceptions of open educational resources in psychology: A narrative review of the literature. Psychology Learning and Teaching, 18(1), 4-20. https://doi.org/10.1177/1475725718799511

Clinton, V., & Khan, S. (2019). Efficacy of open textbook adoption on learning performance and course withdrawal rates: A meta-analysis. AERA Open, 5(3), 1-20. https://doi.org/10.1177/2332858419872212

Clinton-Lisell, V. (2021). Open pedagogy: A systematic review of empirical findings. Journal of Learning for Development, 8(2), 255-268. https://jl4d.org/index.php/ejl4d/article/view/511

Clinton-Lisell, V. (2023). How does OER efficacy vary based on student age and course modality? American Journal of Distance Education, 37(3), 217-233. https://doi.org/10.1080/08923647.2022.2077061

Clinton-Lisell, V., & Gwozdz, L. (2023). Understanding student experiences of renewable and traditional assignments. College Teaching, 71(2), 125-134. https://doi.org/10.1080/87567555.2023.2179591

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695. https://doi.org/10.1080/13562517.2013.827653

Collyer, F. M. (2018). Global patterns in the publishing of academic knowledge: Global North, global South. Current Sociology, 66(1), 56-73. https://doi.org/10.1177/0011392116680020

Crawford, C. E. (2019) The one-in-ten: Quantitative critical race theory and the education of the ‘new (white) oppressed’. Journal of Education Policy, 34(3), 423-444, https://doi.org/10.1080/02680939.2018.1531314

Crenshaw, K. (1991). Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43(6), 1241-1299. https://www.jstor.org/stable/1229039

Croft, B., & Brown, M. (2020). Inclusive open education: Presumptions, principles, and practices. Distance Education, 41(2), 156-170. https://doi.org/10.1080/01587919.2020.1757410

Curby, T., McKnight, P., Alexander, L., & Erchov, S. (2020). Sources of variance in end-of-course student evaluations. Assessment & Evaluation in Higher Education, 45(1), 44-53. https://doi.org/10.1080/02602938.2019.1607249

Deimann, M., & Farrow, R. (2013). Rethinking OER and their use: Open education as bildung. The International Review of Research in Open and Distributed Learning, 14(3), 344-360. https://doi.org/10.19173/irrodl.v14i3.1370

DeRosa, R., & Robinson, S. (2017). From OER to open pedagogy: Harnessing the power of open. In R. Biswas-Diener & R. S. Jhangiani (Eds.), Open: The philosophy and practices that are revolutionizing education and science. Ubiquity Press. https://www.ubiquitypress.com/site/books/10.5334/bbc/read/#epubcfi(/6/2[id001]!/4/1:0)

Domingo, C. R., Gerber, N. C., Harris, D., Mamo, L., Pasion, S. G., Rebanal, R. D., & Rosser, S. V. (2022). More service or more advancement: Institutional barriers to academic success for women and women of color faculty at a large public comprehensive minority-serving state university. Journal of Diversity in Higher Education, 15(3), 365-379. https://doi.org/10.1037/dhe0000292

Fan, Y., Shepherd, L. J., Slavich, E., Waters, D., Stone, M., Abel, R., & Johnston, E. L. (2019). Gender and cultural bias in student evaluations: Why representation matters. PLOS ONE, 14(2), Article e0209749. https://doi.org/10.1371/journal.pone.0209749

Farrow, R., Pitt, R., & Weller, M. (2020). Open textbooks as an innovation route for open science pedagogy. Education for Information, 36(3), 227-245. https://doi.org/10.3233/EFI-190260

Feldon, D. F., Callan, G., Juth, S., & Jeong, S. (2019). Cognitive load as motivational cost.  Educational Psychology Review, 31, 319-337. https://doi.org/10.1007/s10648-019-09464-6

Fialkowski, M. K., Calabrese, A., Tilinghast, B., Titchenal, C. A., Meinke, W., Banna, J. C., & Draper, J. (2020). Open educational resource textbook impact on students in an introductory nutrition course. Journal of Nutrition Education and Behavior, 52(4), 359-368. https://doi.org/10.1016/j.jneb.2019.08.006

Finley, S. C., Gray, B. M., & Martin, L. L. (2018). “Affirming our values”: African American scholars, white virtual mobs, and the complicity of white university administrators. Journal of Academic Freedom, 9, 1-20. https://www.aaup.org/JAF9/%E2%80%9Caffirming-our-values%E2%80%9D-african-american-scholars-white-virtual-mobs-and-complicity-white#.ZG1dWuzMJ6o

Flake, J. K., Barron, K. E., Hulleman, C., McCoach, B. D., & Welsh, M. E. (2015). Measuring cost: The forgotten component of expectancy-value theory. Contemporary Educational Psychology, 41, 232-244. https://doi.org/10.1016/j.cedpsych.2015.03.002

Fraser, F. (1995). From redistribution to recognition? Dilemmas of justice in a “post-socialist” age. New Left Review, 1(212). https://newleftreview.org/I/212/nancy-fraser-from-redistribution-to-recognition-dilemmas-of-justice-in-a-post-socialist-age

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. https://doi.org/10.3102/00346543074001059

Grahe, J. E., Cuccolo, K., Leighton, D. C., & Cramblet Alvarez, L. D. (2020). Open science promotes diverse, just, and sustainable research and educational outcomes. Psychology Learning & Teaching, 19(1), 5-20. https://doi.org/10.1177/1475725719869164

Gillborn, D., Warmington, P., & Demack, S. (2018). QuantCrit: education, policy, ‘big data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158-179. https://doi.org/10.1080/13613324.2017.1377417

Griffiths, R., Mislevy, J., & Wang, S. (2022). Encouraging impacts of an open education resource degree initiative on college students’ progress to degree. Higher Education, 84(5), 1089-1106. https://doi.org/10.1007/s10734-022-00817-9

Hegarty, B. (2015). Attributes of open pedagogy: A model for using open educational resources.  Educational Technology, 55(4), 3-13. https://www.jstor.org/stable/44430383

Hilton, J., III. (2016). Open educational resources and college textbook choices: A review of research on efficacy and perceptions. Educational Technology Research and Development, 64, 573-590. https://link.springer.com/article/10.1007/s11423-016-9434-9

Hilton, J., III. (2020). Open educational resources, student efficacy, and user perceptions: A synthesis of research published between 2015 and 2018. Educational Technology Research and Development, 68(3), 853-876. https://link.springer.com/article/10.1007/s11423-019-09700-4

Hodgkinson-Williams, C. A., & Trotter, H. (2018). A social justice framework for understanding open educational resources and practices in the global south. Journal of Learning for Development, 5(3). https://doi.org/10.56059/jl4d.v5i3.312

Hollich, S. (2022). The unrealized promise of OER: An exploration of copyright, the open movement, and social justice. In C. J. Ivory & A. Pashia (Eds.), Using open educational resources to promote social justice (pp. 3-21). Association of College and Research Libraries. https://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/booksanddigitalresources/digital/9780838936771_OA.pdf

hooks, bell. (2003). Teaching community: A pedagogy of hope. Routledge. https://doi.org/10.4324/9780203957769

Howard, V. J., & Whitmore, C. B. (2020). Evaluating student perceptions of open and commercial psychology textbooks. Frontiers in Education, 5. https://doi.org/10.3389/feduc.2020.00139

Hwang, G.-J., & Chien, S.-Y. (2022). Definition, roles, and potential research issues of the metaverse in education: An artificial intelligence perspective. Computers and Education: Artificial Intelligence, 3, Article 100082. https://doi.org/10.1016/j.caeai.2022.100082

Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010). Making the black box of collaborative learning transparent: Combining process-oriented and cognitive load approaches. Educational Psychology Review, 22(2), 139-154. https://doi.org/10.1007/s10648-010-9131-x

Janssen, J., & Kirschner, P. A. (2020). Applying collaborative cognitive load theory to computer-supported collaborative learning: Towards a research agenda. Educational Technology Research and Development, 68(2), 783-805. https://doi.org/10.1007/s11423-019-09729-5

Jhangiani, R. S., Dastur, F. N., Le Grand, R., & Penner, K. (2018). As good or better than commercial textbooks: Students’ perceptions and outcomes from using open digital and open print textbooks. Canadian Journal for the Scholarship of Teaching and Learning, 9(1). https://doi.org/10.5206/cjsotl-rcacea.2018.1.5

Jordan, J. (2023). Compounded labor: Developing OER as a marginalized creator. In the Library with the Lead Pipe. https://www.inthelibrarywiththeleadpipe.org/2023/compounded-labor-developing-oer-as-a-marginalized-creator/

Kaerwer, K., & Pritchett, M. (2023). Critical race theory in education: How banning its tenets undermines our best hope for equity in education. Behavior and Social Issues, 32, 300-313. https://doi.org/10.1007/s42822-023-00130-9

Katz, S. (2019). Student textbook purchasing: The hidden cost of time. Journal of Perspectives in Applied Academic Practice, 7(1), 12-18. https://jpaap.ac.uk/JPAAP/article/view/349

Kew, S. N., & Tasir, Z. (2022). Learning analytics in online learning environment: A systematic review on the focuses and the types of student-related analytics data. Technology, Knowledge and Learning, 27(2), 405-427. https://doi.org/10.1007/s10758-021-09541-2

Kim, D., Lee, Y., Leite, W. L., & Huggins-Manley, A. C. (2020). Exploring student and teacher usage patterns associated with student attrition in an open educational resource-supported online learning platform. Computers & Education, 156, Article 103961. https://doi.org/10.1016/j.compedu.2020.103961

Kouzoukas, G. (2019). Engaging first generation students. ​​In S. J. Quaye, S. R. Harper, & S. L. Pendakur (Eds.). Student engagement in higher education: Theoretical perspectives and practical approaches for diverse populations (pp. 287-306). Routledge.

Krebs, R., Rothstein, B., & Roelle, J. (2022). Rubrics enhance accuracy and reduce cognitive load in self-assessment. Metacognition and Learning, 17(2), 627-650. https://doi.org/10.1007/s11409-022-09302-1

Ladson-Billings, G. (2014). Culturally relevant pedagogy 2.0: A.k.a. the remix. Harvard Educational Review, 84(1), 74-84. https://doi.org/10.17763/haer.84.1.p2rj131485484751

Lambert, S. R. (2018). Changing our (dis) course: A distinctive social justice aligned definition of open education. Journal of Learning for Development, 5(3). https://doi.org/10.56059/jl4d.v5i3.290

Lantrip, J., & Ray, J. (2021). Faculty perceptions and usage of OER at Oregon community colleges. Community College Journal of Research and Practice, 45(12), 896-910. https://doi.org/10.1080/10668926.2020.1838967

McGowan, V. (2020). Institution initiatives and support related to faculty development of open educational resources and alternative textbooks. Open Learning: The Journal of Open, Distance and e-Learning, 35(1), 24-45. https://doi.org/10.1080/02680513.2018.1562328

Nagle, C., & Vitez, K. (2021). Fixing the broken textbook market. US PIRG. https://uspirg.org/sites/pirg/files/reports/Fixing%20the%20Broken%20Textbook%20Market%2C%203e%20February%202021.pdf

Nouri, A., & Sajjadi, S. M. (2014). Emancipatory pedagogy in practice: Aims, principles and curriculum orientation. The International Journal of Critical Pedagogy, 5(2), 76-87. https://libjournal.uncg.edu/ijcp/article/view/228

Nusbaum, A. T. (2020). Who gets to wield academic Mjolnir?: On worthiness, knowledge curation, and using the power of the people to diversify OER. Journal of Interactive Media in Education, 2020(1), Article 4. https://doi.org/10.5334/jime.559

Nusbaum, A. T., & Cuttler, C. (2020). Hidden impacts of OER: Effects of OER on instructor ratings and course selection. Frontiers in Education, 5. https://www.frontiersin.org/articles/10.3389/feduc.2020.00072

Nusbaum, A. T., Cuttler, C., & Swindell, S. (2020). Open educational resources as a tool for educational equity: Evidence from an introductory psychology class. Frontiers in Education, 4. https://www.frontiersin.org/articles/10.3389/feduc.2019.00152

Open Education Group. (2023). The COUP framework. https://openedgroup.org/coup

Omolade, B. (1993). A black feminist pedagogy.  Women’s Studies Quarterly, 21(3/4), 31-38. https://www.jstor.org/stable/40022003

Otto, D., Schroeder, N., Diekmann, D., & Sander, P. (2021). Trends and gaps in empirical research on open educational resources (OER): A systematic mapping of the literature from 2015 to 2019. Contemporary Educational Technology, 13(4), Article ep325. https://doi.org/10.30935/cedtech/11145

Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6), 7893-7925. https://doi.org/10.1007/s10639-022-10925-9

Paas, F., & van Merriënboer, J. J. (2020). Cognitive-load theory: Methods to manage working memory load in the learning of complex tasks. Current Directions in Psychological Science, 29(4), 394-398. https://doi.org/10.1177/0963721420922183

Pitt, R. (2015). Mainstreaming open textbooks: Educator perspectives on the impact of OpenStax College open textbooks. The International Review of Research in Open and Distributed Learning, 16(4), 133-155. https://doi.org/10.19173/irrodl.v16i4.2381

Rawls, J. (1971). A theory of justice.  Cambridge.

Reeve, J., & Tseng, C.-M. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36(4), 257-267. https://doi.org/10.1016/j.cedpsych.2011.05.002

Reeve, J. (2012). A self-determination theory perspective on student engagement. In S. Christenson, A.L. Reschley, & C. Wylie (Eds) Handbook of research on student engagement (pp. 149-172). Springer.

Reeve, J., & Jang, H. (2022). Agentic engagement. In S. Christenson, A.L. Reschley, & C. Wylie (Eds) Handbook of research on student engagement (pp. 95-107). Springer.

Rideau, R. (2021). “We’re just not acknowledged”: An examination of the identity taxation of full-time non-tenure-track Women of Color faculty members. Journal of Diversity in Higher Education, 14(2), 161-173. https://doi.org/10.1037/dhe0000139

Roberts-Crews, J. (2022, September 22). Assumptions about labor in open education [Keynote address]. Open Texas Conference, Fort Worth, TX, USA.

Sablan, J. R. (2019). Can you really measure that? Combining critical race theory and quantitative methods. American Educational Research Journal, 56(1), 178-203. https://psycnet.apa.org/doi/10.3102/0002831218798325

Samson, R.L., Clinton-Lisell, V., & Fischer, L. (2021). Let students choose: Examining the impact of open educational resources on performance in general chemistry. Journal of Chemical Education, 98(3), 745-755. https://doi.org/10.1021.acs.jcemed.0c00595

Santos, A. (2018). Instructional strategies for game-based learning. In Gamification in education: Breakthroughs in research and practice (pp. 472-481). IGI Global.

Schultz, T. A., & Azadbakht, E. (2021). Open but not for all: A survey of open educational resource librarians on accessibility. College & Research Libraries, 82(5), 755-769. https://doi.org/10.5860/crl.82.5.755

Seiferle-Valencia, M. (2020). It’s not (just) about the cost: Academic libraries and intentionally engaged OER for social justice. Library Trends, 69(2), 469-487. https://doi.org/10.1353/lib.2020.0042

Spica, E. (2023). Inclusive access : A multi-institutional study of academic outcomes from a statewide community college automatic billing e-textbook pilot. Community College Journal of Research and Practice, 47(3), 197-216. https://doi.org/10.1080/10668926.2021.1990161

Sweller, J. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68(1), 1-16. https://doi.org/10.1007/s11423-019-09701-3

Thomas, W. J., & Bernhardt, B. R. (2018). Helping keep the costs of textbooks for students down: Two approaches. Technical Services Quarterly, 35(3), 257-268. https://doi.org/10.1080/07317131.2018.1456844

Thomas, A., Dunbar, E., Wharton, R. M., & Greco, A. N. (2022). The impact of the inclusive access antitrust lawsuit on US college bookstores, book publishers, and educational publishers. Publishing Research Quarterly, 38(3), 475-489. https://doi.org/10.1007/s12109-022-09905-1

Tietjen, P., & Asino, T. I. (2021). What is open pedagogy? Identifying commonalities. The International Review of Research in Open and Distributed Learning, 22(2), 185-204. https://doi.org/10.19173/irrodl.v22i2.5161

Tillinghast, B., Fialkowski, M. K., & Draper, J. (2020). Exploring aspects of open educational resources through OER-enabled pedagogy. Frontiers in Education, 5(76). https://www.frontiersin.org/articles/10.3389/feduc.2020.00076

Tlili, A., Altinay, F., Huang, R., Altinay, Z., Olivier, J., Mishra, S., Jemni, M., & Burgos, D. (2022). Are we there yet? A systematic literature review of open educational resources in Africa: A combined content and bibliometric analysis. PLOS ONE, 17(1), Article e0262615. https://doi.org/10.1371/journal.pone.0262615

Todorinova, L., & Wilkinson, Z. T. (2020). Incentivizing faculty for open educational resources (OER) adoption and open textbook authoring. The Journal of Academic Librarianship, 46(6), Article 102220. https://doi.org/10.1016/j.acalib.2020.102220

Van Allen, J., & Katz, S. (2019). Developing open practices in teacher education: An example of integrating OER and developing renewable assignments. Open Praxis, 11(3), 311-319. https://search.informit.org/doi/abs/10.3316/informit.767590955332474

Vojtech, G., & Grissett, J. (2017). Student perceptions of college faculty who use OER. The International Review of Research in Open and Distributed Learning, 18(4), 155-171. https://doi.org/10.19173/irrodl.v18i4.3032

Weller, M. (2020). Open and free access to education for all. In D. Burgos (Ed.), Radical solutions and open science: An open approach to boost higher education (pp. 1-15). Springer. https://doi.org/10.1007/978-981-15-4276-3_1

Weller, M. (2022). The rise and development of digital education. In O. Zawacki-Richter & I. Jung (Eds.), Handbook of Open, Distance and Digital Education (pp. 1-17). Springer.

Werth, E., & Williams, K. (2021). What motivates students about open pedagogy? Motivational regulation through the lens of self-determination theory. The International Review of Research in Open and Distributed Learning, 22(3), 34-54. https://doi.org/10.19173/irrodl.v22i3.5373

Wiley, D. A. (2021). Open educational resources: Undertheorized research and untapped potential. Educational Technology Research and Development, 69(1), 411-414. https://doi.org/10.1007/s11423-020-09907-w

Wiley, D. (2014, March 5). The access compromise and the 5th R. Improving Learning. https://opencontent.org/blog/archives/3221

Wiley, D., & Hilton Iii, J. L. (2018). Defining OER-enabled pedagogy.  The International Review of Research in Open and Distributed Learning, 19(4). https://doi.org/10.19173/irrodl.v19i4.3601

Winkelmes, M. A., Boye, A., & Tapp, S. (2019). Transparent design in higher education. Stylus Publishing.

Wynants, S., & Dennis, J. (2022). Redesigning a research methods course with personalized, interactive OER: A case study of student perceptions and performance. Journal of the Scholarship of Teaching and Learning, 22(1), 138-153. https://doi.org/10.14434/josotl.v22i1.31706

Young, J., & Young, J. (2022). Decoding the data dichotomy: applying QuantCrit to understand racially conscience intersectional meta-analytic research. International Journal of research & method in education, 45(4), 381-396. https://doi.org/10.1080/1743727X.2022.2093847

SCOPE of Open Education: A New Framework for Research by Virginia Elizabeth Clinton-Lisell, Jasmine Roberts-Crews, and Lindsey Gwozdz is licensed under a Creative Commons Attribution 4.0 International License.