Figure 1. Selection process of identified studies following the PRISMA statement.
Volume 20, Number 2
Christian M. Stracke
Open University of the Netherlands, Heerlen; Korea National Open University, Seoul; East China Normal University, Shanghai
This article discusses the need to innovate education due to global changes to keep its status as a human right and public good and introduces Open Education as a theory to fulfil these requirements. A systematic literature review confirms the hypothesis that a holistic quality framework for Open Education does not exist. For its development, a brief history and definition of Open Education are provided first. It is argued that Open Education improves learning quality through the facilitation of innovative learning designs and processes. Therefore, sources of learning quality and dimensions of quality development are discussed. To support the improvement of the learning quality and design of Open Education, the Reference Process Model of ISO/IEC 40180 (former ISO/IEC 19796-1) is introduced and modified for Open Education. Adapting the three quality dimensions and applying the macro, meso, and micro levels, the OpenEd Quality Framework is developed. This framework combines and integrates the different quality perspectives in a holistic approach that is mapping them to the learning design, processes, and results. Finally, this article illustrates potential adaptations and benefits of the OpenEd Quality Framework. The OpenEd Quality Framework can be used in combination with other tools to address the complexity of and to increase the quality and impact of Open Education. To summarize, the OpenEd Quality Framework serves to facilitate and foster future improvement of the learning design and quality of Open Education.
Keywords: open education, open learning, OpenEd quality framework, learning quality, learning innovations, learning design, history, policies, ISO/IEC 40180
In these challenging and fast-moving times, it is most important to underline: Education is a human right and public good. Education must be continuously provided, innovated, and improved to keep this status in the face of major global challenges (United Nations, 2015). This article describes the needs and potential approaches in theory and practice to meet societal requirements by providing an overview of Open Education as well as introducing the OpenEd Quality Framework for innovating and improving future learning quality and design.
The two main change drivers of our so-called "digital age" are the globalisation and Internet, which modify all parts of our lives, working conditions, and societies as already analysed in detail (Stracke, 2018). That is happening even though the majority of people worldwide (currently 4.2 billion of the global population of 7.4 billion in the year 2016) are still offline and Internet access is very unequal in the Northern and Southern hemispheres (World Bank, 2016). On the other hand, the Internet is more evenly spread than income over the world and the number of Internet users is increasing rapidly (it tripled during the last 10 years from 1.0 to 3.2 billion) and in addition 5.2 billion people have mobile phones and almost everybody (7.0 out of 7.4 billion) is within the mobile coverage (World Bank, 2016).
Globalisation and the Internet have previously challenged and continue to challenge all societies especially in regards to learning and education (education and training in schools, universities, at work, and online; Gaskell & Mills, 2014). On the other hand, globalization and the Internet also offer new opportunities for innovative (formal, non-formal, and informal) learning (Organisation for Economic Co-operation and Development [OECD], 2016). Due to these mutating conditions in societies, there is a current and increasing need to change education, reflecting this ongoing societal revolution in relation to required competences in the future (Weinert, 2001; Westera, 2001; Stracke, 2015). Nevertheless, investments in education and training are more or less stable and not increasing in many countries despite the general recognition of their importance (OECD, 2016).
To our knowledge, there is currently no quality framework for Open Education that is holistic, or which implements a philosophy of quality development with a continuous improvement cycle. Our research question is therefore:
RQ1: How can we derive and develop a holistic quality framework for Open Education from current state-of-the-art literature and research results?
Our key motivation and assumptions are that (1) such a quality framework for Open Education may support the introduction of Open Education and increase the use of Open Education and (2) a quality framework will facilitate the needed change and improvement of learning and education.
Based on our long-term research focus on the quality of Open Education, our hypothesis is:
H1: There is currently no holistic quality framework for Open Education that (1) follows the total quality management philosophy with continuous improvement cycles and (2) addresses all educational levels (micro, meso and macro).
First, we have conducted a systematic literature review to prove (or refute) our initial statement and hypothesis that no holistic quality framework for Open Education exists. A systematic literature review summarizes the state-of-the-art about a selected topic. It uses pre-defined methods and results are documented in a systematic review protocol. This type of review is based on the rigorous analysis of the evidences that arise from a careful evaluation of the available literature according to pre-defined and shared criteria. Thus, it requires a well-structured process that defines the key decisions of the review, i.e., how studies will be identified, analysed, selected, and evaluated (Booth, Sutton, & Papaioannou, 2016). In our systematic literature review, we are following the PRISMA Statement (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009). It consists of a 27-item checklist that regards methods, results, discussion, and funding and a four-phase flow diagram that concerns records identifications, records screening, articles eligibility, and studies included. We followed the PRISMA statement and its four phases as described below and presented in Figure 1. Our systematic literature review has taken into account literature published up until July of 2018.
To achieve broadest results after screening and for the full text analysis, we have defined only two simple selection criteria: (1) literature incorporated must be in English (as new international lingua franca) and (2) literature incorporated must also be available in full text (to keep as many results as possible eligible for analysis independent of their scientific level and document type).
We searched the keywords "Open Education" AND "Quality Framework" on four global databases. As results, we received 173 records from Google Scholar (www.scholar.google.com), 0 records from Web of Science (Clarivate Analytics through University of Maastricht and Open University of the Netherlands [OUNL] account), 0 records from Science Direct (Elsevier through University of Maastricht and OUNL account), and 8 records from Summons (University of Maastricht through OUNL account), leading to a total amount of 181 records. Three records were identified as duplicates and removed, leading to 178 records for the screening. In the screening, 18 records were removed as they were not fulfilling the selection criteria: 11 records were not in English and 7 records were not full text studies. The full texts of the remaining 160 studies were assessed and none of these studies actually presented or referenced a holistic quality framework for Open Education. Among these studies were two publications with the terms quality models and frameworks in the title: Ossiannilsson, Williams, Camilleri, and Brown (2015) and Jansen, Rosewell, and Kear (2017). Ossiannilsson et al. (2015) compare different quality models which focus on online education and summarize that all analysed quality models suffer certain deficiencies and that a holistic quality framework for Open Education is not existing. Jansen, Rosewell, and Kear (2017) explore quality frameworks for Massive Open Online Courses (MOOCs) only, classifying these frameworks as a specific type and mode of Open Education and promoting their own OpenupEd Quality Label, which focuses on self-assessment and benchmarks for MOOCs. Therefore, both studies cannot be considered to provide a holistic quality framework for Open Education.
Thus, no studies could be included in the planned qualitative and quantitative syntheses. Figure 1 provides an overview of the selection process of the studies.
Figure 1. Selection process of identified studies following the PRISMA statement.
As a direct consequence of our systematic literature review, we claim that there is no holistic quality framework for Open Education to our best knowledge. In addition, we have also considered and analysed further publications that we know from our long-term interest in learning quality in Open Education (that were not identified by the keywords of our systematic literature review). Of these publications, there are empirical studies analysing factors for quality education but all of them are analysing specific effects, such as factors in online Higher Education (e.g., Barbera & Linder-VanBerschot, 2011), cross-cultural dimensions of online learning (e.g., Gómez-Rey, Barbera, & Fernández-Navarro, 2016) and different quality perspectives and expectations (e.g., Stracke et al., 2018), all of which do not provide a holistic quality framework for Open Education. Finally, our result is in line with the latest literature review by Esfijani (2018), which articulates a lack of a holistic quality framework in online education, as well as the absence of an integrated view on the quality of online education.
In the following section, we will define Open Education and discuss the dimensions relevant to the research at hand. Based on our discussion results and the identification of key needs (such as covering all educational levels) and documents (such as the first international quality standard ISO/IEC 40180), we will develop and present a proposal for a new and holistic OpenEd Quality Framework to improve the learning quality and design of Open Education.
There is broad consensus that learning and education have to change to reflect and answer the challenges of rapidly growing globalisation and changes in societies (both leading to uncertainty regarding needs for changing personal development as well as learning and education, even in short terms) as discussed above. Learning and education should be changed by opening up to the use of Open Learning and Open Education (Stracke, 2013a, 2017a). But what do Open Learning and Open Education mean?
Open Learning and Open Education have a long history that should not be forgotten and ignored (Nyberg, 1975; Peters, 2008; Peter & Deimann, 2013; Stracke, 2018). Both terms are used to refer to pedagogical theories and approaches which follow a philosophy and thinking that can be characterized by three main beliefs (see for more details Stracke, 2014a, 2015):
In the following, we will use only the term "Open Education.” In general, the difference between "Open Education" and "Open Learning" is that Open Education (which can be classified as both formal and non-formal learning) involves an educator, whereas in Open Learning (often classified as non-formal or informal learning), the learners learn independently, without support of educators.
We have to define Open Education first: While the concept of Open Education is broad and diverse (Gaskell & Mills, 2014), we believe that our following definition targets the core meaning of the term Open Education:
Open Education is designing, realizing, and evaluating learning opportunities with visionary, operational, and legal openness to improve learning quality for the learners.
Open Education is as manifold as the term openness (Wiley, 2009), as it can be related to quite diverse approaches and understandings. Generally, Open Education refers to both, learning innovations and learning quality. It aims to change educational environments and offer a selection of diverse methodologies, tasks, and resources for learners. As expressed in our definition as well as discussed above, improving learning quality has to be the final objective supported by learning innovations. Therefore, Open Education must be adapted for given situations, and in particular, for the specific learners and their needs (Kirschner & van Merriënboer, 2013). The open aspect of Open Education refers not only to the dimensions of “legal” openness (accessibility and availability), but also refers to the dimensions of operational openness (such as open design frameworks) and of visionary openness (such as open policies) (for more details see Stracke, 2017b, 2018).
In the following, we will analyse dimensions of quality development and openness in general before applying these general quality aspects to Open Education, leading to a general framework for Open Education called "OpenEd Quality Framework."
In international discussions about the need to change education and about future learning (from theory, research, and politics but also from press, individuals, and social communities), the main focus is currently on technological innovations and the new opportunities they provide. We suggest that the discussions regarding the topic at hand can be categorized under two separate strands: the learning innovation strand and the learning history strand.
Some theories and experts are claiming brand new and extraordinary chances, sometimes promising new learning eras and paradigms (Stracke, 2014a) even though they are only a fusion of former theories: e.g., the concepts of connectivism by Siemens (2005) or of Social Learning by Hart (2011). Even the arrival of fundamental new ways of learning is promised under the label of "learning 2.0/3.0" in analogy to the terms "Web 2.0/3.0" (Downes 2005; Karrer, 2007; Redecker, 2009). Finally, new concepts and descriptions of our world as a 'flat world' are leading to predictions that the key competence "to learn how to learn" will become the most important asset for all workers due to worldwide changes and faster innovation cycles in business sectors and at work (Friedman, 2006). Those concepts such as 'the flat world' by Friedman (2006) are claiming to constitute a new movement and progress in education as well as in the whole world. However, it is our belief that such claims for a new movement and new competences are just marketing speech and cannot be accepted, as it has been evident in pedagogy for several hundreds of years that "to learn how to learn" is important for learning processes and progress as well as for the development of personality and competences (Dewey, 1966; Piaget, 1953; Rousseau, 1968; Vygotsky, 1988).
The discussion articulated above may best be categorized as part of the learning innovation strand. From this special perspective, it seems that the unique focus on learning innovations is the only pathway and road map for a better education and training in the future as the change and innovation of learning are needed. The underlying, and often hidden argument is that through innovations we would earn many new chances to learn, and without them we are not matching the changing times of globalisation and worldwide Internet as well as the "new digital generation," and the so labelled "digital natives" (Prensky, 2001) even though that they are not existing in reality as it could be proven by several studies (see e. g., Schulmeister, 2008).
On the other hand, there has been a long-term discussion with a longstanding tradition, since the beginning of our culture, about learning quality and what constitutes learning covering a broad range of topics including: the quality of (a) learning objectives and design, (b) learning materials and input, (c) learning processes, and (d) learning outcomes and the achieved knowledge, skills and built competences (Inglis, 2005).
We call this debate the (learning) history strand, as in the past, many theories were developed dealing directly or implicitly with the question how to ensure or to improve learning quality (see for an overview Stracke, 2006). In the educational history, some topics like quality management for education and training are less than 100 years old but general concepts aiming at learning quality have existed for ages.
Surprisingly, both discussion strands articulated above were not interconnected and did not reflect each other (Stracke, 2014a). It seems that those who support learning innovations do not want to refer to theories of the past, and that the authors of learning history do not want to recognise global changes. This led us to an important question that requires urgent attention and an answer in our changing times: What is the relation between learning innovations and history?
Our answer is based on three strong opinions regarding the current societal situation and learning needs that were explained and discussed in detail by Stracke (2013a):
Learning quality is more than learning innovations, and the focus on learning quality is most important for the success of learning processes. Consequently, quality development is the crucial task for learning, education, and training. Learning opportunities have to meet the needs of the learners and to provide the appropriate quality to fulfil their requirements. That can sometimes mean a simple learning course with teacher-centred education, and sometimes a complex sophisticated learning environment with learner-oriented group work facilitated by an educator as moderator, tutor, or enabler and enriched with new learning technologies and innovations including social media and online communities. This means that learning quality cannot be pre-defined but must be adapted to the given situation and learners. In this sense, learning history and learning innovations are two different approaches and points of view that are interdependent, and cannot be reflected solely. They must be analysed in conjunction for achieving the best and appropriate learning opportunity and success. Next to them, standards are building the third source for planning and designing the best learning opportunity and quality (Stracke, 2013) as shown in Figure 2. Standards can provide frameworks and instruments for adapting and reusing plans, designs, patterns, resources, and tools to benefit from return of investment by several repetitive applications and to achieve continuous quality improvement. Moreover, the development of standards and their application and adaptation help all involved stakeholders to discuss and reach consensus about learning quality and the way to achieve it.
Figure 2. The three sources for learning quality.
This overall objective for the continuous improvement of learning quality is called quality development. Quality development has to combine the relevant and appropriate approaches, concepts, and elements from all three sources that learning quality is based on: History (by learning theories and traditions), Innovation (by new learning options), and Standards (by consensus building on learning). In the following, we will discuss first the dimensions for quality development in general that will be transferred to Open Education afterwards.
The debate on learning quality is very old, but discussions and theories on quality development in learning and education began only few years ago. Quality development has to be distinguished from failure reduction, quality assurance, quality management, and total quality management. Failure reduction and quality assurance are focusing products: failure reduction intends to increase the number of usable products whereas quality assurance addresses the improvement of the quality of products. Quality management goes beyond quality assurance and focuses also the production processes to achieve higher quality. Finally, Total Quality Management (TQM) is defined twofold: (1) on the one hand it is considered as quality management plus a continuous improvement cycle and (2) on the other hand it is a broad and holistic concept and philosophy that includes and integrates all aspects and dimensions for improving the quality of products, and thus, going beyond their production processes. Quality development is used here as synonym for the latter holistic definition of TQM.
The concept and philosophy of holistic quality development with continuous improvement cycles were introduced in Japan first and could gain recognition, acceptance, and implementations worldwide. A long-term debate has focussed on quality development in general regarding the different quality issues, aspects, and approaches (Deming, 1982; Juran, 1951, 1992; Stracke, 2006). As articulated by Stracke (2013a) "quality development covers every kind of strategy, analysis, design, realisation, evaluation, and continuous improvement of the quality within given systems" (p. 21). Thus, quality development can be described formally by the selected focus. Quality is not a fixed characteristic belonging to subjects or systems but depends on the point of view and focus. The differentiation of the focus into the three quality dimensions Potential, Process and Result was introduced by Donabedian (1980) in the healthcare sector and has become widely accepted. These three quality dimensions focus on the following questions (see Donabedian, 1980; for the long-term debate on the quality issues, aspects, and approaches see Deming, 1982, 1986; Juran, 1951, 1992; Stracke, 2006):
Quality development requires a long process to be established and integrated throughout a whole organization and even the whole society in the case of public goods like education (Freire, 1970; Volungeviciene, Tereseviciene, & Tait, 2014). Once started, it has to become a continuous improvement cycle to be successful (Crosby, 1980; Deming, 1986). Quality cannot be described and fixed by a simple definition because in itself the concept of quality is too abstract. Potential definitions of quality like "fulfilment of customers' requirements" or "excellent status lacking defects" have to take the perspective from the individuals (such as the learners in education). Therefore, quality has to be defined and specified according to the given context and situation considering the perspectives of stakeholders involved (Donabedian, 1980). It is important to identify the relevant aspects and to specify the suitable criteria. It is necessary to find a consensus amongst the different views and perspectives to gain a common understanding of quality for the given context and situation due to different and sometimes contradictory needs and definitions of quality by all stakeholders (for detailed explanations on context determinations see Crosby, 1980; Deming, 1986; Donabedian, 1980).
The next question is now: How can quality development be addressed and improved in learning, education, and training in the digital age? The concept of Open Education tries to provide a framework in theory and practice for the improvement of the learning quality through the integration of learning innovations leading to opening up education. Therefore, quality development in and by Open Education is becoming not only more and more in vogue but also crucial. It is not a fashion but an increasing requirement due to the huge changes in societies. Thus, the quality dimensions and processes of education will be introduced in the following and applied to Open Education.
Openness in general and Open Education are vague terms and therefore their dimensions are manifold due to their usage in different disciplines and subjects (Stracke, 2018). Educational dimensions and processes are described in the following section so that researchers may apply and use them for designing the structure of Open Education afterwards.
In the following, we introduce the first and unique international quality standard for education ISO/IEC 40180 that can support the design, realization, and evaluation of Open Education. It was developed and approved in consensus by the Working Group 5 "Quality Assurance and Descriptive Frameworks" of the standardisation committee ISO/IEC JTC1 SC36 and issued by the International Standardization Organization (ISO) in 2005 as ISO/IEC 19796-1 (2005). It is currently applied in more than 60 countries worldwide as national standard. ISO/IEC 19796-1 was under official revision that has to regularly take place every five years. The final revision is approved and published as ISO/IEC 40180 (2017) now.
We have selected ISO/IEC 40180 here as a framework to improve the learning quality and design of Open Education. It requires adaptation for each given situation and avoids simplifying evaluation of quality (as often realized by using only one single specified instrument, e.g., for the quality of MOOCs by Margaryan, Bianco, & Littlejohn, 2015). Other special concepts such as design-based research or agile approaches are covered by ISO/IEC 40180: They can be combined with the international quality standard and used for its application and instantiation in specific cases. The Reference Process Model from this international standard ISO/IEC 40180 provides a general framework for designing the structure for learning, education, and training that can be used for Open Education, too. It consists of seven process categories and 38 related processes as shown in Table 1 below.
Table 1
The Reference Process Model of ISO/IEC 40180 (former ISO/IEC 19796-1)
Quality does not exist in a simple manner as we have shown before. First, all stakeholders have to define their own understanding of what the term “quality” stands for in relation to the given context. Then these different perspectives and opinions about quality have to be combined, to be brought into consensus and transferred into practice. The specifications of relevant aspects and criteria to define quality as well as the applications of these criteria into the given context of the organisation are quite abstract by themselves (Stracke, 2010a). For this purpose, the development of ISO/IEC 19796-1 (now ISO/IEC 40180) was started to achieve a common reference framework and the first international quality standard for learning, education, and training based on global consensus.
In a given situation and context, the relevant processes of the quality standard have to be selected and adapted. Figure 3 below presents an example for the selection of relevant processes that are marked in dark grey (for an adaptation model to introduce quality development and in particular ISO/IEC 19796-1 see Stracke, 2010b). The selection of the processes was realized in workshops and discussions among all involved stakeholders.
Figure 3. Example for adaptation of ISO/IEC 40180 (former ISO/IEC 19796-1).
We propose the following modification of the process categories presented in Table 2 below to allow a more simplified version with only four process categories plus evaluation and optimization as overarching activities and tasks that are targeting all other four process categories. The argumentation for the changes is that the two categories "Needs Analysis" and "Framework Analysis" as well as the two categories "Development/Production" and "Implementation" are normally undertaken together whereas the "Evaluation/Optimization" is often realized by different stakeholders. In addition, we want to highlight the importance of the optimization and the involvement of the learners in this crucial process for the continuous quality development.
Table 2
Process Categories of ISO/IEC 40180 and Proposed Modifications
The standard ISO/IEC 40180 presents a good example of an internationally developed and recognised instrument for Open Education. It is valuable and applied worldwide due to its ability to be adapted for each given situation. Such flexible and adaptable instruments are required for the future spreading and implementation of Open Education. In the following, we apply general quality aspects to Open Education leading to a general framework called "OpenEd Quality Framework."
In the following, we want to develop a general framework called "OpenEd Quality Framework." Therefore, we will begin by applying general quality aspects to Open Education including the three dimensions of quality development (as discussed above) as well as the three levels of education (macro, meso, and micro).
We can transfer and apply the three generic quality dimensions that we have analysed above to learning, education, and training in general and in particular to Open Education:
Figure 4 illustrates the quality dimensions and their application to Open Education:
Figure 4. Quality dimensions in Open Education.
In general, learning, education, and training can be divided, like other sectors, into the three levels: macro level, meso level, and micro level (Stracke, 2017b). The needs analysis, design, development, realization, and evaluation of Open Education have to focus and include these three levels:
These three levels can be applied to Open Education as well as to the three quality dimensions as we will explain in the following.
In Open Education, the following key stakeholders and entities are involved at the three levels as shown in Table 3.
Table 3
Key Stakeholders and Entities in Open Education at the Three Levels
We can also transfer the quality dimensions to the Open Education and differentiate them for the three levels as shown in Table 4.
Table 4
Quality Dimensions in Open Education at the Three Levels
In the following, we transfer the quality dimensions and levels of Open Education into a general framework called "OpenEd Quality Framework" as basis for the further development of appropriate instruments and tools to improve the quality of Open Education.
In this section, we develop a general framework called "OpenEd Quality Framework" for the design, realization, and evaluation of Open Education. The Open Education (OpenEd) Quality Framework combines and integrates the quality dimensions in Open Education (cf. Figure 4) with the three levels of Open Education (cf. Table 4) as discussed above. Figure 5 illustrates these relationships.
Figure 5. Quality at macro, meso, and micro level in Open Education.
Furthermore, we can apply the process categories as modified above (cf. Table 2) to the three levels in Open Education as presented in Figure 6 below.
Figure 6. Processes at macro, meso, and micro level in Open Education.
Figure 6 presents the overview of which levels are addressed by the six process categories (e.g., AN is addressing the macro and meso levels, whereas DE is addressing all three levels) and shows in addition the relationship between the three quality dimensions and the process categories. For each of the six process categories and at each level that they are covering, we need appropriate services and instruments to support and improve the overall quality development in Open Education. Some instruments and tools are already developed and in practice such as the Quality Platform Learning (QPL, 2011), the Evaluation Framework for Impact Assessment (EFI, see Stracke, 2014b, 2013b). A general framework for the introduction of quality development also exists: The IDEA(L) framework (Stracke, 2010b), which consists of four phases "Initiate, Do, Evaluate, and Act" (adapted from PDCA cycle presented by Deming, 1982) and was also integrated into the international quality standard ISO/IEC 40180 (2017). Furthermore, the OpenEd Quality Framework can be combined with the Quality Reference Framework (QRF) that was developed for MOOCs as a specific type of Open Education with contributions from several thousands of MOOC learners, designers, facilitators, and providers (Stracke et al., 2018).
The methodologies and philosophies of education have to be adjusted to meet both current and future challenges. We need to modernize and open up education to better fit the given situation. Open Education can improve the quality of education and we have to improve the design of Open Education, to achieve a long-term and sustainable improvement of the learning quality across all educational systems, communities, sectors, and societies worldwide.
This article can only initiate the debate on the importance and impact of Open Education: Open Education can improve the quality of education and we need to improve the learning quality and design of Open Education for its broad acceptance and implementation. Our systematic literature review revealed that a holistic quality framework does not currently exist for Open Education. Therefore, we developed and presented the OpenEd Quality Framework as the first holistic quality framework. It can be used for any type of Open Education and must always be adapted to the given situation. Future research and publications are required and already started to provide more results, tools, insights, recommendations, and argumentations for further discussions and improvements.
We believe in education as a human right and public good. To keep this status due the major global challenges, learning and education have to be changed through the introduction of Open Education. This overview of Open Education in theory and practice presented the needs and potential approaches to meet these requirements. First, Open Education was defined and its history was briefly outlined. The dimensions of quality development and openness were analysed in general. Afterwards, they were transferred and adapted to Open Education. Finally, the OpenEd Quality Framework was developed integrating the modified quality dimensions and three levels of Open Education. It can be combined with other presented instruments, such as the quality standard ISO/IEC 40180 and other specific quality frameworks such as IDEA(L), EFI and the QRF.
To summarize, the OpenEd Quality Framework facilitates and fosters the development and improvement of the learning quality and design of Open Education. We believe in the importance of Open Education for our common future. It can positively impact all our personal lives and developments as well as all learning processes, educational systems, and societies worldwide.
Barbera, E., & Linder-VanBerschot, J. A. (2011). Systemic multicultural model for online education: Tracing connections among learner inputs, instructional processes, and outcomes. Quarterly Review of Distance Education, 12(3), 167-180.
Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review. London: SAGE.
Crosby, P. B. (1980). Quality is free. The art of making quality certain. New York, NY: McGraw-Hill.
Deming, W. E. (1982). Quality, productivity, and competitive position. Cambridge, MA: The MIT Press.
Deming, W. E. (1986). Out of the crisis. Cambridge, MA: The MIT Press.
Dewey, J. (1966). Democracy and education: An introduction to the philosophy of education. New York, NY: The Free Press.
Donabedian, A. (1980). The definition of quality and approaches to its assessment. Explorations in quality assessment and monitoring (Vol. 1). Ann Arbor, MI: Health Administration Press.
Downes, S. (2005, October). E-learning 2.0. eLearn Magazine. Retrieved from http://elearnmag.acm.org/featured.cfm?aid=1104968
Esfijani, A. (2018). Measuring quality in online education: A meta-synthesis. American Journal of Distance Education, 32(1), 57-73. doi: 10.1080/08923647.2018.1417658
Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Herder and Herder.
Friedman, T. (2006). The world is flat. London: Penguin Books.
Gaskell, A., & Mills, R. (2014). The quality and reputation of open, distance, and E-learning: What are the challenges? Open Learning, 29(3), 190-205. doi: 10.1080/02680513.2014.993603
Gómez-Rey, P., Barbera, E., & Fernández-Navarro, F. (2016). The impact of cultural dimensions on online learning. Journal of Educational Technology & Society, 19(4), 225-238.
Hart, J. (2011). Social learning handbook. Wiltshire: Centre for Learning & Performance Technologies.
Inglis, A. (2005). Quality improvement, quality assurance, and benchmarking: Comparing two frameworks for managing quality processes in open and distance learning. The International Review of Research in Open and Distributed Learning, 6(1), 49-61 [1-13]. doi: 10.19173/irrodl.v6i1.221
ISO/IEC 19796-1:2005 (2005). Information technology - Learning, education, and training - Quality management, assurance, and metrics - Part 1: General approach. Geneva: International Organisation for Standardization (ISO) [International Norm].
ISO/IEC 40180:2017 (2017). Information technology - Quality for learning, education, and training - Fundamentals and reference framework. Geneva: International Organisation for Standardization (ISO) [International Norm].
Jansen, D., Rosewell, J., & Kear, K. (2017). Quality frameworks for MOOCs. In M. J. Kinshuk & M. K. Khribi (Eds.), Open education: From OERs to MOOCs (pp. 261-281). Berlin, Heidelberg: Springer.
Juran, J. M. (1951). Quality control handbook. New York, NY: McGraw-Hill.
Juran, J. M. (1992). Juran on quality by design. The new steps for planning quality into goods and services. New York, NY: Free Press.
Karrer, T. (2007). Understanding e-learning 2.0. Learning Circuits (07/2007). Retrieved from http://www.astd.org/Publications/Newsletters/Learning-Circuits/Learning-Circuits-Archives/2007/07/Understanding-E-Learning-20
Kirschner, P., & van Merriënboer, J. J. G. (2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48(3), 169-183. doi: 10.1080/00461520.2013.804395
Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77-83. doi: 10.1016/j.compedu.2014.08.005
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), 1-6. doi: 10.1371/journal.pmed.1000097
Nyberg, D. (1975). The philosophy of open education. London: Routledge and Kegan Paul.
Organisation for Economic Co-operation and Development. (2016). Education at a glance 2016: OECD indicators. Paris: OECD Publishing.
Ossiannilsson, E., Williams, K., Camilleri, A. F., & Brown, M. (2015). Quality models in online and open education around the globe: State of the art and recommendations. Oslo: International Council for Open and Distance Education.
Peter, S., & Deimann, M. (2013). On the role of openness in education: A historical reconstruction. Open Praxis, 5(1), 7-14. doi: 10.5944/openpraxis.5.1.23
Peters, M. A. (2008). The history and emergent paradigm of open education. In M. A. Peters & R. G. Britez (Eds.), Open education and education for openness (pp. 3-15). Rotterdam: Sense Publishers.
Piaget, J. (1953). The origin of intelligence in the child. London: Routledge.
Prensky, M. (2001). Digital natives, digital immigrants - Part II: Do they really think differently? On the Horizon, 9(6), 1-6. doi: 10.1108/10748120110424843
Quality Platform Learning. (2011). Quality platform learning. Essen: Quality Certification for Vocational Education and Training.
Redecker, C. (2009). Review of learning 2.0 practices: Study on the impact of Web 2.0 innovations on education and training in Europe. Luxembourg: Joint Research Centre.
Rousseau, J.-J. (1968). The social contract. Harmondsworth: Penguin.
Siemens, G. (2005). Connectivism: Learning as network-creation. Retrieved from http://elearnspace.org/Articles/networks.doc
Schulmeister, R. (2008). Gibt es eine "Net Generation"? [= Is there a "Net Generation"?] Version 2.0. Retrieved from http://www.zhw.uni-hamburg.de/uploads/schulmeister-net-generation_v2.pdf
Stracke, C. M. (2006). Process-oriented quality management. In U.-D. Ehlers & J. M. Pawlowski (Eds.), Handbook on quality and standardisation in e-learning (pp. 79-96). Berlin: Springer. doi: 10.1007/3-540-32788-6_6
Stracke, C. M. (2010a). Quality development and standards in learning, education, and training: Adaptation model and guidelines for implementations. Информатизация образования и науки. Информике (Informika), 7(3), 136-146. Retrieved from http://opening-up.education/publications/stracke-c-m-2010-quality-development-and-standards-in-learning-education-and-training-adaptation-model-and-guidelines-for-implementations
Stracke, C. M. (2010b). Quality and standards in learning, education, and training: The adaptation model IDEA for the introduction of quality development. In Proceedings of the International Conference on the Past and Future of e-Learning Standards (pp. 26-36). Retrieved from http://opening-up.education/publications/stracke-c-m-2010-quality-and-standards-in-learning-education-and-training-the-adaptation-model-idea-for-the-introduction-of-quality-development
Stracke, C. M. (2013a). Open learning: The concept for modernizing school education and lifelong learning through the combination of learning innovations and quality. In C. M. Stracke (Ed.), Learning innovations and quality: The future of digital resources (pp. 15-28). Retrieved from http://opening-up.education/publications/stracke-c-m-2013-open-learning-the-concept-for-modernizing-school-education-and-lifelong-learning-through-the-combination-of-learning-innovations-and-quality
Stracke, C. M. (2013b). The evaluation framework for impact assessment. In Proceedings of 6th International Conference of Education, Research and Innovations (ICERI) (pp. 4654-4663). Retrieved from http://opening-up.education/publications/stracke-c-m-2013-the-evaluation-framework-for-impact-assessment
Stracke, C. M. (2014a). The concept of open learning for opening up education. In C. M. Stracke, U.-D. Ehlers, A. Creelman, & T. Shamarina-Heidenreich (Eds.), Changing the trajectory: Quality for opening up education (pp. 15-24). Retrieved from http://opening-up.education/stracke-c-m-2014-the-concept-of-open-learning-for-opening-up-education
Stracke, C. M. (2014b). Evaluation framework EFI for measuring the impact of learning, education, and training. Journal of East China Normal University, 2, 1-12. doi: 10.3969/j.issn.1000-5641.2012.02.012
Stracke, C. M. (2015). The need to change education towards open learning. In C. M. Stracke & T. Shamarina-Heidenreich (Eds.), The need for change in education: Openness as default? (pp. 11-23). Retrieved from http://opening-up.education/publications/stracke-c-m-2015-the-need-to-change-education-towards-open-learning
Stracke, C. M. (2017a). Open education and learning quality: The need for changing strategies and learning experiences. In Proceedings of 2017 IEEE Global Engineering Education Conference (EDUCON; pp. 1044-1048). doi: 10.1109/EDUCON.2017.7942977
Stracke, C. M. (2017b). The quality of MOOCs: How to improve the design of open education and online courses for learners? In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies. Novel learning ecosystems. LCT 2017, Part I, LNCS 10295 (pp. 285-293). doi: 10.1007/978-3-319-58509-3_23
Stracke, C. M. (2018). Como a Educação Aberta pode melhorar a qualidade de aprendizagem e produzir impacto em alunos, organizações e na sociedade? [How can Open Education improve learning quality and achieve impact for learners, organizations and in society?]. In M. Duran, T. Amiel, & C. Costa (Eds.), Utopias and distopias da tecnologia na educação a distância e aberta (pp. 499-545). Campinas: & Niterói: UNICAMP & UFF. Retrieved from http://opening-up.education/publications/stracke-c-m-2018-como-a-educacao-aberta-pode-melhorar-a-qualidade-de-aprendizagem-e-produzir-impacto-em-alunos-organizacoes-e-na-sociedade
Stracke, C. M., Kameas, A., Vassiliadis, B., Sgouropoulou, C., Texeira, A. M., Pinto, M., & Vidal, G. (2017). The quality of open online education: Towards a reference framework for MOOCs. In Proceedings of 2017 IEEE Global Engineering Education Conference (EDUCON) (pp. 1712-1715). doi: 10.1109/EDUCON.2017.7943080
Stracke, C. M., & Tan, E. (2018). The quality of open online learning and education: Towards a quality reference framework for MOOCs. In J. Kay & R. Luckin (Eds.), Rethinking learning in the digital age. Making the learning sciences count: The international conference of the learning sciences (ICLS) 2018 (pp. 1029-1032). Retrieved from http://hdl.handle.net/1820/9909
Stracke, C. M., Tan, E., Texeira, A. M., Pinto, M., Kameas, A., Vassiliadis, B., & Sgouropoulou, C. (2018). Gap between MOOC designers' and MOOC learners' perspectives on interaction and experiences in MOOCs: Findings from the global MOOC quality survey. In M. Chang, N.-S. Chen, R. Huang, Kinshuk, K. Moudgalya, S. Murthy, & D. G. Sampson (Eds.), Proceedings 18th IEEE International Conference on Advanced Learning Technologies (ICALT; pp. 1-5). doi: 10.1109/ICALT.2018.0000
Stracke, C. M., Tan, E., Texeira, A., Pinto, M., Vassiliadis, B., Kameas, A., Sgouropoulou, C., & Vidal, G. (2018). Quality reference framework (QRF) for the quality of massive open online courses (MOOCs). Retrieved from http://www.mooc-quality.eu/QRF
United Nations. (2015). Transforming our world: The 2030 agenda for sustainable development. Washington: United Nations. Retrieved from http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E
Volungeviciene, A., Tereseviciene, M., & Tait, A. (2014). Framework of quality assurance of TEL integration into an educational organization. The International Review of Research in Open and Distributed Learning, 15(6), 211-236. doi: 10.19173/irrodl.v15i6.1927
Vygotsky, L. (1988). Thought and language. Cambridge, MA: MIT Press.
Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen (Ed.), Defining and selecting key competencies (pp. 45-66). Seattle: Hogrefe & Huber.
Westera, W. (2001). Competences in education: A confusion of tongues. Journal of Curriculum Studies, 33(1), 75-88. doi: 10.1080/00220270120625
Wiley, D. (2009). Defining “open.” Retrieved from http://opencontent.org/blog/archives/1123
World Bank (2016). World development report 2016: Digital dividends. Washington, DC: World Bank.
Quality Frameworks and Learning Design for Open Education by Christian M. Stracke is licensed under a Creative Commons Attribution 4.0 International License.