International Review of Research in Open and Distributed Learning

Volume 18, Number 2

April - 2017

Review and Content Analysis of the International Review of Research in Open and Distance/Distributed Learning (2000-2015)

Olaf Zawacki-Richter¹, Uthman Alturki², and Ahmed Aldraiweesh^2
1Carl von Ossietzky University of Oldenburg, Germany, ²King Saud University, Saudi Arabia

Abstract

This paper presents a review of distance education literature published in the International Review of Research in Open and Distance/Distributed Learning (IRRODL) to describe the status thereof and to identify gaps and priority areas in distance education research based on a validated classification of research areas. All articles (N = 580) published between 2000 and 2015 were reviewed for this study. An analysis of abstracts using the text-mining tool Leximancer over three 5-year periods reveals the following broad themes over the three periods: the establishment of online learning and distance education institutions (2000-2005), widening access to education and online learning support (2006-2010), and the emergence of Massive Open Online Courses (MOOCs) and Open Educational Resources (OER) (2011-2015). The analysis auf publication and authorship patterns revealed that IRRODL is a very international journal with a high impact in terms of citations.

Keywords: distance education, research, content analysis, text-mining

Introduction

The structure of a research discipline forms the foundation for identifying gaps and priority areas (Mishra, 1998). Lee, Driscoll, and Nelson (2004) emphasize that "understanding trends and issues in terms of topics and methods is pivotal in the advancements of research on distance education" (p. 225). Content analysis of academic journals is a way to do this. However, highly rated peer-reviewed journals in distance education have been around for no more than 30-40 years. The International Review of Research in Open and Distance/Distributed Learning (IRRODL) and the Australian journal Distance Education are the only journals with a special focus on (online) distance education, which are listed in the Social Science Citation Index (SSCI).

In a recent study, Zawacki-Richter and Naidu (2016) analyzed 516 articles that were published in the Australian journal Distance Education with the text-mining tool Leximancer¹. The analysis revealed the following broad research foci over seven time periods: professionalization and institutional consolidation (1980-1984), instructional design and educational technology (1985-1989), quality assurance in distance education (1990-1994), student support and early stages of online learning (1995-1999), the emergence of the virtual university (2000-2004), collaborative learning and online interaction patterns (2005-2009), and interactive learning, Massive Open Online Courses (MOOCs), and Open Educational Resources (OER; 2010-2014).

This study presents a content analysis and a review of research trends and authorship patters based on all full articles that were published in IRRODL between 2000 and 2015 (N = 580 articles). Furthermore, the aim of this analysis is to relate the results to the flow of thematic areas mentioned above in the journal Distance Education.

Thus, this paper addresses questions in the following two areas:

• Research areas covered in the publications: What are the main research topics covered in IRRODL publications and how have they changed between 2000 and 2015? What are the most common research areas and where are potential gaps in distance education research? What is the proportion of empirical and theoretical or descriptive papers published in IRRODL? Is there a trend towards more quantitative, qualitative, or mixed-method research?
• Publication and authorship patterns: Who are the leading contributors of research papers and where do they come from? What are the most cited papers in IRRODL?

Based on the review of research areas and trends, the results can be used to explore the body of knowledge in the field of distance/distributed learning and to identify priority areas for future research projects in order to support the development of a research agenda in the context of online distance education (Zawacki-Richter & Anderson, 2014).

IRRODL in Previous Bibliographic Studies

IRRODL is an open access journal published by Athabasca University Press, Canada. Terry Anderson was the chief editor from 2003 to 2014 and Dianne Conrad and Rory McGreal followed him. Starting in 2006, IRRODL was indexed by the SSCI, which resulted in a significant increase in citations and greatly improved its international reputation (Martinez & Anderson, 2015). In January 2015, the name of the journal was changed from International Review of Research in Open and Distance Learning to International Review of Research in Open and Distributed Learning : "This was done to reflect the journal’s increased emphasis on openness in education and the blurring of boundaries in online learning to include blended and other forms of technology-enhanced learning" (McGreal & Conrad, 2016, para. 2).

IRRODL has been the subject of previous bibliometric studies and literature reviews. Zawacki-Richter and Anderson (2011) carried out a citation analysis to explore the scholarly communication network of 12 academic journals in distance education using social network analysis techniques. Although IRRODL is a relatively young journal, this analysis revealed that IRRODL is one of the central journals in the core of the journal citation network in distance education.

Based on the same sample of journals, Zawacki-Richter, Anderson, and Tuncay (2010) compared the impact of open and closed access journals including IRRODL in terms of citations and found that papers published in open access journals are cited earlier and more often. However, this difference was statistically not significant (p = 0.8).

In another bibliometric study, Martinez and Anderson (2015) investigated the most highly cited and most viewed articles published in IRRODL in the time period between 2008 and 2013 (N = 401). They reported a high Hirsch-Index of h = 30 with an average of about eight citations per paper and identified a set of 33 highly cited papers, i.e., papers that were cited at least 30 times. The most cited paper, with 134 citations, was "Connectivism: Learning theory of the future or vestige of the past" by Kop and Hill (2008). Martinez and Anderson (2015) also looked at the leading authors and their country of origin and found that these highly cited articles came from 11 different countries, based on authors’ affiliations The most common countries identified in the highly cited articles were: United States (24), United Kingdom (12), and Canada (11), followed by Germany (5), Israel (5), Turkey (2), Norway (1), Italy (1), Denmark (1), Bahrain (1), and Australia (1). There were three contributors with two articles in the highly cited selection: David Wiley from the United States, Olaf Zawacki-Richter from Germany, and Rita Kop from Canada.

This finding shows that IRRODL is a very international journal, which is supported by an earlier study of authorship patterns by Zawacki-Richter, Bäcker, and Vogt (2009) based on a sample of 695 articles published in five prominent distance education journals between 2000 and 2008.

The most international journal is IRRODL with only 18.9% of authors from Canada, followed by [Distance Education] DE with 20.5% of authors from Australia, and [Open Learning] OL with 42.2% of authors from the UK. Papers published in IRRODL come from 34 different countries, followed by DE (25 countries), OL (24 countries), [the Journal of Distance Education ] JDE (13 countries), and [the American Journal of Distance Education ] AJDE (only 7 countries). (Zawacki-Richter, Bäcker, and Vogt, 2009, p. 40)

Based on the same sample that included IRRODL, Zawacki-Richter and von Prümmer (2010) looked at gender and collaboration patterns in distance education research. Their study revealed a significant trend towards more collaborative research in distance education as well as significant gender differences noting that "Women are over-represented in research areas such as learner characteristics, learner support or interaction, and communication in learning communities, while men are more concerned with topics stereotypically associated with them: technology and management" (p. 95). There is also a significantly higher propensity of female researchers who apply qualitative methods and mixed-methods research (triangulation).

Olsen, Spring, Young, and West (2013) examined IRRODL to determine trends in research areas and methods, top-authors, and top-cited publications over one decade (2002-2011). They concluded that "IRRODL is increasing its coverage of diverse distance education topics and is looking to publish more empirical data driven research than in the past.... The topic analysis reflects the growing focus on technology-related topics in educational research" (p. 47). The authors with the highest frequency of contributions are Terry Anderson, Patrick Fahy, Olaf Zawacki-Richter, Rita Kop, and Mickey Shanchar. Interestingly, more than half of the most cited papers (8 of 14) were theoretical in nature.

Sample and Methods

Papers Published in IRRODL

For this study, all research articles published in IRRODL between 2000 and 2015 were reviewed (N = 580). Book reviews and editorial notes were excluded from the sample. Table 1 shows the growing number of papers published each year after IRRODL was included in the SSCI in 2006. Table 2 provides an overview of all special issues (N = 17) that were published by IRRODL.

Table 1

Number of Articles per Year Published in IRRODL (Volumes 1-16)

Table 2

Special Issues in IRRODL by Year

*Regional focus issue

Classification of Research Areas

The classification of research areas for this review is based on the framework developed by Zawacki-Richter (2009) that describes the research areas in distance education along three lines of research.

The three levels are:

1. Macro level: Distance education systems and theories.
2. Meso level: Management, organization, and technology.
3. Micro level: Teaching and learning in distance education.

Within these three levels, the research issues were categorized into 15 research areas (for a detailed characterization, refer to Zawacki-Richter, 2009):

Macro level: Distance education systems and theories.

1. Access, equity, and ethics.
2. Globalization of education and cross-cultural aspects.
3. Distance teaching systems and institutions.
4. Theories and models.
5. Research methods in distance education and knowledge transfer.

Meso level: Management, organization, and technology.

1. Management and organization.
2. Costs and benefits.
3. Educational technology.
4. Innovation and change.
5. Professional development and faculty support.
6. Learner support services.
7. Quality assurance.

Micro level: Teaching and learning in distance education.

1. Instructional design.
2. Interaction and communication in learning communities.
3. Learner characteristics.

All 580 articles published in IRRODL between 2000 and 2015 were coded according to this classification scheme.

Computer-Assisted Content Analysis

For the purposes of this study, the content analysis software Leximancer (2011 version) was used to produce a set of concept maps showing the semantic structure of themes and topics of articles and how they are related. Fisk, Cherney, Hornsey, and Smith (2012) indicate that computer-aided content analysis is an appropriate method to map a research domain. The software Leximancer has been used by other researchers for content analysis of academic journals, e.g., the Journal of Cross-Cultural Psychology (Cretchley, Rooney, & Gallois, 2010), the Journal of International Business Studies (Liesch, Håkanson, McGaughey, Middleton & Cretchley, 2011), the Journal of Communication (Lin & Lee, 2012), and Distance Education (Zawacki-Richter & Naidu, 2016).

The software identifies core concepts within textual data (conceptual analysis) and how these concepts are related (relational analysis) by the frequency with which words co-occur in the text. Similar concepts that co-occur in close proximity are clustered in the visual map produced by Leximancer: "The map is an indicative visualization that presents concept frequency (brightness), total concept connectedness (hierarchical order of appearance), direct interconcept relative co-occurrence frequency (ray intensity), and total (direct and indirect) interconcept co-occurrence (proximity)" (Smith & Humphreys, 2006, p. 264). Depending on the connectedness of concepts they form a thematic region. Such a "theme" is named after the most prominent concept in that group.

Titles and abstracts are lexically dense and focus on the core themes and results presented in the articles. An initial overall analysis was run with abstracts and titles of 580 articles, in which common terms (and, not, etc.) were excluded. In addition, Leximancer was requested to merge word variants, e.g., distance and education or open and university.

For the purposes of this study, Leximancer was used to analyze both the entire data set (2000-2015) and each 5-year time period separately. Abstracts and titles of all articles published in IRRODL between Volume 1 (1) in 2000 and Volume 16 (6) in 2015 were collected. A total of 580 full papers, and subsets of data in three 5-year periods were created: 2000-2005² (99 articles), 2006-2010 (138 articles), and 2011-2015 (343 articles).

Limitations

We acknowledge that IRRODL is only one window to look into the structure and flow of research areas in online distance and distributed learning and the selection of the sample for the purposes of this study is subject to interferences:

The most important of these is surely the gatekeeping role of reviewers, editors and journals in general. Quite aside from what one might prefer to do, publication responds to funding possibilities and publishing possibilities, and these in turn respond to connections and selection of a topic, a method, and a choice of potential journal most likely to lead to publication. (Goldenberg & Grigel, 1991, p. 436)

A text-mining tool such as Leximancer is a powerful instrument for the purpose of mapping a research domain (see Fisk, Cherney, Hornsey, & Smith, 2012), but we have to acknowledge that the results must be interpreted with profound knowledge of the subject matter. Leximancer is known to produce stable results as has been noted by Harwood, Gapp, and Stewart (2015). Their analysis revealed encouraging similarities between a Leximancer output and main themes and codes derived from a manual Grounded Theory analysis. However, they remind us that

Leximancer is not a panacea, it still requires analytical sensitivity and judgment in its interpretation, but it is straightforward to probe the data and cross-check via the resultant maps.... Leximancer enables the analyst to make sense of large narrative data sets with minimal manual coding. The result is an efficient and impartial second opinion on open codes (concepts, categories and dimensions) and potential links between them. (Harwood, Gapp, & Stewart, 2015, p. 1041)

Results and Discussion

Content Analysis: Research Areas and Trends

Overall scope of the journal (2000-2015). The concept map in Figure 1 depicts the major topics covered in the articles published over the first 16 years of the journal (2000-2015). The thematic summary includes a connectivity score to indicate the relative importance of the themes. The results reveal that learning has the most direct mentions within the text with 276 (100% relative count), followed by study (66%), online (45%), distance (42%), social (13%), and resources (7%). This is not a surprising result, as this journal deals with the study of online distance education. While this may seem like a trivial observation, it confirms the veracity of Leximancer as a useful text-mining tool.

Figure 1. Concept map of overall scope of the journal IRRODL (2000-2015).

A key focus of the articles published in IRRODL as an open access journal is the potential and role of open and distance learning to provide access to education and educational opportunities. The concepts role, distance, and education are directly connected with educational, resources, and open in the concept map (see Figure 1).

Two major topics covered in the publications (i.e., learning and online ) are connected via the thematic region social, including the concepts of interaction, communication, and support. Thus, learning is seen as a social process that is facilitated by interaction among learners and teachers. The provision of opportunities for interaction, communication, and collaboration between students and their teachers as well as among students, via electronic media, is seen as a constituent element of distance education (Keegan, 1980).

Table 3

Distribution of Research Areas From 2000-2015 (N = 580)

In the three 5-year intervals there are recurring concepts, such as students, online, and education, that refer to very broad and common thematic areas throughout the whole time period. In the following content analysis and interpretation, emphasis is placed on new and emerging concepts in order to describe the changes and trends in distance education research.

Table 3 provides an overview of the 15 research areas in terms of their frequency covered in the publications. The top issues are related to the micro-level of teaching and learning in online distance education: instructional design, learner characteristics, interaction and communication in learning communities, and the application of educational technology to design flexible learning opportunities. Research areas on the bottom of the list are costs and benefits and learner support services (with 12 papers) and globalization of education and innovation and change (with only 10 papers). This corresponds with earlier findings of Zawacki-Richter, Bäcker, and Vogt (2009).

Online learning and distance education institutions (2000-2005). The major themes emerging between 2000 and 2005 are education (100%), students (60%), online (24%), development (14%), research (7%), and analysis (4%) (see Figure 2). The mid-1990s saw growing interest in online learning and the transition to online teaching institutions, which was an enormous process of innovation. Zawacki-Richter and Naidu (2016) call the time period between 2000 and 2004 "the emergence of the virtual university" (p. 257).

Accordingly, many papers deal with the challenges, opportunities, and processes of introducing online learning at higher education institutions. The concepts education, higher, processes, institutions, and delivery are directly connected in the concept map. Also, the results of the manual coding of research areas that are reported in Table 4 show that "Distance teaching systems and institutions" is the top research issue, followed by "Interaction and communication in learning communities," and "Theories and models." There are papers that focus on the development of online course delivery and online student support services worldwide, such as Deakin University as an example for a dual-mode university in Australia (Calvert, 2001), the NKI Internet College in Norway (Paulsen & Rekkedal, 2001), or the Indira Ghandi National Open University (IGNOU) in India (Sharma, 2001). Other articles consider the "hybridization" of whole national higher education systems, such as Brazil (Litto, 2002), or Canada (Shale, 2002).

Figure 2. Concept map of online learning and distance education institutions (2000-2005).

Theoretical papers in this time period elaborate on the pedagogical opportunities that online learning affords. Peters (2000) explores pedagogical and didactical activities in online learning environments and develops a theory for digital learning spaces, and Anderson (2003) discusses the role of new information and communication technologies to optimize the mix between independent study and interactive learning strategies and activities.

Emphasis in empirical research is placed on the students in this new online learning environment. Especially, the building and sustaining a sense of community in the online classroom is a major focus (Rovai, 2002). Researchers investigate questions such as:

Does a social atmosphere develop in online learning discussion groups? What are the different modes of social interaction are manifest in online learning discussion groups? What is the role of the virtual teacher with regards to the social climate in online learning discussion groups? (Oren, Mioduser, & Nachmia, 2002, p. 1)

It is interesting to look at the research areas that are at the bottom of the ranking in terms of the number of published articles in this time period (Table 4): costs and benefits, professional development and faculty support, and access, equity, and ethics. Researchers turned their attention to these neglected research areas in the next time period.

Table 4

Distribution of Research Areas From 2000-2005 (N = 99)

Widening access to education and online learning support (2006-2010). In this second half of the first decade in the new millennium, conventional distance education is in the transition to online learning. As Figure 3 shows, the major topics canvassed in publications over these five years are: students (100%), education (54%), university (15%), open (9%), support (6%), and social (2%). Online and learning are central concepts in the thematic region of students.

Figure 3. Concept map of widening access to education and online learning support (2006-2010).

Around this time the issue of access to educational opportunities and open education came to the fore, moving up from the eighth to the third rank of research areas covered in the publications (see research area of "access, equity, and ethics" in Table 5). Four special issues deal with all kinds of aspects related to openness and open education: Open and Distance Education in Asia (2007), The Role of Distance Learning in the Right to Education (2008), Open and Distance Learning in Africa (2009), and Openness and the Future of Higher Education (2009). Papers in this thematic context explore the emergence of open-source software, course management systems, in particular, in North America (Pan & Bonk, 2007), the role of using ICTs and open and distance learning in increasing access, equity, and quality of rural teachers professional development in China or the potential of open access to reduce the high costs of textbooks at American universities (Baker, Thierstein, Fletcher, Kaur, & Emmons, 2009). Fini (2009) published the first paper in IRRODL about the evaluation of a MOOC, the famous Connectivism and Connective Knowledge (CCK08) course.

There is also growing awareness among scholars and practitioners at this time of the importance of professional development and faculty support, which is critical for the development of high-quality online programs and education innovation (Brindley, Zawacki, & Roberts, 2003). The concept map shows that online, learning, and development, and faculty are linked via the concept quality (see Figure 5). For example, Villar and Alegre (2006) present an online faculty development and assessment system: "The rapid growth of online learning has led to the development of faculty evaluation models that are geared towards the demands of quality improvement of degree programs" (p. 1). Also, Chao, Saj, and Hamilton (2010) highlight that "the issue of quality is becoming front and center as online distance education moves into the mainstream of higher education" (p. 106). Eib and Miller (2006) see faculty development as an on-going need and long-term, continuous effort and emphasize the importance of community building "to create a culture that supports a thoughtful focus on teaching" (p. 1).

Table 5

Distribution of Research Areas from 2006-2010 (N = 138)

Although support emerged as a concept that forms a thematic region of its own, and 14 papers were published that deal with professional development and faculty support, there are interestingly no articles between this 5-year period that deal with student support service systems (see Table 5) on the institutional level of educational management (meso-level). In contrast, Zawacki-Richter and Naidu (2016) found that the provision of student support services was a major research area in the 1990s, when student support was acknowledged as a "critical link in distance education" (Dillon, Gunawardena, & Parker, 1992, p. 29).

However, researchers around this time are fascinated by the enormous opportunities that the new information and communication technologies afford for collaborative online learning and teaching (micro-level). There are several publications in IRRODL that seek to understand how online collaborative learning occurs and how students are best supported to facilitate their learning processes: Barnard-Brak, Paton, and Lan (2010) developed a typology of self-regulated learning profiles in the online learning environment and several studies deal with the role and function of online tutors (e.g., Ng, 2007; Murugaiah & Thang, 2010). The concept of support is connected with access. For example, Long, Vignare, Lappold, and Mallory (2007) reported on a study that investigated how to provide access to computer-mediated communication for learners with special needs like deaf, hard-of-hearing, and ESL students.

The emergence of MOOCs and OER (2011-2015). Over the next five years (2011-2015), the themes students (100%) and education (56%) continue to be critical followed by teaching (36%), courses (20%), system (18%), MOOCs (5%), and OER (3%). From 2011-2015 four special issues were released about open education, especially MOOCs and OER. These issues were titled: Towards a European perspective on MOOCs (2015), OER and MOOCs (2015), Research into MOOCs (2014), and OERs: Opening Access to Knowledge (2013). Another special issue in 2011 dealt with Prior, Experiential and Informal Learning in the Age of Information and Communication Technologies (PLAR), which is also closely related to openness in (higher) education (see Conrad, 2010).

The concept of students, online, courses, and MOOCs are directly connected in the concept map (see Figure 6). Many papers describe and evaluate the development of MOOCs for different target groups: from the integration of MOOCs into secondary school courses (Najafi, Evans, & Federico, 2014) or the area of corporate training (Radford et al., 2014) to contexts of limited resources such as developing countries in Africa (Oyo & Kalema, 2014). Open education does not mean that education is free, as someone has to pay the bill for the sustainable development and delivery of courses and course materials. The issue of costs and economics of open and distance/distributed learning moves up in the ranking slightly in terms of frequency covered in the publications (see Table 6). Some authors raise the question of costs for the development and delivery MOOCs (Hollands & Tirthali, 2014) and financial sustainability of open textbooks (Iii & Wiley, 2011).

Figure 4. Concept map of MOOCs and OER (2011-2015).

Proponents of OER claim that significant cost savings are possible when open textbooks replace traditional textbooks in the classroom. However, Wiley, Iii, Ellington, and Hall (2012) report contradicting findings as open textbooks are deployed in various ways: "Some of these methods cost more than traditional textbooks; however, we did identify and implement a successful model of open textbook adoption that reduces costs by over 50% compared to the cost of adopting traditional textbooks" (p. 262).

Professional development and faculty support remains an important topic in the thematic region of teaching (see concept path teachers - professional - development ). This research area is also ranked fifth in Table 6. Papers published in IRRODL notably explored the potential of open and distance learning for teacher training and the development of competencies for the implementation and use of ICTs in the teaching and learning process (e.g., van Zyl, Els, & Blignaut, 2013; Mhishi, Bhukuvhani, & Sana, 2012).

Table 6

Distribution of Research Areas from 2011-2015 (N = 343)

Methods, Publication, and Authorship Patterns

Leading authors . The total number of different authors who contributed to the 580 articles in this study was 1157. Table 7 lists the 40 authors who contributed to at least three articles. They are from the USA (12), Canada (11), South Africa (4), Australia (2), Israel (2), Denmark (1), Germany (1), the Netherlands (1), Norway (1), Portugal (1), Spain (1), Turkey (1), and the UK (1). The top author with nine publications in IRRODL is Terry Anderson from Athabasca University in Canada. He was the chief editor of IRRODL from 2003 to 2014. Zawacki-Richter, Anderson, and Tuncay (2010) showed that there is a positive bias of editors towards their own journal.

Table 7

Leading Contributors and Number of Their Published Articles

Gender and research methods. The analysis in Table 8 depicts that 324 (55.9%) of the first authors were men and 256 (44.1%) were women. The results appear to confirm the stereotypical view that female researchers (64.2%) are more likely than males (35.8%) to choose qualitative methods. The same pattern was found by Zawacki-Richter and von Prümmer (2010) looking at papers published in five distance education journals between 2000 and 2008. There is a highly significant association between gender and research methods, χ² = 27.62, df = 3, p <.001. However, the association is modest at Cramer's V of.22 (p <.001).

Overall, there seems to be a tendency towards more empirical research. Only 44.5% of all articles published IRRODL are theoretical or descriptive in nature, whereas Zawacki-Richter, Bäcker, and Vogt (2000) reported that IRRODL was the journal that accepted the highest number of descriptive or theoretical papers (56.6 %) between 2000 and 2008.

Table 8

Cross Tabulation of Gender (First Author) and Research Methods

Countries. For the analysis of the geographic distribution of articles, the country of origin of the first author was taken into consideration (64 countries, see Figure 5). The results confirm the earlier finding by Zawacki-Richter, Bäcker, and Vogt (2009) that IRRODL is a very international journal. The articles came from 64 different countries (see Table 9), the majority from the USA, Canada, the UK, Australia, South Africa, Spain, Germany, and Turkey.

Table 9

Distribution of Author’s Country of Origin for Articles in IRRODL

*only the country of the first author was taken into consideration here.

Figure 5. Countries of origin of first authors.

Top-cited articles and authors. Some journals put a strong emphasis on empirical work and discourage, or even disallow, theoretical work, including literature reviews. However, based on a series of literature reviews of educational technology journals, West (2016) consistently found that

theoretical/literature-based articles to be among the most cited in a journal. For example, in Educational Technology Research and Development, when looking at the top-cited articles from each year in the decade, seven were theoretical articles, not reporting new empirical findings. (p. 44)

The following table includes the top 10 most cited articles for the entire time period, as well as the most cited article from each year. The results of the citation analysis confirm the findings by West (2016). Only eight out of the 18 most cited articles are empirical in nature, using quantitative, qualitative, or mixed-methods analysis. Theoretical work is important, and obviously, readers and scholars value these articles.

Table 10

Most Cited Papers From Years 2000-2015 for IRRODL

Conclusion

The aim of this study was to investigate the research areas and trends that are covered in publications in IRRODL and the authorship patterns that apply to these areas. In conclusion, the research areas covered in the three 5-year time periods can be described as follows: 1) Online learning and distance education institutions (2000-2005), 2) Widening access to education and online learning support (2006-2010), and 3) The emergence of MOOCs and OER (2011-2015). The mapping of the contents of IRRODL can be related to the flow of thematic areas over roughly the time periods in the journal Distance Education (see Zawacki-Richter & Naidu, 2016): The emergence of the virtual university (2000-2004), Collaborative learning and online interaction patterns (2005-2009), and interactive learning, MOOCs, and OER (2010-2014). This indicates high similarity between major research topics covered in the publications in these two journals. However, IRRODL has a stronger focus on open education, which is also reflected in the name change from the International Review of Open and Distance Learning to the International Journal of Open and Distributed Learning (McGreal & Conrad, 2016).

With regard to the publication and authorship patterns, major findings of this study may be summarized as follows:

• The 580 contributions published in IRRODL originated from 64 countries, but the majority from wealthy western nations like the USA, Canada, the UK, Australia, South Africa, Spain, and Germany. However, the results also show that IRRODL is an important outlet for scholars from many different and developing countries as well.
• The citation analysis revealed a range of highly cited articles that were published in IRRODL, and interestingly the top-cited papers are theoretical in nature. This finding supports the fact that theoretical papers are important and valued not only by IRRODL's readership but also recognized as influential on the field at large.
• Nevertheless, there seems to be a trend towards more studies in IRRODL that apply empirical research methods (qualitative, quantitative, and mixed). Over the whole time period, only 44.5% of the research articles are theoretical or descriptive in nature.

All in all, the name tells the story: The International Review of Research in Open and Distributed Learning is a very international journal with articles from over 60 countries and with a major focus in the publications placed on openness and access to educational opportunities. IRRODL is an excellent outlet for open scholars worldwide to communicate and share their research findings in a high impact journal.

Notes

1. http://www.leximancer.com
2. The first issue of IRRODL was published at the end of 2000. This issue was added to the first 5-year time period from 2001-2005.

Acknowledgement

The authors would like to extend their sincere appreciation to the Deanship of Scientific search at King Saud University for its funding this Research group NO. RG #---1435--003

References

Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for interaction. International Review of Research in Open and Distance Learning, 4(2).

Baker, J., Thierstein, J., Fletcher, K., Kaur, M., & Emmons, J. (2009). Open textbook proof-of-concept via Connexions. The International Review of Research in Open and Distributed Learning, 10(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/633

Barnard-Brak, L., Paton, V. O., & Lan, W. Y. (2010). Profiles in self-regulated learning in the online learning environment. The International Review of Research in Open and Distributed Learning, 11(1), 61-80.

Brindley, J. E., Zawacki, O., & Roberts, J. (2003). Support services for online faculty: The provider’s and the users’ perspectives. In U. Bernath & E. Rubin (Eds.), Reflections on teaching and learning in an online master program - A case study (pp. 137-165). Oldenburg: Bibliotheks- und Informationssystem der Universität Oldenburg.

Calvert, J. (2001). Deakin University: Going online at a dual mode university. International Review of Research in Open and Distance Learning, 2(2), 1-20.

Chao, I. T., Saj, T., & Hamilton, D. (2010). Using collaborative course development to achieve online course quality standards. The International Review of Research in Open and Distributed Learning, 11(3), 106-126.

Conrad, D. (2010). Achieving flexible learning through recognition of prior learning practice: A case study lament of the Canadian academy. Open Learning, 25(2), 153-161.

Cretchley, J., Rooney, D., & Gallois, C. (2010). Mapping a 40-year history with leximancer: Themes and concepts in the Journal of Cross-Cultural Psychology. Journal of Cross-Cultural Psychology, 41(3), 318-328. doi: http://doi.org/http://dx.doi.org/10.1177/0022022110366105

Dillon, C. L., Gunawardena, C. N., & Parker, R. (1992). Learner support: The critical link in distance education. Distance Education, 13(1), 29-45.

Eib, B. J., & Miller, P. (2006). Faculty development as community building - An approach to professional development that supports communities of practice for online teaching. The International Review of Research in Open and Distributed Learning, 7(2). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/299

Fini, A. (2009). The technological dimension of a Massive Open Online Course: The case of the CCK08 course tools. The International Review of Research in Open and Distributed Learning, 10(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/643

Fisk, K., Cherney, A., Hornsey, M., & Smith, A. (2012). Using computer-aided content analysis to map a research domain: a case study of institutional legitimacy in postconflict East Timor. SAGE Open, 2(4). http://doi.org/10.1177/2158244012467788

Goldenberg, S., & Grigel, F. (1991). Gender, science and methodological preferences. Social Science Information, 30(3), 429-443.

Harwood, I., Gapp, R. P., & Stewart, H. J. (2015). Cross-check for completeness: Exploring a novel use of Leximancer in a Grounded Theory study. The Qualitative Report, 20(7), 1029-1045.

Hollands, F. M., & Tirthali, D. (2014). Resource requirements and costs of developing and delivering MOOCs. The International Review of Research in Open and Distributed Learning, 15(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1901

Iii, J. L. H., & Wiley, D. (2011). Open access textbooks and financial sustainability: A case study on Flat World Knowledge. The International Review of Research in Open and Distributed Learning, 12(5), 18-26.

Keegan, D. (1980). On defining distance education. Distance Education, 1(1), 13-36.

Kop, R., & Hill, A. (2008). Connectivism: Learning theory of the future or vestige of the past? International Review of Research in Open and Distance Learning, 9(3), 1-13.

Lee, Y., Driscoll, M. P., & Nelson, D. W. (2004). The past, present, and future of research in distance education: Results of a content analysis. American Journal of Distance Education, 18(4), 225-241. doi: http://doi.org/http://dx.doi.org/10.1207/s15389286ajde1804_4

Liesch, P. W., Håkanson, L., McGaughey, S. L., Middleton, S., & Cretchley, J. (2011). The evolution of the international business field: a scientometric investigation of articles published in its premier journal. Scientometrics, 88(1), 17-42. doi: http://doi.org/10.1007/s11192-011-0372-3

Lin, J., & Lee, S. T. (2012). Mapping 12 years of communication scholarship: Themes and concepts in the Journal of Communication. In H.-H. Chen & G. Chowdhury (Eds.), The Outreach of Digital Libraries: A Globalized Resource Network (Vol. 7634, pp. 359-360). Berlin, Heidelberg: Springer Berlin Heidelberg. Retrieved from http://link.springer.com/10.1007/978-3-642-34752-8_53

Litto, F. (2002). The hybridization of distance learning in Brazil: An approach imposed by culture. International Review of Research in Open and Distance Learning, 2(2), 1-14.

Long, G., Vignare, K., Rappold, R. P., & Mallory, J. R. (2007). Access to communication for deaf, hard-of-hearing and ESL students in blended learning courses. The International Review of Research in Open and Distributed Learning, 8(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/423

Martinez, R. A., & Anderson, T. (2015). Are the most highly cited articles the ones that are the most downloaded? A bibliometric study of IRRODL. International Review of Research in Open and Distributed Learning, 16(3), 18-40.

McGreal, R., & Conrad, D. (2016). RE: Name change of IRRODL journal. Retrieved from http://www.irrodl.org/home/announcements/irrodlnamechange.pdf

Mhishi, M., Bhukuvhani, C. E., & Sana, A. F. (2012). Science teacher training programme in rural schools: An ODL lesson from Zimbabwe. The International Review of Research in Open and Distributed Learning, 13(1), 72-86.

Mishra, S. (1998). Distance education research: a review of its structure, methodological issues and priority areas. Indian Journal of Open Learning, 7(3), 267-282.

Murugaiah, P., & Thang, S. M. (2010). Development of interactive and reflective learning among Malaysian online distant learners: An ESL instructor’s experience. The International Review of Research in Open and Distributed Learning, 11(3), 21-41.

Najafi, H., Evans, R., & Federico, C. (2014). MOOC integration into secondary school courses. The International Review of Research in Open and Distributed Learning, 15(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1861

Ng, K. C. (2007). Replacing face-to-face tutorials by synchronous online technologies: Challenges and pedagogical implications. The International Review of Research in Open and Distributed Learning, 8(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/335

Olsen, A., Spring, K. J., Young, J., & West, R. E. (2013). International Review of Reserach in Open and Distance Learning, 2002-2011. Educational Technology, (July/August), 43-47.

Oren, A., Mioduser, D., & Nachmias, R. (2002). The Development of social climate in virtual learning discussion groups. International Review of Research in Open and Distance Learning, 3(1), 1-19.

Oyo, B., & Kalema, B. M. (2014). Massive open online courses for Africa by Africa. The International Review of Research in Open and Distributed Learning, 15(6). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1889

Pan, G., & Bonk, C. J. (2007). The emergence of open-source software in North America. The International Review of Research in Open and Distributed Learning, 8(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/496

Paulsen, M. F., & Rekkedal, T. (2001). The NKI Internet College: A review of 15 years delivery of 10,000 online courses. International Review of Research in Open and Distance Learning, 2(2), 1-18.

Peters, O. (2000). Digital learning environments: New possibilities and opportunities. International Review of Research in Open and Distance Learning, 1(1), 1-19.

Radford, A. W., Robles, J., Cataylo, S., Horn, L., Thornton, J., & Whitfield, K. E. (2014). The employer potential of MOOCs: A mixed-methods study of human resource professionals’ thinking on MOOCs. The International Review of Research in Open and Distributed Learning, 15(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1842

Rovai, A. (2002). Building sense of community at a distance. International Review of Research in Open and Distance Learning, 3(1), 1-16.

Shale, D. (2002). The hybridisation of higher education in Canada. International Review of Research in Open and Distance Learning, 2(2), 1-11.

Sharma, R. (2001). Online delivery of programmes: A case study of Indira Gandhi National Open University (IGNOU). International Review of Research in Open and Distance Learning, 2(2), 1-15.

Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Behavior Research Methods, 38(2), 262-279. doi: http://doi.org/10.3758/BF03192778

Villar, L. M., & Alegre, O. M. (2006). Online faculty development and assessment system (OFDAS). The International Review of Research in Open and Distributed Learning, 7(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/367

West, R. E. (2016). Insights from the journal analysis series: What we have learned about educational technology research. Educational Technology, (January-February), 41-47.

Wiley, D., Iii, J. L. H., Ellington, S., & Hall, T. (2012). A preliminary examination of the cost savings and learning impacts of using open textbooks in middle and high school science classes. The International Review of Research in Open and Distributed Learning, 13(3), 262-276.

Zawacki-Richter, O. (2009). Research areas in distance education - a Delphi study. International Review of Research in Open and Distance Learning, 10(3), 1-17.

Zawacki-Richter, O., & Anderson, T. (2011). The geography of distance education - bibliographic characteristics of a journal network. Distance Education, 32(3), 441-456.

Zawacki-Richter, O., & Anderson, T. (Eds.). (2014). Online distance education - towards a research agenda. Athabasca, Edmonton, Canada: Athabasca University Press. Retrieved from http://www.aupress.ca/index.php/books/120233

Zawacki-Richter, O., Anderson, T., & Tuncay, N. (2010). The growing impact of open access distance education journals - a bibliometric analysis. Journal of Distance Education, 24(3). Retrieved from http://www.jofde.ca/index.php/jde/article/view/661

Zawacki-Richter, O., Bäcker, E. M., & Vogt, S. (2009). Review of distance education research (2000 to 2008) - analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning, 10(6), 21-50.

Zawacki-Richter, O., & Naidu, S. (2016). Mapping research trends from 35 years of publications in Distance Education. Distance Education, 37(3), 245-269. doi: https://doi.org/10.1080/01587919.2016.1185079

Zawacki-Richter, O., & von Prümmer, C. (2010). Gender and collaboration patterns in distance education research. Open Learning, 25(2), 95-114.

van Zyl, J. M., Els, C. J., & Blignaut, A. S. (2013). Development of ODL in a newly industrialised country according to face-to-face contact, ICT, and e-readiness. The International Review of Research in Open and Distributed Learning, 14(1), 84-105.

Review and Content Analysis of the International Review of Research in Open and Distance/Distributed Learning (2000-2015) by Olaf Zawacki-Richter, Uthman Alturki, and Ahmed Aldraiweesh is licensed under a Creative Commons Attribution 4.0 International License.