International Review of Research in Open and Distributed Learning

Volume 21, Number 4

November - 2020

Recommender Systems for MOOCs: A Systematic Literature Survey (January 1, 2012—July 12, 2019)

Asra Khalid, Karsten Lundqvist, and Anne Yates
Victoria University of Wellington

Abstract

In recent years, massive open online courses (MOOCs) have gained popularity with learners and providers, and thus MOOC providers have started to further enhance the use of MOOCs through recommender systems. This paper is a systematic literature review on the use of recommender systems for MOOCs, examining works published between January 1, 2012 and July 12, 2019 and, to the best of our knowledge, it is the first of its kind. We used Google Scholar, five academic databases (IEEE, ACM, Springer, ScienceDirect, and ERIC) and a reference chaining technique for this research. Through quantitative analysis, we identified the types and trends of research carried out in this field. The research falls into three major categories: (a) the need for recommender systems, (b) proposed recommender systems, and (c) implemented recommender systems. From the literature, we found that research has been conducted in seven areas of MOOCs: courses, threads, peers, learning elements, MOOC provider/teacher recommender, student performance recommender, and others. To date, the research has mostly focused on the implementation of recommender systems, particularly course recommender systems. Areas for future research and implementation include design of practical and scalable online recommender systems, design of a recommender system for MOOC provider and teacher, and usefulness of recommender systems.

Keywords: recommender system, massive open online course, MOOC, systematic review, implemented recommender system

Introduction

Access to higher education can be restrictive and expensive but it can also be improved by implementing enhanced and novel methods and solutions. Massive open online courses (MOOCs) are a potential solution that have been used for more than a decade. Their spread is enabling learners to satisfy learning needs in an open, participatory, and distributed way. The term MOOC was first introduced in 2008 when the course Connectivism and Connective Knowledge was offered by George Siemens and Stephen Downes (Downes, 2008). Siemens designed this course according to the principles of connectivism, and due to the vast number of participants, it was named a massive open online course (Adham & Lundqvist, 2015). In 2011, at Stanford University, a MOOC different from Siemens and Downes’ was designed. Learning objectives and plans were defined, and it followed a traditional teaching style (Sunar, Abdullah, White, & Davis, 2016). This is known as a content-based MOOC (xMOOC). Currently most MOOCs are not designed on the principles of connectivism, but instead are xMOOCs.

The number of MOOCs and the number of students registered in MOOCs are growing every year. By the end of 2018, more than 900 universities were offering MOOCs with 11,400 courses available, and around 101 million students had registered in them (Shah, 2018), providing learners with a wide variety of choices. With such a high number of courses available, learners now face the problem of selecting courses without being overwhelmed.

With the increase in e-commerce and online business, the number of users attracted to online Web services has increased. Both MOOC providers and online businesses advertise their courses and services while learners search for courses that match their interests and needs. In these situations, recommender systems play an important role, and have attracted the attention of researchers. Recommender systems are algorithms and techniques that recommend matching and relevant courses or services to the learner depending upon their interests, information about which comes from learner profiles and histories gathered by the systems. Recommender systems help MOOC providers grow and learners find more appropriate and customized services tailored to their personalities and interests. An example is provided below.

Mark has a free slot in the evening, and he wants to polish his professional skills by registering in a part-time course Introduction to Java. Mark has no idea about the course, and he does not want to waste his money on something that will not help his career. What will he do? Mark has different options: he can ask his friend who has completed this course, or he can observe details of the course, such as the content, length, pre-requisites, and instructors to reach a decision. In this case, Mark is searching for recommendations or inferring data to generate a recommendation for himself. What should we do if we face the same problem in our online learning life? We could use recommender systems, which help diminish information overload.

Recommender systems discover patterns in considerable datasets to learn the preferences of different users and predict items that correlate to their needs. Here item is a generic term that represents any course, learning element, book, service, application, or product. Recommender systems mostly use machine learning and data mining techniques to achieve their goals (Ricci, Rokach, Shapira, & Kantor, 2010). These systems are used intensively in e-commerce and by retailers to lift their sales and audience and now, increasingly, for learning purposes in MOOCs.

According to Manouselis, Drachsler, Verbert, and Duval (2013), recommender systems can be divided into two broad categories: collaborative filtering recommender systems and content-based recommender systems. There is a third type called the hybrid that contains characteristics of both collaborative filtering and content-based recommender systems.

Collaborative filtering recommender systems perform recommendations on the assumption that people who have had similar taste in the past will make similar choices in the future. This can be compared with real life scenarios in which, when we have to choose from multiple available options, we consider the recommendations of family and friends who have similar interests (Dakhel & Mahdavi, 2013).

Content based recommender systems consider the profile of users and items. Profiles of users can include age, gender, education, and residency area. Characteristics of items, for example in the case of movies, might include actor, genre, category, and type. These recommender systems analyze the items rated by a user and try to design a model that reflects the interests of that user. This model is employed to recommend new items to the user (Lops, de Gemmis, & Semeraro, 2011).

With the increased use of MOOCs, data produced by MOOCs is also expanding. This data contains information about the interests and behaviors of learners and the courses in which they are registered, and that data can be used by a recommender system to make recommendations (Ricci et al., 2010). Recommender systems in MOOCs can help the learner find related learning objects or elements. MOOC providers can also use these systems to inform MOOC design and creation.

The purpose of this systematic literature review was to fully scope and report on: (a) how recommender systems have been used in MOOCs between 2012 and 2019, (b) the trends over this period, and (c) the types of recommender systems yet to be explored. This research reviewed all related work between January 1, 2012 and July 12, 2019, in the English language only. We chose 2012 as the starting year because it was declared the Year of the MOOC by The New York Times (Pappano, 2012) and, from that year, publication of peer-reviewed research on recommender systems in MOOCs started.

Method

According to Fink (2005), a systematic literature review is an organized, comprehensive, and reproduceable method of review. Using this definition as a framework, the purpose of this study was to

• report on work on recommender systems for MOOCs; and
• provide a comprehensive analysis that could be used to find opportunities for research and implementation in the field.

Our methodology consisted of two fundamental steps: data collection and data analysis. The analysis was further divided into quantitative and qualitative analyses.

Data Collection

Gathering data from the literature was performed with care to maximize accuracy. A set of rules describing the criteria for selection of research papers was established. These rules involved four significant points: (a) search terms, (b) research period, (c) sources, and (d) publication type. Search terms are used to find related published work from specific sources, while research period refers to the publication date, and publication type refers to the type of paper, such as journal article, conference paper, book chapter, or review article. The following sections explain these rules in more detail.

Search terms. This review involved two main concepts: massive open online courses and recommender systems. Therefore, we started with the following search terms: “massive open online courses” AND “MOOCs” AND “recommender system.” We added “RS,” a common abbreviation for recommender systems, but that resulted in many unrelated papers. Similarly, we used “MOOC” instead of “MOOCs,” which also resulted in many unrelated papers since MOOC is used as an abbreviation for other terms such as “multiple optical orthogonal code sequences” and “management of organizational change.” We also used “adaptive MOOCs” and “personalized MOOCs” along with “recommender system” and “massive open online courses.” With “personalized MOOCs,” we only found one related paper which was already in our database, whereas the term “adaptive MOOCs” resulted in seven papers, though they were also part of our database. Most of the unrelated papers in the latter case were about making MOOCs adaptive and not about recommending any resource or service to users.

Thus, we finalized the search terms: “massive open online courses” AND “MOOCs” AND “recommender system” because these were the most efficient for locating the literature we were seeking.

Research period. We reviewed papers published between January 1, 2012 and July 12, 2019.

Sources. To determine the sources of research, we followed the same methodology as Liyanagunawardena, Adams, and Williams (2014). We used Google Scholar, academic databases, and the reference chaining technique of Gao, Luo, and Zhang (2012). The initial searching was in Google Scholar, followed by selected academic databases. We chose five databases from the area of computer science and education: the Institute of Electrical and Electronics Engineers (IEEE) Xplore, the Association for Computing Machinery (ACM) journals/Transactions Springer Link, ScienceDirect, and the Education Resources Information Centre (ERIC). Reference chaining was performed at the end to find any further related work.

Publication type. Peer reviewed conference papers, journals, and book chapters were included in this literature review.

Data Analysis

We performed both quantitative and qualitative analysis on the data. In the quantitative analysis, we classified data based on publication year, publication type, and the geographical region of authors. In the qualitative analysis, we used open coding content analysis (Gao et al., 2012). In this technique, there were two phases; first, we read all papers to extract themes and, second, the themes were classified. Then the same process was repeated to refine the classification and synthesis.

Limitations

For this systematic literature review, we only considered:

• articles published between January 1, 2012 and July 12, 2019. (We note that there may have been conference papers presented before July 12, 2019 that were not published by the cutoff date for this study and that they were not included in our literature review.).
• five academic databases and Google Scholar.
• peer reviewed journal articles, conferences, and book sections.
• papers in which the recommender system for MOOCs is proposed, implemented, or discussed as a need, or in which different recommendation algorithms for MOOCs are compared.
• articles that were published in English. While searching Google Scholar and performing reference chaining, we found related articles in other languages, such as French. These other articles are not included.

The Google Scholar search returned more than 30,000 items (13 October 2019). These items included websites, blogs, videos, etc. However, we did not include these resources because they are subjective and usually not considered for peer review. We did, however, include existing literature reviews.

Results and Analysis

Descriptive/Quantitative Analysis

The initial Google Scholar search resulted in 424 research papers. After analyzing titles and abstracts, 124 papers were classified as relevant. After a detailed analysis of each of these papers, we considered only 89 to be related to the topic of recommender systems in MOOCs.

Table 1 contains the results of the searches from the academic databases. Springer Link showed 144 publications, of which 26 were related to our research. IEEE and Springer contained the highest number of related publications, but ERIC revealed no related research papers. Many of the unrelated papers were about recommender systems used in technology enhanced learning other than MOOCs.

Table 1

Distribution of Papers Found in Academic Databases

After searching the databases, we performed reference chaining and found another 10 related papers. As a result, we had 116 papers, which included 88 conference papers, 26 journal articles, and 2 book chapters. Both book chapters were published in 2019. Figure 1 shows yearly distribution of literature in these categories.

Figure 1. Yearly distribution of literature by type: journal article, conference paper, or book chapter. *2019 data includes research published only up to July 12, 2019.

There were no publications on recommender systems in MOOCs in 2012, but subsequently, a gradual increase in the number of publications per year is visible. The highest number of publications was in 2017. (Note that 2019 covers only 6 months.)

Table 2 shows groups of authors who have had more than one publication in this research area. Of 319 authors, we found 68 had at least two papers in this area, and the maximum number of papers from a single author was five.

Table 2

Groups of Authors Having More Than One Publication

Note: * Authors who have publications with more than one group of authors.

At this stage, we analyzed research links between authors and how they are grouped. Figure 2 shows the network of authors who have at least two papers in this area, and their links with other groups of authors.

Figure 2. Network diagram of authors who are linked with other groups of authors. Red nodes indicate authors who have publications with more than one group.

By observing the country of the first author, we determined that the majority of work (43%) is from Europe whereas 24% and 22% of research in this field was performed in Asia and the USA respectively. Ten percent of the research is from Africa, with 1% from Australia. In Asia, most of the research is from China. Figure 3 shows the distribution by country.

Figure 3. Distribution of work with respect to country/region of first author.

We classified the literature into four categories: need, design proposal, implementation, or other. These are defined as follows:

• Need: Papers that mainly focused on the importance of recommender systems in MOOCs.
• Design proposal: Papers in which the author has given an abstract proposal for a recommender system.
• Implementation: Research work in which authors designed an algorithm and implemented it on a dataset.
• Other: All other papers in which authors reviewed the current work, guidelines, or challenges.

Figure 4 shows trends in these categories between 2012 and 2019. Implementation was the main focus of research throughout this period, and from 2016 onwards, the number of published papers in this area rose rapidly. The reason for this rapid increase is that researchers not only implemented new techniques but also implemented their proposals from their 2014 and 2015 research work. Research on design proposals for recommender systems in MOOCs showed a gradual decrease after 2016. A similar pattern is evident in the need category.

Figure 4. Distribution of different types of research work 2012-2019. *2019 data includes research published only up to July 12, 2019.

In the implementation category, some authors also evaluated their work using metrics and baselines. Table 3 illustrates the number of implemented and evaluated recommender systems. Among all implemented systems, 42% were evaluated using different datasets and evaluation techniques. Most authors used datasets of edX and Coursera, but some also created their own datasets. For evaluation, most authors used receiver operating characteristic (ROC), recall and precision metrics, as well as accuracy metrics. The remaining 58% did not evaluate their proposed solutions and instead presented evaluation as future work.

Table 3

Number of Publications on Implemented and Evaluated Recommender Systems

Note: *2019 data includes research published only up to July 12, 2019.

Content/Qualitative Analysis

To carry out a comprehensive review of a topic, it is necessary to conduct an in-depth analysis through synthesis. In a systematic review, synthesis provides a bottom-line statement regarding any gaps and missing links through pooling and exploring the results (Fink, 2005). In this section, we highlight the main issues addressed and major contributions on recommender systems in MOOCs. We found that research could be broadly categorized into seven main themes.

• Thread recommender: Thread recommender involves thread/discussion, question recommendation, and question tag recommendations.
• Learning element recommender: Learning element recommender includes learning activities, suggestions on the study, video lectures, next page recommender, source, and learning path recommenders.
• Course recommender: Only involves course recommendation.
• Student performance recommender: Student performance recommender involves jobs, grades, student difficulty based, student dropout, work plan, and paid task recommenders.
• Peer recommender: Social interactions are a key factor in successful learning, and peer recommender involves systems that recommend related peers or fellow learners to interact with instead of recommending a learning resource or another class to follow. It uses demographics and progress made in a course for recommendations.
• MOOC provider/teacher recommender: This involves curriculum recommendations, news of MOOCs, and MOOC provider feedback.
• Others: This category involves improved and personalized MOOCs, adaptive content, and special user recommender systems.

We found that more than 60% of the literature is on course and learning element recommender systems for MOOCs. A possible reason for this is that universities or institutes that offer MOOCs do so to increase enrolment and throughput, and therefore, they recommend further courses to those already enrolled. Figure 5 shows the percentage distribution of research in different categories.

Figure 5. Distribution of work done on different types of recommender systems in MOOCs.

To analyze the type and trends of work found in the literature, we grouped the research work with respect to the area of MOOCs where the recommender system is applied. Table 4 shows a detailed categorization of different areas of MOOCs where recommender systems have been applied.

Table 4

Distribution of Work in Recommender System Categories

Table 5 shows research on the implementation or proposal of recommender systems in MOOCs. There are some papers in which authors have discussed the recommender systems in a generalized way, while in other papers they have provided guidelines or a literature review of existing work. We have classified these papers into four broad categories: preliminary study; literature review; challenges and effects of recommender systems in MOOCs; and design guidelines. A description of each category follows:

• Preliminary study: All research papers which discuss initial steps of the design of a recommender system in MOOCs. In these papers, the authors discuss steps and possible techniques for preprocessing of data.
• Literature review: We found two related literature reviews. However, these reviews discussed personalized MOOCs and not recommender systems.
• Challenges and effects of recommender systems in MOOCs: Papers in this category target the challenges of implementing a recommender system in MOOCs and the effects on MOOCs after introduction of a recommender system.
• Design guidelines: Papers in which authors have described guidelines to design a recommender system are in this category.

Table 5 shows the distribution of research work by year into these four categories.

Table 5

Yearly Distribution of Research Work Discussing Recommender Systems in MOOCs

Figure 6 shows the trend in the types of recommender systems researched over time. Until 2017, a gradual increase in research was evident. Initially, researchers focused on thread and course recommender systems, which then extended to peer, learning element, and student performance recommenders. By 2016, MOOC provider recommender systems were added to the research stream, and this trend continued in 2017. In 2018, most of the research was into course recommender systems, while no work was found on thread recommenders. Up until July 2019, course and learning element recommenders were the focus of research.

Figure 7 presents the number of research publications based on the different types of recommender systems applied to MOOCs. Overall, course recommender and learning element recommender systems are the most popular areas of research in the application of recommender systems.

Figure 6. Change over time in the types of recommender systems in MOOCs researched. *2019 data includes research published only up to July 12, 2019.

Figure 7. Distribution of research based on type of recommender system in MOOCs (2012 to July 2019).

Figure 8 shows the percentage of published work falling into each of the four broad classifications of research described in the Descriptive/Quantitative Analysis section of this paper. These are: (a) need, (b) design proposal, (c) implementation, and (d) other. The overall focus of research is the implementation of recommender systems in MOOCs. The category of other includes no implementation, and around 70% of this research is about proposed systems only. A possible reason for this could be that the work included in this category is meant only for a specific group of people.

Figure 8. Proportion of literature categorized with respect to type of recommendation and type of research.

Discussion

To the best of our knowledge, this is the first systematic literature review on the use of recommender systems in MOOCs from 2012 to July 12, 2019. Published work falls into three major categories: the need for recommender systems, proposed recommender systems, and implemented recommender systems in MOOCs. We classified the types of recommender systems into seven themes: course recommender, learning elements recommender, peer recommender, thread recommender, student performance recommender, MOOC provider/teachers’ recommender, and others. In this section, we discuss the types and trends of research carried out within each of these themes. We also identify gaps in the current literature which may be areas for future research.

Course Recommender Systems

The implementation of course recommender systems was a key focus of much of the research. This could be due to the availability of data and the interests of MOOC providers, because course recommender systems can help in improving enrolment and the learning experience. From 2013 to 2016, most of the work on the implementation of the recommender system for courses used collaborative and content-based filtering (Onah & Sinclair, 2015a; Piao & Breslin, 2016). Some researchers discussed the need for course recommender systems in MOOCs (Campos et al., 2018a; Fu et al., 2015; Ouertani & Alawadh, 2017). Campos et al. (2018b) implemented a course recommender system using knowledge reuse in ecosystems, and Hou et al. (2016) considered the context of the learner while performing recommendations.

After 2016, along with collaborative and content-based filtering for course recommendation (Boratto et al., 2019; He et al., 2017; Hou et al., 2018; Rabahallah et al., 2018) researchers started to use neural networks, pattern mining, and deep learning for preprocessing of data and recommendations (Agrebi et al., 2019; Jain & Anika, 2018; Jing & Tang, 2017; H. Zhang et al., 2019). We also observed the introduction of association rule mining and hybrid algorithms (Xiao et al., 2018; Y. Li & Li, 2017; Pang et al., 2018) Gope and Jain (2017) used the learning style of the student to recommend courses. Their prototype was based on a learning system model and worked exclusively with edX courses. It scanned every course to identify learning objects and then made recommendations.

Learning Elements Recommender Systems

More than half the work we reviewed focused on the implementation of recommender systems and the work concerns content-based filtering or hybrid algorithms (Cooper et al., 2018a; Itmazi & Hijazi, 2015; Zhao et al., 2018). Researchers have designed recommender systems for different types of learning elements, such as video clips, next page, and additional resources helpful to the learner. Kopeinik (2016) compared existing algorithms that provide a recommendation of learning resources and tags to annotate these resources. Onah and Sinclair (2015b) recommended a suitable path to learners by considering scores on concept-based quizzes. A low score indicated that the learner needed more resources related to a concept, and so this system would recommend instructional material according to a learner’s profile.

While preprocessing the dataset, we observed an increase in the use of neural networks, pattern mining, and machine learning in more recent years (Cooper et al., 2018a; Hajri et al., 2019; Xiao et al., 2018; Pardos et al., 2017; H. Zhang et al., 2019). Cooper and his colleagues researched the recommendation of video lectures to learners by analyzing the content of videos. They designed a user-friendly interface that suggested related videos while learners were watching another video (Bhatt et al., 2018; Cooper et al., 2018a, 2018b; Zhao et al., 2018).

Peer Recommender Systems

Another group of authors completed detailed research in the field of peer recommender systems. They investigated the effect of a peer recommender on overall student performance (Labarthe, Bachelet, et al., 2016; Labarthe, Bouchet, et al., 2016). They compared the results of different peer recommender systems in terms of student engagement (Bouchet, Labarthe, Yacef, et al., 2017). These researchers also attempted to identify the reasons for peer communication usage in MOOCs through conducting surveys of peer recommender user and found that most students express their emotions about course work through communicating with peers (Bouchet, Labarthe, Bachelet, et al., 2017). Reciprocal scores were used by some authors to find and recommend suitable peers for learners (Potts et al., 2018; Prabhakar et al., 2017).

Thread Recommender Systems

Most research on thread recommender systems focused on implementation. We found only two papers related to the algorithm proposals (Cohen et al., 2013; Sunar et al., 2015a). Implementation work was mostly performed by using matrix factorization, collaborative filtering, and content-based filtering for recommender systems (Garg & Tiwari, 2016; Yang, Piergallini, et al., 2014). Mi and Faltings (2016a, 2016b, 2017) used a context tree for online thread recommendations. Yang et al. (2014) designed a recommender system for threads in a forum that recommends questions for students to answer based on their expertise. They also managed learner workload by defining a threshold on the number of questions recommended to each learner. After proposing an initial algorithm, Yang, Shang, et al. (2014) then improved this algorithm by adding sub-modularity to make it computationally less expensive. Agrawal et al. (2015) designed a recommender system that recommends video clips from lectures based on questions asked in forums.

Student Performance Recommender Systems

To increase student performance and engagement, some researchers (Alario-Hoyos et al., 2014; Luacesa et al., 2017; M. Zhang et al., 2019) presented work focused on the design of recommender systems only. Chen et al. (2016) first proposed and then designed and implemented a system that recommends to learners course-related paid tasks from freelancing websites such as Upwork or Witmart (Chen et al., 2018; Chen et al., 2017). The main idea was to make it possible for learners to earn money while using MOOCs.

MOOCs Provider/Teacher’s Recommender Systems

Only two studies paid attention to designing recommender systems for MOOC providers and teachers. Holotescu (2016) designed a chatbot for MOOCs that works with Facebook and provides news about MOOCs that can deliver the latest news about MOOCs to teachers and providers. Zhou et al. (2015) designed an Android application to improve a course, and this application takes feedback from students during the course and makes suggestions to the teacher based on this feedback.

Conclusion

The use of recommender systems in MOOCs presents exciting opportunities to increase the popularity of MOOCs and improve the learners’ experience. Research to date has mostly focused on the implementation of recommender systems in MOOCs, particularly course recommender systems which was the most prolific research line throughout the period.

From 2012 to 2016, researchers modified existing recommender systems that were designed for e-commerce, music, videos, or books, to make them appropriate for use in MOOCs; however, from 2017 onwards, researchers started to apply neural networks, deep learning and data mining techniques in data preprocessing to apply recommender systems in MOOCs. Researchers focused on learners and strived to exploit their learning habits.

Future Directions

Although a considerable number of recommender systems in MOOCs have been proposed and implemented, only a few authors have discussed the time and space complexity of their proposed and implemented algorithms (Ahera & Lobo, 2013; Hou et al., 2018; Mi & Faltings, 2016a, 2017; H. Zhang et al., 2018). MOOCs produce a large amount of data that can be used for recommender systems and researchers should focus on systems that scale well with the increase in data and have linear time and space complexity. In evaluating their solutions, authors have ignored the training and recommendation time that their recommender system is taking.

One reason for overlooking this aspect of their algorithms could be the batch/offline nature of proposed algorithms. Batch/offline algorithms use existing datasets for training and recommendations. For this purpose, algorithms require memory space and time, the amount of which depends upon the type of dataset. Online recommender systems consider only the current context of the user while computing recommendations. In MOOCs, the current context of the user is an important factor, and researchers should put more focus on this. We found only one such work, Mi and Faltings (2016b, 2017), that addressed online recommender systems.

There is also a lack of standardized datasets available for the evaluation of recommender systems in MOOCs. Researchers have mostly used publicly available datasets of Coursera, edX, and, in some cases, datasets from their own institutes to evaluate recommender systems (Aryal et al., 2019; Dai et al., 2017; Kardan et al., 2017; Mi & Faltings, 2016a; Shaptala et al., 2017; Yang, Adamson, et al., 2014; Yang, Piergallini, et al., 2014). Other authors have created datasets (Onah & Sinclair, 2015a; He et al., 2017; Iniesto & Rodrigo, 2019; Zhou et al., 2015). A lack of standardized datasets can be a significant limitation when benchmarking or comparing algorithms or techniques of different researchers. Furthermore, most researchers used datasets from computer science-related courses for testing their recommender systems (Aryal et al., 2019; Bhatt et al., 2018; M. Zhang et al., 2019; Zhou et al., 2015) which limits the research to one academic field.

Scant attention has been paid to designing recommender systems for MOOC providers and teachers. Such systems can help providers and teachers in planning course materials, delivery styles, and the content of the MOOC. Recommender systems could also help providers decide which courses should become MOOCs. The effect of recommender systems on student engagement and completion rates is another useful topic to pursue.

We also observed that previous research has overlooked different types of MOOCs, such as cMOOCs, xMOOCs, and sMOOCs, and has not considered the characteristics of types of MOOCs while designing recommender systems. In future, recommender systems could cater to different types of MOOCs.

References

Adamopoulos, P. (2014a, February). Novel perspectives in collaborative filtering recommender systems. In C.-W. Chung (Chair), 23rd International Conference on World Wide Web (WWW) PhD Symposium. Retrieved from https://pdfs.semanticscholar.org/3958/7fbe80b6e12bfd46209569be9f4c6698892a.pdf

Adamopoulos, P. (2014b). On discovering non-obvious recommendations: Using unexpectedness and neighborhood selection methods in collaborative filtering systems. In B. Carterette, F. Diaz, C. Castillo, & D. Metzler (Eds.), WSDM ’14: Proceedings of the 7th ACM International Conference on Web Search and Data Mining (pp. 655-660). doi: 10.1145/2556195.2556204

Adham, R.S., & Lundqvist, K.O. (2015). MOOCs as a method of distance education in the Arab world: A review paper. European Journal of Open, Distance and E-Learning EURODL, 18(1), 123-138. doi: 10.1515/eurodl-2015-0009

Agrawal, A., Venkatraman, J., Leonard, S., & Paepcke, A. (2015). YouEDU: Addressing confusion in MOOC discussion forums by recommending instructional video clips. In O.C. Santos, J.G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros … M. Desmarais (Eds.), Proceedings of the 8th International Conference on Educational Data Mining (pp. 297-304). Retrieved from https://www.educationaldatamining.org/EDM2015/proceedings/edm2015_proceedings.pdf

Agrebi, M., Sendi, M., & Abed, M. (2019). Deep reinforcement learning for personalized recommendation of distance learning. In Á. Rocha, H. Adeli, L. Reis, & S. Costanzo (Eds.), Advances in Intelligent Systems and Computing, Volume 931. New Knowledge in Information Systems and Technologies. WorldCIST’19 2019 (pp. 597-606). doi: 10.1007/978-3-030-16184-2_57

Ahera, S. B., & Lobo, L. M. R. J. (2013). Combination of machine learning algorithms for recommendation of courses in E-Learning System based on historical data. Knowledge-Based Systems, 51, 1-14. doi: 10.1016/j.knosys.2013.04.015

Alario-Hoyos, C., Leony, D., Estévez-Ayres, I., Pérez-Sanagustín, M., Gutiérrez-Rojas, I., & Kloos, C. D. (2014). Adaptive planner for facilitating the management of tasks in MOOCs. In L. B. Martínez, R. H. Rizzardini, & J. R. H. González (Eds.), Proceedings of the V Congreso Internacional sobre Calidad y Accesibilidad de la Formación Virtual (pp. 517-522). Retrieved from https://pdfs.semanticscholar.org/d007/7611bc09978ce61f489bae9b65974bfbfbea.pdf

Apaza, R. G., Cervantes, E. V., Quispe, L. C., & Luna, J. e. O. (2014). Online courses recommendation based on LDA. In J. A. Lossio-Ventura & H. Alatrista-Salas (Eds.), Proceedings of the 1st Symposium on Information Management and Big Data—SIMBig 2014, Cusco, Peru (pp. 42-48). Retrieved from http://ceur-ws.org/Vol-1318/paper5.pdf

Ardchir, S., Talhaoui, M. A., & Azzouazi, M. (2017). Towards an adaptive learning framework for MOOCs. In A. Esma., R. Umar, & W. Michael. (Eds.), E-Technologies: Embracing the Internet of Things. MCETECH 2017. Lecture Notes in Business Information Processing, vol 289 (pp. 236-251). Springer, Cham. https://doi.org/10.1007/978-3-319-59041-7_15

Aryal, S., Porawagama, A. S., Hasith, M. G. S., Thorade, S. C., Kodagoda, N., & Suriyawansa, K. (2019). MoocRec: Learning styles-oriented MOOC recommender and search engine. In Alaa K. Ashmawy (Chair), 2019 IEEE Global Engineering Education Conference (EDUCOM) (pp. 1167-1172). doi: 10.1109/EDUCON.2019.8725079

Assami, S., Daoudi, N., & Ajhoun, R. (2018). Personalization criteria for enhancing learner engagement in MOOC platforms. In C. -S. González, M. Castro, & M. -L. Nistal (Chairs), 2018 IEEE Global Engineering Education Conference (EDUCON) (pp. 1265-1272). doi: 10.1109/EDUCON.2018.8363375

Assami S., Daoudi N., & Ajhoun R. (2019) Ontology-based modeling for a personalized MOOC recommender system. In Á. Rocha & M. Serrhini (Eds.), Information Systems and Technologies to Support Learning. EMENA-ISTL 2018. Smart Innovation, Systems and Technologies, vol. 111 (pp. 21-28). doi: 10.1007/978-3-030-03577-8_3

Babinec, P., & Srba, I. (2017). Education-specific tag recommendation in CQA systems. In M. Bielikova & E. Herder (Chairs), UMAP ’17: Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization (pp. 281-286). doi: https://doi.org/10.1145/3099023.3099081

Bassi, R., Daradoumis, T., Xhafa, F., Caballé, S., & Sula, A. (2014). Software agents in large scale open e-learning: A critical component for the future of massive online courses (MOOCs). In V. Loia & F. Xhafa (Chairs), Proceedings of the 2014 International Conference on Intelligent Networking and Collaborative Systems (INCoS) (pp. 184-188). doi: 10.1109/INCoS.2014.15

Belarbi, N., Chafiq, N., Talbi, M., Namir, A., & Benlahmar, E. (2019a). A recommender system for videos suggestion in a SPOC: A proposed personalized learning method. In Y. Farhaoui & L. Moussaid (Eds.), Big Data and Smart Digital Environment. ICBDSDE 2018. Studies in Big Data, vol 53. (pp. 92-101). Springer, Cham. https://doi.org/10.1007/978-3-030-12048-1_12

Belarbi, N., Chafiq, N., Talbi, M., Namir, A., & Benlahmar, E. (2019b). User profiling in a SPOC: A method based on User Video Clickstream Analysis. International Journal of Emerging Technologies in Learning (iJET), 14(1), 110-124. doi: 10.3991/ijet.v14i01.9091

Bhatt, C., Cooper, M., & Zhao, J. (2018) SeqSense: Video recommendation using topic sequence mining. In K. Schoeffmann et al. (Eds.), Lecture Notes in Computer Science: Vol. 10705. MultiMedia Modeling. MMM 2018 (pp. 252-263). doi: 10.1007/978-3-319-73600-6_22

Boratto, L., Fenu, G., & Marras, M. (2019) The effect of algorithmic bias on recommender systems for massive open online courses. In L. Azzopardi, B. Stein, N. Fuhr, P. Mayr, C. Hauff, & D. Hiemstra (Eds.), Lecture Notes in Computer Science: Vol. 1143. Advances in Information Retrieval. ECIR 2019 (pp. 457-472). doi: 10.1007/978-3-030-15712-8_30

Bouchet, F., Labarthe, H., Bachelet, R., & Yacef, K. (2017) Who wants to chat on a MOOC? Lessons from a peer recommender system. In C. Delgado Kloos, P. Jermann, M. Pérez-Sanagustín, D. Seaton, & S. White (Eds.), Lecture Notes in Computer Science: Vol. 10254. Digital Education: Out to the World and Back to the Campus. EMOOCs 2017 (pp. 150-159). doi: 10.1007/978-3-319-59044-8_17

Bouchet, F., Labarthe, H., Yacef, K., & Bachelet, R. (2017). Comparing peer recommendation strategies in a MOOC. In M. Bielikova, & E. Herder (Chairs), Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP '17). Association for Computing Machinery (pp.129-134). doi: https://doi.org/10.1145/3099023.3099036

Bousbahi, F., & Chorfi, H. (2015). MOOC-Rec: A case based recommender system for MOOCs. Procedia - Social and Behavioral Sciences, 195, 1813-1822. doi: 10.1016/j.sbspro.2015.06.395

Brigui-Chtioui, I., Caillou, P., & Negre, E. (2017). Intelligent digital learning: Agent-based recommender system. In J.Li. & L. Huang (Chairs), Proceedings of the 9th International Conference on Machine Learning and Computing Association for Computing Machinery (pp. 71-76). doi: https://doi.org/10.1145/3055635.3056592

Burgos, D., & Corbí, A. (2014, October). A recommendation model on personalised learning to improve the user’s performance and interaction in MOOCs and OERs. In IITE 2014: New Challenges for Pedagogy and Quality Education: MOOCs, Clouds and Mobiles. Symposium conducted at the meeting of UNESCO Institute for Information Technologies in Education, Moscow, Russia. Retrieved from https://bit.ly/33Wakle

Campos, R., dos Santos, R. P., & Oliveira, J. (2018a, July). Recommendation systems for knowledge reuse management in MOOCs ecosystems. Paper presented at the XI Workshop de Teses e Dissertações em Sistemas de Informação (WTDSI), Caxias do Sul, RS, Brazil.

Campos, R., dos Santos, R. P., & Oliveira, J. (2018b). Web-based recommendation system architecture for knowledge reuse in MOOCs ecosystems. In 2018 IEEE International Conference on Information Reuse and Integration (IRI) (pp. 193-200). doi: 10.1109/IRI.2018.00036

Chanaa, A., & el Faddouli, N. E. (2019). Context-aware factorization machine for recommendation in massive open online courses (MOOCs). In R. A. Eduardo De Barros, S.B. Dosse, & H. El Fadili (Chairs), 2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems (WITS) (pp. 1-6). doi: 10.1109/WITS.2019.8723670

Chen, G., Davis, D., Krause, M., Aivaloglou, E., Hauff, C., & Houben, G.-J. (2016). Can learners be earners? Investigating a design to enable MOOC learners to apply their skills and earn money in an online market place. IEEE Transactions on Learning Technologies (p. 1). Retrieved from https://angusglchen.github.io/documents/TLT16_Guanliang_Can.pdf

Chen, G., Davis, D., Krause, M., Aivaloglou, E., Hauff, C., & Houben, G.-J. (2018). From learners to earners: Enabling MOOC learners to apply their skills and earn money in an online market place. IEEE Transactions on Learning Technologies, 11(2), 264-274. doi: 10.1109/TLT.2016.2614302

Chen, G., Davis, D., Krause, M., Hauff, C., & Houben, G.-J. (2017). Buying time: Enabling learners to become earners with a real-world paid task recommender system. In A. Wise, P. H. Winne & G. Lynch (Chairs), Proceedings of the Seventh International Learning Analytics and Knowledge Conference (LAK '17). Association for Computing Machinery (pp 578-579). doi: https://doi.org/10.1145/3027385.3029469

Cohen, R., Sardana, N., Rahim, K., Lam, D. Y., Li, M., Maccarthy, O.,... Guo, G. (2013). Recommending messages to users in social networks: A cross-site study. In M. A. Wani, G. Tecuci, M. Boicu, M. Kubat, T. M. Khoshgoftaar, & Naeem (Jim) Seliya(Eds.), 2013 12th International Conference on Machine Learning and Applications (pp. 445-450). doi: 10.1109/ICMLA.2013.160

Cooper, M., Zhao, J., Bhatt, C., & Shamma, D. A. (2018a). MOOCex: Exploring educational video via recommendation. In K. Aizawa, M. Lew, & S. Satoh (Chairs), ICMR ’18: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval (ICMR '18). Association for Computing Machinery (pp. 521-524). doi: https://doi.org/10.1145/3206025.3206087

Cooper, M., Zhao, J., Bhatt, C., & Shamma, D. A. (2018b). Using recommendation to explore educational video. In K. Aizawa, M. Lew, & S. Satoh (Chairs), ACM International Conference on Multimedia Retrieval (ICMR). Retrieved from https://www.fxpal.com/publications/using-recommendation-to-explore-educational-video.pdf

Corbi, A., & Burgos, D. (2014). Review of current student-monitoring techniques used in e-learning-focused recommender systems and learning analytics. The Experience API & LIME model case study. International Journal of Interactive Multimedia and Artificial Intelligence, 2(7), 44-52. doi: 10.9781/ijimai.2014.276

Dai, K., Vilas, A. F., & Redondo, R. P. D. (2017). A new MOOCs’ recommendation framework based on LinkedIn data. In E. Popescu et al. (Eds.), Lecture Notes in Educational Technology: Innovations in Smart Learning (pp. 19-22). doi: 10.1007/978-981-10-2419-1_3

Dakhel, G., & Mahdavi, M. (2013). Providing an effective collaborative filtering algorithm based on distance measures and neighbors’ voting. International Journal of Computer Information Systems and Industrial Management Applications, 5, 524-531. Retrieved from http://www.mirlabs.org/ijcisim/regular_papers_2013/Paper129.pdf

Daradoumis, T., Bassi, R., Xhafa, F., & Caballé, S. (2013, October). A review on massive e-learning (MOOC) design, delivery and assessment. In F. Xhafa, L. Barolli, D. Nace, A. Bui & S. Venticinque (Eds.), 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (pp. 208-213), doi: 10.1109/3PGCIC.2013.37

De Medio, C., Gasparetti, F., Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F., & Temperini, M. (2017). Towards a characterization of educational material: An analysis of Coursera resources. In T.T. Wu, R. Gennari, Y.M. Huang, H. Xie, & Y. Cao (Eds.), Lecture Notes in Computer Science: Vol. 10108. Emerging Technologies for Education. SETE 2016 (pp. 547-557). doi: 10.1007/978-3-319-52836-6_58

dos Santos, H. L., Cechinel, C., Araujo, R. M., & Sicilia, M.-A. (2015). Clustering learning objects for improving their recommendation via collaborative filtering algorithms. In E. Garoufallou, R. Hartley, & P. Gaitanou (Eds.), Communications in Computer and Information Science. Volume 544. MTSR 2015. Metadata and Semantics Research. doi: 10.1007/978-3-319-24129-6_16

Downes, S. (2008). Places to go: Connectivism and connective knowledge. Innovate: Journal of Online Education, 5(1). Retrieved from https://nsuworks.nova.edu/innovate/vol5/iss1/6

Elbadrawy, A., Polyzou, A., Ren, Z., Sweeney, M., Karypis, G., & Rangwala, H. (2016). Predicting student performance using personalized analytics. Computer, 49(4), 61-69. doi: 10.1109/MC.2016.119

Fazeli, S., Rajabi, E., Lezcano, L., Drachsler, H., & Sloep, P. (2016). Supporting users of open online courses with recommendations: An algorithmic study. In Kinshuk, R. Huang, N.-S. Chen, & P. Resta (Chair), 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT) Volume: 1 (pp. 423-427). doi: 10.1109/ICALT.2016.119

Fink, A. (2005). Conducting research literature reviews: From the Internet to paper. Thousand Oaks, California: Sage Publications.

Fu, D., Liu, Q., Zhang, S., & Wang, J. (2015). The undergraduate-oriented framework of MOOCs recommender system. In H. Yang, & L.-F. Kwok (Chairs), 2015 International Symposium on Educational Technology (ISET) (pp. 115-119). doi: 10.1109/ISET.2015.31

Gao, F., Luo, T., & Zhang, K. (2012). Tweeting for learning: A critical analysis of research on microblogging in education published in 2008-2011. British Journal of Educational Technology, 43(5), 783-801. doi: 10.1111/j.1467-8535.2012.01357.x

Garg, V., & Tiwari, R. (2016). Hybrid massive open online course (MOOC) recommendation system using machine learning. In S. G. Malla (Chair), International Conference on Recent Trends in Engineering, Science & Technology (ICRTEST 2016) (pp. 1-5). doi: 10.1049/cp.2016.1479

Gómez-Berbís, J. M., & Lagares-Lemos, Á. (2016). ADL-MOOC: Adaptive learning through big data analytics and data mining algorithms for MOOCs. In R. Valencia-García, K. Lagos-Ortiz, G. Alcaraz-Mármol, J. del Cioppo, & N. Vera-Lucio (Eds.), Technologies and innovation. CITI 2016. Communications in computer and information science, vol 658. (pp. 269-280). doi: https://doi.org/10.1007/978-3-319-48024-4_21

Gope, J., & Jain, S. K. (2017). A learning styles based recommender system prototype for edX courses. In M. M. Kodabagi, S. S. Manvi, V. R. Hulipalled & S.K. Niranjan& (Eds.), 2017 International Conference on Smart Technologies for Smart Nation (SmartTechCon) (pp. 414-419). doi: 10.1109/SmartTechCon.2017.8358407

Hajri, H., Bourda, Y., & Popineau, F. (2017). MORS: A system for recommending OERs in a MOOC. In M. Chang, N-S Chen, Kinshuk, D. G. Sampson, & R. Vasiu (Eds.), IEEE 17th International Conference on Advanced Learning Technologies (ICALT) (pp. 50-52). doi: 10.1109/ICALT.2017.89

Hajri, H., Bourda, Y., & Popineau, F. (2018). A system to recommend open educational resources during an online course. In B. McLaren, R. Reilly, S. Zvacek, & J. Uhomoibhi (Eds.), Proceedings of the 10th International Conference on Computer Supported Education (CDSEU 2018) Vol. 1 (pp 99-109). Retrieved from https://pdfs.semanticscholar.org/24d8/1a0d8874bfb075b90afff58ff4e247ddea85.pdf

Hajri, H., Bourda, Y., & Popineau, F. (2019). Personalized recommendation of open educational resources in MOOCs. In: B. McLaren, R. Reilly, S. Zvacek & J. Uhomoibhi (Eds.), Computer supported education. CSEDU 2018. Communications in computer and information science, vol 1022 (pp 166-190). doi: https://doi.org/10.1007/978-3-030-21151-6_9

Harrathi, M., Touzani, N., & Braham, R. (2017). A hybrid knowlegde-based approach for recommending massive learning activities. In Y. Jararweh & K. Ghedira (Chairs), 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA) (pp. 49-54), doi: 10.1109/AICCSA.2017.150

Harrathi, M., Touzani, N., & Braham, R. (2018). Toward a personalized recommender system for learning activities in the context of MOOCs. In G. De Pietro, L. Gallo, R. Howlett & L. Jain (Eds.), Intelligent Interactive Multimedia Systems and Services 2017. KES-IIMSS-18 2018. Smart Innovation, Systems and Technologies, vol 76 (pp. 575-583). doi: https://doi.org/10.1007/978-3-319-59480-4_57

He, X., Liu, P., & Zhang, W. (2017). Design and implementation of a unified MOOC recommendation system for social work major: Experiences and lessons. In J. Li, B.Chapman, F. Palmieri & H. Mouftah (Chairs), 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC) (pp. 219-223), doi: 10.1109/CSE-EUC.2017.46

Holotescu, C. (2016). MOOCBuddy: A chatbot for personalized learning with MOOCs. In A. Iftene & J. Vanderdonckt (Eds.), Proceedings of the 13th International Conference on Human-Computer Interaction RoCHI’2016 (pp. 91-94). Retrieved from https://pdfs.semanticscholar.org/832c/8de6424644765f98094c0127981120fc66e5.pdf

Hou, Y., Zhou, P., Wang, T., Yu, L., Hu, Y., & Wu, D. (2016, October 11). Context-aware online learning for course recommendation of MOOC big data. Retrieved from arXiv database: 1610.03147

Hou, Y., Zhou, P., Xu, J., & Wu, D. O. (2018, April). Course recommendation of MOOC with big data support: A contextual online learning approach. In D. Rawat (Chair), IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) (pp. 106-111). doi: 10.1109/INFCOMW.2018.8406936

Hussain, M., Zhu, W., Zhang, W., Abidi, S. M. R., & Ali, S. (2018). Using machine learning to predict student difficulties from learning session data. Artificial Intelligence Review, 52, 381-407. doi: 10.1007/s10462-018-9620-8

Iniesto, F., & Rodrigo, C. (2015). Accessible user profile modeling for academic services based on MOOCs. In Interacción ’15: Proceedings of the XVI International Conference on Human Computer Interaction (pp. 1-2). doi: 10.1145/2829875.2829922

Iniesto, F., & Rodrigo, C. (2016). A preliminary study for developing accessible MOOC services. Journal of Accessibility and Design for All, 6(2), 125-149. doi: 10.17411/jacces.v6i2.117

Iniesto, F., & Rodrigo, C. (2018). YourMOOC4all: A MOOCs inclusive design and useful feedback research project. In M. Castro & E. Tovar (Chairs), Learning with MOOCs (LWMOOCs) (pp. 147-150). doi: 10.1109/LWMOOCS.2018.8534644

Iniesto, F., & Rodrigo, C. (2019). YourMOOC4all: A recommender system for MOOCs based on collaborative filtering implementing UDL. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Lecture Notes in Computer Science: Vol. 11722. Transforming Learning with Meaningful Technologies. EC-TEL 2019 (pp. 746-750). doi: 10.1007/978-3-030-29736-7_80

Itmazi, J. A., & Hijazi, H. W. (2015, May). A suggested algorithm of recommender system to recommend crawled-Web open educational resources to course management system. In A. Al-Dahoud (Chair), CIT 2015 The 7th International Conference on Information Technology (pp. 330-337). doi: 10.15849/icit.2015.0050

Jain, H., & Anika. (2018). Applying data mining techniques for generating MOOCs recommendations on the basis of learners online activity. In D. Garg (Ed.), 2018 IEEE 6th International Conference on MOOCs, Innovation and Technology in Education (MITE) (pp. 6-13). doi: 10.1109/MITE.2018.8747056

Jing, X., & Tang, J. (2017). Guess you like: Course recommendation in MOOCs. In A. Sheth (Chair), WI ’17: Proceedings of the International Conference on Web Intelligence (pp. 783-789). doi: 10.1145/3106426.3106478

Jo, Y., Tomar, G., Ferschke, O., Rosé, C. P., & Gašević, D. (2016). Expediting support for social learning with behavior modeling. In T. Barnes, M. Chi & M. Feng (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (pp. 400-405). Retrieved from https://arxiv.org/pdf/1605.02836v3.pdf

Kardan, A. A., Narimani, A., & Ataiefar, F. (2017). A hybrid approach for thread recommendation in MOOC forums. International Journal of Computer and Systems Engineering, 11(10), 2360-2366. Retrieved from https://pdfs.semanticscholar.org/43f1/cc3a373bb825aa49a7b13c68f305bf6e5f4e.pdf

Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in massive open online courses. Computers and Education, 104(C), 18-33. doi: 10.1016/j.compedu.2016.10.001

Kopeinik, S., Kowald, D., & Lex, E. (2016). Which algorithms suit which learning environments? A comparative study of recommender systems in TEL. In: K. Verbert, M. Sharples, & T. Klobučar (Eds.), Adaptive and Adaptable Learning. EC-TEL 2016. Lecture Notes in Computer Science, vol 9891. (pp. 124-138). Springer, Cham. doi: https://doi.org/10.1007/978-3-319-45153-4_10

Labarthe, H., Bachelet, R., Bouchet, F., & Yacef, K. (2016). Increasing MOOC completion rates through social interactions: A recommendation system. In M. Khalil, M. Ebner, M. Kopp, A. Lorenz, & M. Kalz (Eds.), Proceedings of the European Stakeholder Summit on Experiences and Best Practices in and Around MOOCs (EMOOCS 2016) (pp. 471-480). Retrieved from https://www.researchgate.net/publication/293884848_Proceedings_of_the_European_Stakeholder_Summit_on_experiences_and_best_practices_in_and_around_MOOCs_EMOOCS_2016

Labarthe, H., Bouchet, F., Bachelet, R., & Yacef, K. (2016). Does a peer recommender foster students’ engagement in MOOCs? In T. Barnes, M. Chi, & M. Feng (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (pp. 418-423). Retreived from http://www.educationaldatamining.org/EDM2016/proceedings/paper\_171.pdf

Lan, A. S., Spencer, J. C., Chen, Z., Brinton, C. G., & Chiang, M. (2019). Personalized thread recommendation for MOOC discussion forums. In: M. Berlingerio, F. Bonchi, T. Gärtner, N. Hurley & G. Ifrim (Eds.), Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018. Lecture Notes in Computer Science, vol 11052. (pp 725-740). Springer, Cham. https://doi.org/10.1007/978-3-030-10928-8_43

Li, X., Wang, T., Wang, H., & Tang, J. (2018). Understanding user interests acquisition in personalized online course recommendation. In L. Hou U & H. Xie (Eds.), Web and Big Data. APWeb-WAIM 2018. Lecture Notes in Computer Science, vol 11268 (pp 230-242). Springer, Cham. https://doi.org/10.1007/978-3-030-01298-4_20

Li, Y., & Li, H. (2017). MOOC-FRS: A new fusion recommender system for MOOCs. In Y.B Zhu (Chair), 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) (pp. 1481-1488). doi: 10.1109/IAEAC.2017.8054260

Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distributed Learning, 14(3), 202-227. https://doi.org/10.19173/irrodl.v14i3.1455

Lops, P., de Gemmis, M., & Semeraro, G. (2011). Content-based recommender systems: State of the art and trends. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), Recommender Systems Handbook (pp. 73-105). Boston, MA: Springer US.

Luacesa, O., Díeza, J., Alonso-Betanzosb, A., Troncosoc, A., & Bahamondea, A. (2017). Content-based methods in peer assessment of open-response questions to grade students as authors and as graders. Knowledge-Based Systems, 117, 79-87. doi: 10.1016/j.knosys.2016.06.024

Macina, J., Srba, I., Williams, J. J., & Bielikova, M. (2017). Educational question routing in online student communities. In P. Cremonesi & F. Ricci (Chair), RecSys ’17: Proceedings of the Eleventh ACM Conference on Recommender Systems (pp. 47-55). doi: https://dx.doi.org/10.1145/3109859.3109886

Manouselis, N., Drachsler, H., Verbert, K., & Duval, E. (2013). Recommender systems for learning. New York, USA: Springer. doi: 10.1007/978-1-4614-4361-2

Marchal, F., Castagnos, S., & Boyer, A. (2016). A first step toward recommendations based on the memory of users. In A. Esposito (Chair), 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI) (pp. 54-61). doi: 10.1109/ICTAI.2016.16

Margolis, A., López-Arredondo, A., García, S., Rubido, N., Caminada, C., González, D., & Tansini, L. (2019). Social learning in large online audiences of health professionals: Improving dialogue with automated tools [Version 2]. MedEdPublish, 8. doi: 10.15694/mep.2019.000055.2

Mawas, N. E., Gilliot, J.-M., Garlatti, S., Euler, R., & Pascual, S. (2018). Towards personalized content in massive open online courses. In,J. Uhomoibhi. (Chair). Proceedings of the 10th International Conference on Computer Supported Education—Volume 1: CSEDU (pp. 331-339). doi: 10.5220/0006816703310339

Mi, F., & Faltings, B. (2016a). Adapting to drifting preferences in recommendation. Paper presented at the meeting of Neural Information Processing Systems Foundation (NIPS 2016), Barcelona, Spain.

Mi, F., & Faltings, B. (2016b). Adaptive sequential recommendation using context trees. In S. Kambhampati. (Ed.), Twenty-Fifth International Joint Conference on Artificial Intelligence (pp. 4018-4019). Retireved from https://www.ijcai.org/Proceedings/16/Papers/583.pdf

Mi, F., & Faltings, B. (2017). Adaptive sequential recommendation for discussion forums on MOOCs using context trees. In X. Hu, & T. Barnes (Chairs), Proceedings of the 10th International Conference on Educational Data Mining (pp. 24-31). Retireved from https://pdfs.semanticscholar.org/c2c5/284e50e510e9c901db995a15e10d867df975.pdf?_ga=2.189311659.1005401281.1599166549-1010863648.1562893566

Ng, J., Ruta, D., Al-Rubaie, A., Wang, D., Powell, L., Hirsch, B.... Al-Dhanhani, A. (2014). Smart learning for the next generation education environment. In V. Callaghan, & L. Shen (Chairs), 2014 International Conference on Intelligent Environments (pp. 333-340). doi: 10.1109/IE.2014.73

Niu, K., Niu, Z., Su, Y., Wang, C., Lu, H., & Guan, J. (2015). A coupled user clustering algorithm based on mixed data for Web-based learning systems. Mathematical Problems in Engineering, 2015, 1-14. doi: 10.1155/2015/747628

Onah, D. F. O., & Sinclair, J. E. (2015a). Collaborative filtering recommendation system: A framework in massive open online courses. In L. Gómez Chova, A. López Martínez, & I. Candel Torres (Eds.). INTED 2015: Proceedings of the 9th International Technology, Education and Development Conference (pp. 1249-1258). doi: 10.13140/RG.2.1.5023.4409

Onah, D. F. O., & Sinclair, J. E. (2015b). Massive open online courses: An adaptive learning framework. In INTED 2015: Proceedings of the 9th International Technology, Education and Development Conference (pp. 1258-1266). doi: 10.13140/RG.2.1.4237.0083

Ouertani, H. C., & Alawadh, M. M. (2017). MOOCs recommender system: A recommender system for the massive open online courses. In E. Popescu, Kinshuk, M. K. Khribi, R. Huang, M. Jemni, N-S. Chen, & D.G. Sampson (Eds), Innovations in smart learning. lecture notes in educational technology (pp 139-143). doi: https://doi.org/10.1007/978-981-10-2419-1_20

Pang, Y., Liao, C., Tan, W., Wu, Y., & Zhou, C. (2018). Recommendation for MOOC with learner neighbors and learning series. In H. Hacid, W. Cellary, H. Wang, HY Paik, & R. Zhou (Eds.), Web information systems engineering - WISE 2018. Lecture notes in computer science, vol 11234 (pp. 379-394). doi: https://doi.org/10.1007/978-3-030-02925-8_27

Pappano, L. (2012, November 4). The year of the MOOC. The New York Times. Retrieved from https://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html

Paquette, G., Mariño, O., Rogozan, D., & Léonard, M. (2015). Competency-based personalization for massive online learning. Smart Learning Environments, 2(4). doi: 10.1186/s40561-015-0013-z

Pardos, Z. A., Tang, S., Davis, D., & Le, C. V. (2017). Enabling real-time adaptivity in MOOCs with a personalized next-step recommendation framework. In C. Urrea (Chair), L@S ’17: Proceedings of the fourth (2017) ACM conference on learning @ scale (pp. 23-32). doi: http://dx.doi.org/10.1145/3051457.3051471

Piao, G., & Breslin, J. G. (2016). Analyzing MOOC entries of professionals on LinkedIn for user modeling and personalized MOOC recommendations. In J. Vassileva & J. Blustein (Chairs), UMAP ’16: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (pp. 291-292). doi: http://dx.doi.org/10.1145/2930238.2930264

Piedra, N., Chicaiza, J., López, J., & Caro, E. T. (2014, April). Supporting openness of MOOCs contents through of an OER and OCW framework based on Linked Data technologies. In J. Mottok (Chair), 2014 IEEE Global Engineering Education Conference (EDUCON) (pp. 1112-1117). doi: 10.1109/EDUCON.2014.6826249

Popescu, I., Portelli, K., Anagnostopoulos, C., & Ntarmos, N. (2017). The case for graph-based recommendations. In R. Baeza-Yates, X. Tony Hu, & J. Kepner (Chairs), 2017 IEEE International Conference on Big Data (Big Data). (pp. 4819-4821). doi: 10.1109/BigData.2017.8258553

Potts, B. A., Khosravi, H., Reidsema, C., Bakharia, A., Belonogoff, M., & Fleming, M. (2018). Reciprocal peer recommendation for learning purposes. In A. Pardo, K. Bartimote-Aufflick, & G. Lynch (Chairs), LAK ’18: Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp 226-235). doi: https://doi.org/10.1145/3170358.3170400

Prabhakar, S., Spanakis, G., & Zaiane, O. (2017). Reciprocal recommender system for learners in massive open online courses (MOOCs). In H. Xie, E. Popescu, G. Hancke, & M. B. Fernández (Eds.), Advances in Web-Based Learning - ICWL 2017. ICWL 2017. Lecture Notes in Computer Science, vol 10473. (pp. 157-167). doi: https://doi.org/10.1007/978-3-319-66733-1_17

Rabahallah, K., Mahdaoui, L., & Azouaou, F. (2018). MOOCs recommender system using ontology and memory-based collaborative filtering. In O. Camp, & J. Filipe (Chairs), Proceedings of the 20th International Conference on Enterprise Information Systems—Vol. 1 (pp. 635-641). doi: 10.5220/0006786006350641

Rădoiu, D. (2014). Organization and constraints of a recommender system for MOOCs. Scientific Bulletin of the Petru Maior University of Tîrgu Mureş, Romania, 11(XXVIII)(1), 57-61. Retrieved from http://scientificbulletin.upm.ro/papers/2014-1/11_Radoiu%20Dumitru.pdf

Ricci, F., Rokach, L., Shapira, B., & Kantor, P. B. (Eds.). (2011). Recommender systems handbook. New York, NY: Springer-Verlag. doi: 10.1007%2F978-0-387-85820-3

Santos, O. C., & Boticario, J. G. (2015). Practical guidelines for designing and evaluating educationally oriented recommendations. Computers and Education, 81, 354-374. doi: 10.1016/j.compedu.2014.10.008

Santos, O. C., Boticario, J. G., & Pérez-Marín, D. (2014). Extending web-based educational systems with personalised support through User Centred Designed recommendations along the e-learning life cycle. Science of Computer Programming, 88, 92-109. doi: 10.1016/j.scico.2013.12.004

Shah, D. (2018, 11 December). By the numbers: MOOCs in 2018. Class Central MOOC Report. Retrieved from https://www.classcentral.com/report/mooc-stats-2018/

Shaptala, R., Kyselova, A., & Kyselov, G. (2017). Exploring the vector space model for online courses. In I. Pichkalov (Chair), 2017 IEEE First Ukraine Conference on Electrical and Computer Engineering (UKRCON) (pp. 861-864). doi: 10.1109/UKRCON.2017.8100370

Sunar, A. S., Abdullah, N. A., White, S., & Davis, H. C. (2016). Personalisation in MOOCs: A Ccritical Lliterature Rreview. In S. Zvacek, M. T. Restivo, J. Uhomoibhi, & M. Helfert (Eds.), International Conference on Computer Supported Education CSEDU 2015. (pp 152-168). doi: https://doi.org/10.1007/978-3-319-29585-5_9

Sunar, A. S., Abdullah, N. A., White, S., & Davis, H. C. (2015a). Analysing and predicting recurrent interactions among learners during online discussions in a MOOC. In T. Watanabe, & K. Seta (Eds.), Proceedings of the 11th International Conference on Knowledge Management ICKM 2015. Retrieved from https://eprints.soton.ac.uk/381181/1/ICKM2015_ayse_revised.pdf

Sunar, A. S., Abdullah, N. A., White, S., & Davis, H. C. (2015b, September). Analysis of social learning networks on Twitter for supporting MOOCs education. Poster session presented at the meeting of ACM-W Europe womENcourage Celebration of Women in Computing, Uppsala, Sweden.

Sunar, A. S., Abdullah, N. A., White, S., & Davis, H. C. (2015c). Personalisation of MOOCs: The state of the art. In M. Helfert, & M. T. Restivo (Eds.). Proceedings of the 7th International Conference on Computer Supported Education - Volume 1: CSEDU (pp. 88-97). doi: 10.5220/0005445200880097

Symeonidis, P., & Malakoudis, D. (2016, September). MoocRec.com: Massive open online courses recommender system. Poster session presented at the 10^th ACM Conference on Recommender Systems (RecSys 2016), Boston, USA. Retrieved from http://ceur-ws.org/Vol-1688/paper-01.pdf

Symeonidis, P., & Malakoudis, D. (2018). Multi-modal matrix factorization with side information for recommending massive open online courses. Expert Systems With Applications, 118, 261-271. doi: 10.1016/j.eswa.2018.09.053

Taha, E. A. El Kadiri, K.E., & Chrayah, M. (2017). Toward a new framework of recommender memory based system for MOOCs. International Journal of Electrical and Computer Engineering (IJECE), 7(4), 2152-2160. doi: 10.11591/ijece.v7i4.pp2152-2160

Tan, M., & Wu, M. (2018). An association rule model of course recommendation in MOOCs: Based on edX platform. European Scientific Journal, 14(25), 284. doi: 10.19044/esj.2018.v14n25p284

Wang, Y., Liang, B., Ji, W., Wang, S., & Chen, Y. (2017). An improved algorithm for personalized recommendation on MOOCs. International Journal of Crowd Science, 1(3), 186-196. doi: 10.1108/IJCS-08-2017-0021

Wang, Y., Maruyama, N., Yasui, G., Kawai, Y., & Akiyama, T. (2017). A Twitter-based recommendation system for MOOCs based on spatiotemporal event detection. In W. Sterzer (Ed.), iConference 2017 Proceedings Vol. 2 (pp. 152-155). Retrieved from http://hdl.handle.net/2142/98491

Xiao, J., Wang, M., Jiang, B., & Li, J. (2018). A personalized recommendation system with combinational algorithm for online learning. Journal of Ambient Intelligence and Humanized Computing, 9(3), 667-677. doi: 10.1007/s12652-017-0466-8

Yang, D., Adamson, D., & Rosé, C. P. (2014). Question recommendation with constraints for massive open online courses. In A. Kobsa, & M. Zhou (Chairs), RecSys'14: 8th ACM Conference on Recommender Systems (pp. 49-56) doi: http://dx.doi.org/10.1145/2645710.2645748

Yang, D., Piergallini, M., Howley, I., & Rosé, C. P. (2014). Forum thread recommendation for massive open online courses. In M. Mavrikis, & B. M. McLaren (Chairs), EDM 2014: 7th International Conference on Educational Data Mining (pp. 257-260). Retrieved from https://educationaldatamining.org/EDM2014/uploads/procs2014/short%20papers/257_EDM-2014-Short.pdf

Yang, D., Shang, J., & Rosé, C. P. (2014). Constrained question recommendation in MOOCs via submodularity. In J. Li & X.S. Wang (Chairs), CIKM ’14: Proceedings of the ACM International Conference on Information and Knowledge Management (pp. 1987-1990). doi: 10.1145/2661829.2662089

Yanhui, D., Dequan, W., Yongxin, Z., & Lin, L. (2015). A group recommender system for online course study. In S. Li, Y. Dai, & Y. Cheng. (Eds.), Proceedings of the 7th International Conference on Information Technology in Medicine and Education (ITME) (pp. 318-320). doi: 10.1109/ITME.2015.99

Yousef, A. M. F., & Sunar, A. S. (2015, June). Opportunities and challenges in personalized MOOC experience. Paper presented at the ACM WEB Science Conference 2015, Web Science Education Workshop 2015, Oxford, UK.

Zhang, H., Huang, T., Lv, Z., Liu, S., & Yang, H. (2019). MOOCRC: A highly accurate resource recommendation model for use in MOOC environments. Mobile Networks And Applications, 24(1), 34-46. doi: 10.1007/s11036-018-1131-y

Zhang, H., Huang, T., Lv, Z., Liu, S., & Zhou, Z. (2018). MCRS: A course recommendation system for MOOCs. Multimedia Tools and Applications, 77(6), 7051-7069. doi: 10.1007/s11042-017-4620-2

Zhang, H., Yang, H., Huang, T., & Zhan, G. (2017). DBNCF: Personalized courses recommendation system based on DBN in MOOC environment. In F. L. Wang, O. Au, K. K. Ng, J. Shang, & R. Kwan (Eds.), 2017 International Symposium on Educational Technology (ISET) (pp. 106-108). doi: 10.1109/ISET.2017.33

Zhang, M., Zhu, J., Wang, Z., & Chen, Y. (2019). Providing personalized learning guidance in MOOCs by multi-source data analysis. World Wide Web, 22(3), 1189-1219. doi: 10.1007/s11280-018-0559-0

Zhao, J., Bhatt, C., Cooper, M., & Shamma, D. A. (2018). Flexible learning with semantic visual exploration and sequence-based recommendation of MOOC videos. In R. Mandryk & M. Hancock (Chairs), CHI ’18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Paper 329) (pp. 1-13). doi: https://dl.acm.org/doi/abs/10.1145/3173574.3173903

Zhou, M., Cliff, A., Krishnan, S., Nonnecke, B., Crittenden, C., Uchino, K., & Goldberg, K. (2015). M-CAFE 1.0: Motivating and prioritizing ongoing student feedback during MOOCs and large on-campus courses using collaborative filtering. In A. Settle & T. Steinbach (Chairs), Proceedings of the 16th Annual Conference on Information Technology Education SIGITE ’15. (pp. 153-158). doi: http://dx.doi.org/10.1145/2808006.2808020

Recommender Systems for MOOCs: A Systematic Literature Survey (January 1, 2012—July 12, 2019) by Asra Khalid, Karsten Lundqvist, and Anne Yates is licensed under a Creative Commons Attribution 4.0 International License.