Figure 1. Distribution of PM MOOC participants’ final grades.
Volume 21, Number 4
Rawad Chaker1 and Rémi Bachelet2
1Lyon 2 University, France, 2Lille University, Centrale Lille, France
This paper uses data mining from a French project management MOOC to study learners’ performance (i.e., grades and persistence) based on a series of variables: age, educational background, socio-professional status, geographical area, gender, self- versus mandatory-enrollment, and learning intentions. Unlike most studies in this area, we focus on learners from the French-speaking world: France and French-speaking European countries, the Caribbean, North Africa, and Central and West Africa. Results show that the largest gaps in MOOC achievements occur between 1) learners from partner institutions versus self-enrolled learners 2) learners from European countries versus low- and middle-income countries, and 3) learners who are professionally active versus inactive learners (i.e., with available time). Finally, we used the CHAID data-mining method to analyze the main characteristics and discriminant factors of MOOC learner performance and dropout.
Keywords: MOOCs, learner grades, learner dropout, learner performance, academic cohorts, educational data mining, CHAID, low- and middle-income countries, developing countries
Numerous studies have sought to understand who benefits most from MOOCs (Emanuel, 2013; Kizilcec, Pérez-Sanagustín, & Maldonado, 2017; Liyanagunawardena, Williams, & Adams, 2013; Selingo, 2014; Stump, Hilpert, Husman, Chung, & Kim, 2011), since their original promise was education everywhere and for everyone (Lane, 2013; Laurillard, 2014). MOOCs are mainly provided in the English language (Brouns et al., 2017; Colas, Sloep, & Garreta-Domingo, 2016), and their principal audience is from Western cultural regions (e.g., Europe, North America, Australia; Altbach, 2014; Bozkurt, Akgün-Özbek, & Zawacki-Richter, 2017; Jansen, Schuwer, Teixeira, & Aydin, 2015). In 2016, MOOCs in French were the second-most taught courses; and their principal audience, after France and Western French-speaking countries, was based in Africa (i.e., North Africa and French-speaking African countries; Noukakis, Escher, & Aebischer, 2016) and in the Caribbean (largely Haiti).
Despite their expansion, MOOCs have not met initial expectations. The open and online format has, for the most part, benefited educated young learners, people based in high-income countries, and male learners (Christensen et al., 2013; Emanuel, 2013; Zhenghao et al., 2015). The MOOC audience is not as diverse as expected (Albelbisi, Yusop, & Salleh, 2018). Hansen and Reich (2015) found that, even within the United States, MOOCs accentuate the digital divide between social classes rather than reduce it. This reality is more acute in Africa (Noukakis et al., 2016). There is also an imbalance between the use of MOOCs and their added value. Garrido and colleagues (2016) suggest that MOOCs in developing geographical areas are used to gain specific job skills, education, and professional certification; but people are less likely to complete the courses than Western participants. Zhenghao et al. (2015) found that learners from emerging countries report gaining more career and educational benefits from MOOCs than learners from high-income countries. However, these learners sometimes complete modules to gain specific job skills with no reason to complete the entire MOOC. This is often the case with African participants (Noukakis et al., 2016).
Overall, research on MOOCs in developing (i.e., low- or middle-income) countries—and on learner engagement and interaction within culturally and linguistically diverse learner cohorts—is limited (Launois et al., 2019), especially in French-speaking areas. This study compares the benefits of a French professional development MOOC for learners from high-income countries and learners from low- and middle-income countries (LMIC). Our goal was to determine whether the results in this specific context (a professional development MOOC designed for the French-speaking geographical area) would differ from the results of studies on MOOCs conducted in English- or Spanish-speaking regions (Garrido et al., 2016). The use of educational data mining (EDM) allowed us to investigate the details of learners’ performance in the MOOC, including the number of learners who passed, failed, or dropped out. We measured the differences in learner performance based on geographical area, gender, age, prior education, and socio-professional status to identify the most powerful predictor(s) of MOOC success.
We studied the 12th session of a French project management (PM) MOOC (Bachelet & Chaker, 2017), which ran from September to November, 2018. Unlike most MOOC studies, we focused on French-speaking cultural regions: Europe (France and French-speaking European countries), North Africa, Central and West Africa, and the French-speaking Caribbean. Two data sets were used to establish our analyses: results of a questionnaire, which participants completed at the beginning of the session; as well as their scores from weekly evaluations and the final exam, which were used to calculate their final grade.
MOOCs are widely used for professional development (Dodson, Kitburi, & Berge, 2015; Domingo, Paran, Révész, & Palange, 2019; Garrido et al., 2016; Hrdličková & Dooley, 2017); for example, two-thirds of those who took one or more of 24 MOOCs offered through the University of Pennsylvania were employees (Christensen et al., 2013). Online learning programs have been expanding within the continuing education and professional development market (Mori & Ractliffe, 2016), and 2014 was dubbed “The Year of the Corporate MOOC” (Nielson, 2014). For Radford et al. (2014), corporate MOOCs hold an immediate promise of professional development for employers and employees. MOOCs are indeed often “aligned with popular professional development courses in leadership, management, communications, and desktop applications” (Radford, Coningham, & Horn 2015, p. 13).
The MOOC format, which is by definition open and online, can be useful to professional learning, since it enables the transferability of skills between professional practice and learning (Milligan & Littlejohn, 2014). The successful use of MOOCs for professional development depends on “how professionals align their personal learning goals with learning in the MOOC” (Milligan & Littlejohn, 2014, p. 199), as MOOCs attract a broad range of learners with diverse learning dispositions (Milligan, Littlejohn, & Margaryan 2013). Indeed, research emphasizes the importance of environment in professional engagement and participation (Launois et al., 2019; Milligan & Littlejohn, 2014; Mori & Ractliffe, 2016; Murugesan, Nobes, & Wild, 2017).
Many issues have been raised regarding occupational online training, including the need for professionals to self-regulate their online training, as personal dispositions are influenced by their environment (Milligan & Littlejohn, 2014); the importance of allocating time for online training within personal and work schedules, which may interfere with learner commitment (Mori & Ractliffe, 2016); the need to address excess workloads during professional development (Hossain, 2010); and the importance of emphasizing the interrelationship between knowledge and professional skills in context (Milligan & Littlejohn, 2014; Mori & Ractliffe, 2016). International professional development courses online must, therefore, consider participants’ environment in the learning process, especially in LMIC where access to technology and the gap in its use could be a problem (Liyanagunawardena et al., 2013; Zillien & Hargittai, 2009).
When completion rates are the benchmark for student success, evaluating the efficacy of MOOCs proves difficult (Jones, Stephens, Branch-Mueller, & de Groot, 2016). It is now widely known that MOOCs are associated with high attrition rates (Allione & Stein, 2016). This is a complex issue that goes beyond counting the number of students who quit. For example, Koller et al. (2013) show that “the ease of non-completion in MOOCs can be viewed as an opportunity for risk-free exploration” (para. 27). Beyond numbers, the reasons for registering and participating in and dropping out of a MOOC must be considered.
Perspectives on attrition in open online courses depend on how the courses are viewed, as Kizilcec and Halawa (2015) emphasize: MOOCs can be viewed as an open learning environment, where anyone can obtain whatever learning material they want. Indeed, as MOOCs are often free and display low commitment requirements, students tend to set their own participation requirements (Stewart, 2013). MOOCs can also be viewed through a traditional, school-norm lens, where the primary objective is to achieve the goals set by the instructors and course designers, which Tyack and Cuban (1995) refer to as the “grammar of schooling.” For example, people who register for a MOOC may quit after a few weeks because they have acquired the skills or knowledge they wanted and were thus satisfied. Such cases can be considered a learning success from the participant’s point of view, as they benefitted from an informal type of learning. However, the participant can also be considered a dropout from the “grammar of schooling” standpoint. Perhaps herein lies the difference between attrition and dropout.
Since the reasons for attrition are a complex issue in MOOCs, in this study we decided to measure persistence in course assessments; this variable is easy to measure empirically and can be considered a type of attrition. This approach was possible because the format of instruction and assessment in the PM MOOC includes weekly evaluations, as well as a final exam, and follows a schedule with weekly deliverables as opposed to self-paced learning. Students can, certainly, audit the course without completing the evaluations, or shift from active to passive participation; but, missing the weekly evaluations, especially after completing the first ones, can indicate a process of attrition in certain cases, especially in light of the learning goals that the participants initially set.
The French PM MOOC launched its first edition in September 2013 and has been hosted on the Open edX Learning Management System (LMS) since 2018. The common core curriculum of the MOOC consists of four units within the first 4 weeks and an evaluation at the end of each weekly unit. Learners’ global grade is calculated as follows,
Another distinguishing characteristic of this MOOC is the salience of academic cohorts (AC) among its learners. Half of the 6,400 active learners in the September 2018 session were students from partner institutions and had enrolled in the MOOC through their professor. AC students come from French higher-education institutions, and the weight of the MOOC in their curriculum is a powerful incentive for their success. The participation of this “captive” audience is one of the reasons for the high learner completion (success/active learner) rate (56%) in the 12th PM MOOC. According to Jordan (2013), the completion rate in the first edition of the PM MOOC was 50.7%.
Our research questions focus on the relationship between MOOC learners’ demographic backgrounds—in terms of age, gender, geographical area (region), education, and socio-professional status (SPS)—as independent variables; and student performance—in terms of MOOC final grades and dropout rates—as dependent variables (Table 1).
Table 1
Research Questions (RQ) and Variables
| RQ | Dependent variables | Independent variables |
| RQ1 | Grades | Age |
| RQ2 | Gender | |
| RQ3 | Region | |
| RQ4 | Education | |
| RQ5 | SPS | |
| RQ6 | Dropout rates | Age |
| RQ7 | Gender | |
| RQ8 | Region | |
| RQ9 | Education | |
| RQ10 | SPS |
The final question, RQ11, aimed to investigate the characteristics of the best MOOC performers and achievers.
Participants in our study were registered in the 12th edition of the French PM MOOC. MOOC registration characteristics were as follows,
The study questionnaire was posted on the MOOC platform one week before the beginning of the course. It was then made available to all enrolled registrants. Of the 18,302 learners enrolled in the MOOC, 1,792 responded to the questionnaire; 42.2% of the respondents were female, and 155 respondents were AC students (Table 2).
Table 2
Study Participant Demographics (N = 1,792; AC: n = 155)
| Independent variables | n | % |
| Gender | ||
| Female | 756 | 42.2 |
| Male | 1036 | 57.8 |
| SPS | ||
| Business owner | 29 | 1.5 |
| HMPO | 449 | 25.1 |
| Employee | 367 | 20.4 |
| Intermediate professions | 90 | 5.00 |
| Student | 398 | 22.2 |
| Jobseeker | 425 | 23.7 |
| Non-active | 34 | 1.9 |
| Education | ||
| No degree | 9 | 0.40 |
| High school graduate | 67 | 3.70 |
| 2-year technical or university degree | 207 | 11.60 |
| Second year engineering school | 116 | 6.50 |
| Bachelor’s degree | 394 | 22.00 |
| Master’s degree | 890 | 49.70 |
| PhD | 109 | 6.10 |
| Region | ||
| Europe | 1019 | 56.90 |
| North Africa | 127 | 7.10 |
| Central & Western Africa | 592 | 33.00 |
| Caribbean | 29 | 1.60 |
The questionnaire was created using Google Form, and a link for its online completion was posted during Week 0 of the MOOC (i.e., the introductory week). The second source of data was learners’ performance, which was obtained from file extractions through the edX platform.
Demographic background. We asked demographic and sociological questions regarding participants’ age (measured in years), gender, country of residence (the countries were merged into regions), SPS, and prior education.
MOOC certificate as a goal. The question on achievement goal, or intention, was measured using a Likert scale from 1 to 4 for the item: “I want to achieve the course certificate.”
Final grade. We determined the final grade by adding the final exam score (80%), the average score obtained in the four weekly evaluations (19%), and the score obtained in the pre-MOOC test (1%).
Dropout rate. Some researchers have attempted to calculate a context-based dropout rate or to take the participants’ perspective into account (Liyanagunawardena, Parslow, & Williams, 2014). For example, Henderikx, Kreijns, and Kalz (2017), measured the gap between intention to complete a MOOC and actual behavior (i.e., intention-behavior gap). Using a small sample, they measured this gap to study dropout rates based on intention but they did not provide a single indicator for dropout. This paper builds on this approach to calculate a context- and intention-based dropout rate. We first calculated persistence rather than the number of participant dropouts, since the term dropout, in the context of a MOOC, can have multiple definitions. It is indeed difficult to assess why certain participants quit after a few days or a few weeks, since their reasons for registering are often disparate. We calculated the assessment persistence index, based on the scoring system of the PM MOOC, and the number of assessments completed as presented in Table 3 below.
Table 3
Assessment Persistence Scoring System
| Persistence score | Week 1 assessment | Week 2 assessment | Week 3 assessment | Week 4 assessment | Final exam |
| 0 | |||||
| 1 | • | ||||
| 2 | • | • | |||
| 3 | • | • | • | ||
| 4 | • | • | • | • | |
| 5 | • | • | • | • | • |
This detailed scoring system measures not the gross dropout rate (i.e., the number of learners who dropped out before the final exam), but the number of weeks validated by the participants. Scoring student persistence by considering their completed assessments constitutes a bias, since it excludes auditing participants who may be active, but they are not interested in passing exams or obtaining a certificate. Nevertheless, we sought to define a dropout variable that is as close as possible to the academic definition of dropout (as mentioned earlier in the Attrition and Dropout section) by including the course certificate as a goal. As mentioned above, student dropout rates should be measured in context, which requires considering participants’ initial achievement goals or intentions. Hence, we calculated the difference between learners’ achievement intention scores (i.e., the drive to obtain the course certificate) and their assessment persistence scores.
The dropout rate was defined as the distance between the students’ formal learning goals, set at the beginning of the course, and students’ actual achievements. The analysis was achieved by weighing each participant’s achievement intention score out of 5 minus their assessment persistence score, which was measured out of 5 (4 weeks + final exam, Table 3). The resulting variable, which ranges from -5 to +5, is a new continuous variable for measuring student dropout based on 1) the achievement intention set at the beginning of the course (i.e., course certification), and 2) assessment persistence up to and including the final exam. Using our model, we can relate the minimum dropout score (-5) to an underestimated forecast of achievement, the 0 score to an accurate forecast of achievement, and the maximum score (5) to an overestimated forecast of achievement.
The assumption of normality in participants’ final grade scores was not met (DK-S =.239; p <.001). Hence, non-parametric tests were used. Examining the final grade distribution, we observed a U shape, displaying high concentrations on both ends (Figure 1).
Figure 1. Distribution of PM MOOC participants’ final grades.
Analysis shows a negative Spearman rank correlation between final grade and age (rs = -.250; p <.001). Controlling for AC bias, partial correlation shows a lower index but with the same orientation and significance (rs = -.137; p <.001).
Testing the effect of gender on the final grade with the Mann-Whitney U-test did not reveal any significant differences based on gender.
We grouped the countries participants were from by region: Europe (n = 1019, 56.9%), Central and West Africa (n = 592, 33%), North Africa (n = 127, 7.1%), and the Caribbean (n = 29, 1.6%, largely Haiti). As Emanuel (2013), Selingo (2014), and Waldrop (2013) note, in the early 2010s, people from developing countries, from Africa in particular, constituted only a fraction of MOOC registrants (Liyanagunawardena et al., 2013); but these studies focused on English language MOOCs. It is interesting to observe that the French PM MOOC reached a relatively large African audience (40.1%) compared to most Western English-language MOOCs.
Learners from Europe achieved the highest final grade mean among participants (M = 62.58), followed by learners from North Africa (M = 51.02), Central and Eastern Africa (M = 39.80), and the Caribbean (M = 31.50). This is a typical situation where MOOC success is related to the level of economic development of a country or region: From Haiti in the Caribbean (the poorest country in the sample) to Central and West Africa, to North Africa, to Europe. The grade differences between regions are significant, as results of the Kruskal-Wallis H test show: H(3) = 97.50; p <.001 (Figure 2).
Figure 2. Box-plot of final grades by region.
We observed a significant relationship between participants final grade and education (H(6) = 75.46; p <.001). Second year students in engineering school had the highest final grades (M = 81.89; SD = 24.25), while Master’s and PhD degree holders had the lowest grades (M = 49; SD = 40). These findings contradict Morris, Hotchkiss, and Swinnerton’s (2015) finding that “the higher the prior educational attainment, the greater the completion” (p. 7). As such, the enrollment system and participants’ demographic backgrounds must be considered when analyzing the relationship between education and performance in a MOOC.
Significant differences were found between participants’ final grades and socio-professional status (SPS; H(8) = 100.92, p <.001). Participants who were students performed best (M = 72, SD = 30), followed by job seekers (M = 54, SD = 39), and participants in higher managerial and professional occupations (HMPO; M = 52, SD = 39). Time availability appeared to be an important factor in MOOC success. HMPO participants, students, and jobseekers were better able to manage their schedules, compared to employees, intermediate professionals, and workers.
Dropout score distribution violated normality (DK-S =.151; p <.001). As a result, the statistical analyses were non-parametric.
Age was marginally correlated with dropout rate (rs =.172; p < .001). Excluding AC participants (who were forced to enroll and were less prone to attrition) from the analysis suppressed the significant relationship between the two variables, and results reveal that age had no effect on dropout rates. This finding contradicts Guo and Reinecke’s (2014) and Morris et al.’s (2015) finding that older learners are less prone to attrition.
Male participants (M = 1.56; SD = 2.70) had higher dropout rates than female participants (M = 1.21; SD = 2.54): U = 346,018; p =.001. The result was similar when controlling for AC bias (p <.001).
France and French-speaking European participants (FFSE; M = 0.64; SD = 2.57) persisted significantly more than participants from French-speaking developing countries (FSDC; M = 2.47; SD = 2.34): U = 540,402, p <.001. When controlling for AC bias, similar results were found (p <.001). These findings contradict Garrido et al.’s (2016) study on MOOC completion among participants from English- and Spanish-speaking developing countries, but they confirm Kizilcec et al.’s (2017) finding that MOOC completion is higher on average in more- versus less-developed countries. Could the nature of the course and the use of the French language, which is a second language in FSDC countries (Ngalasso, 1992), explain this difference? The dropout rate of FFSE participants is close to 0, which indicates a good forecast of achievement, whereas FSDC participants display a relatively high score of overestimated forecast of achievement.
Since, as results show, the geographical variable influences MOOC performance, we examined whether gender differences in dropout rates could be observed and better explained by dividing gender groups into FFSE and FSDC subsamples (Table 4). We noticed a significant difference between genders only among FFSE participants: Female participants displayed higher dropout rates (M = 0.81; SD = 2.82) than male participants (M = 0.43; SD = 2.68): U = 138,959; p =.032. However, no gender differences where observed among FSDC participants (p =.185). Controlling for AC bias, we expected to observe a difference only within the FFSE subsample, since the learners who were forced to enroll were based in France. We found no significant gender differences in dropout rates in both FFSE and FSDC subsamples. We could argue that the only gender imbalance in dropout rates was caused by AC participants. Indeed, when considering the entire sample, male participant dropout rates were significantly higher than female participants, but when analyzing the geographical subsamples, the gender effect on dropout rates was no longer significant.
Table 4
Dropout Score by Gender and Region (Without AC)
| Gender | FFSE | FSDC | ||
| M | SD | M | SD | |
| Male | 0.43 (1.10) | 2.68 (1.10) | 2.50 (2.50) | 2.35 (2.35) |
| Female | 0.81 (1.05) | 2.42 (2.40) | 2.40 (2.40) | 2.34 (2.34) |
| Total | 0.64 (1.07) | 2.57 (1.75) | 2.47 (2.47) | 2.34 (2.34) |
Controlling for AC bias, no significant relationships were found between dropout rates and prior education. Previous research found different relationships between prior education and MOOC completion. For example, Breslow et al. (2013) found only a marginal association between them, but Morris et al. (2015) found a significant link between higher degrees and MOOC completion. Our results are in line with the overall mixed results regarding education attainment and MOOC dropout rates.
Jobseekers (M = 1.73; SD = 2.62), students (M = 0.25; SD = 2.77), and HMPO (M = 1.26; SD = 2.50) persisted significantly longer than employees (M = 2.09; SD = 2.40) and intermediate professionals (M = 2.32; SD = 2.22; H(7) = 141.24, p <.001). These findings are in line with Morris et al. (2015), who state that “those not working [are] more likely to complete more of their course” (p. 8). These results are also in line with our findings on MOOC grades and SPS (RQ5) and could indicate that availability is a key factor in MOOC achievement: Jobseekers, HMPO, and students display the highest performance rates and the lowest dropout rates.
Overall, the geographical factor was found to be a determinant of MOOC achievement and dropout in separate analyses. To answer the question on the most discriminant characteristics of the best performers, we conducted a tree analysis, with CHAID (Chi-square Automatic Interaction Detection) as an educational data mining method to examine the predictive variables of MOOC success using SPSS. We used this method to determine whether the previous results, which were obtained separately by subsampling, could be verified through an automatic data mining method, such as CHAID analysis. As previously demonstrated, the demographic variables were strong indicators of MOOC performance.
Our predictive variables were region, gender, age, education, and professional status. We excluded the AC participants from our analyses as their presence in the sample would constitute a bias, since they were forced to enroll in the MOOC. If we had included them, the results would have been overly unbalanced between learners from FSDC and FFSE countries for MOOC performance and dropout rates as our previous results demonstrate. Results show that the main discriminant factor (the first node) of final grades and dropout rates is the region variable (Figures 3 and 4): FFSE participants had higher achievement scores than FSDC participants.
Figure 3. CHAID analysis with final grade as dependent variable (without Academic Cohorts). *FSDC: French-speaking developing countries; FFSE: France & French-speaking Europe; SPC: socio-professional categories.
Figure 3 shows that FFSE participants had higher final grades than FSDC participants, region being the first node (F = 25.13; p <.01). The second node was within the FFSE subsample, where job seekers, HMPO, and students (other than AC) had a higher average grade (M = 58.60; SD = 38.90) than employees, intermediate professionals, non-active people, business owners, and workers (M = 43.00; SD = 41.00; F = 13.15; p =.04). There were no node subdivisions among FSDC participants, and the gender variable did not appear to be a significant discriminant subgroup factor. The estimated risk for this model is 14.3%.
Figure 4. CHAID analysis with dropout score as dependent variable (without Academic Cohorts). *FSDC: French-speaking developing countries; FFSE: France & French-speaking Europe.
Figure 4 displays the CHAID analysis with dropout rate as a dependent variable. For this test, we removed the “course certificate as the learning intention,” since it was used to calculate the dropout variable. The result shows that region is the only discriminant factor of dropout: F = 133.34; p <.001 (MFFSE = 1.07; SD = 2.50; MFSDC = 2.48; SD = 2.35), and the estimated model risk is 5.22%.
Finally, we verified learner performance based on the MOOC scoring system and instructional design (i.e., pass or fail). Our goal was to analyze only the achievement or non-achievement factor, without considering grade means. The final grades were mathematically divided into three categories (Figure 5).
Figure 5. Categorization of the MOOC final grades in three groups.
We transformed the final grade data into three discrete grade groups respecting this grading structure (Table 5).
Table 5
Number of MOOC Registrants by Grade Group and Region
| Grade group | Europe without AC students | AC students only | Africa + Caribbean | Total |
| n (%) | n (%) | n (%) | n (%) | |
| 1 | 192 (39.34) | 7 (4.51) | 166 (49.11) | 365 (34.88) |
| 2 | 13 (2.66) | 5 (3.22) | 51 (15.08) | 69 (5.40) |
| 3 | 283 (57.99) | 143 (86.45) | 120 (37.79) | 546 (59.72) |
| Total | 488 (100) | 155 (100) | 337 (100) | 980 (100) |
Figure 6. CHAID analysis with grade group membership as dependent variable (without Academic Cohorts). *FSDC: French-speaking developing countries; FFSE: France & French-speaking Europe.
Figure 6 presents the CHAID analysis for grade group membership. The analysis shows that region is the most discriminant factor of average grade and final exam score group membership (χ2 = 64.91; p <.001): FFSE participants are more present in Group 3 than FSDC participants (58.00% and 35.60%, respectively), and FFSE participants are less present in Group 1 than FSDC participants (39.30% and 49.30%, respectively). Fewer FFSE participants also completed the weekly evaluations but failed in the final exam (Group 2, 2.70% and 15.10%, respectively).
Unlike the results obtained through data mining, where the average grade is the dependent variables, in this analysis age appears as a discriminant factor within the FSDC subgroup. The FSDC subgroup is divided into two nodes: Learners below the age of 27 years had higher achievement scores than learners above the age of 27 years (χ2 = 12.09; p =.021). FSDC participants above the age of 27 years are less present in Group 3 and more present in Group 1 and Group 2 than the rest of the overall sample. Overall, FFSE participants and FSDC participants below the age of 27 years old completed the weekly evaluations and passed the final exam. Conversely, FSDC participants above the age of 27 years tended to underachieve. The overall correct model percentage between observed and expected group membership is 60.60%, with a 39.40% estimated risk.
We found that the biggest gap in MOOC achievement, if we omit students who were forced to enroll in an institutional context, occurred between learners from European and low- and middle-income countries. A U-shaped grade curve was observed in all of our samples. Moreover, the better performance of students and job seekers among FFSE participants highlights the importance of time availability. The results regarding MOOC completion and performance and AC students show that formal for-credit learning is a key driver of MOOC success among participants from FFSE countries. These learners had higher achievement levels than learners who enrolled for professional development reasons, whether they were European or from LMIC.
The definition of dropout must also be considered in context. We chose to consider dropout rates in the context of achieving the learning goal to obtain a certificate, set at the beginning of the course. For other purposes, we could have chosen to weigh dropout rates against other learning intentions. This perspective underscores the multifactorial aspect of online course achievement: Motivation and time availability are necessary but non-sufficient factors in success. The lower grades and higher dropout rates of learners from LMIC emphasize the significance of social and economic determinants of achievement (e.g., learning environment and technology access). The CHAID analyses led us to predict that a specific subsample will underachieve compared to the global sample: Participants above the age of 27 years from LMIC. Based on results from this EDM method, we propose that instructional design for international professional development MOOCs should address the issues that this specific group encounters.
Elias (2011) highlights the challenges inherent to mobile learning in Africa. It is important to consider the access and connectivity problems African learners face (Kaliisa & Picard, 2017) when designing MOOCs. The key to the success of MOOCs in LMIC, thus, necessarily involves investigating the availability of learners as well as technology use and access issues. This could be addressed, for example, by analyzing the mobile data. Indeed, learning data from the 11th edition of the French PM MOOC (March-April, 2018), which was processed by Google Analytics, show that 1) mobile phones represent 35% of African connections and 22.6% of European connections; 2) African connections from mobile phones display a higher attrition rate (-28%) than European mobile connections (-23%) between the first and fourth week of class (Figure 7). One way to intervene effectively would be, for example, to plan lighter and more mobile-responsive online courses.
Figure 7. Number of mobile connections between the first and fourth week of the 11th edition of the French PM MOOC (March-April, 2018).
Another aspect of interest is the content delivered. There is a lack of local and contextualized content in MOOCs and in online education in general, as many studies point out (Czerniewicz, Deacon, Small, & Walji, 2014; King, Luan, & Lopes, 2018; King, Pegrum, & Forsey, 2018; Nkuyubwatsi, 2014; Nti, 2015, as cited in Launois et al., 2019). The digital divide concerns not only access but also use (Zillien & Hargittai, 2009). Liyanagunawardena, Williams and Adams (2013) note that even when there is access to good Internet connectivity, poor digital literacy skills pose a barrier. As Richter and McPherson (2012) assert regarding open educational resources, MOOCs are “produced in Western industrialized countries [and] may not necessarily fit the needs of learners in developing countries” (p. 203). MOOCs are “primarily organized by universities and address topics on an academic level” (Rohs & Ganz, 2015, p. 9).
Our conclusions draw upon student results in one session of the French PM MOOC. This is the main limitation of this research, although we included a relatively large and heterogeneous sample. Nevertheless, this study can pave the way to broader studies involving comparative analyses among different geographical areas within the French-speaking world, since, as noted in the introduction, such studies are limited. Furthermore, we analyzed MOOC success through the prism of formal success (i.e., learners’ final grade). It would be relevant to include among learning benefits participation itself, taking into consideration the cultural and economic context of the participants and their points of view (e.g., on their reasons for participating and self-assessed learning), as some researchers propose (Gamage, Perera, & Fernando, 2016; Guàrdia, Maina, & Sangrà, 2013; Liyanagunawardena et al., 2014).
Many studies (Castillo, Lee, Zahra, & Wagner, 2015; Daniel, Vázquez Cano, & Gisbert, 2015; Nkuyubwatsi, 2014) suggest adapting online learning content to the local contexts of developing countries (Murugesan et al., 2017) and providing guidance and support to the learners (Patru & Balaji, 2016). In order to adapt the French PM MOOC to local contexts, we have implemented a set of interventions, including
Finally, one third of the MOOC tutoring team is based in Africa. These methods could contribute to the high completion rate of African participants in this MOOC compared to most others.
Figure 8. Contextualizing learning: Screenshot of the Facebook deliverable-sharing group.
In conclusion, further research is needed to address the technology learners use to access MOOCs, learners’ geographical and cultural context, and learners’ demographic backgrounds in order to enhance the achievement rate of specific audiences, such as “older” participants from LMIC, as our empirical results show.
Albelbisi, N., Yusop, F. D., & Salleh, U. K. M. (2018). Mapping the factors influencing success of massive open online courses (MOOC) in Higher Education. EURASIA Journal of Mathematics, Science, and Technology Education, 14(7), 2995-3012. https:/doi.org/10.29333/ejmste/91486
Allione, G., & Stein, R. M. (2016). Mass attrition: An analysis of drop out from principles of microeconomics MOOC. The Journal of Economic Education, 47(2), 174-186. https://doi.org/10.1080/00220485.2016.1146096
Altbach, P. G. (2014). MOOCs as neocolonialism: Who controls knowledge? International Higher Education, 75, 5-7. https://doi.org/10.6017/ihe.2014.75.5426
Bachelet, R., & Chaker, R. (2017). Toward a typology of MOOC activity patterns. In C. Delgado Kloos, P. Jermann, M. Pérez-Sanagustín, D. Seaton, & S. White (Eds.) Digital education: Out to the world and back to the campus. EMOOCs 2017. Lecture Notes in Computer Science, vol 10254 (pp. 134-139). Springer, Cham. https://doi.org/10.1007/978-3-319-59044-8_15
Bozkurt, A., Akgün-Özbek, E., & Zawacki-Richter, O. (2017). Trends and patterns in massive open online courses: Review and content analysis of research on MOOCs (2008-2015). The International Review of Research in Open and Distributed Learning, 18(5), 118-147. https://doi.org/10.19173/irrodl.v18i5.3080
Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom research into edX's first MOOC. Research & Practice in Assessment, 8, 13-25. Retrieved from https://www.rpajournal.com/studying-learning-in-the-worldwide-classroom-research-into-edxs-first-mooc/
Brouns, F., Teixeira, A., Morgado, L., Fano, S., Fueyo, A., & Jansen, D. (2017). Designing massive open online learning processes: the sMOOC pedagogical framework. In Open Education: from OERs to MOOCs (pp. 315-336). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-52925-6_16
Castillo, N. M., Lee, J., Zahra, F. T., & Wagner, D. A. (2015). MOOCS for development: Trends, challenges, and opportunities. International Technologies & International Development, 11(2), 35. Retrieved from https://repository.upenn.edu/literacyorg_articles/6
Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. J. (2013, November 6). The MOOC phenomenon: Who takes massive open online courses and why? [Working paper]. University of Pennsylvania. http://dx.doi.org/10.2139/ssrn.2350964
Colas, J. F., Sloep, P. B., & Garreta-Domingo, M. (2016). The effect of multilingual facilitation on active participation in MOOCs. The International Review of Research in Open and Distributed Learning, 17(4), 280-314. https:doi.org/10.19173/irrodl.v17i4.2470
Czerniewicz, L., Deacon, A., Small, J., & Walji, S. (2014). Developing world MOOCs: A curriculum view of the MOOC landscape. Journal of Global Literacies, Technologies, and Emerging Pedagogies, 2(3), 122-139. Retrieved from http://hdl.handle.net/11427/19562
Daniel, J., Vázquez Cano, E., & Gisbert, M. (2015). The future of MOOCs: Adaptive learning or business model? Educational Technology in Higher Education, 12(1), 64-73. http://dx.doi.org/10.7238/rusc.v12i1.2475
Dodson, M. N., Kitburi, K., & Berge, Z. L. (2015). Possibilities for MOOCs in corporate training and development. Performance Improvement, 54(10), 14-21. https://doi.org/: 10.1002/pfi.21532
Domingo, M., Paran, A., Révész, A., & Palange, A. (2019, January). Exploring factors that influence the impact of MOOC learning on participants’ professional practice. UCL Institute of Education, University College London. Retrieved from https://london.ac.uk/sites/default/files/leaflets/ExploringFactorsThatInfluenceTheImpactOfMOOCs_Final.pdf
Elias, T. (2011). 71. Universal instructional design principles for mobile learning. International Review of Research in Open and Distributed Learning, 12(2), 143-156. https://doi.org/10.19173/irrodl.v12i2.965
Emanuel, E. J. (2013). MOOCs taken by educated few. Nature, 503(342). https://doi.org/10.1038/503342a
Gamage, D., Perera, I., & Fernando, S. (2016, March 7-9). Evaluating effectiveness of MOOCs using empirical tools: Learners perspective. The 10th International Technology, Education and Development Conference (pp. 8276-8284). Valencia, Spain. https://doi.org/10.21125/inted.2016.0937
Garrido, M., Koepke, L., Andersen, S., Mena, A., Macapagal, M., & Dalvit, L. (2016). An examination of MOOC usage for professional workforce development outcomes in Colombia, the Philippines & South Africa. Technology & Social Change Group, University of Washington Information School. Retrieved from https://tascha.uw.edu/publications/an-examination-of-mooc-usage-for-professional-workforce-development-outcomes/
Guàrdia, L., Maina, M., & Sangrà, A. (2013). MOOC design principles: A pedagogical approach from the learner’s perspective. eLearning Papers, 33. Retrieved from https://r-libre.teluq.ca/596/1/In-depth_33_4.pdf
Guo, P. J., & Reinecke, K. (2014, March 4-5). Demographic differences in how students navigate through MOOCs [Paper presentation]. The First ACM Conference on Learning @ Scale, 21-30. Atlanta, Georgia. https://doi.org/10.1145/2556325.2566247
Hansen, J. D., & Reich, J. (2015). Democratizing education? Examining access and usage patterns in massive open online courses. Science, 350(6265), 1245-1248. https://doi.org/10.1126/science.aab3782
Henderikx, M. A., Kreijns, K., & Kalz, M. (2017). Refining success and dropout in massive open online courses based on the intention-behavior gap. Distance Education, 38(3), 353-368. https://doi.org/10.1080/01587919.2017.1369006
Hossain, J. (2010). Professional development of higher education teachers: Can ODL contribute? Turkish Online Journal of Distance Education, 11(1), 123-133. Retrieved from https://dergipark.org.tr/en/pub/tojde/issue/16907/176319
Hrdličková, Z., & Dooley, G. (2017, June). AuthorAID capacity development impact study 2017 [Summary report]. INASP. Retrieved from https://www.inasp.info/sites/default/files/2018-04/AuthorAID%20study%20%E2%80%93%20summary%20report.pdf
Jansen, D., Schuwer, R., Teixeira, A., & Aydin, C. H. (2015). Comparing MOOC adoption strategies in Europe: Results from the HOME project survey. International Review of Research in Open and Distributed Learning, 16(6), 116-136. https://doi.org/10.19173/irrodl.v16i6.2154
Jones, K. M., Stephens, M., Branch-Mueller, J., & de Groot, J. (2016). Community of practice or affinity space: A case study of a professional development MOOC. Education for Information, 32(1), 101-119. https://doi.org/10.3233/EFI-150965
Jordan, K. (2013, August 27). MOOC completion rates: The data. Katy Jordan. Retrieved from http://www.katyjordan.com/MOOCproject.html
Kaliisa, R., & Picard, M. (2017). A systematic review on mobile learning in higher education: The African perspective. TOJET: The Turkish Online Journal of Educational Technology, 16(1), 1-18. http://www.tojet.net/articles/v16i1/1611.pdf
King, M., Luan, B., & Lopes, E. (2018). Experiences of Timorese language teachers in a blended massive open online course (MOOC) for continuing professional development (CPD). Open Praxis, 10(3), 279-287. Retrieved from https://www.learntechlib.org/p/187316/
King, M., Pegrum, M., & Forsey, M. (2018). MOOCs and OER in the Global South: Problems and potential. The International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3742
Kizilcec, R. F., & Halawa, S. (2015, March). Attrition and achievement gaps in online learning. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale (pp. 57-66). https://doi.org/10.1145/2724660.2724680
Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, 18-33. https://doi.org/10.1016/j.compedu.2016.10.001
Koller, D., Ng, A., Do, C., & Chen, Z. (2013). Retention and intention in Massive Open Online Courses: In-depth. EDUCAUSE Review, 48(3), 62-63. Retrieved from https://er.educause.edu/articles/2013/6/retention-and-intention-in-massive-open-online-courses-in-depth
Labarthe, H., Bachelet, R., Bouchet, F., & Yacef, K. (2016, February 22-24). Increasing MOOC completion rates through social interactions: A recommendation system [Paper presentation]. eMOOCs 2016, the Fourth MOOC European Stakeholders Summit, 471-480. Graz, Austria. Retrieved from https://hal.archives-ouvertes.fr/hal-01277664/document
Lane, A. (2013, October 23-25). The potential of MOOCs to widen access to, and success in, higher education study [Paper presentation]. EADTU: The Open and Flexible Higher Education Conference 2013, 189-20. Paris, France. Retrieved from http://oro.open.ac.uk/id/eprint/38881
Launois, P., Allotey, P., Reidpath, D., Maher, D., Certain, E., & Ross, B. (2019). Lessons learnt from a professional development MOOC: Engaging culturally and linguistically diverse learners from low-and middle-income countries. European Journal of Open, Distance, and eLearning, 2. Retrieved from https://www.eurodl.org/?p=current&sp=brief&article=809
Laurillard, D. (2014, January 16). Five myths about MOOCs. Times Higher Education. Retrieved from https://www.timeshighereducation.com/comment/opinion/five-myths-about-moocs/2010480.article
Liyanagunawardena, T., Williams, S., & Adams, A. A. (2013). The impact and reach of MOOCs: A developing countries’ perspective. eLearning Papers, 33, 38-46. Retrieved from http://centaur.reading.ac.uk/32452/1/In-depth_33_1.pdf
Liyanagunawardena, T., Parslow, P., & Williams, S. (2014, February 10-12). Dropout: MOOC participants’ perspective [Paper presentation]. EMOOCs 2014, the Second MOOC European Stakeholders Summit, 95-100. Lausanne, Switzerland. Retrieved from http://centaur.reading.ac.uk/36002
Milligan, C., & Littlejohn, A. (2014). Supporting professional learning in a massive open online course. The International Review of Research in Open and Distributed Learning, 15(5), 197-213. https://doi.org/10.19173/irrodl.v15i5.1855
Milligan, C., Littlejohn, A., & Margaryan, A. (2013). Patterns of engagement in connectivist MOOCs. MERLOT Journal of Online Learning & Teaching, 9(2), 149-159. Retrieved from https://jolt.merlot.org/vol9no2/milligan_0613.pdf
Mori, K., & Ractliffe, L. (2016, April). Evaluating the use of a MOOC within higher education professional development training [Paper presentation]. The 25th International Conference Companion on World Wide Web, 831--833. Montreal, Quebec, Canada. https://doi.org/10.1145/2872518.2890577
Morris, N. P., Hotchkiss, S., & Swinnerton, B. (2015, May 18-20). Can demographic information predict MOOC learner outcomes ? [Paper presentation]. eMOOCs 2015, the Third European MOOC Stakeholder Summit, 199-207. Mons Belgium. Retrieved from http://eprints.whiterose.ac.uk/86184/
Murugesan, R., Nobes, A., & Wild, J. (2017). A MOOC approach for training researchers in developing countries. Open Praxis, 9(1), 45-57. http://dx.doi.org/10.5944/openpraxis.9.1.476
Ngalasso, M. M. (1992). Le concept de français langue seconde. Études de linguistique appliquée, 88, 27. Retrieved from https://search.proquest.com/docview/1307659507?accountid=16511
Nielson, B. (2014). The year of the corporate MOOC? LinkedIn. Retrieved from https://www.linkedin.com/today/post/article/20140320165303-16131081-2014-the-year-of-the-corporate-mooc
Nkuyubwatsi, B. (2014, March). Cultural translation in massive open online courses (MOOCs). eLearning Papers, 37, 1-10. Retrieved from shorturl.at/bgoBV
Noukakis, D., Escher, G., & Aebischer, P. (2016). De Lausanne à Yaoundé: L’aventure des MOOCs Pratiques et recherche sur les MOOCs, à Lausanne et en Afrique francophone. Annales des Mines: Réalités Industrielles, 2, 48-53. https://doi.org/10.3917/rindu1.162.0048
Patru, M., & Balaji, V. (2016). Making sense of MOOCs: A Guide for policy-makers in developing countries. OASIS: Commonwealth of Learning Open Access Repository. Retrieved from http://dspace.col.org/handle/11599/2356
Radford, A. W., Robles, J., Cataylo, S., Horn, L., Thornton, J., & Whitfield, K. (2014). The employer potential of MOOCs: A mixed-methods study of human resource professionals’ thinking on MOOCs. The International Review of Research in Open and Distributed Learning, 15(5). https://doi.org/10.19173/irrodl.v15i5.1842
Radford, A. W., Coningham, B., & Horn, L. (2015). MOOCs: Not just for college Students—How organizations can use MOOCs for professional development. Employment Relations Today, 41(4), 1-15. https://doi.org/10.1002/ert.21469
Richter, T., & McPherson, M. (2012). Open educational resources: Education for the world? Distance Education, 33(2), 201-219. https://doi.org/10.1080/01587919.2012.692068
Rohs, M., & Ganz, M. (2015). MOOCs and the claim of education for all: A disillusion by empirical data. The International Review of Research in Open and Distributed Learning, 16(6). https://doi.org/10.19173/irrodl.v16i6.2033
Selingo, J. J. (2014). MOOC U: Who is getting the most out of online education and why. Simon & Schuster.
Stewart, B. (2013). Massiveness + openness = new literacies of participation? MERLOT Journal of Online Learning and Teaching, 9(2), 228-238. Retrieved from https://jolt.merlot.org/vol9no2/stewart_bonnie_0613.pdf
Stump, G. S., Hilpert, J. C., Husman, J., Chung, W., & Kim, W. (2011). Collaborative learning in engineering students: Gender and achievement. Journal of Engineering Education, 100(3), 475-497. https://doi.org/10.1002/j.2168-9830.2011.tb00023.x
Tyack, D. B., & Cuban, L. (1995). Tinkering toward utopia. Harvard University Press.
Waldrop, M. M. (2013). Online Learning: Campus 2.0. Massive open online courses are transforming higher education and providing fodder for scientific research. Nature, 495, 160-163. https://doi.org/10.1038/495160a
Zhenghao, C., Alcorn, B., Christensen, G., Eriksson, N., Koller, D., & Emanuel, E. J. (2015, September 22). Who’s benefiting from MOOCs, and why. Harvard Business Review. Retrieved from https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why
Zillien, N., & Hargittai, E. (2009). Digital distinction: Status‐specific types of Internet usage. Social Science Quarterly, 90(2), 274-291. https://doi.org/10.1111/j.1540-6237.2009.00617.x
Internationalizing Professional Development: Using Educational Data Mining to Analyze Learners' Performance and Dropouts in a French MOOC by Rawad Chaker and Rémi Bachelet is licensed under a Creative Commons Attribution 4.0 International License.