Volume 17, Number 4
Jean-François Colas1, Peter B. Sloep2, and Muriel Garreta-Domingo3
1,3Open University of Catalonia, Barcelona, Spain, 2Welten Institute, Open University of the Netherlands
A new approach for overcoming the language and culture barriers to participation in Massive Open Online Courses (MOOCs) is reported. It is hypothesised that the juxtaposition of English as the language of instruction, used for interacting with course materials, and one's preferred language as the language of participation, used for interaction with peers and facilitators, is preferable to "English only" for participation in a MOOC. The Hands-On ICT (HANDSON) MOOC included seven teams of facilitators, each catering for a different language community. Facilitators were responsible for promoting active participation and peer tutoring. Comparing language groups revealed a series of predictors of intention to learn, some of which became apparent in the first days of the MOOC already. The comparison also uncovered four critical factors that influence participation: facilitation, language of participation, group size, and a pre-existing sense of community. Especially crucial was reaching a sufficient number of active participants during the first week. We conclude that multilingual facilitation activates participation in MOOCs in various ways, and that synergy between the four aforementioned factors is critical for the formation of the learning network that supports a social dynamic of active participation. Our approach suggests future targets for the development of the multilingual and community potential of MOOCs.
Keywords: MOOC, multilinguality, facilitation, language barrier, language of participation, cultural translation, peer tutoring, community, Dunbar's number, group size, learning network, connectivist dynamics, active participation, inactivity, intention to learn
Massive Open Online Courses (MOOCs) have become popular instruments for opening up education to non-traditional student audiences, including those from different cultures and language groups (Deimann & Vogt, 2015; Fitzgerald, Wu, & Witten, 2014). MOOCs are portrayed as being widely accessible. However, English is their predominant language, so one seems to assume that non-English speakers would easily be able to understand and digest English language content. However, research shows that a mere 20% of the world's population is able to profit from English only educational content (Beaven, Comas-Quinn, Hauck, de los Arcos, & Lewis, 2013). Thus, it is fair to assume that many online learners will stumble upon a language barrier in their study of MOOCS if these are offered in English only.
MOOCs, therefore, would do well to become multilingual. This is particularly so for Europe (De Rosa, 2014). There, MOOCs in different languages would promote cross-cultural and multilingual learning, helping to preserve Europe's rich cultural, educational and linguistic heritage (Brouns, Serrano Martínez-Santos, Civera, Kalz, & Juan, 2015; Lundahl, 2014; Sloep & Schuwer, 2015). Furthermore, leveraging Europe's linguistic richness (24 official languages) creates an enormous opportunity for reaching target audiences in other parts of the world as well. Yet, even though Europe recognises it is the moment to latch onto the opportunities offered by MOOCs (e.g., Deimann & Vogt, 2015), European Union (EU) MOOC activities are mainly concentrated in Western and Southern Europe, do not fully reflect the cultural diversity of Europe, and serve a limited number of language communities only (Porto Declaration on European MOOCs, 2014). Even though the majority of Europeans agree that all 24 languages spoken in EU should be treated on a par, English is the language that Europeans are most likely to be able to speak as a second language (Perifanou, Holotescu, Andone, & Grosseck, 2014). In actual fact, then, multilinguality still is a distant ideal.
Perhaps another approach is needed, one in which English plays the part of a lingua franca, that facilitates the translation of courses into any other European language. Initiatives such as the European Multiple MOOC Aggregator (EMMA; http://project.europeanmoocs.eu/project/) capitalise on this idea by providing multilingual access to European MOOCs, thus bringing within reach the ideal of a pan-European, multi-linguistic, cross-cultural approach to online learning that reaches students beyond national boundaries (De Rosa, 2014; Brouns et al., 2015). MOOCs on EMMA are the first to be offered in three languages: Spanish, English and Italian. EMMA translates video subtitles and text content with the help of advanced speech recognition and machine translation technology, thus providing access to any MOOC's learning materials in three languages. The next step would be to expand the number of target languages. Although not foreseen, this is doable. Taking yet one step further, EMMA would also automatically translate interactive content, say from the native speaker's tongue into English via a translation service. At present, however, this is one step too far as the quality of the translation leaves too much to be desired.
And yet, even if automatic translation would be of a high quality, it is a misunderstanding to believe that MOOCs can be made accessible to everyone just by translating them. Translation alone can't fully replace course offerings developed by people who have a deep understanding of the cultural context in which students learn. Contextualisation, or cultural translation, makes MOOCs relevant to students in their respective cultural settings (Nkuyubwatsi, 2014). As the proponents of EMMA judiciously notice, offering MOOCs in multiple languages evokes a new challenge, that of the necessity of tutors to cover the entire course language spectrum (Civera, 2015).
So although translation no doubt increases MOOC accessibility, it still leads to a loss of pedagogical quality. Besides, most likely, cultural translation cannot be automated in the foreseeable future, if at all. We therefore sought to address this issue in a novel way. It is grounded in the distinction between the passive and active command of a language: people may well be able to read a foreign language, yet find it difficult to interact with other people in it. Therefore, for a particular MOOC, we decided to offer to people from different language groups English-only written content. However, we tackled the issue of cultural diversity that direct translation fails properly to address by offering multilingual facilitation, i.e. facilitation in one's own preferred (native or otherwise preferred) language.
Within the virtual learning environment of a MOOC teachers clearly play a different role than in ordinary, face-to-face teaching environments. Indeed, to help overcome barriers to participation in MOOCs new teacher roles have emerged that vary depending on the kind of MOOC. "Teachers" in the connectivist MOOCs, or cMOOCs, express educational goals differently than teachers in the institutionally rooted, highly structured extensions of MOOCs known as xMOOCs: they facilitate self-directed learning rather than provide access to expert knowledge (Ross, Sinclair, Knox, Bayne, & Macleod, 2014). Facilitators need to guide the learners and their communities to get involved and embrace social media practices (Kop, Fournier, & Mak, 2011; Medina-Salguero & Aguaded-Gómez, 2013). Importantly, facilitation also needs to address differences in cultural communication patterns. For learners who are not fully acculturated in the course's cultural backdrop, such differences easily become barriers to participation and learning, and even lead to isolation. This is particularly likely to happen when learners do not know each other well (i), the communication medium is impoverished and contains reduced non-verbal cues only (ii), and meaning is mostly carried by asynchronous, written means of communication (iii) (Rovai, 2007).
This view of facilitation is also espoused by De Waard et al. (2014). They notice that, to encourage multilingual participation in a MOOC, one should create small, open learning groups based in the various learners' native languages or in languages they feel comfortable with. Such "local" open learning groups can offer in-person networking and assistance (Godwin-Jones, 2014). They also provide an environment in which peer learners can tutor each other. The groups then effectively work as peer learning communities, allowing the burden of providing feedback to shift at least in part from the facilitator to the learners themselves (De Vries et al., 2005). Peer learning communities are justified in terms of "sustainability plans for the instructors" that obviously drastically change the teacher role (Ross et al., 2014).
Research shows that such online peer learning communities - sometimes called ad-hoc transient communities - can indeed be effective, efficient and attractive learning aids (Fetter et al., 2012; Sloep, 2009). However, peer tutoring can pose severe problems to many learners. First, they may not have the necessary skills to act as tutors (Hsiao, Brouns, Bruggen, & Sloep, 2015). Second and of particular relevance in the present context, they may not be able to bridge cultural divides. This goes of course especially for many non-native English speakers who participate in MOOCs offered in English (even when facilitation is offered in the learner's own language). As argued such MOOCs are inevitably developed from the developer's own cultural perspective. Participants in such MOOCs cannot but align themselves with that cultural perspective. Although not impossible, it is obviously hard then to bridge cultural divides on one's own. To be able effectively to do so, a facilitator is needed who is conversant with the MOOC's cultural backdrop and that of the open learning groups in it.
The facilitation approach sketched in the above we investigated in the Hands-On ICT (HANDSON) MOOC (http://www.handsonict.eu) a course offered to educators at all educational levels aimed at letting them become more conversant with the use of Information and Communications Technology (ICT) in their teaching. To the best of our knowledge, it is the first MOOC to incorporate various (seven) teams of facilitators, each responsible for promoting participation and peer tutoring in a different language community. Our approach was suggested by exploratory research that the HANDSON project team carried out during an earlier pilot of the course (in which 73% of the participants originated from Greece; Garreta Domingo, Hernández-Leo, Mor, and Sloep 2015b). Our findings indicated that, while most participants (themselves educators) were able to read the learning materials in English, they were reluctant to join the English spoken, live videoconference sessions (Google Hangouts). Crucially, the Greek forum that was offered as an impromptu solution, proved to be used intensely throughout the entire pilot.
Completion rate for this earlier pilot of the MOOC was 9.2%, already beating the 5% rates thus far observed in MOOCs (Jordan, 2014; http://www.katyjordan.com/MOOCproject.html) and those obtained in similar MOOCs for teacher training such as the Open Learning Design Studio's MOOC ( OLDSMOOC, http://www.olds.ac.uk/; 1.2%) or the ICT in Primary Education MOOC (https://www.coursera.org/course/ictinprimary; 5.3%). Also, similar to the trend identified by Jordan (2015), the first two weeks of the earlier pilot appeared to be critical in gaining student engagement. These positive preliminary results led us to further investigate facilitation in the student's native language.
The HANDSON MOOC pilot reported here guided participants through the various steps of a design process, with the aim of teaching them to produce design artefacts (Garreta Domingo, Hernández-Leo, Mor, & Sloep, 2015a). This set-up demanded learners each to develop their own learning project, which was to be tied to their own specific needs and cultural contexts. They were also stimulated to do so co-operatively. Although the learning materials were still provided in English only, the learners could produce artefacts and interact in forums with peers and facilitators in the language of their choice, usually their native tongue. We refer to the language in which the content was developed and made available to the learners as the language of instruction. The language in which the learners produced their artefacts, interacted with each other and with their facilitators we call the language of participation, to emphasise that only through active linguistic interaction genuine participation is achieved. Seven languages of participation were offered to the learners: Bulgarian, Catalan, English, French, Greek, Slovenian or Spanish.
We began by suggesting that multilinguality of MOOCs would address issues with their accessibility. We argued that, although efforts to translate MOOCs have been made, simple translation does not address existing broad cultural differences and that multilingual facilitation is called for. This then is why we developed the approach just described. It involves two research hypotheses. The first addresses the issue of whether the use of English as a language of instruction for anyone, regardless their native language, does not pose an unsurmountable barrier. If it would do so, we would consider our approach as failed:
H1 The use of English as the language of instruction, even for non-native English speakers, does not stand in the way of MOOC access.
The second hypothesis is our main hypothesis: it addresses the question of whether multilingual facilitation indeed fosters productive MOOC engagement. Productive MOOC engagement should be evident not only from the participants' appreciation but predominantly from increased completion rates:
H2 The use of the various participants' preferred languages as the languages of participation increases the success of MOOC engagement.
To tackle the complexity of the language and culture barriers to participation in a MOOC, the HANDSON project used a design-based research approach. It allowed us to study in naturalistic settings the impact of the incremental changes we administered over three pilots. The pilot reported here is the third in succession. Its focus is the evaluation of the effects of creating seven language groups to support multilingual facilitation. Information on the other pilots may be found in Stoyanov, Sloep, De Bie, and Hermans (2014) for pilot 1 and in Garreta Domingo, Hernández-Leo, Mor, and Sloep (2015b) for pilot 2.
This third pilot ran for five weeks, starting October 24, 2014. It was extensively advertised through many different sources and communities as the Learning Design Studio for ICT-based Learning Activities MOOC, because it made extensive use of the Learning Design Studio approach by Mor and Mogilevsky (2013).
The MOOC was implemented in the Canvas Virtual Learning Environment (VLE; www.canvasvle.co.uk/). The activities and (English) resources were made available to all participants as Canvas Modules. Each weekly module corresponded to one of the five phases of the Learning Design methodology: initiate (week 1), investigate (week 2), inspire & ideate (week 3), prototype (week 4) and evaluate & reflect (week 5) (Garreta Domingo et al., 2015a; Stoyanov, Sloep, De Bie, & Hermans, 2014). For the course to be facilitated in seven languages, seven parallel learning groups were created, each one linked to a separate Canvas group that functioned as the group's discussion space. Each language-specific learning group met every week for discussion in a live convergence session.
Each of the seven language groups was served by its own facilitator. Volunteer facilitators with an adequate command of English and of at least one of the seven different languages used were recruited through a call on the HANDSON website and the HANDSON consortium members' networks. They were trained through a series of informal video-conference meetings with the organisers. These sessions were focused on the Learning Design methodology, the MOOC activities, and the MOOC ICT Tools (Canvas, ILDE, Hangouts) (see "Table Pre-Course Activities - Facilitators Training" in the Appendix). In total there were 16 active facilitators: four were Bulgarian, two Catalan, three English, two French, three Greek, and two Spanish; due to the low activity in the small Slovenian group, after the first week facilitation was made available to its members through the English group only.
Participant experience was evaluated at the end of the course through questionnaires about the course and its approach, specifically on facilitation and multilinguality (see Appendix). Data were collected by an e-survey, using the Lime Survey web application. Closed questions were used, based on a 5-degree Likert scale (1 representing the lowest grade and 5 the highest). Data were analysed using descriptive statistics. Eighty-two fully-filled out responses to the survey were received from the participants. The language group distribution of these 82 respondents is given in ‘Table PostPilot3 – Which language group did you join" in the Appendix. As the survey was filled in anonymously, a break-down of responses by language group is not available.
In total 1691 people joined the pilot by creating a username in the Canvas environment of the course (their age and gender was not recorded). Of them, 902 joined one of the seven language groups and were active during the first week at least (Figure 1). Note that being part of a language group was needed to participate in any of the MOOC activities. Although the English and Spanish groups were the biggest ones by a small margin, the 902 participants who joined some language group were distributed fairly evenly over the groups. The only exception was the Slovenian group, counting 19 members only. Excluding the Slovenian group, which was short-lived, the average group size was 147.2 participants.
Figure 1. Distribution of participants in the seven language groups of the HANDSON MOOC.
For four reasons, the set-up deliberately did not include a control group of learners, who would have had to be tutored in a language different than their own. First, methodologically speaking, the entire HANDSON study was set up as a design experiment, with incremental changes administered over three pilots (Burkhardt & Schoenfeld, 2003). The first pilot's set-up was suggested by a literature survey (Stoyanov et al., 2014), and changes were subsequently implemented in pilots 2 and 3 as a result of experiences in pilots 1 and 2 respectively (Garreta Domingo et al., 2015b). Such a design study does not lend itself naturally to randomised experimental designs with control groups. Second, one of the ideas driving the HANDSON project was to beat the usual high dropout rates (cf. Hypothesis 2). Those rates have been reported widely (Jordan, 2014, 2015), so there is little to be gained from measuring them again. Third, there is also an ethical aspect. Is it acceptable to expose learners with a genuine professional development need to a learning environment that, judging from existing research, is very likely to be suboptimal? Our answer was "no." Fourth and most importantly, exposing learners to a randomised experimental design would have implied informing them about the nature of the experiment, including the likelihood of participating in a control group. Providing this information is an ethical and legal obligation. We feared it would cause people to drop out once they realised they participated in the control group. This would have affected the very purpose of the investigation.
Over the five-week duration of the experiment, MOOC participation dwindled, as was to be expected. Figure 2 describes the way in which participation unfolds during the five week period. Overall, out of 798 participants who were active during week 1, 161 completed the first week's module of activities. Among them were the 92 "designers" who eventually completed all the modules of the course and were awarded the designer badge and a course completion certificate as a result. The figure details four participant types: the designers and three kinds of non-designers. "Non-designers" either completed, started, or did not start at all the weekly modules of learning activities.
Figure 2. Participation in weekly modules of activities (all language groups).
The pattern of activity showed a steady fall over the course duration, which was mostly due to the decrease in the number of non-designers, who only started the modules. Their number was 637 at the beginning of the course and went down to only 31 in week 5. The number of non-designers completing the module decreased similarly, so that by the end of week 5, designers represented the large majority of still active participants (92/123=74.8%).
Differences between the seven language groups were observed in the way participation developed over the five weeks of the MOOC (Figures 3-9).
Figure 3. Bulgarian group: Participation in weekly modules of activities.
Figure 4. Catalan group: Participation in weekly modules of activities.
Figure 5. English group: Participation in weekly modules of activities.
Figure 6. French group: Participation in weekly modules of activities.
Figure 7. Greek group: Participation in weekly modules of activities.
Figure 8. Slovenian group: Participation in weekly modules of activities.
Figure 9. Spanish group: Evolution of participation in weekly modules of activities.
All seven groups showed the MOOC-wide decrease described above in the number of non-designers, but with some idiosyncrasies. Notably, the Greek non-designers only starting the module stood out from the rest of the groups, especially in week 1, as they represented only 37% of the total number of participants of the Greek group, while this proportion ranged from 46 to 63% in the other groups. Also, of the 109 non-designers completing modules recorded during weeks 1-4 for all groups, 43 (39.4%) were from the Catalan group. The most salient difference between language groups was however in the number of designers, most of them (80/92=87%) belonging to only three language groups, the Bulgarian, Catalan and Greek ones.
To refine the description of participants' engagement and the comparisons between the language groups, as well as to gain further insights into the possible origins of observed differences, special attention was paid to completion behaviour following accomplishment of week 1. According to Laurillard (2015), activity in week 1 is a much better indicator of "intention to study" than "registration." Indeed, 57% of the participants who completed week 1's module completed the whole course, whereas only 10.2% of the total number of registrants did (Figure 10).
Figure 10. Completion rates for the seven language groups.
The Greek group had the best post-week 1 completion rate. Of the 35 participants active in week 1 (having been awarded the week-1 badge), 28 (80%) finished the course and obtained the "Designer Badge" (certificate of completion). This translated also into the best completion rate overall (26%), taking into account all the members of the Greek group. With similar rates, the Bulgarian group is second to the Greek one. The Catalan group with an overall 19% completion rate ranks third, although with only 54% of those who completed week 1 (also third in rank). No Slovenian participant obtained a badge. The completion rate of week-1 badge owners for the three remaining groups, English (36%), French (29%) and Spanish (12%), was less than half of the Greek rate.
Thus, large differences between language groups were observed in how the participation rates developed over the five weeks of the MOOC; activity during week 1 appeared to be a solid predictor of course completion. To gain insights into the possible causes for such differences, we turned to the participants' evaluation of their experience with the multilingual facilitation. This information was obtained through surveys, especially those on multilinguality and facilitation (Figures 11 and 12, respectively), administered immediately after the course.
The HANDSON MOOC participants who joined a language group were mostly from non-English speaking countries (692/902=76.7%; Figure 1). For 79% of the participants surveyed, the language group represented an environment conducive to their learning activities. Their command of English was quite high (not shown). Only 23% of the HANDSON MOOC participants were basic English users. 7% were native English speakers and the remaining 70% were intermediate level or proficient users. Only 11% of those who dropped out of the course did so because of insufficient English language knowledge (inferred from "the materials were in English and I did not expect that"). For comparison, the most common reason for deciding not to continue the MOOC was lack of time (for 67% of those dropping out; not shown).
Even though reading materials in English did not appear to present a major obstacle to participation and in spite of various announcements to the contrary, a small majority of the participants expected to find all materials translated in their own language (51%) and would certainly have preferred it in their own language (54%) (Figure 11). Accordingly, only 49% found the weekly Hangout live session of videoconference in English easy to follow, whereas the weekly Hangouts in their own or preferred language were better received, and reported as useful for 67%. Despite these notable difficulties with listening to Hangouts in English, in the experience of 65% there was a good balance between the materials and sessions in English and the language groups.
Figure 11. Results of the survey of the participants about multilinguality.
Overall then, although they had a good command of English and the English MOOC reading materials did not present a major obstacle to them, a large majority (76%; not shown) enjoyed being part of a multilingual MOOC. 78% enjoyed contributing in their own language and 79% found it very helpful having a facilitator in their own language. Responses they received from the facilitators were found helpful (82%). Instructions provided by facilitators to the whole group were clear (78%; Figure 12). Facilitators adequately helped to cope with the problems experienced during the learning activities (76%). Overall, respondents were very satisfied with facilitators, their role, help, guide and motivation . In accordance with this positive assessment, only 2% of the participants who dropped out of the MOOC did so because they felt the facilitators did not provide enough guidance (not shown); this reason for dropping out is the least important of all the reasons recorded for the HANDSON MOOC.
Figure 12. Results of the survey of the participants about facilitation.
Thus, in the participants' feedback on the multilingual facilitation no factor stood out as a possible cause for observed engagement differences between language groups. The fact that the English group ranked only fourth in completion rates also seems to confirm that multilingual facilitation performed well enough to compensate for the disadvantage of non-native learning materials, and was thus largely irrelevant for explaining differences in completion behaviours between language groups.
This apparent explanatory irrelevance of the act of facilitation and our inability to find any obvious clues corresponding to observed behaviour differences, prompted us to search for other possible predictors of these different behaviours. Since participant engagement seems to have settled already at the end of the first week, we looked for still earlier determinants which could be identified already in the very first days of the MOOC. We identified two instruments for probing the differential activity of the seven language groups that met our criteria: Mailchimp and Canvas Groups Pageviews.
A Mailchimp campaign was launched on the second day of the MOOC already. It consisted of a welcoming message to the participants and contained a link to further information. It was sent separately to each language group and was phrased in the group's language. The result of this campaign ("open and click rates") were extracted two days later (Figure 13).
Figure 13. Response to early Mailchimp campaign by the seven language groups.
Three language groups ranked above average in their response to the Mailchimp message: the Catalan, Bulgarian and Greek ones. Response rates of the four remaining groups were below average. Thus, already on the second day in the MOOC's history, the differences in engagement between the language groups match those that were observed later on, throughout the remaining runtime of the course. The use of the language group pages in Canvas complement these findings. Use of the MOOC Canvas platform by participants was evaluated through the analysis of the number of times the group pages were viewed (the group page was accessed each time participants engaged in the learning activities of their group). GoogleAnalytics Pageview was used to extract this information during the five weeks the MOOC ran, as well as the week before and after that (Figure 14).
Figure 14. Group Pageviews.
As may be observed in "Average of all groups" (black line), page viewing reached a maximum during week 1, and then decreased steadily during the following weeks. The Catalan group showed a trend very similar to the average of all groups. The Bulgarian and Greek groups stood out with a first week's peak of page viewing roughly double the average value. In contrast, page viewing for the remaining four groups (English, French, Slovenian, and Spanish) was below average during the whole duration of the MOOC. For week-1 page viewing the seven groups ranked as follows: 1st Bulgarian, 2nd Greek, 3rd Catalan, 4th French, 5th Spanish, 6th English and 7th Slovenian. This ranking was maintained over the entire measurement period.
Thus, information from page viewing was in general agreement with data on module completion, especially of week-1 completion, and represented a good indicator of engagement in learning activities. Page viewing, it seems, gives valuable information on participants' daily engagement behaviours, even before completion of week 1 – that is, during the very first days of the MOOC. This behaviour was therefore detailed during a period of 11 days starting from the day of the kick-off hangouts video-conference, preceding by three days the official day 1 of the MOOC up to the first day of week 2 (Figure 15).
Figure 15. Group Pageviews during the first days.
The ordering previously observed for engagement in week 1's learning activities was conserved during the entire MOOC: an average or above average cluster consisting of the Bulgarian, Greek and Catalan groups and a below average cluster of English, French, Slovenian and Spanish groups. This order was already in place during the very first days of the MOOC, even on day 1. For the Greek group, group page viewing had started even the day before, on October 26th, before any official facilitation action had occurred.
Table 1 summarises the measurements recorded on participation throughout the MOOC, from day 1 up to course completion (owing to newly identified early indicators of engagement: Pageviews and Mailchimp). In addition, each of the seven language groups is given a colour code to highlight its ranking for each MOOC event. This illustrates at a glance one of our main findings: the top-three language groups in terms of completion rates (Bulgarian, Catalan and Greek), were already ranked in the three top-most positions in terms of engagement from the first days of the course, that is, before multilingual facilitation could have had any impact.
Table 1
Ranking (#1-7) and Possible Early Predictors of Course Completion for the Seven Language Groups
The HANDSON MOOC experiment sought to investigate the veracity of two hypotheses: i) whether using English only as the language of instruction negatively affects the completion rate (Hypothesis 1) and ii) whether using each participant's native (or preferred) language for facilitation (both by peers and by a dedicated facilitator) would boost participation (Hypothesis 2). We'll now confront these hypotheses in turn with the results obtained.
Hypothesis 1 can best be evaluated by comparing the HANDSON MOOC's completion rates with those obtained in the ICT in Primary Education MOOC (IPE; Laurillard, 2015) as this was a course for teacher training with instruction in English too. In agreement with the findings of Jordan (2014) who found that "approximately 50% of MOOC students who sign up go on to become active users," 55% of those who had registered at the start of the IPE course were active during the first week. With 54%, this figure was similar for HANDSON. Moreover, in both MOOCs, approximately 10% of those active in the first week received a certificate of course completion. Third, overall completion rates were almost identical (IPE 5.3% vs. HANDSON 5.4%). Looking at data on completion for other courses as compiled by Jordan (2014), at that time a 5% completion rate was about the norm. We conclude therefore that our findings confirm our first hypothesis: using English as the language of instruction does not negatively affect completion for an audience for which English is only one of the native languages present.
However, behind the overall rate of 5.4% quite a bit of diversity is hidden. Three groups (Bulgarian 10.8%, Catalan 9.9% and Greek 14.0%) outperformed the IPE reference whereas the other four groups (English 2.0%, French 0.9%, Slovenian 0% and Spanish 0.6%) underperformed with respect to IPE. Note that all these groups were facilitated in the participants' preferred (native) language. It seems that, unlike what Hypothesis 2 assumes, multilingual facilitation in some cases doubles or triples completion but in other cases more than halved it. Hypothesis 2, therefore cannot be confirmed, at least not in its present form. This of course prompts the question why this is so. In our discussion below we will discuss how both groups of languages differed and we will suggest two additional hypotheses - that our data suggest but obviously go untested - for how one may account for the observed difference between them.
Our findings suggest that it is impossible to change the course of events for learner engagement after week 1, or indeed, even after the first day. The strict separation in rankings of the high-completion cluster of the Bulgarian, Catalan and Greek groups on the one hand and the low-completion cluster of English, French, Slovenian and Spanish groups on the other hand remains intact throughout the five weeks the MOOC lasted. This suggests, we surmise, the existence of early threshold values that need to be met in order to ensure subsequent engagement in learning activities.
Of primary importance seems to be the size of the group of fully active participants during week 1, as measured by the number of week-1 badges awarded. Figure 10 shows there is a correlation (Spearman's rho equals 0.927) between final completion rates ("designer badge") and the numbers of participants completing week 1 ("total week 1 badge") (p=.003). Note that no such correlation can be established between final completion rates and the initial size of the group (Spearman's rho.083, p=.860). Indeed, the Greek group had the best post-week-1 completion rate (80%). This rate was lower in groups that were initially larger than the Greek one: English (36%), French (29%) and Spanish (12%). We suggest that the latter groups' low completion rates are due to the low numbers of active participants in week 1 (22, 7 and 16 respectively, while the Bulgarian, Catalan and Greek groups had 33, 48 and 35 active participants, respectively). One may surmise that such low numbers are insufficient to reach a critical threshold number necessary for good operation of the peer-tutoring activities (especially critical during week 1), for building a sense of community, and for ensuring engagement through facilitation of learners for the remaining four weeks ( threshold hypothesis ).
A threshold of 25-30 fully active participants appears a good approximation for the minimal number required to ensure engagement past the first week. This was only obtained in three groups (the Bulgarian, Catalan and Greek groups, cf. Table 1). Interestingly, in the typology of social systems for supporting learning (Dron & Anderson, 2014), 30 persons is the size delimiting a group from a network. Moreover, the emergence of a network-like structure is critical for the development of learning support in large online courses such as cMOOCs (Sloep, 2009; Downes, 2013). Thus staying below the size of 30 is likely to have prevented the low-completion language groups (English, French and Spanish) from adapting to the relatively unstructured learning environment of the HANDSON MOOC.
Furthermore, we suggest that threshold values separating the two clusters of high-completing and low-completing groups not only refer to the numbers of week-1 completers but also include other values: number of group Pageviews and responses to Mailchimp. As these apply even before week-1 completion, there seems to be a chronologically ordered series of predictive threshold values. This extension of the threshold hypothesis predicts that for obtaining at least the 10% completion rate observed on average for the seven groups that actually started the HANDSON MOOC, one would not only need to have at least 30 participants completing week 1 in a group (factor 1) but also, even before that, a Mailchimp (or similar) response rate of at least 80% (factor 2), or an estimated number of course Pageviews of at least 4.0 for week 1 or at least 0.75 for day 1 (factor 3). (The figure of 10% corresponds to 92/902=10.2%, cf. Figure 10; if one wants to compare with other MOOCs, this corresponds to 5.4% if all 1691 registered participants are taken into account.) It would seem therefore that only slight differences early on can have a considerable impact on final completion rates: compare for example the Catalan and Spanish groups on either side of the threshold of 0.75 for day-1 page viewing, with 0.79 and 0.68 respectively, and ending up with a completion rate of 18.6% and 1.1% respectively (Table 1). Most importantly, even though the initial size of a group (factor 1) alone does not give reliable indication on the group's future success, these indicators and thresholds (factors 2 and 3) seem to be strongly suggestive - as early as from day 1 - of the capacity of a group to organise successfully as a learning network of active participants.
The apparently required initial size of a group has some theoretical grounding in the work of Dunbar, that is, in the so-called Dunbar number (Dunbar, 1993; Hill & Dunbar, 2002). This number is used to argue that a course needs to attain about 150 active participants in order to be considered "massive," that is, to enable a "connectivist" dynamic (Downes, 2009, 2013; Dron & Anderson, 2014). The average group size in the HANDSON MOOC of 147.2 seems to qualify. Our findings suggest, however, that a group does not necessarily need 150 active participants to operate as a social learning network. 30 active participants already suffice. Groups in which fewer than 20% of those who signed up were converted into active participants during the first week ended up in the low completion cluster, regardless of whether the initial size of the group was bigger or smaller than Dunbar's number of 150. The groups of the high completion cluster all counted fewer participants than Dunbar's number of 150 and still reached at least 30 active participants thanks to a first-week conversion rate substantially higher than 20%. These observations seem to weaken the theoretical significance of Dunbar's number. However, Hill and Dunbar (2002) discern five different levels of group size, one of them being 21. That number is much closer to our finding of 30 active participants. Indeed, Sloep (2015) argues that 150 is a maximum. Above it, and probably well below it already, one needs supportive structures, such as the already mentioned ad-hoc transient communities ("pop-up communities"), to kick off and maintain a sustainable social dynamic in a learning network. It is tempting, then, to speculate that "inactive participants" are somehow also important for the emergence of a group of 30 active participants. This ties in with work that underscores the importance of lurkers (Preece, Nonneke, & Andrews, 2004) as well as with recent findings on the function of inactivity in decentralised complex systems (Charbonneau & Dornhaus, 2015). Therefore the conversion rate threshold of 20% is likely to be meaningful and might be used, together with the absolute number of active participants, as a reliable indicator of sustainable social dynamics during the first week of a MOOC.
Our discussion thus far suggests an explanation for the success of the Bulgarian, Catalan and Greek groups. But why then failed the other groups, including the initially two largest ones, to engage in the first week's activities? The Greek facilitators may have been very active and significantly more supportive at the start of the course than did the facilitators in other language groups. However, this could hardly have had an impact that explains the lead already taken by the Greek group on the day previous to the start of the MOOC, when facilitation had not officially started yet. Moreover participants surveyed only expressed an overall satisfaction with multilingual facilitation and the group space per language was found to help create a sense of community by 73% of them. Thus, differences in facilitation do not appear to explain the differences in engagement between language groups.
A more likely hypothesis for interpreting the low completion rates observed in the two initially largest, English and Spanish groups invokes the notion of a pre-existing sense of community. Indeed, in contrast to the members of the best performing Bulgarian, Catalan and Greek groups, who share a strong cultural identity, these two large groups included participants from all over the world. Other than sharing a common language they had little in common culturally speaking (this also holds true for the smaller French group). This absence of strong cultural ties and perhaps also of geographical proximity among participants and facilitators may have made the creation of a sense of community much harder, as well as the promotion of peer tutoring and participation in learning activities. Interestingly, out of the six cultural dimensions (Power distance, Individualism, Masculinity, Uncertainty avoidance, Long term orientation, and Indulgence) used to compare various countries (Nkuyubwatsi, 2014), the only dimension for which a clustering is reproduced similar to that for course completion of language groups is ‘Individualism' (not shown). The fact that a low level of individualism corresponds to the high-completion cluster of language groups in the MOOC fits well with the notion of a strong, pre-existing sense of community for these groups.
Along with this purely cultural aspect, the participants of the Bulgarian, Catalan and Greek groups may have also benefited from having been recruited to the MOOC from a common trusted source (as opposed to, for example, impersonal advertisements in MOOC lists for the other four groups). This was especially the case for the participants of the Greek group, who were recruited through the HANDSON partner Ellinogermaniki Agogi. They sent out invitation emails through the e-twinning mailing list to all primary and secondary Greek Schools. A third example of pre-existing connections comes from the Catalan participants, who were linked with each other through a common incentive: completing at least 80% of the learning activities gained them an official title from the Generalitat of Catalunya.
These three factors all point into the direction of a rational decision making process on the part of the participants: before investing time in a MOOC, one wants to make sure that the time is well spent. Lack of space forbids us to explore this community perspective in further depths (but see Kester et al., 2007). Jointly, we will call these three explanations the tried and trusted community hypothesis.
In summary, our discussion points out that for multilingual facilitation to be effective two more conditions need to be fulfilled: an above threshold group size and a tried and trusted community. They appear critical for the formation of the learning network that constitutes the basis for the social dynamics of active participation in MOOCs. Further research is needed to investigate the general validity of this claim and to refine our understanding of how the factors of multilingual facilitation, a threshold group size and pre-existing ties within the group act in concert. Regardless, to the extent that MOOCs are the new form of distance education, which goes in particular for the MOOCs emerging in Europe (Deimann & Vogt, 2015), our findings offer insights that may profitably be used both to design new distance learning offerings and to give distance education theory new inputs that still align with its original tenets and intentions of serving underprivileged and culturally heterogeneous audiences (Lundahl, 2014; Sloep & Schuwer, 2015).
We would like to thank Uroš Godnov, whose group was responsible for the analysis of the surveys in the HANDSON project, all facilitators of the seven language groups for voluntarily donating their precious time to the HANDSON project, and finally the HANDSON team as a whole for their productive collaboration. The HANDSON project - properly, the Hands-On ICT project - has been funded with support from the European Commission's Lifelong Learning Programme. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
Beaven, T., Comas-Quinn, A., Hauck, M., de los Arcos, B., & Lewis, T. (2013). The open translation MOOC: Creating online communities to transcend linguistic barriers. Journal of Interactive Media in Education, 2013(3). doi: 10.5334/2013-18
Brouns, F., Serrano Martínez-Santos, N., Civera, J., Kalz, M., & Juan, A. (2015). Supporting language diversity of European MOOCs with the EMMA platform. In M. Lebrun, M. Ebner, I. De Waard, & M. Gaebel (Eds.), Proceedings of the Third European MOOCs Stakeholder Summit (eMOOCs2015), May, 18-20, 2015, Mons, Belgium (pp. 157–165). Retrieved from http://dspace.ou.nl/handle/1820/6026
Burkhardt, H., & Schoenfeld, A. H. (2003). Improving Educational Research: Toward a More Useful, More Influential, and Better-Funded Enterprise. Educational Researcher, 32(3), 3-14. doi: 10.3102/0013189X032009003
Charbonneau, D., & Dornhaus, A. (2015). When doing nothing is something. How task allocation strategies compromise between flexibility, efficiency, and inactive agents. Journal of Bioeconomics, 17(3), 217-242. DOI 10.1007/s10818-015-9205-4
Civera, J. (2015, June 20). MOOCs in three languages on EMMA platform [Web log post]. Retrieved from http://project.europeanmoocs.eu/moocs-in-three-languages-on-emma-platform/
De Rosa, R. (2013). The Mooc (R)evolution. Where the EMMA project come from. Paper presented at "Apertura e flessibilità nell'istruzione superiore: oltre l'E-Learning?" Conference, Atti Del Convegno Perugia, 13-15 November (pp. 79-85). Reggio Emilia: Sie-L Editore. Retrieved from http://www.sie-l.it/phocadownload/Atti_siel_2014.pdf
De Vries, F. J., Kester, L., Sloep, P. B., Van Rosmalen, P., Pannekeet, K., & Koper, R. (2005). Identification of critical time-consuming student support activities in e-learning. ALT-J, 13(3), 219–229. doi: 10.1080/09687760500376488
De Waard, I., Gallagher, M. S., Zelezny-Green, R., Czerniewicz, L., Downes, S., Kukulska-Hulme, A., & Willems, J. (2014). Challenges for conceptualising EU MOOC for vulnerable learner groups. Proceedings of the European MOOC Stakeholder Summit 2014 (pp.33-42). Retrieved from http://oro.open.ac.uk/40381/2/deWaardEtAl.pdf
Deimann, M., & Vogt, S. (2015). Towards a European perspective on Massive Open Online Courses: The past, the present and the future. The International Review of Research in Open and Distributed Learning, Special Issue: Towards a European Perspective on Massive Open Online Courses, 16(6), i–iii. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2520/3546
Downes, S. (2013, January 17). What Makes a MOOC Massive? [Web log post]. Retrieved from http://halfanhour.blogspot.com.es/2013/01/what-makes-mooc-massive.html
Downes, S. (2009, February 24). Connectivist dynamics in communities [Web log post]. Retrieved from http://halfanhour.blogspot.co.uk/2009/02/connectivist-dynamics-in-communities.html
Dron, J., & Anderson, T. (2014). Teaching crowds: Learning and social media. Athabasca University Press. Retrieved from http://www.aupress.ca/index.php/books/120235
Dunbar, R. I. (1993). Coevolution of neocortical size, group size and language in humans. Behavioral and brain sciences, 16(04), 681-694. doi: 10.1017/S0140525X00032325
Fetter, S., Berlanga, A. J., Sloep, P. B., Van der Vegt, W., Rajagopal, K., & Brouns, F. (2012). Using peer-support to connect learning network participants to each other: An interdisciplinary approach. International Journal of Learning Technology, 7(4), 378-399. doi: 10.1504/IJLT.2012.052212
Fitzgerald, A., Wu, S. and Witten, I. (2014). Flexible Open Language Education for a Multilingual World. Proceedings of OpenCourseWare Consortium Global 2014: Open Education for a Multicultural World. Retrieved from http://cdlh7.free.fr/OCWC_2014/Final_papers/Paper_65.pdf
Garreta Domingo, M., Hernández-Leo, D., Mor, Y., & Sloep, P. B. (2015a). The Learning Design Studio in a 5-weeks MOOC format. In M. Garreta-Domingo, P. Sloep, S. Stoyanov, D. H.- Leo, & Y. Mor (Eds.), Proceedings of the workshop "Design for Learning in Practice", EC-TEL, Toledo, Sept. 18, 2015. (pp. 29–32). Heerlen, Nederland. Retrieved from http://hdl.handle.net/1820/6130
Garreta Domingo, M., Hernández-Leo, D., Mor, Y., & Sloep, P. B. (2015b). Teachers' Perceptions About the HANDSON MOOC: A Learning Design Studio Case. In G. Conole, C. Rensing, J. Konert, & D. Hutchison (Eds.), Design for teaching and learning in a networked world; proceedings of the 10th european conference on technology enhanced learning, EC-TEL 2015 Toledo, Spain, September 15–18, 2015 (pp. 420–427). Berlin / Heidelberg: Springer.
Godwin-Jones, R. (2014). Global reach and local practice: The promise of MOOCS. Language, Learning & Techno logy, 18(3), 5-15.
Hsiao, Y. P., Brouns, F., Bruggen, J. V., & Sloep, P. B. (2015). Effects of training peer tutors in content knowledge versus tutoring skills on giving feedback to help tutees' complex tasks. Educational Studies, 41(5), 1-14. doi: 10.1080/03055698.2015.1062079
Hill, R. A., & Dunbar, R. I. M. (2002). Social Network Size in Humans. Human Nature, 14(1), 53–72. doi: 10.1007/s12110-003-1016-y
Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length and attrition. The International Review of Research in Open and Distributed Learning, 16(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/2112/3340
Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review Of Research In Open And Distributed Learning, 15(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1651/2774
Kester, L., Sloep, P. B., Van Rosmalen, P., Brouns, F., Koné, M., & Koper, R. (2007). Facilitating community building in learning networks through peer-tutoring in ad hoc transient communities. International Journal of Web Based Communities, 3(2), 198–205. Retrieved from http://hdl.handle.net/1820/609
Kop, R., Fournier, H., & Mak, J. S. F. (2011). A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses. The International Review of Research in Open and Distributed Learning, 12(7), 74-93.
Laurillard, D. (2015). UCL IOE Report - Anatomy of a MOOC for Teacher CPD (Substantial update on 06/01/2015). Retrieved from LKL Publication Databases Website http://www.lkl.ac.uk/cms/index.php?option=com_content&task=view&id=605&Itemid=110
Lundahl, L. (2014). Challenges of educational theory in the age of knowledge capitalism. In G. Biesta, J. Allan, & R. Edwards (Eds.), Making a difference in theory (pp. 31–44). New York: Routledge.
Medina-Salguero, R., & Aguaded-Gómez, J. I. (2013). Pedagogical Support in MOOCs: A New Approach to Tutoring. @ tic. revista d'innovació educativa, 11, 30-39. doi: 10.7203/attic.11.3044
Mor, Y., & Mogilevsky, O. (2013). The learning design studio: collaborative design inquiry as teachers' professional development. Research in Learning Technology, 21. Retrieved from http://www.researchinlearningtechnology.net/index.php/rlt/article/view/22054
Nkuyubwatsi, B. (2014). Cultural Translation in Massive Open Online Courses (MOOCs). eLearning Papers, 37, 1-10. Retrieved from http://www.openeducationeuropa.eu/en/article/Cultural-Translation-in-Massive-Open-Online-Courses-(MOOCs)
Perifanou, M., Holotescu, C., Andone, D., & Grosseck, G. (2014). Exploring OERs and MOOCs for Learning of EU Languages. Paper presented at "SMART Social Media in Academia: Research and Teaching" Conference, Timisoara Romania, September 2014. Retrieved from http://www.researchgate.net/publication/276268863_Exploring_OERs_and_MOOCs_for_Learning_of_EU_Languages
Porto Declaration on European MOOCs. (2014, November 27). Retrieved from http://www.home.eadtu.eu/images/News/Porto_Declaration_on_European_MOOCs_Final.pdf
Preece, J., Nonneke, B., & Andrews, D. (2004). The top five reasons for lurking: Improving community experiences for everyone. Computers in Human Behavior, 20, 201–223.
Ross, J., Sinclair, C., Knox, J., Bayne, S., & Macleod, H. (2014). Teacher experiences and academic identity: The missing components of MOOC pedagogy. MERLOT Journal of Online Learning and Teaching, 10(1), 57-69.
Rovai, A. P. (2007). Facilitating online discussions effectively. The Internet and Higher Education, 10(1), 77-88.
Sloep, P. B. (2009). Fostering sociability in learning networks through ad-hoc transient communities. In M. Purvis & B. T. R. Savarimuthu (Eds.), Computer-mediated social networking, ICCMSN 2008, LNAI 5322 (pp. 62–75). Berlin, Heidelberg: Springer.
Sloep, P. B. (2015). Design for Networked Learning. In B. Gros, Kinshuk, & M. Maina (Eds.), The future of ubiquitous learning: Learning designs for emerging pedagogies (pp. 41–58). Berlin, Heidelberg: Springer. doi: 10.1007/978-3-662-47724-3_3
Sloep, P. B., & Schuwer, R. (2015). Of two contrasting philosophies that underpin openness in education and what that entails. In M. Deimann & M. A. Peters (Eds.), The philosophy and theory of open education. New York: Peter Lang Publishing
Stoyanov, S., Sloep, P. B., De Bie, M., & Hermans, V. (2014). Teacher-training, ICT, Creativity, MOOC, Moodle - What pedagogy? In L. Gómez Chova, A. López Martínez, & I. Candel Torres (Eds.), Proceedings of Edulearn 14, the Sixth International Conference on Education and New Learning Technologies (EDULEARN 14) (pp. 5678–5686). Barcelona, Spain: IATED Academy, IATED Digital Library. Retrieved from http://hdl.handle.net/1820/5463
Table PostPilot3 – Which Language Group Did You Join.
Table Pre-Course Activities - Facilitators Training.
| HANDSON pilot 3 / Pre-MOOC training period for facilitators (Aug. 27th - Oct. 24th, 2014) |
|
The Effect of Multilingual Facilitation on Active Participation in MOOCs by Jean-François Colas, Peter B. Sloep, and Muriel Garreta-Domingo is licensed under a Creative Commons Attribution 4.0 International License.