International Review of Research in Open and Distributed Learning

Volume 18, Number 2

April - 2017

Examining MOOCs: A Comparative Study among Educational Technology Experts in Traditional and Open Universities

Nati Cabrera and Maite Fernández-Ferrer
Universitat Oberta de Catalunya (UOC), Universitat de Barcelona (UB)

Abstract

The proliferation of Massive Open Online Courses (MOOCs) in recent years has generated much debate. MOOCs have been presented as technology-based educational practices, but many researchers question if this kind of open courses really respects some of the consolidated principles behind the education offered at universities. In light of this situation, consulting the teachers most closely tied to this type of course can provide an authoritative view of the issue and can allow the most important elements to be highlighted in order to carry out further research. Using a qualitative methodology based on an open questionnaire, this work presents the opinions and perceptions of teachers/lecturers in educational technology regarding these new courses key elements. These key elements are analysed through analysing its controversial definition, their pedagogical advantages and limitations, the functions of a tutor in a MOOC and their assessment (or accreditation). In addition, a comparison is made between the contributions of teachers from a traditional university with a face-to-face model and those from a distance university, which is based entirely on a virtual training offer and which has a greater possibility of coming into direct competition with these Massive Open Online Courses.

Keywords: Massive Open Online Courses, educational technology, open learning, e-learning, qualitative methodology

Introduction

Today's society can be characterized mainly by its consideration of knowledge as a key element in all areas of individuals' activity. In this sense, universal access to education is not only considered important and decisive and, therefore, a right of citizens, it is also viewed as a continuing need, with education being sought throughout a lifetime. All this serves as a strategy for and a guarantee of the productivity and employability of people (Bauman, 2008).

For over two decades, technologies and social networks born of the Internet itself have played an important role, as they facilitate both access to education and the constant training required, constituting what was first coined as the learning or educational society (Delors, 1996) and which later became known as the network society we know today because of the social, political, economic, and cultural changes caused by the spread of networked, digital information and communications technologies (Castells & Cardoso, 2005).

In this context, in which knowledge and technology take pride of place in state agendas and are an integral part of people's activities, more flexible forms of access to education begin to take centre stage, namely online distance education or e-learning, in all its manifestations. Since the dawn of the Net and up to the present day, movements have arisen linked to the network and to the activity that can be generated on it in terms of technology and knowledge, advocating open, shared and free use, such as free software (open source). The open source movement makes use of technological tools under the principles of freedom of use, distribution, study, and modification (Glennie, Harley & Butcher, 2012). This is a movement mainly focused on the creation, use and cataloguing of digital educational materials in order to generate reusable learning resources (Campbell, 2004).

It was against this backdrop that open educational resources (OER) were born. OER are educational resources whose digitized materials are offered freely and openly to teachers, students, and others so that they may use and reuse them in teaching, learning, and research. These materials provide non-discriminatory access and can be adapted, revised, and shared (Valverde Berrocoso, 2014). Open educational resources give right of use for academic purposes (Sánchez, 2013) and were an initiative supported by many universities around the world, so much so that a great many teachers have been trained in the creation and reuse of OER, going on to create a large number of repositories based on these resources in combination with dissemination and familiarization strategies among the academic community (Lehman, 2007).

The existence of a growing demand for flexible education options by an increasingly large amount of students from diverse backgrounds (Conole, 2013) has meant that universities are increasingly interested in expanding their online offering and in making a more efficient use of technologies. As such, in recent years, they have taken a step forward, taking advantage of the emergence of OER to create technological platforms for their integrated management (OpenCourseWare). Most recently, this evolution from open access to educational resources has led to MOOCs (Li Yuan & Powell, 2013; Valverde Berrocoso, 2014). While it is true that online courses already existed, as well as open institutions or organizations, the novelty of these courses has been their promotion of mass access and for free, which, initially, was what most surprised university leaders (Llorens, 2015).

There is a strong relation between this passing from open resources to MOOCs and an experience conducted by George Siemens and Stephen Downes based on the theory of connectivism. This theory is a reflection on how people learn in virtual open spaces and support the values ​​of sharing knowledge through technology. In fact, the first MOOC in history took place in 2008 under the title Connectivism and Connective Knowledge (or CCK08) and with the participation of 2,200 students, although only 24 were expected (Downes, 2012; Kregor, Padgett, & Brown, 2013; Siemens, 2013). Dave Cormier and Bryan Alexander of Canada called it a Massive Open Online Course, or MOOC (Siemens, 2012; Vázquez Cano, López Meneses, & Sarasola, 2013).

There are several factors that influence the motivation of students to enroll in a MOOC (Castaño, 2013; Christensen et al., 2013; Li Yuan & Powell, 2013; Hew & Cheung, 2014):

• The future economic benefits.
• The personal and professional development.
• The pleasure of learning
• The curiosity.
• The desire of students to advance in their jobs.
• The general interest in the subject.
• The possibility of interacting with thousands of students from around the world.

But, once they have enrolled, who are these students? Breslow, Pritchard, DeBoer, Stump, Ho, and Seaton (2013) conducted a descriptive study based on data generated by a MOOC course. Some of the results were that the language used for most students (67%) was English, while the second largest group was the Spanish (16%). Most of the students were between 20 and 30-years-old, although the age range was between adolescents and adults elderly; from which 88% were male. Regarding the education background of the students, 37% had undergraduate level, 28% master's degree, and 27% of secondary education. Eight percent did not state their level of education. In 2015, another study conducted by Engle, Mankoff, and Carbrey examined students profile in a Coursera MOOC. Of the 33,378 students who accessed the course, 29.5% where between 18 to 25-years of age and 30.3% 26-34 years of age. The majority (56.9%) did not speak English as their mother tongue. More than half of the students had bachelors or master's degree. Students with a natural sciences background were overrepresented and those with a humanities background were underrepresented.

The proliferation of MOOCs in recent years has generated much debate (Conole, 2013; Caballo, Caride, Gradaille, & Pose, 2014). According to some authors, these new courses are already revolutionizing the way informal learning takes place (Caballo, Caride, Gradaille & Pose, 2014) which at this moment represents an important part of open education (Hernández, 2014). Although MOOCs arguably constitute a turning point in higher education (Vázquez Cano et al., 2013), it is true that we are still at the experimental stage (Pedreño Muñoz, Moreno, Ramón, & Pernías, 2013). It is still too early to know if we can speak of a new learning methodology, as many questions remain unanswered: Are MOOCs really a new type of training? How is their contribution to the field of education different? What are their limitations? More specifically, what do experts say about these MOOC, referring to experts as teachers in educational technology?

This work, as part of the exploratory phase of a doctoral thesis on these MOOCs in the context of Spain, aims to tackle all these issues allowing to have interesting results which could be used in a larger context. While some theoretical articles already exist on the subject and data on MOOCs in higher education have begun to appear, the study presented here offers two substantial contributions. On the one hand, it consults teachers in the field of educational technology in order to know, in depth, their expert opinion on this new worldwide educational phenomenon. On the other hand, it compares whether there are differences between the perceptions of teachers of educational technology at a traditional university and teachers at a distance university that is, consequently, using technology intensively. Although both groups are experts in the use of technology in education, educational models of both universities are very different. Therefore, the teaching and the experience derived are also distinct. The fact of teaching in an environment with a traditional methodology or in a virtual environment, which makes an intensive use of ICT, can influence judgments and assessments of both groups.

Research Method

Currently there is a lack of information giving us a critical view of this important educational phenomenon, one that allows us to know its strengths and weaknesses. For this reason, this article analyses the impact of new MOOCs, comparing the perceptions of university teachers from a traditional, face-to-face university with over 500 years of history with those from a distance university, with just 20 years of history and a virtual existence. The main objective of this research is to know these educational technology teachers' opinion of MOOCs. Therefore, it proposes a qualitative methodology using open questionnaires as a tool, designed based on an interpretive paradigm that aims to deepen understanding of the elements without generalizing any of the information collected.

This questionnaire was developed based on a comprehensive bibliographic analysis to identify the dimensions and factors of this new training methodology:

Table 1

List of Dimensions Relating to New Massive Open Online Courses

(Source: Own elaboration)

This table was created in the framework of the doctoral thesis to identify the various issues that needed to be investigated during the research. Each of the dimensions were selected to be developed and studied by each of the various data collection instruments. Specifically, for the preparation of the questionnaire to be administrated to university teachers, the following dimensions were took into account: quality, accreditation, structure, design, assessment, and agents.

For the creation of the open questionnaire, a first version was validated by two professors from the University of Barcelona, taking into account the clarity and relevance of the questions. The first one was a lecturer from the Faculty of Education, expert in research and diagnosis methods in education. And the second one was a lecturer whose line of research focuses mainly on the Information and Communication Technologies (ICT) and who works on the design and assessment of interactive learning resources and on the integration of the ICT in educational settings. Based on their both answers, a final and definitive version of the questionnaire was developed.

The final questionnaire from which the results presented and analysed below were taken contained some introductory closed questions to identify the participants and five open questions on the definition of this new mode of learning, its pedagogical advantages and limitations, their teachers' or tutors' functions, and the topic of learning assessment in these new courses. In addition, at the end of the questionnaire there was a space in which the teachers of educational technology could add their considerations or final opinions:

1. What are Massive Open Online Courses?
2. What are the advantages of MOOCs in the field of education?
3. What are the educational limitations of MOOCs in the field of education?
4. Which functions should tutors or teachers in a MOOC develop?
5. And what about the assessment system in a MOOC?

The universities chosen for this research project were the University of Barcelona (UB), as a traditional face-to-face institution of higher education, and the Open University of Catalonia (UOC) as a distance and virtual institution of higher education. The sample included all teachers of educational technology from the Faculty of Education at the UB (26) and the teachers from the eLearn Center at the UOC (14).

The questionnaire developed for this study was forwarded to teachers of educational technology, requesting their cooperation in this research project during the months of February, March, and April of 2013 and 2014. This questionnaire was given online using one of the Google platforms, allowing testing to be carried out easily, guaranteeing the anonymity of the participants and encouraging better processing of basic data statistics with the help of Excel.

Once the answers had been collected, the analysis of the results was carried out based on a categorization process in which a set of meaning units were grouped into categories in order to meet the objectives for which the questionnaire had been intended. For this phase, a nonlinear process based on a constant comparison method and units of meaning and categories from Glaser and Strauss (1967) were followed.

The analysis resulting from this categorization process allowed us to determine the similarities and differences between the opinions and perceptions of educational technology teachers from a traditional university and those from a distance university regarding MOOCs, their pedagogical advantages and disadvantages in the field of education, the role of teachers in them, and, finally, the assessment of learning in this new mode of learning.

Analysis of Research

From the entire sample, a total of 16 teachers from the University of Barcelona and 11 from the Open University of Catalonia answered the open questionnaire. Regarding the characteristics of the participants, the majority were assistant lecturers, in the case of the UB, or lecturers, for both universities, with no representation of full professors for the UOC or postdoc researchers for the University of Barcelona (UB). Of particular note were the 45.5% of senior lecturers in the case of the Open University of Catalonia (UOC), as we can see in the chart shown below:

Figure 1. Comparison of the profile of the participants from the traditional university and the distance university.

Furthermore, in relation to the years of experience of the teachers participating in the research, the teachers of the UB had an average of 14.9 years of experience, while in the case of the UOC, the average was 16.5 years, so the averages were practically the same. Other characteristics of the sample included information on teachers' participation in educational technology through a MOOC. In this case, the participation rate was low for both groups. There is a greater level of involvement in this new type of course for teachers from the distance university, both as teachers and tutors; however, participation as students was much higher in both universities. About half of the respondents said they had not participated in a MOOC in any way, this being significantly higher in the case of the traditional, face-to-face university.

Figure 2. Comparison of participation by teachers from a traditional university and a virtual university in a MOOC.

What are Massive Open Online Courses?

According to the responses of educational technology teachers, MOOCs are related with the concepts of virtuality and massiveness and, while the traditional university also highlights the issue of accessibility and openness, it is the distance university that repeatedly emphasizes the amount of information that this new learning tool can provide. In addition, as we can see in the images below, in defining this new type of course, the technological aspect was far less prominent for the University of Barcelona. However, while the issue of diversity of knowledge, issues and learning interests was an element that did not appear in the Open University of Catalonia's definition, it was quite significant for teachers of educational technology from the traditional university.

Figure 3. Comparison of the "tag clouds" for the definition of MOOCs used by teachers from a traditional university and a virtual university.

What are the Advantages of MOOCs in the Field of Education?

When asked which they felt were the benefits of MOOCs in the teaching and learning field, the items most valued by teachers participating in this study from the traditional university were accessibility and openness (43.8%) and the quality resources (43.8%). The concept of accessibility means that the highest forms of learning are possible for the maximum number of people (Davidson, 2012), and MOOCs currently have a large number of students (Bartolomé, 2014) and learners around the world (Carey, 2012), with all the benefits that this entails. Moreover, according to some authors, this type of education could be viewed as a proposal for high-quality higher education (Fernández Cruz, 2014) that allows any tool to become available online (MoocGuide, 2011), including a selection of learning materials and resources for the personal use of their participants.

University teachers from the virtual university mainly highlighted the variety of emerging and updated topics (45.5%) offered by this new mode of learning, and, indeed, many studies show that most of the users who participate in a MOOC (87%) generally do so because they are interested in the general topic of the course (Castaño, 2013).

I would highlight the open and quality component that these type of courses offer. Open in the sense that they can make available well-selected content to everyone, and this can promote learning among anyone interested. The MOOCs are there waiting for someone to subscribe. Furthermore, the fact that they are virtual enables them to really be massive (P2 UB).

Note that all the quotations of the participants in this research will be cited as follows: A "P" (participant) and the corresponding number to indicate which participant is, and then the acronym for the university to which it belongs. For example, in this quotation, the participant number 2 of the University of Barcelona as we quote: (P2 UB).

The overlapping aspects pointed out by teachers from both universities (UB, traditional university, and UOC, distance and open university) were the courses' massiveness (18.8% UB; 18.2% UOC); the fact that they are free (12.5% UB; 18.2% UOC); online (18.8% UB; 18.2% UOC); that they complement vocational education (31.3% UB; 27.3% UOC); and the way these new courses promote the teaching profession (6.3% UB; 9.1% UOC). As we can see in the chart below, the greatest difference between the two universities, in relation to the advantages of these new courses, were the democratization of education (25% UB; 9.1% UOC); the collaborative learning network they may offer (25% UB; 9.1% UOC); the variety of emerging and updated topics (25% UB; 45.5% UOC); and quality materials and resources (43.8% UB; 18.2% UOC).

Figure 4. Comparison of the perceptions of teachers from both universities about the benefits of the new MOOCs.

Specifically, on the issue of the democratization of education through MOOCs (in regard to accessibility and openness), this was valued highly by University of Barcelona teachers but was given significantly less importance by Open University of Catalonia teachers. One remark in particular stands out:

It is said that they allow access to education for many people with financial difficulties, who cannot attend college. But recent research shows that a high percentage of those following a MOOC are people who already have a higher level of education, or belong to wealthier social classes in their respective countries (P7 UOC).

It is true that the new MOOCs could offer educational opportunities to millions of individuals around the world. However, when one considers the skills and abilities necessary in order for students in developing countries to make use of a MOOC, this theory is easily challenged (Liyanagunawardena et al., 2013). The MOOCs are free courses open to anyone with Internet access (Haynie, 2014), and open is not the same as free. Open means accepting those who want to learn, which is not the same as being ready to learn. Some people will be able to complete the courses on their own, but this is not the only requirement for inclusive education (McAndrew & Scanlon, 2013).

Staying on this topic, it is important to highlight some of the comments from participants referring to the lack of research that exists so far and the need for MOOCs to be questioned or the claim that these courses offer the same advantages as those already offered by e-learning:

It remains to be seen. To answer this question we need more and more research and evidence. If there is anything that I have learned over my past years as a student and as a researcher in the field of educational technology it is to stop placing hopes on new formats and technologies. Because so often the teaching and learning technologies have the problem of not being questioned (P10 UB).

The same as e-learning (P10 UOC).

It is true that, in regard to this new mode of learning, no serious study of the market or definition of a clear business model has been undertaken nor is there any empirical research that reinforces the statements about these new courses' positive impact on institutions (Christensen et al., 2013). Furthermore, none of these models have developed enough efficient mechanisms to gauge the learning attained by participants at the end of the process (Caballo, Caride, Gradaille, & Pose, 2014). Thus, there is now a very real gap in the examination of the quality of the instructional design of MOOCs (Margaryan, Bianco, & Littlejohn, 2014), and, therefore, some authors argue that today we need to talk more about a mode of disclosure than of learning (Sangrà, 2013).

What are the Educational Limitations of MOOCs in the Field of Education?

The third question in the questionnaire for teachers of educational technology was related to the pedagogical limitations of MOOCs in the field of education.

Figure 5. Comparison of the perceptions of teachers from both universities about the pedagogical limitations.

In relation to this aspect, several statements were made attesting that the disadvantages or limitations lie not in the courses themselves but in the use made of them in the context of higher education:

The limitation is not with the courses offered in this format but in thinking of training where you simply have this type of course (P4 UB).

Obviously it has some limitations, but it also has many possibilities. It depends more on the adaptation or use made of these courses (P13 UB).

Specifically, the main pedagogical limitation that most teachers from both universities considered MOOCs to have was in terms of learning assessment (43.8%; 54.5%). Specifically, in the case of the distance university, this had to do with the model of continuous assessment on which this institution of higher education is based:

The lack of a personalized feedback offered by the experts. This part ends up being the students' own responsibility and depends on their capacity to analyse their work and interact with the other students of the course (P4 UOC).

The main criticism in terms of the educational value of MOOCs was that they do not go beyond their basic offering, namely a course based on a traditional class segmented into audiovisual presentations of no more than 15 minutes in which the competence level of the student must rely almost exclusively on rote or conceptual learning and its mechanical assessment through trial and error (Vázquez Cano et al., 2013; SCOPEO, 2013; Fernández Cruz, 2014; Valverde Berrocoso, 2014; Bartolomé & Steffens, 2015). A big problem with a MOOCs is the lack of contact with the teacher, less interaction (Sangrà, 2013) and the fact that there is no real guide or support for learning (Holton, 2012), as MOOCs are characterized precisely by the absence of immediate feedback for students (Hew & Cheung, 2014; Naidu & Barberá, 2014).

Other elements that were identified as a big problem of this new learning tool were the massive size of the group (25%), for the University of Barcelona, and the lack of personalization, for the Open University of Catalonia (45.5%):

There are more resources than courses and they are designed from a standard point of view, not with attending to the specific or individual needs of students in mind (P5 UOC).

Some authors who are more critical of the quality of MOOCs doubt their educational value and emphasize the lack of attention to individual differences (Bartolomé, 2014; Valverde Berrocoso, 2014). The big problem with this type of course is the massive treatment of students without any personalization (Vázquez Cano et al., 2013, p. 33). In this new type of learning, left out of the design is the need to incorporate individual differences to meet the demands of students, basing it instead on a standardization of knowledge (Valverde Berrocoso, 2014) and a single assessment proposal for the wide array of characteristics that participants may have (Gallego Arrufat et al., 2014; Cabero, Llorente, & Vázquez, 2014).

One of the research participants summarized the limitations of this new mode of learning as follows:

The little amount of personal attention, the deficit of collaborative peer learning and the assessment mechanisms that return to the past are a good way to sum it up (P7 UOC).

Although we can not assume that the lack of personal attention is a problem exclusively for MOOCs, because this statement had to be taken in comparison with the personal attention in traditional classes, we can say it is one of the aspects that are not usually taken into account when the instructional design is done.

Which Functions Should Tutors or Teachers in a MOOC Develop?

In a fifth question, teachers were asked about the functions that teachers or tutors in a MOOC should develop. Two participants, one from each university, claimed that they should have the same role as in any face-to-face or blended course. Moreover, there was a participant from the traditional university who stated that he did not believe in the role of tutor in the context of this new learning scenario, assuming that teachers in traditional courses do have control and that they do meet diverse needs, while another individual felt that this was one of the key elements of the new courses.

I do not really believe in the role of the teacher in a MOOC. Teachers in these types of courses have no control over the methodological sequence and they don't know the students well enough to meet the diverse interests, paces and needs (P9 UB).

Here is the problem of sustainability in a MOOC. That it requires teacher intervention. Being massive courses with a very varied profile of participant, it becomes very difficult for teachers to manage feedback, interact, etc. (P5 UB).

Despite this, teachers of educational technology felt that teachers in a MOOC should have a guiding and supportive role and should provide feedback, an opinion that was shared among the participants in the study from both universities.

Accompanying, guiding the whole process of teaching and learning... Teachers following the MOOC philosophy should be open, so they can also learn from others. It is a joint construction and a sharing process. Because it is assumed that in some courses the student may become the teacher and vice versa (P15 UB).

In this new type courses we did not encounter the figure of tutor as such (SCOPEO, 2013). The appearance, immediate dissemination and evolution of MOOCs has brought with them new educational tasks, the characteristics of which still have no fixed parameters (Torres-Díaz, Infante Moro, & Valdiviezo Díaz, 2014). In this context, different roles come about which teachers must integrate into their teaching development. These roles are very specific in nature, for example: a) creators and distributors of content for learning; b) tutors and advisers in understanding concepts and processes, recognizing their significance in the learning process and putting knowledge into practice; c) evaluators of student learning; d) counsellors to guide and facilitate the achievement of the training objectives in academic programmes; e) competency certifiers through the recognition of all kinds of credentials, including university degrees (Rasmussen, 2013).

The two images below reflect the similarities and differences between the perceptions of teachers from each university regarding the role of a tutor or a teacher in this new learning scenario:

Figure 6. Comparison of the "tag clouds" for the definition of teachers' or tutors' functions in a MOOC used by teachers from a traditional university and a virtual university.

Defining the functions or roles of a tutor in this new learning environment was precisely where teachers from the virtual university saw a major difference between MOOCs and the online education or e-learning in which they were involved in their daily routine:

A teacher should be an expert on the topic and guide the learning process, to give guidance to students, to correct mistakes and to explain how to solve them and to promote the regulation of student learning. Obviously, these functions are impossible to develop with thousands of students. That is the difference between a MOOC course and online teaching and learning (P1 UOC).

And What about the Assessment System in a MOOC?

This last question of the study referred to an important aspect of MOOCs: the assessment of learning. Assessment and feedback in a MOOC are a key factor (Sancho, 2013), as these courses' main objective is to support learning and student achievement (Chauhan, 2014). It was in the answers to this question where the major differences between the views and perceptions of the traditional university and the virtual university teachers were found. First of all, the answers showed how teachers of educational technology at the UOC tended to be more against MOOC assessment systems, simply because many of them did not even consider them to be courses:

I have no idea if they need an assessment system. I think they do need learning activities to ensure student learning (P3 UOC).

I honestly do not think you have to incorporate any system of learning assessment. I personally think that MOOCs are the equivalent of buying a workbook and learning about a topic. I do not think these courses enable anyone to do anything (besides learn more about a topic), and for this reason it makes little sense to spend time on an assessment system (P9 UOC).

However, some traditional university teachers have also criticized the issue of assessment in this new type of learning:

I find it hard to think of assessing a course that is being studied voluntarily. There is, however, a need to validate if the course has truly been used well or not, and in some cases there is a need for its certification (P14 UB).

Despite this, the participants of this study named many strategies or instruments that would be important to incorporate in the assessment system of this new type of learning. On the one hand, several teachers from both universities assured that this would depend on the purpose these new courses were meant to serve (teaching and learning or certification):

The assessment system depends on what is intended. In general, I would say that MOOCs aim to certify knowledge and, therefore, a summative assessment would be enough. If it is intended for students to develop certain skills and self-assess themselves, then a formative assessment would also be necessary (P1 UOC).

On the other hand, the elements that most appeared in relation to the assessment of learning in MOOCs were the importance of self-assessment (25%; 45.5%) and peer review (31.3%; 27.3%), whereas a number of other aspects were highlighted by the teachers from the traditional university but not by those from the Open University of Catalonia, such as continuous assessment (25%; 9.1%) or the use of questionnaires, reflective activities, group work and cross-cutting projects (12.5%).

It is one of the pending tasks of any e-learning system. Alternative assessment systems are required to store information about students' achievement from different perspectives and using different tools. The more diverse the sources of information for the assessment used, the more accurate the assessment. Some people believe that with four automated solutions the problem is solved, but they are so wrong. There is much more work to do (P7 UOC).

Among the assessment systems currently used in a MOOC, the most common are automated multiple-choice tests (Conole, 2013), peer review for problem solving or P2P (peer-to-peer) testing (Raposo Rivas, 2014) and automated assessment tests (Sancho, 2013). xMOOCs in particular are based primarily on materials and interactive videos, in addition to these multiple-choice questionnaires (Conole, 2013). Moreover, MOOCs have involved students in the process of learning assessment through strategies such as self-assessment and peer assessment, thereby contributing to a renewal of the existing assessment models typically used in higher education (Valverde Berrocoso, 2014). These two approaches appear more and more frequently in e-learning in general and in MOOCs in particular (Gallego Arrufat et al., 2014).

Conclusion and Future Lines

This work highlights the importance of a taking a critical view of the educational phenomenon known as Massive Open Online Courses, which allow their strengths and weaknesses to be revealed and specific issues that merit further, in-depth analysis to be identified, in order to better understand MOOCs in higher education.

The results of this research, which is an initial exploratory phase of a doctoral thesis on this new type of learning and its assessment, show that there are clear and obvious differences between the perceptions and opinions of teachers of educational technology from a traditional university and those from a distance university. Traditional university teachers are concerned about issues that are not even raised by virtual university teachers, and vice versa. This could be because, for institutions that have already had an online offering or hybrid courses for some time, MOOCs simply represent one more step in a long history rather than a great innovation. Many of the professors from these universities even slightly consider this new mode of learning as regressive from the point of view of the quality of learning, because, basically, it fails to incorporate best practices in online education and also because it is repeating many already-existing errors that occur when educating on a large scale (Hollands & Tirthali, 2014).

In this regard, it is noteworthy, for example, that once asked about the role of the teacher or tutor in a MOOC, teachers from the Open University of Catalonia highlighted functions such as course management, creating resources, guidance, support, and feedback, all important tasks in the configuration of an online teacher profile, depending on the role to be played. However, for teachers from the University of Barcelona, the most important function was to guide and accompany, while feedback and the elaboration and creation of resources were tasks left in the background, with the management of these new courses not even appearing. The new role of teachers in virtual, open, and massive spaces of learning is seen differently by each group, and this will surely be one of the keys to the present and future livelihood of MOOCs. Certain questions must be posed that force us to think about the new role of the teacher in this type of educational scenario, what functions they should have, and what value they can add to the learning process as it takes place in a MOOC.

Although they differed on certain points, learning assessment activities, as a key pedagogical limitation of the new MOOCs, was clearly the element that teachers from both universities agreed on the most. It was also an aspect that both groups of teachers believed was important in determining if MOOCs were indeed a new mode of learning or not. This result suggests the need to explore massive assessment systems, viable online, to ensure the survival of these courses as a new approach to teaching and learning in the future. In this sense, and in light of the results, it is only for a select number of courses or uses that assessment activities represent a mere exercise to certify what has been learned. Assessment must go beyond this and represent an activity that promotes learning itself, thus providing students with the information they need about the process and, even more importantly, information on how to improve it. Students who participate in a MOOC are motivated to learn about a topic and they need the learning model to provide them with the correct educational tools. This view of assessment matches the vision teachers have of the role of a tutor in a MOOC: to guide, to accompany, and to provide feedback. Accordingly, a starting point for promoting such a formative assessment of learning and the required feedback would be to find ways to facilitate access to teachers and their assistants (Jacobs, 2013), although the feasibility of these strategies is difficult to guarantee with the current media.

In any case, the perceptions of teachers of educational technology from both the traditional and distance university regarding these new MOOCs show different views of what the teaching and learning process should be, what students should do on their own in this new learning scenario, the ways that they should be assessed and how to design the different contents (Cabero, Llorente, & Vázquez, 2014).

Some future lines have been also identified from this research. In regard to the students' need for information about the learning process, there is the possibility of introducing learning analytics, data mining, or the analysis of social networks to develop and refine automated assessment and feedback techniques (Chauhan, 2014). Using these techniques would also allow progress to be made in these massive courses, because it would make it possible to adapt, at least in part, the virtual space, the resources and the activities themselves to the specific needs and characteristics of each student, in order to achieve greater learner performance and satisfaction. These strategies perhaps could also help to reduce dropout rates in these new courses (Torres-Díaz, Infante Moro, & Valdiviezo Díaz, 2014).

This study point out a need in the future to detect opportunities to carry out exploratory research that includes the validation of teaching strategies and that further studies the motivation and participation of students on these courses (Hollands & Tirthali, 2014).

Moreover, by way of another clear future line of research, it is important to take into account that the new MOOCs are a great opportunity to experience the way students communicate, exchange information, collaborate, and learn (Raposo Rivas, 2014), and, ultimately, they are tools with which to better understand how to teach and how to learn (Bartolomé, 2014). Due to their online nature and the massive data generated by the students, this new mode of learning is really an opportunity to rethink what is central to teaching and learning processes, analysing what the most appropriate methodologies to create, share, and transfer knowledge, as well as the most appropriate designs and learning management processes, are, always giving more value to curricula that is open and flexible (Caballo, Caride, Gradaille, & Pose, 2014).

References

Bartolomé, A. (2014). Qué se puede esperar de los MOOC [What to expect about MOOCs]. Comunicación & Pedagogía, 40-55.

Bartolomé, A., & Steffens, K. (2015). Are MOOCs promising learning environments? Comunicar, 44, 91-99.

Bauman, Z. (2008). Los retos de la educación en la modernidad líquida [The challenges of education in liquid modernity]. Barcelona: Gedisa.

Breslow, L., Pritchard, D., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying Learning in the Worldwide Classroom Research into edX's First MOOC. Retrieved from http://www.rpajournal.com/dev/wp-content/uploads/2013/05/SF2.pdf

Caballo, M. B., Caride, J. A., Gradaílle, R., & Pose, H. M. (2014). Los massive open on line courses (MOOCS) como extension universitaria [Massive open online courses as a university extention]. Profesorado. Revista de Currículum y Formación del Profesorado, 18(1), 43-61.

Cabero, J., Llorente, M., & Vázquez, A. I. (2014). Las tipologías de MOOC: su diseño e implicaciones educatives [The typologies of MOOC: its design and educational implications]. Profesorado. Revista de Currículum y Formación del Profesorado, 18(1), 14-26.

Campbell, L. (2004). Engaging with the learning object economy. In A. Littlehorn (Ed.), Reusing online resources: A sustainable approach to e-learning. London: Routledge.

Carey, K. (2012). The MOOC-led meritocracy. The Chronicle of Higher Education. Retrieved from http://chronicle.com/blogs/conversation/2012/08/13/the-mooc-led-meritocracy/

Castaño, C. (2013). Tendencias en la investigación en MOOCs. Primeros resultados [Trends in research in MOOCs. First results]. Retrieved from http://ikasnabar.com/papers/castano1

Castells, M., & Cardoso, G. (2005). The network society: From knowledge to policy. Washington, DC: Johns Hopkins Center for Transatlantic Relations.

Chauhan, A. (2014). Massive open online courses (MOOCS): Emerging trends in assessment and accreditation. Digital Education Review, 25, 7-17.

Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. J. (2013). The MOOC phenomenon: Who takes massive open online courses and why? Philadelphia: University of Pennsylvania. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2350964

Conole, G. (2013). MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs. Revista Campus Virtuales, 2(2).

Davidson, C. N. (2012). Size isn't everything. For academe's future, think mash-ups not MOOC's. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Size-Isnt-Everything/136153

Delors, J. (1996). La educación encierra un tesoro [The education holds a treasure]. Madrid: Santillana

Downes, S. (2012). The Rise of MOOCs. Retrieved from http://halfanhour.blogspot.ca/2012/04/rise-of- moocs.html

Engle, D., Mankoff, C., & Carbrey, J. (2015). Coursera's introductory human physiology course: Factors that characterize successful completion of a MOOC. International Review of Research in Open and Distributed Learning, 16(2), 45-68.

Fernández Cruz, F. J. (2014). Los MOOC dentro del abanico de posibilidades de la formación online [MOOCs within the range of possibilities of online learning]. EDUCAWEB. Retrieved from http://www.educaweb.com/noticia/2014/01/13/mooc-dentro-abanico-posibilidades-formacion-online-7974

Gallego, M. J., Gámiz, V., & Gutiérrez, E. (2014). La evaluación en el diseño pedagógico y desarrollo de MOOC [The assessment in the pedagogical design and development of a MOOC]. In Gtea. Universidad de Málaga (Ed.), Creación de cursos MOOC con anotaciones multimedia. Málaga: Universidad de Málaga.

Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: strategies for qualitative research. New York: Aldine Publishing Company.

Glennie, J., Harley, K., & Butcher, N. (2012). Introduction: Discourses in the development of OER practice and policy. In J. Glennie, K. Harley, N. Butcher, & T. van Wyk (Eds.), Open educational resources and change in higher education: Reflections from practice. Vancouver: Commonwealth of Learning.

Guàrdia, L., Maina, M., & Sangrà, A. (2013). MOOC design principles. A pedagogical approach from the learner's perspective. eLearning Papers, 33, 1-6.

Haynie, D. (2014). State Department hosts 'MOOC Camp' for online learners. Retrieved from http://www.usnews.com/education/online-education/articles/2014/01/20/state-department-hosts-mooc-camp-for-online-learners-abroad

Hernández, D. (2014). cTaller: Avaluació dels aprenentatges i certificació [Workshop: Learning assessment and certification]. Barcelona: Universitat Pompeu Fabra.

Hew, F.H., & Cheung, W.S. (2014). Students' and instructors' use of massive open online courses (MOOCs): Motivation and challenges. Educational Research Review, 12, 45-58.

Hollands, F. M., & Tirthali, D. (2014). MOOCs: expectations and reality. Full report. New York: Center for Benefit-Cost Studies of Education, Teachers College, Columbia University.

Holton, D. (2012). What's the "problem” with MOOCs? EdTechDev. Retrieved from https://edtechdev.wordpress.com/2012/05/04/whats-the-problem-with-moocs/

Jacobs, A. J. (2013). Two cheers for Web U! New York Times, 162(56113), 1-7.

Kregor, G., Padgett, L., & Brown, N. (Eds.). (2013), Technology enhanced learning and teaching. Hobart: Tasmanian Institute of Learning and Teaching, University of Tasmania.

Lehman, R. (2007). Learning object repositories. New Directions for Adult and Continuing Education, 113, 57-66.

Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCS: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning, 14(3).

Llorens, F. (2015). Dicen por ahí… [They say there…]. Revista de Investigación en Docencia Universitaria de La Informática, 8(1), 9-10.

Margaryan, A., Bianco, M., & Littlejohn, A. (2014). Instructional Quality of Massive Open Online Courses (MOOCs). Computers & Education, 80, 77-83.

McAndrew, P., & Scanlon, E. (2013). Open learning at a distance: lessons for struggling MOOCs. Science, 342, 1450-1451.

MoocGuide (2011). Benefits and challenges of a MOOC. Retrieved from http://moocguide.wikispaces.com/2.+Benefits+and+challenges+of+a+MOOCMorris

Naidu, S., & Barberà, E. (2014). Special themed issue: Assessment and accreditation in MOOCs. Digital Education Review, 25, 1-6.

Pedreño, A., Moreno, L., Ramón, A., & Pernías, P. (2013). La crisis del modelo actual. Los MOOC y la búsqueda de un modelo de negocio [The crisis of the current model. MOOCs as a search of a business model]. Revista Científica de Tecnología Educativa, 2(2), 54-65.

Raposo Rivas, M. (2014). Orientaciones pedagógicas para los MOOC [Pedagogical guidelines for MOOCs]. In Grupo de Investigación Gtea (Org.), III Workshop internacional sobre Creación de MOOC con anotaciones multimedia. Retrieved from http://gtea.uma.es/congresos/?page_id=417

Rasmussen, C. (2013). When Fathom Met MOOC: Love and Learning in the OER Marketplace. Presented in Annual Meeting of the American Educational Research Association, San Francisco, California. Retrieved from http://www.mhec.org/sites/mhec.org/files/20130430aera_rasmussen_remarks.pdf

Sánchez, M. (2013). Los MOOCs como ecosistema para el desarrollo de prácticas y culturas digitales [MOOCs as an ecosystem for the development of digital practices and cultures]. Campus Virtuales, 1(2), 112-123.

Sangrà, A. (2013). Luces y sombras de los MOOC [Lights and shadows of MOOCs]. Investigación y ciencia, 444, 38-39.

SCOPEO. (2013). SCOPEO informe No2: MOOC: Estado de la situación actual, posibilidades, retos y futuro. Salamanca [SCOPEO report No2: Status of the current situation, possibilities, challenges and future]. España. Retrieved from http://scopeo.usal.es/wp-content/uploads/2013/06/scopeoi002.pdf

Siemens, G. (2012). MOOCs are really a platform. Retrieved from http://www.elearnspace.org/blog/2012/07/25/moocs-are-really-a-platform/

Siemens, G. (2013). Neoliberalism and MOOCs: Amplifying nonsense. Retrieved from http://www.elearnspace.org/blog/2013/07/08/neoliberalism-and-moocs-amplifying-nonsense/

Torres-Díaz, J. C., Infante Moro, A., & Valdiviezo Díaz, P. (2014). Los MOOC y la masificación personalizada [MOOCs and the personalized massification]. Profesorado. Revista de Currículum Y Formación Del Profesorado, 18(1), 63-72.

Valverde Berrocoso, J. (2014). MOOCS: Una visión crítica desde las Ciencias de la Educación. Profesorado. Revista de Currículum y Formación del Profesorado, 18(1), 9-111.

Vázquez Cano, E., López Meneses, E., & Sarasola, J. L. (2013). La expansión del conocimiento en abierto: Los MOOCs [The expansión of open knowledge: the MOOCs]. Barcelona: Octaedro.

Vázquez-Cano, E., & Sevillano, M. L. (2013). ICT strategies and tools for the improvement of instructional supervision. The Virtual Supervision. The Turkish Online Journal of Educational Technology, 12(1), 77-87.

Vázquez, E. (2012). Mobile learning with twitter to improve linguistic competence at secondary schools. The New Educational Review. 29, 134-147.

Yuan, L., & Powell, S. (2013). MOOCs and disruptive innovation: Implications for higher education. eLearning Papers, 33, 1-8.

Examining MOOCs: A Comparative Study among Educational Technology Experts in Traditional and Open Universities by Nati Cabrera and Maite Fernández-Ferrer is licensed under a Creative Commons Attribution 4.0 International License.