International Review of Research in Open and Distributed Learning

Volume 18, Number 2

April - 2017

Student Voices on the Roles of Instructors in Asynchronous Learning Environments in the 21^st Century

Pilar Gómez-Rey¹, Elena Barbera¹, and Francisco Fernández-Navarro^2
1Department of Psychology and Education, Universidad Oberta de Catalunya, Barcelona, Spain; ²Department of Quantitative Methods, Universidad Loyola Andalucía, Sevilla, Spain

Abstract

This paper determines which instructional roles and outputs are important in the 21^st century from the perspective of students in asynchronous learning environments. This research work uses a literature review, in-depth interviews with experts, and a pilot study with students to define the instructors' outputs. Following this, roles are determined by using a quantitative methodology (in a sample of 925 students). To our knowledge, the remaining research works on this topic identify the online instructors' roles by a qualitative analysis. The findings suggest that a new role, the life skill promoter, has emerged. Furthermore, analysis of the remaining roles (pedagogical, designer, social, technical and managerial) showed that: (i) online instructors are, first and foremost, pedagogues; (ii) the design of the particular online program influences the pedagogical and designer roles and; (iii) the managerial role has declined in importance over the years due to the development of more intuitive and transparent online scenarios from the beginning of the course onward.

Keywords: asynchronous learning environments, higher education, instructors' roles, life skills, student's perceptions

Introduction

The post-2015 education agenda encourages debate about the future development of education worldwide, with the goal of improving the quality of education. Enhancing the quality of education requires monitoring the existing quality of education in a wide range of areas, including the quality of the online learning experience (Gómez-Rey, Barbera, & Fernández-Navarro, 2016) or the students' satisfaction with the course (Kuo, Walker, Belland, & Schroder, 2013). In particular, this study focuses on determining which instructional roles are important in the 21^st century from the perspective of students in asynchronous learning environments. Traditionally, instructors' roles have been defined using a qualitative approach in which they are defined in a first stage and then operationalized in a second stage (i.e., a top-down approach). Furthermore, the instructors' roles vary depending on the context (e.g., primary, secondary, postsecondary), content (e.g., science, literature, history), and course goals (e.g., mastery of content, acquisition of skills, problem solving). These two facts impedes reaching consensus in the number of roles that instructors should have when teaching.

This paper uses a set of instructors' roles that were identified after an extensive literature review, in-depth interviews with experts in the field, and a pilot study which analysed students' perceptions. Furthermore, unlike state-of-the-art studies in which instructors' roles are defined a priori using qualitative approaches, identification of instructors' roles in this study was completed using statistical models (exploratory and confirmatory factor analysis). Using this approach also enables us to define the existing relationships/correlations among the roles identified. Finally, it is worth noting that this study uses students' perceptions as the main data source, as such perceptions are at the core of the new learning paradigms (Schweisfurth, 2015). The validity of assessing instructors using students' perceptions has been studied since the late 20th century (Millman & Darling-Hammond, 1989).

Literature Review

Since 1994, online instructor roles, competencies, and outputs have been studied first in distance education scenarios and second in online education environments. For purposes of this study, a role is defined as "a major function which is performed by individuals in a specific field or profession," a competency is defined as "an area of knowledge or skills which is critical to the production of key outputs," and an output is defined as "a product, service, condition, and/or information which results from performing a specific role" (Thach & Murphy, 1995, p. 58).

Thach and Murphy (1995) was an early study that analysed instructors' roles in distance education in which the authors identified 11 online instructor roles (Figure 1) and 10 competencies, including interpersonal communication, planning skills, collaboration/teamwork skills, among others. Experts were in charge of determining these roles, competencies, and outputs through a Delphi study and, finally, a ranking of these roles was also provided, based on the experts' ratings. In subsequent years, Williams (2003) extended the previously mentioned study and added two new emerging instructors' roles to the 11 proposed by Thach and Murphy (1995). These two new roles were the leader/change agent (as distinct from the administrative manager) and trainer. Thach and Murphy (1995) and Williams' (2003) studies adopted the same methodological and reiterated the importance of interpersonal communication and interaction between instructor and learners, thereby emphasizing their influence on the evolution of distance education. Taking a different perspective, Berge (1995a) developed an educational framework (based on a review of the literature) for the roles of a computer conference moderator in distance education based on the following four pillars: pedagogical, social, managerial, and technical. This framework has been extensively adopted in numerous studies, thus reinforcing its potential and importance (Abdulla, 2004; Liu, Bonk, Magjuka, Lee, & Su, 2005).

In recent years, the topic has also been discussed in the distance education literature to analyse whether recent changes in society and education have or have not influenced instructors' roles. For instance, Bezuidenhout (2015) analysed the roles of distance educators' that may impact their perceived workloads. Four main pillars of an academic job were identified in this study: (1) teaching and learning, (2) research, (3) academic citizenship, and (4) community engagement.

With respect to the online learning literature, some studies are of special interest to the goals of this study. These studies include the works of Goodyear, Salmon, Spector, Steeples, & Tickner (2001), Abdulla (2004), Carril, Sanmamed, & Selles (2013) and González-Sanmamed, Muñoz-Carril, & Sangra (2014). In line with the two previously described papers (Thach & Murphy, 1995; Williams, 2003), Goodyear et al. (2001) also studied how experts evaluated the instructors' outputs. They identified 8 roles, different outputs associated with each role, and 23 competencies. Abdullah's PhD thesis later studied the same research topic (roles, competencies and outputs) (Abdulla, 2004) and proposed a new approach to ranking instructors' roles. Abdulla was concerned that there were as yet no studies in which the instructors' roles were analysed using students' perceptions. Motivated by this research gap, Abdulla (2004) extended Thach & Murphy (1995) work using students' perceptions and by adopting Berge's role-based educational model (Berge, 1995a).

Recently, the topic has also been analysed in online scenarios. For instance, the work of Carril et al. (2013) analysed the pedagogical roles and outputs required of online instructors at the University of A Coruña. Through a review of 14 research papers, the author created a table of online teaching roles associated with each study. After determining these roles, the next step was to associate the roles of online instructors with their teaching outputs. A focus group composed of 9 experienced instructors was charged with determining the relation between roles and outputs. A study by Gonzalez-Sanmamed et al. (2014) determined online university instructors' levels of proficiency in their peripheral roles (such as social, evaluative, managerial, technological, counselling, personal, and research). The sample was composed of instructors from a Spanish university. This university did not offer online programs but did offer blended learning.

The number and categorization of instructors' roles are generally determined in the literature according to experts' reports/feedback (Thach & Murphy, 1995; Bezuidenhout, 2015). Some authors simply adopt a well-known, role-based educational model, thus inheriting the number and categorization of the roles according to the model elected (Abdulla, 2004; Liu et al., 2005). In those cases, the educational model is theory-driven (also known as a top-down approach) and, consequently, instructors' roles are firstly identified and are then operationalized using some educational indicators. Unlike state-of-the-art studies of this topic that estimate the number of instructor roles using a top-down approach (qualitative analysis), this study estimates the number of instructor roles using students' perceptions and a bottom-up approach (also known as data-driven) (i.e., a quantitative analysis where first, it is identified the existing indicators of effective instruction, next, existing correlations among indicators are analysed to cluster them in the optimal number of roles, and finally, they are categorized/named based on the indicators encompassing each role).

Figure 1. Methodological framework proposed for identifying the roles of online instructors (N represents the total number of indicators and J the number of roles).

Figure 1 depicts the methodology proposed for identifying online instructors' roles and illustrates the main differences between the previous literature and the approach proposed. Notably, Figure 1 has not included all the literature that is methodologically identical to the frameworks of Thach & Murphy (1995), Williams (2003), and Abdulla (2004).

Defining the Outputs Required of Instructors in Asynchronous Learning Environments

The goal of this study is to define the existing dimensions (called roles in the educational context) and outputs of teaching in asynchronous learning environments according to students' perceptions, discarding the evaluation of instructors' competencies. Current research papers about the roles and competencies of online instructors include competencies, such as interpersonal communication, planning skills, and collaboration/teamwork skills. In this study, the teaching-learning items were evaluated for the online student community (as explained in the Introduction Section); if students were asked to rate these skills directly, we believe that the results would not be reliable. Furthermore, it is important to mention that this study will only focus on teaching duty (not management/service nor research duties), because teaching outputs are evaluated using students' perceptions and, in our opinion, students either do not have the ability to fairly assess their instructors' management/service and research duties or their assessments are not sufficiently accurate.

Three main sources were used to identify and validate qualitative and quantitative the indicators (desired outputs associated with the roles of an online university instructor). Firstly, a literature review done on 28 academic studies, which were published in international peer reviewed journal since 1995 to 2016. The manuscripts selected were organized according to: (i) the goal of our study, and (ii) the person who rates or defines the instructors' outputs, competencies, or roles (Table 1). Secondly, educational experts and practitioners evaluated the suitability of the indicator selected (in several interviews) in the first stage (Appendix A). Finally, the complete reliability of the instrument proposed was assessed with a preliminary pilot study with 30 students.

Table 1

List of Papers Organized According to the Classification Proposed

Methodology

Procedures

Data were collected through an online questionnaire administered to students from the Universitat Oberta de Catalunya, UOC (Spain). In accordance with the ethical policies of the University, the researchers first contacted instructors at the University to inform them about the aim of the study and to invite them to voluntarily participate. Those instructors who accepted the invitation received a consent form, which had to be signed and returned to the researchers in order to receive the online questionnaire. Instructors were in charge of providing their students a consent message as well as with the online questionnaire at the end of the semester (December 2015). The questionnaire was originally written in English and then translated into the official language(s) of the University (Catalan and Spanish). The main researcher in the study and the translator were the same person (a native Spanish speaker). Two native English speakers reviewed the questionnaire: an academic English editor and an English native speaker from South Africa. The first reviewer mainly verified the quality of the text (to ensure that the ideas were communicated clearly and accurately), whereas the second reviewer checked that the text was understandable for questionnaire respondents from different cultures. The online questionnaire was built using the Google Forms tool and, therefore, was also hosted by Google servers. The instructors who were contacted by researchers were teaching in the second semester of 2015 in subjects related to technical and social science disciplines. In this regard, our instructor selection was purposeful. Students were invited via email and/or the subject board to voluntarily fill out the online questionnaire over a time period of four weeks. One reminder was sent to all students after the first two weeks. The final questionnaire was composed of 69 items related to the instructors' roles (scored on a 5-point Likert scale) and 7 items summarizing the main student demographics (Appendix A).

Participants

The study participants were 925 students from the UOC, selected by stratified random sampling. Specifically, a total of 13,885 students enrolled in Psychology (6,364 students), Business Administration (5,270 students), and Computer Science (2,251 students) courses were contacted to participate in the study. The overall response rate was 6.66%, as 925 out of the 13,885 students participated in the study. The response rate in Psychology was 8.09%, in Business Administration it was 6.38%, and in Computer Science it was 3.42%.

Specifically, there were 361 (39%) males and 564 (61%) females. Participants were aged from 18 to 48 (92.1% of the participants) and only 7.9% of the participants were over 49-years-old. Most of them were Spanish (95.9%) and lived in Spain (95.4%). While a small percentage of respondents (12.6%) were students with no family/work commitments, 87.4% needed to balance their studies with family/work commitments. Finally, 42.4% of students were inexperienced in an online environment whereas 57.6% were previously trained in this type of environment.

Data Analysis

Data analyses were run on two different samples to ensure the generalizability of the findings. Thus, the sample of 925 students was divided in two datasets, which were obtained by using stratified sampling based on two demographic variables: degree and sex. The first dataset, of 613 students, was used to assess the factor structure of the scale items through Exploratory Factor Analysis (EFA). The second phase of the study used Confirmatory Factor Analysis (CFA) to test the factor structure of the scale obtained from the first stage through the other independent sample of 312 students.

Results

Exploratory Factor Analysis

An EFA with promax rotation was performed on the first sample of 613 students. Oblique (promax) rotation allows the factors to correlate. Questions that did not correlate significantly to any factor were dropped. We used a factor loading cutoff of 0.40 (McCrae & Terracciano, 2005). Variables that grouped together without any logical meaning were also dropped. The 43 remaining items loaded onto 6-factors. These 6-factors explain 76% of the variance. The loadings of the items on the 6-factors are reported in Figure 2.

Figure 2. Diagram representing the factor structure of the final 43-item educational role-based model. The pedagogical role includes the 28% of the total variance, course designer 18%, social 10%, life skills promoter 8%, technical 6%, and managerial 6%.

Factor 1 reflects the pedagogical attributes of an instructor. In this case, the pedagogical role could be viewed as a main construct that encompasses the following sub-roles: (i) professional, (ii) content expert, and (iii) resource material creator and study guide producer. Firstly, instructors should be good instructors. Secondly, instructors should be content facilitators with an excellent mastery of their subject matter and with a continuous interest in updating their knowledge of their subject as lifelong learners (Selvi, 2010). Thirdly, it is important to provide an adequate, useful, and comprehensible set of materials (syllabus, educational resources, and content material) at the beginning of the semester, thus allowing students to read and prepare in advance (Simonson, Smaldino, Albright, & Zvacek, 2014).

Factor 2 includes the design of instructional strategies for the learning environment and, therefore, corresponds to the designer role. This role has been recognized as a fundamental part of instructors' duties, especially in online environments, due to their particular nature (Alvarez et al., 2009). Online instructors' ultimate aim should be to design of an effective course structure (although this structure should be flexible and may be negotiated if necessary) for an interactive learning experience using a specific teaching methodology.

Factor 3 refers to the social role. The instructional social role is particularly important in the 21^st century framework of learning, where communication and teamwork skills are stressed as the key competencies to be developed by today's students (Fisher & Frey, 2015). Furthermore, this role has special significance in online learning, where there is no physical classroom to promote relationships between students-instructor (group relationship) and student-instructor (individual relationship). Thus, the EFA methodology clustered the two types of relationships in this factor.

All educators want to help their students succeed in life, even more so at university level, where students are being trained for a specific qualification. This fact is particularly reflected in Factor 4, which refers to instructors' role as life skills promoters. This role is encouraged by the European Union, the OECD, and UNESCO. Life skills can be defined as personal management and social skills that are necessary for adequate functioning on an independent basis (Gómez-Rey, Barbera, & Fernández-Navarro, 2015; Willemse, ten Dam, Geijsel, van Wessum, & Volman, 2015). Integrating these skills into the curriculum is a fundamental principle for reshaping education and helping students to reach their full potential, not only as academic achievers, but also as human beings.

Factor 5 corresponds to the technical role. This role has been stressed as one of the main pillars of the online instructor's role since the very beginning of research on this topic. Online instructors should know what digital culture offers, not only technically but also in the educational domain, and be able to use technology effectively. To achieve this, they need to stay updated about latest trends and, consequently, they need to be willing to continually improve their information and communications technology skills. Despite this, technical questions/problems involve a wide range of issues because of the sophistication and continual advancement of technology; therefore, instructors must have the support of specific technical assistance from the university.

Finally, the domain in which the instructor sets the main learning rules, routines, and procedures is represented in Factor 6, which corresponds to the managerial role. The managerial role has been extensively studied in the online learning literature by different authors. Instructors should use management, not to control their students, but rather to help them actively participate in the learning process. This role involves tasks such as the setting of minimal ethical norms, procedural rules (deadlines), and decision-making norms. Incorporating these rules in the syllabus that is provided to the students at the beginning of the course will help them to organize their time and their learning from the very beginning of the semester.

Confirmatory Factor Analysis

A confirmatory factor analysis was performed on the two samples to test the 6-factor solution obtained through EFA. R was used to run the confirmatory factor Analysis (CFA). Table 2 presents the results of the CFA.

Because of the large sample, we will scrutinize the fit indices, rather than the Chi-square, for assessing goodness of fit. The Tucker-Lewis Index (TLI) and Comparative fit index (CFI) are around or above 0.90, which indicates a good fit for the model. Furthermore, the root mean square residual (RMSR) and root mean square error of approximation (RMSEA) are below the usually recommended cut-off values (Hair, Black, Babin, & Anderson 2009). The 6-factor solution represents an acceptable fit based on the usual criteria employed in structural equation modelling.

Table 2

Results of the Confirmatory Factor Analysis

Discussion and Research Implications

Despite the fact that the first 6-factors were retained as representative factors using the EFA and CFA methodologies, it is worth mentioning that from students' point of view, the pedagogical role (F1) is the most important (as the factor with the highest weight of variance), followed by the designer role (F2), social role (F3), life skills promoter role (F4), technical role (F5), and managerial role (F6).

Students value the pedagogical role as most important to their learning. Initially, it may be assumed that online environments are driven by technological concerns instead of pedagogical issues. However, our findings show the latter to be the most important aspect of online scenarios. This result is aligned with existing literature, which finds that online instructors are, first and foremost, pedagogues (Bawane & Spector, 2009). Thus, as discussed by Schlosser and Simonson (2009), the main critical factor in student achievement is the instruction itself rather than the technological media where virtual learning takes place. These factors could explain the critical position of this role. Thus, the prominent position of the role reveals the need to continue to explore the more specific features of this role in the near future through the techno-pedagogical design adopted by the university in its educational model.

The designer role is the second most important factor. In the previous literature, some authors include it as part of the pedagogical role, whereas others separate the two (Goodyear et al., 2001; Bawane & Spector, 2009; Guasch et al., 2010; Carril et al., 2013). Unlike existing research works, this study goes one step further and analyses why this disagreement may exist. Online universities have different ways of designing their courses, which might be the reason why some authors either do or do not separate the designer role from the pedagogical one in their studies. In fact, there are some online courses that are designed by experts (instructional designers) but taught by a different person (instructors); there are other online courses where the same person is in charge of both tasks. In this study, the results have shown two very different roles, which is consistent with what happens in practice. Finally, it is worth mentioning that this role is more important than the remaining ones (F3, F4, F5, and F6) in the learning process. This finding makes sense because online courses must be designed and developed with special attention, in advance, as the basis for a successful learning experience.

In online learning environments, the learning process also has a social component (Guasch et al., 2010; González-Sanmamed et al., 2014). This fact is reflected in our findings, which show that the social role is included within the core of the online instructor's role. This involves the creation of an open space where students and instructors can interact and engage in activities (active learning). This promotes not only the participation and acquisition of knowledge but also the confirmation of learning (whereby instructors confirm what learners have understood about the course's content) (Gorsky & Blau, 2009). This latter factor, the social presence variable, has recently been analysed in the literature as a critical success factor in online learning (Barbera, Gómez-Rey, & Fernández-Navarro, 2016). As seen in Figure 2, students consider the pedagogical role more important than the social role. This finding is consistent with a recent study that found that while instructors tend to focus more on social issues, students are more concerned with pedagogical matters (Gómez-Rey et al., 2016).

An emerging role in the new educational paradigm is the life skills promoter role (to our knowledge, this is the first time that this role has appeared as a part of instructors' roles in online scenarios). The role of education in today's society implies promoting the development of a set of skills that go beyond acquiring what we have called knowledge so far. Thus, education should be understood, as a way to not only enhance the well-being of a society, but also promote positive outlooks and healthy behaviour as human beings. In this way, life skills education has long-term benefits to society in fields such as education, social, health, culture, and economics (Ghasemian & Kumar, 2015). Instructors should be aware of these benefits and include educational strategies in their online teaching to provide learners with adequate information, skills, and desirable attitudes that they will require to make rational decisions. For instance, online instructors should promote multidisciplinary and multicultural teams and provide information about institutional activities, such as academic events or workshops, among others. This enables individuals, during their studies, to acquire skills such as: (i) to behave responsibly (leading to healthy living), (ii) to develop positive attitudes towards themselves and others, (iii) to communicate effectively, and (iv) to develop negotiation skills, to name just a few.

The technical role takes one of the lowest positions in our ranking of instructors' roles. This result might be seen as contradictory due to the nature of the learning environment under consideration. However, our findings are aligned with some recent research, in which instructors noted that most of their online students were already immersed in the digital native community (Barbera et al., 2016; Gómez-Rey et al., 2016) unlike during the last decades of the 20th century, when digital technology was a problem in distance education due to the status of some students as digital immigrants (Prensky, 2001). Furthermore, our finding is also consistent with Gómez-Rey et al. (2016), who empirically showed that learning support is almost irrelevant to success in online learning. Despite these arguments and the findings described regarding the pedagogical role, it is notable that the technical role remains one of an online instructor's several roles (to promote active learning).

Finally, in the last position is the managerial role. Currently, instructional designers clearly state the main objective of the online course at the beginning of the semester, with the goal of helping their students to succeed. This has resulted in online environments becoming more intuitive, adaptive, and designed to enhance student learning, thus reducing the importance of the managerial role. For example, certain studies have shown that some years ago, the tasks of the managerial role occupied a substantial portion of instructors' working time. This high workload could be one of the reasons why this role occupied a top position among online instructors' roles. For instance, in Williams (2003) and Liu et al. (2005), the managerial role was ranked among the top positions whereas in the work of Bawane and Spector (2009), it was grouped together with the least important roles. This change in position over the years could be explained by improvements in course design, which has become clearer and more structured (including all forms of normalization that allow more transparency from the beginning of the course onward). This conclusion is aligned with our findings that justify students' perceptions of the position of the managerial role. Furthermore, this conclusion may have implications in practice as online instructors could dedicate more time to the other roles.

After this analysis, future research works will focus on the operationalization of the life skill promoter role in online learning environments. In this context, a preliminary theoretical study was already presented in Gómez-Rey et al. (2015). This previous work described a potential operationalization of the Capability Approach (CA) for the online learning community, which, in our opinion, covers today's educational needs. Future works can take advantage of this framework by creating an evaluation instrument to evaluate the strengths and weaknesses of an online university in the application of life-long education.

Study Limitations

Although this study expands our knowledge of the teaching work of online instructors in the 21^st century, certain limitations must be acknowledged. First, our study used mostly Spanish students. Future research works will replicate this study across other cultures and settings in order to overcome this limitation. Second, this study was designed using a sample obtained from an asynchronous learning university and, therefore, the results may be specific to asynchronous learning environments.

Acknowledgements

The research work of Pilar Gómez-Rey has been supported with a doctoral grant from the Universitat Oberta de Catalunya. The authors are grateful to the instructors and learners of the university for participating in this study.

References

Abdulla, A. G. (2004). Distance learning students' perceptions of the online instructor roles and competencies (Doctoral dissertation). CiteULike Library (Accession No. 1029039).

Álvarez, I., Guasch, T., & Espasa, A. (2009). University teacher roles and competencies in online learning environments: a theoretical analysis of teaching and learning practices. European Journal of Teacher Education, 32, 321-336.

Bailie, J. L. (2011). Effective online instructional competencies as perceived by online university faculty and students: A sequel study. Journal of Online Learning and Teaching, 7(1), 82-89.

Baran, E., Correia, A. P., & Thompson, A. (2011). Transforming online teaching practice: critical analysis of the literature on the roles and competencies of online teachers. Distance Education, 32, 421-439.

Gómez-Rey, P., Barbera, E., & Fernández-Navarro, F. (2015). Operationalization of the Capability Approach for Online Learning. In L. Gómez Chova, A. López Martínez & I. Candel Torres (Eds.), ICERI2015 Proceedings. Paper presented at the 8th International Conference of Education, Research and Innovation, Sevilla, Spain, 16-18 November (pp. 8200-8207). Spain, ESP: IATED Academy.

Gómez-Rey, P., Barbera, E., & Fernández-Navarro, F. (2016). Measuring teachers and learners' perceptions of the quality of their online learning experience. Distance Education, 37(2), 146-163.

Barbera, E., Gómez-Rey, P., & Fernández-Navarro, F. (2016). A cross-national study of teacher's perceptions of online learning success. Open Learning: The Journal of Open, Distance and e-Learning 31(1), 25-41.

Bawane, J., & Spector, J. M. (2009). Prioritization of online instructor roles: implications for competency-based teacher education programs. Distance Education, 30, 383-397.

Berge, Z. L. (1995a). Facilitating computer conferencing: Recommendations from the field. Educational Technology, 35(1), 22-30.

Berge, Z. L. (1995b). The role of the online instructor/facilitator. Educational Technology, 35(1), 22-30.

Berge, Z. L. (2008). Changing instructor's roles in virtual worlds. Quarterly Review of Distance Education, 9, 407-414.

Bezuidenhout, A. (2015). Implications for academic workload of the changing role of distance educators. Distance Education, 36, 246-262.

Blašková, M., Blaško, R., & Kucharčíková, A. (2014). Competences and competence model of university teachers. Procedia Social and Behavioral Sciences, 159, 457-467.

Carril, P. C. M., Sanmamed, M. G., & Selles, N. H. (2013). Pedagogical roles and competencies of university teachers practicing in the e-learning environment. The International Review of Research in Open and Distributed Learning, 14(3), 462-487.

Chang, C., Shen, H.Y., & Liu, Z.F. (2014). University faculty's perspectives on the roles of e-instructors and their online instruction practice. The International Review of Research in Open and Distributed Learning, 15(3), 72-92.

Coppola, N. W., & Hiltz, S. R., & Rotter, N. G. (2002). Becoming a virtual professor: Pedagogical roles and asynchronous learning networks. Journal of Management Information Systems, 18(4), 169-189.

Cyrs, T. E. (1997). Competence in teaching at a distance. New Directions for Teaching and Learning, 71, 15-18.

Denis, B., Watland, P., Pirotte, S., & Verday, N. (2004). Roles and competencies of the e-tutor. Paper presented at the Networked Learning Conference. England, UK. Retrieved from http://www.networkedlearningconference.org.uk/past/nlc2004/proceedings/symposia/symposium6/denis_et_al.htm

Easton, S. S. (2003). Clarifying the instructor's role in online distance learning. Communication Education 52, 87-105.

Egan, T. M., & Akdere, M. (2005). Clarifying distance education roles and competencies: Exploring similarities and differences between professional and student-practitioner perspectives. The American Journal of Distance Education, 19, 87-103.

Fisher, D., & Frey, N. (2015). Engaging the adolescent learner: Setting the stage for 21st-century learning. Newark, DE: International Literacy Association.

Ghasemian, A., & Kumar, G. V. (2015). The effectiveness of imparting and reviewing life skills education in iran and indian educational system: Opportunities and challenges. International Journal of Psychology and Psychiatry, 3(2), 130-138.

González-Sanmamed, M., Muñoz-Carril, P.C., & Sangra, A. (2014). Level of proficiency and professional development needs in peripheral online teaching roles. The International Review of Research in Open and Distributed Learning, 15(6), 162-187.

Goodyear, P., Salmon, G., Spector, J. M., Steeples, C., & Tickner, S. (2001). Competences for online teaching: A special report. Educational Technology Research and Development, 49, 65-72.

Gorsky, P., & Blau, I. (2009). Online teaching effectiveness: A tale of two instructors. The International Review of Research in Open and Distributed Learning, 10(3), 1-27.

Guasch, T., Alvarez, I., & Espasa, A. (2010). University teacher competencies in a virtual teaching/learning environment: Analysis of a teacher training experience. Teaching and Teacher Education, 26, 199-206.

Hair, J. F, Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate data analysis. Westlake Village, CA: PrenticeHall.

Heuer, B. P., & King, K. P. (2004). Leading the band: The role of the instructor in online learning for educators. Journal of Interactive Online Learning, 3(1), 1-11.

Isman, A., Altinay, Z., & Altinay, F. (2004). Roles of the students and teachers in distance education. Turkish Online Journal of Distance Education, 5(4), 1-10.

Kuo, Y.C., Walker, A. E., Belland, B. R., & Schroder, K. E. (2013). A predictive study of student satisfaction in online education programs. The International Review of Research in Open and Distributed Learning, 14(1), 16-39.

Liu, X., Bonk, C. J., Magjuka, R. J., Lee, S.H., & Su, B. (2005). Exploring four dimensions of online instructor roles: A program level case study. Journal of Asynchronous Learning Networks, 9(4), 29-48.

Millman, J., & Darling-Hammond, L. (1989). The new handbook of teacher evaluation: Assessing elementary and secondary school teachers. Newbury Park, CA: Sage.

McCrae, R. R., & Terracciano, A. (2005). Universal features of personality traits from the observer's perspective: Data from 50 cultures. Journal of Personality and Social Psychology, 88(3), 547.

Prensky, M. (2001). Digital natives, digital immigrants part 1. On the Horizon, 9(5), 1-6.

Queiroz, V., & Mustaro, P. N. (2003). Roles and competencies of online teachers. The Internet Teachers of English as a Second Language Journal, 9(7), 1-6.

Schlosser, L. A., & Simonson, M. R. (2009). Distance education: Definitions and glossary of terms. Charlotte, N.C: Information Age Publishing.

Schweisfurth, M. (2015). Learner-centred pedagogy: Towards a post-2015 agenda for teaching and learning. International Journal of Educational Development, 40, 259-266.

Selvi, K. (2010). Teachers' competencies. Cultura: International Journal of Philosophy of Culture and Axiology, 7(1), 167-175.

Simonson, M., Smaldino, S., Albright, M. J., & Zvacek, S. (2014). Teaching and learning at a distance. Charlotte, NC: Information Age Publishing.

Smith, T. C., & College, A. (2005). Fifty-one competencies for online instruction. The Journal of Educators Online, 2(2), 1-18.

Thach, E. C., & Murphy, K. L. (1995). Competencies for distance education professionals. Educational Technology Research and Development, 43(1), 57-79.

Willemse, T. M., ten Dam, G., Geijsel, F., van Wessum, L., & Volman, M. (2015). Fostering teachers' professional development for citizenship education. Teaching and Teacher Education, 49, 118-127.

Williams, P. E. (2003). Roles and competencies for distance education programs in higher education institutions. American Journal of Distance Education, 17(1), 45-57.

Young, S., (2006). Student views of effective online teaching in higher education. American Journal of Distance Education, 20, 65-77.

Appendix

Sample Interview Questions for Experts

Those experts that accepted an invitation for participating in the study received a draft document including a summary of the research study, the research questions and a list of online instructors' tasks. The interview was a semi-structured interview containing the following questions:

1. Describe your professional (and/or personal) experience relating to the topic.
2. Which of the online instructors' tasks presented do you usually carry out in your online classes? Which tasks do you not carry out in your online classes? Why?
3. In your judgment, which online instructors' tasks are missing from the list? Why?
4. In your opinion, which tasks should not be included in the list of online instructors' tasks? Why?
5. In your opinion, what are the online instructors' tasks that have been incorporated in the 21st Century? Why?

Learners' Questionnaire

The main questions (non-demographic questions) of the questionnaire are presented (Table A.1).

Table A.1

Desired Items for Online Instructors' Outputs

Student Voices on the Roles of Instructors in Asynchronous Learning Environments in the 21st Century by Pilar Gómez-Rey, Elena Barbera, and Francisco Fernández-Navarro is licensed under a Creative Commons Attribution 4.0 International License.