|
|
Gary Long, Carol Marchetti, and Richard Fasse
Rochester Institute of Technology, USA
This paper reports the findings of three studies within a program of research designed to better understand the factors contributing to the academic achievement of students in online courses and the contributions of interaction to online learning. The first study compared the academic achievement of students in the online and face-to-face (F2F) sections of multiple courses. In the second study, an online survey was used to obtain student perceptions of course satisfaction, learning, and communication. These factors were then related, using binary logistic regression analysis, to the amount of interaction that occurred in the students’ respective online courses; information from the myCourses course management system was used to quantify the amount of interaction that occurred in online courses. In the final study, both datasets were used to examine the academic achievement of students in online courses based upon the amount of interaction that had actually occurred. Whenever possible, a subgroup of deaf and hard-of-hearing students was included in the study to increase our understanding of the role that communication plays in the teaching-learning process. Our findings indicate that students enrolled in online courses, especially those designed with high levels of online interaction, receive higher grades and report greater learning than students in comparable F2F courses. In addition, online courses appear to provide deaf and hard-of-hearing students with special benefits in terms of academic achievement through online discussion. Overall, the studies illuminate how the quantity of interaction in online discussions relates to important success factors. Students in online courses with more interaction outperformed students in online courses with less interaction.
Keywords: Online learning; distance learning; online discussion; asynchronous; interaction; deaf; hard-of-hearing
Online learning is growing in popularity and effectiveness. Current estimates indicate that as many as four million students in North America are taking online courses (Allen & Seaman, 2008). In a systematic review of more than 1,000 studies comparing online and F2F learning, Means, Toyama, Murphy, Bakia, and Jones (2009) analyzed 51 studies that met their criteria for inclusion in a meta-analysis and concluded that “on average, students in the online learning conditions performed better than those receiving face to face instruction.” The study points to the dramatic evolution of online options for instruction and student-instructor and student-student interaction. Online learning has come a long way from the days when instruction was delivered via televised broadcasts or videoconferencing that was used to support web-based instruction in early generations of the initial Web. Today students are able to communicate directly with their instructor and each other, using the chat messaging and discussion forum features of their online courses or leaving comments for peers or their own written work in drop boxes, and are able to work collaboratively in virtual teams with student partners or in small groups on projects using online resources. This ease of communication with teachers and peers has changed the nature of online instruction from a passive, synchronous delivery system to an engaging, asynchronous arena for the active exchange of ideas.
Recently, Murugaiah and Thang (2010) utilized an active, collaborative, asynchronous approach to teaching English as a second language online to students at the National University of Malaysia. They incorporated interactive and reflective writing activities which helped to raise students’ awareness of their own learning processes and to become actively engaged learners responsible for their own learning. Faced with as many as 1,000 students for one teacher, the authors created a comfortable learning environment. The instructor initiated communication by asking students to introduce themselves online in English, and over time a learning community developed where students were able to help peers with their English writing. By engaging in coteaching, students learned about written English and how they and others best developed this complex skill.
This paper reports the findings of three studies in a program of research designed to better understand the factors contributing to online learning and the role of interaction in online courses. The first study compared the academic achievement of students in online and face-to-face (F2F) sections of the same course. In the second study, we used an online survey to examine student perceptions of course satisfaction, learning, and communication and related these to the amount of interaction that occurred in their course. Information from the myCourses course management system was used to quantify the amount of interaction that occurred in the online courses. Finally, we used both datasets to examine the academic achievement of students in online courses based upon the amount of actual interaction that occurred. Whenever possible, a subgroup of deaf and hard-of-hearing students was included to better understand the role that effective communication plays in the teaching-learning process for F2F and online courses.
The Rochester Institute of Technology (RIT) is a private university in upstate New York with 17,000 students in nine colleges and institutes. Career education and experiential learning are emphasized in both F2F and online course offerings (http://www.rit.edu/overview/fastfacts.html). Currently, approximately 4% of courses are offered in a fully online format and the majority of the students in these courses are campus-based. Asynchronous online discussion forums are a key component of the online courses.
One of RIT’s colleges is the National Technical Institute for the Deaf, with approximately 1,500 deaf and hard-of-hearing (D/HH) students. In any quarter, approximately 500 D/HH students are enrolled in RIT classes; their learning is supported with sign language interpreters and live captionists. Communication among D/HH students and their hearing peers in the F2F courses is challenging for a variety of reasons. Although the D/HH students can use a support service to participate in class discussions with their hearing peers, they are limited by communication-related challenges such as the processing time required for the interpreter to convey the message. Lack of knowledge of interpretation protocols is also a concern as instructors often call on a hearing student for an answer before the interpreter has finished signing the original question. Furthermore, observation of mixed groups of D/HH and hearing students indicates that in small groups members often communicate directly with each other (hearing to hearing and deaf to deaf) instead of going through a service provider for communication between deaf and hearing participants (Stinson, Liu, Saur, & Long, 1996). Direct communication between D/HH and hearing students is often a challenge and decreases the level of participation by some or all members of the group. These communication difficulties often lead to student passivity (Saur, Layne, Hurley, & Opton, 1986) and decreased learning for the deaf and hard-of-hearing students (Richardson, Marschark, Sarchet, & Sapere, 2010).
The text-based format of online courses and the written communication used in asynchronous discussion forums allows D/HH students to communicate directly with their hearing peers and instructors and may provide them with greater access to information than they have in a F2F, lecture-based class.
The RIT online learning program has been informed primarily by the “Seven Principles for Good Practice in Undergraduate Education” (Chickering & Gamson, 1987) and through active participation in the Sloan-C consortium of schools. The use of asynchronous “anywhere, anytime” strategies for engaging students is a key part of the course design. Most courses use online discussion for whole class participation, small group discussion, and collaboration to support group and team projects.
Student success in online courses was explored from three perspectives: (a) grade distribution analysis, (b) level of interaction analysis, and (c) grades by quartile.
To address how student academic achievement in online courses compared to achievement in face-to-face courses, the grades of all students taking a course offered in both an online and face-to-face format were obtained for the period between the fall quarter of the 2007–2008 academic year and the spring quarter of 2008–2009. Chi-square statistical analysis was used to compare the grades received in the online and face-to-face courses for deaf/hard-of-hearing and for hearing students.
Students enrolled in online courses during the three quarters of the 2007–2008 academic year were surveyed to obtain their perceptions of important factors related to learning. A questionnaire titled Online Discussion: Student Survey (Appendix A) was provided to approximately 2,000 students in each quarter using Clipboard, a web-based survey administration tool. Each student was emailed a link to access the survey. Students who completed the survey were entered into a drawing for a $50 gift certificate at the campus bookstore.
Four hundred and thirty-two online courses were divided into quartiles based on the score for the course from the interaction metric. Each quartile consisted of 108 courses. Student academic achievement was then examined by quartile to see if there was a relationship between the amount of interaction that occurred in online courses and the performance of the students in those courses.
For the level of interaction analysis, Table 1 provides enrollment figures and survey response rates for the overall group and for the hearing-impaired subgroup for each of the three quarters of the academic year. The response rate declined in each quarter as more of the students taking an online course had completed the survey in a previous quarter. In every quarter, the response rate of the deaf and hard-of-hearing students exceeded the overall response rate.
The survey results were analyzed using course interaction as an explanatory variable. The statistic used takes all the interaction in myCourses and averages it by the number of people in the class. This interaction metric included messages posted in the online discussion, public and private comments left in the grade book, feedback left in the dropbox, news announcements posted in the course, calendar items posted in the course, and the number of chat rooms created in the course. Courses were then rank-ordered and divided into quartiles based on the amount of interaction that occurred in that class.
A chi-square analysis was used to compare the grade distributions (A, B, C, D, F) for the hearing and the deaf/hard-of-hearing students enrolled in the online and face-to-face sections of comparable courses (i.e., with each section having the same course number) (Figure 1). Significant differences in grade distributions were found for hearing (X2= 868.3, df = 4 and p < 0.0001) and deaf/hard-of-hearing students (X2 = 101.8, df = 4, and p < 0.0001). In both the hearing and deaf/hard-of-hearing comparisons, the online sections had more As and fewer Cs than expected (while the face-to-face sections had fewer As and more Cs than expected).
Only four colleges had sufficient data (comparable online and face-to-face sections of the same course and large enough sample sizes) for GPA comparisons. All student grades were averaged for the entire set of online courses (with comparable F2F sections) for each college using A=4, B=3, C=2, D=1, F=0. Withdrawals and incompletes were not included in the analysis. The same computations were made for the F2F sections for each college.
Grade point averages for online and face-to-face sections varied among the colleges (Table 2). In the Colleges of Business and Applied Science and Technology, the online sections had a higher GPA than the face-to-face sections of courses; the College of Liberal Arts also had a higher, but less extreme, GPA in the online sections. However, in the College of Science, the GPA of the face-to-face sections was comparable to that of the online sections for the hearing students and higher than the GPA of the online sections for deaf and hard-of-hearing students. The lower GPA of the deaf/HH students in the online sections may be attributed to the symbolic notation and graphics used in the mathematics and science courses, which are not easily communicated online.
The College of Liberal Arts (COLA) was chosen for further analysis. In the COLA, a total of 17 courses were offered in both online and face-to-face formats during the 2007–2008 academic year; deaf and hard-of-hearing students were enrolled in 11 of these courses. Chi-square analyses were performed to compare the grade distributions of the online sections of the COLA courses with the comparable face-to-face sections (Figure 2). The grade distributions of the hearing students were significantly different (X2 = 36.1, df = 4 and p < 0.0001), with online sections earning more As and Fs and fewer Cs than expected. The grade distributions for D/HH students also differed significantly (X2 = 10.5, df = 4 and p = 0.015), but with more As and fewer Cs, Ds, and Fs than expected.
The level of interaction in online sections was examined in four main areas of interest: satisfaction, learning, interaction, and communication. Students in online courses were provided with seven questions, each with a Likert scale of strongly agree, agree, disagree, strongly disagree, or no response (Table 3). The positive and negative responses were combined to create a binary response variable (agree/disagree) and a binary logistic regression model for each question.
Three explanatory factors were included in each regression:
Binary logistic regression fits a linear model, using the natural log of the odds-ratio as the dependent variable. The model is defined as follows:
Ln[Odds-Ratio for Agreement] = Intercept + Q-Code + Hearing-Status + ESL-Status, where
the odds-ratio for agreement is the probability of “Agree” divided by probability of “Disagree,” and Q-Code represents the quartile for a course section in terms of its level of online interaction through the course management system.
The 432 online courses were divided into quartiles based on the score for the course from the interaction metric. Each quartile consisted of 108 courses. Table 4 provides a comparison of theQ-Codes using the course interaction metric, showing the average number of messages per week, the percentage of courses using the “groups” feature, and the average number of course features used. Q1 had the highest level of interaction, while Q4 had the lowest.
From the binary logistic regressions, it was determined that perceived satisfaction and overall learning for the course were not significantly affected by Q-Code, Hearing Status, or ESL Status. Neither of the regressions was significant (G = 5.86, df = 6, p= 0.439 and G = 12.19, df = 6, p = 0.058, respectively).
Learning from other students and interaction with the instructor had significant regressions (G = 98.6, df = 6, p < 0.0001 and G = 21.7, df = 6, p < 0.0001, respectively); however, only Q-Code significantly affected agreement with the statement. As described below, the highest level of agreement occurred for Q1 and decreased with each subsequent quartile of interaction.
Table 5 provides the percentages of agreement for each question within each quartile.
The regressions for increased interaction with other students, improved ability to communicate, and improved ability to express ideas were all significant (with G = 128.7, df = 6, p < 0.0001, G = 83.3, df = 6, p < 0.0001, and G = 63.3, df = 6, p < 0.0001, respectively), with both Q-Code and Hearing Status significantly affecting agreement with the statement.
The following findings were determined for interaction with other students (Table 6).
Figure 3 shows the significant odds-ratio results graphically.
The following findings were found for “improved ability to communicate my ideas because of the online interaction in this course” (Table 7).
In Figure 4, it can be seen that
The following findings were found for “ability to express my ideas more clearly because of the online interactions” (Table 8).
Figure 5 illustrates that
Some of the survey comments from deaf and hard-of-hearing students in the second study illustrate what we believe are important factors to their greater success in online courses. Asynchronous online discussion affords the deaf and hard-of-hearing students the time to more fully comprehend the discussion compared to mediated instruction in the live classroom with an interpreter. This “slower” pace also allows these students to participate more fully because they can take the time to reflect and construct their own ideas and respond to those of peers. Deaf and hard-of-hearing students responded in their own words to the following open-ended question:
What did you like best about this course?
I get to interact more with other students online rather than in class where I tend to clam up—because of my hearing loss and sometimes I don’t always know what to say quickly on the spot, while online, I have more time to reflect and respond.
Liked studying at my own pace. Good interaction with the professor.
Written English, because sometimes I could communicate with other team members online with written English better than with an interpreter.
I like the fact that we have equal access to communication!
What I liked best about this course was that it was easier for me to participate in discussions without getting behind due to a delay through an interpreter. I was on the same ‘playing field’ which was nice.
I love the way the discussions involves the whole class and part of it is because the teacher requires us to make meaningful comments and our own thoughts to the question. It’s the best part actually. (...) The discussions are great because of the interaction between all members of the class.
I liked the discussions online the best in the course. It was interesting to hear other people’s opinions, experiences and perspectives on different topics we covered in class.
In the third study, we examined the grade distributions of all students (hearing, deaf and hard-of-hearing combined) in 432 online courses by the course’s interaction quartile. A chi-square analysis indicates significant differences in the grade distributions (X2 = 133.6, df = 12, p < 0.0001). A closer look revealed the following.
Clearly, students in online courses with more interaction are outperforming those in online courses with less interaction.
In this paper we report on three studies, part of an ongoing research program now at the National Technical Institute for the Deaf (NTID), one of the nine RIT colleges, that suggest the reflective nature of asynchronous online discussion is helping our hearing, deaf and hard-of-hearing students achieve better academic results than in comparable campus-based courses. The first study analyzed grade distributions over a two-year period for online courses and their campus-based equivalents. Although there were some interesting exceptions, students in the online courses earned higher grades on average than the students in the face-to-face courses, and deaf and hard-of-hearing students had an even larger achievement increase than the hearing population.
The second study explored how the quantity of interaction in online courses related to student perceptions of course satisfaction, learning, and ease of communication. While no effect was found for perceived satisfaction or learning, we found that students in the most interactive courses communicated with the instructor and other students “more than I do in most courses.” Students also said that they were better able to communicate/express their own ideas “because of the online interactions in this course.” Our data suggests there are significant benefits for deaf and hard-of-hearing students enrolled in online courses with hearing students.
The third study looked at academic achievement of students enrolled in 432 online courses based on the amount of online interaction that occurred in the course. Students enrolled in online courses with more interaction outperformed students in online courses with less interaction. Students in the second survey study said that online interaction was important to their learning, and findings of the third study confirmed the validity of these perceptions with higher GPAs for more interactive courses.
One limitation of this work is the field nature of our data where students in the online and F2F classes were not randomly assigned. It is possible that with self-selection “brighter” students were going into the online sections, and we have no external measure to determine if this is true or not. However, it is hard to explain why the results were even more striking for the students with a hearing loss and these same students responded the most positively about being able to communicate with peers online in the second study that included the survey. What we do know is that students in the online courses with the most interaction had better academic achievement than students in the courses with the least interaction. This link speaks to the importance of students learning from each other as a critical component of overall learning in the online environment.
It can be extremely difficult for students with a hearing loss to keep up with classroom presentations and to be full participants in discussions in a traditional lecture-focused classroom. All information is taken in through the eyes, so allocating time to the multiple inputs in a classroom (instructor, Power Point slides, interpreter, other students) can be an overwhelming information-processing task. If this complex process isn’t monitored and controlled by the instructor (allowing students time to read slides before speaking, pausing to allow the interpreter to finish signing a question before calling on a student to answer it, pointing to the student answering the question, and allowing one student to respond at a time) communication breaks down and D/HH students no longer have access to the information. Findings of the present study indicate the importance of online peer interaction to learning for all students. It also speaks to the effectiveness of online interactions for facilitating direct communication between hearing and D/HH students and the associated learning that occurs.
In the next phase of our research we will collaborate with online faculty who have taught courses with three or more deaf or hard-of-hearing students during the past year to identify “best practices” for online instruction. We also have about 200 F2F courses on campus each quarter with deaf and hard-of-hearing students enrolled with hearing students and supported with interpreters and notetakers. We hope to collaborate with faculty teaching the F2F classes to redesign their courses to include online discussions that can effectively give all students, especially the deaf and hard-of-hearing, a better educational experience.
Allen, T., & Seaman, J. (2008). Staying the course: Online education in the United States, 2008. Wessley, MA: Sloan-C.
Chickering, A., & Gamson, Zelda, F. (1987, March). Seven principles of good practice in undergraduate education. American Association for Higher Education, Bulletin.
Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009) Evaluation of evidence-based online learning: A meta-analysis and review of online learning studies. U.S. Department of Education Report, Office of Planning Evaluation and Policy Development, Policy and Program Studies Service, 1–66.
Murugaiah, P., & Thang, S. M. (2010). Development of interactive and reflective learning among Malaysian online learners: An ESL instructor’s experience. International Review of Research in Open and Distance Learning, 11(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/842.
Richardson, J. T. E., Marschark, M., Sarchet, T., & Sapere, P. (2010). Deaf and hard-of-hearing students’ experiences in mainstream and separate postsecondary education. Journal of Deaf Studies and Deaf Education, 15(4), 358–382.
Saur, R. E., Layne, C. A., Hurley, E. A., & Opton, K. (1986). Dimensions of mainstreaming. Annals of the Deaf, 131, 325–330.
Stinson, M., Liu, Y., Saur, R., & Long, G. (1996). Deaf college students’ perceptions of communication in mainstream classes. Journal of Deaf Studies and Deaf Education, 1, 40–51.
