|
|
|
|
|
Suhang Jiang, Adrienne E. Williams, Mark Warschauer, Wenliang He, and Diane K. O’Dowd
University of California, Irvine, United States
There is concern that online education may widen the achievement gap between students from different socioeconomic classes. The recent discussion of integrating massive open online courses (MOOCs) into formal higher education has added fuel to this debate. In this study, factors influencing enrollment and completion in a pre-college preparatory MOOC were explored. University of California at Irvine (UCI) students of all preparation levels, defined by math Scholastic Aptitude Test (SAT) score, were invited to take a Bio Prep MOOC to help them prepare for introductory biology. Students with math SAT below 550 were offered the explicit incentive of an early change to the biology major upon successful completion of the MOOC and two additional onsite courses. Our results demonstrate that, among course registrants, a higher percentage of UCI students (>60%) completed the course than non-UCI registrants from the general population (<9%). Female UCI students had a greater likelihood of enrolling in the MOOC, but were not different from male students in terms of performance. University students entering with low preparation outperformed students entering who already had the credentials to become biology majors. These findings suggest that MOOCs can reach students, even those entering college with less preparation, before they enter university and have the potential to prepare them for challenging science, technology, engineering, and mathematics (STEM) courses.
Keywords: MOOCs; STEM; remedial course
The demand for a diverse workforce of individuals with degrees in science, technology, engineering, and mathematics (STEM) has focused attention on how to increase the enrollment and retention of undergraduate STEM majors (National Science Foundation, 2004; National Science Board, 2006; Wilson et al., 2012; National Science Board, 2012). Attrition in STEM majors is particularly high during the first two years of undergraduate education (Seymour & Hewitt, 1997; Shuman et al., 1999; Chang, Cerna, Han, & Sàenz, 2008). At many universities, large introductory STEM courses, often referred to as “gate-keepers,” represent major barriers that discourage students from entering or persisting in STEM disciplines. Moreover, females (Felder, Felder, Mauney, Hamrin, & Dietz, 1995; Brainard & Carlin, 1997; Blickenstaff, 2005), first-generation college students (Engle & Tinto, 2008), students from low-income families (Lam, Srivatsan, Doverspike, Vesalo, & Mawasha, 2006), and under-represented minority students (Cole & Espinoza, 2008) are more likely to drop out from STEM majors.
In recent years, with the number of college applications increasing (Clinedinst & Hawkins, 2014), there are growing numbers of low-income, under-represented minority, and first generation college students in entering classes. These students represent a tremendous resource for developing a diverse community of STEM graduates but they are also at high risk for dropping out since they are often academically under-prepared (Perna & Titus, 2005; Gumport, 2007; Breneman, Jr., & Hoxby, 1998). Providing these incoming students with access to low cost, high quality courses that help them transition to college and be successful in STEM majors is therefore of great importance. However, financial constraints have made it increasingly difficult for the brick-and-mortar campuses to provide transition face-to-face instruction to support these students (Clinedinst & Hawkins, 2014), especially in the summer before initial entry.
The emergence of massive open online courses (MOOCs) potentially affords universities the opportunity to provide students with preparatory courses before they enter university, at relatively low cost. MOOCs are typically developed by university faculty and are free for anyone who has access to a computer and the internet. In general, the completion rate of stand-alone MOOCs is very low (about 5%) (Ho et al., 2014). Enrollment surveys indicate that the students in the new massive courses are largely male and well-educated learners, rather than the underserved (Christensen et al., 2013). A pilot study conducted in San Jose State University, which introduced MOOCs into the curriculum, reported that matriculated students performed better than non-matriculated students. However, the completion rate of the specifically targeted at-risk students was disappointingly low. It was reported that students’ effort was the strongest indicator of their success in the course (Firmin et al., 2014). In addition, students claiming a high intention to complete a MOOC in a pre-survey are much more likely to finish the course than others (Koller et al., 2013). This raises the question of whether it is possible to increase learners’ motivation, engagement, and success in MOOCs by providing external incentives.
The goal of this study was to develop a course that would help under-prepared students who had been accepted to the UCI gain skills and knowledge that would increase their probability of success in a large freshman STEM course, Bio 93. We chose to develop this as a MOOC to determine if this could be done in the context of a course that would also potentially benefit a broad group of individuals in the general population. We first introduce the rationale for providing this course, explain how it was organized, and describe the incentive offered to increase motivation for under-prepared students, and then analyze the extent to which the course achieved our goals.
The study addresses the following questions.
This paper focuses on student success within the Bio Prep MOOC; a second paper will focus on the performance by students in the subsequent face-to-face Bio 93 course.
At UCI, Bio 93 is the first-quarter, introductory biology course that is required for all students majoring in biology or health careers. Author Diane K. O’Dowd has been teaching the class for 10 years, and author Adrienne E. Williams has worked with Diane K. O’Dowd to develop teaching techniques and measure student performance in the course for the past 8 years. Our analysis of data from the last three years indicates that Bio 93 is a major obstacle for under-represented minority students. The drop/fail rate for these groups in Bio 93 is 25-30%, much higher than the 15% overall class average. Further analysis of learner outcome data indicates that high school preparation is a critical factor in passing Bio 93. A high percentage of students who score below 550 on their quantitative Scholastic Aptitude Test (SAT) scores drop or fail Bio 93. Students must score 550 or above to be accepted as a biology major; those below that score are redirected into other majors, most commonly undecided/undeclared.We have investigated ways of creating preparatory courses and learning tools to meet the needs of academically less prepared students, ideally before they enter the university. Online courses offer one way of potentially helping students who have not yet moved to Irvine, and free courses are potentially highly beneficial for this often low-income group. The Bio Prep MOOC was designed by the co-first author Adrienne E. Williams to be accessible to the general public, but was specifically aimed at providing knowledge and critical thinking that is important for success in Bio 93. It was offered on the Coursera MOOC platform.
The MOOC consisted of three modules with the biological themes of membrane transport, protein synthesis and localization, and neurophysiology. Authors Adrienne E. Williams and Diane K. O’Dowd taught the course as two tracks. The Basics track was based on content videos and quizzes only. To complete the Scholars track, students needed to compose written responses to three to five short-answer questions in each of the three units of the course, and provide peer assessments (both numerical scores and comments) on short answers of at least three peers for each question. The Scholars Track also included three independent study projects, where students were required to apply what they have learned from the lectures to new situations. Students were also expected to participate in peer assessment of other students’ project reports to receive a grade for their own report. These additional writing and peer assessment activities were included in the Scholars track based on prior research indicating a correlation with academic success (Liang & Tsai, 2010). Students who successfully completed the Scholars track were awarded a Distinction certificate and those who completed the Basics track were awarded a Normal certificate. Students did not need to pre-select a track but were given the appropriate certificate based on the tasks they fulfilled. In summary, the Scholars track requires more self-regulated study for students than the Basics track.
Like all Coursera courses, Bio Prep was offered to the general public, and approximately 37,000 students signed up. Beyond this general enrollment, we conducted targeted outreach for Bio Prep to incoming freshman students during summer Enrollment in early summer 2013, focusing on two groups in particular. Students accepted into the university as Biological Science majors were told about the course by the Biological Science counselors. They were encouraged to participate to help prepare them for a successful first quarter of Bio 93 but there was no explicit incentive. Another group of students applied to UCI with Biological Sciences as their first choice of major, but this choice was not granted because their math SAT scores were less than 550. Most of these students became Undecided majors. These students were also encouraged to participate in the Bio Prep MOOC by their Summer Enrollment counselors through the Undecided Student Affairs Office and they were additionally provided an explicit but no cost incentive to take the MOOC. Ordinarily, students below the 550 math SAT threshold must complete a full year of biology and chemistry courses successfully before entering the Biological Sciences major. However, students below this threshold were informed that they could transfer to the major after only one quarter if they successfully completed the Bio Prep MOOC Scholars track and passed both Bio 93 and introductory chemistry during fall quarter.
The MOOC was taught by Adrienne E. Williams and Diane K. O’Dowd in the four weeks immediately before fall quarter began at UCI (August 26 to September 23, 2013) to maximize the number of entering UCI students who could take the course.
Subjects in the study included 27,487 MOOC students who had grade records in the Bio Prep MOOC, and the 1,695 UCI students who enrolled in Bio 93 in the fall 2013 quarter and for whom we had complete prior academic records. A subset of students were in both groups (n = 382). This subset was further divided into the “strong math UCI” students (n=226), who had SAT scores of 550 or higher, and the “weak math UCI” students (n=156), who had SAT scores under 550.
Data for the study came from three sources. The first is the SQL file extracted from Coursera database, which includes students’ assignment performance (e.g., quiz and peer assessment) and final performance in the Bio Prep MOOC (e.g., successfully completed the Scholars track and receiving the Distinction certificate). About 37,933 students enrolled in the Bio Prep MOOC and 27,487 students had grade records. A total of 551 students earned the Distinction certificate and 1,971 students earned the Normal certificate.
The second source of data comes from UCI’s Office of Institutional Research, which provides the demographic information and pre-college academic preparation record (i.e., SAT scores) of students who enrolled in the onsite Bio 93 course.
Also, for UCI Bio 93 students who participated in the Bio Prep MOOC, we have the above data associated with their Coursera e-mail address and their UCI identification information.
All MOOC participants (n=27,487) were coded as either UCI students (382) or non-UCI students (27,105). For all UCI Bio 93 students (n=1,695), including those in the MOOC, we gathered SAT math, reading, and writing scores. Literature indicates that online education may widen the academic performance gap between traditionally low-performing students and high-performing students (Xu & Jaggars, 2013). This study therefore examines how students’ previous academic performance influences their participation and performance in the MOOC. Students were also coded as “weak math” if their math SAT scores were below 550. As SAT math 550 is a threshold for the university incentive policy, this variable reflects the effect of an explicit incentive policy on students’ participation and performance in the Bio Prep MOOC.
Each UCI student was identified by gender and ethnicity. Prior studies show contradictory results in terms of which gender is more likely to take online courses and perform better in online learning. The analysis of University of Pennsylvania’s 36 MOOCs indicates that there are significantly more males than females taking MOOCs (Christensen et al., 2013). Some research on distance education shows that there is no gender difference in learning outcomes in online learning while some research indicates that females perform significantly better than males (Xu & Jaggars, 2013). It has been argued that females are more motivated and better at communicating and at scheduling their online learning (McSporran & Young, 2001). Ethnicity was coded as under-represented minority (URM, i.e., Black/African-American, American Indian, and Hispanic) or non-URM. A number of studies report that the educational gap between URMs and non-URMs is wider in online courses than in face-to-face courses (for an overview, see Means, Bakia, and Murphy, 2014).
To analyze the first research question, we present a descriptive assessment of UCI Bio 93 freshmen’s MOOC performance compared to the other MOOC participants. To analyze the second and the third research questions, we construct a logistic regression model and a multinomial logistic regression model to investigate the factors that influence UCI Bio 93 students’ enrollment and final performance in the MOOC separately.
There were differences in performance in the MOOC between the three groups examined: strong math UCI students, weak math UCI students and non-UCI MOOC participants. Typical of many MOOCs, the completion percentage of non-UCI participants was low, with 92% not completing. Of those that completed, approximately 7% earned a Normal certificate and <2% earned Distinction (Figure 1). In marked contrast, the two groups of UCI students had a much higher percentage of completion and Distinction. For the strong math UCI students only 36% did not complete the course, while 37% earned a Normal certificate and 27% earned a Distinction certificate (Figure 1). The percentage of weak math UCI students (n=156) that did not complete the course (31%) was similar to the strong math group, but more earned a Distinction certification (39%) than a Normal certificate (30%) (Figure 1).
Research Questions 2 and 3 focus on which UCI students enroll in and complete the MOOC.
Table 1 shows the odds ratios from the nested logistic regression models predicting UCI Bio 93 students enrolled in the Bio Prep MOOC, using data from the entire cohort of Bio 93 students. As our hypothesis predicted, we found weak math students are more likely to enroll in the course. However, when we control for gender the effect of math is not significant. This indicates that women are more likely to enroll in the MOOC than men. There was no effect of ethnicity on enrollment.
We hypothesized that students with weak math SAT scores would be more likely to complete the MOOC with distinction, given the incentive provided by the university. As predicted, the probability of obtaining the Distinction certificate for students with low SAT math was about 1.66 times greater than the high SAT math students (Table 2, Model 1). This effect remains when the model controls for student preparation, gender and ethnicity. None of the other student characteristics examined significantly affected the prediction of earning a Normal certificate.
Our results demonstrate that the percentage of UCI students completing the MOOC and earning Distinction was much higher than the non-UCI MOOC students. Female students were more likely to enroll than male students, and low math SAT students were more likely to earn Distinction.
There would be little support for investing in development of college preparatory classes if the completion rates are similar to those reported for most MOOCs (~5%) (Ho et al., 2014). A study at San Jose State University indicated that students in a MOOC who are also matriculated and the course counts for university credit have a higher completion rate in three courses (29.8%, 50.0% and 54.3% respectively) than those who are not matriculated (17.6%, 11.9% and 48.7% respectively; Firmin et al., 2014). Our study demonstrates the completion rate for incoming UCI students in a biology prep MOOC can be boosted to over 60% even when there is no course or university credit involved. The exposure to knowledge and/or skills relevant to a first quarter class was sufficient to result in this increase since incoming 1st year students with the credentials to become biology majors completed at approximately the same rate as underprepared students who were given an explicit incentive.
There is considerable concern that online education programs, while effective for some, may amplify rather than narrow educational and social divides (Bolt & Crawford, 2000). Some quasi-experimental studies show that online education reflects the same divide commonly observed in the brick-and-mortar settings. For example, Black and Hispanic students performed more poorly than White students in online courses (Newell, 2007). In an experimental study involving multiple sections of an economics course no significant difference was found between online and lecture courses among students with higher prior GPA; however, among students with lower GPA, online students scored significantly lower than face-to-face lecture students (Figlio, Rush, & Yin, 2010). In the San Jose State University study, the low course pass rates may be due to the target group being at-risk students (Firmin et al., 2014).
In the present study we found that UCI incoming students with weak SAT math skills had a higher probability of completing the MOOC with Distinction than students with strong math skills. It is likely that this difference in completing the MOOC with Distinction is associated with the incentive policy enacted for incoming freshmen who did not meet the SAT math requirement for a biology major. This provided freshmen who obtained a Distinction certificate the opportunity to enter the biology major two quarters earlier than those who did not complete or earned only a Normal certificate. Importantly this incentive did not result in any costs for the students, the instructors, or the university.
Our results also indicate that among UCI students, females were more likely to enroll in the MOOC than male students. This is interesting given previous research showing that the majority (51%~87%) of MOOC population are males (Ho et al., 2014) and it is potentially important given the underrepresentation of women in many STEM fields (Beede et. al, 2011) and women’s greater likelihood to transfer out of STEM majors (Chen & Soldner, 2013). One possible interpretation, consistent with a recent study, is that female students are more motivated and adaptive to online education than male students in an educational setting (Xu & Jaggars, 2013). In contrast, unlike the Xu and Jaggars study, we did not find negative associations between ethnicity and enrollment or performance in the MOOC.
Although previous literature shows that low performance students can be further disadvantaged by online education, our results suggest that a MOOC with no-cost incentives provides an additional learning opportunity for low-performance students. Follow-up research will analyze the impact of the MOOC for students’ academic performance in the onsite Bio 93 course. Additional studies from other institutions and MOOCs will also be important in evaluating the effectiveness of MOOCs as preparatory courses for higher education.
Development and delivery of the MOOC, and study design, was funded by an HHMI Professor grant to Diane K. O’Dowd . Additional funding for analysis of the MOOC data was provided by a MOOC Research Initiative grant to Mark Warschauer and Diane K.O’Dowd and by the Digital Learning Lab, University of California, Irvine.
Beede, D., Julian, T., Langdon, D., McKittrick, G., Khan, B., & Doms, M. (2011). Women in STEM: A gender gap to innovation. Washington, DC: U.S. Department of Commerce, Economics and Statistics Administration.
Blickenstaff, J. C. (2005). Women and science careers: Leaky pipeline or gender filter? Gender and Education, 17(4), 369–386. doi:10.1080/09540250500145072
Bolt, D. B., & Crawford, R. A. K. (2000). Digital divide: Csomputers and our children’s future. TV Books.
Brainard, S. G., & Carlin, L. (1997). A longitudinal study of undergraduate women in engineering and science. In Frontiers in Education Conference, 1997. 27th Annual Conference. Teaching and Learning in an Era of Change Proceedings. (Vol. 1, pp. 134–143). doi:10.1109/FIE.1997.644826
Chang, M. J., Cerna, O., Han, J., & Sàenz, V. (2008). The contradictory roles of institutional status in retaining underrepresented minorities in biomedical and behavioral science majors. The Review of Higher Education, 31(4), 433–464. doi:10.1353/rhe.0.0011
Chen, X., & Soldner, M. (2013). STEM attrition: College students’ paths into and out of STEM fields. Washington, DC: U.S. Department of Education, National Center for Educational Statistics.
Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. J. (2013). The MOOC phenomenon: Who takes massive open online courses and why? (SSRN Scholarly Paper No. ID 2350964). Rochester, NY: Social Science Research Network. Retrieved from http://papers.ssrn.com/abstract=2350964
Clinedinst, M., & Hawkins, D. A. (2014). The state of college admissions 2013. College Admissions Counseling. Retrieved from http://www.collegeadmissionspartners.com/college-admissions-counseling/state-college-admissions-2013/
Cole, D., & Espinoza, A. (2008). Examining the academic success of Latino students in science technology engineering and mathematics (STEM) majors. Journal of College Student Development, 49(4), 285–300.
Felder, R. M., Felder, G. N., Mauney, M., Hamrin, C. E., & Dietz, E. J. (1995). A longitudinal study of engineering student performance and retention. III. gender differences in student performance and attitudes. Journal of Engineering Education, 84(2), 151–163. doi:10.1002/j.2168-9830.1995.tb00162.x
Figlio, D. N., Rush, M., & Yin, L. (2010). Is it live or is it internet? Experimental estimates of the effects of online instruction on student learning (Working Paper No. 16089). National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w16089
Firmin, R., Schiorring, E., Whitmer, J., Willett, T., Collins, E., & Sujitparapitaya, S. (2014). Case study: Using MOOCs for conventional college coursework. Distance Education.
Ho, A. D., Reich, J., Nesterko, S. O., Seaton, D. T., Mullaney, T., Waldo, J., & Chuang, I. (2014). HarvardX and MITx: The first year of open online courses, fall 2012-summer 2013 (SSRN Scholarly Paper No. ID 2381263). Rochester, NY: Social Science Research Network. Retrieved from http://papers.ssrn.com/abstract=2381263
Koller, D., Ng, A., Do, C., & Chen, Z. (n.d.). Retention and intention in massive open online courses. EDUCAUSE Review. Retrieved June 5, 2013, from http://www.educause.edu/ero/article/retention-and-intention-massive-open-online-courses
Lam, P. C., Srivatsan, T., Doverspike, D., Vesalo, J., & Mawasha, P. R. (2006). A ten year assessment of the pre-engineering program for under-represented, low income and/or first generation college students at the University of Akron. Journal of STEM Education: Innovations and Research, 6(3). Retrieved from http://ojs.jstem.org/index.php?journal=JSTEM&page=article&op=view&path%5B%5D=1281
Liang, J.-C., & Tsai, C.-C. (2010). Learning through science writing via online peer assessment in a college biology course. The Internet and Higher Education, 13(4), 242–247. doi:10.1016/j.iheduc.2010.04.004
Means, B., Bakia, M. & Murphy, R. (2014). Learning online: What research tells us about whether, when and how. New York: Routledge.
National Science Board. (2006). Science and engineering indicators 2006. Retrieved from http://www.nsf.gov/statistics/seind06/
National Science Board. (2012). Science and engineering indicators 2012. Retrieved from http://www.nsf.gov/statistics/seind12/
National Science Foundation. (2004). Women, minorities and persons with disabilities in science and engineering. Retrieved from http://www.nsf.gov/statistics/wmpd/2013/start.cfm?CFID=11129262&CFTOKEN=40515627&jsessionid=f0302d61827b5f455e753adf666e73535636
Newell, C. C. (2007). Learner characteristics as predictors of online course completion among nontraditional technical college students. Retrieved from http://purl.galileo.usg.edu/uga_etd/newell_chandler_c_200705_edd http://hdl.handle.net/10724/23943
Seymour, E., & Hewitt, N. M. (1997). Talking about leaving: Why undergraduates leave the sciences. Boulder (Colo.): Westview Press.
Shuman, L. J., Delaney, C., Wolfe, H., Scalise, A., & Besterfield-Sacre, M. (1999). Engineering attrition: Student characteristics and educational initiatives. In Proceedings of the American Society of Engineering Education. Charlotte, NC.
Wilson, Z. S., Holmes, L., deGravelles, K., Sylvain, M. R., Batiste, L., Johnson, M., … Warner, I. M. (2012). Hierarchical mentoring: A transformative strategy for improving diversity and retention in undergraduate STEM disciplines. Journal of Science Education and Technology, 21(1), 148–156. doi:10.1007/s10956-011-9292-5
© Jiang, Williams, Warschauer, He, O’Dowd