Volume 18, Number 5
Allen Rao1, John Hilton III2, and Sarah Harper3
1NetEase Online Open Courses, 2,3Brigham Young University
Over the past decade, great progress has been made in improving the quality and availability of Open Educational Resources (OER). OER proponents often discuss the ability for users to revise and remix OER to make them more suitable for local contexts; however, much OER goes unmodified. This note from the field examines the efforts of NetEase Online Open Courses, a Chinese organization, to take more than two thousand Khan Academy videos and translate them into Chinese. We provide background on this initiative and also analyze site metrics to determine what type of use these derivative OER have received.
Keywords: open educational resources, open access, remix, 5Rs, China, Khan Academy
An important topic in education concerns the use of Open Educational Resources (OER). D'Antoni (2009) cited the William and Flora Hewlett Foundation, a key donor for OER, as stating that OER are "resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others" (p. 4).
A key aspect of this definition is that OER permits free re-purposing; in other words, individuals can legally revise or remix OER. Hilton, Wiley, Stein, and Johnson (2010) examine several issues relating to OER remix and point out that there are multiple gradations in how "open" a resource really is. The resources that are most open allow for complicated revisions, such as translation. Although OER proponents often point to the legal permission to revise as a vital benefit of OER, an important question is whether people actually take advantage of the legal permissions afforded by OER. A related issue surrounds the potential impact of revised OER. Do people use revised versions of OER? In other words, do revisions actually extend the influence of OER? How do users perceive revised OER? Compared with other aspects of OER research, these important questions have yet to be fully examined. The purpose of the present case study is to examine one revision of one set of OER (Khan Academy videos). First, we review the literature pertaining the revising OER, as well as Khan Academy.
Duncan (2009) found that very few OER adopters revise or remix the work. Revision is defined as a modification to the OER, such as a translation, or an addition or deletion of material. When additional resources are combined with an OER, the new work is considered to be a remix. In Duncan's study of an OER collection containing 3,519 utilized components, less than 3% were translated, and less than 3% were modified in some other way. Thus, in this study, less than 6% of the OER in the reviewed set were revised or remixed.
Additionally, Petrides, Nguyen, Jimes, and Karaglani (2008) noted that in the same OER collection observed in Duncan's analysis, 88% of those who altered the open resources were the original authors themselves. If a fundamental foundation of the OER model is the ability for others to modify the materials, then the findings of Petrides et al. (2008) convey a limited extension of a principle objective. Consequently, this report challenges the opinion that people are anxious to refashion resources for their own purposes.
To further explore the customization of open resources, Hilton, Lutz, and Wiley (2012) investigated the degree to which instructors reused, revised, and remixed Flat World Knowledge (FWK) open textbooks. By comparing custom texts to the original material, the examination catalogued additions, deletions, reorders (changing the sequence of the material), and remixes (introducing the content of another FWK resource into the custom text). This study likewise observed low revision and remix rates, with only 7.5% of the texts customized over the two-year period. More significantly, a noteworthy pattern became apparent; the simpler to make the change, the more of those changes were observed. Deletions, the easiest alteration, were found in 60.32% of customized materials, followed by re-orderings in 40.49%. Additions were made in 16.60% of customized materials, and finally, remixes, the most challenging modification, occurred in only 1.62% of the remixes.
Amiel (2013) examined decisions and challenges accompanying the creation of a work by translating and remixing open education resources. Amiel produced an instructional booklet on OER for Brazilian K-12 teachers and chronicled the efforts that it took to create this resource. Amiel identified six different routes a utilized original resource could take. A contribution could be "in complete form attempting to maintain the original meaning and context (revision); adapting language to reflect local meaning and needs (revision/remix); modifying substantially to the point of departing from the original source (remix); only as a source of inspiration, demonstrating a need to create content based on the principles and ideas but not borrowing directly from the source itself (reference); identifying instead other, more compatible resources (search); developing entirely new content when none was available (create)" (Amiel, 2013, pp. 133-134).
When creating a new OER, Amiel selected resources with more open licenses over superior material with narrowed licensing. Amiel found several concerns with this English-based westernized OER that invoked multiple concerns, such as cultural barriers. Amiel concluded that although OER are available to be altered, in practice, performing the actual remix is not an easy task.
While revising OER at the level of discrete learning resources appears to happen relatively irregularly, research indicates that some important books, or large-scale OER projects, do receive extensive revisions. For example, Hilton (2009) found that many people were willing to do large scale revisions (e.g., translation, creating an audio version, etc.) of key works such as Larry Lessig's Free Culture. Carson, Kanchanaraksa, Gooding, Mulder, and Schuwer (2012) point out that in the case of MIT OpenCourseWare, extensive revisions in the form of translations have taken place, including multiple MIT open courses being translated into Chinese, Spanish, Portuguese, Farsi, and Thai. Lee, Lin, and Bonk (2007) chronicle the effort that was undertaken by over two thousand volunteers to produce the Chinese translation. Thus, while OER generally do not appear to be frequently revised, selected large-scale initiatives have been revised, with the revision frequently being translation.
The focus of the present study is a translation of Khan Academy videos into Mandarin; we next discuss additional context for this study by first providing an overview of Khan Academy, as well as what the literature says about its impact on student learning.
Initially founded as a hobby of a hedge fund analyst who was a long distance tutor for his cousins, Khan Academy now has impacted millions of lives across the globe ("Stories," 2016). Salman Khan transitioned from tutoring using Yahoo's Doodle notepad to creating YouTube videos, and after unexpected popularity, established Khan Academy in October 2006. Three years later, Khan abandoned his finance work to focus on his nonprofit educational organization full time—in a walk-in closet (Noer, 2012).
Khan Academy has released more than 5,500 videos on YouTube and its website, khanacademy.org, in addition to hundreds of thousands of practice problems that students can work on at their own pace. Subjects covered in the micro-lectures include math from the elementary to college level, science and engineering, computing, arts and humanities, economics and finance, and test prep for exams such as the SAT, MCAT, and GMAT ("About," 2016).
Murphy, Gallagher, Krumm, Mislevy, and Hafter (2014) examined the efficacy of Khan Academy as a supplemental resource in schools between 2011 and 2013. Two thousand students in grades 5-10 participated in the study; nine sites, 20 schools, and 70 teachers were involved. Findings from the study include the following: student perceptions of their time spent on Khan Academy was highly positive, student engagement was generally high, and a majority of teachers stated that they planned to continue using the Academy after the conclusion of study. Teachers also reported that the program increased their capacity to support students across multiple areas, and that they found Khan Academy reports useful to review student progress. Murphy et al. (2015) found a positive correlation between increased Khan Academy use and "(1) improvements in student test scores, and (2) improvements in three of the four self-reported nonachievement outcomes - math anxiety, math self-concept, and academic efficacy (i.e., belief in one's ability to succeed in academic endeavors)" (p. 12).
In a separate research study, Cohen and Phillips (2015) examined an implementation of Khan Academy across Idaho 47 different schools in 33 that focused on personalized learning. Data from 5,309 students between 3rd and 8th grade showed positive correlations between the usage of Khan Academy and academic progress (Cohen & Phillips, 2015). In a comparison between Khan Academy mission (a personalized set of grade-level content) completion and target growth, "students who completed 0-10% of their mission on average grew as expected," "students who completed 40% or more of their mission on average grew 1.5 times more than their expected growth in one year," and "students who completed 60% or more of their mission on average grew 1.8 times their expected growth in one year" (Cohen & Phillips, 2015, p. 22). Other studies have found learning gains when Khan Academy videos were implemented in classroom settings (Castillo & McIntosh, 2012; Park, 2016).
Khan Academy licenses its videos with a Creative Commons BY-NC-SA license. This open license allows people to retain, reuse, revise, remix, and redistribute the videos. Thus, Khan Academy legally grants permission for people to edit their videos and share them in different formats, as long as it is done for non-commercial purposes.
Because of the value of Khan Academy videos, significant efforts have been made to translate them into other languages. Khan Academy has a volunteer translation portal, by which much of its content is available, following a volunteer translation model that has been used by others (Beaven, Comas-Quinn, Hauck, de los Arcos, & Lewis, 2012). However, in the case of the Chinese translation of Khan Academy videos, the translation was done by NetEase Open Courses, a subsidiary of called NetEase (网易). NetEase is an Internet company based in Beijing, China that focuses a variety of Internet services including email and online gaming. As part of a non-profit initiative within the company, NetEase Open Courses hosts multiple OER and has been involved in a variety of initiatives relating to making educational content freely available, such as working with Coursera to deliver free online courses (Penna, 2013).
In 2012, Khan Academy announced,
At present, 927 videos from Khan Academy have been translated by NetEase Open Courses, covering mathematics, chemistry, biology, geometry, physics, and other disciplines…NetEase is the first portal site in China to partner with Khan Academy to localize its content and this greatly enhances NetEase Open Courses' content and user base. Meanwhile, the cooperation with NetEase will promote the localization of Khan Academy courses in China, to achieve the purpose of providing a world class education for anyone anywhere ("NetEase subtitles," 2012).
As of September, 2016, NetEase has translated 2,931 Khan Academy videos into Chinese, across 88 different courses (e.g., U.S. History, see appendix for complete course list).
In the present study we examine the impact of NetEase's revision of OER, specifically the Khan Academy videos, by asking the following questions:
We used analytics data provided by NetEase Open Courses to determine which videos had been viewed most frequently. The two principal ways that people can access the Khan Academy videos in Chinese are through NetEase's website (http://open.163.com/khan/) and their apps. Unfortunately, accurate analytics data were not available prior to 2016; thus, we used the analytics data available between January 1 - September 30, 2016 to determine the number of page views. These data were provided by NetEase Open Courses and capture visits that took place through the NetEase Open Courses app, as well as their web portal.
Table 1 lists the top 10 most frequently viewed courses between January 1 - September 30, 2016, for both the App and website views.
Table 1
Top Viewed Khan Academy Courses, January 1 - September 30, 2016
| Course | App view | Website views | Total views |
| Finance | 15,185,984 | 1,419,990 | 16,605,974 |
| Microeconomics | 6,921,772 | 921,162 | 7,842,934 |
| Macroeconomics | 6,047,580 | 803,000 | 6,850,580 |
| Linear Algebra | 5,883,658 | 928,620 | 6,812,278 |
| Statistics | 4,167,504 | 1,020,360 | 5,187,864 |
| Chemistry | 3,618,912 | 386,100 | 4,005,012 |
| Physics | 3,443,462 | 422,444 | 3,865,906 |
| Probability | 3,429,712 | 347,886 | 3,777,598 |
| Health and Medicine | 3,164,194 | 305,646 | 3,469,840 |
| Differential Calculus | 2,675,266 | 347,842 | 3,023,108 |
| Total | 54,538,044 | 6,903,050 | 61,441,094 |
Finance, Economics, Math, and Science are the most popular topics for the Chinese Khan Academy videos. Access to videos is much more likely to come from the NetEase app than the website. Most importantly, it is clear that there is a large amount of interest in these videos, with the top 10 courses garnering more than 60 million views.
These data prompt several observations about the ability to revise OER and its attendant usefulness. First, while studies have shown that most OER are not revised, this study clearly demonstrates the deep impact that revision can have. In this study, we found that revised Khan Academy videos received over 60 million page views between January and September of 2016. Given that many of these videos have been available for four years, this potentially puts the total page views at over 200 million. Moreover, the impact of the Chinese-translated Khan Academy videos is further understated by the fact that we have only included data based on visits to NetEase Open Courses. Because these videos are openly licensed, many enterprising users have posted the NetEase Open Courses videos on a variety of websites, such as Youku.com.
Thus, while OER generally do not appear to be revised, in some instances the open licenses that undergird OER allow for deep and significant expansions (e.g., Carson et al., 2012) The present study falls into this category given the large number of views that the Chinese Khan Academy videos have received. This result underscores the benefits of the "Open" in OER, as opposed to resources that are simply "free."
This brief note from the field demonstrates the clear value that revisions can have in extending the influence of OER. Khan Academy videos are evidently a valuable resource and their educational value has been extended as a result of being OER. Without the open license that allowed for the remix of these videos, the viewership would be significantly less. While most OER do not receive dramatic revisions, this study illustrates the value of open licenses and the additional learning that can take place when resources are given open, rather than restricted licenses.
About. (2016). Khan Academy. Retrieved from https://www.khanacademy.org/about
Amiel, T. (2013). Identifying barriers to the remix of translated open educational resources. The International Review of Research in Open and Distributed Learning, 14(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1351
Beaven, T., Comas-Quinn, A., Hauck, M., de los Arcos, B., & Lewis, T. (2012). The open translation MOOC: Creating online communities to transcend linguistic barriers. Journal of Interactive Media in Education. doi: http://doi.org/10.5334/2013-18
Carson, S., Kanchanaraksa, S., Gooding, I., Mulder, F., & Schuwer, R. (2012). Impact of OpenCourseWare publication on higher education participation and student recruitment. The International Review of Research in Open and Distributed Learning, 13(4), 19-32.
Castillo, D., & McIntosh, P. (2012). About Oakland Unity. Khan Academy. Retrieved from https://www.khanacademy.org/educator/empowering-teachers/oakland-unity/a/about-oakland-unity
Cohen, J., & Phillips, D. (2015). Learning gets personal: How Idaho students and teachers are embracing personalized learning through Khan Academy. The Khan Academy Report. Retrieved from http://static1.squarespace.com/static/56733533c647ad862c4e7189/t/567afe22c21b864703fc4e1e/1450901026762/KahnAcademyIdaho-LearningGetsPersonal.pdf
D'Antoni, S. (2009). Open Educational Resources: Reviewing initiatives and issues. Open Learning, The Journal of Open and Distance Learning, 24, 3-10
Duncan, S. M. (2009). Patterns of learning object reuse in the Connexions repository. All Graduate Theses and Dissertations. Paper 423. Retrieved from http://digitalcommons.usu.edu/etd/423
Hilton III, J. (2009). From PDF to MP3: Motivations for creating derivatives. First Monday, 14(9).
Hilton J. III, Wiley, D., Stein, J., & Johnson, A. (2010). The four ‘R's of openness and ALMS analysis: frameworks for open educational resources. Open Learning, 25(1), 37-44.
Hilton, J. III, Lutz, N., & Wiley, D. (2012). Examining the reuse of open textbooks. The International Review of Research in Open and Distributed Learning, 13(2), 45-58.
Lee, M., Lin, M., & Bonk, C. (2007). OOPS, Turning MIT Opencourseware into Chinese: An analysis of a community of practice of global translators. The International Review of Research in Open and Distributed Learning 8(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/463/980
Murphy, R., Gallagher, L., Krumm, A.E., Mislevy, J., & Hafter, A. (2014, March). Research on the Use of Khan Academy in Schools: Research Brief. Menlo Park: SRI International. Retrieved from http://www.sri.com/sites/default/files/publications/2014-03-07_implementation_briefing.pdf
NetEase subtitles videos in Chinese. (2012, November 8). Retrieved from https://www.khanacademy.org/about/blog/post/29838569844/netease-subtitles-videos-in-chinese
Noer, M. (2012, November 2). One man, one computer, 10 million students: How Khan Academy is reinventing education. Forbes. Retrieved from http://www.prisim.com/wp-content/uploads/2013/12/One-Man-One-Computer-10-Million-Students-How-Khan-Academy-Is-Reinventing-Education-Forbes.pdf
Park, S. (2016). Reflection: Impact on my classroom. Khan Academy. Retrieved from https://www.khanacademy.org/educ ator/empowering-teachers/eastside-prep/a/reflection-impact-on-my-classroom
Penna, M. (2013). Coursera partners with NetEase to deliver free online learning in China [Web blog]. Retrieved from https://asiancorrespondent.com/2013/10/coursera-partners-with-netease-to-deliver-free-online-learning-in-china/
Petrides, L., Nguyen, L., Jimes, C., & Karaglani, A. (2008). Open educational resources: Inquiring into author use and reuse. International Journal of Technology Enhanced Learning, 1(1-2), 98-117.
Stories. (2012, November 8). Retrieved from https://www.khanacademy.org/stories
Khan Academy Videos in Chinese: A Case Study in OER Revision by Allen Rao, John Hilton III, and Sarah Harper is licensed under a Creative Commons Attribution 4.0 International License.