|
Catherine Anne Schmidt-Jones
University of Illinois Urbana-Champagne, USA
There have been numerous calls for research that demonstrates how open education resources (OERs) are actually being used. This case study sought to shed light on the users of a well-visited set of modular music-education materials published at Connexions. Respondents to a voluntary survey included teachers, students, self-directed learners, music ensemble participants, and casual learners. Most reported accessing individual modules on their own initiative, as part of a specific, immediate inquiry, rather than responding to institutional directives or following entire online courses. This was supported by computer-log records, which showed that most visitors to a module arrived from an Internet search for terms specific to that module. The study suggests that, for teachers and students as well as self-directed learners, one function of OERs is as a resource for just-in-time, inquiry-based learning.
Keywords: Open education resource; OER; learning objects; inquiry-based learning; just-in-time learning; case study; survey; Connexions
There have been numerous calls for research into the real-world effects of online open-education resources (OERs) on teachers and learners. Walker (2008), for example, has noted that proponents of open education are increasingly being asked to provide evidence of concrete gains in meeting learning objectives. Kanwar et al. (2010) found:
…a dearth of systematic empirical data to help answer the following questions: Who uses OER and for what purpose? What is the impact of the OER? To what extent have OER engendered changes in teachers’ professional practices? Have OER led to improvements in the quality of learning and learning outcomes? (p. 72)
Iiyoshi and Kumar (2008) have also called for research into the value and impact of open education, stating,
. . . one of open education’s most critical questions – how can open educational tools, resources, and knowledge demonstrably improve educational quality? – is rarely mentioned or explored. Unfortunately, this omission from the conversation and action mirrors the education community’s serious lack of engagement in investigating the transformative potential of open education. (p. 429)
The challenge of exploring real-world uses is complicated by the partial nature of the available data. Computer logs indicate numbers of visits to a resource, but not educational outcomes. Voluntary surveys can indicate educational benefits for respondents, but not whether respondents are typical of all visitors. Statistical studies yield persuasive data about the use of an OER by one population, but, as this case demonstrates, the very openness of an OER means that it may be affecting many different, widely dispersed populations. Also, data gathered about one OER cannot be assumed to be relevant to another. OERs can vary in a number of important ways as demonstrated by the contrast, outlined below, between just two of the wide variety of resources available. Numerous studies will be needed to provide enough “data points” to begin to develop a well-rounded understanding of the effects of OERs in the real world.
This case study is meant to be one such data point. I gathered both survey and computer-log data about a highly visited, modular, music-education resource. The results suggest that, although the modules were organized into online courses, most users were accessing modules individually through search engines. Even visitors who were formally teaching or studying the subject appeared to be using this OER as a resource for self-directed inquiry. From Dewey (1938) to Freire (1970) to Knowles (1984), many researchers and theorists have suggested that education organized around the learner’s present problems and interests is highly motivating and effective. One powerful consequence of near-constant access to extensive online resources is the ability to easily conduct an extensive inquiry around nearly any problem or interest (Cope & Kalantzis, 2009). Calls to assess the impact of OERs (such as those quoted above) often focus on use by educators, but educators are often reluctant to adopt technology. As Hennessy, Ruthven, and Brindley (2005) have shown, even teachers who are enthusiastic about adopting technology in the classroom experience significant barriers, including lack of access to equipment, reliability issues, lack of training and support, insufficient time for instruction in both subject matter and technology use, and policies that limit use of technology during testing and assessment. The results of this study suggest that the most significant educational effects of this set of materials may be due to informal rather than classroom-based use. Although these results are not generalizable, the widespread use of the Internet for informal inquiry-based learning, combined with current constraints on formal, classroom-based uses, suggests that it is possible that informal use may be a significant source of the educational impact created by many OERs.
This case study focused on 103 music-education modules published at Connexions (see http://cnx.org/). Connexions is a large and fast-growing (Ochoa, 2010) online repository and management system for educational materials. It is supported by Rice University, but is freely available to any educator who wishes to publish or manage content at the site. Authors are encouraged to publish materials in short modules, and site-based tools and Creative Commons licensing ensure that modules can easily be viewed, downloaded, altered, rearranged and reused in new courses at the site, or republished elsewhere, as needed, by any educator or learner.
The Connexions modules that were part of this study varied in format (textbook, encyclopedia entry, and lesson plan), in content area (Western music theory and notation, acoustics, and ethnomusicology), and in content level (from music novice to specialist). They are published individually but are heavily interlinked and arranged into multiple overlapping courses.
Numbers from this study cannot be generalized to any population; instead, the implications of the case arise from the fact that a very large number of people who chose to look at a resource were offered an opportunity to comment, and a wide variety of users responded. During the six months that I collected computer-log data, a total of 627,521 visits were recorded for all modules. Modules varied greatly in popularity, ranging from over 3,000 visits per month for some of the basic music theory texts to fewer than 10 visits per month for some of the lesson plan modules. Because I was interested in visitors’ reactions to the materials, the invitation to participate in the survey was at the end of each module, and so would only have been noticed by relatively engaged users who had at least skimmed through an entire module. The 488 respondents who completed the survey were less than a tenth of a percent of all visitors, but, as Fowler (1993) has explained, such non-statistical surveys can be useful for exploring the “range of ideas or opinions that people have or the way that variables seem to hang together” (p.10). I took this exploratory-case-study approach, searching for common patterns and issues. For the sake of clarity, the percentages reported below are percentages of all completed surveys (N = 488) or of all logged visits (N = 627,521). When a number might be interesting either as a percentage of one category of respondents or as a percentage of all respondents, I report it as a number of respondents rather than a percentage.
The research question was: Who uses these materials, for what purposes? The survey was voluntary and anonymous. Available from March 2010 to March 2011 through a link (to Survey Monkey) at the end of each module, the survey gathered some demographic data (age, gender, and country of residence), but focused mainly on the respondents’ music-education background, reason for accessing the materials, and opinions about the materials. (See survey instrument at http://cnx.org/content/m34234/latest/.) The survey was the focus of the study, but for 6 months during the survey (from mid-April to mid-November 2010), I also collected computer-log data about all visits, focusing on the number, source (for example, search term or referring site), and country of origin of visits to each module. I compared those results to the number and country of origin of survey responses for each module, looking for evidence of ways in which survey respondents did or did not appear to be representative of all visitors.
Reported ages ranged from 11 to 87 years, with a mean of 37 (see Figure 1). The age distribution was somewhat skewed towards younger people, but interest among older learners was also strong. Respondents describing themselves as students ranged in age from 11 to 73 years (with a mean of 26), and those describing themselves as self-directed learners ranged from 12 to 87 years (with a mean of 38). The male-female ratio of respondents was 55% - 45%.
Visits from 117 countries were logged, and survey responses arrived from 41 of those countries. For the seven countries that generated large numbers of visits, the proportion of survey responses was similar to the proportion of all visits, with the exception of the Philippines, which generated a large percentage of visits but few survey responses (see Figure 2).
Survey responses suggested that visitors came from a wide variety of educational situations. Music learning can take place in very different contexts. Music classes and lessons typically involve explicit instruction, similar to classes in other subject areas. However, many learners choose to study music primarily, or entirely, within the context of an ensemble. Ensemble is a general term referring to any group of musicians that play together, including school bands, youth orchestras, church choirs, and teen-organized “garage bands.” As Wenger (1998) has described, learning often takes place within communities of practice, in which understanding is gained through a “process of being active participants in the practices of social communities and constructing identities in relation to these communities” (p. 4). Music ensembles often function as Wengerian communities of practice. Participants are mutually engaged in an endeavor, for example preparing to perform together, and teaching and learning take place as an integral part of that endeavor.
Finally, as evidenced by the many “Teach Yourself to Play...” books available, there are also music learners who are working on their own, without regular opportunities to learn directly from other musicians. The survey used the music-learning venue as a way of categorizing respondents under the assumption that such contexts probably have important effects on the users’ music-education needs and goals. In a multiple choice question, respondents were asked to choose a self-description indicating their reason for looking at the materials. An open-ended question then requested details about the multiple-choice response (see Figure 3).
Reported benefits were spread fairly evenly among the respondent categories. The survey asked respondents to rate how helpful they had found the module, on a 5-point scale from “not at all” (= 1) to “extremely” (= 5) helpful. The mean of 3.998 corresponded very closely with “very helpful” (= 4). Means within each category were close to the overall mean, ranging from 3.84 (among students) to 4.20 (among teachers).
Self-directed learners were the largest group of respondents (190, or 39.26% of respondents). This group also appeared to be the most deeply engaged with the materials; over 55% of the respondents who reported visiting the materials multiple times in the previous year were self-directed learners.
Common goals mentioned by self-directed learners included the following:
Goal descriptions included the following:
“Teaching yourself,” simply for enjoyment and self-improvement, is perhaps a more common pursuit in music than in other subjects, and 13% of self-directed learners reported that they were not interested in formal music classes or lessons. Comments included the following:
However, over half of the self-directed learners reported that they had not received as much formal music education as they would have liked. Money or cost was the most common reason given. Other circumstances that were cited as restricting access to music education included a lack of family support, geographic isolation, inflexibility in school schedules, and lack of access for those who are past “school age.” Characteristic comments include:
The inquiries of the 73 respondents (15.08%) who chose “Just curious/other” described a very wide variety of goals, projects, and inquiries:
The 109 respondents (22.52%) who indicated they were using the online materials to supplement formal instruction included students in
Some – particularly those in lessons – appeared to be satisfying personal curiosities raised by their studies, while others appeared to be working to meet course expectations:
Eleven respondents (2.27%) cited ensemble directing, and 38 respondents (10.12%) participation in a musical ensemble, as their reason for looking at the materials. Respondents mentioned a wide variety of teaching, performing, and informal ensembles:
Comments from the 60 respondents (12.4%) who identified themselves as teachers mentioned
They appeared to be conducting a wide variety of self-directed inquiries with the goal of improving their teaching practices:
The previous comment was the only survey response that named a specific course published at Connexions (Basic Elements of Music), and the context is in reference to adapting the materials in order to create another course (the American folk music tradition). In Hatakka’s (2009) study, instructors reported problems with reusing open resources that were packaged in large chunks, such as entire lectures, because only portions of the lecture were relevant to their course:
So rather than using full lectures or courses in their content development they prefer to use smaller pieces of information that can be fitted into their content more easily. This means that the granularity of the objects is of importance and can be an inhibiting factor for reuse (p. 8, emphasis in the original).
It could be, then, that users who prefer entire online courses are choosing other resources, while the modular materials in this study are unusually attractive or helpful to those who prefer small pieces of information. The examples above suggest that these include not only teachers who want to customize their curriculum, but also students who need supplementary materials and independent learners pursuing inquiries not well-served by standard courses.
As Stake (1995) explains, the emphasis in case study should be on understanding the object of study as a unique case, rather than on producing generalizations. It is perhaps better suited to raising questions and issues than to finding answers. One issue raised by this study concerns the distribution of the respondents in terms of educational situation. To date, there have been few attempts to characterize the distribution of users of an OER; the most visible is the evaluation of the Massachusetts Institute of Technology (MIT) OpenCourseWare (OCW). The distribution found by Carson (2006) for MIT’s OCW was substantially similar to the distribution in the present study, which is interesting because the two resources are different in so many ways. Major differences include the following:
The Connexions Project complements the MIT project in that it provides not only a rapidly growing collection of free scholarly material but also a set of free software tools to help authors publish and collaborate; instructors build rapidly and share custom courses; and learners explore the links among concepts, courses, and disciplines. It focuses on building and supporting communities of digital object consumers and producers who credential material (p. 15).
Given the differences between the two resources, a similarity in patterns of use becomes a point of interest. There appear to be similarities in the educational roles of survey respondents (see figure 4). Proportions of teachers and self-directed learners are similar for both resources. At first glance, the student proportions look very different because this study included categories not in the MIT study: ensemble participants and directors. As described above, ensembles often function as communities of practice in which learning about music is an integral aspect of the act of playing music with others. Although such learning is much less explicit than classroom-based learning, the role of music director in these music-making communities is in some ways parallel to that of the classroom teacher, and the role of the learning musician parallels that of the student.
Seen in this light, the proportions of students, teachers, and self-directed learners begin to look even more similar to those found for OCW. Because the two resources are so different, this similarity may reflect an underlying “distribution of typical OER users” more than it reflects the distribution of interest in a specific resource. If so, then the greatest educational effects of OERs may be due to direct, informal use by learners.
I chose to study this resource because I am the author of the modules; one goal of the study was to collect feedback that would help me improve the materials. My position as author gave me easy access to computer log and survey data. However, there is evidence that it also affected the survey responses. As Bergquist and Ljungberg (2001) explain, in the social calculus of Internet-based open-information communities, acceptance of a free gift of information may create a sense of moral obligation to pay back the giver in some way. In this case, the final survey question was an invitation to comment on the survey itself. Only 166 (34%) responded to this question, and 80 of those comments were simply expressing appreciation or gratitude for the materials, with some explicitly linking their gratitude to their survey participation:
The dynamics involved in this interaction – my personal interest in improving the materials, and the possibility that survey responses were colored by attitudes about me as provider of the materials – are part of the context for understanding this case.
This case demonstrates that one function of OERs is to serve as readily available resources for informal, inquiry-based learning. Of the users who were sufficiently engaged with the materials to respond to the survey, over 39% identified themselves as self-directed music learners, and a further 15% chose “just curious/other” to describe their visit. The remaining respondents reported involvement in music teaching or learning in a variety of settings; however, even for this group, the search for online information was typically self-chosen and self-directed, serving as an informal supplement to the formal curricula of classrooms and lessons, or the implicit, community-of-practice-style curricula of performing ensembles.
Satisfied users were very diverse in age, geographic location, previous experience in the subject, educational role, and goals. Most survey respondents were seeking specific knowledge to resolve an immediate question, problem, or curiosity. Evidence from computer logs supported this result; most visitors arrived at a module by searching the Internet for terms specific to that module.
The preference for seeking small portions of knowledge as needed, rather than entire courses, may have been partly due to specific characteristics of the OER, such as the subject area or publication format. However, for the question “who uses this OER,” the results of this case were similar to the results found by Carson (2006) for MIT’s OCW, a very different resource. The differences make a direct comparison problematic in large part because so many music learners are based in ensemble communities-of-practice rather than in classrooms. Still, in both studies, the largest group of respondents categorized themselves as self-directed learners, with students the second largest group, and educators representing fewer than 20% of respondents. If ensemble-based learners can be considered “students” in the sense that, like students, they have regularly scheduled opportunities for face-to-face learning, then the proportions found in the two studies begin to look very similar. Do other OERs also attract more self-directed learners than students, and more students than educators? Citing the results of the MIT-OCW study, Kanwar et al. (2010) assert that the
. . . role of OER is increasingly changing from primarily a teaching to a learning resource. This reflects the wider change within education from a teacher-centered to a learner-centered model. Instead of teachers and educational institutions being the main users, it is students who now constitute the majority of users of OER. (p. 67)
Looking at a very different resource, the present case supports the conclusion that direct, informal use by both students and self-directed learners is the main use of OERs. Education researchers, policy-makers, and OER developers may want to consider the best ways to understand and support this type of use and take steps to make it truly available to all learners.
Further research could also clarify the extent to which OERs are being used, by both learners and educators, as resources for individualized, just-in-time inquiries. In particular, research that focuses on the user rather than the resource, seeking to understand the frequency, extent, and educational significance of informal online inquiries in the lives of various types of learners and educators, would help provide a better context for understanding the educational value of just-in-time OER use. Given that OERs appear to be empowering learners to direct their own learning activities in pursuit of their own learning purposes, a high educational value can reasonably be expected. As Dewey (1938) has stated:
There is, I think, no point in the philosophy of progressive education which is sounder than its emphasis upon the importance of the participation of the learner in the formation of the purposes which direct his activities in the learning process. (p. 67)
Atkins, D. E., Brown, J. S., & Hammond, A. L. (2007). A review of the open educational resources (OER) movement: Achievements, challenges, and opportunities (Report to the William and Flora Hewlett Foundation). Retrieved from http://www.oerderves.org/2007/03/a-review-of-the-open-educational-resources-oer-movement/
Bergquist, M., & Ljungberg, J. (2001). The power of gifts: Organizing social relationships in open source communities. Information Systems Journal, 11, 305-20.
Carson, S. (2006). 2005 Program evaluation findings report: MIT OpenCourseWare. Retrieved from http://ocw.mit.edu/about/site-statistics/
Cope, B., & Kalantzis, M. (Eds.). (2009). Ubiquitous learning. Urbana: University of Illinois Press.
Dewey, J. (1938/1997). Experience & education. New York: Touchstone.
Fowler, Floyd J. Jr. (1993). Survey research methods. Newbury Park: Sage Publications.
Freire, P. (1970). Pedagogy of the oppressed. New York: Continuum.
Hatakka, M. (2009). Build it and they will come? Inhibiting factors for reuse of open content in developing countries. The Electronic Journal on Information Systems in Developing Countries, 37(5), 1-16.
Hennessy, S., Ruthven, K., & Brindley, S. (2005). Teacher perspectives on integrating ICT into subject teaching: commitment, constraints, caution, and change. Journal of Curriculum Studies 37(2), 155-192.
Iiyoshi, T., & Kumar, M. S. V. (Eds.). (2008). Opening up education: The collective advancement of education through open technology, open content, and open knowledge. Cambridge, MA: The MIT Press.
Kanwar, A., Kodhandaraman, B., & Umar, A. (2010). Toward sustainable open education resources: A perspective from the global south. The American Journal of Distance Education, 24(2), 65-80. DOI: 10.1080/08923641003696588
Knowles, M. (1984). Andragogy in action: Applying modern principles of adult learning. San Francisco and London: Jossey-Bass Inc.
Ochoa, X. (2010). Connexions: A social and successful anomaly among learning object repositories. Journal of Emerging Technologies in Web Intelligence, 2(1), 11-22.
Stake, R. (1995). The art of case study research. Thousand Oaks, CA: Sage Publications.
Walker, E. (2008). Evaluating the results of open education. In T. Iiyoshi & M. S. V. Kumar (Eds.), Opening up education: The collective advancement of education through open technology, open content, and open knowledge. Cambridge, MA: The MIT Press.
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge: Cambridge University Press.