International Review of Research in Open and Distributed Learning

Volume 18, Number 4

June - 2017

What Impacts do OER Have on Students? Students Share Their Experiences with a Health Psychology OER at New York City College of Technology

Cailean Cooney
New York City College of Technology, The City University of New York

Abstract

This article reports findings from a study conducted with students in three sections of a Health Psychology course that replaced a traditional textbook with open educational resources (OER) as the primary course material. The purpose of the study was to learn how OER impacted students. Data were collected in Fall 2015 with students from New York City College of Technology (City Tech), of the City University of New York (CUNY), a comprehensive college located in Brooklyn. Students were assigned the OER by their course instructor, who developed it as part of a library funded OER pilot initiative. Two research instruments were employed: one-on-one interviews and short surveys. Both interview and survey items asked students about how they engaged with the OER as their primary assigned course material. They shared feedback about the overall organization of the OER, ease of use, methods used to access the OER and complete coursework, benefits and challenges, and differences and similarities to using a traditional print textbook.

Findings indicate that most students were able to access the OER more easily than traditional textbooks and responded positively to the variety of learning materials and assignments the OER assembled. Most students reported that course readings were equal to or better than traditional textbooks and would be willing to register for a course offering a similar resource in the future. A small amount of students reported minor usability issues. Also, few students had difficulties obtaining technology necessary to access the OER.

Keywords: open educational resources, oer, open education, open digital pedagogy, assessment, evaluation, student feedback

Introduction

The purpose of this study was to learn how the use of open educational resources (OER) in place of traditional textbooks has impacted students' academic experiences. Research was conducted with students in three sections of a Health Psychology course at New York City College of Technology, a comprehensive college located in Brooklyn, New York. Through interviews and surveys, students were asked what methods they used to complete course assignments from the OER, how the OER was organized, and what it was like to navigate. They were also asked about the benefits and challenges of using the OER, and how it compared to a traditional textbook. The feedback collected in this study was intended to help assess and improve the OER program at the College, particularly the faculty development component of the initiative, and to encourage faculty reflection and foster best practices for developing, revising, and teaching with OER. Topics of value to discuss with faculty include student technology access and usage, user experience and navigation of OER sites, preferences for types of learning materials, and more. The findings from this study also facilitate an exemplar of positive and critical feedback about aspects of the Health Psychology OER that can provide concrete examples for instructors developing OER in the future.

Open educational resources are "teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others" (Hewlett, 2016, para. 7). In addition to being cost free, open educational resources are released in the public domain, or have a Creative Commons license which allows the creator to retain their copyright and decide how their work gets used, shared, and remixed by others (Creative Commons, 2016). In contrast to textbooks, OER can be frequently updated, customized, or reworked by instructors in local contexts.

City Tech's OER Initiative

City Tech was motivated by the advantages that OER present for reducing or obviating the price of textbooks and providing more consistent access to curricular materials, particularly in light of the population City Tech serves. The college is one of 24 colleges that comprise the City University of New York (CUNY), the United States' largest urban public higher education institution that is legislatively mandated as the "vehicle for the upward mobility of the disadvantaged in the City of New York" (City University of New York [CUNY], 2016, para. 4). Pitt, Ebrahimi, McAndrew, and Coughlan (2013) contend that providing easy access to and customization of OER is especially important among community college and commuter student populations who often juggle a number of different responsibilities, and come from a variety of backgrounds, and levels of academic experience. Such attributes are consistent with the City Tech student population. More than 85% of City Tech students receive financial aid and 44% are among the first generation to attend college. The student population is diverse: 32% of students are Hispanic or Latino, 30% are Black or African American, and 20% are Asian. Fifty percent of students at City Tech report a household income of less than $25,000 per year and about 10% earn less than $10,000 per year (CUNY Office of Institutional Research and Assessment, 2014).

In addition to attending college, half of City Tech students work: 21% work 35 hours or more per week, and 27% work between 21 to 34 hours per week. Students' most commonly reported earning money in order to pay for living expenses, tuition, and fees. Since City Tech students devote much of their time earning a living and fulfilling non-academic responsibilities in addition to college, 43% report that work responsibilities affect their academic performance (CUNY Office of Institutional Research and Assessment, 2014). Moreover, Regalado and Smale's (2014, 2015) ethnographic study of CUNY students' academic and technological behaviors reveals the importance of understanding how access to technology, commuting, and other day to day realities converge with academic careers in meaningful ways. Their findings were encouragement for this study to include questions about students' access and use of online based course materials particularly because faculty interviewed in their study voiced concerns about students' technology skills in academic settings.

The OER initiative at City Tech is run out of the College's Library and launched in Fall 2014. The pilot initiative awarded stipends for faculty to replace textbooks with a curated resource composed of openly licensed educational materials, as well as library digital subscription content, and other cost free educational materials. The initiative was modelled after similar programs launched at UMass Amherst and Temple University in the early 2010s (Bell, 2014; UMass Amherst Libraries, 2016). Completed OER were hosted on the OpenLab, City Tech's unique WordPress powered open source collaborative platform for teaching and learning. Data were collected in December 2015 at New York City College of Technology (City Tech).

Study Significance

This study was intended to engage students and foster a more inclusive and equitable platform on which to consider big pedagogical initiatives, in this case the creation/adoption of open educational resources as the primary curricular material in lieu of traditional textbooks. It aims to broaden existing discourse on OER adoption beyond methods often used to measure efficacy, by instead foregrounding dialogue from the learner's perspective, in line with the Freirian philosophy of working with students as involved subjects, not objects (Freire, 1996). The parameters of the OER program, and the attributes of the OER created, are contextualized to the institutional setting and students, which informs the data collected in this study.

There are a few distinctions in the City Tech OER initiative worth mentioning. First, the initiative is not an open textbook adoption program; rather, it supports faculty creation of OER by curating multiple open and alternative educational materials that constitute a collection of course materials. Another unique feature of the City Tech initiative is the OER hosting platform. OER are accessed via the City Tech OpenLab, an open teaching and learning system (as opposed to closed systems like BlackBoard) where it is possible for the City Tech community to interact with each other and the public (OpenLab, 2016). To support the open pedagogy philosophy, the OpenLab has several out of the box, mobile optimized environments for students and faculty to create their own websites including portfolios, projects, and clubs. Faculty have additional authorization to create course sites that can be open to the public, thus accessible to students after their City Tech careers end.

Features of the Health Psychology OER

In this study, students shared their experiences with and assessment of the OER in their interdisciplinary Healthy Psychology course. This course was particularly suitable for the OER initiative because the curriculum tackles contemporary issues from various disciplines such that the instructor felt there was no suitable textbook on the market.

The learning materials that were collected and posted together on a course site are a combination of existing OER, instructor generated OER, and external sites that are linked out to, including library subscription materials, and free to view materials that are not Creative Commons licensed. Learning objects consisted of video lectures, news, and scholarly articles, PowerPoint slides, and readings from Boundless, a for-profit company, which at the time the study was conducted, provided free textbook content with customized quiz and assessment activities on a digital platform. Since the course site includes all required assignments and readings, the syllabus, due dates, etc., it is distinct from a traditional textbook because everything is in one online location - effectively a full course "hub" housed on a WordPress powered site on the City Tech OpenLab.

Figure 1. Detail of the Health Psychology landing page.

Figure 2. Detail of the Assignment Outline page.

Figure 3. Detail of the Assignments (by subject) drop down menu.

Figure 4. Detail of the mobile version view.

Literature Review

Current literature discussing the impacts of OER programs in U.S. postsecondary institutions is growing but diminutive. The literature reveals that a frequent method of assessing these programs, particularly the efficacy of the OER, employs student performance metrics such as exam scores, pass rates, GPA, final grades, assessment instruments specific to disciplines, and withdrawal rates and credits earned. Student and faculty surveys or questionnaires are another method commonly used to assess OER programs. Rarely are interviews and focus groups conducted to learn about student perceptions of, and experiences with OER in depth.

Hilton's (2016) recent synthesis of literature assessing OER programs published through September 2015 discusses 16 studies, nine of which utilized student performance measures to explore the efficacy of the OER in comparison to formerly assigned textbooks, with learning outcomes and course enrollment trends. Hilton analyzed and critiqued the rigor of the methods used in these studies and cited flaws and limitations in research design. These included suboptimal sample sizes for control and treatment groups, and multiple uncontrollable variances. Hilton concluded that despite the varying results of the nine studies, students' learning did not decrease with OER, and their performance remained the same.

Surveys are also a frequent component of OER assessment studies that seek to learn about students' perceptions of OER. At the time of writing this, I am aware of eight published articles (Bliss, Hilton, Wiley, & Thanos, 2013a; Bliss, Robinson, Hilton, & Wiley, 2013b; Feldstein, Martin, Hudson, Warren, Hilton, & Wiley, 2012; Hilton, Gaudet, Clark., Robinson, & Wiley, 2013; Lindshield & Adhikari, 2011; Lindshield & Adhikari, 2013; Pitt et al., 2013; Petrides, Jimes, Middleton-Detzner, Walling, & Weiss, 2011) and two institutional reports (Affordable Learning Georgia, 2015; TideWater Community College, 2015) that conducted student surveys. Two studies evaluating open textbook adoptions: the multi-institutional Project Kaleidoscope initiative (Bliss et al., 2013a, 2013b), and the open statistics textbooks project at De Anza College in California (Illowsky, Hilton, Whiting, & Ackerman, 2016) employed questionnaires. In contrast to this study, most of the above mentioned research assessed open textbooks with larger student samples. Trends in student responses indicate that OER users were satisfied with the cost savings afforded (Bliss et al., 2013a; Hilton et al., 2013; Petrides et al., 2011). Of the surveys and questionnaires that asked students to compare the OER to a traditional textbook, the majority of students found the OER to be comparable (Bliss et al., 2013a, 2013b) or preferred the OER (Feldstein et al., 2012; Illowsky et al., 2016; Lindshield & Adhikari, 2013; Petrides et al., 2011; TideWater Community College, 2015). Most respondents also reported they would recommend the OER to classmates (Hilton et al., 2013; Pitt, et al., 2013; TideWater Community College, 2015).

The studies had very few negative student responses. Of the two isolated groups that received negative feedback in the textbook transformation grants in Georgia, they both cited problems with usability, organization, and the writing style of the resources included in the OER (Affordable Learning Georgia, 2015). Writing style was also a theme expressed in feedback among students at De Anza College who assessed their open textbooks by considering the level of comprehensiveness, readability, and applicability (Illowsky et al., 2016). Hilton's (2016) literature review indicates that a minority of students across seven of the studies mentioned above found the OER to be inferior.

Several other findings from the above studies are also worth noting. Of those surveyed, 72% of students who used an online Flat World Knowledge textbook in Virginia State University's business program preferred it because of its enhanced portability over textbooks and 62% of students agreed the Flat World Knowledge textbook helped them to engage more with their course than a traditional textbook. The researchers acknowledged that students may have had biased preference toward the OER since it was free (Feldstein et al., 2012). Students who used an open version of a textbook called a flexbook in a Human Nutrition course at Kansas State University also preferred the ability to access multiple formats of the text such as PDF and Google documents versions (Lindshield & Adhikari, 2013).

To date, I am aware of only two other studies that interviewed or conducted focus groups with students. Petrides et al. (2011) studied the impacts of the Community College Open Textbook Project (CCOTP) in 2008, a program conducted with community colleges throughout the U.S. The second study that conducted interviews and focus groups, reported their survey and student performance analysis and did not discuss results from focus groups and interviews (Pitt et al., 2013). The findings of Petrides et al. (2011) support that students most commonly noted cost savings and easier access to the materials as benefits. Portability was a deciding factor in using the open textbook for their course according to 30% of student respondents. Both focus group and survey data supported the researchers' findings that "students revealed they incorporated the open textbook into their current learning practices, specifically with regard to technology use" (Petrides et al., 2011, p. 45). Their analysis also indicates that 77% of students preferred using print versions (hard copy or loose prints) of the open textbook during class time. Despite this preference, a minority of students reported printing out either the entire textbook or portions of it. They also report that students found the open textbook "supported increased interactivity with course materials" (Petrides et al., 2011, p. 45).

Methods

To understand how using an OER impacted students, interviews and surveys were conducted upon receiving approval from the college Institutional Review Board, and obtaining appropriate consent forms from participants. Participant eligibility was limited to students enrolled in the Health Psychology course whose section was assigned the OER created by their instructor in the OER pilot program, who also taught all three sections. Surveys and interviews were conducted anonymously. Both instruments were designed to stimulate dialogue about usability, and the curricular materials comprised in the Health Psychology OER.

Two students from each of the three Health Psychology course sections were recruited for interviews on a voluntary first-come, first-served basis. A total of six students participated in the semi-structured one-on-one interviews lasting approximately 45 minutes each and were compensated for their time with $15 MetroCards. Participants were read and provided with a print copy of standard instructions about the interview process to maintain the consistency of each interview (Creswell, 2009) and each of the 12 interview questions were read aloud to students via script. It was also stated that both the participant and the interviewer could ask questions to clarify and follow up as needed. This interview format provided the flexibility to pose open-ended questions that counterbalanced close ended survey items in order to glean more nuanced and detailed perspectives from students in their situated learning environments.

Interview items asked where, with what devices, and how students used the OER to complete coursework. Students were also asked for their feedback on the usability of the OER, their perspectives on the resource as their primary course material, and how using the OER compared to a traditional textbook.

There were a total of 86 students eligible to complete the survey from three sections of the Health Psychology course, and there were a total of 67 respondents. The aim of the survey protocol was to capture as much student feedback as possible to compare with student interviews. Several preceding studies were consulted (Feldstein et al., 2012; Hilton et al., 2013; Bliss et al., 2013; Affordable Learning Georgia, 2015) that asked similar questions in students surveys. Additionally, the OER Research Hub's shared list of survey questions, and some online learning literature focusing on usability and instructional design (Young & Hoerig, 2013; Lee, 2014) were also consulted for additional guidance on survey items. Paper surveys were distributed to students at the end of each class section. I provided students with a brief description of the research project and the purpose of the survey, and reminded students that the process was entirely voluntary, anonymous, and was unrelated to their course or grades at City Tech.

The survey consisted of 12 questions that addressed demographics, and whether students had prior experience with the OpenLab. Another item asked about routine methods of accessing traditional textbooks. Remaining questions addressed the OER in relation to accessing the resource, completion of coursework, usability, navigation and ability to locate materials, quality, and students' overall evaluation of the OER, especially compared to a traditional textbook. Likert scales captured student responses with frequencies, levels of agreement, likelihoods, and levels of difficulty. There were also multiple-choice questions, some of which included an option for open ended responses.

Research Design Challenges

Nomenclature was a challenging aspect of creating the research protocols. Students were not expected to be able to define or be familiar with open educational resources and the acronym: "OER." Therefore, in both the survey and interview protocols, these terms are omitted. Instead, the OER are described in plain, discernable language. For this study, it is appropriate to refer to the Health Psychology resource as an OER, but it was consistently referenced to students as the Critical Health Psychology course site and discrete learning objects within the site were referred to as "course materials" on the Critical Health Psychology course site.

Results and Discussion

Demographics

Demographics indicate that students surveyed in three sections of Health Psychology were diverse in their levels of academic experience and disciplinary tracks. Students represented a wide distribution with fifteen distinct majors reported among the 67 survey respondents. Computer Systems Technology - a baccalaureate program - was the most frequently reported major with 24 (36%) students. The next most prevalent majors were Hospitality Management and Nursing; each with eight (12%) students. The third most frequently reported major was Human Services, with five (7%) students. The remaining 22 were divided among 11 different majors, with one to three students per major. Students who were interviewed studied Human Services, Computer Systems Technology, Radiologic Technology, Electrical Engineering Technology, and Nursing. The majors reported indicate that students from both baccalaureate and associate degrees were well represented in the study sample.

Table 1

Distribution of Student Majors (n=67)

Of those surveyed, 80% of students reported having taken between three to eight semesters of college - this was the same range reported by students in the interviews - and the remaining 20% of students completed between 9 and 14 semesters. Students also indicated whether they used the OpenLab platform prior to this course: 38 (57%) students used the OpenLab prior to taking the Health Psychology course, and 29 (43%) students had not.

Access and Technology

When surveyed about what methods they used to access their assigned readings from the OER, 70% of students reported reading on their personal laptop or computer all or most of the time. This is an interesting finding given the transient nature of urban commuter students, and the ubiquity of mobile phone usage. Nevertheless, reading on a smartphone was the next most common method that 31% of students reported doing all or most of the time.

Fifty six percent of students reported reading with a City Tech computer regularly (31%) to all or most of the time (25%). Students least frequently read by printing their readings out: 49% of students rarely to never printed readings, while only 16% printed all or most of the time. Half of the students interviewed mentioned not printing anything either. The second least common method of accessing the readings was via tablets: 40% of students said they rarely to never used tablets. The drop in tablet usage contrasts Regalado and Smale's (2014) earlier reports that indicated a rise in tablet usage in 2011.

Table 2

Distribution of Student Responses Regarding How They Access the Assigned Readings From the Health Psychology OER (n=66)

Printing

Respondents indicated why they printed readings if they had done so. Of those who responded, the majority reported printing so they could take notes on the page (65%), while the second most common reason reported was a preference for paper (41%). The response rate for this question was 46. In the prior question there were 66 respondents. Based on this deviation in number of respondents, one could infer that about 30% of students found the question not applicable to them, which suggests roughly that percentage of students did not print at all.

Table 3

Distribution of Student Responses to the Question: "If You Printed Out Any Readings for the Course, Why Did You Print?" (n=46)

Completing Assignments

While most of the students interviewed in this study mentioned using a mobile phone to complete readings and quizzes online from time to time (out of convenience, or to maximize their time), two students preferred completing course assignments with computers on campus. The remaining four students used a combination of smart phones, tablets, laptops, and desktops. During interviews, students shared how they accessed the OER on a daily basis.

Depends on the day, and like what I'm doing. I live far out in Brooklyn so my train goes outside, so sometimes I'll use it on my phone while I'm on the train... I have a Kindle so I can access it from my Kindle, or if I'm home I just use my laptop.
- Student

The device they used and their work location depended on multiple variables: schedule and type of assignment, preference for working environment, and level of convenience.

[I use the] school computer cuz I don't like using the wifi here cuz it's like a whole process, but when, sometimes I'm on the bus I use my actual data from my phone. And then if I'm home I use my laptop or my actual... [desktop].
- Student

As the findings from Regalado and Smale's (2014) earlier study demonstrated, students' abilities to engage in academic work hinged on access to computers at home or on campus. All of the students interviewed - with one exception - mentioned routine use of school computer labs to work on assignments. The student who did not mention using the computer labs preferred working off campus, and found it easier to work outside of home, so Starbucks was her typical site for academic work. One student completed most of his work at the Library, citing it was optimal to working at home, while a second student preferred completing readings at home, and a third preferred writing assignments at home.

Challenges Accessing the OER

Students most frequently cited difficulties with accessing the OER via campus wifi (27%), and seven (10%) students had difficulty accessing wifi off campus. Though a small percentage of the total sample, this presents a notable barrier to accessing primary course materials, especially given that wifi on campus is unreliable. Three (4%) students reported trouble finding a device to access the OER on campus and two (3%) reported difficulty finding a device to access the OER off campus. Although these numbers are small, they are an indicator that some of our students face tremendous access barriers that should be mitigated with design solutions, and improved services, and infrastructure.

Students also reported difficulties associated with usability that will be discussed in the next section. Due to the design of the survey item, there is no indication whether reported challenges were persistent or isolated, so it is unknown whether students overcame them.

Table 4

Reported Challenges Associated with Accessing Materials from the OER (n=63)

Organization and Usability

Students who were interviewed commonly referred to the site as well organized and easy to use. Other words students used to describe the OER were helpful, user-friendly, and a "guideline to the semester." They also mentioned how the OER is categorized and has tabs or pages for different course components. The "due dates" page on the OER was most frequently mentioned when students described how the OER site was organized.

One student voiced that there was an initial learning curve to understand how to navigate the OER. Despite initial differences in comfort levels and familiarity with the OpenLab platform, all of the students interviewed found it easy to gain proficiency with the OER.

When asked how they would describe the OER to a new user, one student said people would adapt to it right away because it is comparable to technology we routinely use, likening it to social media.

Log on... navigate through it. Wherever you need to go you can find it. Use the search option.
- Student

Students also provided recommendations for prospective users. One said they would encourage people new to the OER to click through everything on the site to get a feel for it and become comfortable navigating. A second student recommended printing out everything on the OER (the tabs/pages that comprise the site) so they know and have a copy of everything they need, including the list of due dates. A third student said that becoming familiar with the OpenLab platform by signing up for an account in advance would help.

The survey also asked students what it was like to locate materials on the OER. Students responded positively by reporting that it was easy (52.2%) or extremely easy (35.8%) to find what they needed (totaling 88%). Seven (10.4%) students had a neutral response, and one student found navigating the OER to be difficult. Paradoxically, a sizable minority of students (24%) also reported having trouble knowing where to find the resources (Table 4). It is unclear whether survey respondents were able to overcome the challenge, though students who were interviewed reported moving beyond initial navigational issues, and did not mention difficulties finding resources aside from one mention of a few broken links.

Table 5

Distribution of Student Responses to the Question: "How Would You Rate Your Overall Ability to Find What You Need on the Critical Health Psychology Course Site?" (n=67)

Preferences and Impacts on Learning Habits

All students who were interviewed mentioned their satisfaction with, and preference for having all the course materials in one place. Two students really appreciated having a full, up-to-date guide covering the semester long course in contrast to past professors who might assign work verbally or on a weekly basis. Both also mentioned liking the option to complete work ahead of time.

It's like having the whole course in the website so you know what to follow...You don't need to wait for the professor to say, "next week this is due, and this is how you do it," it's just like, ok, so next week this is due, and it's explains it to you on the OpenLab.
- Student

I like looking ahead sometimes... when I have free time... all the material and all the homeworks were available already. So I like that... if I wanna do something earlier than the due date I could.
- Student

Most students referred to the due dates as a "go to" that linked to all the supporting materials needed to complete their assignments. This functionality was a welcome departure from their descriptions of scheduling time to visit the library to make copies of assignments.

Two students had contrasting responses to how the OER impacted their academic habits. They both responded in the context of having better access to the OER which created a new level of convenience. One student found this to be a positive influence because she was able to maximize and allocate time to complete work more efficiently. Instead of saving all of her course work for what she described as a "congested" window of six or eight hours when she was home, she could complete some work prior to arriving home, and free up more time to do other course work.

the little short times that I do have that are free, I actually utilize it in order to study, in order to read. While before, if I had the textbook, I wouldn't do that cuz it's kind of a hassle bring[ing] the textbook... I would just listen to music... [and] when I get home, I'll read those chapters.
- Student

The other student found the added convenience almost made her procrastinate over completing the work more. This student also mentioned that she probably studied more frequently because of the accessibility.

Survey respondents rated how each of the course materials supported their learning process, if at all. The responses were very positive, with the majority of students agreeing that each material supported their learning, suggesting that students liked using a variety of educational materials. Students reported that the required assignments and quizzes were most supportive of their learning process.

Table 6

Level of Agreement About Course Materials From the OER That Supported the Learning Process (n=66)

OER Versus Textbook

When asked how the OER was different from a traditional textbook, most of the students responded in the interviews by sharing features of the OER that they liked. Half of the students began their responses by saying the OER was easier to access and more convenient because everything was available in one place - the OER site. Half of the students also contrasted the OER with textbooks by mentioning the cost savings, and most of the students (four of the six interviewees) also shared satisfaction with not having to carry a textbook around. One student mentioned how the OER saved her from developing back pain - something that had happened with textbooks in the past. Other students shared their relief at not having to worry about transporting a textbook to and from class, to work, and elsewhere, which alleviated complications as they planned their academic and life schedules. A couple of students described the OER as helpful and a time saver. A student remarked on how the OER freed her from having to borrow the textbook from a library, something that was time consuming and created stress over worrying when the return date approached. Another student said that textbooks, while required, were rarely utilized in some courses anyway.

Half of the students interviewed preferred the composition and format of the OER much more than a textbook.

It made me more focused... you know, with the textbook- I don't like- I can get kinda lazy about it. But I go on the computer and, it's something that I like to do so it kind of motivates me more to do it.
- Student

Another student felt the quizzes that accompanied the readings on Boundless really helped her test her knowledge and review areas that she didn't initially understand.

[When using] a regular textbook, I would just read it and think in my mind, "Ok you understand everything." But you actually didn't because, I mean, no one forces you to read the questions [in the back] of the textbook.
- Student

A couple of students found the course readings to be far less overwhelming than a textbook because they could focus in on the important components.

When you have the textbook, it's a little bit overwhelming with all the information that's in it. You have to look through it. You just feel like, ok, I have to read these chapters. So it's overwhelming to see how much you have to actually do. But if you have- you print it out from the site, you say, ok, this is what I have to do. This is the only thing I need to do. So it's like you're focused on only what you printed out.
- Student

One student mentioned her preference for watching videos, a learning habit she had developed independently by searching for YouTube videos related to other courses that did not integrate multimedia into the curriculum.

Survey respondents also shared their perceptions of the quality of the OER readings. Eighty one percent of students found their quality to be somewhat to much better than a textbook.

Table 7

Response Rating on the Quality of the OER Course Readings Compared to a Textbook (n=67)

Survey respondents also rated aspects of their course experience against a traditional textbook and the results were largely positive. Students were most positive about the OER increasing their satisfaction with the learning experience, and increased engagement with course lessons in comparison to a textbook. By contrast, 32% of students did not think using the OER instead of a textbook affected their grade and 35% reported it had no effect on their confidence. Twenty percent of students also reported that the OER did not impact their class participation or their interest in the course.

Table 8

Level of Agreement About How the OER Impacted the Learning Experience (n=66)

Most survey respondents indicated they were likely to register for a course with a similar resource again (89), with 6 students (9%) neutral, and 1 student unlikely to do so in the future.

Table 9

Response for Likelihood Students Would Register for a Class Using a Resource Similar to the Health Psychology OER Instead of a Traditional Textbook (n=66)

Criticism

Students suggested that adding email notifications for important approaching deadlines, as BlackBoard does, would be helpful, making PDF versions of the readings would be an improvement, and that navigating to the OER site within the OpenLab platform could be easier. One student shared that there were glitches with using the drop down menu on the OER with smartphones and tablets, and that the OER had a few broken links. Overall, criticism of the OER was minimal.

Conclusion and Recommendations

Study findings indicate that students used a combination of digital devices to access the course readings, yet despite the prevalence of handheld and portable devices, readings and completion of assignments happened most frequently on personal and/or school laptops and desktops. The majority of students also preferred the OER to traditional textbooks. The latter is consistent with previous studies, but distinct in that most studies were about open textbooks as opposed to a curated OER site. Preceding studies also report that students were attracted to OER for the cost savings and ease of use and that was true for students in this study as well. Students in this study also voiced their satisfaction with not having to carry a textbook as they traveled throughout the day.

Convenient access to course materials is an issue of particular gravity among underserved populations and urban commuter students who typically balance many commitments in addition to their studies. The manner students discussed their satisfaction with the convenience of having all the course materials organized pragmatically in one location suggests it is worth considering how the scaffolding of the course on the OER impacts students' academic motivation and agency. Other preferences of students distinctly aligned to improved learning habits and enhanced teaching methodologies. For instance, students also expressed preference for more targeted and concise readings with accompanying quizzes that align with active reading and formative assessment. I believe students' reports on the convenience of OER prompts us toward deeper analysis and inquiry on this front, particularly relating to instructional design and integration of active and high impact learning methods.

Educators and administrators are often in the position to make decisions that have lasting impact on students. In this study, I hoped to expose student voices to bring learners' narratives in dialogue with pedagogical praxis. The emergence of OER is an important pedagogical shift that needs constructive channels of dialogue among learners and educators to be implemented effectively. OER practitioners can benefit from conducting research that yields in depth data by employing qualitative methods such as student focus groups and interviews that explore usability, learning habits, and technology access and proficiency. Students have valuable perspectives as demonstrated by the results of prior OER studies that incorporated student critiques (Petrides et al., 2011; Illowsky et al., 2016). The recommendations students shared on how their open textbook could be become more interactive with annotation tools, audiovisual elements, and a venue for online dialogue with professors

Limitations

By design, this study was not meant to contribute statistical data; the inclusion of the survey had the primary function of informing interview data. While the strength of this study is its reliance on student perceptions, human subjects also introduce biases and their data cannot be externally validated. Additionally, recruitment for students interviewed could have favored those with more schedule flexibility and better time coordination skills, thus may have impacted the qualitative data sample and demonstrates that achieving a truly representative student sample is not possible.

References

Affordable Learning Georgia. (2015). Textbook transformation grants round one: Spring semester 2015 final report summary. Retrieved from http://affordablelearninggeorgia.org/documents/ALG_R1_Final_Report.pdf

Bell, S. (2014). Ditch your textbook: Moving to OER and Alt-Textbooks. Temple University Faculty Herald, 44(4). Retrieved from http://www.temple.edu/herald/44_4/DitchYourTextbooks.htm

Bliss, T., Hilton, J., Wiley, D., & Thanos, K. (2013a). The cost and quality of open textbooks: Perceptions of community college faculty and students. First Monday, 18, 1.

Bliss, T., Robinson, T. J., Hilton, J., & Wiley, J. (2013b). An OER COUP: College teacher and student perceptions of open educational resources. Journal of Interactive Media in Education. 2013(1). Retrieved from http://jime.open.ac.uk/articles/10.5334/2013-04/

City University of New York [CUNY]. (2016). Mission & history [Web blog]. Retrieved from http://www2.cuny.edu/about/history/

Creative Commons. (2016). Share your work [Web blog]. Retrieved from https://creativecommons.org/share-your-work/

Creswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE Publications.

CUNY Office of Institutional Research and Assessment. (2014). 2014 Student experience survey. New York: CUNY Office of Institutional Research and Assessment.

Feldstein, A. M., Martin, M., Hudson, A., Warren, K., Hilton, J., & Wiley, D. (2012). Open textbooks and increased student access and outcomes. European Journal of Open, Distance and E-Learning, 2, 1-9. Retrieved from http://eric.ed.gov/?id=EJ992490

Freire, P. (1996). Pedagogy of the oppressed. (M. B. Ramos, Trans.). New York: Penguin. (Original work published 1968).

Hewlett. (2016). Open educational resources. Retrieved from http://www.hewlett.org/programs/education-program/open-educational-resources

Hilton, J. (2016). Open educational resources and college textbook choices: a review of research on efficacy and perceptions. Educational Technology Research and Development, 1-18. Retrieved from http://link.springer.com/article/10.1007/s11423-016-9434-9

Hilton, J., Gaudet, D., Clark, P., Robinson, J., & Wiley, D. (2013). The adoption of open educational resources by one community college math department. T he International Review of Research in Open and Distributed Learning, 14(4). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1523

Illowsky, B. S., Hilton, J., Whiting, J., & Ackerman, J. D. (2016). Examining student perception of an open statistics book. Open Praxis, 8(3), 265-276. Retrieved from http://openpraxis.org/index.php/OpenPraxis/article/view/304/218

Lee, J. (2014). An exploratory study of effective online learning: Assessing satisfaction levels of graduate students of mathematics education associated with human and design factors of an online course. The International Review of Research in Open and Distributed Learning, 15(1).

Lindshield, B., & Adhikari, K. (2011). The Kansas State University human nutrition (HN 400) flexbook. EDUCAUSE Review. Retrieved from http://er.educause.edu/articles/2011/12/the-kansas-state-university-human-nutrition-hn-400-flexbook

Lindshield, B., & Adhikari, K. (2013). Online and campus college students like using an open educational resource instead of a traditional textbook. Journal of Online Learning & Teaching, 9(1), 1-7. Retrieved from http://jolt.merlot.org/vol9no1/lindshield_0313.htm

OpenLab at City Tech. (2016). About the OpenLab. Retrieved from https://openlab.citytech.cuny.edu/about/

Petrides, L., Jimes, C., Middleton-Detzner, C., Walling, J., & Weiss, S. (2011). Open textbook adoption and use: Implications for teachers and learners. Open learning, 26(1), 39-49.

Pitt, R., Ebrahimi, N., McAndrew, P., & Coughlan, T. (2013). Assessing OER impact across organisations and learners: experiences from the Bridge to Success project. Journal of Interactive Media in Education, 2013(3). Retrieved from http://jime.open.ac.uk/article/view/2013-17/501

Regalado, M., & Smale, M. A. (2014). Commuter students using technology. EDUCAUSE Review. Retrieved from http://er.educause.edu/articles/2014/9/commuter-students-using- technology

Regalado, M., & Smale, M. A. (2015). "I am more productive in the library because it's quiet": Commuter students in the college library. College & Research Libraries, 76(7), 899-913. Retrieved from http://crl.acrl.org/content/76/7/899.full.pdf

TideWater Community College. (2015). THE "Z-DEGREE": Removing textbook costs as a barrier to student success through an OER-based curriculum. Retrieved from https://web.tcc.edu/academics/zdegree/docs/Z-Degree%20Booklet%20for%20Hewlett.pdf

UMass Amherst Libraries. (2016). Open education initiative. Retrieved from https://www.library.umass.edu/services/teaching-and-learning/oer/open-education- initiative/

Young, A., & Hoerig, B. (2013). Utilizing student feedback to inform faculty development activities for online course development and delivery. International Journal on ELearning, 12(4), 439-453.

What Impacts do OER Have on Students? Students Share Their Experiences with a Health Psychology OER at New York City College of Technology by Cailean Cooney is licensed under a Creative Commons Attribution 4.0 International License.