International Review of Research in Open and Distributed Learning

Volume 18, Number 6

September - 2017

Using Design-Based Research to Develop Meaningful Online Discussions in Undergraduate Field Experience Courses

Carol Johnson, PhD¹, Laurie Hill, PhD², Jennifer Lock, PhD¹, Noha Altowairiki¹, Chris Ostrowski¹, Luciano da Rosa dos Santos³, and Yang Liu^1
1University of Calgary, ²St. Mary's University, ³Mount Royal University

Abstract

From a design perspective, the intentionality of students to engage in surface or deep learning is often experienced through prescribed activities and learning tasks. Educators understand that meaningful learning can be furthered through the structural and organizational design of the online environment that motivates the student towards task completion. However, learning engagement is unique for each student. It is dependent on both how students learn and their intentions for learning. Based on this challenge, the design of online discussions becomes a pedagogical means in developing students' intentionality for the adoption of strategies leading to deep learning. Through a Design-Based Research (DBR) approach, iterative design of online learning components for undergraduate field experience courses were studied. For this paper, the focus of the research is on examining factors that influenced deep and surface levels of learning in online discussion forums. The results indicate that design factors (i.e., student engagement, group structures, and organization) influence the nature and degree of deep learning. From the findings, two implications for practice are shared to inform the design and scaffolding of online discussion forums to foster deep approaches to student learning.

Keywords: instructional design, online discussions, deep learning, participation

Introduction

Online and blended approaches have become a prevalent means for delivering courses in post-secondary institutions during the past decade. While these approaches offer accessibility and flexibility to students who are geographically dispersed, the online course environment complements and supports student learning in a variety of on-campus learning contexts. Some on-campus classes, using a blended approach, also incorporate opportunities for online discussions so that student conversations can continue beyond the classroom (Zhang, Gao, Ring, & Zhang, 2007). Online learning environments (e.g., discussion forums and collaborative learning tasks) have the potential to engage students in a range of interactive experiences that enhance learning (Garrison, 2011; Garrison, Anderson, & Archer, 2000; Harasim, 2012). The problem to be investigated is the intentionality in the design and scaffolding in the online discussion areas. That is, when creating and nurturing student engagement through higher-order cognitive activities, attention needs to be given to the nature of the questions used, the formation of activities, and organizational structures that lead to deep learning. Identifying the extent to which online interactions demonstrate meaningful student learning becomes key to understanding the learning potential that is afforded through such things as online discussion forums.

Online course designs within a Learning Management System (LMS) can include a variety of elements such as email communication (asynchronous), chat rooms (synchronous), discussion forums, repository spaces for course content, and dropbox folders for student document submissions. These features offer students interactive opportunities including student-to-student, student-to-instructor, and student-to-content (Moore & Kearsley, 2005). Student-driven decisions of how to engage in an online environment impacts learning outcomes (Balter, Cleveland-Innes, Pettersson, Scheja, & Svedin, 2013). From the viewpoint of student learning, this paper examines findings of the online learning components within a two-year study informed by Design-Based Research (DBR). The implications and conclusions of the findings demonstrate the need for implementing online course design and management strategies to promote greater learning opportunities through asynchronous discussions.

Background

Framing the Theoretical Context

One shift in higher education is the growing emphasis placed on the learner and the teaching and learning interaction (Harasim, 2000; Lublin, 2003). Conceptions of good teaching have changed to highlighting teaching that engages students in course content. Rather than a focus on instructor actions, teaching practices have shifted focus to facilitating students' learning and encouraging student self-initiative in learning task completion (Harasim, 2012; Lublin, 2003). In an evaluation of the meta-analysis of online learning completed by Means, Toyama, Murphy, Bakia, and Jones (2009), it is suggested that instructors have become more focused in how students approach learning and how to engage students in a deep understanding of subject curricula. Further, Francescucci and Foster (2013) found "instructor-directed and collaborative/interactive learning results in significantly better outcomes than does independent/active learning" (p. 81).

Focused on meaningful learning, developed through interaction and collaboration, our theoretical context is based on the premise of deep learning as identified by Biggs (1999). Biggs suggested that good teaching fosters development of deep learning approaches by students. Collaborative tasks, group work, and simulations are instructional strategies that encourage deep learning (Chatti, Jarke, & Specht, 2010; Darling-Hammond, 2006). Motivation and attitude factor into the approach a student takes toward learning (Balter et al., 2013). Students are not bound by one approach. Rather they make choices on how to approach learning based on the context and their learning goals. For example, students choose how they will craft their discussion posts and the level of detailed learning they aim for. While deep learning is preferable (Donnison & Penn-Edwards, 2012), Diseth (2003) noted that deep and surface approaches are not contradictory. Simpson (2003) suggested that a balance between surface and deep is a realistic approach to learning in higher educational settings.

Deep Learning

In the early research of deep- and surface-level processing, Marton and Säljö (1976) identified the learning approaches of students as the exploration of the "what" and "how much" (p. 4) of a topic or content item. Entwhistle and Entwhistle (2005) defined deep learning as,

the intention to understand, and that intention evokes the processes of learning which allow deep levels of understanding to be reached. The deep approach involves relating new information to prior knowledge and experience in ways which transform the information and create personal meaning. (p. 145)

Deep learning was originally defined within Marton, Dall'Alba, and Beaty's (1993) six conceptions of learning: "1) increasing one's knowledge; 2) memorizing and reproducing; 3) applying; 4) understanding; 5) seeing something in a different way; and 6) changing as a person" (as cited in Burnett, Pillay, & Dart, 2003, p. 56). The surface approach focuses on fact acquisition, memorization, and the seeking of quantitative amounts of knowledge. Consequently, the surface approach to learning is often about "achieving course requirements with the minimum of effort" (Donnison & Penn-Edwards, 2012, p. 10).

A pathway for learning involves the interconnections of deep and surface learning approaches while intentionality of learning can help students navigate toward deep learning (Diaz & Diniz, 2014; Marton, Hounsell, & Entwhistle, 2005). For example, memorization can be a strategy for developing deep learning:

Committing meaning to memory is more complex than committing words to memory, because meaning implies a distinction between words and meaning. Understanding (meaning) is more complex than remembering (meaning) because both imply permanence, but the former implies an act of constituting (or finding) meaning as well. (Marton, Hounsell, & Entwhistle, 2005, p. 57)

Lublin (2003) further explained the usefulness of memorization for certain areas of learning, such as when learning a foreign language. The caveat is the singular use of this learning strategy can hinder deep learning. The inclusion of multiple strategies such as memorization, application, and creation become pathways for adopting intentionality of learning particular content. The achievement of deep learning is dependent on the variety of pathways that students adopt such as memorizing vocabulary terms which can permit deeper discourse and knowledge exchanges in the topic area.

Designing for Deep Learning in an Online Context

Purposeful design that incorporates online communication between students and instructors, as well as between students and peers fosters effective learning interaction (Tu & Corry, 2003) and also encourages collaboration. Through online discussion forums and question prompts, students can be encouraged to consider and reflect on their experiences and consolidate their learning (Eastman, 1995; Garrison, 2011). This suggests online learning tasks should promote constructivist thinking and provide students with deep-learning opportunities to think critically, make connections, and share exemplars of their learning. To elicit metacognitive responses of reflection and application from students, faculty must provide well-structured, deep-learning discussion questions. That is, "asynchronous discussion designs may sound as easy as an instructor posting questions and the students responding, but an effective design, management, and strategy is required for students to internalize [learning]" (Tu & Corry, 2003, p. 304). If discussion questions are not well-designed, students' postings and responses tend to limit the nature of the interaction and may not result in effective student learning (Tu & Corry, 2003).

Opportunities for students' deep learning are enhanced when they have choices for learning interactions (Lublin, 2003). Effective learning designs can offer opportunities for students to make choices for adopting personal learning strategies and can be exemplified in the instructor's purposeful design of asynchronous discussions. For example, this may mean that an instructor includes related discussion topics before and after specific content items are introduced. Tu and Corry (2003) further summarized the importance of careful planning in course design. Specifically, balance is required to assist students in amplifying learning; when the "structure is too rigid, dialogue and interaction are stifled... [In contrast] students are likely to become confused or lose interest when the discussion is totally without structure" (p. 304). Student engagement is best assured when students understand what is expected of them in the online environment. This awareness includes: a clear understanding of the discussion cycle, the depth or number of required discussion threads, the frequency of discussion, the number of postings, the expectations for interaction between students and between students and instructors, and evaluation criteria (Tu & Corry, 2003).

As student populations in online environments become more diverse, instructors need to seek additional strategies and ways to meet the learning needs of their students (Meyer, Rose, & Gordon, 2014). Meeting student needs requires that course information and learning tasks are accessible for all learners. Accessibility should address both the applications used for students to retrieve learning content and how students demonstrate their knowledge and understanding. "To this end, application of the Universal Design framework to education holds much promise in helping educators design learning environments that are maximally accessible for diverse learner populations" (Kumar & Widman, 2014, p. 126). With an emphasis on personalized student learning and a grounding in neuroscience, Universal Design for Learning (UDL) is a useful instructional framework that facilitates opportunities for student success (Kumar & Widelan, 2014). UDL encompasses three key components to enable students to: 1) access learning content, 2) demonstrate learning, and 3) participate in learning (Meyer et al., 2014). UDL components offer a powerful framework to enhance student learning experiences; however, it requires a strategic implementation of purposeful design to succeed (i.e., thought and planning of UDL principles).

Strategies for Assessing Deep Learning in Online Discussions

Online instructors need to understand how students engage with course content (Garrison, 2011). This involves understanding the cognitive and social processes activated in an online environment (Henri, 1992). Specific to online discussions, this implies that instructors understand how their students interact online and the students' preferred learning strategies. For example, Garrison's (2011) teaching presence highlighted the need for instructors to be attentive to the content and development of student discussion posts. They should also be alert to the choices of students' interactions (e.g., frequency of posting) and collaboration (e.g., with whom they post) in the shared space online. Attentiveness to students' decisions allows the instructor to "offer immediate support to the individual [student] and the collective learning process" (Henri, 1992, p. 118).

Assessing students' level of engagement in deep learning versus surface learning interactions can help instructors identify areas to better support learning. Such measures of engagement can provide helpful critique to students for identifying their understanding of content knowledge and meaning in addition to understanding their learning approaches and strategies in the online environment (Henri, 1992). To this end, Henri (1992) developed an analytical framework for understanding online discussion posts that allows the instructor to identify the learning approaches used by students and the cognitive meaning and engagement within their writing. His framework highlighted five dimensions that take place within an online learning environment: "participative, social, interactive, cognitive, and metacognitive" (p. 124). His operational definitions and indicators for each dimension allow instructors to interpret the meaning of students' written text and offers insight into the students as learners.

Table 1 highlights indicators present in typical discussion posts for each dimension. In-depth examples of each indicator is described in Henri's (1992) work.

Table 1

Summary of Henri's (1992) Five Dimensions of Content Analyses

The five dimensions help to explain the depth to which students understand the content topic as they use or do not use reflection, application of content knowledge, and critical thinking skills in their writing. To refine his description of the cognitive dimension, Henri (1992) developed criteria for distinguishing surface processing from in-depth processing. This refinement was informed by the work of Marton, Hounsell, and Entwistle (1984), and by Entwistle and Waterston (1988). Their studies "indicate[d] that the learning process is influenced by the level at which information processing occurs" (Henri, 1992, p. 130). An in-depth processing on the part of students is linked to cognitive skills such as interpreting, inferring, providing proof, or developing solutions (Henri, 1992). Through a content analysis of student online discussion writing, an overall sense of students' deep- and surface-learning can be identified.

Context of the Study

This study was conducted over a two-year period in a two-year after-degree Bachelor of Education program at a western Canadian university. Students in the program have already completed a minimum of one degree or are simultaneously completing a combined degree in another discipline. The field experience component for students occurs once in each of the four semesters. Field experience - also referred to as practicum - is where students work with mentor teachers to develop skills and competencies in K-12 classroom settings. Instructors for field experience courses supervise field experiences and help to bridge theory (i.e., what is taken up in program courses) to practice (i.e., what is observed in action in classroom and school settings).

Each field experience course is supported by learning in an online environment. Each field experience instructor is given a Desire2Learn (D2L) course shell for sharing information and engaging with students (e.g., asynchronous discussions and email) during their practicum. In this online space, a variety of content is shared along with access to asynchronous discussion forums.

For the purposes of this article, the data focuses on the discussion forums for two iterations of one field experience course that was four weeks in length. The selected course occurred at the same time each academic year. The instructors in the first iteration were not necessarily the same as those in the second iteration and each instructor was responsible for facilitating their online environment during the field experience.

Method

Design-Based Research Methodology

Reeves (2000) identified that the use of design-based research allows for the strength of "reflection" (p. 9) to promote problematization and solutions through the exploration of design. A Design-Based Research (DBR) methodology was used to develop and inform the implementation of an innovation, in this case using multiple means of representation and engagement through the Universal Design for Learning (UDL) framework (Meyer, Rose, & Gordon, 2014), in online field experience courses. The innovation particular to this study is that a master online course shell was created by the research team (i.e., DBR team) for each field experience course and then duplicated so each instructor had their own course shell. By providing this template, common content and information was shared, and it also allowed instructors to customize in response to the needs of their particular cohort of field experience students.

The flexibility of DBR and its iterative cycles of testing and refining the solution are based on evidence-informed decisions (McKenney & Reeves, 2012). The methodology fostered multiple iterations including design, the collection of feedback, the re-design based on feedback, and opportunities for immediate stakeholder feedback on the design of the online course environments in D2L.

An initial gap analysis was conducted to help conceptualize the design of the online experience in D2L using principles of UDL. In this paper we focus on the discussion forum data in the DBR process. Discussion forum data received after the implementation of the first iteration informed the re-design for the second iteration. The following three examples reflect how the DBR team used the data to guide the refinement and re-design. First, examples of each of Henri's (1992) dimensions were depicted in sample discussion postings. This provided students with a model for what each of the dimensions looked like for developing their own responses. Second, the inclusion of a course tour video assisted students in better navigation access to the discussion areas (and the rest of the D2L environment). Third, how-to resources were included for students on how to create an audio or video version of their discussion postings.

Data Sources and Analyses

DBR is known to involve a mixed methods approach to data collection (Anderson & Shattuck, 2012). DBR allows for the exploration of quantitative online survey data combined with student and faculty semi-structured interviews and student discussion posts. Student online survey and student and instructor interview data were collected directly after the completion of the field experience four-week course.

Online survey questions elicited information on the design of the online component in D2L and the collaborative nature of the online environment. A pilot study occurred with the questionnaire to verify that questions addressed key issues in terms of the design of the online environment using UDL. The survey contained two parts: 1) students used a five-point Likert scale to rate the design of the online environment; and 2) students responded to open-ended questions. Descriptive statistics were used to analyze part one and thematic analysis was completed for part two.

Individual and focus group interviews with instructors occurred after completion of the field experience course. During the interviews, questions explored how the re-design influenced online interaction and collaboration, engagement fostered by the online environment, and elements that helped facilitate learning in the online environment as well as what changes they would recommend to enhance the online student experience for field experience courses. Interview data provided qualitative understanding of the interactions experienced in the online space. Member checking was used with interview transcripts (Denzin & Lincoln, 1998).

Interviews were analyzed using first- and second-cycle thematic coding (Miles, Huberman, & Saldaña, 2014; Saldaña, 2013). First-cycle coding involved In Vivo coding (Saldaña, 2013) followed by descriptive coding to identify themes resulting from the interview data. Themes were then categorized to identify primary themes from the data corpus.

Online discussion postings in D2L were analyzed using Henri's (1992) computer conferencing and content analyses to assess text and multimedia discussions. We used this framework to analyze content of the discussion forums where instructors and students had the opportunity to post through text, audio, and video. Examples were provided in the online environment to illustrate how multimedia could be used in parallel or instead of traditional text. The overall course design encouraged the use of multiple forms of representation in the discussions.

Henri's (1992) form of content analysis "transforms qualitative data to quantitative data through the coding of student responses and ratings made by multiple researchers" (Darabi, Arrastia, Nelson, Cornille, & Liang, 2011, p. 219). As a method for evaluation, these dimensions are helpful in developing a deeper understanding of the content and meaning of online entries. As well, this method helps to foster greater understanding of the learning approaches and strategies implemented by students (Henri, 1992).

Student and instructor posts were collected in the first year, iteration one (n = 21), and in the second year, iteration two (n = 20). Data collected from the online course discussions in iterations one and two (319 posts and 330 posts, respectively) were coded independently by four researchers using Henri's (1992) five analytical dimensions. These dimensions (participative, social, interactive, cognitive, and metacognitive) were applied to students' posts guided by the indicators developed by Henri. The researchers used interrater reliability to confirm indicators.

When researchers are directly involved as both designers and researchers, as in this study, issues of credibility can arise. Barab and Squire (2004), noted the intimate connection of designer and researcher invested within a project as problematic for credibility of the work. However, Anderson and Shattuck (2012) suggested that while a valid critique, researcher bias can be a valuable strength when other safeguards for academic rigour and trustworthiness are upheld. In this study, academic rigour and trustworthiness were addressed through multiple processes.

Results

Findings from the content analyses of the discussion posts are shared first to highlight the varied approaches to learning visible in the online discussion area. Influential design factors are addressed in the second section. Both data sources illuminate the specific learner context of the online component of the two iterations of one field experience course.

Content Analyses of Online Discussion Posts

In each of the four weeks, students were given a new discussion question from their instructor. The discussion questions encouraged students to find links between their on-campus coursework and experiences in the practicum environment. Students were expected to respond to each question and reply to their peers to develop pedagogical insights and connections, and extend understanding of classroom practices.

Table 2

Dimension Range for Discussion Posts in Iteration One and Iteration Two*

* All posts were coded as participative. Itr = iteration, n = participants, NP = discussion posts.

Findings represented in Table 2 reflect the range of dimensions (e.g., social, interactive, cognitive, and metacognitive) for both iterations. Given that the students enrolled in the field experience courses over two years were unique to each iteration, and instructors had personal choice in the number and kind of questions posed, the data cannot to be considered for comparison. The table highlights the diversity of the classification of postings and the tendency for discussion posts to be focused in the interactive and cognitive dimensions.

Social dimensions. Social dimension posts contained information that, while relevant to the student practicum experience, were "not related to formal content or subject matter" (Henri, 1992, p. 125). These posts often pertained to general aspects of the learning tasks or shared information. The following is an example of a social dimension post. For example, one student posted:

Hi everyone! I went to the school yesterday and I was assigned a specific parking space, so make sure you ask for it when you go in. There was also a ton of parking space at the community center beside the school, as well as street parking. (Student discussion post)

While social dimension posts aid in developing a sense of connection and community within the group, they do not further the direct learning of students. However, if "affective support plays a greater or lesser role in the learning process" (Henri, 1992, p. 127) then social posts can contribute to the overall learning process for students. In both iterations of the field course we examined, social dimension posts accounted for 1.25% of posts in the first iteration and 12.12% of posts in the second iteration.

Interactive dimensions. In this study, 62.70% of posts in the first iteration and 57.27% in the second iteration were deemed as interactive posts. Each of these posts aligned to Henri's (1992) description of "chain of connected messages" (p. 125) that often focus on the learning dynamics (e.g., collaboration, group interaction, discussion, etc.) among students. Such posts may offer support or encouragement to peers by articulating new ideas and insights. The following student interactive discussion post example was in reply to a peer:

You are so right. I also learned during this practicum that relationships are crucial to teaching! I found that just learning their names and learning something interesting about each student does wonders in how they respect you and how comfortable they are opening up to you and approaching you. In building a strong relationship with students it makes the learning easier because you not only have an idea of how each student learns best but makes students want to learn with you. (Student discussion post)

This post exemplifies the aspects of active engagement of the student and the collaborative element in the learning process found in the online environment. Another discussion post demonstrates an implicit interaction between students. "I really like the idea of doing individual check-ins with students. I think it is very important to take the time for one on one conversations in order to gain an understanding for how each student is doing." This student also is responding indirectly to an expressed idea without referring to the original comment. Both posts describe the students' preferences for a particular learning approach that is found in the interactive dimension.

Cognitive dimensions. The cognitive dimension is defined by the overt demonstration of cognitive skills that support significant learning. These skills include "understanding, reasoning, the development of critical skills, and problem resolution" (Henri, 1992, p. 129). We identified and tabulated 35.11% in the first iteration and 29.9% in second iteration as cognitive discussion posts. The discussion post below, reflects an example of Henri's (1992) cognitive dimension. This student demonstrated an in-depth cognitive processing as their writing reflected a linking of ideas that allowed for an interpretation within a larger perspective.

I could not agree more about the importance of lining up theory with the students' individual needs-especially when it comes to differentiation. I felt that way in my practicum as well. It is so important to think about the need for the student first and foremost. Theory is the academic side to learning and I can understand the importance of learning it all. However, I think it is more important to focus on how these theories can benefit children in a positive way. I think more focus should be placed on how we can connect theory to the classroom and improve the lives of our students. (Student discussion post)

Understanding the learning approach and level of cognitive thought demonstrated by students can allow instructors to provide the most appropriate support.

Metacognitive dimensions. While the metacognitive process is "difficult to observe" (Henri, 1992, p. 131), and measure within a traditional teaching and learning situation, it is possible to discern learning processes through the application of Henri's (1992) framework. A small percentage of posts marked as metacognitive: 0.94% in the first iteration and 1.51% in the second iteration. Metacognitive discussion posts are characterized by the application of higher level thinking skills such as critical reflection, application of the new knowledge in practice, evaluation of one's own learning and self-awareness (Henri, 1992). The following student discussion post shows elements of the metacognitive dimension found in the online discussion forum of the field experience course:

I think that the element of the unexpected is one of the most powerful aspects of the whole experience. To that end, the metaphor I would choose for this practicum is a roller coaster - maybe a roller coaster in the dark! This has been truly emotionally, mentally, cognitively, and physically [challenging] for the past four weeks, and though the lows have been quite low, the high aspects definitely make up for it. I think that this contributes to the greatest lesson that I've learned - that teaching is impossible without relationship. This has been essential for me. I think it speaks to my personality as well, as relationship is essential to how I function in most aspects of life, and it of course follows that this would be the same in teaching, where relationship is essential, no matter what. (Student discussion post)

This metacognitive post exemplifies application in thinking, self-awareness and critical thinking on the part of the student. The student's skills and approach to learning are evident in this writing and allow the instructor to more clearly understand their learning needs.

Data analyses from the student survey in both iterations (n = 17; n = 4) also suggest deep learning occurred in the online discussions. For example, one student noted that the discussion forum "allowed me to think critically about my experiences," and another student declared "[n]ew ideas were shared by my classmates. These new ideas made me reflect on my own learning." Building on the strength of the collaborative experiences and the applications made from the online discussion forum, another student indicated in the survey response that,

It was a great tool for peer collaboration, and I liked being able to post something I had experienced or was thinking about and get other opinions on the situation. I found this to be a really useful tool in dealing with challenges in the field. (Student survey response)

These findings suggest the online discussion area provided opportunities for students to engage in metacognitive analyses that lead to deep learning.

Students and instructors had the opportunity through the affordances of the technology to use multiple forms of representation such as text, audio, and video. The analyses of the online discussion forums revealed only text was used, which also included hyperlinks. Conversely, instructors used multimedia for some aspects of the online environment such as news announcements, weekly reminders, and content resources. This leads to the question, does familiarity of text-based discussions take priority over exploration of using integrated audio and video in discussion forums?

Discussion prompts. From the content analysis of the online discussion posts, the bulk of the posts reflected the interactive dimension (iteration one 62.7% and iteration two 52.7%), seconded by the cognitive dimension (iteration one 35.11% and iteration two 29.09%). When looking at the nature of the discussion prompts, these instructor-led questions tended to ask students about the nature of their experiences in their practicum. Here are three examples that reflect the general nature of the questions students were to discuss: 1) "What are the various forms of assessment used by your partner teacher?" 2) "How is differentiation attended to in your lesson plan?" and 3) "Describe one way you might utilize formative assessment in your subject area." From the discussions that we were given permission to analyze, we were unable to identify questions that encouraged and/or required students to engage in meta-cognitive thinking. It was difficult to identify questions that required students to engage in higher-order cognitive and/or meta-cognitive.

Design Factors that Influence the Nature of Student Approaches to Learning

Findings from the student surveys for both iterations (n=21) and instructor interviews (n=8) first iteration and (n=3) second iteration, identified how the D2L course shell impacted participation, interaction, collaboration, and reflection of users. From these responses the following three themes were identified: 1) Student Engagement, 2) Group Structures, and 3) Organization. These design themes highlight aspects of course shell design that support student approaches to learning.

First, student engagement suggests that users are able to choose how they interact and the extent of this engagement in using the discussion forums. When designing the online discussion, attention was given to the layout, and the inclusion of audio, video, and graphics. From the survey data, it was evident the students appreciated how the discussion forums were structured and framed. One student appreciated topical organization of the discussion forums: "I like how the discussion board was divided into topics which allowed me to focus what I'd like to talk about in regards to my practicum." Another way of engaging students was how instructors helped foster the personalization of the online experience. One student commented on how the instructor provided more personal spaces for discussion by saying: "I was glad to see how my instructor made it her own by creating chats for us to share resources and common questions."

Together, these two student examples highlight the importance of interaction and collaboration enabled through the discussion forum design.

Second, group structures in online discussions are relevant to the success of student participation and collaboration. For example, a student stated in the survey, "[The discussion area] is for collaboration. It is easy to see what [others] are doing, sharing ideas, resources they use. It is easy to share information." Another student confirmed the importance of group learning when they wrote,

It has given me the opportunity to record my thoughts, while hearing feedback from peers and instructors. I was also able to focus my thoughts on pertinent areas of education through reviewing and responding to the weekly topics… The sense of freedom of expression in the online community is nice, as is the ability to think through responses and anecdotes prior to comment.

Together, such statements support a need for focused online discussion areas that encourage peer collaboration; a necessary part of group learning.

Third, the organization and the organizational framework of forum discussions were found to be a related theme. Specifically, the research team developed a clear discussion framework for students and instructors to participate in the online environment. Organizational features that have an impact on student interaction, such as number of discussion threads, weekly content themes within the discussion cycle, and guiding questions, were design features implemented to support students in expressing their understanding. Students found the guided questions with discussion themes as being helpful for encouraging their reflections on the course content. Evidencing this, a student stated in the survey, "I like how the areas for us to post our discussions was divided into weekly sections. It made it easier to focus on a specific topic rather than have conversations straying everywhere." Organization of the forum discussion area also allowed for instructor flexibility and personalized pedagogy.

Further, one instructor noted in her interview the inclusion of multiple forms of communication allowed for bettering student learning. Specifically, her use of text and audio helped elicit comprehension. She stated,

For the students who are auditory learners, it's very important to have that option for them, because there were students that I sent the written statements out to in an email to them. Everything that I had in the email was perfectly clear, but you could tell that they were an auditory learner because they asked the same questions of what was actually in the writing. You just knew that they needed to hear it in a different way or explain it in different way. [Online discussions] provide the option of having auditory.

This use of multimedia in the instructor's pedagogical approach to the online discussion area supported a variety of student learning needs and furthered student participation.

Discussion of the Findings

From the findings, the study demonstrates a variety of approaches to learning, including deep learning that can be found in online discussions. As outlined in the above section, content analyses of online discussion posts, we found a range of approaches to student learning taking place. These approaches included the participative, social, interactive, cognitive and metacognitive dimensions of Henri's (1992) Five Dimensions. The most common dimension found in the discussion posts was the interactive dimension, followed by the cognitive dimension: application of previous knowledge to new contexts, deepening of understanding, learning, and reconceptualization of teaching practices

The findings demonstrate that online discussions can promote a variety of approaches to learning. These findings are connected with Lublin's (2003) examination of teaching and learning strategies that encourage deep learning. In the study, this is addressed through the design of the online discussion area through the themes of student engagement, group structures, and organization. These themes can be identified as substantive design considerations to promote student interaction, reflection and collaboration to enhance such elements as metacognitive and cognitive learning outcomes.

Francescucci and Foster (2013) underscored the value of collaborative learning spaces in the online environment. From this perspective, their study asserts a link between learning and collaboration with peers. In our study, students recognized collaboration as a valuable process of their online coursework. Students' experienced theory to practice feedback loop. That is, learning that relates to students' practicum experiences was seen as motivational and therefore valued as supporting their future professional role. Students demonstrated a variety of approaches to learning in the discussion area. Those who did exemplify the cognitive and metacognitive discussion posts suggested strong motivation for learning. This aligns with Balter et al., (2013) who suggested that learning tasks are taken up by students when there are connections to student motivations.

Learning designs that allow for meaningful interactions and communications are important components when developing approaches to deep learning (Redmon & Lock, 2008; Tu & Corry, 2003). Careful planning and design become paramount decisions made by the instructors as they seek to engage students in the online discussion area (Garrison, 2011). Further, the study identifies that the use of DBR can aid educators in the development of a variety of learning approaches through informed conversation with its stakeholders. The opportunities for deep learning approaches to be realized were informed by both the research itself and the iterative process and design practice.

Implications for Practice

There are there two practical implications for the promotion of learning through online discussion forums within field experience courses. The first implication addresses design aspects. When designing the discussion area for online field experience course shells, varying degrees of meaningful cognitive exchanges made by the students were identified. This suggests that carefully constructed instructor questions and responses to students' discussion posts have the potential to further extend and expand student understanding, so that approaches to deep learning are enhanced. Second, instructors need to scaffold and probe during their facilitation to have students engage in analysis, synthesis and reflection as part of the learning environment in the online environment. Promoting deep learning approaches through critical thinking, connections between classroom experiences and theory, the sharing of examples, and problem solving in online discussions can help students in creating new understanding.

Limitations and Future Study

The study focused on creating an online environment that allowed for flexibility, accessibility, and ease of interaction between students in ways that supported a variety of learning approaches, including deep learning. The study is limited by the access to and the availability of participants involved. Each year a different group of students and instructors were in the course and as a result participants varied. There was no way to measure the effects of the design changes from one year to the next and therefore causality cannot be determined.

The final limitation involved the roles and responsibilities of the DBR team, who was also the design team, with regard to the development of course material. The team was tasked with transforming the pre-determined curriculum into the online environment. This limited the team in focusing on design elements (e.g., clarity and organizational structure of the online area) and they did not play an integral role in the implementation of the courses.

The impact of the cognitive presence of students' discussion posts was not measured over the course of the two-year design iteration. A future study on a comparable population of students over the series of course levels and consistency of instructors would be beneficial in understanding the connections to student approaches to deep learning. Further, future analysis on the level of cognitive skill involved in the discussion posts could be explored by utilizing the second model developed by Henri (1992). This second model, Analytic Model: Processing Information Model, built on the work of Marton, Hounsell, and Entwistle (1984), distinguished between explicit surface and in-depth processing. It allows instructors to identify the learning approach taken by the student within the cognitive dimension (Henri, 1992).

Due to the specific focus of the study on the design and creation of the online course shell, questions remain regarding the impact of the curriculum and associated discussion questions on students' deep learning. That is, did the crafting of the discussion question hold influence on the level of cognitive and metacognitive responses from students? Furthermore, did the design of the online environment with its ease of access, opportunities for engagement and demonstration of learning impact student understanding of practicum themes and change their teaching practices? Together, future studies that focus specifically on student learning outcomes are warranted.

Conclusion

With the growth of online and blended learning approaches, greater attention needs to be given to the design and facilitation of approaches that promotes a variety of learning approaches for student engagement. With a focused outcome for students to take up their learning through a variety of nuanced approaches, the inclusion of student engagement, use of groups in the discussion areas, and organizational structures are design aspects pertinent to student learning. However, care needs to be given in the crafting of the questions, how discussions are developed through scaffolding, and asking probing questions that push from surface to deep learning. Building upon this design, development and facilitation work is the need to gather data to inform the refinement of the online discussion forum. DBR provides such an evidence-informed process. As such, consideration needs to be given to purposeful planning that engages learners in meaningful interaction opportunities to achieve deep learning.

References

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education Research? Educational Researcher, 41(1), 16-25. Retrieved from http://edr.sagepub.com/content/41/1/7.full.pdf+html

Balter, O., Cleveland-Innes, M., Pettersson, K., Scheja, M., & Svedin, M. (2013). Student approaches to learning in relation to online course completion. Canadian Journal of Higher Education, 43(3), 1-8. Retrieved from http://journals.sfu.ca/cjhe/index.php/cjhe/article/view/184673

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1-14. http://dx.doi.org/10.1207/s15327809jls1301_1

Biggs, J. (1999). Teaching for quality learning at university. Buckingham, United Kingdom: Society for Research into Higher Education and Open University Press.

Burnett, P., Pillay, H., & Dart, B. (2003). The influences of conceptions of learning and learner self-concept on high school students' approaches to learning. School Psychology International, 24(1), 54-66. http://dx.doi.org/10.1177/0143034303024001621

Chatti, M., Jarke, M., & Specht, M. (2010). The 3P learning model. Journal of Educational Technology & Society, 13(4), 74-85. Retrieved from http://www.ifets.info/issues.php?id=49

Darabi, A., Arrastia, M. C., Nelson, D. W., Cornille, T., & Liang, X. (2011). Cognitive presence in asynchronous online learning: A comparison of four discussion strategies. Journal of Computer Assisted Learning, 27(1), 216-227. http://dx.doi.org/10.1111/j.1365-2729.2010.00392.x

Darling-Hammond, L. (2006). Powerful teacher education: Lessons from exemplary programs. San Francisco, CA: Jossey-Bass.

Denzin, N. K., & Lincoln, Y. S. (Eds.). (1998). The landscape of qualitative research: Theories and Issues. Thousand Oaks, CA: Sage Publications.

Diaz, S., & Diniz, J. (2014). Towards an enhanced Learning Management System for blended learning in Higher Education incorporating distinct learners' profiles. Journal of Educational Technology & Society, 17(1), 307-319. Retrieved from http://www.ifets.info/issues.php?id=49

Diseth, A. (2003). Personality and approaches to learning as predictors of academic achievement. European Journal of Personality, 17(2), 143-155. http://dx.doi.org/10.1002/per.469

Donnison, S., & Penn-Edwards, S. (2012). Focusing on first year assessment: Surface or deep approaches to learning? The International Journal of the First Year in Higher Education, 2(1), 9-20. http://dx.doi.org/10.5204/intjfyhe.v3i2.127

Entwhistle, N., & Entwhistle, A. (2005). Revision and the experience of understanding. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The experience of learning: Implications for teaching and studying in higher education (3rd ed., pp. 145-155). Edinburgh, Scotland: University of Edinburgh, Centre for Teaching, Learning and Assessment. Retrieved from http://www.docs.hss.ed.ac.uk/iad/Learning_teaching/Academic_teaching/Resources/Experience_of_learning/EoLChapter9.pdf

Entwistle, N. J., & Waterston, S. (1988). Approaches to studying and levels of processing in university students. British Journal of Educational Psychology, 58(3), 258-265. http://dx.doi.org/10.1111/j.2044-8279.1988.tb00901.x

Eastman, D. V. (1995). Alone but together: Adult distance study through computer conferencing. Cresskill, NJ: Hampton Press.

Francescucci, A., & Foster, M. (2013). The VIRI (virtual, interactive, real-time, instructor-led) classroom: The impact of blended synchronous online courses on student performance, engagement, and satisfaction. Canadian Journal of Higher Education, 43(3), 78-91.

Garrison, D. R. (2011). E-learning in the 21st century: A framework for research and practice (2nd ed.). New York, NY: Routledge.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.

Harasim, L. (2000). Shift happens: online education as a new paradigm in learning. The Internet and Higher Education, 3(1-2), 41-61. http://dx.doi.org/10.1016/S1096-7516(00)00032-4

Harasim, L. (2012). Learning theory and online technologies. New York, NY: Routledge.

Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing: The Najaden papers (pp. 117-136). New York, NY: Springer.

Kumar, K., & Wideman, M. (2014). Accessible by design: Applying UDL principles in a first year undergraduate course. Canadian Journal of Higher Education, 44(1), 125-147. Retrieved from http://journals.sfu.ca/cjhe/index.php/cjhe/article/view/183704

Lublin, J. (2003). Deep, surface and strategic approaches to learning. Centre for Teaching and Learning, University College Dublin.

Marton, F., Hounsell, D. J., & Entwistle, N. J. (Eds.) (1984). The experience of learning. Edinburgh, Scotland: Scottish Academic Press.

Marton, F., Hounsell, D., & Entwistle, N. (2005). The experience of learning: Implications for teaching and studying in higher education (3rd ed.). Edinburgh: University of Edinburgh, Centre for Teaching, Learning and Assessment.

Marton, F., & Säljö, R. (1976). On qualitative differences in learning: I-outcomes and processes. British Journal of Educational Psychology, 46(1), 4-11. http://dx.doi.org/10.1111/j.2044-8279.1976.tb02980.x

McKenney, S. E., & Reeves, T. C. (2012). Conducting educational design research. Milton Park, Abingdon, Oxon: Routledge.

Means, B., Yoyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and a review of online learning studies. US Department of Education. Office of Planning, Evaluation, and Policy Development. Policy and Program Studies Service.

Meyer, A., Rose, D.H., & Gordon, D. (2014) Universal design for learning: Theory and practice. Wakefield, MA: CAST

Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Thousand Oaks, CA: Sage Publications.

Redmond, P. & Lock, J. V. (2008). Investigating deep and surface learning in online collaboration. Proceedings of the Society for Information Technology & Teacher Education International Conference Annual, USA (pp. 4295-4301). Chesapeake, VA: Association for the Advancement of Computing in Education.

Moore, M., & Kearsley, G. (2005). Distance education: A systems view. Toronto, ON: Nelson.

Reeves, T. C. (2000). Socially responsible educational technology research. Educational Technology, 40(6), 19-28.

Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). Los Angeles, CA: Sage Publications.

Simpson, R. (2003). A search for the complete education: Balancing the needs for survival and fulfillment. Innovation Higher Education, 28(2), 91-105. http://dx.doi.org/10.1023/B:IHIE.0000006288.13507.4b

Tu, C. H., & Corry, M. (2003). Designs, management tactics, and strategies in asynchronous learning discussions. The Quarterly Review of Distance Education, 4(3), 303-315.

Zhang, T., Gao, T., Ring, G., & Zhang, W. (2007). Using online discussion forums to assist a traditional English class. International Journal on E-Learning, 6(4), 623-643.

Using Design-Based Research to Develop Meaningful Online Discussions in Undergraduate Field Experience Courses by Carol Johnson, Laurie Hill, Jennifer Lock, Noha Altowairiki, Chris Ostrowski, Luciano da Rosa dos Santos, and Yang Liu is licensed under a Creative Commons Attribution 4.0 International License.