|
Yu-Hui Ching
Boise State University, USA
This study explored the impact of role-playing on the quality of peer feedback and learners’ perception of this strategy in a case-based learning activity with VoiceThread in an online course. The findings revealed potential positive impact of role-playing on learners’ generation of constructive feedback as role-playing was associated with higher frequency of problem identification in the peer comments. Sixty percent of learners perceived the role-play strategy useful in assisting them to compose and provide meaningful feedback. Multiple motivations drove learners in making decisions on role choice when responding to their peers, mostly for peer benefits. Finally, 90% of learners reported the peer feedback useful or somewhat useful. Based on the findings of this study, we discussed educational and instructional design implications and future directions to further the line of research using role-play strategy to enhance peer feedback activity.
Keywords: Role-play; peer feedback; case-based learning; online learning; VoiceThread; higher education
Case-based learning uses a “problem-based approach” to engage learners in solving “real or hypothetical problem situations, reflecting the kinds of experiences naturally encountered in the discipline under study” (Ertmer & Russell, 1995, p. 24). These complex problems usually involve rich but not complete information, dynamic factors, limited resources, and multiple stakeholders with competing perspectives (Jonassen, 1997, 2000). In this case-based context, learners are charged to analyze the given information, identify strengths and weaknesses of provided approach, generate a solution or reflect on other possible solutions as alternatives (Pozzi, 2010). Case-based discussion enables learners to bridge the gap between theory and practice through application and transfer of knowledge and skills when they solve realistic problems (Ertmer & Russell, 1995; Winter & McGhie-Richmond, 2005). In addition, the rich information and narratives in the case resemble vivid human experiences that may be easier to assimilate and reflect upon as opposed to abstract information (Pozzi, 2010). However, case-based learning is likely to impose heavy cognitive loads on novice learners due to the complexity of real-world problems and the demanding task of applying newly learned knowledge in practice. When overwhelmed by the rich information in the cases, learners may find it even more challenging to apply newly learned knowledge and skills during the problem-solving processes. In this situation, support from an online learning community may benefit learners when they tackle complicated issues embedded in the case scenarios. Support, in the form of peer feedback, can help learners validate ideas, identify problems, and expand the spectrum of thinking.
Peer feedback activities engage learners in cognitive interactions of sharing relevant experiences, exchanging ideas, and negotiating meanings. Peer feedback refers to “a communication process through which learners enter into dialogues related to performance and standards” (Liu & Carless, 2006, p. 280). Peer feedback can be part of peer assessment in which “students engage in reflective criticism of the products of other students and provide them with feedback, using previously defined criteria” (Van der Pol, Van den Berg, Admiraal, & Simmons, 2008, p. 1805). When providing peer feedback, learners may “consider the amount, level, value, worth, quality or success of the products or outcomes of learning of peers of similar status” (Gielen et al., 2011, p. 137). While peer feedback can be employed for formative or summative assessment, research suggests that it benefits learners the most when it is used for formative purpose with no grades on the peers’ work involved (Nicol, 2008). Formative feedback, consisting of comments on strengths, weaknesses, and/or suggestions (Falchikov, 1996), is presented in a nonevaluative, supportive, and timely manner during the learning process for the purpose of improving learning (Shute, 2008). After reviewing 26 peer assessment studies, Van Zundert et al. (2010) found that enabling students to revise their work on the basis of peer feedback improved domain-specific skills.
The social constructivist view of learning underpins the use of peer feedback. This view emphasizes learning as a social activity and asserts that learners’ interactions with people in the environment lead to their cognitive growth (Gunawardena, et al., 1997; Schunk, 2008). When providing and receiving peer feedback, learners have the opportunity to articulate and clarify their own thinking, to view peers’ ideas, and to negotiate and make sense of different perspectives. Through this interactive process, learners collaboratively explore the given issues and develop more comprehensive knowledge on the issues, and achieve deeper understanding toward the subject.
Benefits have been found for both receiving and providing peer feedback. When receiving feedback, learners invite peers to contribute experiences and perspectives to enrich their own learning process (Ertmer, et al., 2007). When providing feedback, learners actively engage in articulating their evolving understanding of the subject matter (Liu & Carless, 2006). They also apply the learned knowledge and skills when assessing others’ work. This process involves learners in thinking about quality, standards, and criteria that they may use to evaluate others’ work, which helps them become critical thinkers and reflective learners (Liu & Carless, 2006). In their study of peer assessment in an undergraduate technology application course, Li, Liu, and Steckelberg (2010) found a positive and significant relationship between the quality of peer feedback that students provided for others and the quality of the students’ own final products, controlling for the quality of the initial projects. They concluded that active engagement in reviewing peers’ projects might facilitate learning performance. Examining how undergraduate peer reviewers learned from giving comments, Cho and Cho (2011) found that students improved their writing more by giving comments than by receiving comments. Giving comments involves evaluative and reflective activities in which students identified good writing, problematic areas in the writing, and possible ways to solve the problem.
Despite the cognitive benefits of peer feedback activity, research identified both cognitive challenges and affective barriers of this activity on learners. Providing peer feedback is a cognitively demanding task for learners because they have to use their knowledge and skills to review, clarify, and evaluate other peoples’ work (van Gennip et al., 2010). Especially, learners may not possess the domain knowledge or skills to provide useful and meaningful feedback (Palloff & Pratt, 1999) as learners are often novices in the field. As such, they may provide feedback at a superficial level that does not lead to critical thinking of their own (Li et al., 2010), nor does it contribute to peers’ learning. Affectively, students may have anxiety about giving feedback (Ertmer, et al., 2007) or little confidence in assessing their peers (Venables & Summit, 2003) if they are not used to this activity, as they do not want to appear to be criticizing peers’ work. For example, Ellison and Wu (2008) found that college students were uncomfortable providing peer feedback on blogs. In addition, peer feedback may not be perceived as valid by the receivers as peer reviewers are usually not regarded as a “knowledge authority” by feedback receivers (Gielen, et al., 2010), and, thus, learners refuse to take the feedback seriously. In addition, learners’ peer feedback performance also varies depending on their characteristics, such as thinking style and level of academic achievement (Van Zundert et al., 2010). Lu and Law (2011) found that learners’ ability to identify problems in peer work and give suggestions was a significant predictor of the feedback providers’ learning performance. Davies (2006) found that “better” students were more willing to criticize their peers than weaker students.
Recently, researchers started to identify instructional intervention to train or scaffold learners in providing better peer feedback. For example, Sluijsmans et al. (2004) found that training that engaged learners in defining assessment criteria positively influenced learners’ peer assessment skills. Ching and Hsu (2013a) explored graduate students’ peer feedback activities in an online course and examined different types of feedback provided by peers. They found that when guiding questions were used to scaffold learners’ peer feedback activity, learners seemed to generate more feedback consisting of “Suggestion” and “Problem Identification.” In the current study, a different instructional approach was taken to scaffold learners to provide constructive peer feedback. This study explored whether a role-play strategy helps improve the generation of constructive peer feedback in a case-based problem solving learning activity in an online course. Specifically, in this study, the constructive feedback is defined as feedback that involves identifying problematic areas, asking questions to probe deeper thinking, and providing suggestions to address the problematic or weak areas.
Role-playing is a teaching method that has been used widely for “experiential learning” (Russell & Shepherd, 2010) and that “provides an imaginary context in which issues and behaviors may be explored by participants who take on a specific role or character” (Bell, 2001, p. 256). It is considered to be particularly effective for learning about complex social/human systems (Russell & Shepherd, 2010), and has been implemented to develop group decision-making (Bos & Shami, 2006; Pata et al., 2005), motivate learners (Wishart et al., 2007), improve communication skills (Chien et al., 2003), and develop problem-solving abilities (Hou, 2012). When participating in role-play, learners explore a complex scenario that requires resolution through discussion, debate, and negotiation among roles with different points of view (Russell & Shepherd, 2010). It gives opportunities for learners to apply knowledge in contexts and receive the consequences of actions in safe environments. Research shows that role-playing benefits learning in several ways (Dracup, 2008). First, it produces deep-level learning outcomes that resonate for a long time (Bolton & Heathcote, 1999). Second, it engages both learners and instructors. That is, learners tend to enjoy the experiential learning experiences (Raphael & O’Mara, 2002) and the instructors tend to find the experiences rewarding (Bolton & Heathcote, 1999). Third, role-play can help learners develop problem-solving abilities by assuming different roles situated in complex problem scenarios relevant to the professional domain (Hou, 2012).
Role-playing can be an innovative way to elicit constructive feedback from peers. In a case-based learning environment, learners can play different roles of stakeholders to provide meaningful feedback to their peers. Role-playing allows feedback providers to use a specific lens to anchor their analysis, interpret, and evaluate peers’ work, which may result in more directed and constructive feedback. Role-based peer feedback can be more in-depth and critical, which helps result in more comprehensive problem solutions.
The process of role-based peer feedback activity can be enhanced with a Web 2.0 tool that enables easy communication and smooth collaboration. In this study, we used VoiceThread as the virtual learning environment. VoiceThread is a tool that meets a variety of criteria of a useful online role-play environment (Russell & Shepherd, 2010), such as authenticity, asynchronous communication, student accessibility, and low set-up costs. Like other Web 2.0 tools, VoiceThread featuring audio or video presentations can be used as a cognitive tool that allows learners to organize and present their knowledge for deep learning (Hsu, Ching, & Grabowski, 2009). Its text, audio and video comment-sharing function also makes it possible for learners to actively interact with peers in an authentic and meaningful environment (Ching & Hsu, 2011). With its affordance to create multimedia artifacts, learners can share their ideas in multiple formats (e.g., texts, images, audio, and video) from which they can further build their understanding of the learning materials (Hsu, Ching, & Grabowski, 2014). Role-playing and commenting with VoiceThread also augments communication as the process emulates face-to-face interaction, providing learners the opportunity to see and hear each other. Research has found that learners felt more connected when they use audio-based and video-based discussion on VoiceThread (Ching & Hsu, 2013b) compared to text-based discussion in discussion forums.
This study explored the impact of role-playing on the quality of peer feedback and learners’ perceptions of the role-play strategy in a case-based learning activity with VoiceThread in an online course. Specifically, the study answers the following research questions:
Participants were graduate students in an online master’s program in a northwestern state university in the United States. Twenty of the 39 students in an online instructional design course participated in this study on a voluntary basis. Forty-five percent of the participants were males and 55% were females. Fifty percent of the participants were younger than 40 years old. This online course was mainly hosted on the Moodle learning management system (LMS) where the course instructor posted course materials, and made regular announcements regarding course requirements and reminders. VoiceThread was used as an additional learning environment where learners worked on one of the major activities, the one concerned in this study.
Students participated in a case-based role-playing peer feedback activity as part of the required course work. This activity lasted for four weeks and involved learners in analyzing an instructional design (ID) case individually, creating a presentation on the analysis on VoiceThread, participating in role-playing peer feedback to three peers, revising one’s own original analysis based on peer feedback and submitting final analysis for grading. Students chose a relevant ID case out of the three cases that represent scenarios in different professional contexts. When providing feedback to peers, students were asked to use the role-play strategy (role play) by assuming a stakeholder’s role of their own choice in the case scenario and providing constructive feedback from the perspective of the selected stakeholder. For example, in a case situated in an elementary school setting, many learners chose the associate principal’s role to construct their feedback from. Specifically, two prompts were provided to help learners construct their feedback: (1) How does the analysis address your existing (the stakeholder’s) concerns and/or needs? (2) What are some concerns you may have toward the analysis (and the solutions if there are any)? Figure 1 shows a screenshot of the VoiceThread presentation of the case analysis created by a participant in this study. This presenter’s avatar is on the upper left and two peer commenters provided feedback on the case analysis.
We collected data from two sources to answer the aforementioned research questions. Peer feedback entries were used to answer research question 1 and a survey of learner perceptions to answer research question 2, 3, and 4. First, peer feedback entries on VoiceThread were collected and analyzed using content analysis. Content analysis is a research method that builds on procedures to make valid inferences from text (Anderson, Rourke, Garrison, & Archer, 2001). In this study, a person can give directed comments at individual presentation slides on VoiceThread. We treated a comment given by a person on an individual slide as one complete message and we used one complete message as the unit of analysis like in other studies that applied content analysis (e.g., Gunawardena et al., 1997). For example, when one learner commented at five different slides throughout a presentation, five complete messages were generated for analysis. Using this method, 123 entries of peer feedback were identified on 20 VoiceThread presentations. On average, each learner generated 6.47 pieces of feedback on VoiceThread with a range from 2 to 27 entries. One learner did not provide feedback on VoiceThread but on a Moodle discussion forum; as such, his/her feedback entries were not included in this analysis.
All the feedback entries retrieved from VoiceThread were coded for cognitive and affective categories using the coding scheme presented in Table 1. This coding scheme was adapted based on the scheme in Lu and Law (2011) and Ching and Hsu (2013a). Each peer feedback entry can be coded into multiple categories.
Second, a survey with open-ended questions was administered after the end of the peer feedback activity to solicit participants’ experiences and their decision-making process during their participation in the activity. Specifically, we asked questions on learners’ perceptions of the role-play strategy when providing feedback, the method learners used to select the role to take, whether they changed to different roles when providing feedback to different peers, and the usefulness of the received peer feedback. The responses to the survey questions were then examined using the constant-comparative approach espoused by Lincoln and Guba (1985). We initially examined the survey responses to group similar comments into themes, and evaluated the fit between each student response and the theme. We then gave each theme a suitable label, and selected and reported representative statements for each theme.
When examining peer feedback entries, it was found that not all learners adopted the role-play strategy when they provided feedback. As a result, we distinguished feedback entries generated with the strategy from those without the strategy. Table 2 presents the descriptive data showing the detailed breakdown of the percentage of different types of comments generated with or without role play. In total, 76 entries of messages were generated with role play and 47 entries of messages were generated without role play. Problem Identification is the least frequent category (11%) in the comments generated without role play, followed by Question, Suggestion, and Support. When role play was adopted, the frequency of all the coded categories was increasing. Particularly, Problem Identification, the least present category without role play, doubled in frequency. To test if the increases in these categories were associated with role play, chi-squared tests of independence were performed on these categorical data.
Overall, 28% of comments made by learners engaged in role play were classified as Problem Identification, compared to 11% of those made by learners who did not take a particular role. The results of a chi-square test (X2 (1, N = 123) = 5.03, p < .05) supported the conclusion that the use of the role-play strategy was associated with a higher proportion of comments that contains Problem Identification compared to those generated without the use of the strategy. However, the results of chi-square tests did not support the conclusion that the use of the role-play strategy was associated with increase of comments in the following categories: Question (X2(1, N = 123) = .07, p > .05), Suggestion (X2(1, N = 123) = .24, p > .05), nor Support (X2(1, N = 123) = 1.38, p > .05).
The results of chi-square tests of independence showed significant association between role play and Problem Identification. Problem Identification has been the least frequent category reported in the past literature of peer feedback concerning whether learners were able to provide constructive comments (Ching & Hsu, 2013a), as well as in the current study when learners did not adopt intervention to generate comments. It can be argued that Problem Identification is a type of comment that learners often do not feel comfortable producing compared to other types of comments. Studies have shown that learners did not feel comfortable identifying problems in peers’ work because they did not want to appear to be criticizing peers (e.g., Ertmer, et al., 2007), to risk the interpersonal relationships, or to use their new knowledge to evaluate. The association of role play with higher frequency of Problem Identification in the peer comments seems to suggest that through role play, learners were able to overcome the affective barriers of critiquing peers and point out the weakness for improvement. Being able to detect problematic areas in peers’ work could lead to better learning in the domain knowledge of the feedback providers (Lu & Law, 2011). However, this finding needs to be further validated by studies with research design that can establish the causal relationship between the intervention, role play, and the behavior of problem identification in peer feedback.
When learners were asked about their experiences of using the role-play strategy to provide feedback to peers, it was found that the role-play strategy alleviated cognitive challenges of peer feedback, made the activity more engaging, and relieved the affective barriers of providing peer feedback. However, some learners also reported that the strategy limited the feedback they could provide.
Sixty percent of learners perceived the role-play strategy useful in enabling them to compose and provide meaningful feedback. Commenting based on a role helped learners delve deeper into the issues in the case study, and create more directed and focused comments. For example, two learners commented the following.
I thought that by taking on a role of a specific person or people, it helped focus my feedback. I was able to look at and respond to other people’s analysis through a specific perspective.
I liked focusing on a role as it made me look deeper into the questions and forced me to see it through their glasses, so to speak.
As the case scenarios used in this study presented complex instructional design issues that involved multiple stakeholders, taking a role to give peer feedback alleviated cognitive challenges because it made it easier for learners to only take into consideration one stakeholder’s perspective to process the analysis, instead of trying to be comprehensive by taking into account multiple perspectives.
Learners also thought the role-play strategy made the peer feedback activity more engaging and authentic as they were put into stakeholders’ shoes to make sense of the analysis and use the corresponding perspective to address complex issues. For example, one learner commented that
Picking up a specific role and giving feedback was actually fun. It gives the students a chance to see what it will be like should they want to become an instructional designer. It is like working on a project at work and getting feedback from your boss.
A particularly interesting and critical perspective revealed by two learners indicated that the role-play strategy made them feel more comfortable critiquing peers’ work during the process, as learners did not feel they were personally attacking others’ work.
It felt a bit freeing - I wasn’t “personally attacking” something in their project, and it gave me a better foundation/better criteria on which to give positive feedback (something beyond “I liked it!”).
Looking at it from a stakeholder’s viewpoint, it only helps in the analysis to rectify flaws. It helps eliminate personal opinions and focuses on constructive feedback which is relevant to the case.
On the other hand, 20% of students found the role-play strategy limiting, instead of enabling. In all these cases, learners reported that they could have given more comments to peers if they were not limited to the specific role they were taking. Two learners commented that
It (role-playing) was limiting in a way because I then could not speak to things outside that character’s experience or expected knowledge base. Maybe I took the task too seriously, but I think I might have been able to make more meaningful comments in a place or two had I had more freedom.
I thought it would have been more helpful to comment from a global perspective from multiple viewpoints. It would give the presenter feedback that was more well-rounded.
The findings to this research question show that learners mostly had positive perceptions of the role-play strategy. Particularly, two learners commented that using the role-play strategy freed them from the uncomfortable feeling of critiquing peers’ work. This finding seems to provide an explanation for the content analysis results that role play is associated with the increased frequency of problem identification in the peer feedback. Taking a role empowers learners to pinpoint problematic or weak areas relevant to the perspective without learners being afraid of hurting a peer’s feelings or risking the interpersonal relationships with peers.
However, the findings also revealed that there is a need to establish psychological safety among the learners if the goal is to offer a task-oriented environment for peer feedback. Psychological safety refers to “a shared belief that it is safe to take interpersonal risks in a group of people” (Van Gennip et al., 2010, p. 282), and “a sense of confidence that the team will not embarrass, reject, or punish someone for speaking up” (Edmondson, 1999, p. 354). When learners appreciate differences in opinions as opportunities to frame a problem (Edmondson, 1999) instead of disagreement, they are likely to improve their performance of providing constructive comments.
Based on the design of the learning activity, learners were given the freedom to choose the roles they preferred to play when providing peer feedback. When asked about their decision-making regarding role choices, learners identified multiple reasons for picking the roles, including peer-benefit and self-benefit reasons.
About 30% of learners tried to help their peers by picking a role that enabled making valuable comments on peers’ analysis. Participants reported that they would listen to the peer presentation and choose a role that could contribute the most. For example, two learners commented that
Based on who I thought would be the most “antagonistic” to their approach - the “devil’s advocate.”
After I listened to the other person’s analysis, depending on what they said, I chose who would be most concerned with the information that they provided.
Some learners picked roles based on the relevance of the roles to the case study. For example, one learner picked the role that seemed to be the controlling stakeholder whose opinion matters the most, and another learner picked a role that could provide a different perspective. There are also learners who picked the roles most people could relate to, the roles interesting to themselves as learners, or the roles relevant to their past experiences. Lastly, 20% of the learners made a decision randomly.
Learners were further asked about whether they took the same role when they provided feedback to three or more peers. Forty percent of students took the same role when responding to different peers due to multiple reasons such as time saving, being able to compare several analyses from the same perspective, and being able to provide more constructive feedback by delving into one specific role in depth. Fifty-five percent of students took different roles when responding to multiple students. Some of them felt playing different roles was more interesting; others wanted to expand their own learning by seeing a case from different perspectives; still others changed roles because they thought they could provide more useful feedback that way. In summary, various motivations drove learners in making decisions on role choice when responding to their peers. The majority of learners reported altruistic reasons as they intended to provide the most helpful feedback for their peers. Only very few learners were more interested in the opportunities for self-learning and growth, as they picked various roles to explore and broaden their own understanding of the issues presented in the cases.
The decisions for learners to play consistent or different roles have implications for their own learning. By taking the same role, learners are able to use one perspective to examine multiple peer analyses and alternative solutions, leading to a deeper and more focused understanding toward the underlying issues that are important to the stakeholder. Through altering roles, learners gain the opportunity to explore analyses and solutions from different stakeholders’ perspectives. A design challenge, thus, emerges regarding how much freedom we give to learners in terms of role choice. A study on online role-playing found that familiarity with the role being played helped role engagement and appropriate voice when performing the role (Cornelius, et al., 2011). In this study, the goal is to have learners provide constructive feedback to other learners. Therefore, it may be more reasonable to have learners identify a role that they can relate to the most, and develop a deep understanding of the issues critical to the chosen perspective. This way, learners can offer more meaningful and constructive feedback to their peers.
Overall, 90% of learners reported the peer feedback they received useful or somewhat useful. Sixty percent of the learners perceived that peer feedback was useful. Learners reported that feedback helped them identify holes in their case analysis and ideas that had been overlooked or that could be analyzed deeper. Useful peer feedback also gave them suggestions on how to improve their work. A sample comment reads, “The feedback was very useful. In particular, one of the roles that was chosen was at direct odds with my analysis, so it was very eye-opening.”
Thirty percent of learners found the feedback somewhat useful. In these instances, learners wish they could receive more specific and deep feedback that could be used to improve their work. For example, a learner commented the following : “Moderately. It did not seem as in depth as the feedback I had tried to give.”
Out of all the learners, 10% reported that they did not receive feedback either because they posted their case analysis presentation late or there were technical issues that prevented making their presentation accessible to their peers.
Based on the findings of this study, the role-play strategy appears to have a great potential in facilitating learners to compose constructive feedback that identifies problems for improvement. We would recommend online educators and instructional designers to use this strategy to achieve the learning goal of improving peer feedback quality. The current findings tied in role-play with prompting questions that guided learners’ feedback construction. While current intervention increases the frequency of problem identification in the comments, in the future, prompting questions can be expended to elicit other types of comments. That is, prompts can also guide learners to ask more probing questions and provide suggestions that address identified issues based on the role they play.
The results of this study shed light on training learners to overcome affective barriers of providing constructive peer feedback. Previous research has identified that learners do not feel comfortable critiquing peers’ work (Ertmer, et al., 2007; Smith et al., 2002; Venables & Summit, 2003) and training on peer feedback or assessment has helped learners to form more positive attitudes toward this activity (Smith et al., 2002; Cheng & Warren, 1997). This study contributes to the existing research by demonstrating an additional strategy to train learners. That is, when training novices to provide peer feedback, having them play a role to evaluate peers’ work can help them be concerned less about how they appear to others in a community where interpersonal relationships are valued and maintained. As shown in this study, learners can feel more comfortable critiquing peers’ work by taking a stakeholder’s role as it “frees” learners from the feeling of attacking peers.
The current finding also shows that not everyone perceived role-playing as helpful for generating peer feedback, which may be due to individual differences. The role-play strategy may not have the same impact on learners of different cognitive styles or different abilities. Research has found that learner cognitive style has an effect on the peer feedback they provided. In a study conducted by Lin, Liu, and Yuan (2001), they defined that learners with a higher executive learning style were more willing to follow instructional rules than those who emphasized independence and creativity. They found that learners with a high executive thinking style provided peer feedback of better quality than those with a low executive thinking style, where feedback quality was defined as high when it offered suggestions for the next step of modifying and explaining the peers’ reading summary. Yu, Liu, and Chan (2005) found positive relations between levels of academic achievement and the peer assessment skill. Davis (2006) found that learners of lower performance tended to be less critical whereas learners of higher performance were more critical. As the current study did not take into account learner characteristics, future research may explore how the role-play strategy interplays with learner cognitive styles or academic achievement, so that more personalized intervention can target learners’ diverse needs.
While peer feedback remains a challenging skill for some learners even with the role-play strategy, it may be worthwhile to couple this strategy with established effective intervention to better enhance learners’ skills in providing constructive feedback. For example, Sluijsmans et al. (2004) found that training which engaged learners in defining assessment criteria positively influenced learners’ peer assessment skills. Research on role-playing has suggested that an anonymous role-play activity may help participants engage with their roles so that they reply to roles rather than to their peers, which improves the freedom to speak in-role with honest disclosure (Cornelius, Gordon, & Harris, 2011). Future research may investigate whether peer feedback quality can be further improved by combining intervention that fosters learners’ understanding of the assessment criteria and an anonymous role-playing environment.
This study explored the impact of role-playing on the quality of peer feedback and learners’ perception of this strategy in a case-based learning activity with VoiceThread in an online course. The findings revealed potential positive impact of role-playing on learners’ generation of constructive feedback as role-playing was associated with higher frequency of problem identification in the peer comments. Sixty percent of learners perceived the role-play strategy useful in assisting them to compose and provide meaningful feedback. Multiple motivations drove learners in making decisions on role choice when responding to their peers, mostly for peer benefits. Finally, ninety percent of learners reported the peer feedback useful or somewhat useful. Based on the findings of this study, we discussed educational and instructional design implications and future directions to further the line of research using role-play strategy to enhance peer feedback activity.
In this study, we used multiple data sources, including peer comments posted on VoiceThread and students’ responses to open-ended survey questions to explore how the role-play strategy impacted peer feedback, learners’ perceptions, and decision-making processes in a case-based asynchronous online discussion using VoiceThread. However, the findings of this current study should be interpreted with caution due to the limited number of participants and specific learning contexts and tasks (e.g., adult learners in an online learning environment). Future research is encouraged to replicate this study in different learning contexts with learners of different characteristics. Research studies using experimental design need to be conducted to verify the current findings and to yield research results that can be generalized to broader educational contexts.
Bell, M. (2001). Online role-play: Anonymity, engagement and risk. Educational Media International, 38(4), 251-260.
Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2). Retrieved from http://sloanconsortium.org/jaln/v5n2/assessing-teacher-presence-computer-conferencing-context
Bolton, G., & Heathcote, D. (1999). So you want to use role play? London: Trentham.
Bos, N., & Shami, N. S. (2006). Adapting a face-to-face role-playing simulation for online play. Educational Technology Research and Development, 54(5), 493-521.
Cheng,W., & Warren, M. (1997). Having second thoughts: Student perceptions before and after a peer assessment exercise. Studies in Higher Education, 22, 233-239.
Chien, L. D., Muthitacharoen, A. M., & Frolick, M. N. (2003). Investigating the use of role play training to improve the communication skills of IS professionals: Some empirical evidence. Journal of Computer Information Systems, 43(3), 67-74.
Ching, Y. -H., & Hsu, Y. -C. (2011). Design-grounded assessment: A framework and a case study of Web 2.0 practices in higher education. Australasian Journal of Educational Technology, 27(5), 781-797.
Ching, Y.-H., & Hsu, Y.-C. (2013a). Peer feedback to facilitate project-based learning in an online environment. The International Review of Research in Open and Distance Learning, 14(5), 258-276.
Ching, Y.-H., & Hsu, Y.-C. (2013b). Collaborative learning using VoiceThread in an online graduate course. Knowledge Management & E-Learning, 5(3), 298-314.
Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Sciences, 39, 629-643.
Cornelius, S., Gordon, C., & Harris, M. (2011). Role engagement and anonymity in synchronous online role play. The International Review of Research in Open and Distance Learning, 12(5), 57-73.
Davies, P. (2006). Peer assessment: Judging the quality of students’ work by comments rather than marks. Innovations in Education and Teaching International, 43, 69-82.
Dracup, M. (2008). Role play in blended learning: A case study exploring the impact of story and other elements. Australasian Journal of Educational Technology, 24(3), 294–310.
Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350-383.
Ellison, N., & Wu, Y. (2008). Blogging in the classroom: A preliminary exploration of student attitudes and impact on comprehension. Journal of Educational Multimedia and Hypermedia, 17(1), 99-122.
Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., et al. (2007). Using peer feedback to enhance the quality of student online postings: An exploratory study. Journal of Computer-Mediated Communication, 12(2), article 4. Retrieved from http://jcmc.indiana.edu/vol12/issue2/ertmer.html
Ertmer, P. A., & Russell, J. D. (1995). Using case studies to enhance instructional design education. Educational Technology, 35(4), 23–31.
Falchikov, N. (1996, July). Improving learning through critical peer feedback and reflection. Paper presented at the HERDSA Conference 1996: Different approaches: Theory and practice in Higher Education, Perth, Australia.
Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304-315. doi:10.1016/j.learninstruc.2009.08.007
Gunawardena, C. N., Lowe, X., Constance, A., & Anderson, T. (1997). Analysis of a global debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
Hou, H.-T. (2012). Analyzing the learning process of an online role-playing discussion activity. Educational Technology & Society, 15(1), 211-222.
Hsu, Y. -C., Ching, Y.-H., & Grabowski, B. (2009). Web 2.0 technologies as cognitive tools of the new media age. In L. W. H. Tan & R. Subramaniam (Eds.), Handbook of research on new media literacy at the K-12 level: Issues and challenges (pp. 353-371). Hershey, PA: IGI Global.
Hsu, Y. -C., Ching, Y.-H., & Grabowski, B. (2014). Web 2.0 applications and practices for learning through collaboration. In M. Spector, D. Merrill, J. Elen & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed.) (pp. 747-758). Springer Academics.
Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45, 65-94.
Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research & Development, 48(4), 63-85.
Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assess: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536.
Lin, S. S. J., Liu, E. Z. F., & Yuan, S. M. (2001). Web-based peer assessment: Feedback for students with various thinking-styles. Journal of Computer Assisted Learning, 17, 420-432.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage Publications.
Liu, N. -F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279-290.
Lu, J., & Law, N. (2011). Online peer assessment: Effects of cognitive and affective feedback. Instructional Science, 40(2), 257-275.
Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace. San Francisco, CA: Jossey-Bass.
Pata, K., Sarapuu, T., & Lehtinen, E. (2005). Tutor scaffolding styles of dilemma solving in network-based role-play. Learning and Instruction, 15(6), 571-587.
Pozzi, F. (2010). Using jigsaw and case study for supporting online collaborative learning. Computers & Education, 55(1), 67–75. doi:10.1016/j.compedu.2009.12.003\
Raphael, J., & O’Mara, J. (2002). A challenge, a threat and a promise: Drama as PD for teacher educators. Melbourne Studies in Education, 43(2), 77-87.
Schunk, D. H. (2008). Learning theories: An educational perspective (5th ed.). Upper Saddle River, NJ: Pearson.
Shute, V. J. (2008) Focus on formative feedback. Review of Educational Research, 78(1), 153-189.
Sluijsmans, D. M. A., Brand-Gruwel, S., Van Merrienboer, J. J. G., & Martens, R. L. (2004). Training teachers in peer-assessment skills: Effects on performance and perceptions. Innovations in Education and Teaching International, 41, 60-78.
Smith, H., Cooper, A., & Lancaster, L. (2002). Improving the quality of undergraduate peer assessment: A case for student and staff development. Innovations in Education and Teaching International, 39, 71-81.
Yu, F. Y., Liu, Y. H., & Chan, T. W. (2005). A web-based learning system for question-posing and peer assessment. Innovations in Education and Teaching International, 42, 337-348.
Van der Pol, J., Van den Berg, I., Admiraal, W. F., & Simons, P. R. J. (2008). The nature, reception, and use of online peer feedback in higher education. Computers & Education, 51, 1804-1817.
Van Gennip, N. a. E., Segers, M. S. R., & Tillema, H. H. (2010). Peer assessment as a collaborative learning activity: The role of interpersonal variables and conceptions. Learning and Instruction, 20(4), 280–290. doi:10.1016/j.learninstruc.2009.08.010
Van Zundert, M., Sluijsmans, D., & Van Merrienboer, J. (2010). Effective peer assessment processes: research findings and future directions. Learning and Instruction, 20(4), 270-279.
Venables, A., & Summit, R. (2003). Enhancing scientific essay writing using peer assessment. Innovations in Education and Teaching International, 40, 281-290.
Wishart, J. M., Oades, C. E., & Morris, M. (2007). Using online role play to teach internet safety awareness. Computers & Education, 48(3), 460-473.