International Review of Research in Open and Distributed Learning

Volume 17, Number 2

February - 2016

Learners’ Interpersonal Beliefs and Generated Feedback in an Online Role-Playing Peer-Feedback Activity: An Exploratory Study

Yu-Hui Ching and Yu-Chang Hsu

Boise State University, USA

Abstract

Peer feedback affords interaction and critical thinking opportunities for learners in online courses. However, various factors prevent learners from taking advantage of these promising benefits. This study explored learners’ perceptions of the interpersonal factors in a role-playing peer-feedback activity, and examined the types of peer feedback that learners generated when playing a role. Participants were 16 graduate students engaged in an online role-playing peer-feedback activity. The results from survey responses revealed learners’ positive interpersonal beliefs, including psychological safety and trust, toward the role-playing peer-feedback activity. In addition, more than sixty percent of the participants reported being more comfortable critiquing peers’ work when playing a role. The content analysis of the peer-feedback entries indicated that learners were able to generate highly constructive feedback entries. In addition to adding supportive comments, those feedback entries identified problems, asked questions, and provided suggestions. The results show that role-play strategy has great potential to enhance learners’ interpersonal beliefs in peer-feedback activity and their feedback quality.

Keywords: role-play, peer feedback, interpersonal, feedback type, online discussion, voicethread

Introduction

The unprecedented growth in the number of online learners in recent years calls for the development of successful online learning skills and effective online teaching pedagogies. According to Allen and Seaman’s (2014) report, a total of 7.1 million students in the United States took at least one online course during Fall 2012, compared to 1.6 million students during Fall 2002. In online learning environments where face-to-face interaction is lacking or limited, potential issues, such as the lack of human contact and reduced connection among participants, can result in disengagement or attrition of learners (Palloff & Pratt, 2007). Pedagogies that provide opportunities for building presence and interactions are critical to ensure learning success in such environments. Peer feedback, a form of social learning that enables high degrees of interaction among learners (Gielen, Dochy, & Onghena, 2011), has the potential to alleviate the aforementioned issues in online learning environments. Socially, learners are able to interact with peers in the learning community for mutual development during the peer-feedback activity. Pragmatically, learners receive frequent and greater volume of feedback in a timely manner than the feedback that the instructor could provide alone (Topping, 1998; van der Pol, van den Berg, Admiraal, & Simons, 2008). Pedagogically, peer feedback exemplifies professional practices that colleagues collaborate and offer input to improve individuals’ work (van der Pol et al., 2008). Moreover, learners gain opportunities to apply knowledge and skills in an authentic context, develop critical thinking, and reflect on ideas from multiple perspectives (Stavredes, 2011).

For the purposes of formative assessment, peer feedback involves learners in considering “the amount, level, value, worth, quality or success of the products or outcomes of learning of peers of similar status” (Topping, 1998, p. 250). In a peer-feedback activity, learners perform the roles of both feedback providers and receivers. As feedback providers, learners contribute their expertise and experiences to support peers’ learning processes (Ertmer et al., 2007). Learners can also broaden their perspectives and deepen their understanding of the learning tasks when they are exposed to peers’ strategies of tackling an assignment. During the reviewing process, learners ponder upon quality, standards, and criteria used for evaluation, a process that contributes to their development of critical and reflective thinking skills (Ching, 2014; Liu & Carless, 2006). As feedback receivers, learners benefit from the constructive comments that help refine and enrich their initial ideas. Empirical studies have established the relationship of peer feedback and learning outcomes. The quality of peer feedback provided by undergraduate students was found to be positively correlated with feedback providers’ own final products, controlling for the quality of the initial projects (e.g., Li, Liu, & Steckelberg, 2010). In addition, undergraduate students improved their writing more by providing feedback to peers than by receiving feedback from peers (Cho & Cho, 2011), as providing feedback engaged students in identifying strength and weakness in writing and possible solutions to improve writing.

Despite the aforementioned educational benefits, cognitive challenges and interpersonal issues have prevented students from taking full advantage of peer feedback activities (Ching, 2014). Reviewing and commenting are cognitively demanding tasks for students as these tasks involve comprehending the work and comparing it to standards (van Gennip, Segers, & Tillema, 2010). These tasks are especially challenging for novice learners who do not possess the domain knowledge to understand the work or who are ignorant of the standards for a given type of work (Nilson, 2003; Palloff & Pratt, 1999). As such, they may provide feedback at a superficial level failing to demonstrate critical thinking (Li et al., 2010), or feedback that is unrelated to the objectives and requirements of the assignment (Nilson, 2003). To address the cognitive challenges of providing peer feedback, different strategies were suggested in the literature. For example, students were trained to define assessment criteria, which led to their improved peer assessment skills (Sluijsmans, Brand-Gruwel, van Merrienboer, & Martens, 2004). Guiding questions provided by the instructor helped improve the quality of peer feedback (Ching & Hsu, 2013a). In addition, Nilson (2003) proposed that the guiding questions need to be shifted away from judgmental questions (e.g., Is the central idea clear throughout the paper?) to identification tasks (e.g., List the types of supporting evidence given in the paper) and personal reactions (e.g., What do you find most compelling about the paper?). To perform the identification tasks and personal reactions, peer reviewers only need to comprehend and analyze the work instead of evaluating the work against unfamiliar criteria. In turn, the outcomes of the identification and personal reaction tasks can help feedback receivers become aware of whether their ideas are communicated clearly to their readers.

Although rarely examined in the empirical studies, critical interpersonal factors have impacts on the learning benefits of peer feedback activities (van Gennip, Segers, & Tillema, 2009). As peer feedback is a type of social and collaborative learning, learners’ interpersonal beliefs inevitably come into play during the feedback giving and receiving process (van Gennip et al., 2010). Trust, one of the critical interpersonal factors in the context of peer assessment, is defined as learners’ belief in a person’s ability of assessment (van Gennip et al., 2009). Inexperienced students often believe that instructors should be responsible for assessment (Ballantyne, Hughes, & Mylonas, 2002). These students question their own abilities to provide useful feedback (McDowell, 1995), and have little confidence in their own or peers’ abilities in assessment (Venables & Summit, 2003). Due to learners’ lack of trust in peers’ abilities, they may refuse to take peer feedback seriously as they do not deem peer feedback as valid as that provided by the “knowledge authority” (Gielen, Peeters, Dochy, Onghena, & Struyven, 2010). For example, Lin, Liu, and Yuan (2002) found that high school students did not feel that their peers were equipped with the knowledge to evaluate their work, while undergraduate students reported more neutral feeling on the trust of their peers as assessors. Secondary-vocational students were found to have increased trust in peers as assessors at the end of a peer assessment activity (van Gennip et al., 2010). Reader responses may be able to mitigate the issue of the lack of trust. When readers offer personal reactions in peer feedback activities, learners may shift their attention from the locus of knowledge authority to how the work is perceived by the readers. In the case of providing reader responses, the peer-reviewer takes on the role of a reader, instead of an evaluator who will make judgment on the work (Nilson, 2003). As a result, the comments are more perceptual instead of judgmental, reflecting readers’ understanding and honest personal reactions to the work.

Another influential interpersonal factor in peer feedback is psychological safety, defined as “a belief that it is safe to take interpersonal risks in a group of people (Edmondson, 1999)” (van Gennip et al., 2009, p. 43). The issue of lacking psychological safety is commonly evidenced in peer review activities where students felt uncomfortable or anxious about critiquing other’s work because they believed that critique may ruin the relationship (Ellison & Wu, 2008; Ertmer et al., 2007; Topping, Smith, Swanson, & Elliot, 2000). Anonymity has been used to improve psychological safety in the peer feedback activities, aiming to create an environment that students feel more comfortable to offer critical and constructive feedback. For example, two instructors have found that their first year college students avoided giving constructive feedback due to their fear of offending their peers in the class (Getchell & Amicucci, 2014). Therefore, they designed and experimented a cross-institutional peer review process that mimicked a blind peer review process to help relieve the interpersonal burden from students. They found that students became more comfortable criticizing their cross-institutional peers’ work. In addition, technology has aided the creation of an anonymous environment to improve learners’ perceived psychological safety for peer review. In the context of computer-mediated discussion, learners generated better ideas when anonymous critical members challenged their ideas (Connolly, Jessup, & Valacich, 1990). Learners also became more willing to give negative feedback if their responses could be anonymous (Freeman & Bamford, 2004) and felt more protected when they explored thorny issues through anonymous online interactions with peers (Dracup, 2012). Moreover, web-based peer review systems have been developed to simulate blind review process in academic practices to support writing and peer-review processes (e.g., SWoRD, scaffolded writing and rewriting in the discipline, in Cho & Schunn, 2007). In such web-based systems, students can work anonymously or with pseudonyms both as author and reviewer to ensure that their critical reviewing is not compromised by the revealing of identity.

However, anonymity may not be the most feasible or applicable strategy to use with a group of online learners who have established relationships and whose identities are easily identified though the responses. An alternative method that allows learners the use of different personas when providing peer feedback was explored in Ching (2014). This study showed that the use of a role-play strategy was associated with students’ generation of more constructive feedback. In addition, some learners reported that the role-play strategy alleviated their affective and interpersonal barriers when critiquing their peers. Building on previous findings (Ching, 2014), this study adopted a role-play strategy to enhance learners’ perceptions and behaviors in an online peer-feedback activity, focusing on learners’ interpersonal beliefs and peer feedback performance.

Role-playing Peer-Feedback Activity

Role-play is an instructional strategy that “provides an imaginary context in which issues and behaviors may be explored by participants who take on a specific role or character” (Bell, 2001, p. 256). It serves several purposes, including enabling students to practice their learning, and creating a concrete and meaningful basis for discussion (McKeachie, 1986). A role-play activity situates learners in a complex scenario that enables them to discuss, debate, and negotiate by assuming different points of view (Russell & Shepherd, 2010). In such an activity, learners are able to apply knowledge in context and receive the consequences of actions safely (Ching, 2014; Dracup, 2012). Educational benefits associated with the role-play strategy include deep-level learning outcomes that resonate for a long time (Bolton & Heathcote, 1999; Dracup, 2008), engaging experiential learning experiences for learners (Raphael & O’Mara, 2002), and rewarding teaching experiences for instructors (Bolton & Heathcote, 1999). Role-play has been widely adopted in various disciplines to achieve diverse learning outcomes: to train medical students and healthcare practitioners to learn communication skills (Lane & Rollnick, 2007; Nestel & Tierney, 2007), to develop business school students in group decision-making skills (Bos & Shami, 2006), to help educators explore issues of quality in education (Beach & Doerr-Stevens, 2011), and to build collaboration and leadership skills of future educational leaders (Howard, McClannon, & Wallace, 2014).

The role-play strategy has the potential to enhance not only the cognitive aspect, but also the interpersonal aspect of a peer-feedback activity. Cognitively, playing a role gives the feedback providers a stance to analyze, interpret, and respond to others’ work, leading to more focused responses and reduced cognitive burden of addressing every strength and weakness they see (Ching, 2014). Interpersonally, role-playing has the potential to address the issues of lacking psychological safety and trust. When playing a role, learners take on perspectives separated from their own to critique peers’ work, so they can worry less about hurting the relationships (Ching, 2014), thus lessening the issue of psychological safety. On the other hand, role-playing can augment the authenticity of task when learners review peers’ work from an important stakeholder’s perspective. The issue of lacking trust in peers’ abilities as assessors may be mitigated when learners are instructed to provide their readers’ responses from the stakeholder’s perspective, instead of casting judgment on the quality of the work. Although role-play seems to be a promising strategy to enhance the cognitive and interpersonal aspects of peer feedback activities, few empirical studies have verified its effect and provided evidence.

Research Purpose and Questions

This study explored learners’ perceptions of the interpersonal factors critical in a role-playing peer-feedback activity, as well as the types of peer feedback learners generated. Specifically, the study answers the following research questions:

1. What are learners’ perceptions of interpersonal factors in a role-playing peer-feedback activity?
   1. How do learners feel about critiquing peers’ work when using the role-play strategy?
   2. What are learners’ perceptions of psychological safety and trust in the role-playing peer-feedback activity?
2. What types of peer feedback do learners provide when they adopt the role-play strategy?

Research Method

Participants and the Context

Sixteen graduate students in an educational online master’s program in a northwestern state university in the United States participated in this study voluntarily. These students were recruited from an Instructional Design course lasting for 16 weeks. Sixty-three percent of the participants were males. Fifty percent of the participants were between 31 to 40 year old, 31% were younger than 30 years old, and 19% were older than 41 years old. The online course in which these graduate students were enrolled was hosted on the Moodle learning management system. This asynchronous online course provided ample opportunities for student interactions using discussion boards in Moodle. Various instructional strategies were used in this hands-on course in order to embed opportunities for knowledge application, including the use of project-based learning method, case studies, and peer-feedback activities.

Learning Activity

This study was situated in a mandatory role-playing peer-feedback activity that lasted for four weeks and was a modified version of a previous study (Ching, 2014). Each learner individually analyzed a case representing a complex instructional design problem, created an asynchronous audio or video presentation of their analysis using VoiceThread, and published their presentation on the VoiceThread platform. Learners then posted the URLs to their VoiceThread presentations in a designated discussion forum on Moodle to inform their peers about their published presentations. Following the instruction, learners played a self-selected stakeholder’s role to provide constructive feedback on peers’ presentations. The peer commenters listened to asynchronous presentations and recorded their comments in either text, audio, or video format on VoiceThread. Lastly, learners incorporated peer feedback they received to revise their original analysis and submitted a written analysis for a grade. VoiceThread was selected as the tool for collaboration because it affords multimodal communication in which users can share text, audio or video comments (Ching & Hsu, 2013b; Hsu, Ching, & Grabowski, 2014). The design of the current peer-feedback activity allowed learners to take advantage of multimodal communication as they could choose a communication mode with which they felt most comfortable. In addition, VoiceThread met a variety of criteria of a useful online role-play environment (Russell & Shepherd, 2010) including authenticity, asynchronous communication, student accessibility, and low set-up costs (Ching, 2014).

The activity instruction was redesigned to purposefully reinforce the use of the role-play strategy for constructive feedback. In the previous implementation (Ching, 2014), it was found that some of the feedback entries were provided without learners using the role-play strategy. To improve the adoption of the strategy, the instruction emphasized the importance of role-play and asked students to identify the role they took in the beginning of their responses. As a result, all students adopted the strategy when providing peer feedback in this implementation. To elicit constructive feedback, four prompts were provided to help learners construct their feedback: (a) how does the analysis address your existing (the stakeholder's) concerns and/or needs, (b) what are some questions you have to help improve your peer’s analysis and solutions, (c) what are the concerns you may have toward the analysis (and the solutions if there are any), and (d) what are some suggestions you can provide to help peers improve their analysis and solutions? These questions were aimed to elicit honest readers’ responses from the feedback providers and to reduce the anxiety of assessing the quality of peers’ work.

Figure 1 shows a screenshot of the VoiceThread presentation of the case analysis created by a participant in this study. This presenter’s avatar is the topmost image in the left panel and two peers commented on the case analysis.

Figure 1. An example of role-playing peer-feedback activity on VoiceThread.

Data Collection and Analysis

We used two data sources to answer the aforementioned research questions: participants’ responses to a survey and peer feedback entries. First, a survey consisting of open-ended questions and Likert scale questions was administered at the end of the peer-feedback activity to answer the first research question. The open-ended questions elicited learners’ perception of critiquing peers’ work when playing a role and their general perception of the role-play strategy. In addition, five Likert scale questions addressed learners’ perception of Psychological Safety, and six Likert scale questions addressed learners’ perception of Trust. These questions asked learners to rate their perceptions using the following four-point scale: Strongly Disagree (1), Disagree (2), Agree (3), Strongly Agree (4). In terms of the open-ended survey questions, we grouped similar survey responses into themes, and evaluated the fit between each student response and the theme. We then tallied the response frequency of each theme.

Second, peer feedback entries on VoiceThread and in the designated Moodle discussion forum were collected and analyzed using content analysis to answer the second research question. Content analysis is a data analysis technique that involves procedures to make valid inferences from text (Anderson, Rourke, Gerrison, & Archer, 2001). To ensure consistency and comparability among the messages provided on VoiceThread and in the discussion forum, we treated a comprehensive comment addressing a whole presentation as one complete message. On VoiceThread, students could give one comprehensive comment to address the whole presentation or several shorter comments on different presentation slides. In the cases that students provided shorter feedback entries at various slides of a presentation, we combined the entries into one complete message. We used one complete message as the unit of analysis, like in other studies that applied content analysis (e.g., Gunawardena, Lowe, Constance, & Anderson, 1997). Using this method, 26 entries of peer feedback were identified for analysis. All the identified feedback entries were coded for cognitive and affective categories using the coding scheme presented in Table 1. This coding scheme was adapted from the scheme used in previous studies (Ching, 2014; Ching & Hsu, 2013a; Lu & Law, 2012). Each peer feedback entry could be coded into multiple categories. The first author and a graduate assistant individually coded the peer feedback entries and the percentage of agreement on the codings was at.85.

Table 1

Coding Scheme for the Types of Comments

Results and Discussions

Learners’ Perceptions of the Interpersonal Factors of Role-Playing Peer-Feedback Activity

Psychological safety. Overall, students perceived the psychological safety aspect positively in the role-playing peer-feedback activity. They enjoyed both providing and receiving peer feedback ( M = 3.00, M = 3.21, respectively), believing that it is important to learn how to give constructive feedback to peers ( M = 3.57), and how to receive peer feedback without taking it personally ( M = 3.57). They also disagreed that they felt reluctant to give negative feedback to classmates ( M = 2.50). Table 2 shows the medians, means, and standard deviations of the indicators of psychological safety. When examining the standard deviations of the items, two statements relevant to giving feedback seem to generate the most varied responses: “I enjoy giving peer feedback” ( SD =.70), and “I felt reluctant to give negative feedback to my classmates” ( SD =.83).

Table 2

Descriptive Statistics of the Indicators of Psychological Safety

Note. The scale: Strongly Disagree (1), Disagree (2), Agree (3), Strongly Agree (4).

Trust. The results also showed that learners had positive perception on the interpersonal factor of trust, meaning that students had positive beliefs of peers’ and their own assessment abilities. Learners reported that their peers had adequate knowledge to comment on their work (M = 3.28), and they felt that the peer feedback was helpful (M = 3.07) and sufficient (M = 3.36) to improve their own work. In addition, learners were satisfied with the overall quality of feedback they received (M = 3.29). Regarding the trust of one’s own assessment ability, learners were satisfied with the overall quality of feedback they provided to their peers (M = 3.07) and they believed that they benefited from providing feedback to peers’ work (M = 3.07). Table 3 shows the medians, means, and standard deviations of the indicators of trust. When examining the standard deviations of the items, learners had most varied responses on the following two statements: “The peer feedback I received was helpful to improve my case analysis” (SD =.74), and “I have benefited from providing feedback to peers' work” (SD =.74).

Table 3

Descriptive Statistics of the Indicators of Trust

Note. The scale: Strongly Disagree (1), Disagree (2), Agree (3), Strongly Agree (4).

Perception of critiquing peers’ work. Learners were asked how they felt about critiquing peers’ work when playing a role in an open-ended survey question. Out of 16 students who responded to the survey, 62.5% indicated that playing a role made them feel more comfortable critiquing peers’ work. Sample reasons include “no worries about feeling” and “less self conscious during the peer feedback process.” Two participants commented that,

I felt more comfortable critiquing while playing a role because I was able to justify my comments based on the selected characters’ beliefs and information provided through the case study.

Actually, yea, that did help. Often times it is hard to be critical of peers that you have never met and do not have rapport with.

Two learners (12.5%) reported that they already felt comfortable about peer critiquing so the role-play strategy did not make a difference. One learner felt less comfortable because he or she was not used to pretending to be someone else. Another person felt that playing a role over which one had no choice presented challenges. Two other participants commented that the strategy helped to provide more focused feedback but did not comment on their feeling of critiquing peers with this strategy.

Overall, learners reported positive perception of their psychological safety and trust of their peers’ assessment ability after the use of a role-play strategy in an online peer-feedback activity. This supported the finding of a previous study that the role-play strategy relieved learners’ interpersonal barriers in providing constructive feedback to peers (Ching, 2014). Learners did not feel that they were “personally attacking” others’ work. In addition, the findings also agreed with the role-play literature stating that “Being in-role and online provides freedom of expression, …and encourages risk-taking that can include actions such as trying out novel methods or solutions” (Cornelius, Gordon, & Harris, 2011, p. 62). Stepping into an alternative identity helps learners to concentrate mainly on the learning aspect of the collaborative activity, instead of the interpersonal relationships involved in completing the tasks. As such, the learning performance is more likely to be augmented and learners are more likely to achieve the shared goal (i.e., improving the case analyses collectively) of a collaborative activity (Ching & Hsu, 2011). In a web-based or online learning environment, it is possible to structure an anonymous peer-feedback activity to alleviate the interpersonal issues that can complicate the collaborative activity. However, anonymity may not be the most feasible and applicable strategy to use with a group of online students who have established relationships and whose identities are easily identifiable though the responses. The role-play strategy as enacted in this study, although not resulting in anonymity, had a similar effect of relieving the interpersonal burden associated with critiquing peers’ work. In addition, the guiding questions also helped learners focus on providing readers’ responses in their feedback instead of judging the quality of peers’ work. Together, the role-play strategy with guided questions resulted in learners’ positive perception of psychological safety and trust.

The Impact of Role-Playing on the Types of Peer Feedback Provided

Using the coding scheme presented in Table 1, 26 entries of peer feedback were analyzed. Table 4 presents the descriptive data showing the detailed breakdown of the percentage of different types of comments generated with the role-play strategy. Overall, 77% of the entries identified at least one problem, about 73% of the entries contained at least one question, and about 54% of the entries made suggestions. In addition, 88% of the peer feedback entries contained supportive messages.

Table 4

Percentage of the Types of Comments Generated with Role-Play Strategy

The results of the content analysis showed that when students employed the role-play s trategy to provide peer feedback, their comments were likely to identify problems, ask questions, and provide suggestions, in addition to showing support. Previous studies indicate that learners are least likely to identify problems in their peers’ work when they are not being prompted to do so, or when they do not adopt a role-play strategy (Ching, 2014; Ching & Hsu, 2013b), due to cognitive and interpersonal reasons (e.g., Ertmer et al., 2007). Affirming previous findings in Ching (2014), the results of this study demonstrated that the use of the role-play strategy resulted in a high percentage of feedback entries containing problem identification. To identify problematic area in peer’s work, learners need to use their newly learned knowledge and skills for analysis. The ability to detect problematic areas in peers’ work is associated with better domain knowledge learning of the feedback provider (Lu & Law, 2012). Furthermore, when playing a role, learners were willing to question peers’ ideas, either for clarification or for probing deeper thinking. The high percentage of problem identification and question comments in the peer feedback may suggest that learners overcome the interpersonal barriers of critiquing peers and focus more on completing the assigned cognitive task. Taking together the findings from the types of comments and learners’ interpersonal beliefs, the lower percentage of suggestion type of comments coupled with the higher variation in scores on the usefulness of feedback (i.e., item 4 in Table 3) may suggest that learners are looking for specific takeaways to help them improve their own work from peer comments. However, this relationship needs to be further verified in future studies.

Conclusion and Implications

This exploratory study aimed to contribute to the online learning research and practice through exploring learners’ perception of interpersonal factors in a role-playing peer-feedback activity, as well as the types of peer feedback learners generated when playing a role. The findings of this study revealed learners’ positive interpersonal beliefs, including psychological safety and trust, toward the role-playing peer-feedback activity. These factors have been identified to have critical impact on learners’ acceptance of and their performance in administering peer feedback (van Gennip et al., 2009; van Gennip et al., 2010). Also, more than 60% of the participants reported being more comfortable critiquing peers’ work when playing a role, which was also evidenced in the peer-feedback entries they generated. In addition to the feedback that showed support, the content analysis of the peer-feedback entries indicated that learners were able to generate highly constructive feedback entries that identified problems, asked questions, and provided suggestions. This study contributes to the peer feedback literature by examining both the interpersonal factors and the types of feedback that learners provided, as these interpersonal factors have rarely been examined in the context of peer-feedback research (van Gennip et al., 2009; van Gennip et al., 2010). Based on the results of this exploratory study, we suggest that distance learning educators and instructional designers consider incorporating the role-play strategy to enhance learners’ interpersonal beliefs in a peer-feedback activity and improve their feedback quality.

While the role-play strategy helped 62.5% of the participants feel more comfortable critiquing peers’ work, it did not have the same impact on all the learners as some learners still felt reluctant to give negative feedback to their classmates. Previous studies have identified that cognitive style and level of academic achievement were predictors of learners’ peer feedback performance (Davies, 2006; Van Zundert, Sluijsmans, & van Merrienboer 2010). Learners’ initial interpersonal beliefs may also impact their generation of constructive feedback. For example, two participants in this study reported that they were comfortable critiquing peers without using the role-play strategy. Future studies can investigate whether the role-play strategy has similar effects for learners of various levels of interpersonal beliefs in terms of generating constructive feedback. We suspect that the role-play strategy can have stronger effects in assisting learners with less positive interpersonal beliefs to generate constructive feedback.

Another promising direction for future research lies in whether the role-play strategy benefits both males and females in providing more constructive feedback. Prior studies revealed gender differences in the online text-based communication pattern and discourse. That is, females are more likely than males to express agreement and respond positively in online communication (Guiller & Durndell, 2006; Guiller & Durndell, 2007), probably because females tend to care more about interrelationships (Tannen, 1991) and connection with peers (Ching & Hsu, 2015). Playing a role may have greater impact on females in terms of alleviating their interpersonal barriers of critiquing peers and promoting psychological safety in the learning environment.

Role engagement contributes to successful learning in role-play (Cornelius et al., 2011). In this study, learners were asked to select a role and provide feedback based on the perspective of the role. It is unclear how the role is portrayed by the learners and how the provided feedback genuinely reflects the perspective of the role. In the current study, about 50% of the comments were provided with a supervisory role (e.g., school principal or plant manager). A supervisory role may have empowered learners to identify problems and asked questions in their feedback to peers, as these behaviors are often associated with a supervisory role. The roles learners play may empower or limit learners’ responses and interactions with peers. For example, learners who played a manager role often adopted a confident and formal voice (Cornelius et al., 2011), while other learners felt frustrated when their character role limited their agency in the role-play (Dracup, 2012). An examination into the role engagement of learners may shed light on the potential need for additional instructional support. For example, having learners create biographic profiles for the adopted personae may help prepare them to step into the shoes of the selected role for generating representative feedback (Beach & Doerr-Stevens, 2011; Dracup, 2012). In doing so, learners can create projective identities from which they can speak (Beach & Doerr-Stevens, 2011; Gee, 2004). Future studies may look into how the roles are portrayed by learners and whether certain types of roles lead to the generation of more constructive feedback.

Lastly, a limitation of this study lies in the lack of association between survey responses and the feedback entries; therefore, no correlation could be examined between interpersonal beliefs and students’ feedback performance. Future studies with a design that can correlate the interpersonal beliefs and feedback performance are needed to investigate how the interpersonal beliefs relate to students’ peer feedback behaviors and performance. In addition, the findings of this exploratory study need to be interpreted with caution due to the limited number of participants, the specific learning contexts (i.e., adult learners in an online learning environment), and the learning tasks (i.e., a case analysis). Future research is encouraged to verify the current findings with larger samples of learners, or learners of different characteristics in alternative educational contexts.

References

Allen, I. E., & Seaman, J. (2014). Grade change: Tracking online education in the United States. Babson Park, MA: Babson Survey Research Group and Quahog Research Group. Retrieved from http://www.onlinelearningsurvey.com/reports/gradechange.pdf

Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5 (2). Retrieved from http://cde.athabascau.ca/coi_site/documents/Anderson_Rourke_Garrison_Archer_Teaching_Presence.pdf

Ballantyne, R., Hughes, K., & Mylonas, A. (2002). Developing procedures for implementing peer assessment in large classes using an action research process. Assessment & Evaluation in Higher Education, 27, 427-441. doi: 10.1080/0260293022000009302

Beach, R., & Doerr-Stevens, C. (2011). Using social networking for online role-plays to develop students’ argumentative strategies. Journal of Educational Computing Research, 45 (2), 165-181. doi: 10.2190/EC.45.2.c

Bell, M. (2001). Online role-play: Anonymity, engagement and risk. Educational Media International, 38 (4), 251-260. doi: 10.1080/09523980110105141

Bolton, G., & Heathcote, D. (1999). So you want to use role play ? London, UK: Trentham.

Bos, N., & Shami, N. S. (2006). Adapting a face-to-face role-playing simulation for online play. Educational Technology Research and Development, 54 (5), 493-521. doi: 10.1007/s11423-006-0130-z

Ching, Y.-H. (2014). Exploring the impact of role-playing on peer feedback in an online case-based learning activity. The International Review of Research in Open and Distributed Learning, 15 (3), 292-311.

Ching, Y.-H., & Hsu, Y.-C. (2011). Design-grounded assessment: A framework and a case study of Web 2.0 practices in higher education. Australasian Journal of Educational Technology, 27 (5), 781-797.

Ching, Y.-H., & Hsu, Y.-C. (2013a). Peer feedback to facilitate project-based learning in an online environment. The International Review of Research in Open and Distributed Learning, 14 (5), 258-276.

Ching, Y.-H., & Hsu, Y.-C. (2013b). Collaborative learning using VoiceThread in an online graduate course. Knowledge Management & E-Learning, 5 (3), 298-314.

Ching, Y.-C., & Hsu, Y.-C. (2015). Online graduate students' preferences of discussion modality: Does gender matter? Journal of Online Learning and Teaching, 11 (1), 31-41.

Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Sciences, 39, 629-643. doi: 10.1007/s11251-010-9146-1

Cho, K., & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers & Education, 48, 409-426.

Cornelius, S. Gordon, C., & Harris, M. (2011). Role engagement and anonymity in synchronous online role play. The International Review of Research in Open and Distributed Learning, 12 (5), 57-73.

Connolly, T., Jessup, L. M. and Valacich, J. S. (1990). Effects of anonymity and evaluative tone on idea generation in computer-mediated groups. Management Science, 36 (6), 689-703. doi: 10.1287/mnsc.36.6.689

Davies, P. (2006). Peer assessment: Judging the quality of students’ work by comments rather than marks. Innovations in Education and Teaching International, 43, 69-82. doi: 10.1080/14703290500467566

Dracup, M. (2008). Role play in blended learning: A case study exploring the impact of story and other elements. Australasian Journal of Educational Technology, 24 (3), 294–310.

Dracup, M. (2012). Designing online role plays with a focus on story development to support engagement and critical learning for higher education students. Journal of Learning Design, 5 (2), 12–24. doi: 10.5204/jld.v5i2.104

Ellison, N., & Wu, Y. (2008). Blogging in the classroom: A preliminary exploration of student attitudes and impact on comprehension. Journal of Educational Multimedia and Hypermedia, 17 (1), 99-122.

Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G.,...,& Mong, C. (2007). Using peer feedback to enhance the quality of student online postings: An exploratory study. Journal of Computer-Mediated Communication, 12 (2), 412-433. doi: 10.1111/j.1083-6101.2007.00331.x

Freeman, M. A., & Bamford, A. (2004). Student choice of anonymity for learner identity in online learning discussion forums. International Journal on E-Learning, 3 (3), 45–53.

Gee, J. P. (2004). Situated language in learning: A critique of traditional schooling. London: Routledge.

Getchell, K., & Amicucci, A. N. (2014). Responding with the Golden Rule: A cross-institutional peer review experiment. Teaching English in the Two-Year College, 42 (1), 55-64.

Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20 (4), 304-315. doi: 10.1016/j.learninstruc.2009.08.007

Gielen, S., Dochy, F., & Onghena, P. (2011). An inventory of peer assessment diversity. Assessment & Evaluation in Higher Education, 36, 137–155. doi: 10.1080/02602930903221444

Gunawardena, C. N., Lowe, X., Constance, A., & Anderson, T. (1997). Analysis of a global debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17 (4), 397-431. doi: 10.2190/7MQV-X9UJ-C7Q3-NRAG

Guiller, J., & Durndell, A. (2006). “I totally agree with you”: Gender interactions in educational online discussion groups. Journal of Computer Assisted Learning, 22, 368–381. doi: 10.1111/j.1365-2729.2006.00184.x

Guiller, J., & Durndell, A. (2007). Students’ linguistic behavior in online discussion groups: Does gender matter? Computers in Human Behavior, 23, 2240–2255. doi: 10.1016/j.chb.2006.03.004

Howard, B. B., McClannon, T. W., & Wallace, P. R. (2014). Collaboration through role play among graduate students in educational leadership in distance learning. American Journal of Distance Education, 28 (1), 51-61. doi: 10.1080/08923647.2014.868665

Hsu, Y.-C., Ching, Y.-H., & Grabowski, B. (2014). Web 2.0 applications and practices for learning through collaboration. In M. Spector, D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed.) (pp.747-758). New York: Springer Academics. doi: 10.1007/978-1-4614-3185-5_60

Lane, C., & Rollnick, S. (2007). The use of simulated patients and role-play in communication skills training: A review of the literature to August 2005. Patient Education and Counseling, 67, 13–20. doi: 10.1016/j.pec.2007.02.011

Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assess: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41 (3), 525-536. doi: 10.1111/j.1467-8535.2009.00968.x

Lin, S. S. J., Liu, E. Z. F., & Yuan, S. M. (2002). Student attitudes toward networked peer assessment: Case studies of undergraduate students and senior high school students. International Journal of Instructional Media, 29, 241–256.

Liu, N.-F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11 (3), 279-290. doi: 10.1080/13562510600680582

Lu, J., & Law, N. (2012). Online peer assessment: Effects of cognitive and affective feedback. Instructional Science, 40 (2), 257-275. doi: 10.1007/s11251-011-9177-2

McDowell, L. (1995). The impact of innovative assessment on student learning. Innovations in Education and Training International, 32, 302-313. doi: 10.1080/1355800950320402

McKeachie, W. J. (1986). Teaching tips: A guidebook for the beginning college teacher. Lexington, MA: DC. Heath & Co.

Nestel, D., & Tierney, T. (2007). Role-play for medical students learning about communication: Guidelines for maximising benefits. BMC Medical Education, 7. doi: 10.1186/1472-6920-7-3

Nilson, L. B. (2003). Improving student peer feedback. College Teaching, 51 (1), 34-38. doi: 10.1080/87567550309596408

Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace. San Francisco, CA: Jossey-Bass.

Palloff, R. M., & Pratt, K. (2007). Building online learning communities. San Francisco, CA: Jossey-Bass.

Raphael, J., & O'Mara, J. (2002). A challenge, a threat and a promise: Drama as PD for teacher educators. Melbourne Studies in Education, 43 (2), 77-87. doi: 10.1080/17508480209556404

Russell, C., & Shepherd, J. (2010). Online role-play environments for higher education. British Journal of Educational Technology, 41 (6), 992-1002. doi: 10.1111/j.1467-8535.2009.01048.x

Sluijsmans, D. M. A., Brand-Gruwel, S., van Merrienboer, J. J. G., & Martens, R. L. (2004). Training teachers in peer-assessment skills: Effects on performance and perceptions. Innovations in Education and Teaching International, 41, 60-78.

Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. San Francisco, CA: Jossey-Bass.

Tannen, D. (1991). You just don’t understand: Women and men in conversation. London, UK: Virago Press.

Topping, K. (1998). Peer-assessment between students in colleges and universities, Review of Educaitonal Research, 68, 249-276. doi: 10.3102/00346543068003249

Topping, K. J., Smith, E. F., Swanson, I., & Elliot, A. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment and Evaluation in Higher Education, 25, 146-169. doi: 10.1080/713611428

van der Pol, J., van den Berg, I., Admiraal, W. F., & Simons, P. R. J. (2008). The nature, reception, and use of online peer feedback in higher education. Computers & Education, 51, 1804-1817. doi: 10.1016/j.compedu.2008.06.001

van Gennip, N. A. E., Segers, M. S. R., & Tillema, H. H. (2009). Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features. Educational Research Review, 4, 41-54. doi: 10.1016/j.edurev.2008.11.002

van Gennip, N. A. E., Segers, M. S. R., & Tillema, H. H. (2010). Peer assessment as a collaborative learning activity: The role of interpersonal variables and conceptions. Learning and Instruction, 20 (4), 280–290. doi: 10.1016/j.learninstruc.2009.08.010

van Zundert, M., Sluijsmans, D., & van Merrienboer, J. (2010). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20 (4), 270-279.

Venables, A., & Summit, R. (2003). Enhancing scientific essay writing using peer assessment. Innovations in Education and Teaching International, 40, 281-290. doi: 10.1080/1470329032000103816

Learners’ Interpersonal Beliefs and Generated Feedback in an Online Role-Playing Peer-Feedback Activity: An Exploratory Study by Yu-Hui Ching and Yu-Chang Hsu is licensed under a Creative Commons Attribution 4.0 International License.