July - 2005

Technical Evaluation Report

48. Rubrics and Exemplars in Text-Conferencing

Allan Zahara
Athabasca University - Canada's Open University

Abstract

The author draws on his K-12 teaching experiences in analyzing the strengths and weaknesses of asynchronous, text-based conferencing in online education. Issues relating to Web-based versus client-driven systems in computer-mediated conferencing (CMC) are examined. The paper also discusses pedagogical and administrative implications of choosing a synchronous conferencing approach.

Introduction

What are the main pedagogical strengths and weaknesses of asynchronous text-conferencing in Kindergarten to Grade 12 (K-12) education? Since asynchronous methods are the most commonly used forms of online conferencing, the paper will focus less on comparing synchronous and asynchronous modes, than on evaluating asynchronous conferencing in its own right. The author draws upon his experience in developing evaluation criteria and modeling of asynchronous conferencing methods in a group-paced, Grade 12 Political Science course. As indicated in previous reports in this series, successful computer-mediated conferencing (CMC) depends upon a sound pedagogy. Discourse, collaborative, and inquiry-based learning are the pedagogies favoured in the current paper, with dialogue as the common thread in all of these approaches. As Rorty (1980) suggests, we should see "conversation as the ultimate context within which knowledge is to be understood."

In the form of Web-based discussion/ bulletin boards, asynchronous CMC has been in use long enough in the K-12 environment for us to recognize that it presents new opportunities and challenges. The most exciting potential lies in the expansion of opportunities for students to overcome the constraints of the traditional classroom, time, space, and access to information and experts. In addition, students gain opportunities to record, develop, and refine their knowledge in a structured database, as part of a community of online learners such as The Knowledge Forum. Nevertheless, distinct problems have emerged in the use of discussion boards. The following observations, though derived from the post-secondary environment, are confirmed by the author's K-12 experiences.

[M]any current dialogical approaches to computer-supported collaborative learning (CSCL). . . use generic 'off the shelf' computer-mediated communication (CMC) systems. Hence, these approaches tend to be technology-led, and take virtually no account of the importance of pragmatic level features. Instead, these systems operate as mere conduits of dialogue, and fail to provide the structure, management, and guidance that is often necessary to support and mediate effective educational collaboration. Further, it has been noted that online tutors at the UK Open University and elsewhere consistently comment that using generic CMC systems is problematic. They point out that these systems often encourage educational discourse that is superficial (e.g., incoherent and with no agreed closure), ambiguous (e.g., lack of shared meanings and little appreciation of different points of view) or simply unmanageable (e.g., too many contributors and too much dialogue) (Ritchie and Peters, 2001).

CMC Rubrics and Exemplars

Part of the solution to CMC problems is to ensure an adequate structure for the CMC assignment, with clear expectations and criteria for evaluating student performance. Useful tools include carefully designed rubrics for the evaluation of student contributions, and the study of exemplars of successful previous work. Rubrics are best developed with the students themselves. As conferencing participants, they should learn to differentiate between research postings, which provide data in the form of facts and expert opinion, and discussion postings, which involve the negotiation of meaning between participants. Both data and discourse are essential to the pedagogical framework, and therefore need to be explicitly examined prior to student involvement. The rubric is a useful tool for achieving this level of understanding. Exemplars provide students who may have difficulty moving beyond rote learning to knowledge as discourse, with participation models.

Careful structuring of time is important to the success of asynchronous CMC. In contrast with synchronous CMC modes, the asynchronous mode usually provides extra time in which to compose thoughts, and classroom momentum should be encouraged to sustain interest and opportunities for meaningful interchange. The author has found that short assignments or assignment segments, with a 36-72 hour window for short discourse-based inquiry, strikes an effective balance with the flexibility that students' value in conducting online learning activities.

The authenticity of the learning task is also essential to creating an engaging CMC learning opportunity. This means grounding assignments in topics which are meaningful to the students and significant beyond the classroom. Inquiry-based learning is a means of achieving this. Students can be engaged in directed research relating to their own agenda. This has been successfully combined with collaborative discourse in CMC through the use of 'cooperation scripts' (Hron and Friedrich, 2003). These inquiry-based sentence fragments or 'learning stems' are used to focus collaboration and are built into the CMC tool. Useful examples include: "How do you know that . . . " and "My theory is . . ."(Knowledge Forum, 2003). The goal is to provide an adequate balance between structure and student-directed discourse and inquiry.

These principles are applied in structured CMC activities including the Facilitation of Dialogue Games (Ritchie, 2001). This approach uses game theory to structure guided student inquiry, in dialogue with a teacher/ tutor/ facilitator: "Within this game, the student is questioned and encouraged to express their understanding of a domain and to refine this in response to the tutor-system reasoning about the learner's explanations examining their completeness, consistency and generality, and consequently challenging, critiquing, and probing the student's explanatory model" (Ritchie, 2001). Another means of adding authenticity to CMC is to include 'tele-mentors' or experts who are brought virtually into the classroom to assist student inquiry. In collaborative or individual inquiry, students are assisted by an expert mentor who guides them through a defined inquiry process (O'Neill, 2000). Klemm (1999) suggests a checklist for the successful moderation of CMC that uses such approaches:

Hron and Friedrich (2003) describe the different functions played by the teacher/ moderator/ facilitator. The moderation functions listed in Figure 1 include organization, motivation, expertise, and didactics. This taxonomy is useful in pointing out that expertise is far from being the only criterion for successful teaching in this relatively new medium.

Figure 1. CMC moderator functions (after Hron and Friedrich, 2003)

 

Consideration of the learners' characteristics is crucial to good CMC teaching. Sabry and Baldwin (2003) describe the influence of CMC on learning styles as follows.

Providing a variety of asynchronous interactions through, for example, the use of discussion (bulletin) board for different activities such as different group assignments, discussions, brainstorming, activities and problem solving exercises can help Sequential learners to get involved in a progressive manner and be able to see the development of the argument, while also giving opportunities for Global learners to obtain a holistic view of the discussion through the linking of different discussions to subject topics that constitute the whole (Sabry and Baldwin, 2003).

K-12 educators must increasingly consider not just learning style differences, but also the challenges of learning in a second language. Young (2003) found an increased willingness to use a second language through the lessening of 'psychological barriers' in asynchronous CMC activities. Although grammar and vocabulary were not notably improved, students demonstrated increased critical thinking and problem-solving skills. These positive effects may be related to the effects of asynchronous CMC noted by Hron and Friedrich (2003) - e.g., being able to compose a message 'in peace,' the existence of a long term record for reference, and the absence of social cues requiring appropriate timing, as in face-to-face communication. As K-12 educational researchers and practitioners move from the present early stages of combining sophisticated pedagogy with next-generation CMC, a systematic approach to design, application, evaluation, and feedback will be required. As Ravenscroft and Matheson (2002) indicate:

. . . the aim is a much closer fit between empirical research, design, implementation and evaluation in educational technology research and development, that has direct implications for designing tools supporting (CMC) . . . The research is considering pedagogy, technology and dialogue context in the design of educational interactions, in ways that treat designs, like theories, as something that are developed, evaluated and refined rather than 'delivered' (Ravenscroft and Matheson, 2002).

References

Hron, A., and Friedrich, H. (2003). A review of web-based collaborative learning: factors beyond technology. Journal of Computer Assisted Learning, 19(1), 70 – 79.

Klemm, W. (1999). Eight ways to get students more engaged in on-line conferences. Journal of Higher Education, 26(1), 62 – 64.

Knowledge Forum (2003). Introducing Knowledge Forum 4.5 for Elementary, Middle School, High School, and Professional Development. Retrieved June 15, 2005 from: http://www.knowledgeforum.com/K-12/k12.htm

O'Neill, K., Abeygunawardena, H., Perris, K., and Punja, Z. (2000). The Telementor's Guidebook: A field manual for supporting student inquiry on-line. Retrieved June 15, 2005 from: http://www.stanford.edu/class/educ298/2002/ONeill-telementoring.pdf

Ravenscroft. A., and Matheson, M. (2002). Developing and evaluating dialogue games for collaborative e-learning. Journal of Computer Assisted Learning, 18(1), 93.

Ritchie, G., and Peters, S. (2001). Using narratives in conferences to improve the CMC learning environment. Journal of Computer Assisted Learning, 17(4), 376.

Rorty, R. (1980). Philosophy and the Mirror of Nature. Oxford: Blackwell.

Sabry, K., and Baldwin, L. (2003). Web-based interaction and learning styles. British Journal of Educational Technology, 34(4), 443 – 454.

Young, S. (2003). Integrating ICT into second language education in a vocational high school. Journal of Computer Assisted Learning, 19(4), 447 – 461.

The next report in the series discusses the role of instructional design in learning object development.

N.B. Owing to the speed with which Web addresses are changed, the online references cited in this report may be outdated. They can be checked at the Athabasca University software evaluation site: http://cde.athabascau.ca/softeval/. Italicised product names in this report can be assumed to be registered trademarks.

JPB. Series Editor, Technical Evaluation Reports