Supporting quality learning in online discussion forums is an intricate task, particularly for e-tutors aspiring to facilitate vigorous interactive learning environments. I argue that the key to successful online discussion forums is the ability of e-tutors to provide learners with feedback well informed in the meaning making and knowledge advancement processes emanating from learner interactions. In this paper, a newly developed concept of providing e-tutors with the information they require is explored, exhibiting the Event Centre (EC) concept, through which tutors are able to obtain periodic “snapshots” of the occurrences throughout discussion forums, which highlight processes of meaning construction and knowledge advancement. The EC concept provides e-tutors with visual images that depict the links and routes through which participants using text messages convey meaning, construct knowledge, and create Socio-Informational networks within discussion forums.
Keywords: e-learning; online discussion forums; e- tutoring; visualising social networks; monitoring online learning; online constructivist learning
The technology applied by Online Discussion Forums accommodate the potential for e-tutors and e-learners to engage in continuing tutorials, rich in dialogues and reflections, and generate processes of meaning construction and knowledge advancement. Accomplishing the full potential entailed in the technology requires feedback support that is well informed in meaning construction and knowledge advancement processes, that in turn support students’ understanding and knowledge construction (Rowntree, 1997).
In my role as a “participant observer” in an online tertiary course, I realised that reading and responding to students’ messages on a message-by-message basis had its merits, particularly in encouraging interactions and social negotiations of ideas and meaning making (Jonassen, Davidson, Collins, Campbell, and Bannan Haag, 1995). At times, however, I felt that this sporadic manner of response to students was lacking the overview needed to support ongoing processes through which knowledge is developed and advanced over a period of time. As an e-tutor, I needed prompt access to the process as a whole, so as to identify significant turning points or defining moments in the meaning making processes. The more intensive the interactions in the discussion forum became, the more I felt the need for a tool that would enable more than merely sporadic, message-by-message feedback to support students.
In this paper, I argue that for e-tutors to be able to provide learners with relevant, well informed feedback, there needs to be a mechanism that would enable:
The two issues raised here reveal the complexity of the information needed, as they imply the analysis of large quantities of information, while maintaining close investigation of the content conveyed, mindful of key instances in the processes of meaning and knowledge construction.
I will begin this paper by addressing issues related to using online discussion forums as interactive learning environments; I will then review existing approaches for analysing Online Learning Discussion Forums (OLDF); and proceed to describe the development of a new concept presented in this paper, called the Event Centre (EC) concept, which attempts to provide e-tutors with an accessible tool of analysis based on visualisations of the occurrences in discussions forums, and significant instances of the meaning construction process which emerge throughout the discussion. In the final section, I outline the potential entailed in the EC concept for providing relevant feedback, thereby supporting meaning making and knowledge construction (albeit, not exhibiting the processes themselves, as this would be beyond the scope of this paper).
Online learning discussion forums are predominantly embedded in e-learning, which is primarily a form of technology-mediated learning used for the benefit of distance learners. Historically, distance education has always relied on technology for reaching learners. Earlier forms of technology used for distance learning – i.e., television – lacked the central essence of quality teaching, specifically the ability to interact with the learners. The necessity to provide interactivity turned the attention to computer-based learning(Rosenberg, 2001). Computer-based learning provided Human-machine interactivity; however, it failed to provide human feedback so important for successful learning (Laurillard, 1993).
Apart from the inability to cater for human feedback, all earlier used learning technologies were primarily based on “the sage in the box” model. This predominant model of teaching and learning in technology-mediated learning was based on that of the “traditional classroom” concept, wherein the teacher or an alternative source of authority, transmitted a fixed body of knowledge to students. This traditional approach has been contested by constructivist principles, where learners are encouraged to take an active part in the learning process and construct their knowledge by interacting with learning materials and their peers(Sherry, 1996). Distance learning was able to embrace constructivist principles with the arrival of computer networks, allowing people to interact with other people for mutually constructing knowledge. For distance learning, particularly in tertiary education, the arrival of computer networks introduced the potential for a paradigm shift in the perception of teaching and learning (Kanuka and Anderson, 1999).
Jonassen and colleagues (1995) believe that the computer networks infrastructure running online discussion forums, enable the application of constructivist learning theories, which emphasis the importance of social interactions for the construction of knowledge. Constructivist principles provide a set of guidelines for creating learner-centred, collaborative environments that support reflective and experiential processes. Within the constructivist approach, learners and teachers are urged to construct meaning, understanding, and relevant practice together through social interactions (Jonassen et al., 1995). This approach is associated with the Vygotskian view that learners co-construct meanings actively and continuously in a social context (Young, 1997, p.107). For Vygotsky (1978) shared meaning is created especially in the Zone of Proximal Development (ZPD), which he defines as: “the difference between the development of the individual’s performance in ‘independent problem solving,’ and in ‘problem solving under adult guidance’ or in collaboration with more capable peers” (Vygotsky, 1978, p.86).
Tharp and Gallimore(1991) argue that teaching occurs when assistance is offered at points in the ZPD at which performance requires assistance. They propose that there are three major mechanisms for assisting learners through the ZPD: modelling, contingency management, and feedback (Tharp and Gillmore,1991 as cited by Bliss, Askew, and Macrea, 1996). In this paper, I will focus on the third mechanism – feedback.
A survey conducted by McCollum et al., (McCollum, Calder, Ashby, and Morgan, 1995) showed that students ranked feedback, or assistance in learning, as the highest factor in determining course quality. Rowntree (1997) describes feedback as “the key to quality in education and training” (p. 58), and suggests that “feedback is what enables us all to learn from our experience . . . What learners need is something personal – a response from another human being that challenges or confirms their understanding and helps them overcome errors or encourage them towards new insights” (Rowntree, 1997, p.58).
Laurillard (1993), in her analysis of university teaching, emphasises the importance of feedback provided through dialogic interactions, wherein tutors can provide intrinsic and adaptive feedback. Laurillard suggests that: “No simulation or technology is able to give truly intrinsic or fully customised feedback, the closest they can manage is ‘extrinsic feedback.”’ Online tests self-assessment questions and other artificial sources of formative feedback cannot provide the degree of depth or insight required for customised learning assistance . . . in the ways a human tutor can” (Laurillard, 1993, p. 153). The emphasis Laurillard puts on human dialogue and feedback highlights the importance of giving e-tutors the necessary tools for providing what she refers to as “customised learning assistance.”
Customisation of the learning process provides an opportunity for students to have course materials interpreted in ways meaningful to them. This is why tutors need to be aware of students’ conceptual processes and difficulties (Ramsden, 1988). Providing tutors the tools necessary for obtaining this information is crucial for the customisation of the learning and the provision of relevant feedback.
The literature acknowledges the importance of feedback for learning, and implies that e-tutors involved in implementing constructivist learning approaches in online environments would benefit from having analysis tools to help them identify significant instances of meaning construction, so as to better support these processes. In the search for such analysis tools, the following section summarises existing research done in the realm of online discussion forums.
Prevalent research approaches studying Online Learning Discussion Forums (OLDF), encompass a wide variety of research methodologies applied across a wide array of research perspectives, ranging from quantitative measurements of “log-in frequencies” (Monroe, 2003), to descriptive “quantitative content analysis” that measures the frequency of contributions per student (Bullen, 1998; Weis and Morrison, 1998, as cited by Rourke and Anderson, 2004), through to inferring from quantitative content analysis for assessing learning processes in online contexts (Kanuka and Anderson, 1998), or alternatively applying “qualitative content analysis” techniques for studying the quality of the messages as artefacts of critical thinking and argumentation content (Jeong, 2003).
Quantitative and qualitative content analysis techniques provide valuable information about learning processes; however, Rourke, Anderson, Garrison, and Archer (2001) argue that content analysis techniques present researchers with a number of difficulties, some of which are of pragmatic nature, as content analysis is a time consuming technique, particularly as described in the scenario portrayed by Rourke and Anderson (2004). Other difficulties are of a methodological nature, for example objectivity, reliability, replicability (Rourke et al., 2001).
Although OLDF could predominantly be described as social constructivist learning environments (Ferdig, Roehler, and Pearson, 2002), most studies conducted within the realm do not seem to attribute much attention to the interactions and the social dynamic processes occurring in OLDF, and the meaning making and knowledge advancement these portray (Collins and Berge, 2001; Jeong, 2003), although Levin, Haesun and Riel (1990) attempted to study interactions by developing an approach they called “Inter-Message Reference Analysis,” where messages were analysed by coders who determined whether a message was referring to a previous message. These researchers, however, admit that this method proved ambiguous in some cases (Wang, 2000).
Adopting the “login” quantitative approach would grant e-tutors reasonable access to periodic updates concerning students’ login behaviour patterns. Information obtained through this approach, however, is stripped of the content conveyed by participants. On the other hand, content analysis techniques, both quantitative and qualitative, are not readily inclined to periodic updates, as the processing of text messages is time consuming, and at times not generalisable even within a specific OLDF.
Furthermore, neither “login” nor “content analysis” techniques provide a descriptive, analytic framework that would enable studying online interaction and the affect these may bear on learning, just by looking at the data, without having to make any inferences (Rourke and Anderson, 2004).
As such, I argue that the approaches presented in the available research are not well-equipped for the analysis requirements needed for supporting constructivist learning and providing learners with informed feedback.
The idea underlying the Event Centre – herein called the EC concept – simply suggests that people participating in online discussion forums, or more specifically, discussion threads, are in fact taking part in various conversational events, which linked together comprise a discussion thread. Each of these conversational events may involve constant or varying groups of participants.
In attempting to study the evolvement of discussion threads, their content and participants’ activities, I chose to break down the process into three stages:
First stage – “who talked to whom?”
Second stage – “who talked to whom” and on “which occasion?”
Third stage – “who talked to whom,” on “which occasion,” and “about what?”
In the first stage, in attempting to discover “who talked to whom,” I applied Web usage mining, which allowed me to discover and analyse useful information from the Web-data, and use it for trailing users’ behaviour on the Web (Wang, 2000), as well as discover patterns of usage (Srivastava, Cooley, Deshpande, and Tan, 2000). The e-learning system running the discussion forums comprising my data-set is based on an SQL server, which automatically generates a triple coding for every message posted to forums:
By manually extracting the server’s generated key codes, I created Table 1, which lists in the left column the names of participants coupled with the “Thread” and “Post” keys. Combining the participant with the message coding enabled me to associate a participant with a specific message. The right column of Table 1 lists the “Thread” key coupled with “Parent” key of the messages sent by the participant showing on the left.
Table 1 contains the information needed for discovering “who posted,” “which message,” as well as “in response to whom.” The manner in which the information is represented, however, is not very clear, and is quite inaccessible when trying to analyse whole discussion threads, which are much longer than the small sample shown in Table 1.
I converted Table 1 into a binary matrix, (Table 2). The zeros (0) in the matrix indicate no entries; the ones (1) indicate entries. The matrix format is better suited for visually depicting who responded to whom, by simply following columns containing more than one entry.
Table 2 shows that Bob and John responded to the same parenting messages marked 100. Identifying the original or “Parent” message and its responses was possible by following the second part of the key codes, which are highlighted in bold fonts in Table 3. Identical second parts of the code meant people were responding to the same message, creating a conversational event where participants responded to an originating message.
To gain a clearer visualisation of the conversational events and their participants, I converted the matrix of Table 3 into a graph. Figure 1 marks participants in grey circles and conversational events in black shapes. The white square identifies the initial message at the start of the discussion.\
Identifying the conversation events completed the Second Stage relating to “who talked to whom” and “on which occasion.”
The concept emerging from this process perceives the activities in online discussion forums as interactions between people, communicating with other people in different conversational events, which I chose to name “Event Centres.” An Event Centre (EC) is formed when one or more participants respond to a message posted on the discussion thread. The various ECs, and the participants contributing to them, form networks of people connected by events.
Addressing the third stage – “who talked to whom,” “on which occasion,” and “about what,” the EC concept enables the visualisation of the participants of each EC with the relevant messages and responses associated with it. Figure 2 indicates the participants in each EC, by linking participants to the messages they contributed in the particular EC. (Full content of messages can be found in the referred appendices).
A Web-view of this concept is available on:http://etalk.bravehost.com. This Web-view demonstrates the EC concept, which I manually constructed myself. Further technological development needs to be done, however, for the EC to be accessible and ready for practical use by e-tutors or researches interested in processes occurring in online discussion forums.
By archiving the messages generated by a group of 19 educators participating in a Master’s level distance-learning course, I used the EC concept to analyse the data collected. The group I observed used the online discussion forums activated throughout the course for dialogue and reflection. The course lasted 10 months, during which 32 different discussion forums were activated, producing 299 discussion threads. For the purpose of this study, I chose to include the 131 threads showing three and above contributions, while ignoring the shorter threads.
By processing the 131 threads, using the method described in Tables 2 and 3, and Figure 1, reoccurring patterns in the dynamics underpinning the emergence of Event Centres (EC) within my data, could be detected. In categorising the patterns that emerged, I focused on following the links spawned from one EC to the next. For visual clarity purposes, I discoloured the activities of the members in each EC, highlighting only the ones linking one EC to another.
Six observable categories of patterns emerged from my analysis. Two uninvolved colleagues and myself, carried out categorising each of the 131 threads, according to the six categories.
Category 1 reveals Uni-Focal dynamics, resulting in a single EC consisting of the initial EC (marked in a square shape). This pattern emerges when all participants respond to the initial EC, creating a star-like shape of all responses directed to one single point. This is why I chose to call this category the “Star.”
Category 2 reveals Duo-Focal dynamics, resulting in two EC; one is the initial EC (marked in a square shape), initiating the emergence of an additional EC. This pattern emerges when one or more participants engage in responding to a response made to the initial EC, so that a second EC emerges.
Category 3a, The Chain, reveals multiple EC linked in a chain like format, suggesting that the dynamics in this case were produced by respondents replying to a previous message.
In Category 3b, Multi-Chains emerged when responses triggered by various EC on the primary chain (inside the dotted area) emanating from the initial EC (marked as a square), evolved into multiple chains or threads of discussion.
In Category 4a, The Branch, the initial EC (marked as a square), generates two separate responses, which may provoke some continuum in either branches.
Category 4b, The Branch and Chain, depicts chains evolving from each of the branches creating two or at times several chains.
The Multi Chain (3b) differs from The Branch and Chain (4b), in that in the 3b Multi Chain pattern, a primary discussion is emerging from the initial message, and this is the primary chain that spawns additional chains; whereas in The Branch and Chain (4b) pattern, two or more chains emanate directly from the initiating message. For the purpose of this paper, I defined chains as a sequence of three and above messages, excluding the initial message. At this point, this definition is arbitrary, and further investigation into this matter is needed.
Studying the content of the messages posted in the discussion forums comprising my data set, using the EC as my unit of analysis, I was able to identify two key discussion topics:
Using “quantitative content analysis” techniques, I coded all the EC in my data set. I created a third category titled the “mix”’ for cases in which the content of the EC related to both categories of discussion topics.
The graph below shows the total number of messages coded in each of the coding categories:
Categories 1, 2, 3a, and 4a, show an average of 45 percent course related discussions, while Categories 3b and 4b show an increase of 10 percent, with a soaring 55 percent of course related discussions.
The above findings indicated a relationship between patterns, and represent structures created by members’ actions and the amount of information flowing within them. This realisation pointed me in the direction of Social Network Analysis (SNA), which is an intellectual tool for the study of social structures generated by social action (Scott, 2000; Wellman and Berkowitz, 1988). Social Network Analysis maps social structures and describes their patterns using tools derived from mathematical graph theory, thereby enabling the visualisation of structures as well as mathematical analysis of the social actions and relationships constructing them (Wellman and Berkowitz, 1988).
Pursuing the relation between structure and information flow, I chose to apply one of the mathematical tools used by SNA, the “Information Centrality” routine. “Information” in the context of “Information Centrality” routine refers to “the level of ability to transmit, to communicate”(Stephenson and Zelen, 1989). Stephenson and Zelen’s concept suggests that the more central a participant is in the network, the higher their ability to transmit or communicate. The individual’s centrality – or “Actor Information Centrality” as it is referred to in SNA literature – can be extended to “Group Information Centrality,” measuring the average “information,” (or the ability to communicate), across all the individuals in the network(Wasserman and Faust, 1994, p.192-8)
By measuring Information Centrality across all 131 threads of my data, and categorising them in their appropriate category across the six various categories, I was able to measure the level of “information” in each category. Table 4 ranks each of the six the categories’ levels of information centrality in a descending order.
Table 4 distinctly shows that Category 1, The Star, holds the highest levels of Information Centrality; while Category 3b, The Multi-Chain, and Category 4b, The Branch and Chain, are at the low end of the scale.
SNA’s Information Centrality measures distinctly rated Category 1, The Star, as the most efficient pattern for disseminating information, exhibiting a level of 9.1, which is far above all the other categories. High levels of dissemination do not necessarily ensure that all the information disseminated is course related. However, the Star pattern showing this measurement indicates the dynamics in which information is conveyed.
By re-engaging SNA as a tool for the study of social structures generated by social action (Scott, 2000; Wellman and Berkowitz, 1988), I looked at what the patterns might tell me about the dynamics of information flow in different pattern categories. On one end of the spectrum, there was The Star pattern (Category 1), which illustrated centralised dynamics wherein all participants respond to a single message sent by a specific person, hence rendering that person a central position in the group, controlling all transfer of information. This type of dynamics grants the central participant high levels of power, control, and authority. At the other end of the spectrum, The Multi Chain (Category 3b) and The Branch and Chain (Category 4b) show decentralised dynamics, where no single person can be identified as having more control or authority than the others (Ioannides, 2003). In The Multi-Chain and The Branch and Chain, the dynamics seem to flow among dyads of participants, granting all participants less hierarchical opportunities for communication and access to information, although some participants in the network hold a more central position than others, for examples participants situated at the intersection between chains. Figure 10 demonstrates these differences – i.e., the participant marked by the arrow, is at a more central position than the member marked in the circle.
Figure 10.
Comparing the findings of SNA Information Centrality with those of Content Analysis within the context of learning paradigms reveals an interesting notion. Content Analysis clearly identified pattern Category 3b, The Multi Chain, and Category 4b, The Branch and Chain, as conveying high levels of course related discussions; however they ranked low on the SNA Information Centrality measurements (Category 3b at 0.16, and Category 4b. at 0.36). Whereas Category 1, The Star, rated high in the Information Centrality (9.1), but not as high as course related discussions are concerned (44 percent in Category 1, as compared to 55 percent in Categories 3b and 4b).
These two different measuring concepts highlight the debate around the meaning of learning. The SNA Information Centrality approach implies the paradigm that perceives learning as the dissemination of information – or the “sage in the box” paradigm (Sherry, 1996). In this paradigm, Category 1, The Star, would be the preferred model, as it is obviously capable of effective dissemination of information. Categories 3b and 4b, on the other hand, appeal to the paradigm perceiving learning as a two-way communication process wherein learners engage in dialogue among themselves, with their e-tutor, with the learning materials, and social interactions are perceived as supporting the construction of knowledge (Sherry, 1996).
The two different methods of measurements highlight two different perceptions of the meaning of information:
The SNA Information Centrality measurement implies the perception of information is a commodity, which is disseminated from a central resource, accessible and reachable by all participants.
Contents Analysis implies the perception that information is an entity that is constructed by many participants, each contributing their portion, and each enjoying the mutual processes of sharing and constructing an entity collaboratively.
E-tutors are expected to support quality learning, not by simply transmitting information, but rather by facilitating intensive and engaging debates among learners. Achieving these expectations require informed decisions regarding the support and feedback needed at various points of the learning process. Creating such an intensive and closely supported learning environment, however, may increase workloads for e-tutors. In attempting to sustain quality learning, e-tutors will face a conflict situation wherein the more intensive the discussion, the more messages will be generated, thereby creating an ever increasing quantity of information for e-tutors to read, process and provide feedback on.
The EC concept suggests a way of reducing the amount of messages e-tutors will need to process, while obtaining the high levels of information vital for sustaining informed decision making to support quality learning. The visual patterns represented by the six categories described earlier in the paper can serve as indicators, highlighting the probability of members’ involvement in course related discussions, as shown earlier, where patterns depicted in Category 3b, The Multi Chain, and Category 4b, The Branch and Chain, showed levels of 55 percent course related discussions. E-tutors can utilise these patterns as content indicators to identify levels of course related discussions, and focus their reading around a selected sample of messages from each of the pattern categories emerging in their discussion forums. The dynamics of the interaction represented by the EC patterns may indicate instances where critical thinking, or social presence, can be detected; however, pursuing these issues is beyond the scope of this paper.
Visual patterns also reflect on the style of moderation and the tutor intervention practiced, illustrating centralised and non-centralised dynamics of interactions, and indicating levels of authority and influence delegated to participants in the discussion. For example, Category 1, The Star, depicts a centralised situation where a specific individual holds more authority and can practice more influence than any other participant in the group. Having obtained this information, the e-tutor can decide whether this style of moderation practiced meets the learning goals of a particular task or phase in the process.
These visual patterns also highlight influential or dominant individuals within the participants, pointing out those individuals who assume critical positions within the network. e-Tutors could benefit from this information, as they could focus on reading messages generated by these key participants, reducing their necessity to read the less influential messages posted by less critically positioned participants. Additionally, e-tutors may choose to assign the influential individuals as co moderators, acting as “teachers’ assistance.”
The graphic representations facilitated by the EC, enable e-tutors with a powerful tool for conducting periodic formative assessment of the learning dynamics within the discussion forums. The visual maps afforded by the EC concept clearly depicts learning situations where participants are engaged in dialogue or whether students are expecting ready answers form the “sage on the stage.” Although some points in the learning process may require the “sage on the stage” strategy, I argue that the ability to discern which pattern is occurring would better inform e-tutors’ decisions as to whether this is the “desired pattern” at a specific point in the learning process.
The graphic representations of the EC enable clear view of significant points in the process of meaning making and knowledge construction. Observing a division of the discussion into more than one chain of conversation may have significant implications on the meaning making process. For example, Figure11 below indicates two major sub-divisions of the conversations (marked in circles).
The EC concept enables the construction of two-mode networks, one showing the social, and the other the content dynamics underpinning the processes occurring in the Online Learning Discussion Forum (OLDF). The EC concept maps the processes forming socio-informational networks, where human dynamics intertwine with the content they produce. I argue that these obtainable periodic visual maps of the socio-informational networks would provide e-tutors with the much needed mechanism that would manage, as well as process, large quantities of information, thereby supplying e-tutors with the data they require for assessing the situation at hand, and offering the required supportive feedback at various stages in order to ensure quality learning process.
The EC concept addresses the key issues mentioned in this paper, by providing a new concept outside the prevalent Content Analysis on the one hand, and Login files on the other. It looks at possibilities of obtaining a “micro snapshot,” while keeping the macro picture in tact; it looks into the content, while obtaining the dynamics underpinning it; and finally, it acknowledges the importance of the social dynamics at play in knowledge construction processes.
The scope of this paper does not allow me to elaborate on the meaning making processes and knowledge construction as such, because describing these processes would require the attention of the whole paper.
The EC concept was primarily developed for automated processing; however, further development is needed to achieve full automation and accessibility to practicing e-tutors.
The EC concept offers e-tutors periodic snapshots of the interactions occurring in Online Learning Discussion Forums (OLDF), helping them visualise the dynamics underpinning the content, and the ways in which they affect the meaning making processes.
The visualisation offered by the EC concept help facilitate:
Further study and development is needed for making the EC concept accessible to e-tutors. Development of the technological aspects of the EC concept will enable its application using any of the prevalent Learning Management Systems – i.e., WebCT, Blackboard and many others.
Bliss, J., Askew, M., and Macrea, S. (1996). Effective Teaching and Learning: scaffolding revisited. Oxford review of Education, 22(1), 37 – 61.
Collins, M., and Berge, Z. L. (2001). Resources for moderators and facilitators of online discussion. Retrieved April 15, 2004 from: http://www.emoderators.com/moderators.shtml
Ferdig, R. E., Roehler, L. R., and Pearson, D. P. (2002). Building Electronic Discussion Forums to Scaffold Pre-Service teacher Learning: Online Conversations in the Reading Classroom Explorer. Ann Arbor, MI.: Centre for the Improvement of Early Reading Achievement, University of Michigan.
Ioannides, Y. (2003). Topologies of Social Interactions. ECON Papers: Department of Economics Tufts University.
Jeong, A. C. (2003). The Sequential Analysis of Group Interaction and Critical Thinking in Online Threaded Discussions. The American Journal of Distance Education, 17(1), 25 – 43.
Jonassen, D., Davidson, M., Collins, M., Campbell, J., and Bannan Haag, B. (1995). Constructivism and Computer Mediated Communication in Distance Education. The American Journal of Distance Education, 9(2), 7 – 26.
Kanuka, H., and Anderson, T. (1998). Online Social Interchange, Discord, and Knowledge Construction. Journal of Distance Education, 13(1). 57 – 74. Retrieved April 7, 2004 from: http://cade.icaap.org/vol13.1/kanuka.html
Kanuka, H., and Anderson, T. (1999). Using Constructivism in Technology-Mediated Learning: Constructing Order out of the Chaos in the Literature. Radical Pedagogy, 1(2). Retrieved April 7, 2004 from: http://radicalpedagogy.icaap.org/content/issue1_2/02kanuka1_2.html
Laurillard, D. (1993). Rethinking University Teaching: A framework for the effective use of educational technology. New York: Routledge.
Levin, J. A., Haesun, K., and Riel, M. M. (1990). Analysing Instructional Interactions on Electronic message Networks. In L. M. Harasim and M. Turoff (Eds.), Online Education: Perspectives on a New Environment (p. 185-215). New York: Praeger.
McCollum, A., Calder, J., Ashby, A., Thorpe, M., and Morgan, A. (1995). Quality and effectiveness in vocational education. Paper presented at the One World Many Voices, proceedings of the 17th World Conference for Distance Education, Birmingham, UK.
Monroe, B. (2003). Fostering Critical Engagement in Online Discussion: Te Washington State University Study (newsletter). Washington, DC.: Washington Center for improving the Quality of Undergraduate Education.
Ramsden, P. (Ed.). (1988). Improving Learning: New perspectives. London: Kogan Page.
Rosenberg, M. J. (2001). E-Learning: Strategies for Delivering Knowledge in the Digital Age. New York: McGraw-Hill.
Rourke, L., and Anderson, T. (2004). Validity in Quantitative Content Analysis. Unpublished manuscript.
Rourke, L., Anderson, T., Garrison, R., and Archer, W. (2001). Methodological Issues in the Content Analysis of Computer Conference Transcripts. International Journal of Artificial Intelligence in Education, 12(1), 8 – 22.
Rowntree, D. (1997). Making Materials-Based Learning Work. London: Kogan Page.
Scott, J. (2000). Social Network Analysis: A handbook. London: Sage.
Sherry, L. (1996). Issues in Distance Learning. International Journal of Educational Telecommunications, 1(4), 337 – 365.
Srivastava, J., Cooley, R., Deshpande, M., and Tan, P. N. (2000). Web Usage Mining: Discovery and Applications of Usage Patterns from Web Data. SIGKDD Explorations, 1(2), 12 – 23.
Stephenson, K., and Zelen, M. (1989). Rethinking Centrality: Method and examples. Social Networks, 1989(11), 1 – 37.
Tharp, R. G., and Gallimore, R. (1991). A theory of teaching as assisted performance. In P. Light, S. Sheldon and M. Woodhead (Eds.) Learning to Think (p. 42-62). London: Routledge in association with The Open University UK.
Vygotsky, L. (1978). Mind in Society: The development of higher psychological processes. Cambridge MA.: Harvard University Press.
Wang, Y. (2000). Web Mining and Knowledge Discovery of Usage Patterns. Retrieved October 7, 2003, from:
Wasserman, S., and Faust, K. (1994). Social Network Analysis: Methods and applications. New York: Cambridge University Press.
Wellman, B., and Berkowitz, S. D. (Eds.) (1988). Social Structures: A network approach New York: Cambridge University Press.
Young, G. (1997). Adult Development, Therapy, and Culture: A postmodern synthesis. New York: Kluwer Academic/ Plenum Publishers.