April - 2002

Technical Evaluation Report

8. Fully Featured vs. Lean-and-Mean?

Jon Baggaley
Centre for Distance Education
Athabasca University - Canada's Open University

Abstract

With students in an “Educational Conferencing” graduate course, a comparison was conducted between the merits of an online collaborative package containing a wide range of features and alternative conferencing software containing only a threaded discussion and search feature only. The responses of fourteen students taking part in the study revealed a clear preference for the simpler approach. The study has implications for the selection of software tools in the design of online course environments and for the re-design of the product ratings system used in the current series of evaluation studies.

Introduction

The collaborative tools reviewed in this series of evaluation studies vary greatly in their complexity. Whereas some contain a comprehensive range of tools (text and multimedia conferencing, file and image transfer, polling, etc.), others include one or two basic features on a “stand-alone” basis. Many of the reviews published in these reports to date have given high marks to the products with the most features. However, the teachers who supervise these evaluation studies have noted previously that a majority of students prefers simpler methods in their distance education (DE) studies, and tends to resist adoption of more complex methods.

In January-February 2002, Athabasca University’s Centre for Distance Education (CDE) conducted a comparative study of two contrasting conferencing products with the graduate students in its “Educational Conferencing” course (MDDE661):

1. Anyboard is a text-conferencing package that had received the highest ratings in an earlier study for its comprehensive range of features (automated file and image transfer, spell-check, polling, etc.: see Report 4); and

2. WWWBoard is simple software, restricted to a threaded discussion and search tool. This software has been used by the CDE’s graduate programme for the past three years, within a customised online framework of course materials and stand-alone Internet audio techniques. To justify the students’ discussion of this familiar product, two format changes were made: the addition of a pagination feature, which divides the discussion into separate pages after every ten discussion threads; and a change in page colour.

Method

In review of the two products during the CDE’s ‘Educational Conferencing’ course, 16 students were invited to take part in the discussion, and 14 did so. Their edited contributions are presented below.

Product One: The Anyboard product was introduced first, in the second week of the course.

JB, Instructor: A good way to appreciate the wide range of possibilities in computer-mediated conferencing is to compare various approaches. We invite you to place some test postings on this board, and to try out some of its features. All areas of the board are open to you, although for the sake of easy navigation, some of them could be removed if they do not have any particular use in distance education.
Student A: This CMC seems to have some advantages over simple discussion board. For one there is the chat. As well I like how you can link URL’s. I will try linking to my homepage.
Student B: Yes, this board is quick and easy. Your page is crystal clear and I like the options – very easy to use.
Student C: The format for adding links and attachments is good. I tried switching to browser view but it wouldn’t load.
Student B: I’m trying out a few features. One is to send an attachment which seems easy enough (can they be sent in any format? (doc, pdf, etc.)
Student D: Does this board seem a little cluttered or is it just me? Perhaps those of you who are familiar with this program can tell me what the best pops and whistles are.
Student E: It is not your imagination. This product’s screen is VERY cluttered.
Student F: Just trying this out, but like others, I find it unnecessarily cluttered, and can’t quite understand why the rate link is there. What am I to rate??
Student G: What is the rate link about? Is it to rate the message?
Student I: I thought I’d try out polling a question and see how it could be incorporated in a needs assessment. If anyone is interested, attached is the polling zip file (37Kb).
Student G: I believe my polling file is to big. I will try again later with another one.
Student J: Seems to be an extensive package, but the menu bar is a little hard to follow.
Student E: It is nice to see that you can add an image to the posting without having to use HTML.
Student A: My first attempt to attach a picture failed due to size restrictions. The photo was 407 Kb so that explains why it failed. Took some time but this picture should be around 100 kb.

Product Two. Two weeks after the first discussion, the second product (WWWBoard) was introduced.

JB, Instructor: Welcome now to this modified version of the usual MDDE discussion board. We invite you to place some test postings on the board, commenting on some of its features.
Student E: Okay; this is not as cluttered as the other format.
Student K: What a relief to move back to a screen format that clearly differentiates action area (i.e., post reply, link, search, etc.) from postings area. I found the initial view of Anyboard quite confusing (even intimidating!). Student/smiley faces are nice, but there’s way too much happening on the screen for a new user. Maybe a bit of graphic design would help?
Student F: Yes, this is cleaner and clearer.
Student C: I like the easy link to external sites.
Student K: I think the search function is very handy – saves having to note name, posting date or even print material that you want to refer to from time to time.
Student L: This is a much more ‘friendly’ screen to read. I find it a bit irritating though that you can’t collapse the response postings into the first post of each main thread.
Student E: I don’t see enough here to make me change. However, we’ll see if the 10-thread page feature is a help or a hindrance.
Student D: This format looks like the old WebCT bulletin board. It always worked well, but I like the multiple page access. Very convenient and clear. Generally a nice board but can one create more than one link per message??

Summary Discussion. Three weeks after the second discussion, a final discussion was introduced.

JB, Instructor: Now that you have each experienced a wide array of these tools, which in general do you prefer – an online conference containing many optional features, or one with relatively few features? Your opinions on this topic will be useful in the context of this particular course, and in decisions about future web site policy.
Student E: I am moving to the position of favouring “lean and mean.” I accomplish what I need with stand-alone tools: email and file attachments; ftp; Internet audio; course discussion boards. I use what I need when I need it, without having to load some cumbersome or screen-crowding thing.
Student G: I prefer an online conference containing relatively few features. I get technically frustrated very easily, so for me the simpler and easier the better. Features I feel should be included in an online conference are email, file transfer and a method to replay the discussion to a group or to an individual. I found the first method (Anyboard) to have a distracting font – too large and colorful. The page layout was also not as organized as the second method (WWWBooard).
Student H: I like email for asynchronous, private messaging, but prefer a conference board when trying to follow a discussion by a group. I like the simplicity of WWWBoard. Black text on a white background is easy to read and provides a no-nonsense, get down to business arena for discussion. I am not a fan of screen colors and clutter when it is the text that is important. I remember using another method, which was bizarre! The text was brown on a mustard yellow background. I couldn’t even read it. Many of us complained, and the colors were changed, along with the structuring of the threads, which was also difficult to follow.
Student C: The beauty of an exercise such as this is the ability to “play with the toys,” trying out a variety of programs that may or may not add to the distance learning experience. The danger of this playing around can be going for the “latest” techno tools for the sake of using something neat when a simpler approach would work as well or even better. If I were teaching Math or Art online, a whiteboard and synchronous conferencing tool would definitely be a plus (in my opinion) whereas, if my students and I are critiquing a literature text, a simple, single function conferencing board would be more than adequate for the job.
Student F: I agree with your points re: the beauty and the danger of these techno-toys/tools. I also agree about matching the tools to the task. I must add that, for me, an audio component adds a richness I hadn’t expected. I suppose it is partly because most of my day (work and general living) is spent in a second language environment, so being able to discuss the points/ content in my mother tongue is worth having to rearrange my schedule to be online when everyone else is. So in looking for a package, I’d start with one that offered an asynchronous text tool with synchronous audio capabilities.
Student M: I personally prefer the lean and mean approach to technology, a simple program.
Student N: I prefer the fully featured approach where one has the ability to use a variety of tools depending upon the task being performed. I would like the option of using audio and video with someone who is capable of using this feature and then using only text with another person who may have bandwidth problems that cause audio and video conferencing to be an annoyance. I also prefer to use software that comes with the operating system. As a training consultant, I do not like to recommend software that is relatively little used by individuals just because I like the program. I am more likely to do my best by using software that is already in use by the majority of my clients.
Student I: I prefer the flexibility of a full-featured system. I tend to opt for utilities from established vendors – higher likelihood that they will maintain their service and that a significant portion of my clients will have heard of it. More and more, I find value in a system that also tracks users’ logins, time online and what activities they were engaged in. This feature normally only comes with a fully featured system.

Analysis of Comments

The students were classified in terms of whether or not their response(s) clearly favoured one of the two methods above the other. Eight of the 14 respondents indicated a clear preference for the “lean and mean” method, while two preferred the more complex method. The addition of less clear responses raises this comparison to nine versus four. The students who prefer the more complex method tend to be those with a particular fascination for the online technologies in their own right, and who enjoy exploring them. The majority of students, however, indicates that they lack the time and patience to explore software that contains a wide range of features, which they may not need for the specific online activities at hand.

Conclusions

The conclusions of this product comparison coincide with those of similar software preference studies conducted in the same Athabasca University graduate programme during the past three years. The current study has two main implications. Firstly, it confirms that distance education students do not wish to use fully featured online collaborative tools when simple tools will suffice. Secondly, it suggests that the ratings system used in this particular series of evaluation reports to date should be modified so that it no longer includes an “overall rating” by which each software product is directly compared with others. Since reviewers’ overall ratings are usually based on the comprehensiveness of a product, they may not reflect the main purpose of these evaluations at all – i.e., to identify online approaches that serve distance education students’ needs simply and directly, and are popular with them because they do so. The greater virtue of such studies lies in the extent to which they generate a comparative checklist of product features for the benefit of those who know exactly which features they need for specific purposes.

Acknowledgements

To my colleague Dr. Patrick Fahy, co-designer and co-instructor of Athabasca University’s graduate course on Educational Conferencing (MDDE661); and to the course’s Winter 2002 students whose opinions made this report possible.

The next report in this series will review dual-platform collaborative software (PC and Mac).

N.B. Owing to the speed with which Web addresses are changed, the online references cited in this report may be outdated. They can be checked at the Athabasca University software evaluation site: cde.athabascau.ca/softeval/. Italicised product names in this report can be assumed to be registered trademarks.
JPB. Series Editor, Technical Notes