Abstract

Learning statistics can be challenging for many students, due to their inability to engage in statistical reasoning and application of techniques. This challenge becomes compounded in online learning contexts where students are spatially and temporally separated from the teacher. This paper describes and explains a case of theory-driven interventions designed to enhance the learning experiences of students enrolled in two similar business statistics units, one for undergraduate and the other for postgraduate programs. The paper based its claims primarily on the analysis of data from a student evaluation of teaching survey. This study affirmed the importance of a pedagogy-first approach. It argued that the interventions, which were effective in enhancing the student learning experience, were underpinned by a robust pedagogical analysis of the teaching and learning issues using both constructive alignment and transactional distance theory lenses.

Keywords: constructive alignment, online learning, pedagogy, statistics teaching, transactional distance

Enhancing Online Teaching of Business Statistics: A Pedagogy Before Technology Approach

Business statistics is one of the fundamental subjects in business-related degrees. The purpose of this subject is to equip students with skills of analysing data in the context of business. Thus, a key goal in teaching statistics is to equip students with the ability to use data to reason appropriately and apply the right techniques in solving real-world business problems. However, learning statistics can be challenging for many students, due to their inability to engage in statistical reasoning and application of techniques (Garfield & Ben-Zvi, 2007; Selvanathan & Selvanathan 1998; Tishkovskaya & Lancaster, 2012). This challenge is compounded in online learning contexts where students are spatially and temporally separated from the teacher (Mills & Raju, 2011). This paper outlines a case which exemplified these challenges obtained and described the interventions that were employed to address them. This study based its claims primarily on the analysis of data from a student evaluation of teaching survey. The aim was to describe and explain theory-driven interventions that were designed to enhance the learning experiences of students enrolled in two similar business statistics units, one for undergraduate and the other for postgraduate programs. A key thesis of this paper was that strategic and meaningful change in teaching and learning happens when a pedagogy-first approach to technology-based learning and teaching interventions is taken. In particular, it argued for a thorough front-end pedagogical analysis of the teaching learning issues if the ensuing changes are to be sustainable and effective.

The outline of this paper is as follows: first, the institutional context is described, followed by analysis of the teaching and learning challenges. Then, the interventions are explained, followed by an evaluation of their impact. The paper concludes with discussion and general recommendations.

Context

The developmental work of this study took place within a regional Australian university with centres or campuses both within and outside the state. As a dual sector institution, it offers vocational education and training (VET) as well as higher education (HE) courses. The majority of the HE enrolments were external, meaning over 50% students studied off-campus, primarily through online delivery mode. Within the HE Faculty of Arts and Society, the business and accounting discipline has run accounting and business courses. Most courses for internally enrolled students were offered from the main campus with some students in accounting and business also enrolled in the interstate campus. There were also external students, meaning most units had these three groups of students—main campus, interstate, and external.

Across the business and accounting discipline, there were two similar foundational business statistics units, one for undergraduates (UG unit) and the other for postgraduate students (PG unit). These units provided core quantitative skills, particularly the use of data to make business decisions, to first-year students as well as non-cognate postgraduate students in business and accounting, respectively. Consequently, the units were usually taken by a high number of students across the two campuses and the off-campus cohort.

Over the years these units have been taught by different lecturers, some of them sessional staff, with a high turnover. Consequently, there had not been any major revisions and there were issues related to teaching and learning. The second author had lectured in these units since 2018, while the first author was an education developer familiar with previous issues related to the teaching and learning of these units and other units with quantitative content across the university. These challenges had manifested themselves through poor student performance and satisfaction as well as staff frustration. In the end of the semester unit evaluation survey, students revealed a substantially lower satisfaction towards the structure of the unit, the appropriateness of the assessment activities in relation to the unit learning outcomes, and the unit’s ability to meet students’ expectations (see Table 1 for further information). Consequently, the overall unit performance, in terms of the student satisfaction, raised serious concerns (see Figure 5).

The Problem

As outlined above, the nature of the business statistics units and their learning and teaching context had posed a set of pedagogical challenges to the lecturers who taught them, and dissatisfaction to some students. In this section, to gain a more nuanced understanding, these challenges are problematised by way of a theoretical analyses based on constructive alignment model and transactional distance theory.

Constructive Alignment Issues

Appropriately designed curriculum is central to effective teaching and learning in any unit of study. The outcomes-based education approach, which the Australian system follows, uses constructive alignment, which refers to the deliberate alignment of intended learning outcomes (ILOs), teaching and learning activities (TLAs), and assessment tasks (ATs) to help learners construct meaning. This approach, key to effectiveness, is intended to ensure that learners engage with TLAs that optimise their chances of achieving the intended learning outcomes, demonstrable through appropriate assessment (Biggs, 2003, 2011).

The starting point to ensuring constructive alignment is to have clear and observable ILOs, as these define the curriculum for a particular unit of study. The TLAs and assessment, on the other hand, reflect the pedagogy employed to facilitate students’ achievement of the ILOs. Boitshwarelo and Vemuri (2017) argued for a closer curriculum-pedagogy connection, pointing out that the way the curriculum is designed can limit or aid good pedagogy. Using their curriculum-pedagogy alignment framework, they argued that learning outcomes represented different knowledge types, and that effectiveness was achieved when the attendant pedagogical approaches, strategies, and assessment types were closely aligned. Figure 1 presents this framework (Boitshwarelo & Vemuri 2017).

Figure 1

Curriculum-Pedagogy Alignment Framework

From “Conceptualizing strategic alignment between curriculum and pedagogy through a learning design framework,” by B. Boitshwarelo, & R. Vemuri, 2017, International Journal for Academic Development, 22(4), p. 285 Copyright [2017] by Taylor & Francis Group. Reprinted with permission.

In the case of the units in question, while the ILOs were deemed appropriate for their respective levels and contents, it was the view of the lecturer and previous students’ feedback that assessment did not adequately align with the key learning outcomes related to application of statistical concepts. In particular, the ILO, ‘to identify, discuss, and apply quantitative and qualitative tools and methods commonly used in decision making in private and public sector organisations,’ required that, among other things, students develop procedural and contextualised knowledge. This, in turn, required practice-based and situated learning (problem-based) TLAs respectively and creative use of narrative media or, even more suitably, interactive and/or adaptive forms (refer to Figure 1). However, in contrast, there was a lack of opportunity for students to apply their understanding of statistical concepts. This could partly be because of the fact that the assessment tasks did not require such engagement with the material. Therefore, while the TLAs, perhaps, aligned with the documented assessment specifications, the actual assessment task did not help students to achieve this particular ILO, meaning some of the curriculum expectations were not adequately met.

Feedback from the students’ evaluation of teaching survey revealed that they felt that the assignment was poorly structured, and it was more suitable for a closed book exam than for a problem-solving assignment. They further stated that the assignment questions did not prompt students to practically apply statistical theories to solve real-world problems. Consequently, students did not receive the training they should have in order to increase their employability in the current labour market that called for hands-on experience with analysing data.

In general, good practice in teaching statistics is to teach theoretical concepts and procedures as well as let students analyse real-world data using data analysis software and interpret results. However, this aspect was lacking in the unit and, in particular, no data analysis software was incorporated in teaching the two units. The qualitative feedback from students clearly indicated that they would like to learn how to solve business-related problems and how to use statistical software/programs to gain hands-on data analysis experience. After an analysis of the learning outcomes, unit content, previous actual assessment tasks, and student evaluation reports of previous offerings, it was realised that the unit did not adequately facilitate the learning of appropriate statistical reasoning. It also lacked application-based opportunities to solve real-world business problems. The predominant pedagogical approaches used in this unit were either expository or practice-based which did not do full justice to the contextual knowledge intentions. Similarly, the assessment was largely of an objective and/or performance nature and not authentic as expected of contextualised knowledge. As illustrated in Figure 2 through the annotation, the challenge was to bring that into alignment.

Figure 2

Constructive Misalignment and Realignment

Adapted from “Conceptualizing strategic alignment between curriculum and pedagogy through a learning design framework,” by B. Boitshwarelo, & Vemuri, R, 2017, International Journal for Academic Development, 22(4), p. 285. Copyright [2017] by Taylor & Francis Group. Adapted with permission.

Transactional Distance Issues

As described earlier in the Context section, there were a significant number of students who studied externally, primarily online, and traditionally known as distance education students. The idea of transactional distance has been used in distance education to describe the psychological or communication gap that separates the teacher and the learner (Moore, 1991; Moore & Kearsley, 1996). A student, separated by physical distance from their education provider and/or teacher, experiences a bigger transactional distance, necessitating the use of strategies to bridge or reduce that gap.

The key concepts in transactional distance theory are dialogue, structure, and autonomy (Moore, 1991; Moore 1993; Moore and Kearsley, 1996). Dialogue primarily refers to the interaction between a learner and their teacher during a learning experience, whether synchronous or asynchronous. Other interactions are also important such as with other learners (Quong et al., 2018); Structure describes the rigidity or flexibility of the learning environment design in terms of such elements as the learning outcomes, learning activities, interactions, and assessments. Some learning environments are more democratic and others more prescriptive in nature (e.g., with a defined path or a tightly controlled sequence of learning events). The third element, autonomy, refers to the level of self-directedness or independence a student has in controlling their learning.

Moore and Kearsley (1996) submitted that more dialogue in a distance learning environment reduces transactional distance. However, if an environment is highly structured at the expense of dialogue, then transactional distance may increase, depending on the nature of the content and learning outcomes. High structure and low dialogue necessitate high learner autonomy due to lessened communication and a more prescribed learning experience. As a general principle, it seems, blending highly structured learning environments with increased dialogue reduces transactional distance (Benson & Samarawickrema, 2009). However, this is all dependent on the nature of the learning outcomes, the content, and the characteristics of the learner amongst other things (Moore, 2004) with some subject matter by their very nature requiring greater dialogue than others.

In the case of these units, which required the use of problem-solving procedures, the structure of the learning experiences was of key importance. A key deficiency in the unit resources was the absence of opportunity for students to experience the demonstration of statistical procedures and techniques in an engaging and interactive way; most of the materials were presented in a static text-format, such as PowerPoint slides that described the problem and presented a complete solution already worked out. This appeared to be a critical issue among external students. For example, in the qualitative feedback of the student evaluation survey, students noted their dissatisfaction about not clearly demonstrating the steps involved in solving problems, and as a result, how they relied more on YouTube videos than on their recorded lectures. The lecturers in the discipline were unable to record step-by-step demonstrations mainly because the classroom-based Camtasia technology at the university did not allow for recording any external writing, such as on a whiteboard. Therefore, the external students who listened to lecture recordings could not see the extra explanations the lecturer provided on the whiteboard. This matter had been raised by a number of students in their feedback and they expressed their frustration and dissatisfaction over this matter. External students were concerned that they were not receiving the same level of resources as were face-to-face students. Thus, the perceived lack of structure, particularly by external students, widened the transactional distance.

The Interventions and Related Outcomes

To address the pedagogical issues outlined above, appropriate learning design and technological interventions were done. The interventions had a two-fold purpose.

1. To improve constructive alignment through:
• introducing an application-based approach teaching and learning approach, using EXCEL for statistical problem solving and decision making
• improving assessment items and aligning them with the contextualised knowledge of the relevant learning outcomes as well as with application-based, problem-solving learning activities, and thus ensuring that there is assessment, for and of, applied skills
2. To bridge the transactional distance through developing and making available step-by-step demonstration of quantitative problem-solving procedures.

These initiatives are detailed below explaining how they enhanced student engagement and learning of statistics. To evaluate the impact of the interventions, customised student evaluation of teaching survey questions were developed in addition to the regular ones.

Introducing an Application-Based Approach

Past student feedback revealed that the way the units were taught did not provide any applications-based training to enhance students’ employability skills. Taking this feedback into consideration, when the units were redeveloped by the second author, she introduced an applications-based approach, and incorporated Excel as a teaching tool. The attraction of Excel is that it did not require licence arrangements for students and, most importantly, was available at every workplace. The problems set for Excel applications were expected to provide students with hands-on experience in analysing real-world datasets and using statistical skills required in the future at their workplace to solve business problems and for decision making. The introduction of Excel was well received by students and their qualitative feedback indicated that they found Excel-based activities useful in enhancing their data analysis skills (Figure 3).

Figure 3

The Excel Activities Were Useful to Gain Data Analysis Skills

Improving Assessment Tasks

The structure of the written assignment was revised so that students were required to analyse a real-world dataset using Excel data analysis tools introduced in the unit. While estimating results for the analysis required the students to use the Excel knowledge they gained from the unit, writing up the findings required them to interpret the results using the statistical techniques they learned. To develop students’ interest and sense of necessity in learning Excel data analysis skills, Excel activities were integrated through homework problems and later into assessment tasks. In this way, students gained hands-on experience in both theoretical knowledge and application of statistical concepts. By way of promoting constructive alignment (Biggs & Tang, 2011), a question focusing on the interpretation of Excel generated results was also included in the final examination.

Step-By-Step Demonstration of Problem-Solving Procedures

The need to demonstrate procedures for external students was two-fold: first, to actually demonstrate the problem-solving procedures for students, and second, to use Excel for analysis.

Demonstrating the step-by-step approach to solve statistical problems to face-to-face students was not an issue, as they could easily follow synchronously in class, and even ask questions. However, the key challenge related to reducing the transactional distance and providing equivalent and/or similar learning experience for external students. Recording demonstration lectures was the obvious solution, which has been done elsewhere. However, solutions that are common elsewhere (e.g., 360 Lecture Capture or using SurfacePro notebooks) were not plausible due to a lack of funding. The authors explored other options possible with the existing technology to create demonstration lectures and so allow the external students to enjoy a learning experience similar to that of the face-to-face students.

The solution was to record the lecture using the Blackboard Ultra Collaborate platform in a Collaborate-enabled lecture room. In this environment, a smart pen was used to write on the whiteboard, which was then captured by the projector and displayed as a live annotation to students. This approach had not been used before at the university; some experimenting and setting equipment was necessary to make it work. Students followed the lecture as the procedures were demonstrated either synchronously or through recordings. In addition to lectures, weekly online tutorials were conducted using this same method. Feedback from one external student, who previously attended face-to-face sessions whenever she could, revealed that for the first time, she felt that external students received the same resources as face-to-face students. This improvement was further evident in feedback from a student who withdrew from the UG unit in 2017 and re-enrolled in 2018.

I would like to say that I am very happy with the UG unit this year. The recordings in the learning area for each week were VERY HELPFUL. The lecturer who recorded these sessions explained the concepts in a way that really helped me understand. When I had attempted to take this unit before I dropped out before census date because I had such difficulties with the learning materials. Very, very happy with the unit this time around.

Additionally, face-to-face students, particularly those with non-English speaking background, revealed that these recordings provided a valuable reference if they could not grasp some content during the lecture. As evident from student feedback in Figure 4, students regarded the new form of lecture recordings as big step forward in their learning experience.

Similarly, the application-based approach using Excel to teach face-to-face students was not challenging as they can always approach the lecturer for help. However, once again the challenge was to enhance the consistency of the Excel training across all campuses and external students. The solution to this problem was to record Excel videos and upload them into learning management system as part of the learning materials.

Figure 4

Weekly Recordings with Annotated Notes Helped Me Understand Statistical Procedures

Concomitant with creating the demonstration videos was improving the overall organisation of the unit, which had been a concern. Doing so improved student satisfaction. As can be seen in Tables 1 and 2, before introducing the changes, (i.e., in 2016 and 2017) the student evaluation of teaching survey question “this unit was well organized” was significantly low, particularly for both units. In fact, until 2017, the unit rating for this question was noticeably lower than that of the average university rating for the same question.

The interventions described above have led to sustainable impact over a number of offerings of the two units, across campuses, with different lecturers, and for both internally and externally enrolled students. Table 1 shows improvements across three evaluation items for both the UG and PG unit, even exceeding the institutional average, particularly for the PG unit. The high performance of the two units was sustained even during COVID-19 period.

Table 1

Change in Unit Performance After Applying the Changes (Out of Four)

Note. For comparison, 2019 and 2021 ratings (which were out of 7) were converted to align with prior ratings out of 4. SS refers to the summer semester. Standard deviations are given in parenthesis.

Similarly, Figure 5 shows that overall student satisfaction improved significantly as a result of the interventions.

Figure 5

Overall Performance of Units After Applying Interventions

Discussion and Conclusions

This paper has given an account of learning design and technological intervention into business statistics units at a regional university in Australia, with a mix of face-to-face and externally enrolled students. This intervention responded to the epistemological nature of the units, and their requirement to facilitate procedural and contextualised knowledge. The intervention, which has been effective in enhancing the student learning experience, was underpinned by a robust pedagogical analysis of teaching and learning issues. The pedagogical diagnosis revealed constructive alignment issues as well as transactional distance challenges. Thus, while the interventions were largely technological, the underpinning intention was, first, to refine the constructive alignment among ILOs, TLAs, and assessment especially as it related to applying business statistics methods and techniques to solve authentic problems. A second intention was to reduce the transactional distance by improving the structure and dialogue of the units through recording a series of demonstrational videos and improving the units’ overall organisation.

The technological interventions were not novel, however we believe they were fit for purpose as they were a result of a strategic approach that took a pedagogy-first strategy. We propose therefore that learning and teaching issues are often complex and a technology-first approach to solving such problems is seldom efficient as it tends to find or manufacture problems that fit the vendor-driven technology solutions. From a curriculum perspective, the constructive alignment lens enhanced effectiveness by helping realign curriculum intentions and pedagogical actions (Boitshwarelo & Vemuri, 2017). It was apparent that refining the alignment of assessment with the intended learning outcomes also necessitated changes in the teaching and learning approach, and consequently the technologies used and how.

Good constructive alignment ensures a robust curriculum-pedagogy connection which in turn needs robust learning environments to mediate effective learning. For online distance students, the learning environment must effectively bridge the transactional distance. What is to be learned, and who are the learners (and their level of autonomy) determines the nature and extent of structure of the learning environment and the level of dialogue. In the case of these statistics units, the external students were of particular interest and needed to learn statistical procedures and develop skills in application-based problem solving thus requiring some structured learning in the form of well sequenced demonstrational videos. Rather serendipitously, this intervention became highly efficacious with the onset of COVID-19 shutdowns in March 2020 when internally enrolled students also had to study externally (online).

While these conclusions are based on a specific case and based almost solely on student evaluation of teaching data, the principles of pedagogy-based learning design and interventions are universally applicable and have been illustrated through this case. Constructive alignment and transactional distance theory are just examples of the frameworks to analyse learning problems and were perhaps the most suitable in this context.

Acknowledgements

The authors wish to acknowledge funds received under the COVID-19 Supplementary Funding Pool Scheme (CSFPS) from Charles Darwin University to undertake this research.

This paper was endorsed by the Institutional Chair of the Ethics Committee as low risk and appropriate as an evaluation or practice piece rather than a fully-fledged report on research on humans, hence its publication in the Field Notes section.

References

Benson, R., & Samarawickrema, G. (2009). Addressing the context of e‐learning: Using transactional distance theory to inform design. Distance Education, (1), 5-21. https://doi.org/10.1080/01587910902845972

Biggs, J. (2003). Aligning teaching for constructing learning. Higher Education Academy, 1(4), 1-4. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=1aa6531a3ec223588e907c7180c307047b84b00d

Biggs, J., & Tang, C. (2011). Teaching for quality learning at university (4th ed.). McGraw-Hill Education.

Boitshwarelo, B., & Vemuri, R. (2017). Conceptualizing strategic alignment between curriculum and pedagogy through a learning design framework. International Journal for Academic Development, 22(4), 278-92. https://doi.org/10.1080/1360144X.2017.1367298

Garfield, J., & Ben‐Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistical Review, 75(3), 372-396. https://doi.org/10.1111/j.1751-5823.2007.00029.x

Mills, J. D., & Raju, D. (2011). Teaching statistics online: A decade’s review of the literature about what works. Journal of Statistics Education, 19(2). https://doi.org/10.1080/10691898.2011.11889613

Moore, M. G. (1991). Editorial: Distance education theory. The American Journal of Distance Education 5(3), 1-6. https://doi.org/10.1080/08923649109526758

Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22-38). Routledge.

Moore, M. G. (2004). Editorial: Constructivists, don’t blame the tools! The American Journal of Distance Education, 18(2), 67-72. https://doi.org/10.1207/s15389286ajde1802_1

Moore, M. G., & Kearsley, G. G. (1996). Distance education: A system view. Wadsworth.

Quong, J., Snider, S. L., & Early, J. (2018). Reducing transactional distance in online and blended courses through the use of a closed social media platform. Journal of Educational Technology Systems, 47(1), 79-100. https://doi.org/10.1177/0047239518766

Selvanathan E. A., & Selvanathan, S. (1998). Teaching statistics to business students: Making it a success. In 5th International Conference of Teaching Statistics, Conference Proceedings. Singapore.

Tishkovskaya, S., & Lancaster, G. A. (2012). Statistical education in the 21st century: A review of challenges, teaching innovations and strategies for reform. Journal of Statistics Education, 20(2). https://doi.org/10.1080/10691898.2012.11889641

Enhancing Online Teaching of Business Statistics: A Pedagogy Before Technology Approach by Bopelo Boitshwarelo and Maneka Jayasinghe is licensed under a Creative Commons Attribution 4.0 International License.