International Review of Research in Open and Distributed Learning

Volume 19, Number 2

April - 2018

Applying Learning Analytics to Explore the Effects of Motivation on Online Students' Reading Behavioral Patterns

Jerry Chih-Yuan Sun, Che-Tsun Lin, and Chien Chou
National Chiao Tung University, Taiwan

Abstract

This study aims to apply a sequential analysis to explore the effect of learning motivation on online reading behavioral patterns. The study's participants consisted of 160 graduate students who were classified into three group types: low reading duration with low motivation, low reading duration with high motivation, and high reading duration based on a second-order cluster analysis. After performing a sequential analysis, this study reveals that highly motivated students exhibited a relatively serious reading pattern in a multi-tasking learning environment, and that online reading duration was a significant indicator of motivation in taking an online course. Finally, recommendations were provided to instructors and researchers based on the results of the study.

Keywords: learning analytics, motivation, sequential analysis, online learning, behavioral pattern

Introduction and Background

In recent years, learning analytics has been a hot topic in information technology. It is a technology that allows for the analysis of highly complex information and deals with the professional storage, management, analysis, and imaging of such complex information (Chen, Chiang, & Storey, 2012). Furthermore, the technology of learning analytics involves administration, research, teaching, learning, and support resources (Long & Siemens, 2011). Led by the trend in the recent innovation of information technology, learning analytics in education is an application which gathers large amounts of data to process, collect, measure, and analyze the content or behavior of students generated in the online learning process. The information obtained from learning analytics can allow us to gain a more in-depth understanding and determine the best methods in learning and teaching. Learning analytics was listed on the NMC Horizon Report from 2011 to 2014 as an innovative and potential agenda item in higher education in recent years, underscoring its importance in education research (Johnson, Adams, & Cummins, 2012; Johnson Adams, Cummins, Estrada, Freeman, & Ludgate, 2013; Johnson, Becker, Estrada, & Freeman, 2014).

Learning analytics was formed through two coinciding trends: the increase in the application of virtual learning environments in education, and the expansion of data mining techniques used in business organizations (Agudo-Peregrina, Iglesias-Pradas, Conde-González, & Hernández-García, 2014). Based on this viewpoint, Agudo-Peregrina et al. (2014) believed that learning analytics is often regarded as educational data mining (EDM). The International Educational Data Mining Society defines EDM as follows: Educational Data Mining is an emerging discipline, concerned with developing methods for exploring the unique and increasingly large-scale data that come from educational settings, and using those methods to better understand students and the settings in which they learn in. The methods utilized by EDM include statistics, graphics technology, social network analysis (SNA), online analytical processes (OLAP), regression, cluster analysis, correlation analysis, and text mining (Black, Dawson, & Priem, 2008). Macfadyen and Dawson (2010) considered that learning behavior tracking logs, such as on-line hours, the number of interaction in discussion forums, and the duration and time required to complete the online test, were meaningful data for analysis. This study investigated learners' reading behavior in an online context through a behavioral sequence analysis, which is a technique of learning analytics.

E-learning is a teaching format that combines the application of education and information technology, and which allows students and instructors to engage in learning activities at different temporal and spatial distances (Raab, Ellis, & Abdon, 2001). However, after reviewing students' online activities such as login frequency, frequency of accessing course materials, number of bulletin board messages posted, and number of synchronous discussions attended, Hung and Zhang (2008) found that the majority of the students tended to demonstrate passive learning behavioral patterns. In other words, most online learning activities involved reading of materials rather than posting messages or participating in synchronous discussions. As a result, Chen and Jang (2010) proposed to incorporate learning motivation into learning analytics to further explore the relationship between student's motivation, online learning behavior, and learning achievement. Learning motivation is a psychological variable that significantly impacts learning behavior, engagement, effort, and learning achievement (Bandura, 1977; Schunk, Meece, & Pintrich, 2013; Sun & Rueda, 2012). Learning motivation affects the students' choice of learning tasks, effort, and degree of perseverance that is invested in certain tasks, as well as the mental state of the student (Schunk et al., 2013). Deci and Ryan (1985) defined motivation both in terms of intrinsic motivation and extrinsic motivation according to the self-determination theory. Empirical studies have pointed out that both intrinsic motivation and extrinsic motivation influence learning attitude, self-learning efficacy, learning achievement, and overall acceptance of e-learning (Saadé, He, & Kira, 2007; Tseng & Tsai, 2010; Yoo, Han, & Huang, 2012).

Engagement is a cognitive process in learning tasks and an indicator of active participation, emotion, and other aspects in learning (Pellas, 2014). In online learning, online time spent and other behavior records represent the behavior engagement of the student, such as the frequency in which they log in and out of their online platforms, and the frequency in which they open and close reading materials, download and upload exercises, and post in internal discussion forums. "Online time" appears to have a positive impact on online learning achievement (Hu, Lo, & Shih, 2014; Ma, Han, Yang, & Cheng, 2015; Macfadyen & Dawson, 2010). Skinner, Furrer, Marchand, and Kindermann (2008) categorized learning engagement into behavioral engagement and emotional engagement. Behavioral engagement is defined as a continuous behavior that includes the effort spent on learning, attention, and continuity of behavior from the beginning to the execution of a learning task. Emotional engagement includes feelings such as interest, enjoyment, and frustration from defeat. Related studies have pointed out that learning motivation and user engagement levels have an impact on the actual learning behavior (Kong, Kwok, & Fang, 2012; Wang & Lin, 2007). As seen from the previous study results, both learning motivation and user engagement have a significant impact on the behavior process in online learning. However, existing research related to online learning motivation has primarily focused on behavioral aspects, such as outcomes and performance determined by one's motivation "after learning," and on the psychological aspect, such as emotions and one's level of satisfaction. Hou (2012b) suggested that gamers' behavioral patterns in educational games include collaborative learning and problem-solving. Existing studies have applied behavioral sequence analysis to investigate users' behavioral patterns during social interactions in an online learning context (Hou, 2012a; Hou & Wu, 2011; Sun, Kuo, Hou, & Lin, 2017), and behavioral patterns of knowledge construction from a cognitive perspective (Hou & Wu, 2011; Yang, Li, Guo, & Li, 2015). Furthermore, behavioral sequence analysis had been utilized in an online two-tier test strategy to diagnose students' misconceptions and to examine the pattern of students' test-learning behaviors (Yang, Chen, & Hwang, 2015). Lai and Hwang (2015) discovered that, compared to low-achieving students, high-achieving students tended to exhibit patterns of behaviors in which they repeatedly searched for additional learning materials, often attempted an input value, and observed the corresponding outcome. Chen and Jang (2010) discovered that there were no significant correlations between learning motivation and weekly login frequency, and the number of clicks on the online platform. However, changes in behavior during the learning process, as well as the relationship between learning motivation and behavioral patterns in an online environment still await further exploration, with extra attention needed to assess the differences among high and low motivated students. Therefore, the present study refers to the recommendation of Chen and Jang (2010) to combine learning motivation and user engagement to investigate the behavior pattern in online learning. This study aims to understand and compare the reading behavior patterns that students with different levels of motivation exhibit in order to provide recommendations to instructors and researchers for future use or design of online learning courses.

The sequential analysis in Bakeman (1997) is often used in peer discussions of the learning process, which includes factors such as behavior, the knowledge construction process, and problem solving skills (Eryilmaz, Chiu, Thoms, Mary, & Kim, 2014; Hou, 2012a, 2012b; Hou & Wu, 2011; Liu, Cheng, & Huang, 2011). However, aside from teacher-student and peer interactions, online learning also includes interactions between students and e-materials (Moore, 1989). Gil-Flores, Torres-Gordillo, and Perera-Rodríguez (2012) argue that the digital culture resulting from ubiquitous information communication technology has a significant impact on the learning environment, and digital reading has become a crucial skill that a student must possess in the contemporary learning environment. Liu (2005) revealed that the digital reading environment is a topic deserving of an in-depth study as the digital environment continues to evolve. In recent years, researchers have started applying sequential analysis to investigate the interactive process happening on the discussion platform among students (Hou, 2012a; Hou & Wu, 2011). However, in terms of online reading behavioral patterns (i.e., the way students read online course materials), further studies are still needed. Therefore, this study used a sequential analysis to investigate and compare the differences in behavior patterns of students in reading online learning materials and in assessing their learning motivation. The research questions posed to the participants were as follows:

1. What are the frequencies of different types of online reading behavior?
2. What are the differences in the reading behavioral pattern of online students with different type?

Methods

Research Procedures and Participants

This study adopted the cross-sectional survey research method. Cross-sectional survey research refers to the investigation of data collected at a specific point in time or a short period of time (Ann, 2006). This study adopted a convenient sampling method. The participants were graduate students from three national universities in Taiwan who were invited to participate in this research study via campus bulletins, email, and community website marketing. The study design is shown in Figure 1. Participants read the digital learning materials from each of the four units and then took a test. If their test scores were lower than 80, the participants could re-read the learning materials and try to pass the same test again. After passing the test, the participants were prompted to complete a post-test questionnaire that measured their levels of learning motivation. The research ethics course designed for the study aimed to ensure a mastery of basic knowledge. Therefore, we defined the passing threshold to be 80% for all unit tests and permitted learners to take the tests repeatedly. The participants were required to finish the entire course online. The system recorded participants' online learning behaviors during the learning and testing processes. The open period for the online course information collection was 2 months. After the researcher informed the participants that the online course would be open for 2 months, the participants were free to control their own study pace without interference from the researcher. Out of the 249 students enrolled, 170 completed the course. After the invalid data was eliminated in the questionnaire review session, a total of 160 students had valid information; among them, 64 were female and 96 were male. The average age was 23.46 and the standard deviation for age was 2.46 years. The distribution of participants' schools and grade levels is presented in Figure 2. There were more participants from the schools of engineering and computer science (48%) than from other departments, and the majority of the participants were first-year master's students (57%).

Figure 1. Experimental design.

Figure 2. Distribution chart of participants' school and grade level.

Instruments

This study used the digital learning motivation scale proposed by Yoo et al. (2012) that measured employees' acceptance levels in digital learning and further categorized them into six sub-factors. There were 20 items in the original scale after translation into Chinese. Among the six sub-factors, "social influence" (three items) did not fit the online learning environment, as its online course did not include any interactive function among the teacher and students or among peers, and that items such as "(i)n general, the organization has supported the use of e-learning" did not fit the context of this study, in which our participants were asked to carry out the study autonomously, and was therefore eliminated. The scale used in this study was a 5-point Likert scale with 1 representing strongly disagree - no confidence and 5 representing strongly agree - full confidence. A confirmatory factor analysis revealed that the factor loadings of three items were less than.40. Therefore, these three items were removed from the scale. A further test after excluding the unqualified items yielded good fit indices (χ2 = 127.05, df = 67, p < .001, CFI = 0.96, RMSEA = 0.08, SRMR = 0.05), indicating good construct validity. The final scale contained 14 items. The five sub-factors were effort expectancy, attitude towards e-learning, anxiety, performance expectancy, and facilitating conditions; and their corresponding Cronbach's α values were .81, .84, .89, .67, and .80, respectively. The total Cronbach's α value for the 20 items was .88, which was consistent with the validity standard of .70 suggested by (Nunnally & Bernstein, 1994). The factor loadings of effort expectancy (three items) ranged from .59 to .81 (e.g., "So far, I think the e-learning platform is easy to use"). The factor loadings of attitude towards e-learning (four items) ranged from .55 to .80 (e.g., "using e-learning is fun"). Anxiety contained three items with reversed wording and factor loadings ranged from .77 to .83 (e.g., "I hesitate to use e-learning because of making a mistake"). The factor loadings of the two items of performance expectancy were .41 and .93 (e.g., "I think the e-learning tool is useful"). Facilitating conditions contained two items, with factor loadings of .71 and .77 (e.g., "I have sufficient knowledge to use the e-learning tool").

Online Learning Platform and Materials

Austin, Gorsuch, Lawson, and Newberry (2011) believed that in higher education, ethics courses are regarded as being of high value, and in the majority of disciplines, research ethics is considered as a compulsory basic science. Therefore, this study utilized a research ethics course in our online learning program. The course materials used in this study were the ethics education materials taken from the Taiwan Research Ethics Education Program by Taiwanese Ministry of Education. This program aims to raise the awareness of research ethics and to develop digital learning resources for Taiwanese higher education institutions. The aforementioned online course materials were used in this study to develop a proprietary platform for the learning management system. The platform was built on the Microsoft Internet Information Service server using Microsoft Visual Studio 2013 as the development tool, ASP.NET 4.5 C# as the programing language framework, and Microsoft SQL Server 2008 R2 as the database. Microsoft Windows Server 2008 R2 was used as the operation system. A screenshot of the online course is shown in Figure 3.

Figure 3. Screenshot of the online course.

The Flash multimedia audio and video animation learning materials in this course were divided into four chapters, each with 30-90 seconds of audio and video on the front page. The four units of the course are Ethics and Responsibility of Academic Research, What is Research Misconduct, Basic Concepts of Plagiarism, and How to Avoid Plagiarism. Each unit contained video, audio, and text materials. Each chapter took the user approximately 30-40 minutes to learn. The number of questions included in the four unit tests were 25, 13, 17, and 16 respectively (a total of 71 questions). All questions are multiple choices with four options. There were buttons labeled "next page" and "previous page" in the system that allowed students to sequentially or repeatedly run the pages. The system interface also provided a unit navigation menu, allowing students to switch freely between the pages.

Data Analysis Method

Black et al. (2008) pointed out that combining questionnaires and actual log data can reveal the relationship between the mental states of learners and their actual behavior. Therefore, this study adopted learning motivation and reading duration as the base of the cluster analysis. Data collected in the study included the rating of the digital learning motivation scale, the total online reading time, and the behavioral coding logs. A cluster analysis and a behavioral sequence analysis were applied to the data analysis. Predictive Analytics Software (PASW) 22 was used to perform the cluster analysis. This study used learning motivation and e-textbook reading duration as the two variables for cluster analysis. In the first stage, Ward's method was used to determine the number of clusters as the preliminary optimal. In the second stage, K-mean was used to establish the grouping results. No significant correlation was found between learning motivation and e-textbook reading duration in the pre-test performed in this study (r =.04, p =.62), and no collinearity of variables was observed, indicating that the data was fit for grouping. Ward's method was adopted to divide students into two to five groups sequentially. The variance analysis of the clustering validity is listed in Table 1. There was a substantial increase of the coefficients of determination (R²) in the transition from three to four groups, while the increase was not apparent in the transition from four to five groups. Although four and five groups had higher coefficients of determination, the number of participants in each group was small (< 30), yet there was a large discrepancy in the number of participants in each group. Therefore, we decided to use three groups for the K-means cluster analysis. After K-means clustering, the groups were named as follows: low reading duration with low motivation (LRLM group), low reading duration with high motivation (LRHM group), and high reading duration (HR group), representing three different types of student learners. ANOVA revealed that the scores of all three groups showed significant variances in terms of pretest learning motivation (F = 78.40, p < .01) and online e-textbook reading duration (F = 234.23, p < .01). A post hoc comparison showed that the learning motivation of the LRHM group was significantly greater than that of the LRLM group (p < .01) and the HR group (p < .01). Furthermore, the post hoc comparison showed that the learning motivation of the HR group was substantially greater than that of the LRLM group (p < .01). In addition, the reading duration of the HR group was significantly greater than that of the LRLM group (p < .01) and the LRHM group (p < .01), and no apparent difference was found in reading duration between the LRLM group and the LRHM group (p =.54). This confirms that the characteristics of the three groups were indeed different, and therefore proves validity of this grouping.

Table 1

Cluster Analysis Results Using the Ward's Method

After the students were classified into different groups, a sequential analysis was then used to examine the sequential behavioral pattern in each type of student. In this study, the online learning behavior was encoded based on the guidelines proposed by Bakeman (1997). A pilot test along with the observation and interview of two graduate students was conducted. The results indicated that the time the participants took to finish the course was shorter than expected. During the interview, the participants also indicated that the learning pace was not fast and the material covered was not difficult. Therefore, this study established 120 seconds as the average amount of time needed to flip through a page of intensive reading. If a page was turned before the learning material was finished playing, it would be classified as incomplete reading of the learning materials. To sum up the results from this pilot test, a definition of behavior codes used in this study is listed in Table 2. A self-developed program system was used in this study to automatically transform the behavior records into codes to avoid human error and unnecessary omissions. There were a total of 13,035 converted codes. We then used the Multiple Episode Protocol Analysis (MEPA) software to conduct a sequential analysis.

Table 2

Online Reading Behavioral Coding Scheme

Results

To address research question (1), the sequential codes of each group were statistically organized based on the results of the cluster analysis conducted in this study. Related data are as shown in Table 3 and Figure 4.

Table 3

Descriptive Statistics of the Sequential Behavioral Codes of the Observed Values in Each Cluster

*Note. LRLM group means "low reading duration with low motivation group"; LRHM group means "low reading duration with high motivation group"; HR group means "high reading duration group."

Figure 4. Pie chart of behavior percentage in each cluster.
*Note. LRLM group means "low reading duration with low motivation group"; LRHM group means "low reading duration with high motivation group"; HR group means "high reading duration group."

To address research question (2), the data was converted into an adjusted residuals table of sequential codes, as shown in Table 4, with the horizontal axis representing "start of behavior" and the vertical axis representing "end of behavior." For the z value shown in the adjusted residuals table, if the value was greater than 1.96, the change in sequential behavior of that group reached the significance level of p < .05 (Bakeman, 1997; Hou, 2012b); that is, there is a significant relationship between the order of sequence. The sequential behavioral pattern among the three groups in online reading is shown in Figure 5.

Table 4

The Adjusted Residuals Table of the Three Student Groups

*Note. LRLM group means "low reading duration with low motivation group"; LRHM group means "low reading duration with high motivation group"; HR group means "high reading duration group."

Figure 5. Diagram of sequential behavioral patterns in online reading of all three groups.

Discussion

Online Students Exhibit Different Reading Behavior Frequencies

According to the behavior frequency pie chart (Figure 4), the allocation of behavior outcomes in the two groups of low reading duration is quite similar, but is considerably different from the high reading duration group. In the high reading duration group, 68.3% of students fell into the "intensive reading" category of the overall learning behavior, whereas the intensive reading behavior in the low reading duration with low motivation and low reading duration with high motivation groups was only 34% and 39.9%, respectively. "Skim reading" behavior was only exhibited by 18.5% of the students in the high reading duration group, while the percentages of the other two groups were quite high - 36.5% and 34%, respectively. Among the students in the group of high reading duration group, only for 2.4% exhibited behavior of "not completing the unit test," but the percentages of students in the other two groups who exhibited this same behavior was 14.5% and 13.8%. In comparing the differences in behavioral proportions due to different learning motivation factors, the results showed that reading duration could significantly affect learning behavior. Students with high reading duration tended to be more seriously engaged in learning activities, and these students not only tended to exhibit page retention time spent on the e-textbook learning materials in accordance with expectations, but also tended to exhibit a relatively lower rate of occurrence in the behavior of "not completing the unit test."

In addition, for both the group of low reading duration with low motivation and the group of low reading duration with high motivation, the next behavior after "passing the unit test" was "not completing the unit test" (D→E). However, such a sequence was not observed in the high reading duration group. Thus, we could see that after passing a unit test, the two low reading duration groups had the tendency to quickly transition into the test challenge in the following unit, rather than spending time reading the learning materials in the following unit. Such results indicate that these two low reading duration groups seem to have a speculative mentality: they hoped to finish the course quickly and therefore skipped reading the learning materials and took the test right away. This result is consistent with the study of Hu et al. (2014) who pointed out that online learning time can significantly predict the learning performance.

The Online Students of Different Types Have Different Reading Behavior Patterns

Although the behavior of digital media multitasking is believed to lower task performance (Gardner, 2008), we found that the group of low reading duration with high motivation exhibited a different behavior sequence than the other two groups. The group of low reading duration with high motivation showed that the behavior of "intensive reading" tended to follow the behavior of "intensive reading" and "multi-tasking reading" (A→B, B→A), and "multi-tasking reading" behavior also tended to follow "intensive reading" behavior (A→B). The findings suggest that when multi-tasking under the "intensive reading" environment, students may have been delayed in flipping through the learning materials and were therefore encoded as performing "multi-tasking reading." However, when encoded as performing "multi-tasking reading," this meant that a student's attention had not completely detracted from the learning materials, and that he or she had returned to the page with "intensive reading" behavior. Other data in this study showed that the group of low reading duration with high motivation exhibited 944 instances of "intensive reading," whereas the low reading duration with low motivation group only showed 715 instances of "intensive reading." Therefore, for students who prefer the multi-tasking online learning environment, highly motivated students can easily switch back from multi-tasking to the learning materials and refocus their learning. Many studies in the past have widely viewed learning motivation as an important factor in the actual learning behavior that is exhibited (Kong et al., 2012; Schunk et al., 2013). However, we found that highly motivated students may prefer to exhibit a motivated online multi-tasking reading behavior pattern; that is, in which students' learning behavior is characterized in terms of focused multi-tasking.

Conclusion, Limitations, and Future Research

This study aims to explore the effects of learning motivation on online reading behavioral patterns. In response to the research question (1), the study found that the high reading behavior duration group presented a higher ratio of intensive reading and a lower ratio of non-completion of the unit test. However, the other two groups (low reading duration with low motivation and low reading duration with high motivation groups) manifested a lower proportion of intensive reading behavior, higher proportion of skim reading behavior, and a higher proportion of non-completion of the unit. In response to research question (2), we discovered that, after taking a unit test, the low reading duration with low motivation and low reading duration with high motivation groups had a behavioral tendency to bypass the reading materials and instead participate directly in the next unit test. In addition, the behavioral patterns of the low reading duration with high motivation group showed the characteristic of interchange between intensive reading and multi-tasked reading. In summary, this study indicates that "online reading duration" in the online learning system is a strong indicator for reading motivation in students. In addition, this study proposes that multi-tasking learning should be considered in the future design of online learning courses to provide appropriate teaching strategies or platform functions. Learning motivation is an important factor in enhancing "intensive reading" in students. With the aid of online course materials, instructors can further enhance students' learning motivation. We propose that a record function should be added for "online reading duration" in the online course platform in future online courses. Coupled with an interface with graphical presentation, instructors or system managers can better understand the learning situation so that flexible adjustments to the online learning materials can be made accordingly, and so that the effectiveness of the learning platform and student performance can be evaluated more accurately.

In terms of research limitations, we used the time spent on reading the online materials to define the student's behavior as "intensive reading," "multi-tasking reading," or "skim reading." However, these standards are not definitively certain in terms of effectiveness. As the length of each animation page in the learning materials varies, applying the same amount of time as the encoding standard may not fully explain the actual usage behavior in the students. Therefore, the analysis in this study may be limited. In addition, this study adopted a convenient sampling method to recruit graduate students from three national universities in Taiwan; therefore, readers should be cautious when applying our findings to the entire population. This study adopted reading duration as an indicator of behavioral engagement. Future research is suggested to further explore other influential factors of behavioral patterns in online learning, such as emotional and cognitive engagement, perceptual habits in reading, learning attitudes, gender, and academic background. Given that the research ethics course adopted in this study aimed to establish learners' basic knowledge, criterion-referenced tests that allowed repeated participation were utilized; therefore, the results of the test were not included in the discussion. Future studies are advised to adopt a topic with a more comprehensive question pool so as to explore the impact of online learning behavior patterns on learning performance. Finally, with the large amount of data saturation that exists today, using the back-end quantitative data on the server to undertake a massive analysis is a worthwhile direction to take in the developmental stages. We suggest that in subsequent research the correlation between the back-end data in the server and the actual usage behavior in the front end should be explored through observation and interviews. Furthermore, we suggest that data triangulation should be applied to such an investigation to examine the actual reading situations and page retention time to serve as the foundation for massive data analysis and to further understand reading behavior patterns in online courses.

Acknowledgement

This research was supported by the Ministry of Science and Technology in Taiwan through Grant No. MOST 105-2511-S-009-013-MY5. The authors would like to thank the students who participated in this study and Huei-Tse Hou, Chao-Hsiu Chen, Yu-Yan Lin, and all of the reviewers for their useful comments.

References

Agudo-Peregrina, Á. F., Iglesias-Pradas, S., Conde-González, M. Á., & Hernández-García, Á. (2014). Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in Human Behavior, 31(0), 542-550. doi: 10.1016/j.chb.2013.05.031

Ann, L. K. (2006). Study design III: Cross-sectional studies. Evidence-Based Dentistry, 2006(7), 24-25. doi: 10.1038/sj.ebd.6400375

Austin, K. A., Gorsuch, G. J., Lawson, W. D., & Newberry, B. P. (2011). Developing and designing online engineering ethics instruction for international graduate students. Instructional Science, 39(6), 975-997.

Bakeman, R. (1997). Observing interaction: An introduction to sequential analysis. United Kingdom: Cambridge University Press.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215.

Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11(2), 65-70. doi: 10.1016/j.iheduc.2008.03.002

Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165-1188.

Chen, K. C., & Jang, S. J. (2010). Motivation in online learning: Testing a model of self-determination theory. Computers in Human Behavior, 26(4), 741-752. doi: 10.1016/j.chb.2010.01.011

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.

Eryilmaz, E., Chiu, M. M., Thoms, B., Mary, J., & Kim, R. (2014). Design and evaluation of instructor-based and peer-oriented attention guidance functionalities in an open source anchored discussion system. [Article]. Computers & Education, 71, 303-321. doi: 10.1016/j.compedu.2013.08.009

Gardner, J. S. (2008). Simultaneous media usage: Effects on attention. Virginia Polytechnic Institute and State University.

Gil-Flores, J., Torres-Gordillo, J.-J., & Perera-Rodríguez, V.-H. (2012). The role of online reader experience in explaining students' performance in digital reading. Computers & Education, 59(2), 653-660. doi: 10.1016/j.compedu.2012.03.014

Hou, H.-T. (2012a). Analyzing the learning process of an online role-playing discussion activity. Educational Technology & Society, 15(1), 211-222.

Hou, H.-T. (2012b). Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG). Computers & Education, 58(4), 1225-1233. doi: 10.1016/j.compedu.2011.11.015

Hou, H.-T., & Wu, S.-Y. (2011). Analyzing the social knowledge construction behavioral patterns of an online synchronous collaborative discussion instructional activity using an instant messaging tool: A case study. Computers & Education, 57(2), 1459-1468. doi: 10.1016/j.compedu.2011.02.012

Hu, Y. H., Lo, C. L., & Shih, S. P. (2014). Developing early warning systems to predict students' online learning performance. Computers in Human Behavior, 36, 469-478. doi: 10.1016/j.chb.2014.04.002

Hung, J. L., & Zhang, K. (2008). Revealing online learning behaviors and activity patterns and making predictions with data mining techniques in online teaching. MERLOT Journal of Online Learning and Teaching, 4(4), 426-437.

Johnson, L., Adams, S., & Cummins, M. (2012). The NMC horizon report: 2012 higher education edition. Austin, Texas: The New Media Consortium.

Johnson, L., Adams, S., Cummins, M., Estrada, V., Freeman, A., & Ludgate, H. (2013). The NMC horizon report: 2013 higher education edition. Austin, Texas: The New Media Consortium.

Johnson, L., Becker, S., Estrada, V., & Freeman, A. (2014). The NMC horizon report: 2014 higher education edition. Austin, Texas: The New Media Consortium.

Kong, J. S. L., Kwok, R. C. W., & Fang, Y. (2012). The effects of peer intrinsic and extrinsic motivation on MMOG game-based collaborative learning. Information & Management, 49(1), 1-9. doi: 10.1016/j.im.2011.10.004

Lai, C.-L., & Hwang, G.-J. (2015). A spreadsheet-based visualized mindtool for improving students' learning performance in identifying relationships between numerical variables. Interactive Learning Environments, 23(2), 230-249.

Liu, C.-C., Cheng, Y.-B., & Huang, C.-W. (2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57(3), 1907-1918. doi: 10.1016/j.compedu.2011.04.002

Liu, Z. (2005). Reading behavior in the digital environment: Changes in reading behavior over the past ten years. Journal of Documentation, 61(6), 700-712.

Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, September/October, 31-40.

Ma, J., Han, X., Yang, J., & Cheng, J. (2015). Examining the necessary condition for engagement in an online learning environment based on learning analytics approach: The role of the instructor. The Internet and Higher Education, 24, 26-34. doi: 10.1016/j.iheduc.2014.09.005

Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an "early warning system" for educators: A proof of concept. Computers & Education, 54(2), 588-599. doi: 10.1016/j.compedu.2009.09.008

Moore, M. G. (1989). Editorial: Three types of interaction. American Journal of Distance Education, 3(2), 1-7.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychological theory (3rd ed.). New York: McGraw-Hill.

Pellas, N. (2014). The influence of computer self-efficacy, metacognitive self-regulation and self-esteem on student engagement in online learning programs: Evidence from the virtual world of Second Life. Computers in Human Behavior, 35, 157-170. doi: 10.1016/j.chb.2014.02.048

Raab, R. T., Ellis, W. W., & Abdon, B. R. (2001). Multisectoral partnerships in e-learning: A potential force for improved human capital development in the Asia Pacific. The Internet and Higher Education, 4(3), 217-229.

Saadé, R. G., He, X., & Kira, D. (2007). Exploring dimensions to online learning. Computers in Human Behavior, 23(4), 1721-1739. doi: 10.1016/j.chb.2005.10.002

Schunk, D. H., Meece, J. L., & Pintrich, P. R. (2013). Motivation in education: Theory, research, and applications (4th ed.). Upper Saddle River, NJ: Pearson.

Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement and disaffection in the classroom: Part of a larger motivational dynamic? Journal of Educational Psychology, 100(4), 765-781.

Sun, J. C.-Y., Kuo, C.-Y., Hou, H.-T., & Lin, Y.-Y. (2017). Exploring learners' sequential behavioral patterns, flow experience, and learning performance in an anti-phishing educational game. Educational Technology & Society, 20(1), 45-60.

Sun, J. C.-Y., & Rueda, R. (2012). Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education. British Journal of Educational Technology, 43(2), 191-204. doi: 10.1111/j.1467-8535.2010.01157.x

Tseng, S.-C., & Tsai, C.-C. (2010). Taiwan college students' self-efficacy and motivation of learning in online peer assessment environments. Internet and Higher Education, 13(3), 164-169. doi: 10.1016/j.iheduc.2010.01.001

Wang, S.-L., & Lin, S. S. J. (2007). The effects of group composition of self-efficacy and collective efficacy on computer-supported collaborative learning. Computers in Human Behavior, 23(5), 2256-2268. doi: 10.1016/j.chb.2006.03.005

Yang, T.-C., Chen, S. Y., & Hwang, G.-J. (2015). The influences of a two-tier test strategy on student learning: A lag sequential analysis approach. Computers & Education, 82, 366-377.

Yang, X., Li, J., Guo, X., & Li, X. (2015). Group interactive network and behavioral patterns in online English-to-Chinese cooperative translation activity. Internet and Higher Education, 25, 28-36.

Yoo, S. J., Han, S.-H., & Huang, W. (2012). The roles of intrinsic motivators and extrinsic motivators in promoting e-learning in the workplace: A case from South Korea. Computers in Human Behavior, 28(3), 942-950. doi: 10.1016/j.chb.2011.12.015

Applying Learning Analytics to Explore the Effects of Motivation on Online Students' Reading Behavioral Patterns by Jerry Chih-Yuan Sun, Che-Tsun Lin, and Chien Chou is licensed under a Creative Commons Attribution 4.0 International License.