International Review of Research in Open and Distributed Learning

Volume 23, Number 4

November - 2022

Translating and Validating the Community of Inquiry Survey Instrument in Brazil

Cibele Duarte Parulla, Anne Marie Weissheimer, Marlise Bock Santos, and Ana Luísa Petersen Cogo
Universidade Federal do Rio Grande do Sul

Abstract

Massive open online courses (MOOCs) have emerged as an affordable way to distribute knowledge and democratize education. The examination of online courses calls for theoretical models and instruments that contemplate its particularities. The community of inquiry (CoI) framework has been used in several studies to analyze the effectiveness of online education and hybrid education, including MOOCs. This study aimed to translate and validate the Community of Inquiry Survey instrument (Arbaugh et al., 2008) into Brazilian Portuguese, and used a two-stage methodological design for translating and validating a questionnaire. In the first stage, we carried out translation, back-translation, and cross-cultural adaptation. We translated the 34 items while maintaining the survey’s original format. In the expert evaluation phase, all items were considered understandable and essential for inclusion in the Brazilian Portuguese version of the CoI instrument. In the second stage, a prospective cross-sectional study was conducted to validate the questionnaire, and data was collected from participants completing the Nursing Assessment MOOC available on the Lúmina platform. A total of 690 responses were gathered. The resulting instrument produced excellent results, and the three presences achieved high reliability indexes, clearly indicating their adequacy. Furthermore, this study proved the validation of the CoI instrument, maintaining the three-factor structure previously reported in the literature corresponding to the three presences: teaching, social, and cognitive presence. We recommend further studies to evaluate the need for excluding or altering cognitive presence items.

Keywords: Instrument validation, community of inquiry, exploratory factor analysis, massive open online courses, instrument translation, Brazilian Portuguese

Translating and Validating the Community of Inquiry Survey Instrument in Brazil

Massive open online courses (MOOCs) have emerged as a way to distribute knowledge produced by renowned institutions, and to democratize teaching on different platforms in an affordable and low-cost manner (Barba et al., 2018). Moreover, during the COVID-19 pandemic, MOOCs became an important educational tool with increased enrollments and flexibility, given that several providers reduced fees (Impey & Formanek, 2021). Even in a post-pandemic projection, MOOCs can be considered a staple tool. They can also provide learning experiences for higher education students, such as learning with minimal supervision and instructor-learner interaction. Furthermore, universities can maintain this teaching format as a replacement for or complement to offerings in some theoretical disciplines (Safri et al., 2020).

However, the examination of online courses requires theoretical models and instruments that contemplate their particularities. The community of inquiry (CoI) framework is a theoretical model that has guided the development of online learning and evaluation of its effectiveness (Garrison et al., 2000). It was deliberately intended to guide the development of online and computer-mediated education.

The CoI was based on the ideas of John Dewey, and has constituted an important reinforcement to the use of constructivist theories of learning in higher education (Garrison et al., 2000). The CoI model considered three elements: teaching presence, social presence, and cognitive presence (Garrison et al., 2000). Teaching presence consists of granting, facilitating, and directing cognitive and social processes, to achieve learning outcomes with personal meaning and educational value (Garrison et al., 2000). Social presence is the ability of students to project themselves socially and emotionally, and to be perceived as real people in mediated communication (Garrison et al., 2000). Finally, cognitive presence is the extent in which students are able to construct and confirm meaning through sustained reflection and discourse, which allows a continuous evaluation of the organization of critical thinking and reflections throughout the course (Garrison et al., 2000, 2001).

Developed by Arbaugh et al. (2008), the CoI Survey instrument consisted of 34 items with a 5-point Likert scale, designed to measure students’ perceived levels in teaching presence, social presence, and cognitive presence. In 2007, this research instrument was applied in four different institutions located in the United States and Canada. The study participants were enrolled in graduate courses in education or administration. A total of 287 students volunteered to answer the survey, with a response rate of 43% (Arbaugh et al., 2008). In 2008, the Community of Inquiry Survey instrument was formally proposed and validated to strengthen and expand the use of the CoI.

Consistent with the design of the instrument, items 1 to 13 (teaching presence) loaded more strongly into factor 1. Items 14 to 22 (social presence) had more influence on factor 2. Finally, items 23 to 34 (cognitive presence) had more influence on factor 3. Cronbach’s alpha yielded internal consistencies equal to 0.94 for teaching presence, 0.91 for social presence, and 0.95 for cognitive presence (Arbaugh et al., 2008).

This instrument has been used in different countries, and has been translated into and validated in many languages in order to expand its use (Ma et al., 2017; Moreira et al., 2013; Olpak & Kiliç Çakmak, 2018; Velázquez et al., 2019; Yu & Richardson, 2015). Up to now, the instrument has not yet been validated and adapted to Brazilian Portuguese, limiting its use in this language, which is an imminent need for the development of research evaluating distance education.

The CoI has been used in several studies to analyze the effectiveness of online education and hybrid education; it has also been used to evaluate MOOCs. These studies explored teaching, social, and cognitive presence in contexts beyond strictly instructional and impersonal models to those that enable interaction (Caskurlu et al., 2020; Stenbom, 2018; Velázquez et al., 2019; Yu & Richardson, 2015). However, despite confirming the potential for the use of the CoI to provide a better understanding of learning processes in MOOCs, it is still necessary to explore how course design affects the three CoI presences (Kovanović et al., 2018).

In nursing teaching, we found reports of the CoI being used in hybrid education at undergraduate and graduate levels, characterized by the development of collaborative activities with exchanges between participants, especially using discussion forums (Miils et al., 2016; Phillips et al., 2013; Stephens & Hennefer, 2013). In a study to investigate the level of knowledge of the CoI structure and its applicability to the design of online and hybrid courses in Australian higher education nursing schools, the results showed that instructors classified the three presences (i.e., teaching, social, and cognitive) of the CoI framework as applicable to teaching nursing online (Smadi et al., 2019). Also, instructors who were familiar with the CoI structure reported that they would probably recommend the structure of the CoI to a colleague (Smadi et al., 2019).

This current study was justified by the need to use instruments cross-culturally and adapted to other languages, in order to expand their applicability for other cultures. Specifically, this study aimed to translate and validate the CoI Survey instrument for Brazilian Portuguese.

Method

Study Design

We followed Beaton et al.’s (2000) methodological design for translating and validating a questionnaire. Before starting the process, e-mail consent was obtained to translate and validate the CoI Survey instrument from English into Brazilian Portuguese. In the first stage, translation, back-translation, and cross-cultural adaptation were conducted. In the second stage, a prospective cross-sectional survey was conducted to validate the translated questionnaire.

First Stage: Translating and Adapting the Questionnaire to Portuguese

The first translation stage was performed by two independent translators who translated the CoI to Portuguese. The researchers then synthesized the two versions, resulting in a single Portuguese version. In the back-translation step, the Portuguese version was translated back into English by two others translators who both had mastery of English. For one of the translators, English was their mother tongue. All items were evaluated for semantic, idiomatic, cultural, and conceptual equivalences. The final Portuguese version was composed by the researchers’ consensus.

To finish this stage, the Portuguese questionnaire, translated in the online format, was made available as a pre-test to a convenience sample of 30 nursing students participating in the Nursing Assessment MOOC conducted by the Universidade Federal do Rio Grande do Sul and hosted on the Lúmina platform.

Second Stage: Questionnaire Validation Process

To validate the questionnaire, data was collected from the Nursing Assessment MOOC available on the Lúmina platform using an online form. Between September 2019 and February 2020, 1,063 students responded to the research instrument. Incomplete responses were excluded, resulting in a total of 690 responses.

Data Analysis

Exploratory factor analysis was performed using the Statistical Package for Social Sciences (SPSS, version 21). The variables were described by mean and standard deviation. Cronbach’s alpha coefficient was used to determine the internal consistency of the instrument. In evaluating the instrument structure along three subscales, factor analysis by principal components and with varimax rotation was applied. To verify the adequacy of the sample for factor analysis, the Keyser-Meyer-Olsen (KMO) measurement was obtained. The association between the domains (i.e., teaching, social, and cognitive presence) was measured by Pearson’s correlation test. A 5% significance level was adopted (p < 0.05).

Ethical Issues and Permissions

All the procedures adopted in this study complied with the criteria on Ethics in Research with Human Beings, according to Resolution no. 466 (December 12, 2012) of the National Health Council of Brazil (Government of Brazil, 2012). This research was approved by the Research Ethics Committee of the Universidade Federal do Rio Grande do Sul.

Results

First Stage: Translating the Questionnaire

The 34 items of the Arbaugh et al. (2008) survey were translated without any difficulty while maintaining the original format (Appendix).

Second Stage: Validating the Questionnaire

The sample of 690 resulted in a KMO measurement of 0.96. The sample size (N = 690) for this study was considered adequate, meeting the recommendation of 10 or more respondents per item of the questionnaire under validation (Nunnally, 1978). The measure of adequacy of the KMO sample demonstrated that the factor analysis was reliable. Table 1 shows the total explained variance of the main components. Specifying a three-factor solution accounts for 65.1% of the total variance. More than half (57.4%) of the total variation of this three-factor solution was attributed to the first two factors. Component analysis suggested a fourth additional factor; however, it did not show as significant variation as did the first three factors, which was also apparent in the sedimentation graph (Figure 1).

Table 1

Total Variance for the CoL Instrument Explained by Factor Analysis with Varimax Rotation

Figure 1

Sedimentation Graph of Factor Analysis of CoL Instrument

Table 2 lists the load factor for each of the 34 items of the CoI in terms of the three factors. Items 1 to 13 (teaching presence) had more influence on factor 2, and items 14 to 22 (social presence) had more influence on factor 1. Items 23 to 34 (cognitive presence) had more influence on factor 3, except for items 27 and 28 that presented stronger loads for factor 1. Cronbach’s alpha produced internal consistencies of 0.94 for teaching presence, 0.95 for social presence, and 0.91 for cognitive presence. The three domains showed a positive correlation with each other (Table 3).

Table 2

Component Rotation Matrix for CoI Instrument

Note. Observation: Extraction method: Principal Component Analysis.

Table 3

Correlations Between the CoI Instrument Domains

Note. Pearson correlation, significant at level 0.01** (2 ends).

Discussion

The suggestion of adding a fourth factor to the model has been discussed since the original validation of the instrument. However, as well as the data found in this study, the sedimentation graph fails to inform the possibility of a fourth additional factor, considering the intense decrease in the magnitude of the eigenvalues of the first and second factors (Arabaugh et al., 2008). The structure of three factors was also maintained and confirmed in the Korean version; this study used exploratory and confirmatory factor analysis to evaluate its validity found 63.82% of the explanation of the structure within three factors (Yu & Richardson, 2015). Despite keeping the initial structure, the Chinese version of the instrument suggested additional emphasis on teaching presence (Ma et al., 2017). In a study conducted specifically with MOOCs, the analysis suggested a six-factor model to better adjust the data (Kovanović et al., 2018).

We observed that two items of cognitive presence, items 27 (debating and searching for relevant information helped us solve content-related issues) and 28 (online discussions were valuable in helping us understand different perspectives), pointed more strongly to social presence. In part, this phenomenon can be explained by the MOOC course design. It is possible that discrepancies between the dynamics of traditional online courses and MOOCs affect students’ perceptions of the three presences of the CoI (Kovanović et al., 2018); studies have suggested that further research is needed into the role of social presence in MOOCs (Poquet et al., 2018, Stranach, 2017). However, in the validation for the Korean language, conducted with online graduate courses, the authors also suggested the removal of two items from social presence, as well as one item of teaching presence, resulting in a model of 31 items (Yu & Richardson, 2015). On the other hand, the Turkish version of the instrument maintained the 34-item structure (Olpak & Kiliç Çakmak, 2018).

Conclusion and Limitations

This study aimed to translate and validate the CoI Survey for Brazilian Portuguese. The resulting instrument produced excellent results, and the three presences achieved high reliability indicators, demonstrating their adequacy.

Furthermore, this study further validated the CoI instrument, maintaining the structure of three factors—teaching presence, social presence, and cognitive presence. Using the validated and adapted CoI makes it possible to increase its use in the Brazilian context in order to support the development and evaluation of distance education in Brazil. In addition, applying the instrument translated to Brazilian Portuguese allows for further studies to compare different educational realities. We recommend conducting studies to evaluate the need to exclude or alter cognitive presence items.

As a limitation of this study, we emphasize that the validation of the instrument translated into Brazilian Portuguese took place in a single MOOC and not in a blended format. Thus, we suggest further studies to use and evaluate the instrument validated in other online course formats.

References

Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C., & Swan, K. P. (2008). Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample. Internet and Higher Education, 11(3-4), 133-136. https://doi.org/10.1016/j.iheduc.2008.06.003

Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2018). The role of students’ motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32(3), 218-231. https://doi.org/10.1111/jcal.12130

Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 16(2), 3186-3191. https://doi.org/10.1097/00007632-200012150-00014

Caskurlu, S., Maeda, Y., Richardson, J. C., & Lv, J. (2020). A meta-analysis addressing the relationship between teaching presence and students’ satisfaction and learning. Computers and Education, 157(September 2019), 103966. https://doi.org/10.1016/j.compedu.2020.103966

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. https://doi.org/10.1016/S1096-7516(00)00016-6

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23. https://doi.org/10.1080/08923640109527071

Government of Brazil. (2012). Resolução n° 466, de 12 de dezembro de 2012 [Resolution No. 466, December 12, 2012]. National Health Council, Ministry of Health. http://www.conselho.saude.gov.br/web_comissoes/conep/index.html

Impey, C., & Formanek, M. (2021). MOOCS and 100 days of COVID: Enrollment surges in massive open online astronomy classes during the coronavirus pandemic. Social Sciences & Humanities Open, 4(1), 100177. https://doi.org/10.1016/j.ssaho.2021.100177

Kovanović, V., Joksimović, S., Poquet, O., Hennis, T., Čukić, I., de Vries, P., Hatala, M., Dawson, S., Siemens, G., & Gašević, D. (2018). Exploring communities of inquiry in massive open online courses. Computers & Education, 119, 44-58. https://doi.org/10.1016/j.compedu.2017.11.010

Ma, Z., Wang, J., Wang, Q., Kong, L., Wu, Y., & Yang, H. (2017). Verifying Causal Relationships Among the Presences of the Community of Inquiry Framework in the Chinese Context.  The International Review of Research in Open and Distributed Learning, 18(6). https://doi.org/10.19173/irrodl.v18i6.3197

Moreira, J. A., Ferreira, A. G., & Almeida, A. C. (2013). Comparing communities of inquiry of Portuguese higher education students: One for all or one for each? Open Praxis, 5(2), 165-178. http://dx.doi.org/10.5944.

Mills, J., Yates, K., Harrison, H., Woods, C., Chamberlain-Salaun, J., Trueman, S., & Hitchins, M. (2016). Using a community of inquiry framework to teach a nursing and midwifery research subject: An evaluative study. Nurse Education Today, 43, 34-39. https://doi.org/10.1016/j.nedt.2016.04.016

Olpak, Y. Z., & Kiliç Çakmak, E. (2018). Examining the reliability and validity of a Turkish version of the community of inquiry survey. Online Learning, 22(1), 147-161. https://doi.org/10.24059/olj.v22i1.990

Phillips, D., Forbes, H., & Duke, M. (2013). Teaching and learning innovations for postgraduate education in nursing. Collegian, 20(3), 145-151. https://doi.org/10.1016/j.colegn.2012.05.003

Poquet, O., Kovanović, V., de Vries, P., Hennis, T., Joksimović, S., Gašević, D., & Dawson, S. (2018). Social presence in massive open online courses. International Review of Research in Open and Distributed Learning, 19(3), 43-68. https://doi.org/10.19173/irrodl.v19i3.3370

Safri, S. N. W., Mohi, Z. & Hanafiah, M. H. (2020). Massive open online course (MOOC): Our saviour during COVID-19 pandemic? Journal of Tourism, Hospitality & Culinary Arts, 12(3), 120-128. https://fhtm.uitm.edu.my/images/jthca/Vol12Issue3/Chap_8.pdf

Smadi, O., Parker, S., Gillham, D., & Müller, A. (2019). The applicability of community of inquiry framework to online nursing education: A cross-sectional study. Nurse Education in Practice, 34, 17-24. https://doi.org/10.1016/j.nepr.2018.10.003

Stenbom, S. (2018). A systematic review of the community of inquiry survey. Internet and Higher Education, 39(June), 22-32. https://doi.org/10.1016/j.iheduc.2018.06.001

Stephens, M., & Hennefer, D. (2013). Internationalising the nursing curriculum using a community of inquiry framework and blended learning. Nurse Education in Practice, 13(3), 170-175. https://doi.org/10.1016/j.nepr.2012.08.010

Stranach, M. (2017). Social presence in two massive open online courses (MOOCs): A multiple case study [Unpublished doctoral thesis]. University of Calgary. http://dx.doi.org/10.11575/PRISM/27816

Velázquez, B. B., Gil-Jaurena, I., & Encina, J. M. (2019). Validation of the Spanish Version of the ‘Community of Inquiry’ survey. Revista de Educación a Distancia, 59(4). http://dx.doi.org/10.6018/red/59/04

Yu, T. & Richardson, J. C. (2015). Examining reliability and validity of a Korean version of the community of inquiry instrument using exploratory and confirmatory factor analysis. The Internet and Higher Education, 25, 45-52. https://doi.org/10.1016/j.iheduc.2014.12.004

Appendix: Dimensions and Items of the Brazilian CoI Survey Instrument - Comunidade Investigativa 

Presença de Ensino

Planejamento e Organização

1. O instrutor informou claramente os tópicos importantes do curso.
2. O instrutor informou claramente os objetivos principais do curso.
3. O instrutor apresentou instruções claras sobre como participar das atividades de aprendizagem do curso.
4. O instrutor informou claramente as datas e os prazos principais para entrega das atividades de aprendizagem.

Facilitação

1. O instrutor auxiliou a identificar áreas de concordância e discordância nos tópicos do curso que me ajudou a aprender.
2. O instrutor, ao orientar a atividade, auxiliou na compreensão dos tópicos do curso de forma que ajudou a esclarecer meu pensamento.
3. O instrutor auxiliou a manter os participantes do curso envolvidos e participativos em diálogos produtivos.
4. O instrutor auxiliou a manter os participantes do curso nas atividades de forma que me ajudou a aprender.
5. O instrutor incentivou os participantes a explorarem novos conceitos neste curso.
6. As ações do instrutor reforçaram o desenvolvimento do senso de comunidade entre os participantes do curso.

Instrução de direcionamento

1. O instrutor auxiliou em direcionar a discussão para questões relevantes de forma que me ajudou a aprender.
2. O instrutor forneceu retorno que me ajudou a compreender meus pontos fortes e fracos.
3. O instrutor forneceu retorno em tempo hábil.

Presença social

Expressão afetiva

1. Ter contato com os demais participantes deu-me a sensação de pertencimento no curso.
2. Eu fui capaz de formar impressões distintas sobre alguns dos participantes do curso.
3. A comunicação online ou através da internet é um excelente meio para interação social.

Comunicação aberta

1. Eu me senti confortável me comunicando online.
2. Eu me senti confortável participando das discussões do curso.
3. Eu me senti confortável interagindo com outros participantes do curso.

Coesão do grupo

1. Eu me senti confortável em discordar de outros participantes do curso, mantendo a sensação de confiança.
2. Eu senti que meu ponto de vista foi reconhecido por outros participantes do curso.
3. As discussões online me auxiliaram a desenvolver a sensação de colaboração.

Presença cognitiva

Evento disparador

1. A proposição de problemas aumentou o meu interesse nas questões de curso.
2. As atividades do curso instigaram minha curiosidade.
3. Eu me senti motivado a explorar questões relacionadas ao conteúdo.

Exploração

1. Eu utilizei várias fontes de informação para explorar problemas propostos neste curso.
2. O debate e a busca por informações relevantes me ajudaram a resolver questões relacionadas ao conteúdo.
3. As discussões online foram valiosas para me ajudar a valorizar diferentes perspectivas.

Integração

1. A combinação de novas informações me ajudou a responder questões que surgiram em atividades do curso.
2. As atividades de aprendizagem me ajudaram a construir explicações/soluções.
3. A reflexão sobre o conteúdo do curso e as discussões me ajudaram a compreender os conceitos fundamentais das aulas.

Resolução

1. Eu posso descrever maneiras para testar e aplicar o conhecimento gerado neste curso.
2. Eu desenvolvi soluções para os problemas do curso que podem ser aplicadas na prática.
3. Eu posso aplicar o conhecimento gerado neste curso no meu trabalho ou em outras atividades não relacionadas ao curso.

Escala de Likert:

1 = discordo totalmente
2 = discordo
3 = neutro
4 = concordo
5 = concordo totalmente

Translating and Validating the Community of Inquiry Survey Instrument in Brazil by Cibele Duarte Parulla, Anne Marie Weissheimer, Marlise Bock Santos, and Ana Luísa Petersen Cogo is licensed under a Creative Commons Attribution 4.0 International License.