A System of Indicators for the Quality Assessment of Didactic Materials in Online Education
DOI:
https://doi.org/10.19173/irrodl.v22i1.5069Keywords:
online education, didactic materials, quality, indicators, system, assessmentAbstract
The quality of didactic materials is a source of concern for teachers, users, and educational institutions that offer online education. There is a lack of indicators to help assess the quality of three key types of didactic materials commonly used in online education: didactic units (i.e., materials that contain program contents), didactic guides (i.e., materials that provide information), and additional didactic materials (materials to deepen knowledge). The objective of this article is to present a system of indicators designed to assess the quality of these types of didactic materials and guide their creation process. The system was developed based on a critical analysis of existing models designed to assess the quality of digital didactic materials. The system was validated by 16 international experts in online education, and a trial application of the system assessed five didactic guides and didactic units used by online universities in three different countries. Results of the validation process were triangulated with relevant literature, allowing the authors to make decisions regarding changes to the system in terms of maintaining, reformulating, or removing indicators. The resulting system comprises 43 assessment indicators and serves as a guide for designers, teachers, and users in the creation and selection of didactic materials for use in online education and in the assessment of their quality.
References
Asinsten, J. C. (2011). Producción de material didáctico en ambientes virtuales de aprendizaje [Production of didactic material in virtual learning environments]. Virtual Educa. https://virtualeduca.org/documentos/manual_del_contenidista.pdf
Bautista Liébana, J. R., Martínez Romero, R., & Sainz Ibáñez, M. (2001). La evaluación de materiales didácticos para la educación a distancia [The evaluation of didactic materials for distance education]. Revista Iberoamericana de Educación a Distancia, 4(1), 73–95. http://revistas.uned.es/index.php/ried/article/view/1192/1095
Blanco Gil, R., Casado García, L., Martín Ruiz, F. J., Mediano Villanueva, F. J., Ramos Ruiz, J., & Utiel Ortega, C. (2010). Didáctica de la tecnología [Teaching technology]. Grao.
Bravo Ramos, J. L. (2005). Elaboración de materiales educativos para la formación a distancia [Development of educational materials for distance learning] Instituto de Ciencias de la Educación (ICE) UPM. http://www.ice.upm.es/wps/jlbr/Documentacion/Libros/Elabora_mat_img.pdf
Cepeda Romero, O., ; Gallardo, I. M., & Rodríguez Rodríguez, J. (2017), La evaluación de los materiales didácticos Digitales [The evaluation of digital didactic materials]. Revista Latinoamericana de Tecnología Educativa, 16(2). https://doi.org/10.17398/1695-288X.16.2.79
Espinoza, O., & González, L. E. (2006). Procesos universitarios dinámicos. El modelo de gestión de la calidad total [Dynamic university processes. The total quality management model]. Calidad de la Educación, 1(24), 15–34. http://dx.doi.org/10.31619/caledu.n24.265
Fernández-Pampillón Cesteros, A. M., Domínguez Romero, E., & Armas Ranero, I. (2012a). Guía para la producción y evaluación de materiales didácticos digitales [Guide for the production and evaluation of digital didactic materials]. Universidad Complutense de Madrid. https://eprints.ucm.es/id/eprint/12533/1/COdAv1_1_07jul2012.pdf
Fernández-Pampillón Cesteros, A. M., Domínguez Romero, E., & Armas Ranero, I. (2012b). Diez criterios para mejorar la calidad de los materiales didácticos digitales [Ten criteria to improve the quality of digital didactic materials]. In A. Sanz Cabrerizo, J. A. López Orozco, & A. Baratas Díaz (Eds.). VII jornadas de campus virtual UCM: valorar, validar y difundir campus virtual (pp. 25–34). Universidad Complutense de Madrid. https://eprints.ucm.es/id/eprint/20297/
García Aretio, L. (2014). Bases, mediaciones y futuro de la educación a distancia en la sociedad digital [Bases, mediations, and future of distance education in the digital society]. Sintesis.
García Martín, A., Briones Peñalver, A. J., Busquier Sáez, S., García Cascales, M. S., De Miguel-Gómez, M. D., Mulas Pérez, J., Pérez García, J., Vicente Chicote, C., García Sanchez, A. J., Martínez Conesa, E. J., Serrano Martínez, J. L., Pérez Pérez, J., & Sánchez Pérez, L. Á. (2010). Manual de elaboración de guías docentes adaptadas al EEES. [Manual for preparing teaching guides adapted to the EEES]. Universidad de Cartagena. https://repositorio.upct.es/xmlui/bitstream/handle/10317/1747/isbn9788469350317.pdf
Hyla, M. (2016). Przewodnik po e-learningu. [A guide for e-learning]. Wolters Kluwer.
Illum Hansen, T., & Toke Gissel, S. (2017). Quality of didactic materials. IARTEM e-Journal, 9(1), 122–141. https://doi.org/10.21344/iartem.v9i1.601
Marciniak, R. (2016). Autoevaluación de programas de educación universitaria virtual [Self-assessment of higher virtual education programmes] (Doctoral dissertation). Autonomous University of Barcelona, Barcelona, Spain.
Marciniak, R. (2017). A methodological proposal for the Project design of the online course: Pilot application. APERTURA. Revista de Innovación Educativa, 9(2), 74-95. http://www.udgvirtual.udg.mx/apertura/index.php/apertura/article/view/991/797
Marciniak, R. (2018). Quality assurance for online higher education programmes: Design and validation of an integrative assessment model applicable to Spanish universities. The International Review of Research in Open and Distributed Learning 19(2), 126-154. http://www.irrodl.org/index.php/irrodl/article/view/3443
Marzal, M. A., Calzada-Prado, J., & Vianello, M. (2008). Criterios para la evaluación de la usabilidad de los recursos educativos virtuales: un análisis desde la alfabetización en información. [Criteria for evaluating the usability of virtual educational resources]. Information Research, 13(4), 387. http://informationr.net/ir/13-4/paper387.html
Morales Morgado, E. M., García Peñalvo, F. J., & Olmos Migueláñez, S. (2010). Diseño de objetos de aprendizaje para potenciar el desarrollo de competencias y su evaluación con
HEODAR [Design of learning objects to enhance the development of skills]. J. Sánchez (Ed.). IE 2010, Segundo Congreso Iberoamericano de Informática Educativa (pp. 683–690). Universidad de Chile. http://www.tise.cl/volumen6/TISE2010/Documento97.pdf
National Agency for Quality Assessment and Accreditation (ANECA). (2019). Guide to self-evaluation: Ex post accreditation for official Bachelor's Degrees, Master's Degrees and Doctoral Degrees ACREDITA Procedure. ANECA. http://www.aneca.es/eng/content/view/full/12612
National Distance Education University of Spain. (n.d.). El protocolo de evaluación de materiales didácticos a distancia [The evaluation protocol for distance learning materials]. http://www.uned.es/iued/subsitio/html/documentos/Protocolo_MADI.pdf
Nesbit, J. C., Belfer, K., & Vargo, J. (2002). A convergent participation model for evaluation of learning objects. Canadian Journal of Learning and Technology, 28(3), 105–120. https://doi.org/10.21432/T25C8C
Nielsen, J., & Morkes, J. (1998). Applying writing guidelines to Web pages. Nielsen Norman Group. https://www.nngroup.com/articles/applying-writing-guidelines-web-pages/
Opdenacker, L., Stassen, I., Vaes, S., Waes, L., & Jacobs, G. (Eds.). (2009). QuADEM:Manual for the quality assessment of digital educational material. University of Antwerp.
Padrón, C., Díaz, P., & Aedo, I. (2006). MD2 method: The didactic materials creation from a model based perspective. In: W. Nejdl & K. Tochtermann (Eds.), Innovative Approaches for Learning and Knowledge Sharing. Lecture Notes in Computer Science (vol. 4227, pp. 366–382). Springer. https://doi.org/10.1007/11876663_29
Rodríguez Rodríguez, J., & Martínez Bonafé, J. (2016). Libros de texto y control del currículum en el contexto de la sociedad digital. Revista Latinoamericana de Tecnología Educativa, 16(2), 79–95. https://doi.org/10.17398/1695-288X.16.2.79
Rushby, N., & Surry, D. (2016). Wiley handbook of learning technology. John Wiley and Sons. https://doi.org/10.1002/9781118736494
Shattuck, K. (2014). Assuring quality in online education: Practices and processes at theteaching, resource, and program levels. Sterling: Stylus Publishing.
Spanish Association for Standardization and Certification (AENOR). (2017). UNE 71362. Calidad de los materiales educativos digitales [UNE 71362. Quality of digital educational materials]. Madrid: AENOR.
Vizoso, C. M. (2018, November 29). Calidad de los MED: Materiales educativos digitales[Quality of the Digital Educational Materials]. Global Campus Nebrija. https://www.nebrija.com/medios/nebrijaglobalcampus/2018/11/29/calidad-de-los-med-materiales-educativos-digitales/
Vlachopoulos, D. (2016). Assuring quality in online course design: The roadmap. The International Review of Research in Open and Distance Learning, 17(6), 183–205. http://dx.doi.org/10.19173/irrodl.v17i6.2784
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International Licence. The copyright of all content published in IRRODL is retained by the authors.
This copyright agreement and use license ensures, among other things, that an article will be as widely distributed as possible and that the article can be included in any scientific and/or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
