Evidence-based eLearning Design: Develop and Trial a Prototype Software Instrument for Evaluating the Quality of eLearning Design Within a Framework of Cognitive Load Theory

Abstract

A major research direction within higher education in Australia and internationally is the evaluation of learning design quality and the extent to which the design–teaching–learning–evaluation cycle is evidence based. The quest for increased evidence-based learning design, which has been influenced by evidence-based medical research standards, is driven by its link to improved learning outcomes, higher learner engagement levels and lower attrition rates. Cognitive Load Theory (CLT) has risen to prominence over the past three decades as an evidence-based framework for informing instructional design in traditional, blended and multimedia learning environments. CLT approaches learning from the perspective of engaging specific strategies to manage the loads imposed on a limited working memory in order to form and automate long-term memory schemas. CLT operates on the premise that optimal learning conditions may be obtained by aligning pedagogical strategies with the structure and functions of human cognitive architecture and the individual learner’s prior knowledge. CLT has contributed a suite of strategies derived from a unified model of human cognitive architecture and validated through randomised controlled trial (RCT) experiments as exerting strengthening effects on learning, thus suiting the CLT framework for use as an evidence-based standard in this study. Up to this point, a single digital system has not yet been developed for managing, monitoring and evaluating the implementation and impact of CLT strategies at scale. The key contribution of this study is a new prototype software instrument called Cognitive Load Evaluation Management System (CLEMS) that addresses this issue and also provides a model for its implementation. CLEMS is underpinned by a personalised model of teacher–learner interactions defined as mediative–adaptive in nature that includes diagnostic conversations (DCs) for identifying barriers to learning, interventions called Nodes of Expertise (NOEs) for advancing learners to new levels of understanding of complex knowledge, and validation conversations (VCs) for evaluating learner progress. In addition, the heutagogical or self-directed learning capability of learners, including motivation, has been brought to the fore as a significant factor contributing to schema automation. A qualitative Design-based Research (DBR) methodological approach was used to develop CLEMS, which emerged over three research iterations through the synthesis of literature review findings and empirical data from expert focus groups. Emergent data was continuously triangulated between research iterations and ongoing literature reviews to refine the design and development of CLEMS from a theoretical model to an operational digital prototype. The conceptual framework of the study has been derived from Critical Realism (CR) which posits an ontological–epistemological view of reality that is stratified and multi-mechanistic, thus aligning with the complex nature of authentic learning environments as well as the multi-faceted model of human cognitive architecture contributed by CLT. The implications of the study have been discussed with reference to stakeholders including teachers, learners and educational institutions. Recommendations for future research include the ongoing development of CLEMS for the systematic implementation of CLT strategies at scale.