Aligning learning design and learning analytics through instructor involvement: a MOOC case study

Abstract

This paper presents the findings of a mixed-methods research that explored the potentials emerging from aligning learning design (LD) and learning analytics (LA) during the design of a predictive analytics solution and from involving the instructors in the design process. The context was a past massive open online course, where the learner data and the instructors were accessible for posterior analysis and additional data collection. Through a close collaboration with the instructors, the details of the prediction task were identified, such as the target variable to predict and the practical constraints to consider. Two predictive models were built: LD-specific model (with features based on the LD and pedagogical intentions), and a generic model (with cumulative features, not informed by the LD). Although the LD-specific predictive model did not outperform the generic one, some LD-driven features were powerful. The quantity and the power of such features were associated with the degree to which the students acted as guided by the LD and pedagogical intentions. The leading instructor’s opinion about the importance of the learning activities in the LD was compared with the results of the feature importance analysis. This comparison helped identify the problems in the LD. The implications for improving the LD are discussed.