Orchestration Load Indicators and Patterns: In-the-Wild Studies Using Mobile Eye-Tracking

Abstract

Orchestration load is the effort a teacher spends in coordinating multiple activities and learning processes. It has been proposed as a construct to evaluate the usability of learning technologies at the classroom level, in the same way that cognitive load is used as a measure of usability at the individual level. However, so far this notion has remained abstract. In order to ground orchestration load in empirical evidence and study it in a more systematic and detailed manner, we propose a method to quantify it, based on physiological data (concretely, mobile eye-tracking measures), along with human-coded behavioral data. This paper presents the results of applying this method to four exploratory case studies, where four teachers orchestrated technology-enhanced face-to-face lessons with primary, secondary school, and university students. The data from these studies provide a first validation of this method in different conditions, and illustrate how it can be used to understand the effect of different classroom factors on orchestration load. From these studies, we also extract empirical insights about classroom orchestration using technology.