Design of conversational agents for CSCL: comparing two types of agent intervention strategies in a university classroom

Abstract

The use of Conversational Agents (CAs) in computer-supported collaborative learning (CSCL) has shown promising results regarding students' productive dialogue and learning. Yet, limited work has explored the connection between the configuration of the CA behavior, the nature of the learning task, and the student behavior in authentic educational settings. In this work, we describe a pedagogical design space of CAs for collaborative learning composed of three dimensions: task design, domain model, and agent intervention strategies. We conduct an initial field study in a university classroom comparing two types of agent intervention strategies based on student participation, dialogue, and satisfaction. 54 university students worked in pairs in the same collaborative brainstorming task with a CA tool and were randomly assigned in two CA conditions with a) knowledge-based prompts to connect two domain concepts, b) social prompts to link their partners' contributions. The results show that students who received knowledge-based prompts significantly exchanged more messages with evidence of explicit reasoning and were more satisfied with the agent and their discussion during the task. Students from both conditions reported problems like the lack of context-awareness and timely interventions by the agent. We discuss the relation between the agent intervention strategies and the task design towards seeking design recommendations for CAs in CSCL.