Prediction and localization of student engagement in the wild

Abstract

Digital revolution has transformed the traditional teaching procedures, students are going online to access study materials. It is realised that analysis of student engagement in an e-learning environment would facilitate effective task accomplishment and learning. Well known social cues of engagement/disengagement can be inferred from facial expressions, body movements and gaze patterns. In this paper, student’s response to various stimuli (educational videos) are recorded and cues are extracted to estimate variations in engagement level. We study the association of a subject’s behavioral cues with his/her engagement level, as annotated by labelers. We have localized engaging/non-engaging parts in the stimuli videos using a deep multiple instance learning based framework, which can give useful insight into designing Massive Open Online Courses (MOOCs) video material. Recognizing the lack of any publicly available dataset in the domain of user engagement, a new ‘in the wild’ dataset is curated. The dataset: Engagement in the Wild contains 264 videos captured from 91 subjects, which is approximately 16.5 hours of recording. Detailed baseline results using different classifiers ranging from traditional machine learning to deep learning based approaches are evaluated on the database. Subject independent analysis is performed and the task of engagement prediction is modeled as a weakly supervised learning problem. The dataset is manually annotated by different labelers and the correlation studies between annotated and predicted labels of videos by different classifiers are reported. This dataset creation is an effort to facilitate research in various e-learning environments such as intelligent tutoring systems, MOOCs, and others.