This paper aims to investigate students’ behavioral engagement(On-
Task vs. Off-Task) in authentic classrooms. We propose a two-phased approach
for automatic engagement detection: In Phase 1, contextual logs are utilized to
assess active usage of the content platform. If there is active use, the appearance
information is utilized in Phase 2 to infer behavioral engagement. Through
authentic classroom pilots, we collected around 170 hours of in-the-wild data
from 28 students in two different classrooms using two different content plat-
forms (one for Math and one for English as a Second Language (ESL)). Our data
collection application captured appearance data from a 3D camera and context
data from uniform resource locator (URL) logs. We experimented with two test
cases: (1) Cross-classroom, where trained models were tested on a different
classroom’s data; (2) Cross-platform, where the data collected in different
subject areas (Math or ESL) were utilized in training and testing, respectively.
For the first case, the behavioral engagement was detected with an F1-score of
77%, using only appearance. Incorporating the contextual information improved
the overall performance to 82%. For the second case, even though the subject
areas and content platforms changed, the proposed appearance classifier still
achieved 72% accuracy (compared to 77%). Our experiments proved that the
accuracy of the proposed model is not adversely impacted considering different
set of students or different subject areas.