When Should a Tutor Intervene? Adaptive Hint Timing from Student Interaction Traces in Online Programming Exercises

Abstract

Providing hints too early can reduce productive struggle, while delayed assistance may cause students to disengage. This study investigates whether interaction traces can be used to determine when an online programming tutor should intervene. We develop an adaptive hint policy using recent compilation outcomes, code-editing activity, time since the last successful execution, and repeated error patterns to estimate when a learner is unlikely to make progress without assistance. The system was evaluated in a six-week controlled study involving 1,284 undergraduate students completing 38,760 programming exercises. Students were assigned to adaptive-hint, fixed-time-hint, or no-automatic-hint conditions. The adaptive group completed 81.6% of assigned exercises, compared with 74.2% and 69.8% in the fixed and no-hint groups, respectively. More importantly, students receiving adaptive hints requested 23.7% fewer direct solution reveals than those receiving fixed-time assistance. On a delayed assessment administered one week later, the adaptive group scored 6.4% higher than the fixed-hint group. Analysis of intervention timing shows that repeated compilation failures combined with declining edit activity were substantially more predictive of abandonment than elapsed time alone. These findings suggest that effective tutoring depends not only on the content of assistance but also on identifying appropriate moments for intervention.