Beyond Helmet Detection: Temporal Recognition of Unsafe Worker–Equipment Interactions in Industrial Video

Abstract

Many vision-based workplace safety systems focus on detecting personal protective equipment, while accidents frequently arise from unsafe interactions between workers and moving equipment. This paper studies the recognition of such interactions from continuous industrial video. We construct a temporal detection pipeline that tracks workers and equipment, estimates their relative motion, and identifies patterns including unsafe approach, restricted-zone entry, vehicle crossing, and prolonged proximity to operating machinery. Evaluation is performed on 1,920 hours of video from 47 cameras across three industrial facilities, containing 8,614 manually annotated unsafe interaction events. The proposed method achieves an event-level F1-score of 87.9%, compared with 71.6% for frame-based object detection with rule-defined safety zones and 82.3% for a video classification baseline. Incorporating relative trajectory information reduces false alarms from workers merely passing behind equipment by 41.2%. At a processing rate of 27.8 FPS on a single edge GPU, the system supports near-real-time analysis of 1080p camera streams. Performance remains above 84% F1 under low-light and partially occluded conditions. The results demonstrate that modeling worker–equipment interactions over time provides a more reliable basis for industrial safety monitoring than detecting isolated objects or zone violations alone.