Real-time vision-based worker localization & hazard detection for construction

Abstract Despite training, construction workers often fail to recognize a significant proportion of hazards in construction environments. Therefore, there is a need for developing technology that assists workers and safety managers in identifying hazards in complex and dynamic construction environments. This study develops a framework for an automated system that detects hazardous conditions and objects in real-time to assist workers and managers. The framework consists of three independent pipelines for localization of workers, semantic segmentation of the visual scene around workers, and detection of static and dynamic hazards. The framework can be used to automate and augment the hazard detection ability of workers and safety managers in construction workplaces. In addition, the framework offers several computing contributions including an improved real-time worker localization method and an efficient architecture for integrating pipelines for entity localization and object detection. A system developed based on the proposed framework as a proof of concept and was tested in indoor and outdoor construction environments. It achieved over 93% accuracy.