Feedback-based object detection for multi-person pose estimation

Abstract In this paper, a novel method is proposed for increasing the performance through coupling of top-down models adjusting the object detector based on a new loss function. Generally, object detectors and keypoint estimators are sequentially used in real-time multi-person pose estimations; however, these two models are separately trained. Therefore, the results of the object detector are not optimized for the keypoint estimator. To solve this problem, we analyze the relationship between the two models and propose a feedback-based loss optimization in the object detector, based on the estimation results of the keypoint estimator. In addition, the resulting bounding box of the object detector is readjusted to improve the accuracy of the keypoint estimation model. The experimental results demonstrate that the proposed approach can perform real-time operations with a high frame rate similar to that of the baseline model. Moreover, it achieved an accuracy of 74.2 average precision (AP), which is higher than the state-of-the-arts model including the human detector used in the experiment.