Automated detection and analysis of social behaviors among preweaning piglets using key point-based spatial and temporal features

Abstract In the pig industry, the social behaviors of preweaning piglets are critical indicators of their liveability, growth, health, and welfare status. In this study, we developed a novel method based on convolutional neural networks (CNNs) that extracts high-quality spatiotemporal features to detect different types of social behaviors among preweaning piglets: snout-snout and snout-body social nosing, and snout-snout and snout-body aggressive/playing behavior. In our method, piglets were first detected and individually tracked using an integrated CNN-based network. Upon piglet detection, the key points of the piglet body were detected using a key point detector and a 1-second-tracking-unit strategy was adopted to reduce the influence of tracking errors and missing the detection of piglets. To detect suspected social behaviors in a tracking unit, we constructed a self-adaptive spatial affinity kernel function to represent the relationships between piglets, and pairwise piglets with spatial affinity higher than 0.7 were kept for further analysis, while the others were filtered out. Thereafter, high-quality spatiotemporal features consisting of the piglets’ spatial affinity and the corresponding linear motion and angular motion intensities were extracted and input into a support vector machine to hierarchically detect and classify piglet social behaviors. In addition, the proportions of the four detected social behaviors associated with the interaction points were analyzed in short video clips and a long video that covered the whole daytime. Our method achieved a favorable detection performance with a recall of 0.9687, a precision of 0.9309, and an F1 of 0.9494 in the short video clips, and a recall of 0.9576, a precision of 0.9187, and an F1 of 0.9377 in the 8-h long video episode. This shows the feasibility of the detection of social behaviors for piglets using CNNs.