Behavior classification of goats using 9-axis multi sensors: The effect of imbalanced datasets on classification performance

Abstract Recent developments of small electronic instruments have enabled the classification of animal behavior using simultaneous measurements of various bio-logging data such as acceleration, magnetism, and angular velocity. Following technological progress, studies on the behavioral classification of ruminants combining measurements based on accelerometers, magnetometers, and gyroscopes have received attention. However, while the issue of class imbalance has recently become a serious challenge in classification by machine learning, few behavioral classification studies on livestock animals have focused on the effect of equalizing the prevalence of each behavior to improve the problem caused by the imbalance of data on classification performance. The aims of this study were to classify the behaviors of goats using a back-mounted 9-axis multi sensor (a tri-axial accelerometer, a tri-axial gyroscope, and a tri-axial magnetometer) with machine learning algorithms, and to evaluate changes in the predictive scores by equalizing the prevalence of each behavior. The behaviors of three goats grazing on an experimental pasture were logged for approximately 12 h with the multi sensors. The behaviors were recorded at 1-second intervals with time-lapse cameras throughout the experimental period. Three behaviors were classified: lying, standing, and grazing. Over 100 different variables were extracted from the raw sensor data, and classification was executed by inputting the variables into two supervised machine learning algorithms: K-nearest neighbors (KNN) and decision tree (DT). Moreover, because the prevalence of standing was low compared to that of grazing, the number of observations of each behavior in the training datasets for classification models was equalized by undersampling. As expected, the results indicated that the overall accuracies of both algorithms using all variables derived from the three sensors were higher than those using only variables from the acceleration data. Furthermore, both the algorithms using the variables from the acceleration and magnetism data could classify the behaviors as accurate as the algorithms using variables from all sensor data. Balancing the prevalence of each behavior resulted in a decrease in the F1 scores of the lying and grazing classifications but a slight increase in those of the standing classification by DT. In conclusion, our results suggest that, in addition to tri-axial acceleration, tri-axial magnetism is useful for classifying lying, standing, and grazing activities of ruminants and that equalizing the number of data for each behavior is important to correctly assess the predictive accuracy of behavioral classifications, particularly for the behavior with low prevalence.