A novel dataset and deep learning-based approach for marker-less motion capture during gait

Abstract Background The deep learning-based human pose estimation methods, which can estimate joint centers position, have achieved promising results on the publicly available human pose datasets (e.g., Human3.6 M). However, these datasets may be less efficient for gait study, particularly for clinical applications, because of the limited number of subjects, their homogeneity (all asymptomatic adults), and the errors introduced by marker placement on subjects’ regular clothing. Research question How a new human pose dataset, adapted for gait study, could contribute to the advancement and evaluation of marker-less motion capture systems? Methods A marker-less system, based on deep learning-based pose estimation methods, was proposed. A new dataset (ENSAM dataset) was collected. Twenty-two asymptomatic adults, one adult with scoliosis, one adult with spondylolisthesis, and seven children with bone disease performed ten walking trials, while being recorded both by the proposed marker-less system and a reference system – combining a marker-based motion capture system and a medical imaging system (EOS). The dataset was split into training and test sets. The pose estimation method, already trained on the Human3.6 M dataset, was evaluated on the ENSAM test set, then reevaluated after further training on the ENSAM training set. The joints coordinates were evaluated, using Bland-Altman bias and 95% confidence interval, and joint position error (the Euclidean distance between the estimated joint centers and the corresponding reference values). Results The Bland-Altman 95% confidence intervals were substantially improved after finetuning the pose estimation method on the ENSAM training set (e.g., from 106.9 mm to 17.4 mm for the hip joint). With the new dataset and approach, the mean joint position error varied from 6.2 mm for ankles to 21.1 mm for shoulders. Significance The proposed marker-less system achieved promising results in terms of joint position errors. Future studies are necessary to assess the system in terms of gait parameters.