Vision Based Upper Limbs Movement Recognition Using LSTM Neural Network

This paper presents the theoretical background and the implementation of a Long Short-Term Memory (LSTM) Neural Network architecture to recognize arm movements from video clips. The pose points (corresponding to the position of six body parts: shoulders, elbows and wrists) are extracted with a pre-trained Convolutional Pose Machine. Those points generate sequences over time with 66 (x, y) pairs, which are the input for a neural network, to classify them in 20 movement classes. Our architecture has 128 LSTM cells and presented \(92.5\%\) of accuracy on testing data and an execution time of around 6.64 ms.