FPGA-Based Gesture Recognition with Capacitive Sensor Array using Recurrent Neural Networks

This work presents a prototype of an FPGA-based hand motion recognition system using a capacitive sensor array (CSA). The prototype system is being developed as a tool to evaluate upper-limb motor skills for assistive or rehabilitative applications. A light-weight gesture segmentation algorithm was developed that uses summation and moving average filtering of quantized capacitive sensing data to segment motions. The time-series hand motions are then recognized through a recurrent classifier based on long short-term memory (LSTM) neural networks. The classifier model is trained on uni-stroke hand written digit ('0'–'9') samples obtained from four volunteers. A total of 12,000 hand motion samples are collected. The accuracy of 10-fold and leave-one-user-out cross-validation accuracy is respectively 97.5% and 91.3% using a two-layer LSTM network. The LSTM classifier is implemented on a Zynq FPGA device. The experiment demonstrated that the FPGA implementation of the LSTM-based classifier can achieve real-time gesture classification with capacitive sensor data.