Distillation of a CNN for a high accuracy mobile face recognition system

In face recognition systems, the use of convolutional neural networks (CNNs) permits to achieve good accuracy performances, which derive largely from a huge number of well-trained parameters. While using online services any mobile device can suffice for an accurate identification, in the offline scenario, implemented on a wearable mobile hardware, it is difficult to achieve both real-time responsiveness and high accuracy. In this paper we present a solution to replace a large open source face recognizer network (provided as part of the dlib libraries), distilling its learned knowledge into a less demanding CNN. The former is used as an expert oracle that provides the targets, while the latter is trained on the same input image, following a regression approach. In addition to lightness, our CNN is trained to use smaller input images, naturally allowing the recognition of identities in a wider distance range and with a reduced amount of computation. This eventually permits the porting of the network into a dedicated mobile accelerating hardware. The hypothesis we want to demonstrate is that since the feature space topology has been deeply explored during the training of the expert network, and due to the fact that no information is created during the up sampling of a tiny face to the input size of the expert oracle, the smaller network can provide the same accuracy at a reduced computational cost.