Deep 3D face recognition using 3D data augmentation and transfer learning

Abstract. Deep convolutional neural networks (DCNNs) have achieved humancomparable performance on challenging 2D face databases, outperforming all previous shallow methods. However, current 3D face recognition research still focuses on non-deep-learning methods due to the lack of large-scale 3D face databases. To address this, this paper proposes new 3D data augmentation methods: pose-based and channel-based augmentation. Experiments on three databases show that deep convolutional neural networks can be used effectively for 3D face recognition. Using a pose-augmented training set, an eight-layer convolutional neural network achieved 100%, 99% and 99% of validation accuracy on Photoface-10, FRGC-10 and Photoface-50 databases respectively. Also, successful channel-based augmentation, with five more artificial sessions per a three-channel image, showed that feature extraction in DCNNs is channel-invariant. It was found that transfer learning from a pre-trained deep neural network (e.g. VGG19) works for 3D face recognition, by fine-tuning the last few fully connected layers using 3D facial scans. The performance of a bespoke DCNN was compared to a fine-tuned pre-trained DCNN. The bespoke DCNN could achieve competitive performance if enough training data were provided; however, transfer learning from a pre-trained model provides an advantage in training time, being at least 3 times faster