Synthetic Thermal Image Generation for Human-Machine Interaction in Vehicles

Thermal infrared imaging holds promise for human-machine interaction in vehicles owing to superior performance in low-light and low-visibility conditions, and the potential for monitoring human psycho-physiological state. However, the shortage of large-scale 2D thermal image datasets and public benchmarks has hindered progress of deep-learning-based solutions. To tackle this problem, we develop a pipeline for creating a synthetic thermal image dataset. Firstly, 3D models of human heads are generated from uncalibrated TIR images (without additional visible or depth images) using photogrammetry techniques. A synthetic dataset of 100k images of 640×480 resolution are then generated by rendering each of the five 3D models for a range of head poses, camera positions and backgrounds using commercial animation software. The effectiveness of the approach is evaluated using a number of deep learning algorithms that may enable human-machine interaction such as head pose estimation and face detection. The neural networks are trained on the new synthetic thermal dataset, before fine tuning on real world data where possible.