End-to-end machine learning for experimental physics: using simulated data to train a neural network for object detection in video microscopy

We demonstrate a method for training a convolutional neural network with simulated images for usage on real-world experimental data. Modern machine learning methods require large, robust training data sets to generate accurate predictions. Generating these large training sets requires a significant up-front time investment that is often impractical for small-scale applications. Here we demonstrate a ‘full-stack’ computational solution, where the training data set is generated on-the-fly using a noise injection process to produce simulated data characteristic of the experimental system. We demonstrate the power of this full-stack approach by applying it to the study of topological defect annihilation in systems of liquid crystal freely-suspended films. This specific experimental system requires accurate observations of both the spatial distribution of the defects and the total number of defects, making it an ideal system for testing the robustness of the trained network. The fully trained network was found to be comparable in accuracy to human hand-annotation, with four-orders of magnitude improvement in time efficiency.