Analysis of neural network takeover-time predictions for shared-control autonomous driving

Autonomous driving systems may encounter situations where it is necessary to transfer control to the human driver, for instance when encountering unpredictable dangerous road conditions. To be able to do so safely, the autonomous system needs an estimate of how long it will take for the human driver to take control of the vehicle. A neural network can be used for making such predictions. However ensuring that such a neural network can be used in safety-critical situations is very challenging. We discuss our recent efforts for building, analysing and formally verifying a neural network built for predicting takeover time in a shared-control autonomous driving system. The network was trained on data collected from a (semi-) autonomous driving simulator. We evaluated several techniques for the analysis of the neural network as follows. We performed robustness and sensitivity analysis for the neural network, using the Marabou formal verification tool. We evaluated off-the-shelf attribution tools to determine the important features upon which the neural network makes its predictions. We investigated trust and confidence analysis to better understand the neural network outputs. And finally, we performed adversarial training to improve the quality of the neural network. We discuss our results and outline directions for future work.