Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems

Automated and autonomous driving systems (ADS) are a transformational technology in the mobility sector. Current practice for testing ADS uses virtual tests in computer simulations; search-based approaches are used to find particularly dangerous situations, possibly collisions. However, when a collision is found, it is not always easy to automatically assess whether the ADS should have been able to avoid it, without relying on offline analyses by domain experts. In this paper, we propose a definition of avoidable collision that does not rely on any domain knowledge, but only on the fact that it is possible to reconFigure the ADS (in our case, the path planner component provided by our industry partner) in a way that the collision is avoided. Based on this definition, we propose two search-based approaches for finding avoidable collisions. The first one (named sequential approach), based on current industrial practice, first searches for a collision, and then searches for an alternative configuration of the ADS which avoids it. The second one (named combined approach), instead, searches at the same time for the collision and for the alternative configuration which avoids it. Experiments show that the combined approach finds more avoidable collisions, even when the sequential approach doesn’t find any; indeed, the sequential approach, in the first search, may find too severe collisions for which there is no alternative configuration that can avoid them.