A Framework for Evidence-Based Licensure of Adaptive Autonomous Systems

Abstract : Adaptive autonomous systems of interest to DoD have great potential to complement human performance in a wide range of missions. Thisparticularly is true for adaptive systems that learnthose whose behavior on a given set of inputs may change over time, even after thesystem has been fielded. Such adaptation, however, makes exhaustive testing, certification, and licensure of the final system impossible.The challenge is to establish high confidence that the system will perform dependably and behave as intended while safely, securely,reliably, and effectively carrying out the assigned missions. This paper adapts approaches from other disciplines, such as softwareassurance theory, where safety as well as performance is paramount. The key features of the approach include taking and retainingperformance data from the beginning of development, establishing a formal means to assess actual performance compared with desiredperformance throughout the development, and recognizing that system testing will need to continue beyond a fielding decision. An additionalcomplexity is that performance testing needs to address how the autonomous modules make decisions and provide a standard for doingthat. That is, test, evaluation, verification, and validation of such systems must inform testers about how a system satisfies requirementsduring development and about its potential to make effective system changes after fielding.