Automatic Target Recognizer Performance Evaluation

Automatic target recognizer performance evaluationLloyd G. Clark, James C. Rachal, and Vincent J. VeltenMission Avionics Division, Air Force Wright Aeronautical Laboratories (AFWAL /AARF),Wright -Patterson Air Force Base, Ohio 45433AbstractThe evolution of automatic target recognizers (ATRs) and their application to militaryproblems has necessitated the development of new techniques for the evaluation of thesedevices. The evaluation of today's ATR must provide some measure of the expected perfor-mance in a real scenario. It is essential that the evaluation techniques evolve at thesame rate as the ATR technology and possess the same relative maturity. The AutomaticTarget Recognizer Working Group (ATRWG) Evaluation Committee was formed as a forum for theexchange of ideas and to lead in the development of ATR performance measures. This paperpresents the standards and procedures developed by the DOD and accepted by ATRWG's Evalu-ation Committee. Also included is a brief history of the evaluation of modern ATRs.Definitions of performance as well as testing procedures are outlined that are uniform,accurate, extendable, and usable for evaluation of ATRs. A scientific methodology isbeing developed for evaluating improvements in targeting effectiveness for various DODmissions and scenarios.IntroductionIn recent years, techniques for evaluating automatic target recognizer performance haveevolved toward the determination of the usefulness of the device to solve a military prob-