A Comment on the PCAST Report: Skip the 'Match'/'Non-Match' Stage

This letter comments on the report “Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-comparison Methods” released in September 2016, by the President’s Council of Advisors on Science and Technology (PCAST). The report advocates a two-stage procedure for evaluation of forensic evidence. The first stage is a “match”/“non-match” determination, and the second stage is an empirical assessment of sensitivity (correct acceptance) and false alarm (false acceptance) rates. Almost always, quantitative data from feature-comparison methods are continuously-valued and have within-source variability. We explain why a two-stage procedure is not appropriate for this type of data, and recommend more appropriate statistical procedures.