Evaluating the usage of fault localization in automated program repair: an empirical study

Fault localization techniques are originally proposed to assist in manual debugging by generally producing a rank list of suspicious locations. With the increasing popularity of automated program repair, the fault localization techniques have been introduced to effectively reduce the search space of automated program repair. Unlike developers who mainly focus on the rank information, current automated program repair has two strategies to use the fault localization information: suspiciousness-first algorithm (SFA) based on the suspiciousness accuracy and rank-first algorithm (RFA) relying on the rank accuracy. However, despite the fact that the two different usages are widely adopted by current automated program repair and may result in different repair results, little is known about the impacts of the two strategies on automated program repair. In this paper we empirically compare the performance of SFA and RFA in the context of automated program repair. Specifically, we implement the two strategies and six well-studied fault localization techniques into four state-of-the-art automated program repair tools, and then use these tools to perform repair experiments on 60 real-world bugs from Defects4J. Our study presents a number of interesting findings: RFA outperforms SFA in 70.02% of cases when measured by the number of candidate patches generated before a valid patch is found (NCP), while SFA performs better in parallel repair and patch diversity; the performance of SFA can be improved by increasing the suspiciousness accuracy of fault localization techniques; finally, we use SimFix that deploys SFA to successfully repair four extra Defects4J bugs which cannot be repaired by SimFix originally using RFA. These observations provide a new perspective for future research on the usage and improvement of fault localization in automated program repair.