Variation in recombination rate affects detection of FST outliers under neutrality

Genome scans can potentially identify genetic loci involved in evolutionary processes such as local adaptation and gene flow. Here, we show that recombination rate variation across a neutrally evolving genome gives rise to different sampling distributions of mean Fst, a common population genetic summary statistic. In particular, we show that in low recombination regions the distribution of Fst estimates may have a longer tail than in more highly recombining regions. Determining outliers from the genome-wide distribution without accounting for local recombination rate may therefore increase the frequency of false positives in low recombination regions and be overly conservative in more highly recombining ones. We perform genome-scans on simulated and empirical Drosophila melanogaster datasets and, in both cases, find patterns consistent with this neutral model. Our results highlight a flaw in the design of genome scan studies and suggest that without estimates of local recombination rate, it is very difficult to interpret the genomic landscape of population genetic summary statistics.