Theoretical solutions for the evaluation of discrimination capacity of species distribution models

The discriminating capacity (i.e., ability to correctly classify presences and absences) of species distribution models (SDMs) is commonly evaluated with metrics such as the Area Under the Receiving Operating Characteristic Curve, the Kappa statistic and the True Skill Statistic (TSS). AUC and Kappa have been repeatedly criticised, but the TSS has fared relatively well since its introduction, mainly because it has been considered as independent of prevalence. In addition, discrimination metrics have been contested because they should be calculated on presence-absence data, but are often used on presence-only or presence-background data. With examples and simulations, we demonstrate here that TSS is misleading because of its dependence on prevalence. We then introduce an alternative set of metrics −based on similarity indices− that represents a theoretical solution to evaluate model discrimination capacity. In the ideal situation where species and sample prevalence (respectively, true and sampled ratio of sites occupied by species) are equal −corresponding to a perfectly random presence-absence sampling or evaluation of virtual species, similarity indices are appropriate to evaluate model discrimination capacity. In situations where sample and species prevalence are not equal −corresponding to non-random presence-absence samplings or presence-background samplings, similarity metrics can correctly evaluate discrimination capacity provided that an estimate of species prevalence can be obtained. However, estimates of species prevalence are challenging to obtain for presence-absence schemes and unlikely to be possible for presence-pseudoabsence schemes. Our recommendations are therefore to use similarity metrics in the specific case of virtual species, and, until a robust framework to estimate species prevalence is developed, focus on reliability metrics in other cases.