Effect of imperfect detection on the estimation of niche overlap between two forest dormice

Abstract: Quantification of niche overlap represents an important topic in several aspects of ecology and conservation biology, although it could be potentially affected by imperfect detection, i.e., failure to detect a species at occupied sites. We investigate the effect of imperfect detection on niche overlap quantification in two arboreal rodents, the edible dormouse (Glis glis) and the hazel dormouse (Muscardinus avellanarius). For both species, we used Generalized Linear Mixed Models (GLMM) to estimate the occurrence probability and Occupancy Models (OM) to calculate occurrence and detection probabilities. By comparing these predictions through niche equivalency and similarity tests, we first hypothesised that methods correcting for imperfect detection (OM) provide a more reliable estimate of niche overlap than traditional presence/ absence methods (GLMM). Furthermore, we hypothesised that GLMM mainly estimate species detectability rather than actual occurrence, and that a low number of sampling replicates provokes an underestimation of species niche by GLMM. Our results highlighted that GLMM-based niche overlap yielded significant outcomes only for the equivalency test, while OM-based niche overlap reported significant outcomes for both niche equivalency and similarity tests. Moreover, GLMM occurrence probabilities and OM detectabilities were not statistically different. Lastly, GLMM predictions based on single sampling replicates were statistically different from the average occurrence probability predicted by GLMM over all replicates. We emphasized how accounting for imperfect detection can improve the statistical significance and interpretability of niche overlap estimates based on occurrence data. Under a habitat management perspective, an accurate quantification of niche overlap may provide useful information to assess the effects of different management practices on species occurrence.