Advancing statistical decision-making in sports science

Developed within sports science as an alternative to null hypothesis significance tests, the magnitude-based decisions (MBD) procedure aimed to emphasise effect sizes and discourage dichotomous decision-making. The use of MBD has been banned by some sports science journals following claims it lacks a theoretical foundation and leads to high Type I error rates. To address these claims, we first generalise contour-enhanced funnel plots to allow for ranges of meaningful effect sizes, then relate regions defined in these plots to the decisions made by MBD. Next, using a novel construction to accommodate testing of the one hypothesis at multiple levels, we mathematically show how MBD fits within a class of multiple decision procedures. We have implemented a theoretically sound version of MBD as a visualisation tool that supports the generalisation of funnel plots. The use of MBD could encourage researchers to plan test directionalities, test levels and error definitions, and the visualisation tool may help stakeholders engage with the design of analyses and interpretation of trial findings.