Data presentation in rodent stroke studies and the predictive value of confidence intervals

The clinical failure of neuroprotective agents stems partly from inappropriate statistical presentation of preclinical data, which causes an overestimation of effect size and underpowered clinical studies. We searched for studies utilizing neuroprotective agents in a rodent middle cerebral artery occlusion model. We identified all experimental groups demonstrating statistically significant claims of neuroprotection within these studies and calculated the mean, 95% confidence intervals (CI), and meta-analyses of effect size for each agent. The lower limits of the CI (LLCI) of effect size were less than 0.2 in 161/221 (73%) of all experimental groups, corresponding to small effects. After meta-analysis, 29/60 (48%) and 11/18 (61%) of the agents had an effect size LLCI < 0.2 for infarct volume and neurological function, respectively. This difference was statistically significant (p < 0.05). These results suggest that the preclinical neuroprotective effect size of many of these drugs is small, although that of neurological function is smaller and is thus a more conservative and appropriate estimate of effect.