In practice, optimal screening designs for arbitrary run sizes are traditionally generated using the D-criterion with factor settings fixed at ±1, even when considering continuous factors with levels in [−1,1]. This article identifies cases of undesirable estimation variance properties for such D-optimal designs and argues that generally A-optimal designs tend to push variances closer to their minimum possible value. New insights about the behavior of the criteria are gained through a study of their respective coordinate-exchange formulas. The study confirms the existence of D-optimal designs comprised only of settings ±1 for both main effect and interaction models for blocked and unblocked experiments. Scenarios are also identified for which arbitrary manipulation of a coordinate between [−1,1] leads to infinitely many D-optimal designs each having different variance properties. For the same conditions, the A-criterion is shown to have a unique optimal coordinate value for improvement. We also compare how Bayesian versions of the A- and D-criteria balance minimization of estimation variance and bias. Multiple examples of screening designs are considered for various models under Bayesian and non-Bayesian versions of the A- and D-criteria.
In practice, optimal screening designs for arbitrary run sizes are traditionally generated using the D-criterion with factor settings fixed at ±1, even when considering continuous factors with levels in [−1,1]. This article identifies cases of undesirable estimation variance properties for such D-optimal designs and argues that generally A-optimal designs tend to push variances closer to their minimum possible value. New insights about the behavior of the criteria are gained through a study of their respective coordinate-exchange formulas. The study confirms the existence of D-optimal designs comprised only of settings ±1 for both main effect and interaction models for blocked and unblocked experiments. Scenarios are also identified for which arbitrary manipulation of a coordinate between [−1,1] leads to infinitely many D-optimal designs each having different variance properties. For the same conditions, the A-criterion is shown to have a unique optimal coordinate value for improvement. We also compare how Bayesian versions of the A- and D-criteria balance minimization of estimation variance and bias. Multiple examples of screening designs are considered for various models under Bayesian and non-Bayesian versions of the A- and D-criteria.
Weighted optimality criteria allow an experimenter to express hierarchical interest across estimable functions through a concise weighting system. We show how such criteria can be implicitly influenced by the estimable functions' minimum variances, leading to nonintuitive variance properties of the optimal designs. To address this, we propose a new optimality and evaluation approach that incorporates these minimum variances. A modified c-optimality criterion is introduced to calculate an estimable function's minimum variance while requiring estimability of all other functions of interest. These minimum variances are then incorporated into a standardized weighted A-criterion that has an intuitive weighting system. We argue that optimal designs under this criterion tend to satisfy the conditions of a new design property we call weight adherence that sets appropriate expectations for how a given weighting system will influence variance properties. A practical, exploratory approach is then described for weighted optimal design generation and evaluation. Examples of the exploratory approach and weight adherence are provided for two types of factorial experiments.
Introduction: Pruritus (or itch) research has gained momentum in the last decades and use of animal models to study itch behavior are a vital part of the research. Recent studies have found that many fields using animal models, including neuroscience, are predisposed toward using male animals in preclinical research. To address sex bias in animal research, the National Institutes of Health (NIH) began requiring researchers to include sex as a variable beginning in June 2015. Here, we test whether researchers studying itch are biased toward using males in preclinical research. Methods: The NIH’s PubMed database was searched for primary research articles written between August 2007 and December 2018 using the words “Itch” and “Pruritus.” The following information was extracted from articles fitting our inclusion criteria: type of itch (acute or chronic), the animal model and the sex of the animals used, and whether researchers considered sex as a variable. z -Tests, binomial tests, and the Cochran-Armitage test for trend were used to explore relationships between animal models and the usage of both sexes. Results: We found 5.3%±1.2% of papers in a given year used 1 of our 4 animal models. Mice were the most frequently used animal model, followed by rats, nonhuman primates, and dogs. Overall, researchers used male animals regardless of the animal model used. In preclinical research conducted on both male and female animals, sex was not considered a variable in a majority of these studies. Finally, since 2015, there has not been a change in the usage of male or female mice. Briefly, the incidence of papers utilizing both sexes has not changed. Discussion: We have found that itch researchers have a bias towards males in animal research. This bias has not changed since the NIH’s mandate to include sex as a variable in preclinical research.
Weighted optimality criteria allow an experimenter to express hierarchical interest across estimable functions through a concise weighting system. We show how such criteria can be implicitly influenced by the estimable functions’ minimum variances, leading to nonintuitive variance properties of the optimal designs. To address this, we propose a new optimality and evaluation approach that incorporates these minimum variances. A modified c-optimality criterion is introduced to calculate an estimable function’s minimum variance while requiring estimability of all other functions of interest. These minimum variances are then incorporated into a standardized weighted A-criterion that has an intuitive weighting system. We argue that optimal designs under this criterion tend to satisfy the conditions of a new design property we call weight adherence that sets appropriate expectations for how a given weighting system will influence variance properties. A practical, exploratory approach is then described for weighted optimal design generation and evaluation. Examples of the exploratory approach and weight adherence are provided for two types of factorial experiments.
In practice, optimal screening designs for arbitrary run sizes are traditionally generated using the D-criterion with factor settings fixed at +/- 1, even when considering continuous factors with levels in [-1, 1]. This paper identifies cases of undesirable estimation variance properties for such D-optimal designs and argues that generally A-optimal designs tend to push variances closer to their minimum possible value. New insights about the behavior of the criteria are found through a study of their respective coordinate-exchange formulas. The study confirms the existence of D-optimal designs comprised only of settings +/- 1 for both main effect and interaction models for blocked and un-blocked experiments. Scenarios are also identified for which arbitrary manipulation of a coordinate between [-1, 1] leads to infinitely many D-optimal designs each having different variance properties. For the same conditions, the A-criterion is shown to have a unique optimal coordinate value for improvement. We also compare Bayesian version of the A- and D-criteria in how they balance minimization of estimation variance and bias. Multiple examples of screening designs are considered for various models under Bayesian and non-Bayesian versions of the A- and D-criteria.
Null hypothesis significance testing is a statistical tool commonly employed throughout laboratory animal research. When experimental results are reported, the reproducibility of the results is of utmost importance. Establishing standard, robust, and adequately powered statistical methodology in the analysis of laboratory animal data is critical to ensure reproducible and valid results. Simulation studies are a reliable method for assessing the power of statistical tests, however, biologists may not be familiar with simulation studies for power despite their efficacy and accessibility. Through an example of simulated Harlan Sprague-Dawley (HSD) rat organ weight data, we highlight the importance of conducting power analyses in laboratory animal research. Using simulations to determine statistical power prior to an experiment is a financially and ethically sound way to validate statistical tests and to help ensure reproducibility of findings in line with the 4R principles of animal welfare.
The purpose of this article is to persuade experimenters to choose A-optimal designs rather than D-optimal designs for screening experiments. The primary reason for this advice is that the A-optimality criterion is more consistent with the screening objective than the D-optimality criterion. The goal of screening experiments is to identify an active subset of the factors. An A-optimal design minimizes the average variance of the parameter estimates, which is directly related to that goal. While there are many cases where A- and D-optimal designs coincide, the A-optimal designs tend to have better statistical properties when the A- and D-optimal designs differ. In such cases, A-optimal designs generally have more uncorrelated columns in their model matrices than D-optimal designs. Also, even though A-optimal designs minimize the average variance of the parameter estimates, various cases exist where they outperform D-optimal designs in terms of the variances of all individual parameter estimates. Finally, A-optimal designs can also substantially reduce the worst prediction variance compared with D-optimal designs.
