Increased Power Through Design and Analysis
5
Citation
16
Reference
10
Related Paper
Citation Trend
Abstract:
This study was suggested by the frequent encounters of the authors with beginning researchers who tend to avoid the more powerful research designs and statistical procedures such as the Solomon Four-Group design and analysis of covariance. The combination of an effective design and a powerful statistical analysis technique resulted in a significant finding for only 44 learning disabled students. Researchers are encouraged to seek pre-existing or easily obtainable covariates to incorporate into the statistical treatment whenever added power is required.Keywords:
Power analysis
Statistical Analysis
Analysis of covariance
Statistical power
Research Design
This study examined the use of the Principal Component Analysis (PCA) approach in incorporating covariates in the Analysis of Covariance (ANCOVA) under different experimental setups. Simulated data were used for the study, and the statistical programs were developed in R statistical software to generate the datasets for different experimental setups by varying (i) number of covariates, (ii) degree of correlation among covariates, (iii) number of treatments, (iv) difference between treatment means, and (v) number of replicates. Thousand simulations were performed for each experimental setup, and the impact of the PCA approach was assessed by means of power of the test through the proportion of rejections of H0: no difference between adjusted treatment means, in 1000 simulations. The use of PCs led to a significant gain in the power of the test in ANCOVA when there is a higher number of interrelated covariates with a limited number of observations. The impact was higher with the increase of number of covariates as well as the correlation between covariates. It can be concluded that by accommodating covariates by means of PCs, the efficiency in ANCOVA can be increased, especially if there are many covariates to be included in the analysis with a limited number of observations.
Analysis of covariance
Statistical power
Statistical Analysis
Cite
Citations (1)
Background Educational and developmental psychologists often examine how groups change over time. Two analytic procedures – analysis of covariance (ANCOVA) and the gain score model – each seem well suited for the simplest situation, with just two groups and two time points. They can produce different results, what is known as Lord's paradox. Aims Several factors should influence a researcher's analytic choice. This includes whether the score from the initial time influences how people are assigned to groups. Examples are shown, which will help to explain this to researchers and students, and are of educational relevance. It is shown that a common method used to measure school effectiveness is biased against schools that serve students from groups that are historically poor performing. Methods and results The examples come from sports and measuring educational effectiveness (e.g., for teachers or schools). A simulation study shows that if the covariate influences group allocation, the ANCOVA is preferred, but otherwise, the gain score model may be appropriate. Regression towards the mean is used to account for these findings. Conclusions Analysts should consider the relationship between the covariate and group allocation when deciding upon their analytic method. Because the influence of the covariate on group allocation may be complex, the appropriate method may be complex. Because the influence of the covariate on group allocation may be unknown, the choice of method may require several assumptions.
Analysis of covariance
Relevance
Cite
Citations (4)
We propose to discuss at length several examples from standard text books. All of these examples deal with analysis of covariance (ANCOVA) models and related analyses of data. We intend to capitalize on our understanding of optimal covariate designs (OCDs) in different ANCOVA models and re-visit these examples with a view to suggest optimal/nearly optimal designs for estimation of the covariate parameter(s). As we will see, for some examples our task is very much routine but for others, it is indeed a highly non trivial exercise. We intent to cover a total of six examples—divided in two parts. This is Part I—dealing with two examples.
Analysis of covariance
Cite
Citations (1)
Abstract Analysis of covariance (ANCOVA) is a statistical method that may be viewed as an extension of analysis of variance (ANOVA) when, in addition to one or more factors (discrete explanatory variables, typically group membership), it is required to account for possible differences due to a continuous variable(s), usually called covariate(s) or concomitant variable(s). The latter variable(s) co‐varies with a dependent variable under consideration (response, outcome), and it is of interest to examine whether group differences on the latter may be related to group differences on the covariate(s). Typically, a covariate is highly correlated with a response variable; that is, the covariate contains information about the response variable and therefore possibly also about group differences on the outcome measure(s). Empirical settings in which ANCOVA is appropriate usually have at least one categorical factor and one or more continuous covariates.
Analysis of covariance
Categorical variable
Variables
Cite
Citations (0)
Analysis of Coy ariance (ANCOVA) is a data analysis method that is often employed to control extraneous sources of variation in non-equivalent group designs. It is commonly belie% ed that so long as the covariate is highly correlated with the dependent variable there is nothing to lose in employing ANCOVA, ev en in non-randomized studies. This paper examines some of the that lead to successful and unsuccessful criterion source adjustments, and demonstrates that under certain circumstances, may perform in a manner antithetical to its intended purpose. CONFOUNDING COVARIATES IN NONRANDOMIZED STUDIES INTRODUCTION The analysis of covariance (ANCOVA), as employed in educational research practice, is routinely used for one or both of two purposes. The first of these purposes is to attain an increase in the power of a statistical test. As an example, a researcher might randon-.1y assign students to various treatment groups with subsequent outcome measures being evaluated by means of an analysis of variance (ANOVA). In the event that ancillary information pertaining to the students is available in the form of measures that (a) correlate with the outcome measures and (b) do not reflect treatment effects, then the power of the ANOVA test may be augmented by the introduction of a covariate, similar to the use of blocks in different design contexts. Under random assignment covariate scores for students in various treatment groups are sampled from identical populations. Covariates are also commonly employed to adjust criterion measures so as to ameliorate group differences that are unrelated to treatments. That is, the second use of is to control an extraneous variable. For example, an educational researcher may be unable to randomly assign students to treatments and subsequently become aware of differences between the groups in terms of intellectual ability. In the event that outcome measures are related to intellectual ability (e.g., scores on a reading test), then the researcher might employ IQ scores as a covariate in the model in order to control for group differences in intellectual ability. Unlike the first use of ANCOVA, in this instance groups do differ on the covariate measure and it is precisely because of this difference that the covariate is used. For a further discussion on this topic and related issues, see Cochran (1957, 1970), Elashoff (1969), Evans and Anastasio (1968). Fisher (i932), Harris (1963), Levin and Subkoviak (1977), Linn (1981), and Lord (1960). The use of covariates is not a substitute for random assignment of experimental units. but its proliferation is apparently promoted by the belief that covariates in non-equivalent group designs can only improve the level of precision in the data analysis. The argument contends that, at best, in nonrandom assignment will control at least some of the sources of extraneous variation, and at worst, will not be biased from traditional ANOVA results. This in turn can only lead to greater confidence in the validity of results than would have been realized had the covariate not been employed in the model. While it has been clearly pointed out that, ANCOVA provides the appropriate adjustment only under a very limited set of conditions (Porter and Raudenbush, 1987, P. 390) and randomization is a primary condition, the propensity of usage in nonrandomized studies in education suggests that it is not commonly known that under certain circumstances the use of ANCOV A will result in the introduction of eA:traneou.s influences into the analysis. Not only does the fail to provide precision in this situation, but it will operate in a manner antithetical to its purpose. PURPOSE OF THIS PAPER The purpose of this paper is to explicate and focus attention on the problem of unsuccessful adjustments made in data analysis through misuse of covariates. It will be helpful to review some basic assumptions of psychometric test score theory. This background will form the basis for the subsequent discussion. Then, examples of successful and unsuccessful criterion score adjustments in situations where the null hypothesis of no treatment effect, as well as in situations where the null hypothesis is false, will be presented. CLASSICAL TEST SCORE THEORY Classical test score theory conceives of the raw score earned by a student on a given test as a function of two basic components, expressed as:
Analysis of covariance
Repeated measures design
Cite
Citations (0)
An ATI Analysis is an analysis of covariance with an interaction. In the previous chapter, we looked at a two or three-factor crossed ANOVA. This test involves one or more covariates, whereby one of the covariates may be an interaction between two variables. The test involves one or more fixed effects and one or more covariates. A covariate is a latent variable that may indirectly impact the significance of a model.
Analysis of covariance
Interaction
Variables
Cite
Citations (0)
Analysis of Covariance (ANCOVA) is a technique that is frequently used in neuroimaging studies to control for covariates. An assumption of ANCOVA is that the between-subjects factor and the covariate are independent. In some observational studies in the neuroimaging literature, this assumption is violated. The question that these studies attempt to answer is what the difference would be between group means on the dependent variable if the group means on the covariate were equal. However, when there is a dependency between the between-subjects factor and the covariate, then correcting for differences between groups on the covariate may misrepresent the factor and distort its definition. Moreover, the situation where all subjects have equal scores on the covariate may be unrealistic. If the assumption of independence is violated, there are several procedures to follow. Generally, it is of crucial importance to consider the question what it means to correct for a variable that has a relationship with the factor under study. In case of an observational study, ANCOVA does not facilitate the estimation of the causal effect of the between-subjects factor. When two variables are related, there is no statistical method available to correct for this relationship.
Analysis of covariance
Independence
Cite
Citations (3)
Analysis of covariance (ANCOVA) is commonly used in psychiatric research to statistically adjust for illness severity when making group comparisons. A fundamental assumption of the procedure is that the groups' differences in outcome are constant across all levels of baseline severity. This is the classic assumption of equal linear slopes. The plausibility of this assumption is rarely reported in manuscripts that use an ANCOVA. A statistical model is discussed that can be used to test the assumption, and if the assumption is not fulfilled, the model can incorporate a covariate by group interaction. An application of this model, the multiple linear regression approach to analysis of covariance, is provided using data from a large clinical trial for patients with panic disorder. The authors recommend that manuscripts reporting results from an ANCOVA also report the results of the test of the covariate by group interaction and account for the interaction when the slopes are heterogeneous.
Analysis of covariance
Cite
Citations (11)
Analysis of covariance (ANCOVA) assesses group differences on a dependent variable (DV) after the effects of one or more covariates are statistically removed. By utilizing the relationship between the covariate(s) and the DV, ANCOVA can increase the power of an analysis. MANCOVA is an extension of ANCOVA to relationships where a linear combination of DVs is adjusted for differences on one or more covariates. The adjusted linear combination of DVs is the combination that would be obtained if all participants had the same scores on the covariates. That is, MANCOVA is similar to MANOVA, but allows a researcher to control for the effects of supplementary continuous IVs, termed covariates. In an experimental design, covariates are usually the variables not controlled by the experimenter, but still affect the DVs. Consequently, although not as effective as random assignment, including covariates may reduce both systematic and within-group error by equalizing groups being compared on important characteristics. This chapter discusses the assumptions of MANCOVA, sample size requirements, and strengths and limitations of MANCOVA. An annotated example is also provided.
Analysis of covariance
Cite
Citations (1)
Four misconceptions about the requirements for proper use of analysis of covariance (ANCOVA) are examined by means of Monte Carlo simulation. Conclusions are that ANCOVA does not require covariates to be measured without error, that ANCOVA can be used effectively to adjust for initial group differences that result from nonrandom assignment which is dependent on observed covariate scores, that ANCOVA does not provide unbiased estimates of true treatment effects where initial group differences are due to nonrandom assignment which is dependent on the true latent covariable if the covariate contains measurement error, and that ANCOVA requires no assumption concerning the equality of within-groups and between-groups regression. Where treatments actually influence covariate scores, the hypothesis tested by ANCOVA concerns a weighted combination of effects on covariate and dependent variables.
Analysis of covariance
Cite
Citations (19)