Reconsidering the impact of behavior prediction questions on illegal drug use: The importance of using proper analytic methods

Social scientists often habitually employ ANOVA methods when analyzing data from experiments when other analytic approaches are required instead. This paper illustrates how traditional analytic approaches can lead to incorrect research conclusions by reanalyzing data from a recent study by Williams, Block, and Fitzsimons (2006a). Because the non‐negative dependent variable (illegal drug use) was super skewed and had a large majority of zero values, the use of improper statistical tests and the presence of just a few extreme, outlying observations produced the illusion that asking people to predict their likelihood of drug use increased that behavior significantly, when in fact it did not. The effect of behavior prediction questions on frequency of exercise also turns out to be non‐significant when analyzed properly. As this example illustrates, experimental researchers should choose and implement appropriate analytic approaches carefully. Jon Krosnick is University Fellow at Resources for the Future. We t...