Response order effects in the Youth Tobacco Survey: Results of a split-ballot experiment

Response order effects are changes in answers to close-ended questions that arise by varying the order of the response options (Krosnick and Alwin 1987). Two types of response order effects have been documented in previous studies: primacy effects and recency effects (Bishop and Smith 2001; Holbrook et al. 2007; Knauper 1999). Primacy effects, when response options presented earlier in a list of options are selected more often than ones at the end, have been typically observed in paper and pencil self-administered surveys (Krosnick and Alwin 1987). Recency effects, where response options presented later in a list are selected more often, primarily occur in surveys where questions are presented orally (Krosnick and Alwin 1987). Response order effects have been documented with adult respondents, but few studies have examined these effects with children or adolescents (Fuchs 2005). It is believed that no studies on response order effects have been conducted on a tobacco-specific survey. Questionnaires often include non-substantive response options, such as “no opinion” or “don't know,” to allow respondents who have no true opinions to select these options. People with lower education are more likely to be attracted to the “no opinion” response option (Krosnick et al. 2002). As a result, it is recommended that in children's surveys, researchers should minimize the use of non-substantive response categories (Bell 2007). However, in circumstances where it is not possible to avoid using non-substantive response options, no research has shown if the order of these response options has any effect on responses. We assess whether primacy and/or recency effects occur in a self-administered tobacco-related questionnaire among youth and will determine the effects of reversing the order of response options and specifically changing the position of a non-applicable response category. This study is one of very few studies to examine the effect of response order in children's responses, and it is the first to investigate the impact of moving a non-applicable response category from first to last position. To the best of our knowledge, it is also the first to document the presence of response order effects in a tobacco survey and differentiate the effects for tobacco users verses non-users.