"When Often Becomes Always, and Sometimes Becomes Never: Miscomprehension in Surveys"

There is ongoing concern within the literature about the quality of survey data used in research. Typically efforts have focussed on increasingly sophisticated analytic approaches to ameliorate these problems. This paper draws on linguistic theory to suggest that the process of item interpretation poses a considerable threat to construct validity. We explore three types of miscomprehension: instructional miscomprehension (where respondents do not follow instructions); sentential miscomprehension (where item syntax is inappropriately enriched or depleted) and lexical miscomprehension (where different meanings of words are employed). We report on two studies. The first combines open-ended questions and Likert scales to ascertain what respondents think survey items mean. The second study draws on the results of the first and allows respondents to select from among neutral items and those which have been either sententiallly or lexically miscomprehended. We find strong evidence for all three forms of miscompr...