Strategic Bias in Choice Experiments

A claimed advantage of choice experiments (CE) over other stated preference techniques is that they are less susceptive to strategic behaviour (SB); however, this assertion has yet to be extensively tested (Hanley et al. (2001)). While the presence of strategic behaviour has been explored in the context of contingent valuation studies (Bohm (1972), Posavac (1998), Whittington et al. (1990)), there is little evidence to support the claim that strategic bias is mitigated through choice experiments. Strategic behaviour occurs when “respondents deliberately misrepresent their preferences in order to influence the decision making process” (Hanley, Mourato, & Wright, 2001). This behavioural inefficiency can lead to incorrect inference. Furthermore, it is an issue for policy if respondents misrepresenting their true preferences are influencing decisions, but strategic behaviour reveals information about true preferences, as long as one can identify when that is occurring and how to account for it (Carson and Groves 2007).  This research is a first step in that direction. We know of two papers that explicitly evaluate SB in the CE context.  Burton (2010) runs an induced value CE where respondents are rewarded for their ability to distort the parameter values within the subsequently estimated conditional logit model.  Carson et al. (2015) develop an induced-value lab experiment, where the CE is treated as a referendum across 3 alternatives, and the respondent’s payout depends on which choice is in the majority. This clearly violates the Gibbard-Sattherwaite necessary condition for incentive compatibility, and hence SB results may ensue. Although both of these approaches benefits from using real payments conditional upon choices, the context within which the CE is framed is far removed from how CEs are typically used.  In this paper we seek to investigate SB within a more realistic representation of the end use of the CE in the decision process. We use respondents’ perception of this end use as the root of their motivation to act strategically. In this consideration, strategic behaviour arises when they consider the choice experiment as a mechanism to elicit preferences for a finite set of policy/management contenders. This tournament between discrete policy outcomes is not one of the choice sets presented to respondents in the CE, but is the one that the respondent aims to influence by preference misrepresentation in the choice experiment. We use a Monte Carlo simulation to estimate the effect of strategic behaviour when respondents believe that the choice experiment will be used to elicit preferences for some set of policy alternatives. Two classes of respondents are defined, differing by their utility functions.  One of the classes has an incentive to act strategically due to their expectation of their preferred outcome “losing” in the final tournament. This is a form of the Gibbard-Sattherwaite result, but the mechanism is mediated by the estimation of preferences rather than an explicit referendum.  We simulate “strategic bias” decision rules the respondents can implement to secure their second-best outcome. We reveal that only a small percentage of the sample population must exhibit strategic behaviour for the preferred policy outcome to switch from one which is probabilistically most preferred to one which is inferior, but which represents a net gain for those acting strategically. We emulate this simulated environment in an induced value lab experiment. This experiment outlines three possible policy alternatives and respondents are told to use the CE to increase the probability, ex post, that one of these alternatives is most preferred. We find that respondents are able to conceptualise the task and implement decision rules based on the final set of policy alternatives. Furthermore, over 20% of respondents misrepresented their preferences and revealed evidence of strategic bias. The findings suggest that respondents can successfully influence policy outcomes through their choices in the CE. Results from the simulation and induced value lab experiment illustrate three important findings: (i) their exists decision rules based on the final tournament which respondents can use in the CE to influence the policy outcome; (ii) only small percentages of the population are needed to act in a plausibly strategic manner to induce this “policy switch”; and (iii) real respondents, with real incentives to do so, are able to conceptualise the “strategic bias” task and implement it. On-going research will explore how sensitive the results are to (i) respondents’ misperception of the final policy tournament, (ii) experimental designs generated using different criteria; and (iii) varying the conditions in which a respondent chooses to behave strategically.