Best–worst scaling

Best–worst scaling (BWS) techniques involve choice modelling (or discrete choice experiment – 'DCE') and were invented by Jordan Louviere in 1987 while on the faculty at the University of Alberta. In general with BWS, survey respondents are shown a subset of items from a master list and are asked to indicate the best and worst items (or most and least important, or most and least appealing, etc.). The task is repeated a number of times, varying the particular subset of items in a systematic way, typically according to a statistical design. Analysis is typically conducted, as with DCEs more generally, assuming that respondents makes choices according to a random utility model (RUM). RUMs assume that an estimate of how much a respondent prefers item A over item B is provided by how often item A is chosen over item B in repeated choices. Thus, choice frequencies estimate the utilities on the relevant latent scale. BWS essentially aims to provide more choice information at the lower end of this scale without having to ask additional questions that are specific to lower ranked items. Best–worst scaling (BWS) techniques involve choice modelling (or discrete choice experiment – 'DCE') and were invented by Jordan Louviere in 1987 while on the faculty at the University of Alberta. In general with BWS, survey respondents are shown a subset of items from a master list and are asked to indicate the best and worst items (or most and least important, or most and least appealing, etc.). The task is repeated a number of times, varying the particular subset of items in a systematic way, typically according to a statistical design. Analysis is typically conducted, as with DCEs more generally, assuming that respondents makes choices according to a random utility model (RUM). RUMs assume that an estimate of how much a respondent prefers item A over item B is provided by how often item A is chosen over item B in repeated choices. Thus, choice frequencies estimate the utilities on the relevant latent scale. BWS essentially aims to provide more choice information at the lower end of this scale without having to ask additional questions that are specific to lower ranked items. Louviere attributes the idea to the early work of Anthony A. J. Marley in his PhD thesis, who together with Duncan Luce in the 1960s produced much of the ground-breaking research in mathematical psychology and psychophysics to axiomatise utility theory. Marley had encountered problems axiomatising certain types of ranking data and speculated in the discussion of his thesis that examination of the 'inferior' and 'superior' items in a list might be a fruitful topic for future research. The idea then languished for three decades until the first working papers and publications appeared in the early 1990s. The definitive textbook describing the theory, methods and applications was published in September 2015 (Cambridge University Press) by Jordan Louviere (University of South Australia), Terry N Flynn (TF Choices Ltd.) and Anthony A. J Marley (University of Victoria and University of South Australia). The book brings together the disparate research from various academic and practical disciplines, in the hope that replication and mistakes in implementation are avoided. The three authors have (individually and together) already published many of the key academic peer-reviewed articles describing BWS theory, practice, and a number of applications in health, social care, marketing, transport, voting, and environmental economics. However, the method has now become popular in the wider research and practitioner communities, with other researchers exploring its use in areas as diverse as student evaluation of teaching, marketing of wine, quantification of concerns over ADHD medication, the importance of environmental sustainability, and priority-setting in genetic testing. There are two different purposes of BWS – as a method of data collection, and/or as a theory of how people make choices when confronted with three or more items. This distinction is crucial, given the continuing misuse of the term maxdiff to describe the method. As Marley and Louviere note, maxdiff is a long-established academic mathematical theory with very specific assumptions about how people make choices: it assumes that respondents evaluate all possible pairs of items within the displayed set and choose the pair that reflects the maximum difference in preference or importance.