Adapting Smart Methodology for use with Subjective Rating Data

Decision making based on multiattribute utility theory involves analysis of complex decisions. Each possible course of action or choice is evaluated on a number of important characteristics, and the individual merit scores are then recombined into a single, global utility value for that option. This analytic approach reduces the demands placed on decision makers, and permits review and reconsideration of each step in the decision process. The Simple Multiattribute Rating Technique (SMART) is a method which employs uncomplicated data collection methods, yet produces results similar to more methodologically demanding techniques. Unfortunately, descriptions of SMART procedures do not describe a number of special tests and procedures that can be used with subjective rating data to improve their reliability and validity. A number of biases can occur in rating data, such as: lieniency and severity effects, the tendency for raters to make heavy use of one or the other extreme of rating scale values; sequential e...