Can Performance Raters Be More Accurate? Investigating the Benefits of Prior Knowledge of Performance Dimensions

Performance appraisals are not and probably will never be infallible reflections of human behavior. They are prone to many sources of error, most of which are due to raters' cognitive distortions (Landy and Farr, 1980; Vance et al., 1983). A significant amount of performance appraisal research has focused on how raters process information, including how information is acquired, how it is stored and organized in memory and how it is retrieved and integrated into performance evaluations (Ilgen et al., 1993). Much of this research has relied on theories that can be considered "limited capacity" theories of information processing (Lord and Maher, 1990), in that they concentrate on the cognitive shortcuts that individuals take to reduce or simplify a large amount of information in a limited amount of cognitive space. When raters use inappropriate simplifications, such as relying on general impressions of persons that are not related to relevant performance dimensions, rating errors occur. This can cause unfortunate consequences in organizations. Inaccurate decisions regarding performance may ultimately affect the productivity of the organization, and on an individual level, inadequate judgments can be devastating. The purpose of this paper is to describe a study investigating some of these simplification processes. A recent line of research addresses "on line" and "memory based" processing. Hastie and Park (1986) describe on line processing as judgments made from "working memory" at the time the information is encoded. When we are asked to serve behavior in order to evaluate performance, on line processing is engaged. As we observe the person's behavior, we immediately interpret that behavior in accordance to relevant performance criteria. Memory based processing, on the other hand, could be compared to a "traditional" model of judgment (Borman, 1978), wherein bits of information are encoded directly into long-term memory and later, when a performance judgment is required, these memories are accessed to evaluate the behavior. On line and memory based processing differ in fundamental ways. On line processing, since it manages a large amount of information to make immediate judgments, will rely on impressions and other shortcuts in order to handle the incoming information. Thus, raters using on line processing will be more likely to depend on dimensions of expected behavior in order to group information into meaningful patterns. Memory based processors, since they do not need to organize the incoming information in order to make an immediate judgment, will tend to record information into long-term memory in noncategory based configurations. Therefore, on line processors will be more likely to remember information that is relevant to the person judgment being made. However, since, they are organizing information around a specific conception of behavior, they may remember fewer discrete behaviors than memory based processors, who are not constrained by a cognitive category. In performance appraisal research, on line processing has been initiated by telling raters in advance that they must evaluate the behavior of those they will observe. Memory based processing has been introduced when the rating task was not known until after the behavior had been observed. Recent research has studied these two processing modes in relationship to performance appraisal. In one study (Murphy et al., 1989), researchers informed one group of subjects that their primary task in viewing a videotape of a college professor's lecture was to evaluate his performance. Another group was told that although they would have to rate the lecturer's performance, their main task was to prepare for an exam over the content of the lecture. This study found that raters observing solely to evaluate performance tended to be more accurate both in ratings and recognition of critical behaviors. However, when ratings were collected up to one week later, the group whose primary purpose was to learn the material showed more accurate performance ratings. …