Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation

While bibliometrics is widely used for research evaluation purposes, a common theoretical framework for conceptually understanding, empirically studying, and effectively teaching their usage is lacking. In this paper, we develop one such framework: the fast-and-frugal heuristics research program, proposed originally in the context of the cognitive and decision sciences, lends itself particularly well for understanding and investigating the usage of bibliometrics in research evaluations. Such evaluations represent judgments under uncertainty in which typically neither all possible outcomes, nor their consequences, and probabilities are known, knowable, or can be reliably estimated. In such situations of fuzzy and incomplete information, good descriptive and prescriptive models of human behavior are heuristics. Heuristics are simple strategies that, by exploiting the structure of environments, can aid people to make smart decisions. Relying on heuristics does not mean trading off accuracy against effort: while reducing complexity, heuristics can yield better decisions than more information-greedy procedures in many environments. This is well-documented in a large literature, cutting across psychology, computer science, crime, the law, medicine, and other domains. We outline the fast-and-frugal research program, illustrate examples of past research on heuristics outside the field of bibliometrics, explain why heuristics are especially suitable for studying the usage of bibliometrics, and outline a corresponding conceptual framework.