Using ORCID, DOI, and Other Open Identifiers in Research Evaluation

An evaluator’s task is to connect the dots between program goals and its outcomes. This can be accomplished through surveys, research, and interviews, and is frequently performed post hoc. Evaluation is hampered by a lack of data that clearly connect a program with its outcomes and, in particular, by ambiguity about who has participated in the program and what contributions they have made. Manually making these connections is very labor-intensive, and algorithmic matching introduces errors and assumptions that can distort results. In this paper, we discuss the use of identifiers in evaluation -- for individuals, their contributions, and the organizations that sponsor them and fund their work. Global identifier systems are uniquely positioned to capture global mobility and collaboration. By leveraging connections between local infrastructures and global information resources, evaluators can map data sources that were previously either unavailable or prohibitively labor-intensive. We describe how identifiers, such as ORCID iDs and DOIs, are being embedded in research workflows, how this is affecting data availability for evaluation purposes, and provide examples of evaluations that are leveraging identifiers. We also discuss the importance of provenance and preservation in establishing confidence in the reliability and trustworthiness of data and relationships, and in the long-term availability of metadata describing objects and their inter-relationships. We conclude with a discussion on opportunities and risks for the use of identifiers in evaluation processes.