Is this what success looks like? Mismatches between the aims, claims, and evidence used to demonstrate impact from knowledge exchange processes at the interface of environmental science and policy

Abstract As anthropogenic pressures on the environment grow, science-policy interaction is increasingly needed to support evidence-informed decision-making. However, there are many barriers to knowledge exchange (KE) at the science-policy interface, including difficulties evaluating its outcomes. The aims of this study are to synthesize the literature to elucidate the a) intended and b) claimed outcomes of KE processes at the interface of environmental science and policy, as well as the c) evidence used to evaluate them and d) methods used for collecting evaluation data. Results from systematically identifying and analyzing 397 articles show that co-production, knowledge brokerage, boundary organizations, and social connections were the most common strategies for KE. KE processes commonly aimed, claimed and referred to evidence regarding the usability of knowledge (e.g. credibility, salience, legitimacy) and social outcomes (e.g. networking, awareness, learning, trust-building). They also aimed for deeper policy/economic/societal impacts and actual use of scientific knowledge within decision-making. These additional goals, however, were seldom claimed to have been achieved, although products (e.g. maps/tools) and process attributes (e.g. equity, power-relations, transparency) were commonly used for evidencing impact. Hence, this study found that success from KE at the interface of environmental science and policy comes in diverse forms and showed a divergence between what studies aim for (ambitious) and what they evidence or claim as an achievement (more modest). This may represent failures of KE processes to reach intended goals, shortcomings in evaluation literature/approaches, or mismatches between timescales of evaluation and impact. Overall, this suggests a need to better align goals with evaluation measures to plan, facilitate, and appreciate the diverse impacts of KE processes.