In search for a benchmark of impact, effectiveness and efficiency of innovation instruments

This report for the TAFTIE Task Force on Benchmarking Impact, Effectiveness and Efficiency (TFBIEE) looks at the accomplishments of innovation policy instruments in an international context. The Task Force was set up to gain a combined insight in innovation agencies’ impact, effectiveness and efficiency. In order to fulfil TAFTIE’s ambition to demonstrate the added value of innovation agencies to society, it is important to understand and demonstrate how the policies these agencies implement have an impact. The main purpose of this project is to learn from each other and improve the agencies’ impact. A comparison or benchmark between the agencies in the Task Force would contribute to joint policy learning. As agencies have quite different mandates, tasks, resources and policy portfolios, a comparison on agency level will be distorted by these significant differences. In addition, it is mostly at programme level that impacts are established through evaluation studies. This study was a first experiment to assess to what extent it is possible to compare effectiveness between specific types of innovation instruments across different countries. It was however not intended to be another handbook on evaluation, but a reference guide to use alongside the many existing guidelines and handbooks. The reference model seems to have worked well to make an assessment of the strengths and weaknesses of the evaluation studies in the benchmark, even though these assessments need to take account of many contextual factors.