Educational Evaluation in the Public Policy Setting

Abstract : The essays that compose this report criticize, from federal, state, and local perspectives, current methods of evaluating government-sponsored education programs. A major recurring theme is that experimental design methods, most commonly used by the U.S. Department of Education, do not provide adequate information for policy makers' needs. The essays recommend other methods that address policy makers' immediate concerns, including such issues as resource use and distribution of funds, fidelity of implementation, and needs of target groups, in addition to the traditional focus on student outcomes. Underlying this theme is the view that the proper relationship between evaluation and policy making is important, but ill understood, with the consequence that contemporary evaluation methods are frequently of little value in formulating education policy. Chapter 1 of the report summarizes the main findings of Chapters 2 through 5, and discusses a possible new approach to evaluation through the social effects of government programs. Chapter 2 is a critique of the experimental design approach, with recommendations for new perspectives on evaluation. Chapter 3 analyzes federal program evaluation from the local perspective, and argues that present evaluation methods are generally not useful from the local perspective. It also points out the absence of any evaluation approach that can successfully deal with the sources of local variations. Chapter 4 describes an Executive Branch attempt to evaluate Title I of the Elementary and Secondary Education Act on behalf of Congress. The strategy adopted, which was generally successful, was to define the aims of the evaluation in light of Congress's policy making authority and concerns. Chapter 5 discusses, for an audience of state government legislators and administrators, how education programs can be more successfully evaluated at the state level.