Estimating Heterogeneous Treatment Effects and the Effects of Heterogeneous Treatments with Ensemble Methods

Randomized experiments are increasingly used to study political phenomena because they can credibly estimate the average effect of a treatment on a population of interest. But political scientists are often interested in how effects vary across sub-populations— heterogeneous treatment effects —and how differences in the content of the treatment affects responses—the response to heterogeneous treatments. Several new methods have been introduced to estimate heterogeneous effects, but it is difficult to know if a method will perform well for a particular data set. Rather than use only one method, we show how an ensemble of methods—weighted averages of estimates from individual models increasingly used in machine learning—accurately measure heterogeneous effects. Building on a large literature on ensemble methods, we show the close relationship between out of sample prediction and accurate estimation of heterogeneous treatment effects and demonstrate how pooling models leads to superior performance to individual methods across diverse problems. We apply the ensemble method to two experiments, illuminating how ensemble method for heterogenous treatment effects facilitates exploratory analysis of treatment effects. ∗Assistant Professor, Department of Political Science, Stanford University; Encina Hall West 616 Serra St., Stanford, CA, 94305 †Ph.D. candidate, Department of Communication, Stanford University, 450 Serra Mall, Building 120, Room 110, Stanford, CA, 94305 ‡Ph.D. candidate, Department of Communication, Stanford University, 450 Serra Mall, Building 120, Room 110, Stanford, CA, 94305