Policy Evaluation during a Pandemic.

National and local governments have implemented a large number of policies, particularly non-pharmaceutical interventions, in response to the Covid-19 pandemic. Evaluating the effects of these policies, both on the number of Covid-19 cases as well on other economic outcomes is a key ingredient for policymakers to be able to determine which policies are most effective as well as the relative costs and benefits of particular policies. In this paper, we consider the relative merits of common identification strategies exploiting variation in policy choices made across different locations by checking whether the identification strategies are compatible with leading epidemic models in the epidemiology literature. We argue that unconfoundedness type approaches are likely to be more useful for evaluating policies than difference in differences type approaches due to the highly nonlinear spread of cases during a pandemic. For difference in differences, we further show that a version of this problem continues to exist even when one is interested in understanding the effect of a policy on other economic outcomes when those outcomes also depend on the number of Covid-19 cases. We propose alternative approaches that are able to circumvent these issues. We apply our proposed approach to study the effect of state level shelter-in-place orders early in the pandemic.