To RCT or not to RCT? The ongoing saga of randomised trials in quality improvement

Williams et al 1 describe a well-conducted cluster randomised trial of a stoke quality improvement (QI) initiative, which aimed to improve two inpatient stroke indicators with strong evidence linking them to improved patient outcomes. They randomised five hospitals to receive a QI intervention, and six to receive only indicator feedback. In aggregate, they found evidence of improvement in one indicator, in the intervention group, relative to the control, but this was not sustained once the intervention period ended. The design, execution and analysis of the study were textbook for a cluster randomised controlled trial (RCT) design, aligning well with the CONSORT statement, the gold standard for RCT execution.2 There is much debate within the improvement field about the value of RCTs in determining the effectiveness of improvement interventions. In 2007, Donald Berwick's monologue ‘eating soup with a fork’ provided a convincing argument for why the RCT was necessary for evidence-based medicine, but inadequate for evaluating complex social interventions such as collaboratives and campaigns. Since then, there has been an apparent ‘cooling’ in the appetite of improvement practitioners to adopt RCT methods in attempts to understand the overall impact of improvement initiatives. Against this backdrop, we applaud the authors in their attempt, which goes against the trend, but disappointingly, once again, offers conflicting and weak evidence of beneficial effect despite adherence to rigorous method. So what does this study teach us about whether or not to embrace RCTs in improvement? …