Two Achilles Heels of Surgical Randomized Controlled Trials: Differences in Surgical Skills and Reporting of Average Performance

Abstract Randomized controlled trials (RCT)s of surgery are fundamentally different from RCTs of medications because it is difficult to blind or mask a surgical procedure or perform “sham’ operations. An additional challenge is the variation in skills and surgical proficiency of participating centers and surgeons. Addressing heterogeneity in surgical proficiency remains of paramount importance, especially when RCTs involve a new or complex procedure such as minimally invasive radical surgery. In the presence of such heterogeneity, it is very cumbersome to objectively evaluate and monitor surgical skills so that most trials simply report associations that are averaged across surgeons and hospitals/centers. Such reporting is non-transparent because the rates of complications and adverse outcomes are reported only as averages, and these averages may not apply to the individual participating surgeons or centers. These factors, coupled with the inherent non-generalizability of findings from such RCTs – due to the strict inclusion and exclusion criteria for enrollment – may lead to conclusions that no longer apply to real life for individual surgeons or centers. Case in point is a recently published non-inferiority RCT that reported that minimally invasive radical hysterectomy was associated with lower rates of disease-free survival (86% versus 96.5% at 4.5 years) and overall survival (93.8% versus 99% at 3 years) than open abdominal radical hysterectomy in patients with cervical cancer. However, RCTs involving two competing complex or new procedures may be affected by tremendous confounding due to variations in surgical proficiency and also non-standardization for other confounding factors such as patient selection categories (i.e. stage of cancer) and adjuvant post-operative therapies that may affect long-term survival. The purpose of this Viewpoint is not to provide an exhaustive review of the trial but to use it as an illustration to focus on two challenging areas that most RCTs of a new complex surgical procedure suffer from: un-adjusting or not correcting for surgical skill variability and non-transparent reporting of averaged results. We provide suggestions to overcome these deficiencies through robust methodological and statistical approaches.