Analysis of variability in high throughput screening data: applications to melanoma cell lines and drug responses

// Kuan-Fu Ding 1, 2 , Darren Finlay 4 , Hongwei Yin 3 , William P.D. Hendricks 3 , Chris Sereduk 3 , Jeffrey Kiefer 3 , Aleksandar Sekulic 3 , Patricia M. LoRusso 5 , Kristiina Vuori 4 , Jeffrey M. Trent 3 , Nicholas J. Schork 1, 2, 3 1 J. Craig Venter Institute, La Jolla, San Diego, CA, USA 2 University of California, San Diego, CA, USA 3 The Translational Genomics Research Institute, Phoenix, AZ, USA 4 Sanford Burnham Prebys Medical Discovery Institute, La Jolla, San Diego, CA, USA 5 Yale University, New Haven, CT, USA Correspondence to: Nicholas J. Schork, email: nschork@jcvi.org Keywords: melanoma, high-throughput screening, drug screens, computational modeling, variability Received: July 14, 2016      Accepted: January 27, 2017      Published: February 15, 2017 ABSTRACT High-throughput screening (HTS) strategies and protocols have undergone significant development in the last decade. It is now possible to screen hundreds of thousands of compounds, each exploring multiple biological phenotypes and parameters, against various cell lines or model systems in a single setting. However, given the vast amount of data such studies generate, the fact that they use multiple reagents, and are often technician-intensive, questions have been raised about the variability, reliability and reproducibility of HTS results. Assessments of the impact of the multiple factors in HTS studies could arguably lead to more compelling insights into the robustness of the results of a particular screen, as well as the overall quality of the study. We leveraged classical, yet highly flexible, analysis of variance (ANOVA)-based linear models to explore how different factors contribute to the variation observed in a screening study of four different melanoma cell lines and 120 drugs over nine dosages studied in two independent academic laboratories. We find that factors such as plate effects, appropriate dosing ranges, and to a lesser extent, the laboratory performing the screen, are significant predictors of variation in drug responses across the cell lines. Further, we show that when sources of variation are quantified and controlled for, they contextualize claims of inconsistencies and reveal the overall quality of the HTS studies performed at each participating laboratory. In the context of the broader screening study, we show that our analysis can also elucidate the robust effects of drugs, even those within specific cell lines.