Reply to “Comment on ‘On the Utility of ToxCast™ and ToxPi as Methods for Identifying New Obesogens’”

Perspectives | Correspondence A Section 508–conformant HTML version of this article is available at http://dx.doi.org/10.1289/EHP1122. The Correspondence section is not peer-reviewed. Personal opinions expressed herein are the sole responsibility of the authors. ­ EHP neither endorses nor disputes the content of the letters it publishes. Reply to “Comment on ‘On the Utility of ToxCast™ and ToxPi as Methods for Identifying New Obesogens’” http://dx.doi.org/10.1289/EHP1122 Refers to http://dx.doi.org/10.1289/EHP881 We thank Houck et al. for taking the time to critique our paper, which evaluated the utility of ToxCast™ bioactivity data and the ToxPi tool for predicting PPARγ acti- vation and induction of adipogenesis. Our science-based critique of the performance of certain ToxCast™ assays should have elicited a response that engaged with the substance of our paper. Instead, Houck et al. highlight irrelevant or marginally relevant points and criticize us for analyses performed by scien- tists at the U.S. Environmental Protection Agency (EPA) while ignoring what we consider to be fundamental problems with ToxCast™ assays. Here we respond to their major points. In their first point Houck et al. suggest that we failed to cite the correct publications for the NovaScreen®, GeneBLAzer®, Attagene, and Tox21 assays. We cited the original developers of these assays and Knudsen et al. (2011) for using them in ToxCast™ Phase I to profile 309 chemicals. We did not perform targeted testing of Tox21 assays. In their second point Houck et al. make multiple criticisms regarding the assays, our analysis of them, and possible reagent differ- ences. We want to clarify that we undertook this project as a collaboration with Dr. Kristina Thayer of the National Toxicology Program (NTP) in 2010 as an activity related to the January 2011 NTP workshop “Role of Environmental Chemicals in Diabetes and Obesity” (Thayer et al. 2012). Prior to the workshop, Dr. Thayer provided us with a list of PPARγ activators generated by EPA scientists during late 2010. We agreed to test the top 20 ranked chemicals if they were provided. In rechecking our material transfer agree- ments, we found that the top 20 PPARγ hits were indeed provided by the EPA at the request of the NTP; it was our under- standing that these were from the same stocks as were used in ToxCast™. We added an additional chemical, chloro­thalonil (purchased separately), because the ToxCast™ Novascreen® assay results suggested it bound avidly to PPARγ (AC 50 = 0.54 μM). Following the workshop, we were provided with a second group of compounds (the “ToxPi chemicals”) by the NTP (material transfer agreement dated 27 October 2011). A 12 ToxPi chemicals were generated by predic- tion models based on assays applicable to biological processes associated with diabetes and obesity, assays that were nominated by experts in obesity, diabetes, and metabolism. The adipocyte differentiation model, which my group helped construct, was based on assay results for PPARγ, PPRE, RXR, GR, LXR, LXRE, SREBP1, and C/EBP; see Table S2 in Auerbach et al. (2016) and Figure 3 in our paper. Prediction models were generated by Dr. David Reif, then of the EPA, and the results provided to the investigators by the NTP, together with test chemicals. These models reflected the data analysis pipeline used by the EPA in 2011. It was our understanding that these chemicals were from ToxCast™ stocks, but whether this was the case is unclear in our correspondence. We tested all of the chemi- cals provided to us in good faith and reported the results. Changes in the EPA data processing pipeline caused the lists of prioritized chemi- cals to change numerous times between 2011 and 2015. Auerbach et al. (2016) used a data processing pipeline that evolved during the period of 2014–2015. Their prediction models utilized ToxCast™ Phase II data on 1,860 compounds; the original 2011 ToxPi prioritized chemicals were based on 309 ToxCast™ Phase I chemicals. Therefore, the targeted testing analysis originally contem- plated and undertaken by multiple groups was no longer straightforward, and Auerbach et al. (2016) was published as a review article. Surprisingly, none of the chemicals on the 2011 list that we identified as active on PPARγ, on RXR, and in adipogenesis assays were included on the 2015 list of priori- tized chemicals (see Table 2 in Auerbach et al. 2016). Three chemicals appeared on both lists (tebufenpyrad, pyridaben, and fenpyroximate) but did not induce adipo- genesis. That none of the active chemicals from the 2011 ToxPi list appeared in the 2015 list, whereas 3 inactive chemicals were incorrectly predicted to be active on both lists, indicates that the prioritization process needs improving. It is instructive to consider where the chemicals we found to be active are ranked by Auerbach et al. (2016). The 1,890 ToxCast Phase II chemicals were ranked from 1 to 810 by EPA scientists using the 2011 ToxPi list of assays; position 810 comprised 1,050 chemicals with a score of 0 (See Table S3 in Auerbach et al. 2016). The chemicals active in our assays were ranked as follows: tebupirimfos, 48; t ­ riphenyltin hydroxide, 71; spirodiclofen, 96; triflumizole, 150; zoxamide, 223; bisphenol A, 290; quinoxyfen, 444; flusilazole, 490; fludioxonil, 525; forchlorfenuron, 663; and pymetrozine and acetamiprid, 810. Therefore, the source of the low ranking is the ToxCast™ assays themselves, not the source of the chemicals. Since both the 2011 and the 2015 lists were generated by EPA scientists, it is unclear how our analysis of which chemicals to test was faulty, as Houck et al. allege. Houck et al. state that we did not consider chemical source in our discussion of why the assay results in our study disagreed with ToxCast™ results. While it is possible that one batch of a chemical has a contami- nant that produces spurious activity, or has degraded such that the active material is no longer active, we minimized this possibility by receiving test chemicals from reputable sources (the NTP and the EPA). These chemicals were used exclusively for receptor activation assays and in most of the adipo- genesis assays. In some cases, the stock of chemicals provided to us was exhausted, and we repurchased them from commercial sources. We did not observe differences in the ability of the repurchased chemicals to induce adipogenesis from the originals provided by the NTP and the EPA. Houck et al. further state that we did not consider methodological and platform differences in our criticism of the results of ToxCast™ assays. We have been doing nuclear receptor activation assays since 1992 and have contributed to the development of these technologies and the interpretation of results. While it can be made to appear that there are differences in our techniques that would obviate comparison of receptor activa- tion assays across species or platforms, this is not the case for PPARγ and virtually all other nuclear receptors (with the exception of the xenobiotic receptors SXR/PXR and CAR, which exhibit strong species selectivity for a subset of compounds). Occasional differences arise in PPARγ activation across species, but these are exceptional. It is also possible that a chemical can act as a receptor agonist in one cell type and be inactive or a receptor antagonist in another. Such chemicals are uncommon, and one would not expect to find many, if any, among the 37 PPARγ or PPRE activators identified by the ToxCast™ assays we tested. Whether chemicals can be metabolized to active forms by the cells used in the Attagene assays is also unlikely to be a valid criti- cism. The ability of chemicals to cause hits in the NRF2 assays noted by Houck et al. (information that was not conveyed to us volume 125 | number 1 | January 2017 • Environmental Health Perspectives