Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
Confirmatory Results

Revisiting inconsistency in large pharmacogenomic studies

Zhaleh Safikhani, Mark Freeman, Petr Smirnov, Nehme El-Hachem, Adrian She, Rene Quevedo, Anna Goldenberg, Nicolai Juul Birkbak, Christos Hatzis, Leming Shi, Andrew H Beck, Hugo JWL Aerts, John Quackenbush, Benjamin Haibe-Kains
doi: https://doi.org/10.1101/026153
Zhaleh Safikhani
1Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
2Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mark Freeman
1Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Petr Smirnov
1Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nehme El-Hachem
3Institut de recherches cliniques de Montréal, Montreal, Quebec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Adrian She
1Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rene Quevedo
1Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
2Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anna Goldenberg
4Hospital for Sick Children, Toronto, Ontario, Canada
5Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nicolai Juul Birkbak
6University College London, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christos Hatzis
7Section of Medical Oncology, Yale University School of Medicine, New Haven, Connecticut; USA
8Yale Cancer Center, Yale University, New Haven, Connecticut, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Leming Shi
9Fudan University, Shanghai City, China
10University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrew H Beck
11Department of Pathology, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts. USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hugo JWL Aerts
12Department of Cancer Biology, Dana-Farber Cancer Institute, Boston, Massachusetts. USA
13Department of Radiation Oncology and Radiology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John Quackenbush
14Department of Biostatistics and Computational Biology and Center for Cancer Computational Biology, Boston, Massachusetts, USA
15Department of Cancer Biology, Dana-Farber Cancer Institute, Boston, Massachusetts, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Benjamin Haibe-Kains
1Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
2Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
5Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Preview PDF
Loading

ABSTRACT

Background: In 2012, two large pharmacogenomic studies, the Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE), were published, each reported gene expression data and measures of drug response for a large number of drugs and hundreds of cell lines. In 2013, we published a comparative analysis that reported gene expression profiles for the 471 cell lines profiled in both studies and dose response measurements for the 15 drugs characterized in the common cell lines by both studies. While we found good concordance in gene expression profiles, there was substantial inconsistency in the drug responses reported by the GDSC and CCLE projects. Our paper was widely discussed and we received extensive feedback on the comparisons that we performed. This feedback, along with the release of new data, prompted us to revisit our initial analysis. Here we present a new analysis using these expanded data in which we address the most significant suggestions for improvements on our published analysis: that drugs with different response characteristics should have been treated differently, that targeted therapies and broad cytotoxic drugs should have been treated differently in assessing consistency, that consistency of both molecular profiles and drug sensitivity measurements should both be compared across cell lines to accurately assess differences in the studies, that we missed some biomarkers that are consistent between studies, and that the software analysis tools we provided with our analysis should have been easier to run, particularly as the GDSC and CCLE released additional data.

Methods: For each drug, we used published sensitivity data from the GDSC and CCLE to separately estimate drug dose-response curves. We then used two statistics, the area between drug dose-response curves (ABC) and the Matthews correlation coefficient (MCC), to robustly estimate the consistency of continuous and discrete drug sensitivity measures, respectively. We also used recently released RNA-seq data together with previously published gene expression microarray data to assess inter-platform reproducibility of cell line gene expression profiles.

Results: This re-analysis supports our previous finding that gene expression data are significantly more consistent than drug sensitivity measurements. The use of new statistics to assess data consistency allowed us to identify two broad effect drugs — 17-AAG and PD-0332901 — and three targeted drugs — PLX4720, nilotinib and crizotinib — with moderate to good consistency in drug sensitivity data between GDSC and CCLE. Not enough sensitive cell lines were screened in both studies to robustly assess consistency for three other targeted drugs, PHA-665752, erlotinib, and sorafenib. Concurring with our published results, we found evidence of inconsistencies in pharmacological phenotypes for the remaining eight drugs. Further, to discover “consistency” between studies required the use of multiple statistics and the selection of specific measures on a case-by-case basis.

Conclusion: Our results reaffirm our initial findings of an inconsistency in drug sensitivity measures for eight of fifteen drugs screened both in GDSC and CCLE, irrespective of which statistical metric was used to assess correlation. Taken together, our findings suggest that the phenotypic data on drug response in the GDSC and CCLE continue to present challenges for robust biomarker discovery. This re-analysis provides additional support for the argument that experimental standardization and validation of pharmacogenomic response will be necessary to advance the broad use of large pharmacogenomic screens.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted September 11, 2015.
Download PDF

Supplementary Material

Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Revisiting inconsistency in large pharmacogenomic studies
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Revisiting inconsistency in large pharmacogenomic studies
Zhaleh Safikhani, Mark Freeman, Petr Smirnov, Nehme El-Hachem, Adrian She, Rene Quevedo, Anna Goldenberg, Nicolai Juul Birkbak, Christos Hatzis, Leming Shi, Andrew H Beck, Hugo JWL Aerts, John Quackenbush, Benjamin Haibe-Kains
bioRxiv 026153; doi: https://doi.org/10.1101/026153
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Revisiting inconsistency in large pharmacogenomic studies
Zhaleh Safikhani, Mark Freeman, Petr Smirnov, Nehme El-Hachem, Adrian She, Rene Quevedo, Anna Goldenberg, Nicolai Juul Birkbak, Christos Hatzis, Leming Shi, Andrew H Beck, Hugo JWL Aerts, John Quackenbush, Benjamin Haibe-Kains
bioRxiv 026153; doi: https://doi.org/10.1101/026153

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One
Subject Areas
All Articles
  • Animal Behavior and Cognition (3698)
  • Biochemistry (7809)
  • Bioengineering (5689)
  • Bioinformatics (21330)
  • Biophysics (10595)
  • Cancer Biology (8199)
  • Cell Biology (11961)
  • Clinical Trials (138)
  • Developmental Biology (6777)
  • Ecology (10419)
  • Epidemiology (2065)
  • Evolutionary Biology (13900)
  • Genetics (9726)
  • Genomics (13094)
  • Immunology (8164)
  • Microbiology (20058)
  • Molecular Biology (7871)
  • Neuroscience (43147)
  • Paleontology (321)
  • Pathology (1280)
  • Pharmacology and Toxicology (2264)
  • Physiology (3362)
  • Plant Biology (7246)
  • Scientific Communication and Education (1315)
  • Synthetic Biology (2010)
  • Systems Biology (5547)
  • Zoology (1132)