Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Analysis
  • Published:

Using RNA sample titrations to assess microarray platform performance and normalization techniques

Abstract

We have assessed the utility of RNA titration samples for evaluating microarray platform performance and the impact of different normalization methods on the results obtained. As part of the MicroArray Quality Control project, we investigated the performance of five commercial microarray platforms using two independent RNA samples and two titration mixtures of these samples. Focusing on 12,091 genes common across all platforms, we determined the ability of each platform to detect the correct titration response across the samples. Global deviations from the response predicted by the titration ratios were observed. These differences could be explained by variations in relative amounts of messenger RNA as a fraction of total RNA between the two independent samples. Overall, both the qualitative and quantitative correspondence across platforms was high. In summary, titration samples may be regarded as a valuable tool, not only for assessing microarray platform performance and different analysis methods, but also for determining some underlying biological features of the samples.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: RNA samples.
Figure 2: Percentage of genes showing the monotonic titration responses Āi > i > i > i and i > i > i > Āi plotted against the linear Āi / i and i / Āi ratios, respectively, for each commercial whole-genome microarray platform, using various normalization methods.
Figure 3: Impact of normalization on the distributions of titrating genes as a function of signal intensity.
Figure 4: Titration-response concordance for each commercial whole-genome microarray platform, using different normalization methods, with data from each platform separated by site and fold-change direction.

Similar content being viewed by others

References

  1. Barczak, A. et al. Spotted long oligonucleotide arrays for human gene expression analysis. Genome Res. 13, 1775–1785 (2003).

    Article  CAS  Google Scholar 

  2. Barnes, M., Freudenberg, J., Thompson, S., Aronow, B. & Pavlidis, P. Experimental comparison and cross-validation of the Affymetrix and Illumina gene expression analysis platforms. Nucleic Acids Res. 33, 5914–5923 (2005).

    Article  CAS  Google Scholar 

  3. Dobbin, K.K. et al. Interlaboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays. Clin. Cancer Res. 11, 565–572 (2005).

    CAS  PubMed  Google Scholar 

  4. Dorris, D.R. et al. Oligodeoxyribonucleotide probe accessibility on a three-dimensional DNA microarray surface and the effect of hybridization time on the accuracy of expression ratios. BMC Biotechnol. 3, 6 (2003).

    Article  Google Scholar 

  5. Hughes, T.R. et al. Expression profiling using microarrays fabricated by an ink-jet oligonucleotide synthesizer. Nat. Biotechnol. 19, 342–347 (2001).

    Article  CAS  Google Scholar 

  6. Irizarry, R.A. et al. Multiple-laboratory comparison of microarray platforms. Nat. Methods 2, 345–350 (2005).

    Article  CAS  Google Scholar 

  7. Larkin, J.E., Frank, B.C., Gavras, H., Sultana, R. & Quackenbush, J. Independence and reproducibility across microarray platforms. Nat. Methods 2, 337–344 (2005).

    Article  CAS  Google Scholar 

  8. Li, J., Pankratz, M. & Johnson, J.A. Differential gene expression patterns revealed by oligonucleotide versus long cDNA arrays. Toxicol. Sci. 69, 383–390 (2002).

    Article  CAS  Google Scholar 

  9. Naef, F., Socci, N.D. & Magnasco, M. A study of accuracy and precision in oligonucleotide arrays: extracting more signal at large concentrations. Bioinformatics 19, 178–184 (2003).

    Article  CAS  Google Scholar 

  10. Shippy, R. et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61 (2004).

    Article  Google Scholar 

  11. Yuen, T., Wurmbach, E., Pfeffer, R.L., Ebersole, B.J. & Sealfon, S.C. Accuracy and calibration of commercial oligonucleotide and custom cDNA microarrays. Nucleic Acids Res. 30, e48 (2002).

    Article  Google Scholar 

  12. Chudin, E. et al. Assessment of the relationship between signal intensities and transcript concentration for Affymetrix GeneChip arrays. Genome Biol. 3, RESEARCH0005 (2002).

  13. MAQC Consortium. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).

  14. Shi, L. et al. Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics 6 (Suppl.) S12 (2005).

    Article  Google Scholar 

  15. Thompson, K.L. et al. Use of a mixed tissue RNA design for performance assessments on multiple microarray formats. Nucleic Acids Res. 33, e187 (2005).

    Article  Google Scholar 

  16. Bolstad, B.M., Irizarry, R.A., Astrand, M. & Speed, T.P. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics 19, 185–193 (2003).

    Article  CAS  Google Scholar 

  17. Irizarry, R.A. et al. Summaries of Affymetrix GeneChip probe level data. Nucleic Acids Res. 31, e15 (2003).

    Article  Google Scholar 

  18. Irizarry, R.A. et al. Exploration, normalization, and summaries of high density oligonucleotide array probe level data. Biostatistics 4, 249–264 (2003).

    Article  Google Scholar 

  19. Irizarry, R.A., Wu, Z. & Jaffee, H.A. Comparison of Affymetrix GeneChip expression measures. Bioinformatics 22, 789–794 (2006).

    Article  CAS  Google Scholar 

  20. Parrish, R.S. & Spencer, H.J. III. Effect of normalization on significance testing for oligonucleotide microarrays. J. Biopharm. Stat. 14, 575–589 (2004).

    Article  Google Scholar 

  21. Guide to probe logarithmic intensity error (PLIER) estimation. Affymetrix Technical Note <http://www.affymetrix.com/support/technical/technotes/plier_technote.pdf>

  22. Statistical algorithms description document. Affymetrix <http://www.affymetrix.com/support/technical/whitepapers/sadd_whitepaper.pdf>

  23. Cope, L.M., Irizarry, R.A., Jaffee, H.A., Wu, Z. & Speed, T.P. A benchmark for Affymetrix GeneChip expression measures. Bioinformatics 20, 323–331 (2004).

    Article  CAS  Google Scholar 

  24. Wu, Z. & Irizarry, R.A. Stochastic models inspired by hybridization theory for short oligonucleotide arrays. J. Comput. Biol. 12, 882–893 (2005).

    Article  CAS  Google Scholar 

  25. Sendera, T.J. et al. Expression profiling with oligonucleotide arrays: technologies and applications for neurobiology. Neurochem. Res. 27, 1005–1026 (2002).

    Article  CAS  Google Scholar 

  26. Wu, Z., Irizarry, R.A., Gentleman, R., Martinez Murillo, F. & Spencer, F. A model based background adjustment for oligonucleotide expression arrays. J. Am. Stat. Assoc. 99, 909–917 (2004).

    Article  Google Scholar 

  27. Seo, J., Gordish-Dressman, H. & Hoffman, E.P. An interactive power analysis tool for microarray hypothesis testing and generation. Bioinformatics 22, 808–814 (2006).

    Article  CAS  Google Scholar 

  28. Hwang, D., Schmitt, W.A. & Stephanopoulos, G. Determinatoin of minimum sample size and discriminatory expression patterns in microarray data. Bioinformatics 18, 1184–1193 (2002).

    Article  CAS  Google Scholar 

  29. Tibshirani, R. A simple method for assessing sample sizes in microarray experiments. BMC Bioinformatics 7, 106 (2006).

    Article  Google Scholar 

  30. Page, G.P. et al. The PowerAtlas: a power and sample size atlas for microarray experimental design and research. BMC Bioinformatics 7, 84 (2006).

    Article  Google Scholar 

  31. Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This study used a number of computing resources, including the high-performance computational capabilities of the Biowulf PC/Linux cluster at the US National Institutes of Health in Bethesda, Maryland (http://biowulf.nih.gov). This research was supported in part by the Intramural Research Program of the US National Institutes of Health, National Library of Medicine.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Richard Shippy.

Ethics declarations

Competing interests

The following authors declare competing financial interests: R.S., S.F-S., P.W., X.G., E.C., Y.A.S., R.A.S., A.B.L., N.N., Y.T., S.C.B., & J.A.W. In addition, J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.

Supplementary information

Supplementary Fig. 1

Linear regression fitting of Ci/Bi vs. Ai/Bi for each platform, normalization method and site to estimate the true mixture coefficients αcc for the C sample from the data. (DOC 773 kb)

Supplementary Fig. 2

Illustration providing a possible explanation for the asymmetry (i.e., median A>B (βC) ≠ medianA<B (βC) and median A>B (βD) ≠ medianA<B (βD)) in the box plots on either side of Figure 4 (particularly the differences in median values). (DOC 60 kb)

Supplementary Fig. 3

Percentage of genes achieving the specified level of apparent power between the RNA titration mixtures and independent samples. (DOC 56 kb)

Supplementary Fig. 4

Impact of different normalization methods on signal CVs for each microarray platform separated by site and sample. (DOC 429 kb)

Supplementary Fig. 5

Derivation of formulas within Box 1 to clarify calculations for mRNA fractions. (DOC 42 kb)

Supplementary Table 1

Gene counts for each commercial whole genome microarray platform separated by site. (DOC 43 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shippy, R., Fulmer-Smentek, S., Jensen, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat Biotechnol 24, 1123–1131 (2006). https://doi.org/10.1038/nbt1241

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt1241

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing