RT Journal Article SR Electronic T1 benchmarkR: an R package for benchmarking genome-scale methods JF bioRxiv FD Cold Spring Harbor Laboratory SP 018200 DO 10.1101/018200 A1 Xiaobei Zhou A1 Charity W. Law A1 Mark D. Robinson YR 2015 UL http://biorxiv.org/content/early/2015/04/17/018200.abstract AB Summary: benchmarkR is an R package designed to assess and visualize the performance of statistical methods for datasets that have an independent truth (e.g., simulations or datasets with large-scale validation), in particular for methods that claim to control false discovery rates (FDR). We augment some of the standard performance plots (e.g., receiver operating characteristic, or ROC, curves) with information about how well the methods are calibrated (i.e., whether they achieve their expected FDR control). For example, performance plots are extended with a point to highlight the power or FDR at a user-set threshold (e.g., at a method’s estimated 5% FDR). The package contains general containers to store simulation results (SimResults) and methods to create graphical summaries, such as receiver operating characteristic curves (rocX), false discovery plots (fdX) and power-to-achieved FDR plots (powerFDR); each plot is augmented with some form of calibration information. We find these plots to be an improved way to interpret relative performance of statistical methods for genomic datasets where many hypothesis tests are performed. The strategies, however, are general and will find applications in other domains.Availability: The benchmarkR package is available from: https://github.com/markrobinsonuzh/benchmarkRContact: mark.robinson@imls.uzh.ch