TY - JOUR T1 - Hypothesis testing in the presence of noisy experimental replications JF - bioRxiv DO - 10.1101/268151 SP - 268151 AU - Diego Vidaurre AU - Mark W. Woolrich AU - Theodoros Karapanagiotidis AU - Jonathan Smallwood AU - Thomas E. Nichols Y1 - 2018/01/01 UR - http://biorxiv.org/content/early/2018/02/20/268151.abstract N2 - We propose a simple procedure based on permutation testing that provides a way of combining the results from many individual tests that refer to the same hypothesis. This is needed when testing a measure whose value is obtained from a noisy process, which can be repeated multiple times, referred to as replications. Examples of a noisy process can be: (i) computational, e.g. when using an approximate inference algorithm (e.g. ICA) for which different runs can produce different results or (ii) observational, if we have the capacity to acquire data multiple times, and the different acquired data sets can be considered noisy examples of the underlying data that we are attempting to estimate; that is, we are not interested in the individual replications but on the unobserved process behind. This method can also be used when we intend to test multiple hypotheses, each with access to various replications, while correcting for the familywise error rate. Using both simulations and real data, we show that the proposed approach compares favourably to more standard approaches to this problem. ER -