Permutation and parametric tests for effect sizes in voxel-based morphometry of gray matter volume in brain structural MRI

Permutation testing has been widely implemented in voxel-based morphometry (VBM) tools. However, this type of non-parametric inference has yet to be thoroughly compared with traditional parametric inference in VBM studies of brain structure. Here we compare both types of inference and investigate what influence the number of permutations in permutation testing has on results in an exemplar study of how gray matter proportion changes with age in a group of working age adults. High resolution T1-weighted volume scans were acquired from 80 healthy adults aged 25-64years. Using a validated VBM procedure and voxel-based permutation testing for Pearson product-moment coefficient, the effect sizes of changes in gray matter proportion with age were assessed using traditional parametric and permutation testing inference with 100, 500, 1000, 5000, 10000 and 20000 permutations. The statistical significance was set at P<0.05 and false discovery rate (FDR) was used to correct for multiple comparisons. Clusters of voxels with statistically significant (PFDR<0.05) declines in gray matter proportion with age identified with permutation testing inference (N≈6000) were approximately twice the size of those identified with parametric inference (N=3221voxels). Permutation testing with 10000 (N=6251voxels) and 20000 (N=6233voxels) permutations produced clusters that were generally consistent with each other. However, with 1000 permutations there were approximately 20% more statistically significant voxels (N=7117voxels) than with ≥10000 permutations. Permutation testing inference may provide a more sensitive method than traditional parametric inference for identifying age-related differences in gray matter proportion. Based on the results reported here, at least 10000 permutations should be used in future univariate VBM studies investigating age related changes in gray matter to avoid potential false findings. Additional studies using permutation testing in large imaging databanks are required to address the impact of model complexity, multivariate analysis, number of observations, sampling bias and data quality on the accuracy with which subtle differences in brain structure associated with normal aging can be identified.