Neuroimaging group studies are typically performed with the assumption that subjects used are randomly drawn from a population of subjects. The population of subjects is assumed to have a distribution of effect sizes associated with it that are Gaussian distributed. However, in practice, group studies can include "outlier" subjects whose effect sizes are completely at odds with the general population for reasons that are not of experimental interest. If ignored, these outliers can dramatically affect the inference results. To solve this problem, we propose a group inference approach which includes inference of outliers using a robust general linear model (GLM) approach. This approach models the errors as being a mixture of two Gaussian distributions, one for the normal population and one for the outliers. Crucially the robust GLM is part of a traditional hierarchical group model which uses GLMs at each level of the hierarchy. This combines the benefits of outlier inference with the benefits of using variance information from lower levels in the hierarchy. A Bayesian inference framework is used to infer on the robust GLM, while using the lower level variance information. The performance of the method is demonstrated on simulated and fMRI data and is compared with iterative reweighted least squares and permutation testing.