Ancient and diverse antibiotic resistance genes (ARGs) have previously been identified from soil, including genes identical to those in human pathogens. Despite the apparent overlap between soil and clinical resistomes, factors influencing ARG composition in soil and their movement between genomes and habitats remain largely unknown. General metagenome functions often correlate with the underlying structure of bacterial communities. However, ARGs are proposed to be highly mobile, prompting speculation that resistomes may not correlate with phylogenetic signatures or ecological divisions. To investigate these relationships, we performed functional metagenomic selections for resistance to 18 antibiotics from 18 agricultural and grassland soils. The 2,895 ARGs we discovered were mostly new, and represent all major resistance mechanisms. We demonstrate that distinct soil types harbour distinct resistomes, and that the addition of nitrogen fertilizer strongly influenced soil ARG content. Resistome composition also correlated with microbial phylogenetic and taxonomic structure, both across and within soil types. Consistent with this strong correlation, mobility elements (genes responsible for horizontal gene transfer between bacteria such as transposases and integrases) syntenic with ARGs were rare in soil by comparison with sequenced pathogens, suggesting that ARGs may not transfer between soil bacteria as readily as is observed between human pathogens. Together, our results indicate that bacterial community composition is the primary determinant of soil ARG content, challenging previous hypotheses that horizontal gene transfer effectively decouples resistomes from phylogeny.