Climate warming is increasingly leading to marked changes in plant and animal biodiversity, but it remains unclear how temperatures affect microbial biodiversity, particularly in terrestrial soils. Here we show that, in accordance with metabolic theory of ecology, taxonomic and phylogenetic diversity of soil bacteria, fungi and nitrogen fixers are all better predicted by variation in environmental temperature than pH. However, the rates of diversity turnover across the global temperature gradients are substantially lower than those recorded for trees and animals, suggesting that the diversity of plant, animal and soil microbial communities show differential responses to climate change. To the best of our knowledge, this is the first study demonstrating that the diversity of different microbial groups has significantly lower rates of turnover across temperature gradients than other major taxa, which has important implications for assessing the effects of human-caused changes in climate, land use and other factors.