Sperm competition is a pervasive selective force in evolution, shaping reproductive anatomy, physiology and behaviour. Here, we present comparative evidence that varying sperm competition levels account for variation in the male reproductive anatomy of rodents, the largest and most diverse mammalian order. We focus on the sperm-producing testes and the accessory reproductive glands, which produce the seminal fluid fraction of the ejaculate. We demonstrate a positive association between relative testis size and the prevalence of within-litter multiple paternity, consistent with previous analyses in which relative testis size has been found to correlate with sperm competition levels inferred from social organization and mating systems. We further demonstrate an association between sperm competition level and the relative size of at least two accessory reproductive glands: the seminal vesicles and anterior prostate. The size of the major product of these glands-the copulatory plug-is also found to vary with sperm competition level. Our findings thus suggest that selection for larger plugs under sperm competition may explain variation in accessory gland size, and highlight the need to consider both sperm and non-sperm components of the male ejaculate in the context of post-copulatory sexual selection.