The importance of macrostructural white matter changes, including white matter lesions and atrophy, in intact brain functioning is increasingly being recognized. Diffusion tensor imaging (DTI) enables measurement of the microstructural integrity of white matter. Loss of white matter integrity in aging has been reported, but whether this is inherent to the aging process itself or results from specific white matter pathology is unknown. In 832 persons aged 60 years and older from the population-based Rotterdam Study, we measured fractional anisotropy (FA) and directional diffusivities in normal-appearing white matter using DTI. All subjects' DTI measures were projected onto a common white matter skeleton to enable robust voxelwise comparison. With increasing age, multiple regions showed significant decreases in FA or increases in axial or radial diffusivity in normal-appearing white matter. However, nearly all of these regional changes were explained by either white matter atrophy or by white matter lesions; each of which related to changes in distinct brain regions. These results indicate that loss of white matter integrity in aging is primarily explained by atrophy and lesion formation and not by the aging process itself. Furthermore, white matter atrophy and white matter lesion formation relate to loss of integrity in distinct brain regions, indicating the two processes are pathophysiologically different.