We examined the associations of current alcohol consumption with brain morphometric measures in a healthy, community-dwelling cohort. Cranial T1-weighted 3D-structural MRI scans were obtain in 383 adults (men=211) aged 60-64 years, randomly selected form the larger PATH Through Life study. Voxel-based morphometric analyses were applied to detect regional gray matter and white matter volume changes related to reported weekly alcohol consumption (mean 7.04+/-8.15 drinks per week). Alcohol consumption in men had a linear association with greater gray matter in bilateral superior and medial frontal gyrus, bilateral middle occipital gyrus, right inferior parietal gyrus, bilateral precentral gyrus, left paracentral gyrus, left uncus and left inferior occipital gyrus, and with lesser white matter in bilateral superior temporal and left parahippocampal gyrus, after adjustment for age, education, total intracranial volume, smoking, hypertension, diabetes and hyperlipidemia. In women, there was no significant linear association between alcohol consumption and total or regional brain volumes. Our results showed a dose-related, sexually dimorphic impact of alcohol on brain tissue volumes independent of cerebrovascular risk factors. These findings are consistent with an inverse-U association between alcohol use and brain morphometry, while suggesting an increased vulnerability of white matter to alcohol-related brain damage.