Gait and balance impairment is highly prevalent in older people. We aimed to assess whether and how single markers of small vessel disease (SVD) or a combination thereof explain gait and balance function in the elderly. We analysed 678 community-dwelling healthy subjects from the Lothian Birth Cohort 1936 at the age of 71-74 years who had undergone comprehensive risk factor assessment, gait and balance assessment as well as brain MRI. We investigated the impact of individual SVD markers (white matter hyperintensity - WMH, microbleeds, lacunes, enlarged perivascular spaces, brain atrophy) as seen on structural brain MRI and of a global SVD score on the patients' performance. A regression model revealed that age, sex, and hypertension significantly explained gait speed. Among SVD markers white matter hyperintensity (WMH) score or volume were additional significant and independent predictors of gait speed in the regression model. A similar association was seen with the global SVD score. Our study confirms a negative impact of SVD-related morphologic brain changes on gait speed in addition to age, sex and hypertension independent from brain atrophy. The presence of WMH seems to be the major driving force for SVD on gait impairment in healthy elderly subjects.