To investigate associations between genetic, linguistic, and geographic variation in Africa, we type 50 Y chromosome SNPs in 1122 individuals from 40 populations representing African geographic and linguistic diversity. We compare these patterns of variation with those that emerge from a similar analysis of published mtDNA HVS1 sequences from 1918 individuals from 39 African populations. For the Y chromosome, Mantel tests reveal a strong partial correlation between genetic and linguistic distances (r=0.33, P=0.001) and no correlation between genetic and geographic distances (r=-0.08, P>0.10). In contrast, mtDNA variation is weakly correlated with both language (r=0.16, P=0.046) and geography (r=0.17, P=0.035). AMOVA indicates that the amount of paternal among-group variation is much higher when populations are grouped by linguistics (Phi(CT)=0.21) than by geography (Phi(CT)=0.06). Levels of maternal genetic among-group variation are low for both linguistics and geography (Phi(CT)=0.03 and 0.04, respectively). When Bantu speakers are removed from these analyses, the correlation with linguistic variation disappears for the Y chromosome and strengthens for mtDNA. These data suggest that patterns of differentiation and gene flow in Africa have differed for men and women in the recent evolutionary past. We infer that sex-biased rates of admixture and/or language borrowing between expanding Bantu farmers and local hunter-gatherers played an important role in influencing patterns of genetic variation during the spread of African agriculture in the last 4000 years.