A robust positive correlation between height and intelligence, as measured by IQ tests, has been established in the literature. This paper makes several contributions toward establishing the causes of this association. First, we extend the standard bivariate ACE model to account for assortative mating. The more general theoretical framework provides several key insights, including formulas to decompose a cross-trait genetic correlation into components attributable to assortative mating and pleiotropy and to decompose a cross-trait within-family correlation. Second, we use a large dataset of male twins drawn from Swedish conscription records and examine how well genetic and environmental factors explain the association between (i) height and intelligence and (ii) height and military aptitude, a professional psychologist's assessment of a conscript's ability to deal with wartime stress. For both traits, we find suggestive evidence of a shared genetic architecture with height, but we demonstrate that point estimates are very sensitive to assumed degrees of assortative mating. Third, we report a significant within-family correlation between height and intelligence (p^ = 0.10), suggesting that pleiotropy might be at play.