Gene flow between populations that are adapting to distinct environments may be restricted if hybrids inherit maladaptive, intermediate phenotypes. This phenomenon, called extrinsic postzygotic isolation (EPI), is thought to play a critical role in the early stages of speciation. However, despite its intuitive appeal, we know surprisingly little about the strength and prevalence of EPI in nature, and even less about the specific phenotypes that tend to cause problems for hybrids. In this study, we searched for EPI among allopatric populations of the butterfly Euphydryas editha that have specialized on alternative host plants. These populations recall a situation thought typical of the very early stages of speciation. They lack consistent host-associated genetic differentiation at random nuclear loci and show no signs of reproductive incompatibility in the laboratory. However, they do differ consistently in diverse host-related traits. For each of these traits, we first asked whether hybrids between populations that use different hosts (different-host hybrids) were intermediate to parental populations and to hybrids between populations that use the same host (same-host hybrids). We then conducted field experiments to estimate the effects of intermediacy on fitness in nature. Our results revealed strong EPI under field conditions. Different-host hybrids exhibited an array of intermediate traits that were significantly maladaptive, including four behaviors. Intermediate foraging height slowed the growth of larvae, while intermediate oviposition preference, oviposition site height, and clutch size severely reduced the growth and survival of the offspring of adult females. We used our empirical data to construct a fitness surface on which different-host hybrids can be seen to fall in an adaptive valley between two peaks occupied by same-host hybrids. These findings demonstrate how ecological selection against hybrids can create a strong barrier to gene flow at the early stages of adaptive divergence.