Several authors have proposed that speciation frequently occurs when a population becomes fixed for one or more chromosomal rearrangements that reduce fitness when they are heterozygous. This hypothesis has little theoretical support because mutations that cause a large reduction in fitness can be fixed through drift only in small, inbred populations. Moreover, the effects of chromosomal rearrangements on fitness are unpredictable and vary significantly between plants and animals. I argue that rearrangements reduce gene flow more by suppressing recombination and extending the effects of linked isolation genes than by reducing fitness. This unorthodox perspective has significant implications for speciation models and for the outcomes of contact between neospecies and their progenitor(s).