Genes that are differentially expressed between the sexes (sex-biased genes) are among the fastest evolving genes in animal genomes. The majority of sex-biased expression is attributable to genes that are primarily expressed in sex-limited reproductive tissues, and these reproductive genes are often rapidly evolving because of intra- and intersexual selection pressures. Additionally, studies of multiple taxa have revealed that genes with sex-biased expression are also expressed in a limited number of tissues. This is worth noting because narrowly expressed genes are known to evolve faster than broadly expressed genes. Therefore, it is not clear whether sex-biased genes are rapidly evolving because they have sexually dimorphic expression, because they are expressed in sex-limited reproductive tissues, or because they are narrowly expressed. To determine the extend to which other confounding variables can explain the rapid evolution of sex-biased genes, I analyzed the rates of evolution of sex-biased genes in Drosophila melanogaster and Mus musculus in light of tissue-specific measures of expression. I find that genes with sex-biased expression in somatic tissues shared by both sexes are often evolving faster than non-sex-biased genes, but this is best explained by the narrow expression profiles of sex-biased genes. Sex-biased genes in sex-limited tissues in D. melanogaster, however, evolve faster than other narrowly expressed genes. Therefore, the rapid evolution of sex-biased genes is limited only to those genes primarily expressed in sex-limited reproductive tissues.