Epistasis (gene-gene interaction) is thought to play an integral role in the genetic basis of complex traits, and a significant amount of research has been invested into identifying this phenomenon in human disease. However, the overall success of empirical studies of epistasis in humans is unclear, as such studies are rarely systematically evaluated. Here, we have selected asthma as an example of a well-studied, complex human disease, and provide a critical analysis and replication attempt of nearly all prior reports of epistasis for this disease. Of 191 previously reported interactions, we find that 39.8% were not originally identified using an explicit test for interaction and thus may not have been true epistatic effects to begin with. Moreover, directions of effect were not described for 46.1% of the interactions, which prevents their rigorous replication. In the original studies, attempts at replication were made for 15.2% of the interactions, and 7.3% were actually replicated. In the current study, we were able to evaluate 85.9% of the interactions using a large asthma dataset from the GABRIEL Consortium. None of these interactions could be replicated based on strict criteria. However, we found nominally significant (p < 0.05) evidence in support of 23.8% of the evaluated interactions. Although many reports of epistasis are not robustly supported in the published literature, our results suggest that at least some of these reports may have been true-positive examples of epistasis. In general, improvements in empirical studies of epistasis are called for, in order to better understand the importance of this phenomenon in human disease.