Increasing evidence suggests that the composition of the human gut microbiome is important in the etiology of human diseases; however, the personal factors that influence the gut microbiome composition are poorly characterized. Animal models point to sex hormone-related differentials in microbiome composition. In this study, we investigated the relationship of sex, body mass index (BMI) and dietary fiber intake with the gut microbiome in 82 humans. We sequenced fecal 16S rRNA genes by 454 FLX technology, then clustered and classified the reads to microbial genomes using the QIIME pipeline. Relationships of sex, BMI, and fiber intake with overall gut microbiome composition and specific taxon abundances were assessed by permutational MANOVA and multivariate logistic regression, respectively. We found that sex was associated with the gut microbiome composition overall (p=0.001). The gut microbiome in women was characterized by a lower abundance of Bacteroidetes (p=0.03). BMI (>25 kg/m2 vs. <25 kg/m2) was associated with the gut microbiome composition overall (p=0.05), and this relationship was strong in women (p=0.03) but not in men (p=0.29). Fiber from beans and from fruits and vegetables were associated, respectively, with greater abundance of Actinobacteria (p=0.006 and false discovery rate adjusted q=0.05) and Clostridia (p=0.009 and false discovery rate adjusted q=0.09). Our findings suggest that sex, BMI, and dietary fiber contribute to shaping the gut microbiome in humans. Better understanding of these relationships may have significant implications for gastrointestinal health and disease prevention.