Changing demographics have made aging and age-related chronic diseases an enormous and growing biomedical and societal challenge. The biological processes of aging may involve a role for the gut microbiota. Aspects of host physiology such as immune homeostasis and energy balance are profoundly influenced by the microbiota. Immune dysregulation characterizes old age and constitutes a major pathomechanism underlying frailty and age-associated chronic diseases. A growing body of literature implicates age-related perturbations in the gut microbial ecology as contributing to a global inflammatory state in the elderly. A better understanding of the nature and determinants of the host-microbe relationship in old age has the potential to translate into strategies that promote healthy aging and extend life span. This review summarizes our current understanding of the configuration of the age-related gut microbiota and its likely role in determining the immune phenotype in the elderly. It also highlights the specific components of the microbiota that can be targeted to modulate the age-related chronic inflammation.