Children exposed to various forms of adversity early in life are at increased risk for a broad range of developmental difficulties, affecting both cognitive and emotional adjustment. We review a growing body of evidence suggesting that exposure to adverse circumstances affects the developing brain in ways that increase risk for a myriad of problems. We focus on two forms of adversity, one in which children are exposed to childhood maltreatment in family environments, and another in which children are exposed to extreme psychosocial deprivation in contexts of institutional rearing. We discuss ways in which each of these experiences represent violations of species-expected caregiving conditions, thereby imposing challenges to the developing brain. We also review emerging data pointing to the effectiveness of early intervention in remediating neurodevelopmental consequences associated with maltreatment or institutional rearing. We conclude by discussing implications of this work for public health efforts and highlight important directions for the field.