Infant faces elicit early, specific activity in the orbitofrontal cortex (OFC), a key cortical region for reward and affective processing. A test of the causal relationship between infant facial configuration and OFC activity is provided by naturally occurring disruptions to the face structure. One such disruption is cleft lip, a small change to one facial feature, shown to disrupt parenting. Using magnetoencephalography, we investigated neural responses to infant faces with cleft lip compared with typical infant and adult faces. We found activity in the right OFC at 140 ms in response to typical infant faces but diminished activity to infant faces with cleft lip or adult faces. Activity in the right fusiform face area was of similar magnitude for typical adult and infant faces but was significantly lower for infant faces with cleft lip. This is the first evidence that a minor change to the infant face can disrupt neural activity potentially implicated in caregiving.