A number of conflicting claims have been advanced regarding the role of the left inferior frontal gyrus, inferior parietal lobe and posterior middle temporal gyrus in action recognition, driven in part by an ongoing debate about the capacities of putative mirror systems that match observed and planned actions. We report data from 43 left hemisphere stroke patients in two action recognition tasks in which they heard and saw an action word ('hammering') and selected from two videoclips the one corresponding to the word. In the spatial recognition task, foils contained errors of body posture or movement amplitude/timing. In the semantic recognition task, foils were semantically related (sawing). Participants also performed a comprehension control task requiring matching of the same verbs to objects (hammer). Using regression analyses controlling for both the comprehension control task and lesion volume, we demonstrated that performance in the semantic gesture recognition task was predicted by per cent damage to the posterior temporal lobe, whereas the spatial gesture recognition task was predicted by per cent damage to the inferior parietal lobule. A whole-brain voxel-based lesion symptom-mapping analysis suggested that the semantic and spatial gesture recognition tasks were associated with lesioned voxels in the posterior middle temporal gyrus and inferior parietal lobule, respectively. The posterior middle temporal gyrus appears to serve as a central node in the association of actions and meanings. The inferior parietal lobule, held to be a homologue of the monkey parietal mirror neuron system, is critical for encoding object-related postures and movements, a relatively circumscribed aspect of gesture recognition. The inferior frontal gyrus, on the other hand, was not predictive of performance in any task, suggesting that previous claims regarding its role in action recognition may require refinement.