Human vision is an active process that involves shifting attention across the visual scene, with or without moving the eyes. Such shifts of attention can be generated at will (endogenously) or be triggered automatically, i.e., generated in response to exogenous stimuli including socially relevant cues such as someone else's gaze. What are the common and distinct brain mechanisms involved in these processes? To address this question, we carried out a quantitative effect-location meta-analysis of 59 brain-imaging experiments whose results were published using standardized coordinates. For each condition of interest, namely voluntary and visually triggered eye movements, voluntary and visually triggered (covert) shifts of attention, and perception of someone else's gaze, we computed activation likelihood estimation (ALE) maps. Those maps represent at each voxel of the brain the probability of reporting a signal change related to the condition of interest. For eye movements, this analysis confirmed the spatial location of the frontal eye fields, supplementary eye fields, and parietal saccade-related regions. The map of covert shifts of attention demonstrated highest similarity with the map of saccadic eye movements. Gaze perception showed common activation likelihood with the other conditions in the right intraparietal sulcus and in the lateral precentral gyrus. It demonstrated more similarity with the reflexive than with the voluntary saccades and shifts of attention. We propose that a core network of frontoparietal and temporal brain regions is recruited when we shift the focus of our attention with or without eye movements in response to the appearance of a visual target, as well as when we see someone else shift his or her gaze.