Visual search for a target object among distractors often takes longer when more distractors are present. To understand the neural basis of this capacity limitation, we recorded activity from visually responsive neurons in the frontal eye field (FEF) of macaque monkeys searching for a target among distractors defined by form (randomly oriented T or L). To test the hypothesis that the delay of response time with increasing number of distractors originates in the delay of attention allocation by FEF neurons, we manipulated the number of distractors presented with the search target. When monkeys were presented with more distractors, visual target selection was delayed and neuronal activity was reduced in proportion to longer response time. These findings indicate that the time taken by FEF neurons to select the target contributes to the variation in visual search efficiency.