Neuropsychological and functional imaging studies have associated the conceptual processing of numbers with bilateral parietal regions (including intraparietal sulcus). However, the processes driving these effects remain unclear because both left and right posterior parietal regions are activated by many other conceptual, perceptual, attention, and response-selection processes. To dissociate parietal activation that is number-selective from parietal activation related to other stimulus or response-selection processes, we used fMRI to compare numbers and object names during exactly the same conceptual and perceptual tasks while factoring out activations correlating with response times. We found that right parietal activation was higher for conceptual decisions on numbers relative to the same tasks on object names, even when response time effects were fully factored out. In contrast, left parietal activation for numbers was equally involved in conceptual processing of object names. We suggest that left parietal activation for numbers reflects a range of processes, including the retrieval of learnt facts that are also involved in conceptual decisions on object names. In contrast, number selectivity in right parietal cortex reflects processes that are more involved in conceptual decisions on numbers than object names. Our results generate a new set of hypotheses that have implications for the design of future behavioral and functional imaging studies of patients with left and right parietal damage.