While human cognitive abilities are clearly unique, underlying changes in brain organization and function remain unresolved. Here we characterized the transcriptome of the cortical layers and adjacent white matter in the prefrontal cortexes of humans, chimpanzees and rhesus macaques using unsupervised sectioning followed by RNA sequencing. More than 20% of detected genes were expressed predominantly in one layer, yielding 2,320 human layer markers. While the bulk of the layer markers were conserved among species, 376 switched their expression to another layer in humans. By contrast, only 133 of such changes were detected in the chimpanzee brain, suggesting acceleration of cortical reorganization on the human evolutionary lineage. Immunohistochemistry experiments further showed that human-specific expression changes were not limited to neurons but affected a broad spectrum of cortical cell types. Thus, despite apparent histological conservation, human neocortical organization has undergone substantial changes affecting more than 5% of its transcriptome.