The superior olivary complex is a group of brainstem nuclei involved in hearing and includes the medial superior olive (MSO) and the lateral superior olive (LSO), surrounded by periolivary cell groups. The structure and functional roles of the MSO and LSO have been the subject of many investigations in laboratory animals and it has largely been assumed that these findings are directly transferable to humans. However, little is known conclusively regarding the detailed organization of the human superior olivary complex. The goal of this study is to provide a detailed analysis of the cytoarchitecture of the human MSO and LSO. Results from the examination of eight human brainstems confirm the existence of a conserved MSO and provide evidence of a prominent and highly ordered LSO. Unbiased stereological estimates of neuronal number indicate approximately 15,500 neurons in the MSO and 5600 neurons in the LSO. Additionally, a three-dimensional model of the MSO and LSO was constructed and provides evidence that the human LSO is composed of medial and lateral segments. Finally, an analysis of neuronal morphology, in Nissl stained and Golgi impregnated tissue, provides evidence of multiple neuronal classes within each nucleus and further that these neurons demonstrate a precise geometric arrangement (depending on the nucleus) that is suggestive of isofrequency laminae.