Lateralization of function in auditory cortex has remained a persistent puzzle. Previous studies using signals with differing spectrotemporal characteristics support a model in which the left hemisphere is more sensitive to temporal and the right more sensitive to spectral stimulus attributes. Here we use single-trial sparse-acquisition fMRI and a stimulus with parametrically varying segmental structure affecting primarily temporal properties. We show that both left and right auditory cortices are remarkably sensitive to temporal structure. Crucially, beyond bilateral sensitivity to timing information, we uncover two functionally significant interactions. First, local spectrotemporal signal structure is differentially processed in the superior temporal gyrus. Second, lateralized responses emerge in the higher-order superior temporal sulcus, where more slowly modulated signals preferentially drive the right hemisphere. The data support a model in which sounds are analyzed on two distinct timescales, 25-50 ms and 200-300 ms.