Speech perception depends strongly on precise encoding of the temporal structure of sound. Although behavioural studies suggest that communication problems experienced by older adults may entail deficits in temporal acuity, much is unknown about the effects of age on the neural mechanisms underlying the encoding of sound duration. In this study, we measured neuromagnetic auditory evoked responses in young, middle-aged and older healthy participants listening to sounds of various durations. The time courses of cortical activity from bilateral sources in superior temporal planes showed specific differences related to the sound offsets indicating the neural representation of onset and offset markers as one dimension of the neural code for sound duration. Model free MEG source analysis identified brain areas specifically responding with an increase in activity to increases in sound duration in the left anterior insula, right inferior frontal, right middle temporal, and right post-central gyri in addition to bilateral supra-temporal gyri. Sound duration-related changes in cortical responses were comparable in all three age groups despite age-related changes in absolute response magnitudes. The results demonstrated that early cortical encoding of the temporal structure of sound presented in silence is little or not affected by normal aging.