The purpose of this study was to examine associations among hearing thresholds, cognitive ability, and speech understanding in adverse listening conditions within and between groups of younger, middle-aged, and older adults. Participants repeated back sentences played in the presence of several types of maskers (syntactically similar and syntactically different competing speech from one or two other talkers, and steady-state speech-shaped noise). They also completed tests of auditory short-term/working memory, processing speed, and inhibitory ability. Results showed that group differences in accuracy of word identification and in error patterns differed depending upon the number of masking voices; specifically, older and middle-aged individuals had particular difficulty, relative to younger subjects, in the presence of a single competing message. However, the effect of syntactic similarity was consistent across subject groups. Hearing loss, short-term memory, processing speed, and inhibitory ability were each related to some aspects of performance by the middle-aged and older participants. Notably, substantial age-related changes in speech recognition were apparent within the group of middle-aged listeners.