Temporal coherence and attention in auditory scene analysis

doi:10.1016/j.tins.2010.11.002

Trends in Neurosciences

Volume 34, Issue 3, March 2011, Pages 114-123

https://doi.org/10.1016/j.tins.2010.11.002 Get rights and content

Humans and other animals can attend to one of multiple sounds and follow it selectively over time. The neural underpinnings of this perceptual feat remain mysterious. Some studies have concluded that sounds are heard as separate streams when they activate well-separated populations of central auditory neurons, and that this process is largely pre-attentive. Here, we argue instead that stream formation depends primarily on temporal coherence between responses that encode various features of a sound source. Furthermore, we postulate that only when attention is directed towards a particular feature (e.g. pitch) do all other temporally coherent features of that source (e.g. timbre and location) become bound together as a stream that is segregated from the incoherent features of other sources.

Section snippets

The auditory scene analysis problem

Humans and other animals routinely detect, identify and track sounds coming from a particular source (e.g. someone's voice, a conspecific call) among sounds emanating from other sources (e.g. other voices, heterospecific calls, ambient music and street traffic) (Figure 1). The apparent ease with which they determine which components and attributes in a sound mixture arise from the same source belies the complexity of the underlying biological processes. By analogy with the scene segmentation

Temporal coherence in auditory scene analysis

Problems inherent to auditory scene analysis are similar to those found in visual scene analysis. However, there are a few notable unique aspects. In particular, whereas natural and artificial visual scenes often contain a large proportion of static or slow-moving elements, auditory scenes are essentially dynamic, containing many fast-changing, relatively brief acoustic events (referred to as tokens in Box 1) 30, 31. Therefore, an essential aspect of auditory scene analysis is the linking over

Is streaming a pre-attentive process?

A widely held view by which has emerged from electrophysiological studies in humans 28, 29, 67, 68, 69, 70, is that auditory streams are formed pre-attentively in the auditory system, much like the extraction of low-level features in early pre-cortical stages. Depending on the listener's intentions and guided by representations of previously encountered auditory objects (or streams) that are now stored in memory, attention would simply serve to enhance the perception of a particular stream in

Summary

Here, we proposed two ideas within an overall framework to explain the perception of auditory scenes. The first is that auditory stream formation is critically dependent on the temporal coherence between neural responses to sounds in the auditory cortex. Specifically, when stimulus-induced cortical responses are temporally coherent, the features they represent can potentially become perceptually unified (or bound) as one stream, distinct from other temporally incoherent responses. This

Acknowledgments

This work was supported by the following grants to the authors: NIH R0107657, MURI N000141010278, AFOSR FA9550-09-1-0234 and NSF CAREER award IIS-0846112.

Glossary

Auditory scene analysis: processes by which sequential and concurrent acoustic events are analyzed and organized into auditory streams.
Auditory stream: series of sounds perceived by the listener as a coherent entity and, as such, can be selectively attended to among other sounds. The word ‘stream’ emphasizes the fact that sounds usually unfold over time. Although sounds coming from different physical sound sources typically form separate streams, this is not always the case. For example, a choir

References (104)

R.P. Carlyon
How the brain separates sounds
Trends Cogn. Sci.
(2004)
I. Nelken
Processing of complex stimuli and natural scenes in the auditory cortex
Curr. Opin. Neurobiol.
(2004)
D.G. Sinex
Spectral processing and sound source determination
Int. Rev. Neurobiol.
(2005)
C. Alain
Breaking the wave: effects of attention and learning on concurrent sound perception
Hear. Res.
(2007)
C. Micheyl
The role of auditory cortex in the formation of auditory streams
Hear. Res.
(2007)
I. Winkler
Modeling the auditory scene: predictive regularity representations and perceptual objects
Trends Cogn. Sci.
(2009)
S. Shamma et al.
Behind the scenes of auditory perception
Curr. Opin. Neurobiol.
(2010)
C. Micheyl et al.
Pitch, harmonicity and concurrent sound segregation: Psychoacoustical and neurophysiological findings
Hearing Res.
(2010)
Y.I. Fishman
Neural correlates of auditory stream segregation in primary auditory cortex of the awake monkey
Hear. Res.
(2001)
C. Micheyl
Perceptual organization of tone sequences in the auditory cortex of awake macaques
Neuron
(2005)

D. Pressnitzer

Perceptual organization of sound begins in the auditory periphery

Curr. Biol.

(2008)

M.L. Sutter

Spectral processing in the auditory cortex

Int. Rev. Neurobiol.

(2005)

J.C. Middlebrooks

Binaural response-specific bands in primary auditory cortex (AI) of the cat: topographical organization orthogonal to isofrequency contours

Brain Res.

(1980)

M. Elhilali

Temporal coherence in the perceptual organization and cortical representation of auditory scenes

Neuron

(2009)

A. Luczak

Multivariate receptive field mapping in marmoset auditory cortex

J. Neurosci. Methods

(2004)

T. Rahne

A multilevel and cross-modal approach towards neuronal mechanisms of auditory streaming

Brain Res.

(2008)

B.G. Shinn-Cunningham

Object-based auditory and visual attention

Trends Cogn. Sci.

(2008)

J.B. Fritz

Auditory attention – focusing the searchlight on sound

Curr. Opin. Neurobiol.

(2007)

E. Niebur

Synchrony: a neuronal mechanism for attentional selection?

Curr. Opin. Neurobiol.

(2002)

R. VanRullen

Spike times make sense

Trends Neurosci.

(2005)

J.J. DiCarlo et al.

Untangling invariant object recognition

Trends Cogn. Sci.

(2007)

A.S. Bregman

Auditory Scene Analysis: The Perceptual Organization of Sound

(1990)

E.C. Cherry

Some experiments on the recognition of speech, with one and two ears

J. Acoust. Soc. Am.

(1953)

M.A. Bee et al.

The cocktail party problem: what is it? How can it be solved? And why should animal behaviorists study it?

J. Comp. Psychol.

(2008)

J.H. McDermott

The cocktail party problem

Curr. Biol.

(2009)

N. Marrone

Evaluating the benefit of hearing aids in solving the cocktail party problem

Trends Amplif.

(2008)

D. Wang et al.

Computational Auditory Scene Analysis: Principles, Algorithms, and Applications

(2006)

R.R. Fay

Sound source perception and stream segregation in nonhuman vertebrate animals

A. Bidet-Caulet et al.

Neurophysiological mechanisms involved in auditory perceptual organization

Front Neurosci.

(2009)

J.S. Snyder et al.

Toward a neurophysiological theory of auditory stream segregation

Psychol. Bull.

(2009)

W. Hartmann et al.

Stream segregation and peripheral channeling

Mus. Percep.

(1991)

M.W. Beauvois et al.

Computer simulation of auditory stream segregation in alternating-tone sequences

J. Acoust. Soc. Am.

(1996)

S. McCabe et al.

A model of auditory streaming

J. Acoust. Soc. Am.

(1997)

J.S. Kanwal

Neurodynamics for auditory stream segregation: tracking sounds in the mustached bat's natural environment

Network

(2003)

M.A. Bee et al.

Primitive auditory stream segregation: a neurophysiological study in the songbird forebrain

J. Neurophysiol.

(2004)

Y.I. Fishman

Auditory stream segregation in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration

J. Acoust. Soc. Am.

(2004)

M.A. Bee et al.

Auditory stream segregation in the songbird forebrain: effects of time intervals on responses to interleaved tone sequences

Brain Behav. Evol.

(2005)

E. Sussman

An investigation of the auditory streaming effect using event-related brain potentials

Psychophysiol.

(1999)

E.S. Sussman

The role of attention in the formation of auditory streams

Percept. Psychophys.

(2007)

H. Attias et al.

Temporal low-order statistics of natural sounds

Adv. Neural Inf. Process. Syst.

(1997)

N.C. Singh et al.

Modulation spectra of natural sounds and ethological theories of auditory processing

J. Acoust. Soc. Am.

(2003)

J. Vliegen

The role of spectral and periodicity cues in auditory stream segregation, measured using a temporal discrimination task

J. Acoust. Soc. Am.

(1999)

N. Grimault

Auditory stream segregation on the basis of amplitude-modulation rate

J. Acoust. Soc. Am.

(2002)

B.C.J. Moore et al.

Factors influencing sequential stream segregation

Acta Acustica

(2002)

S. Boehnke et al.

The relation between auditory temporal interval processing and sequential stream segregation examined with stimulus laterality differences

J. Percept. Psychophys.

(2005)

C.E. Schreiner et al.

Topography of excitatory bandwidth in cat primary auditory cortex: single-neuron versus multiple-neuron recordings

J. Neurophysiol.

(1992)

C.E. Schreiner

Order and disorder in auditory cortical maps

Curr. Opin. Neurobiol.

(1994)

H. Versnel

Ripple analysis in the ferret primary auditory cortex. III. Topographic and columnar distribution of ripple response parameters

Aud. Neurosci.

(1995)

N. Kowalski

Analysis of dynamic spectra in ferret primary auditory cortex. I. Characteristics of single-unit responses to moving ripple spectra

J. Neurophysiol.

(1996)

N. Kowalski

Analysis of dynamic spectra in ferret primary auditory cortex. II. Prediction of unit responses to arbitrary dynamic spectra

J. Neurophysiol.

(1996)

Cited by (311)

The effects of aging and hearing impairment on listening in noise
2024, iScience
The study investigates age-related decline in listening abilities, particularly in noisy environments, where the challenge lies in extracting meaningful information from variable sensory input (figure-ground segregation). The research focuses on peripheral and central factors contributing to this decline using a tone-cloud-based figure detection task. Results based on behavioral measures and event-related brain potentials (ERPs) indicate that, despite delayed perceptual processes and some deterioration in attention and executive functions with aging, the ability to detect sound sources in noise remains relatively intact. However, even mild hearing impairment significantly hampers the segregation of individual sound sources within a complex auditory scene. The severity of the hearing deficit correlates with an increased susceptibility to masking noise. The study underscores the impact of hearing impairment on auditory scene analysis and highlights the need for personalized interventions based on individual abilities.
Slow neural oscillations explain temporal fluctuations in distractibility
2023, Progress in Neurobiology
Human environments comprise various sources of distraction, which often occur unexpectedly in time. The proneness to distraction (i.e., distractibility) is posited to be independent of attentional sampling of targets, but its temporal dynamics and neurobiological basis are largely unknown. Brain oscillations in the theta band (3 – 8 Hz) have been associated with fluctuating neural excitability, which is hypothesised here to explain rhythmic modulation of distractibility. In a pitch discrimination task (N = 30) with unexpected auditory distractors, we show that distractor-evoked neural responses in the electroencephalogram and perceptual susceptibility to distraction were co-modulated and cycled approximately 3 – 5 times per second. Pre-distractor neural phase in left inferior frontal and insular cortex regions explained fluctuating distractibility. Thus, human distractibility is not constant but fluctuates on a subsecond timescale. Furthermore, slow neural oscillations subserve the behavioural consequences of a hitherto largely unexplained but ever-increasing phenomenon in modern environments – distraction by unexpected sound.
The effect of topic familiarity and volatility of auditory scene on selective auditory attention
2023, Hearing Research
Selective auditory attention has been shown to modulate the cortical representation of speech. This effect has been well documented in acoustically more challenging environments. However, the influence of top-down factors, in particular topic familiarity, on this process remains unclear, despite evidence that semantic information can promote speech-in-noise perception. Apart from individual features forming a static listening condition, dynamic and irregular changes of auditory scenes—volatile listening environments—have been less studied. To address these gaps, we explored the influence of topic familiarity and volatile listening on the selective auditory attention process during dichotic listening using electroencephalography. When stories with unfamiliar topics were presented, participants’ comprehension was severely degraded. However, their cortical activity selectively tracked the speech of the target story well. This implies that topic familiarity hardly influences the speech tracking neural index, possibly when the bottom-up information is sufficient. However, when the listening environment was volatile and the listeners had to re-engage in new speech whenever auditory scenes altered, the neural correlates of the attended speech were degraded. In particular, the cortical response to the attended speech and the spatial asymmetry of the response to the left and right attention were significantly attenuated around 100–200 ms after the speech onset. These findings suggest that volatile listening environments could adversely affect the modulation effect of selective attention, possibly by hampering proper attention due to increased perceptual load.
Functional network properties of the auditory cortex
2023, Hearing Research
The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.
Atypical cortical processing of bottom-up speech binding cues in children with autism spectrum disorders
2023, NeuroImage: Clinical
Individuals with autism spectrum disorder (ASD) commonly display speech processing abnormalities. Binding of acoustic features of speech distributed across different frequencies into coherent speech objects is fundamental in speech perception. Here, we tested the hypothesis that the cortical processing of bottom-up acoustic cues for speech binding may be anomalous in ASD. We recorded magnetoencephalography while ASD children (ages 7–17) and typically developing peers heard sentences of sine-wave speech (SWS) and modulated SWS (MSS) where binding cues were restored through increased temporal coherence of the acoustic components and the introduction of harmonicity. The ASD group showed increased long-range feedforward functional connectivity from left auditory to parietal cortex with concurrent decreased local functional connectivity within the parietal region during MSS relative to SWS. As the parietal region has been implicated in auditory object binding, our findings support our hypothesis of atypical bottom-up speech binding in ASD. Furthermore, the long-range functional connectivity correlated with behaviorally measured auditory processing abnormalities, confirming the relevance of these atypical cortical signatures to the ASD phenotype. Lastly, the group difference in the local functional connectivity was driven by the youngest participants, suggesting that impaired speech binding in ASD might be ameliorated upon entering adolescence.
Cross-Modal Interactions Between Auditory Attention and Oculomotor Control
2024, Journal of Neuroscience

View all citing articles on Scopus

View full text

Trends in Neurosciences

OpinionTemporal coherence and attention in auditory scene analysis

Section snippets

The auditory scene analysis problem

Temporal coherence in auditory scene analysis

Is streaming a pre-attentive process?

Summary

Acknowledgments

Glossary

Trends Cogn. Sci.

Curr. Opin. Neurobiol.

Int. Rev. Neurobiol.

Hear. Res.

Hear. Res.

Trends Cogn. Sci.

Curr. Opin. Neurobiol.

Hearing Res.

Hear. Res.

Neuron

Curr. Biol.

Int. Rev. Neurobiol.

Brain Res.

Neuron

J. Neurosci. Methods

Brain Res.

Trends Cogn. Sci.

Curr. Opin. Neurobiol.

Curr. Opin. Neurobiol.

Trends Neurosci.

Trends Cogn. Sci.

Auditory Scene Analysis: The Perceptual Organization of Sound

Some experiments on the recognition of speech, with one and two ears

J. Acoust. Soc. Am.

The cocktail party problem: what is it? How can it be solved? And why should animal behaviorists study it?

J. Comp. Psychol.

The cocktail party problem

Curr. Biol.

Evaluating the benefit of hearing aids in solving the cocktail party problem

Trends Amplif.

Computational Auditory Scene Analysis: Principles, Algorithms, and Applications

Sound source perception and stream segregation in nonhuman vertebrate animals

Neurophysiological mechanisms involved in auditory perceptual organization

Front Neurosci.

Toward a neurophysiological theory of auditory stream segregation

Psychol. Bull.

Stream segregation and peripheral channeling

Mus. Percep.

Computer simulation of auditory stream segregation in alternating-tone sequences

J. Acoust. Soc. Am.

A model of auditory streaming

J. Acoust. Soc. Am.

Neurodynamics for auditory stream segregation: tracking sounds in the mustached bat's natural environment

Network

Primitive auditory stream segregation: a neurophysiological study in the songbird forebrain

J. Neurophysiol.

Auditory stream segregation in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration

J. Acoust. Soc. Am.

Auditory stream segregation in the songbird forebrain: effects of time intervals on responses to interleaved tone sequences

Brain Behav. Evol.

An investigation of the auditory streaming effect using event-related brain potentials

Psychophysiol.

The role of attention in the formation of auditory streams

Percept. Psychophys.

Temporal low-order statistics of natural sounds

Adv. Neural Inf. Process. Syst.

Modulation spectra of natural sounds and ethological theories of auditory processing

J. Acoust. Soc. Am.

The role of spectral and periodicity cues in auditory stream segregation, measured using a temporal discrimination task

J. Acoust. Soc. Am.

Auditory stream segregation on the basis of amplitude-modulation rate

J. Acoust. Soc. Am.

Factors influencing sequential stream segregation

Acta Acustica

The relation between auditory temporal interval processing and sequential stream segregation examined with stimulus laterality differences

J. Percept. Psychophys.

Topography of excitatory bandwidth in cat primary auditory cortex: single-neuron versus multiple-neuron recordings

J. Neurophysiol.

Order and disorder in auditory cortical maps

Curr. Opin. Neurobiol.

Ripple analysis in the ferret primary auditory cortex. III. Topographic and columnar distribution of ripple response parameters

Opinion
Temporal coherence and attention in auditory scene analysis