Processing of changes in visual speech in the human auditory cortex

Brain Res Cogn Brain Res. 2002 May;13(3):417-25. doi: 10.1016/s0926-6410(02)00053-8.

Abstract

Seeing a talker's articulatory gestures may affect the observer's auditory speech percept. Observing congruent articulatory gestures may enhance the recognition of speech sounds [J. Acoust. Soc. Am. 26 (1954) 212], whereas observing incongruent gestures may change the auditory percept phonetically as occurs in the McGurk effect [Nature 264 (1976) 746]. For example, simultaneous acoustic /ba/ and visual /ga/ are usually heard as /da/. We studied cortical processing of occasional changes in audiovisual and visual speech stimuli with magnetoencephalography. In the audiovisual experiment congruent (acoustic /iti/, visual /iti/) and incongruent (acoustic /ipi/, visual /iti/) audiovisual stimuli, which were both perceived as /iti/, were presented among congruent /ipi/ (acoustic /ipi/, visual /ipi/) stimuli. In the visual experiment only the visual components of these stimuli were presented. A visual change both in audiovisual and visual experiments activated supratemporal auditory cortices bilaterally. The auditory cortex activation to a visual change occurred later in the visual than in the audiovisual experiment, suggesting that interaction between modalities accelerates the detection of visual change in speech.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Acoustic Stimulation
  • Adult
  • Auditory Cortex / physiology*
  • Discrimination Learning
  • Female
  • Gestures
  • Humans
  • Magnetic Resonance Imaging
  • Magnetoencephalography*
  • Male
  • Middle Aged
  • Photic Stimulation
  • Psychoacoustics
  • Recognition, Psychology
  • Speech Perception / physiology*
  • Time Factors
  • Visual Perception / physiology*