Role of the auditory system in speech production

Abstract

This chapter reviews evidence regarding the role of auditory perception in shaping speech output. Evidence indicates that speech movements are planned to follow auditory trajectories. This in turn is followed by a description of the Directions Into Velocities of Articulators (DIVA) model, which provides a detailed account of the role of auditory feedback in speech motor development and control. A brief description of the higher-order brain areas involved in speech sequencing (including the pre-supplementary motor area and inferior frontal sulcus) is then provided, followed by a description of the Hierarchical State Feedback Control (HSFC) model, which posits internal error detection and correction processes that can detect and correct speech production errors prior to articulation. The chapter closes with a treatment of promising future directions of research into auditory–motor interactions in speech, including the use of intracranial recording techniques such as electrocorticography in humans, the investigation of the potential roles of various large-scale brain rhythms in speech perception and production, and the development of brain–computer interfaces that use auditory feedback to allow profoundly paralyzed users to learn to produce speech using a speech synthesizer.

Introduction

Speech production is a highly complex motor act, involving the finely coordinated activation of approximately 100 muscles in the respiratory, laryngeal, and oral motor systems. To achieve this task, speakers utilize a large network of brain regions. This network includes regions involved in other motor tasks, such as the motor and somatosensory cortical areas, cerebellum, basal ganglia, and thalamus, as well as regions that are more specialized for speech and language, including inferior and middle prefrontal cortex and superior and middle temporal cortex. Our goal in this chapter is to describe the critical role of the auditory system in speech production. We will first discuss the role of sensory systems in motor control broadly and summarize the long history of ideas and research on the interaction between auditory and motor systems for speech. We then describe current research on speech planning, which strongly implicates the auditory system in this process. Two large-scale neurocomputational models of speech production are then discussed. Finally, we will highlight some future directions for research on speech production.

Movement is absolutely dependent on sensory information. We know where and how to reach for an object because we see its location and shape; we know how much force to exert while we are holding the object because we feel the pressure of the object on our hand and the weight on our limb; and we know how to initiate any of these movements because our sensory systems tell us where our limb is in relation to our body and the object. British neurologist and physiologist Henry Charlton Bastian (1837–1915) wrote on the topic of movement control in 1887, stating, “It may be regarded as a physiological axiom, that all purposive movements of animals are guided by sensations or by afferent impressions of some kind” (Bastian, 1887, p. 1). Experimental work over the decades backs up these claims. This work has found, for example, that blocking somatosensory feedback from a monkey's limb (while leaving motor fibers intact) causes the limb to go dead. With training the monkey can learn to reuse it clumsily, but only with visual feedback; blindfold the animal and motor control degrades dramatically (Sanes et al., 1984). Similar symptomology can be found in humans suffering from large-fiber sensory neuropathy, which deafferents the body sense while leaving motor fibers intact (Sanes et al., 1984).

Speech is no different. Without the auditory system, as in prelingual-onset peripheral deafness, normal speech development cannot occur. Importantly, it is not just during development that auditory information is critical. Experimental or naturally caused manipulations of acoustic input can have dramatic effects on speech production. For example, delayed auditory feedback induces non-fluency (Yates, 1963), altering feedback in the form of pitch or the formant frequency structure results in automatic and largely unconscious compensation in speech articulation (Burnett et al., 1998, Houde and Jordan, 1998, Larson et al., 2001), and exposure to a different linguistic environment can induce changes in the listener–speaker's articulation (picking up accents; Sancier and Fowler, 1997). Furthermore, although individuals who become deaf as adults can remain intelligible for years after they lose hearing, they show some speech output impairments immediately (including impaired ability to adjust pitch loudness in different listening conditions), and over time their phonetic contrasts become reduced (e.g., Perkell et al., 2000) and they exhibit articulatory decline (Waldstein, 1989).

The speech research literature contains numerous theoretic proposals that strongly link speech perception and speech production. Notable examples include the motor theory of speech perception (Liberman et al., 1967, Liberman and Mattingly, 1985), which posits that speech perception involves translating acoustic signals into the motor gestures that produce them, as well as acoustic theories of speech production (e.g., Fant, 1960, Stevens, 1998), which highlight the importance of acoustic or auditory targets in the speech production process. In the following sections we elaborate on the roles of auditory information in recent neural models of speech planning and execution.