New Results

Evidence for a Spoken Word Lexicon in the Auditory Ventral Stream

Srikanth R. Damera, Lillian Chang, Plamen P. Nikolov, James A. Mattei, Suneel Banerjee, Laurie S. Glezer, Patrick H. Cox, Xiong Jiang, Josef P. Rauschecker, Maximilian Riesenhuber
doi: https://doi.org/10.1101/2022.10.09.511436

Abstract

The existence of a neural representation for whole words (i.e., a lexicon) is a common feature of many models of speech processing. Prior studies have provided evidence for a visual lexicon containing representations of whole written words in an area of the ventral visual stream known as the “Visual Word Form Area” (VWFA). Similar experimental support for an auditory lexicon containing representations of spoken words has yet to be shown. Using fMRI rapid adaptation techniques, we provide evidence for an auditory lexicon in the “Auditory Word Form Area” (AWFA) in the human left anterior superior temporal gyrus that contains representations highly selective for individual spoken words. Furthermore, we show that familiarization with novel auditory words sharpens the selectivity of their representations in the AWFA. These findings reveal strong parallels in how the brain represents written and spoken words, showing convergent processing strategies across modalities in the visual and auditory ventral streams.

Highlights

  • Individual auditory word form areas (AWFA) were defined via an auditory localizer

  • The AWFA shows tuning for individual real words but not untrained pseudowords

  • The AWFA develops tuning for individual pseudowords after training

Introduction

Speech perception is perhaps the most remarkable achievement of the human auditory system and one that likely is critically dependent on its overall cortical architecture. It is generally accepted that the functional architecture of auditory cortex in human and nonhuman primates comprises two processing streams1–3. There is an auditory dorsal stream that is involved in the processing of auditory space and motion4 as well as in sensorimotor transformations such as those required for speech production5–8. There is also an auditory ventral stream specialized for recognizing auditory objects such as spoken words. This stream is organized along a simple-to-complex feature hierarchy2, akin to the organization of the visual ventral stream9.

Visual object recognition studies support a simple-to-complex model of cortical visual processing in which neuronal populations in the visual ventral stream are selective for increasingly complex features and ultimately visual objects along a posterior-to-anterior gradient extending from lower-to-higher-order visual areas10,11. For the special case of recognizing written words, this simple-to-complex model predicts that progressively more anterior neuronal populations are selective for increasingly complex orthographic patterns 12,13. Thus, analogous to general visual processing, orthographic word representations are predicted to culminate in representations of whole visual words – an orthographic “lexicon”. Evidence suggests that these lexical representations are subsequently linked to concept representations in downstream areas like the anterior temporal lobe 14–17. The existence of this orthographic lexicon in the brain is predicted by neuropsychological studies of reading 18. Indeed, functional magnetic resonance imaging (fMRI) 19,20 and, more recently, electrocorticographic (ECoG) data 21,22 have confirmed the existence of such a lexicon in a region of the posterior fusiform cortex known as the Visual Word Form Area (VWFA) 12,23.

It has been proposed that an analogous simple-to-complex hierarchy exists in the auditory ventral stream as well24,25 extending anteriorly from Heschl’s Gyrus along the superior temporal cortex (STC)2,26. Yet, the existence and location of a presumed “auditory lexicon”, i.e., a neural representation for the recognition (and storage) of real words, has been quite controversial 27: The traditional, “posterior” view is that the auditory lexicon should be found in posterior superior temporal cortex (pSTC)28. In contrast, a notable meta-analysis 26 provided strong evidence for the existence of word-selective auditory representations in anterior STC (aSTC), consistent with imaging studies of speech intelligibility 29,30 and proposals for an “auditory word form area” (AWFA) in the human left anterior temporal cortex26,31. Such a role of the aSTC is compatible with nonhuman primate studies that show selectivity for complex communication calls in aSTC 32–34 and demonstrate, in humans and nonhuman primates, that progressively anterior neuron populations in the STC pool over longer timescales 35–38. In this “anterior” account of lexical processing, the pSTC and speech-responsive regions in the IPL are posited to be involved in “inner speech” and phonological reading (covert articulation), but not auditory comprehension6,39. Yet, despite this compelling alternative to traditional theories, there is still little direct evidence for an auditory lexicon in the aSTC.

Investigating the existence and location of auditory lexica is critical for understanding the neural bases of speech processing and, consequently, the neural underpinnings of speech processing disorders. However, finely probing the selectivity of neural representations in the human brain with fMRI is challenging, in part because it is difficult to assess the selectivity of these populations. Many studies have identified speech processing areas by contrasting speech stimuli with various non-speech controls 29,40,41. However, these coarse contrasts cannot reveal what neurons in a particular auditory word-responsive ROI are selective for, e.g., phonemes, syllables, or whole words. More sensitive techniques such as fMRI rapid adaptation (fMRI-RA 42,43) are needed to probe the selectivity of speech representations in the brain and resolve the question of the existence of auditory lexica. In the current study, we used fMRI-RA to test the existence of lexical representations in the auditory ventral stream. Paralleling previous work in the visual system that used fMRI-RA to provide evidence for the existence of an orthographic lexicon in the VWFA 19,20,44, we first performed an independent auditory localizer scan that we used to identify the AWFA in individual subjects, and then conducted three fMRI-RA scans that probed the representation in the AWFA and its plasticity. The first two scans consisted of real words and pseudowords (pronounceable nonwords), respectively. These scans revealed an adaptation profile consistent with lexical selectivity in the putative AWFA for real words, but not novel pseudowords, directly replicating results for written words in the VWFA. We then tested the lexicon hypothesis by predicting that training subjects to recognize novel pseudowords would add them to their auditory lexica, leading them to exhibit lexical-like selectivity in the AWFA following training, as previously shown for written words in the VWFA 19. To do so, we conducted a third fMRI-RA scan after pseudoword training. Results from this scan showed real word-like lexical selectivity to the now-familiar pseudowords following training, supporting the role of the AWFA as an auditory lexicon shaped by experience with auditory words.

Results

Auditory Localizer Scan Identifies Bilateral Speech-Selective ROI in the Auditory Ventral Stream

An independent auditory localizer scan was used to identify a putative auditory word form area (AWFA) near previous literature coordinates (Fig. 2A). To do so, analogous to prior studies of written word representations in the VWFA 19,20,44, we first identified group-level auditory speech-selective areas (see Materials and Methods) by examining the “(Real Words) vs.Silence” contrast thresholded at p≤0.001 masked by the “RW vs. Scrambled Real Words” contrast thresholded at p≤0.05 at the group-level cluster-corrected at the FDR p ≤ 0.05. This revealed several clusters of activation in the superior temporal, frontal, and inferior temporal cortices (Fig. 2B). A local peak in the left STG was identified at MNI: -62, - 14, 2 (Fig. 2C) near the literature coordinates26 of MNI: -61, -15, -5. Individual subject AWFA ROIs (Fig. 2D) were created by building a 50-voxel ROI around the local peak of each subject (mean ± std: -62 ± 2, -14.9 ± 3, 2.6 ± 2.6) closest to the group peak (see Materials and Methods).

Figure 1:
Figure 1: Rapid adaptation and Auditory Localizer Experimental Paradigms.

(A) Shows the slow clustered acquisition paradigm used in the auditory localizer scan. Each trial was 9s long with 1.5s of volume acquisition and 7.5s of silence. During the silent period, the subject heard 4 sounds from one of 5 stimulus classes and performed a 1-back task. (B) Shows the rapid clustered acquisition paradigm used for the RA scans 45,46. Each trial was 3.36s long with 1.68s of volume acquisition. During the silent period, two spoken words were played to the subject with a 50ms ISI. The first word acted as a prime and the second word the target. (C and D) Shows the experimental paradigm for Real Words (RW) and Pseudowords (UTPW/TPW), respectively. The prime was followed by a target word that was either the same word (SAME), a word that differed from the target by one phoneme (1PH), or a word that shared no phonemes with the target (DIFF). Furthermore, subjects were presented with silence trials that served as an explicit baseline. During the task, subjects were asked to attend to all the words and respond when they heard the oddball stimulus (RW or PW containing the rime ‘-ox’ e.g., “socks”) in either the prime or target position.

Figure 2:
Figure 2: Identifying the Auditory Word Form Area (AWFA).

(A) Proposed location of the Auditory Word Form Area (MNI: -61, -15, -5) adapted from 26. Color bar reflects the ALE 47 statistic. (B) Shows the “RW-Silence” contrast (p ≤ 0.001) masked by the “RW-Scrambled Real Words” contrast (p ≤ 0.05) in the auditory localizer scan. Only clusters significant at the FDR p ≤ 0.05 level are shown. Colors reflect t-statistics. (C) Marks the peak in the left STG (MNI: -62, -14, 2). (D) Shows the Auditory Word Form Area defined in individual subjects.

View this table:
Supplementary Table 1

Individual Subject AWFA ROIs

Lexical Selectivity for Real Words but not Pseudowords in the AWFA

The first two fMRI-RA scans were performed with real words (RW) and pseudowords (“untrained pseudowords”, UTPW), respectively. In these experiments, we predicted the lowest signal for the “SAME” condition since the two identical words presented in that condition would repeatedly activate the same neural populations, thereby causing maximal adaptation. Likewise, we predicted the least amount of adaptation for the “DIFF” condition because two words that share no phonemes should activate disjoint groups of neurons, irrespective of whether responsiveness to auditory words in the localizer scan in that ROI was due to neurons selective for phonemes, syllables or whole words. Finally, we tested specific predictions regarding responses in the 1PH condition. Specifically, if neurons in the AWFA ROI are tightly tuned to whole real words (i.e., if the AWFA contains an auditory lexicon), then, as in the DIFF condition, the two similar but non-identical real words in the 1PH condition should have minimal neural overlap and therefore no adaptation should occur and response levels in the 1PH condition should be comparable to that of the DIFF condition (as found for written real words in the VWFA20). In contrast, if neurons in the AWFA were tuned to sublexical phoneme combinations, then there should be a gradual release from adaptation from SAME to 1PH to DIFF, with 1PH < DIFF, as sublexical overlap would continue to increase from 1PH to DIFF. For pseudowords, we predicted that there would be a gradual increase in the BOLD signal paralleling the increasing dissimilarity (i.e., SAME to 1PH to DIFF). This is thought to reflect low-level activation of RW-tuned neurons to phonologically similar pseudowords. These predictions mirror findings in the VWFA for written words19,20,44 and are compatible with an experience-driven increase in selectivity of neurons in the AWFA to real words because of extensive auditory experience with and the need to discriminate among real words but not pseudowords.

To test our hypotheses, we ran a 2-way repeated measures ANOVA on AWFA responses to investigate the relationship between lexicality (real words, RW, and untrained pseudowords, UTPW) and word similarity (SAME, 1PH, and DIFF). The ANOVA was run on subjects that successfully completed both the RW and UTPW scans (n=24). This revealed a significant main effect of similarity (F2,46 = 42.597; p = 3.4E-11) but no significant main effect of lexicality (F1,23 = 2.547; p = 0.124). Critically, however, the analysis revealed a significant interaction between lexicality and word similarity (F2,46 = 4.092; p = 0.023). Planned paired t-tests revealed an adaptation profile in the AWFA that was consistent with tight neural tuning for individual real words (Fig. 3A): There was a significant difference in the mean percent signal between the DIFF vs. SAME conditions (t(23) = 7.86; p <0.001) and the 1PH vs. SAME (t(23) = 5.71; p < 0.001). However, there was no significant difference between the DIFF and 1PH conditions (t(23) = 1.95; p = 0.189). Similar results were obtained using all 26 subjects who completed the RW scan (Supplementary Fig. 1). In contrast, the adaptation profile for UTPWs was not consistent with lexical selectivity (Fig. 3B): There was a significant response difference between the DIFF vs. SAME conditions (t(23) = 7.03; p < 0.001), the 1PH vs. SAME (t(23) = 3.28; p = 0.010), and, critically, also for the DIFF vs. 1PH conditions (t(23) = 3.89; p = 0.002). Finally, we ran a whole-brain analysis to test if lexical representations were localized to the anterior STG or more distributed in nature. Specifically, for RW the conjunction of DIFF vs. SAME and 1PH vs. SAME after excluding voxels where DIFF vs. PH was p < 0.05 produced a cluster in the left anterior STG (FWE-corrected p < 0.05; MNI: -62, -14, 6) within 3.5mm from the average coordinate (MNI: -62, -14.9, 2.6) of our individual ROIs. Importantly, this analysis for UTPWs produced no significant clusters. Thus, the whole-brain analysis also supports the special status of the AWFA as the location of a lexicon for spoken real words.

Figure 3:
Figure 3: Evidence for Auditory Lexical Representations in the AWFA.

Within-subject (n=24) adaptation profile for auditory real words (RWs) and untrained pseudowords (UTPWs). Patterns of release from adaptation are compatible with tight tuning to individual real words consistent with an auditory lexicon. In contrast, UTPWs show a graded release from adaptation as a function of phonological similarity. ***, **, *, and N.S. mark: p ≤0.001, ≤0.01, ≤0.05, and not significant (>0.1), all Bonferroni-corrected for multiple comparisons.

Planned paired t-tests revealed an adaptation profile that was consistent with tight neural tuning for individual RWs (SFig. 1). There was a significant difference in the mean percent signal between the DIFF vs. SAME conditions (t(25) = 6.94; p <0.001) and the 1PH vs. SAME (t(25) = 5.36; p < 0.001). However, there was no significant difference between the DIFF and 1PH conditions (t(25) = 1.70; p = 0.3039).

Supplementary Figure 1:
Supplementary Figure 1: Evidence for Auditory Lexical Representations in the AWFA.

Adaptation profile for auditory real words (RWs) using the full dataset (n=26). RWs show tight tuning to individual real words consistent with an auditory lexicon. ***, **, *, and N.S. mark: p ≤0.001, ≤0.01, ≤0.05, and not significant (>0.1), all Bonferroni-corrected for multiple comparisons.

Adaptation Patterns to Pseudowords in the AWFA Exhibit Lexical Selectivity After, but not Before Familiarization

Next, in the pre- and post-training scans (UTPW and TPW, respectively), we tested the hypothesis that familiarization with previously novel pseudowords drives the formation of lexical selectivity in the AWFA. To do so, we examined the adaptation profiles for RWs, pseudowords before training (untrained pseudowords, UTPW), and pseudowords after training (trained pseudowords, TPW) in subjects who had completed all three scans (n = 16). We ran a 2-way repeated measures ANOVA to investigate the relationship between lexicality (RW, UTPW, and TPW) and similarity (SAME, 1PH, and DIFF). This revealed a significant main effect of similarity (F2,30 = 43.023; p = 2.95E-9) but not a significant main effect of lexicality (F2,30 = 2.398; p = 0.116). Critically, there was again a significant interaction between lexicality and similarity (F4,60 = 4.144; p = 0.012).

Consistent with the full dataset (Fig. 3), planned paired t-tests revealed an adaptation profile that was consistent with tight neural tuning for individual RWs (Fig. 4). There was a significant difference in the mean percent signal change between the DIFF vs. SAME conditions (t(15) = 7.28; p < 0.001) and 1PH vs. SAME (t(15) = 5.85; p < 0.001). However, there was no significant difference between the DIFF and 1PH conditions (t(15) = 0.750; p = 1). In contrast, the adaptation profile for UTPW was not consistent with lexical selectivity (Fig. 4): There was a significant difference between the DIFF vs. SAME conditions (t(15) = 7.84; p < 0.001) and the DIFF vs. 1PH conditions (t(15) = 4.57; p = 0.001), but not 1PH vs. SAME (t(15) = 2.16; p = 0.141). Crucially, the adaptation profile for PW after training (i.e., TPW) was consistent with lexical selectivity (Fig. 4): There was a significant difference in the mean percent signal between the DIFF vs. SAME conditions (t(15) = 5.55; p < 0.001) and the 1PH vs. SAME (t(15) = 3.17; p = 0.019). However, there was no significant difference between the DIFF and 1PH conditions (t(15) = 1.71; p = 0.327).

Figure 4:
Figure 4: Auditory Lexical Representations Emerge in the AWFA for Pseudowords After Familiarization Training.

Within-subject (n = 16) adaptation profile for auditory real words (RW), untrained pseudowords (UTPW), and trained pseudowords (TPW). RW adaptation profile shows tuning to individual real words consistent with an auditory lexicon. Untrained pseudowords (UTPW) show a graded release from adaptation as a function of phonological similarity. Importantly, following familiarization training, adaptation patterns in the AWFA to the same pseudowords (now trained, TPW) reveal tight lexical tuning, similar to RW. ***, **, *, and N.S. mark: p ≤0.001, ≤0.01, ≤0.05 and not significant (>0.1), all Bonferroni-corrected.

The AWFA Connects to the Language Network

We next calculated task-based functional connectivity in the auditory localizer dataset (n=26) to examine the connections between the AWFA and the rest of the brain. This seed-to-voxel analysis (Fig. 5) showed that the AWFA is highly connected with brain regions previously shown 48 to be involved in language processing such as the inferior frontal gyrus (local peak at MNI -46, 12, 26) and premotor cortex (local peak at MNI -50, -8, 48). Especially noteworthy is the connectivity between the AWFA and a cluster in the left posterior fusiform cortex (circled in red; MNI: -44, -46, -14) that encompases the reported location of the VWFA (MNI: -45, -54, -20) 12,49.

Figure 5:
Figure 5: Functional Connectivity of the AWFA.

Whole-brain functional connectivity of the AWFA during the auditory localizer task (n=26). Results are thresholded at a voxel-wise p ≤ 0.001 and cluster-level p ≤ 0.05, FWE corrected. Cluster corresponding to the literature coordinates of the VWFA is circled in red.

View this table:
Supplementary Table 2.

Significant Clusters of Connectivity With the AWFA

Discussion

Cognitive models of speech comprehension 50–52 have proposed the existence of an auditory lexicon. More recent models of speech comprehension in the brain 2,3 have posited the existence of such an auditory lexicon in an auditory ventral “what” stream. Yet, significant disagreement exists between these models about the location of such an auditory lexicon, and no studies to date have directly tested the existence of an auditory lexicon for speech comprehension in the brain. In prior work 19,20,44, we have used fMRI rapid adaptation techniques to establish the existence of an orthographic lexicon in a region of the visual ventral stream known as the Visual Word Form Area, VWFA (subsequently confirmed by human neurophysiological recordings 21,22,53). In the present study, we leveraged these techniques to test the existence of an auditory lexicon in the auditory ventral stream, particularly in the “Auditory Word Form Area” (AWFA) of the anterior STG 26,31. We first defined the ROI for an individual auditory word form area (AWFA) through an independent auditory localizer scan. We then showed in RW and UTPW RA scans that, consistent with an auditory lexicon, spoken RWs engaged distinct neural populations in the AWFA, with even a single phoneme change causing full release from adaptation, whereas pseudowords did not exhibit this lexical adaptation profile but instead showed a graded release from adaptation as a function of phoneme overlap. This graded release from adaptation suggests that neurons in the AWFA, like the VWFA 20, exhibit broader tuning to pseudowords due to experience-driven refinement of tuning of neurons to real words but not pseudowords. Thus, these results directly replicated analogous findings of highly selective lexical tuning to real words but not pseudowords in the VWFA 20. Then, in the TPW RA scan, we showed that training subjects to recognize a set of pseudowords led to the development of lexical selectivity for these TPWs, again replicating previous results for written words in the VWFA 19. Finally, the AWFA was connected to an area in the left posterior fusiform cortex coincident with the reported location of the VWFA, further supporting analogous roles of the AWFA and VWFA in the processing of spoken and written words, respectively.

Our novel evidence of auditory lexical representations in the brain informs current cognitive models of speech comprehension. While all these models map acoustic input to meaning, they disagree on whether auditory lexica exist 18,54. Our data do not only present compelling evidence for the existence of an auditory lexicon – they additionally place its location in the anterior STG where it is ideally suited to interface with semantic representations located further anteriorly in the temporal lobe 16,55, thereby completing the mapping of speech sounds to meaning. Furthermore, this anterior STG location is consistent with prior studies demonstrating other familiar “auditory objects” 56 such as human voices 57–59 or musical instruments 60. Moreover, such a simple-to-complex progression in selectivity from simple perceptual features over lexical representations to semantic representations in the anteroventral auditory processing stream along the superior temporal cortex is a direct counterpart of the ventral visual stream in the inferior temporal cortex 9,14, revealing convergent processing strategies across speech modalities.

Materials and Methods

Overall Procedure

In this study, participants completed an auditory localizer scan and three RA experiments over the course of three scanning sessions. In the first session, subjects completed the auditory localizer and real word (RW) scans. The auditory localizer scan was used to identify a candidate AWFA region in the anterior auditory ventral stream2 – adopting the approach used in the visual/orthographic case for defining the VWFA 19,20,44. The RW RA scan was used to test if the candidate AWFA exhibited lexical selectivity for RWs. In the remaining two scanning sessions subjects completed a pre- and then a post-training PW scans, respectively, separated by 6 behavioral training sessions outside of the scanner.

During the pre-training scan, subjects were presented with the then untrained pseudowords (UTPW). Next, subjects completed six behavioral training sessions consisting of a 2-back and a novel/familiar task designed to familiarize and assess subject familiarity with the UTPW, respectively. Subjects completed a maximum of one behavioral session each day and those participants that achieved at least 80% accuracy on the novel/familiar task by the final session qualified for the post-training scan. During the post-training scan subjects were presented with the same set of pseudowords, now called trained pseudowords (TPW). The pre- and post-training scans were used to test if the candidate AWFA exhibited lexical selectivity for TPW (i.e., after training) but not to the UTPW (i.e., before training).

Participants

We recruited a total of 28 right-handed healthy native English speakers for this study (ages 18-34, 12 females). This sample size was based on power analyses conducted on prior studies. In two prior studies (Glezer et al., 2015, 2009) that used fMRI rapid adaptation to test the existence of a visual lexicon the effect size (Cohen’s d) for the crucial paired t-test that differentiated RWs and UTPWs d = 0.87 and 0.69. In another study (Chevillet et al., 2013) that used fMRI rapid adaptation to test phoneme category invariance the effect size (Cohen’s d) for the paired t-test between the adaptation for within versus across category exemplars was d = 0.93. Using G*Power 3.1 (dependent samples t-test, minimal effect size = 0.69, two-tail α = 0.05, and power = 0.8) found that the sample size is 19. Georgetown University’s Institutional Review Board approved all experimental procedures, and written informed consent was obtained from all subjects before the experiment. Two subjects were excluded from further analyses for performing 2 standard deviations below the average on the in-scanner task. Furthermore, two subjects dropped out of the study after completing the RW RA scan. In total, we analyzed 26 subjects for the RW RA scan and 24 of those 26 for the pre-training UTPW scan. Due to subject dropout (n=4) or failure to achieve 80% accuracy on the novel/familiar task (n=4), 16 out of the 24 subjects were analyzed for the post-training TPW scan.

Stimuli

Real word stimuli, for the RW RA experiment, were chosen using the English Lexicon Project 61. Analogous to our studies of the neural representation of written words 19,20, three sets of fifty high-frequency (>50 per million), monosyllabic RWs that were 3-5 phonemes in length were created. One set of words (target words) served as the reference for the other two lists. The second set was created by altering each target word by a single phoneme to create another real word. The third set was created by selecting for each target word another real word that had the same number of (but no shared) phonemes. All three of these lists were matched on the number of phonemes, orthographic neighborhood, and phonological neighborhood. To create the UTPW/TPW we used MCWord (Medler, D.A., & Binder, 2005) to generate four sets of 50 target PWs, 3-5 phonemes in length. One set to be trained and three sets to remain untrained and serve as foils in the training task. As with the RW stimuli, we then used the target PW as the reference to generate a set of pseudowords each differing by one phoneme from the target word, and another set of pseudowords matched to the target pseudowords by number of phonemes but not sharing any phonemes. As in prior work 19,20 real word and pseudoword sets were matched for length, bigram and trigram frequency, and phonological neighborhood. All stimuli were recorded using a 44.1-kHz sampling rate in a sound-isolated booth by a trained female native speaker of American English.

Auditory Localizer Scan

The auditory localizer scan was used to independently identify auditory speech-selective areas. In this scan, subjects were randomly presented with trials from one of five conditions: “Real Words”, “Pseudowords”, “Scrambled Real Words”, “Scrambled Pseudowords”, and “Silence”. “Real Words” and “Pseudowords” were one syllable long; the lists were matched for length, orthographic and phonologic neighborhood. “Scrambled Real Words” and “Scrambled Pseudowords” were generated by randomly rearranging 200 ms by 1-octave tiles of the constant Q spectrogram62 for each real word or pseudoword, respectively, and then reconstructing a time-domain waveform with an inverse transform33. Each trial in the auditory localizer scan was 9s long and began with 2.5 s of silence (during which 1.68 s of scanner acquisition occurred) followed by a ∼3-4 s stimulus presentation period and concluding with silence. During the stimulus presentation period, subjects heard four stimuli from a given condition and responded with a left-handed button press if any of the four stimuli was a repeat within that block. In total, there were 145 trials per run with 25 trials of each of the five conditions and five one-back trials for each of the four non-silence conditions. An additional 18 s of fixation were added to the start and end of each run. Subjects completed five runs of the task.

Rapid Adaptation (RA) Scans

There were three RA scans (i.e., RW, UTPW, TPW) performed on different days. In the rapid adaptation (RA) scans, subjects heard a pair of words (prime/target) on each trial. The words in each pair were either identical (SAME), differed by a single phoneme (1PH), or shared no phonemes at all (DIFF). These pairs were generated by using the three matched word lists described above. To engage subjects’ attention, we asked subjects to perform an oddball rime detection task in the scanner. To do so, we created an additional condition (oddball) in which a word or pseudoword containing the oddball rime ‘-ox’ (e.g., socks, grox) was presented in lieu of either the prime or target word. Participants were asked to attentively listen to all stimuli and respond with a left-handed button press when an oddball stimulus was heard. In all three scans, the number of repetitions of each word was counterbalanced across all conditions to control for long-lag priming effects63. Trial order and timing was adjusted using M-sequences64. Each trial was 3.36 s long and consisted of a 1.68 s silent period followed by 1.68 s stimulus presentation period during which the word pairs were presented. Following prior auditory RA studies45,46 we presented stimuli with a 50ms ISI. In total, there were 25 trials of each condition (SAME, 1PH, DIFF, oddball, and silence) for a total of 125 trials per run. An additional 10.08 s of fixation was added to the start and end of each run. Subjects completed 4 runs for each scan.

Data Acquisition

MRI data were acquired at the Center for Functional and Molecular Imaging at Georgetown University on a 3.0 Tesla Siemens Prisma-fit scanner. We used whole-head echo-planar imaging sequences (flip angle = 70°, TE = 35 ms, FOV = 205mm, 102×102 matrix) with a 64-channel head coil. Building off previous auditory localizer65 and RA paradigms in our lab 46, a slow (TR = 9000 ms, TA = 1680 ms) and a fast (TR = 3360 ms, TA = 1680 ms) clustered acquisition paradigm were used for the auditory localizer and RA scans, respectively. 54 axial slices were acquired in descending order (thickness = 1.8 mm, 0.18 mm gap; in-plane resolution = 2.0 × 2.0 mm2). A T1-weighted MPRAGE image (resolution 1×1×1 mm3) was also acquired for each subject.

fMRI Data Preprocessing

Image preprocessing was performed using SPM12 (http://www.fil.ion.ucl.ac.uk/spm/software/spm12/). The first three volumes of each run were discarded to allow for T1 stabilization, and the remaining EPI images were spatially realigned to the mean BOLD reference image. No slice-time correction was done given the presence of temporal discontinuities between successive volumes in clustered acquisition paradigms 66. EPI images for each subject were co-registered to the anatomical image. The anatomical image was then segmented and the resulting deformation fields for spatial normalization were used to normalize the functional data to the standard MNI space. Next, we smoothed the normalized functional images with a 4-mm FWHM Gaussian kernel. Finally, a first-level model containing regressors for each condition and the six motion parameters from realignment was fit. In the auditory localizer scan, the regressors were: “Real Words”, “Pseudowords”, “Scrambled Real Words”, “Scrambled Pseudowords”, “Button Press”, and “Silence”. In the RA scans, the regressors were: “SAME”, “1PH”, “DIFF”, “Button Press”, and “Silence”.

Defining the Auditory Word Form Area (AWFA)

The auditory word form area (AWFA) was determined at the individual subject-level using the auditory localizer scan. Analogous to our studies of the visual word form area 19,20,44, for each subject we first defined an “RW-Silence” contrast and an “RW-Scrambled Words” contrast. Next, at the group level we masked the “RW-Silence” t-statistic map with the “RW-Scrambled Words” t-statistic map thresholded at p ≤ 0.05 before applying a p ≤ 0.001 voxel-wise threshold. The resulting map had a peak at MNI: -62, -14, 2. These coordinates are consistent with the hypothesized locus (MNI: -61, - 15, -5) of the AWFA from prior studies 26. We then created the same “RW-Silence” masked by “RW-Scrambled Words” maps at the individual-subject level using the same thresholds as above. Then, for each subject we identified the local peak in the resulting maps closest to the group peak (MNI: -62, -14, 2). Finally, to create each individual subject’s AWFA, we created a region of interest (ROI) consisting of the 50 closest voxels to each individual’s local peak.

Behavioral Training

Subjects were trained to recognize 150 auditory PWs (TPW, see above). Following previous work19, a 2-back training task, in which subjects had to detect repeats of PW separated by another PW, was used to familiarize subjects with the PWs. Each session of the 2-back task consisted of 15 blocks of 75 trials each with self-paced breaks between each block. Each trial lasted 1.5 s during which subjects heard a single PW and had to respond if they heard a 2-back repeat (i.e., if the current PW was the same as the PW before the last). Each block lasted 112.5 s, and each session lasted for a total task length of 28.125 min excluding breaks. Following the 2-back task, subjects’ familiarity with the trained PWs was assessed using a novel/familiar task. For this task, we developed 3 sets of foils for each of the 150 PWs. Each foil differed from its base PW by a single phoneme. Each session of the novel/familiar task consisted of 3 blocks of 100 trials with self-paced breaks between each block. Each trial lasted 1.5 s during which subjects heard a single PW and had to respond with either the ‘left’ or ‘right’ arrow key to indicate a novel (i.e., a foil) or familiar (i.e., trained) PW. In total, each block lasted 150 s for a total task length of 7.5 min excluding breaks. Over the course of the novel/familiar task sessions each foil list was paired with the trained PW list only twice with at least 2 days since the last pairing. Six sessions of both tasks were performed over the course of ∼8 days. To proceed to the post-training rapid adaptation (RA) scan, subjects had to achieve at least 80% accuracy on the novel/familiar task by their 6th session.

Task-Based Functional Connectivity

After preprocessing, we used the CONN toolbox 21.a 67 to calculate seed-to-voxel stimulus-driven functional connectivity from the AWFA during auditory word processing in the auditory localizer scans. Word processing was included as the primary task condition by combining the onset times and durations for RW and PW into one condition. The individual subject (n=26) AWFA seeds were used for the analysis. We then performed denoising with confounds including the subject-specific motion regressors computed during preprocessing in SPM, as well as modeling nuisance covariates including cerebrospinal fluid and white-matter signals and their derivatives, following the CompCor strategy 68 as implemented in CONN. Data were then band-pass filtered (0.008-0.009 Hz) with linear detrending. Seed-to-voxel functional connectivity was performed using a weighted-GLM computing the bivariate correlation between the AWFA seed with the whole brain. Due to the sparse acquisition type of this analysis, no HRF weighting was performed. Group-level significance was determined with a voxel threshold of p<0.001 and a cluster threshold of p<0.05, FDR-corrected.

Competing Interests

The authors declare no competing financial interests

Acknowledgements

We thank Iain DeWitt for providing the auditory scrambling code used for the localizer scans. The research described in this paper was funded by NSF (grant BCS-1756313). Additional support was provided by the NIH (grant 1S10OD023561). This work also used resources provided through the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562

References

  1. 1.↵
    Rauschecker, J. P. & Tian, B. Mechanisms and streams for processing of “what” and “where” in auditory cortex. Proc National Acad Sci 97, 11800–11806 (2000).
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    Rauschecker, J. P. & Scott, S. K. Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nature Neuroscience 12, 718–724 (2009).
    OpenUrlCrossRefPubMedWeb of Science
  3. 3.↵
    Hickok, G. & Poeppel, D. The cortical organization of speech processing. Nature Reviews Neuroscience 8, rn2113 (2007).
  4. 4.↵
    Heijden, K. van der, Rauschecker, J. P., Gelder, B. de & Formisano, E. Cortical mechanisms of spatial hearing. Nat Rev Neurosci 20, 609–623 (2019).
    OpenUrlCrossRefPubMed
  5. 5.↵
    Rauschecker, J. P. Where, When, and How: Are they all sensorimotor? Towards a unified view of the dorsal pathway in vision and audition. Cortex 98, 262–268 (2018).
    OpenUrlCrossRefPubMed
  6. 6.↵
    Rauschecker, J. P. An expanded role for the dorsal auditory pathway in sensorimotor control and integration. Hearing Research 271, 16–25 (2011).
    OpenUrlCrossRefPubMed
  7. 7.
    Hickok, G., Houde, J. & Rong, F. Sensorimotor Integration in Speech Processing: Computational Basis and Neural Organization. Neuron 69, 407–422 (2011).
    OpenUrlCrossRefPubMedWeb of Science
  8. 8.↵
    Archakov, D. et al. Auditory representation of learned sound sequences in motor regions of the macaque brain. Proc National Acad Sci 117, 15242–15252 (2020).
    OpenUrlAbstract/FREE Full Text
  9. 9.↵
    Kravitz, D. J., Saleem, K. S., Baker, C. I., Ungerleider, L. G. & Mishkin, M. The ventral visual pathway: an expanded neural framework for the processing of object quality. Trends in Cognitive Sciences 17, 26–49 (2013).
    OpenUrlCrossRefPubMedWeb of Science
  10. 10.↵
    Felleman, D. J. & Essen, D. C. V. Distributed Hierarchical Processing in the Primate Cerebral Cortex. Cereb Cortex 1, 1–47 (1991).
    OpenUrlCrossRefPubMedWeb of Science
  11. 11.↵
    Hubel, D. H. & Wiesel, T. N. Ferrier lecture - Functional architecture of macaque monkey visual cortex. Proc Royal Soc Lond Ser B Biological Sci 198, 1–59 (1977).
    OpenUrl
  12. 12.↵
    Dehaene, S., Cohen, L., Sigman, M. & Vinckier, F. The neural code for written words: a proposal. Trends Cogn Sci 9, 335–341 (2005).
    OpenUrlCrossRefPubMedWeb of Science
  13. 13.↵
    Vinckier, F. et al. Hierarchical Coding of Letter Strings in the Ventral Stream: Dissecting the Inner Organization of the Visual Word-Form System. Neuron 55, 143–56 (2007).
    OpenUrlCrossRefPubMedWeb of Science
  14. 14.↵
    Damera, S. et al. From Shape to Meaning: Evidence for Multiple Fast Feedforward Hierarchies of Concept Processing in the Human Brain. Neuroimage 117148 (2020) doi:10.1016/j.neuroimage.2020.117148.
    OpenUrlCrossRef
  15. 15.
    Malone, P. S., Glezer, L. S., Kim, J., Jiang, X. & Riesenhuber, M. Multivariate Pattern Analysis Reveals Category-Related Organization of Semantic Representations in Anterior Temporal Cortex. Journal of Neuroscience 36, 10089–10096 (2016).
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    Ralph, M. A. L., Jefferies, E., Patterson, K. & Rogers, T. T. The neural and computational bases of semantic cognition. Nat Rev Neurosci 18, 42–55 (2017).
    OpenUrlCrossRefPubMed
  17. 17.↵
    Liuzzi, A. G., Bruffaerts, R. & Vandenberghe, R. The medial temporal written word processing system. Cortex (2019) doi:10.1016/j.cortex.2019.05.002.
    OpenUrlCrossRefPubMed
  18. 18.↵
    Coltheart, M. Are There Lexicons? Q J Exp Psychology Sect 57, 1153–1172 (2004).
    OpenUrl
  19. 19.↵
    Glezer, L. S., Kim, J., Rule, J., Jiang, X. & Riesenhuber, M. Adding Words to the Brain’s Visual Dictionary: Novel Word Learning Selectively Sharpens Orthographic Representations in the VWFA. J Neurosci 35, 4965–4972 (2015).
    OpenUrlAbstract/FREE Full Text
  20. 20.↵
    Glezer, L. S., Jiang, X. & Riesenhuber, M. Evidence for highly selective neuronal tuning to whole words in the “visual word form area”. Neuron 62, 199–204 (2009).
    OpenUrlCrossRefPubMedWeb of Science
  21. 21.↵
    Hirshorn, E. A. et al. Decoding and disrupting left midfusiform gyrus activity during word reading. Proceedings of the National Academy of Sciences 113, 8162–8167 (2016).
    OpenUrlAbstract/FREE Full Text
  22. 22.↵
    Woolnough, O. et al. Spatiotemporal dynamics of orthographic and lexical processing in the ventral visual pathway. Nat Hum Behav 1–10 (2020) doi:10.1038/s41562-020-00982-w.
    OpenUrlCrossRef
  23. 23.↵
    Dehaene, S. & Cohen, L. The unique role of the visual word form area in reading. Trends in Cognitive Sciences 15, 254–262 (2011).
    OpenUrlCrossRefPubMedWeb of Science
  24. 24.↵
    Kell, A., Yamins, D., Shook, E. N., Norman-Haignere, S. V. & McDermott, J. H. A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy. Neuron 98, 630-644.e16 (2018).
    OpenUrl
  25. 25.↵
    Rauschecker, J. P. Cortical processing of complex sounds. Curr Opin Neurobiol 8, 516–521 (1998).
    OpenUrlCrossRefPubMedWeb of Science
  26. 26.↵
    DeWitt, I. & Rauschecker, J. P. Phoneme and word recognition in the auditory ventral stream. Proceedings of the National Academy of Sciences 109, E505–E514 (2012).
    OpenUrlAbstract/FREE Full Text
  27. 27.↵
    Bogen, J. E. & Bogen, G. M. Wernicke’s Region - Where is it? Ann Ny Acad Sci 280, 834–843 (1976).
    OpenUrlCrossRefPubMedWeb of Science
  28. 28.↵
    Geschwind, N. The Organization of Language and the Brain: Language disorders after brain damage help in elucidating the neural basis of verbal behavior. Science 170, 940–944 (1970).
    OpenUrlFREE Full Text
  29. 29.↵
    Scott, S. K., Blank, C. C., Rosen, S. & Wise, R. J. S. Identification of a pathway for intelligible speech in the left temporal lobe. Brain 123, 2400–2406 (2000).
    OpenUrlCrossRefPubMedWeb of Science
  30. 30.↵
    Binder, J. R. et al. Human Temporal Lobe Activation by Speech and Nonspeech Sounds. Cereb Cortex 10, 512–528 (2000).
    OpenUrlCrossRefPubMedWeb of Science
  31. 31.↵
    Cohen, L., Jobert, A., Bihan, D. L. & Dehaene, S. Distinct unimodal and multimodal regions for word processing in the left temporal cortex. Neuroimage 23, 1256–1270 (2004).
    OpenUrlCrossRefPubMedWeb of Science
  32. 32.↵
    Rauschecker, J., Tian, B. & Hauser, M. Processing of complex sounds in the macaque nonprimary auditory cortex. Science 268, 111–114 (1995).
    OpenUrlAbstract/FREE Full Text
  33. 33.↵
    Ortiz-Rios, M. et al. Functional MRI of the vocalization-processing network in the macaque brain. Front Neurosci-switz 9, 113 (2015).
    OpenUrl
  34. 34.↵
    Tian, B., Reser, D., Durham, A., Kustov, A. & Rauschecker, J. P. Functional Specialization in Rhesus Monkey Auditory Cortex. Science 292, 290–293 (2001).
    OpenUrlAbstract/FREE Full Text
  35. 35.↵
    Jasmin, K., Lima, C. F. & Scott, S. K. Understanding rostral–caudal auditory cortex contributions to auditory perception. Nat Rev Neurosci 20, 425–434 (2019).
    OpenUrlCrossRefPubMed
  36. 36.
    Hamilton, L. S., Edwards, E. & Chang, E. F. A Spatial Map of Onset and Sustained Responses to Speech in the Human Superior Temporal Gyrus. Curr Biol 28, 1860-1871.e4 (2018).
    OpenUrlCrossRefPubMed
  37. 37.
    Hullett, P. W., Hamilton, L. S., Mesgarani, N., Schreiner, C. E. & Chang, E. F. Human Superior Temporal Gyrus Organization of Spectrotemporal Modulation Tuning Derived from Speech Stimuli. The Journal of Neuroscience 36, 2014–2026 (2016).
    OpenUrlAbstract/FREE Full Text
  38. 38.↵
    Kajikawa, Y. et al. Auditory Properties in the Parabelt Regions of the Superior Temporal Gyrus in the Awake Macaque Monkey: An Initial Survey. J Neurosci 35, 4140–4150 (2015).
    OpenUrlAbstract/FREE Full Text
  39. 39.↵
    DeWitt, I. & Rauschecker, J. P. Wernicke’s area revisited: Parallel streams and word processing. Brain Lang 127, 181–191 (2013).
    OpenUrlCrossRefPubMed
  40. 40.↵
    Evans, S. et al. The Pathways for Intelligible Speech: Multivariate and Univariate Perspectives. Cereb Cortex 24, 2350–2361 (2013).
    OpenUrl
  41. 41.↵
    Okada, K. et al. Hierarchical Organization of Human Auditory Cortex: Evidence from Acoustic Invariance in the Response to Intelligible Speech. Cereb Cortex 20, 2486–2495 (2010).
    OpenUrlCrossRefPubMedWeb of Science
  42. 42.↵
    Grill-Spector, K. & Malach, R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol 107, 293–321 (2001).
    OpenUrlCrossRefPubMedWeb of Science
  43. 43.↵
    Krekelberg, B., Boynton, G. M. & Wezel, R. J. A. van. Adaptation: from single cells to BOLD signals. Trends Neurosci 29, 250–256 (2006).
    OpenUrlCrossRefPubMedWeb of Science
  44. 44.↵
    Glezer, L. S. et al. Uncovering phonological and orthographic selectivity across the reading network using fMRI-RA. Neuroimage 138, 248–56 (2016).
    OpenUrlCrossRefPubMed
  45. 45.↵
    Jiang, X., Chevillet, M. A., Rauschecker, J. P. & Riesenhuber, M. Training Humans to Categorize Monkey Calls: Auditory Feature- and Category-Selective Neural Tuning Changes. Neuron 98, 405-416.e4 (2018).
    OpenUrl
  46. 46.↵
    Chevillet, M. A., Jiang, X., Rauschecker, J. P. & Riesenhuber, M. Automatic Phoneme Category Selectivity in the Dorsal Auditory Stream. The Journal of Neuroscience 33, 5208–5215 (2013).
    OpenUrlAbstract/FREE Full Text
  47. 47.↵
    Laird, A. R. et al. ALE meta-analysis: Controlling the false discovery rate and performing statistical contrasts. Hum Brain Mapp 25, 155–164 (2005).
    OpenUrlCrossRefPubMedWeb of Science
  48. 48.↵
    Wilson, S. M., Saygin, A. P., Sereno, M. I. & Iacoboni, M. Listening to speech activates motor areas involved in speech production. Nat Neurosci 7, 701–702 (2004).
    OpenUrlCrossRefPubMedWeb of Science
  49. 49.↵
    Kronbichler, M. et al. The visual word form area and the frequency with which words are encountered: evidence from a parametric fMRI study. Neuroimage 21, 946–953 (2004).
    OpenUrlCrossRefPubMedWeb of Science
  50. 50.↵
    McClelland, J. L. & Elman, J. L. The TRACE model of speech perception. Cognitive Psychol 18, 1–86 (1986).
    OpenUrl
  51. 51.
    Morton, J. Interaction of information in word recognition. Psychol Rev 76, 165–178 (1969).
    OpenUrlCrossRefWeb of Science
  52. 52.↵
    Marslen-Wilson, W. D. & Welsh, A. Processing interactions and lexical access during word recognition in continuous speech. Cognitive Psychol 10, 29–63 (1978).
    OpenUrl
  53. 53.↵
    Lochy, A. et al. Selective visual representation of letters and words in the left ventral occipito-temporal cortex with intracerebral recordings. Proc National Acad Sci 115, E7595–E7604 (2018).
    OpenUrlAbstract/FREE Full Text
  54. 54.↵
    Woollams, A. M. Lexical is as lexical does: computational approaches to lexical representation. Lang Cognition Neurosci 30, 395–408 (2015).
    OpenUrl
  55. 55.↵
    Ueno, T., Saito, S., Rogers, T. T. & Lambon Ralph, M. A. Lichtheim 2: Synthesizing Aphasia and the Neural Basis of Language in a Neurocomputational Model of the Dual Dorsal-Ventral Language Pathways. Neuron 72, 385–396 (2011).
    OpenUrlCrossRefPubMedWeb of Science
  56. 56.↵
    Griffiths, T. D. & Warren, J. D. What is an auditory object? Nat Rev Neurosci 5, 887–892 (2004).
    OpenUrlCrossRefPubMedWeb of Science
  57. 57.↵
    Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P. & Pike, B. Voice-selective areas in human auditory cortex. Nature 403, 309–312 (2000).
    OpenUrlCrossRefPubMedWeb of Science
  58. 58.
    Bodin, C. et al. Functionally homologous representation of vocalizations in the auditory cortex of humans and macaques. Curr Biol 31, 4839-4844.e4 (2021).
    OpenUrl
  59. 59.↵
    Staib, M. & Frühholz, S. Distinct functional levels of human voice processing in the auditory cortex. Cereb Cortex (2022) doi:10.1093/cercor/bhac128.
    OpenUrlCrossRef
  60. 60.↵
    Leaver, A. M. & Rauschecker, J. P. Cortical Representation of Natural Complex Sounds: Effects of Acoustic Features and Auditory Object Category. J Neurosci 30, 7604–7612 (2010).
    OpenUrlAbstract/FREE Full Text
  61. 61.↵
    Balota, D. A. et al. The English Lexicon Project. Behavior Research Methods 39, 445–459 (2007).
    OpenUrlCrossRefPubMedWeb of Science
  62. 62.↵
    Brown, J. C. Calculation of a constant Q spectral transform. The Journal of the Acoustical Society of America 89, 425–434 (1991).
    OpenUrlCrossRefWeb of Science
  63. 63.↵
    Henson, R., Shallice, T. & Dolan, R. Neuroimaging Evidence for Dissociable Forms of Repetition Priming. Science 287, 1269–1272 (2000).
    OpenUrlAbstract/FREE Full Text
  64. 64.↵
    Buračas, G. T. & Boynton, G. M. Efficient Design of Event-Related fMRI Experiments Using M-Sequences. Neuroimage 16, 801–813 (2002).
    OpenUrlCrossRefPubMedWeb of Science
  65. 65.↵
    Damera, S. R. et al. Metamodal Coupling of Vibrotactile and Auditory Speech Processing Systems Through Matched Stimulus Representations. (2021) doi:10.1101/2021.05.04.442660.
    OpenUrlAbstract/FREE Full Text
  66. 66.↵
    Perrachione, T. K. & Ghosh, S. S. Optimized Design and Analysis of Sparse-Sampling fMRI Experiments. Front Neurosci-switz 7, 55 (2013).
    OpenUrl
  67. 67.↵
    Whitfield-Gabrieli, S. & Nieto-Castanon, A. Conn: A Functional Connectivity Toolbox for Correlated and Anticorrelated Brain Networks. Brain Connectivity 2, 125–141 (2012).
    OpenUrl
  68. 68.↵
    Behzadi, Y., Restom, K., Liau, J. & Liu, T. T. A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37, 90–101 (2007).
    OpenUrlCrossRefPubMedWeb of Science
Back to top
Posted October 10, 2022.
Download PDF
Email
Evidence for a Spoken Word Lexicon in the Auditory Ventral Stream
Srikanth R. Damera, Lillian Chang, Plamen P. Nikolov, James A. Mattei, Suneel Banerjee, Laurie S. Glezer, Patrick H. Cox, Xiong Jiang, Josef P. Rauschecker, Maximilian Riesenhuber
bioRxiv 2022.10.09.511436; doi: https://doi.org/10.1101/2022.10.09.511436
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area