Contrary neuronal recalibration in different multisensory cortical areas

The adult brain demonstrates remarkable multisensory plasticity by dynamically recalibrating itself based on information from multiple sensory sources. After a systematic visual–vestibular heading offset is experienced, the unisensory perceptual estimates for subsequently presented stimuli are shifted toward each other (in opposite directions) to reduce the conflict. The neural substrate of this recalibration is unknown. Here, we recorded single-neuron activity from the dorsal medial superior temporal (MSTd), parietoinsular vestibular cortex (PIVC), and ventral intraparietal (VIP) areas in three male rhesus macaques during this visual–vestibular recalibration. Both visual and vestibular neuronal tuning curves in MSTd shifted – each according to their respective cues’ perceptual shifts. Tuning of vestibular neurons in PIVC also shifted in the same direction as vestibular perceptual shifts (cells were not robustly tuned to the visual stimuli). By contrast, VIP neurons demonstrated a unique phenomenon: both vestibular and visual tuning shifted in accordance with vestibular perceptual shifts. Such that, visual tuning shifted, surprisingly, contrary to visual perceptual shifts. Therefore, while unsupervised recalibration (to reduce cue conflict) occurs in early multisensory cortices, higher-level VIP reflects only a global shift, in vestibular space.

. Thus, in both cases ( Fig. 2A, B), both the vestibular and the visual cues 197 shifted in the direction required to reduce the cue conflict (i.e. in opposite directions).

225
This result is also consistent with our previous study. Thus, the behavioral results from 226 the original study (performed in the Angelaki laboratory) were replicated in these 227 experiments (in the Chen laboratory) using a new set of monkeys, with simultaneous 228 neuronal recording. In the following sections, we present how neuronal responses in 229 areas MSTd, PIVC, and VIP (Fig. 3, 4, and 5, respectively) recalibrated in comparison to the behavioral shifts.
shifts 233 Responses of an example neuron recorded from MSTd during unsupervised 234 recalibration are presented in Figure 3A. Behaviorally, the vestibular PSE shifted

314
To test whether this mismatch between behavior and tuning for visual cues in VIP 315 relates to specific subtypes of neurons, we sorted the VIP data into three subsets:  . 1B). Therefore, the contrary shifts of visual tuning in VIP seem to reflect a general 324 feature of this cortical area, rather than an anomaly of a subgroup of neurons. 326 The tuning curves in were seen for both vestibular and visual cues during the stimulus (Fig. 6A). The   (Fig. 6C). This pattern is in sustained activity correlated with subsequent vestibular choices, and was contrary to 355 visual choices. Thus the sustained activity is not generically choice related, but rather 356 in accordance with recalibrated vestibular function.

358
Correlations between neuronal and behavioral PSE shifts, using the neuronal activity at heading, separately for rightward and leftward choices, (Fig. 7). In the 373 pre-recalibration block vestibular responses were strongly choice related (Fig. 7A, left   374 plot) -neuronal responses to the same heading stimulus were larger when followed    0.47, paired t-test; Fig. 8A) cues. For these statistical 398 comparisons and for plotting we used the squared partial correlations (which quantify 399 the amount of unique variance explained by choice or heading). We did not observe 400 any changes in partial correlations in areas PIVC and MSTd (Supplemental Fig. 2). 401 Lastly, there was no evidence for differences between post-and pre-recalibration    423 To investigate the neuronal bases of unsupervised cross-sensory recalibration, 424 we first replicated the behavioral results from our previous study (Zaidel et al., 2011).  However, choice-related activity cannot explain the results here, because the 503 predicted shifts in neuronal tuning would be in the same direction as the altered 504 choices (behavioral shifts), whereas we found contrary visual recalibration. To 505 understand contrary shifts that could arise despite strong choice-related activity in VIP, 506 we investigated choice tuning in VIP neurons. We found that choice tuning in VIP 507 decreased during unsupervised calibration. This allowed contrary shifts to be exposed.  The data and analysis code for this study have been uploaded to github and can be 741 found at https://github.com/FuZengBio/Recalibration.