Abstract
Synchrony is a crucial cue indicating whether sensory signals are caused by single or independent sources. In order to be integrated and produce multisensory behavioural benefits, signals must co-occur within a temporal integration window (TIW). Yet, the underlying neural determinants and mechanisms of integration across asynchronies remain unclear. This psychophysics and electroencephalography study investigated the temporal constraints of behavioural response facilitation and neural interactions for evoked response potentials (ERP), inter-trial coherence (ITC), and time-frequency (TF) power. Participants were presented with noise bursts, ‘taps to the face’, and their audiotactile (AT) combinations at seven asynchronies: 0, ±20, ±70, and ±500 ms. Behaviourally we observed an inverted U-shape function for AT response facilitation, which was maximal for synchronous AT stimulation and declined within a ≤70 ms TIW. For ERPs, we observed AT interactions at 110 ms for near-synchronous stimuli within a ≤20 ms TIW and at 400 ms within a ≤70 ms TIW consistent with behavioural response facilitation. By contrast, AT interactions for theta ITC and ERPs at 200 ms post-stimulus were selective for ±70 ms asynchrony, potentially mediated via phase resetting. Finally, interactions for induced theta power and alpha/beta power rebound emerged at 800-1100 ms across several asynchronies including even 500 ms auditory leading asynchrony. In sum, we observed neural interactions that were confined to or extending beyond the behavioural TIW or specific for ±70 ms asynchrony. This diversity of temporal profiles and constraints demonstrates that multisensory integration unfolds in a cascade of interactions that are governed by distinct neural mechanisms.
Significance Statement Integrating information across audition and touch is critical for effective interactions with our environment. We are faster to swat a mosquito when we perceive a prick on the skin together with hearing the mosquito’s buzzing. Importantly, we should integrate signals only when they co-occur within a temporal integration window (TIW) and are hence likely to originate from a common source. This psychophysics/electroencephalography study unravels a multitude of neural interactions governed by different temporal constraints: interactions were confined to a TIW for ERPs, specific for one particular asynchrony for inter-trial coherence, and extending beyond the behavioural TIW for induced low frequency power. This diversity of temporal profiles demonstrates that distinct neural mechanisms mediate a cascade of multisensory integration processes.
Acknowledgements
FP7 ERC Starting Grant multsens (U.N.) and FP7 Marie Curie IntraEuropean Fellowship ISMINO (J.M.Z. and U.N.). We thank Christoph Braun and Elisa Leonardelli for assistance with the tactile device and Mate Aller for assistance with EEG and stimulus setup.
Footnotes
Conflict of Interest: The authors declare no competing financial interests.
