Multisensory Identification of Natural Objects in a Two-Way Crossmodal Priming Paradigm
Abstract
Abstract. The question of how vision and audition interact in natural object identification is currently a matter of debate. We developed a large set of auditory and visual stimuli representing natural objects in order to facilitate research in the field of multisensory processing. Normative data was obtained for 270 brief environmental sounds and 320 visual object stimuli. Each stimulus was named, categorized, and rated with regard to familiarity and emotional valence by N = 56 participants (Study 1). This multimodal stimulus set was employed in two subsequent crossmodal priming experiments that used semantically congruent and incongruent stimulus pairs in a S1-S2 paradigm. Task-relevant targets were either auditory (Study 2) or visual stimuli (Study 3). The behavioral data of both experiments expressed a crossmodal priming effect with shorter reaction times for congruent as compared to incongruent stimulus pairs. The observed facilitation effect suggests that object identification in one modality is influenced by input from another modality. This result implicates that congruent visual and auditory stimulus pairs were perceived as the same object and demonstrates a first validation of the multimodal stimulus set.
References
(2005). Functional imaging of human crossmodal identification and object recognition. Experimental Brain Research, 166, 559– 571
(1993). Common factors in the identification of an assortment of brief everyday sounds. Journal of Experimental Psychology: Human Perception and Performance, 19, 250– 267
(2004). Integration of auditory and visual information about objects in superior temporal sulcus. Neuron, 41, 809– 823
(1999). International affective digitized sounds (IADS): Stimuli, instruction manual and affective ratings(Tech. Rep. No. B-2). Gainesville, FL: The Center for Research in Psychophysiology, University of Florida.
(2004). Size matters: effects of stimulus size, duration and eccentricity on the visual gamma-band response. Clinical Neurophysiology, 115, 1810– 1820
(2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex, 11, 1110– 1123
Eds. (2004). The handbook of multisensory processes. Cambridge, MA: Massachusetts Institute of Technology.
(2004). Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology, Paris, 98, 1-3)– 191-205
(2003). Top-down attentional processing enhances auditory evoked gamma band activity. Neuroreport, 14, 683– 686
(2002). Auditory novelty oddball allows reliable distinction of top-down and bottom-up processes of attention. International Journal of Psychophysiology, 46, 77– 84
(2005). What is novel in the novelty oddball paradigm? Functional significance of the novelty P3 event-related potential as revealed by independent component analysis. Brain Research. Cognitive Brain Research, 22, 309– 321
(1997). Transfer between vision and haptics: Memory for 2-D patterns and 3-D objects. Psychonomic Bulletin and Review, 4, 403– 410
(2001). Dynamic predictions: Oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2, 704– 716
(1996). Naming norms for brief environmental sounds: Effects of age and dementia. Psychophysiology, 33, 462– 475
(2003). Stimulus characteristics and task category dissociate the anterior and posterior aspects of the novelty P3. Psychophysiology, 40, 198– 208
(1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473– 490
(2001). Visual-auditory events: Cross-modal perceptual priming and recognition memory. Consciousness and Cognition, 10, 425– 435
(2003). Neuroimaging studies of priming. Progress in Neurobiology, 70(1), 53– 81
(2007). Late auditory evoked potentials asymmetry revisited. Clinical Neurophysiology, 118, 1274– 1285
(1998). Emotion, motivation, and anxiety: Brain mechanisms and psychophysiology. Biological Psychiatry, 44, 1248– 1263
(1988). The international affective picture system[Photographic slides]. Gainesville, FL: Center for Research in Psychophysiology, University of Florida.
(1991). An investigation of cross-modality effects in implicit and explicit memory. The Quarterly Journal of Experimental Psychology A, 43(1), 19– 33
(2005). Auditory temporal processes in normal-hearing individuals and in patients with auditory neuropathy. Clinical Neurophysiology, 116, 669– 680
(2004). Multisensory visual-auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex, 14, 452– 465
(2004). Cross-modal object recognition. In G.A. Calvert, C. Spence, & B.E. Stein (Eds.), The handbook of multisensory processes (pp. 123-139). Cambridge, MA: MIT Press.
(2004). Revisiting Snodgrass and Vanderwart's object set: The role of surface detail in basic-level object recognition. Perception, 33, 217– 236
(2005). An online task for contrasting auditory processing in the verbal and nonverbal domains and norms for younger and older adults. Behavior Research Methods, 37(1), 99– 110
(2004). Multisensory convergence in early cortical processing. In G.A. Calvert, C. Spence, & T.R. Stanford (Eds.), The handbook of multisensory processes(pp. 295-309). Cambridge, MA: Massachusetts Institute of Technology.
(2005). Multisensory contributions to low-level, “unisensory” processing. Current Opinion Neurobiology, 15, 454– 458
(2005). Multisensory processing and oscillatory gamma responses: Effects of spatial selective attention. Experimental Brain Research, 166, 411– 426
(2004). How to select stimuli for environmental sound research and where to find them. Behavior Research Methods, Instruments and Computers, 36, 590– 598
(1980). A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6, 174– 215
(1993). The merging of the senses. Cambridge, MA: MIT Press.
(2001). On specification and the senses. The Behavioral and Brain Sciences, 24, 195– 213
(1995). Priming the identification of environmental sounds. The Quarterly Journal of Experimental Psychology A, 48, 741– 761
(1996). From auditory image to auditory percept: Facilitation through common processes?. Memory and Cognition, 24, 296– 304
(1996). Speed of processing in the human visual system. Nature, 381, 520– 522
(1990). Priming and human memory systems. Science, 247, 301– 306
(1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638– 667
(2004). Visual, haptic and cross-modal recognition of objects and scenes. Journal of Physiology, Paris, 98, 1-3)– 147-159
(1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, Monograph, Supplement, 9, 1– 27
