Functional MRI mapping of dynamic visual features during natural viewing in the macaque
Introduction
During an animal's natural visual experience, the brain receives information about the environment in a manner that differs from that in most electrophysiological or fMRI experiments. For example, when a macaque monkey observes its conspecifics, visual input consists not of a series of isolated presentations, but of an evolving thread of dynamic, high- and low-level features superimposed in space and time. While it is straightforward to record neural or fMRI activity while monkeys observe natural movies, few such experiments have been carried out, likely because of the challenges inherent in the data analysis and interpretation. However, given that the brain evolved and operates under such conditions, it may be of great value to use natural paradigms that can complement more conventional approaches to assess functional responses in the brain.
Human fMRI experiments have begun to develop and apply such paradigms, often by having subjects watch commercial movies. Various experiments have used voxel time courses to assess the shared signal variation across brain areas (Bartels and Zeki, 2004, Bartels and Zeki, 2005), map intersubject correlations (Hasson et al., 2004, Hasson et al., 2008), and measure the degree to which brain activity is predicted by certain features (Bartels et al., 2008, Hanson et al., 2007, Huth et al., 2012). These studies have shown that, despite the challenges of superimposed features, it is possible to create functional maps and assess aspects of functional brain organization under natural viewing conditions. One study has gone further, matching the voxel time courses in the human brain to those in macaques watching the same movies with the aim of establishing homological correspondence between the species (Mantini et al., 2012, Mantini et al., 2013).
The macaque extrastriate visual cortex is characterized by having a large number of specialized areas. Visual signals are analyzed by regions apparently dedicated for the processing of motion (Maunsell and Van Essen, 1983), complex spatial structure (Fujita et al., 1992), and spatial cues (Andersen, 1985). Testing with fMRI has identified functional networks, such as that specialized for the processing of faces (Tsao et al., 2003). Does this apparent division of labor, derived from conventional testing, govern neural activity in the macaque's brain during natural viewing?
Here we explore this question by measuring fMRI activity in macaques freely viewing extended natural videos containing diverse social and nonsocial content. Using a family of feature models extracted from each video, we assessed the relative contribution of different visual features throughout the brain. We first show that face feature models yield maps that bear striking similarity to the face patches identified using conventional block design. We next show that in the face patches, as well as in neighboring regions of V4, TEO and TE, the motion feature models, whose time courses are largely uncorrelated with the face feature models, are the primary drivers of the fMRI voxel time courses. These findings demonstrate that natural viewing paradigms can be valuable assays of functional specialization in the macaque brain, and additionally underscore the strong contribution of magnocellular input to ventral stream visual processing during natural vision.
Section snippets
Subjects
Three adult female rhesus monkeys (M1, M2, M3) participated in the study. Prior to training, monkeys were implanted with a custom-designed and fabricated fiberglass headpost, which was used to immobilize the head during testing. All procedures were approved by the Animal Care and Use Committee of the US National Institutes of Health (National Institute of Mental Health) and followed US National Institutes of Health guidelines. Surgery was performed using sterile procedures (see Maier et al.,2008
Results
The main goal of the study was to measure the contribution of different stimulus features to fMRI responses throughout the brain during natural viewing. The fMRI data were obtained from three monkeys, who repeatedly viewed up to eighteen 5-min natural videos inside the bore of a 4.7 T vertical scanner. The movies varied in their content, but generally depicted conspecific and heterospecific monkeys, humans, and other animals engaged in a variety of social and nonsocial behaviors (see Video S1
Discussion
We used fMRI data collected during the free viewing of natural movies to investigate how the brain processes visual features that co-occur during natural vision. We found that we could readily map the locations of the face patches across three animals using a model based on the presence of faces. Additionally, we found that motion dominated the responses throughout the ventral visual stream, both within known motion selective regions in the lower bank of the STS (Bartels et al., 2008, Fischer
Acknowledgments
We would like to thank Tahir Haque for scoring the videos, Charles Zhu and Dr. Frank Ye for assistance with fMRI scanning, Katy Smith, George Dold and David Ide for excellent technical assistance and development, Drs. Takaaki Kaneko and Saleem Kadharbatcha for helpful discussion, Hang Joon Jo, Ziad Saad and Colin Reveley for guidance in fMRI analysis, and Drs. Chia-chun Hung and Brian Scott for comments on the manuscript. Functional and anatomical MRI scanning was carried out in the
References (61)
- et al.
Brain dynamics during natural viewing conditions–a new guide for mapping connectivity in vivo
NeuroImage
(2005) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages
Comput. Biomed. Res.
(1996)- et al.
Facial expression and gaze-direction in human superior temporal sulcus
Neuropsychologia
(2007) - et al.
Activity of inferior temporal neurons in behaving monkeys
Neuropsychologia
(1979) - et al.
Brain-to-brain coupling: a mechanism for creating and sharing a social world
Trends Cogn. Sci. (Regul. Ed.)
(2012) - et al.
A continuous semantic space describes the representation of thousands of object and action categories across the human brain
Neuron
(2012) - et al.
Integration of shape and motion cues in biological motion processing in the monkey STS
NeuroImage
(2012) - et al.
The ventral visual pathway: an expanded neural framework for the processing of object quality
Trends Cogn. Sci. (Regul. Ed.)
(2013) - et al.
Repeated fMRI using iron oxide contrast agent in awake, behaving macaques at 3 Tesla
NeuroImage
(2002) - et al.
Human-monkey gaze correlations reveal convergent and divergent patterns of movie viewing
Curr. Biol.
(2010)
Uncovering the visual “alphabet”: advances in our understanding of object perception
Vis. Res.
Ocolomotor adaptation: adaptive mehans in gaze control
Science
Functional brain mapping during free viewing of natural scenes
Hum. Brain Mapp.
Natural vision reveals regional specialization to local motion and to contrast-invariant, global flow in the human brain
Cereb. Cortex
FMRI responses to video and point-light displays of moving humans and manipulable objects
J. Cogn. Neurosci.
Object representations in the temporal cortex of monkeys and humans as revealed by functional magnetic resonance imaging
J. Neurophysiol.
Visual responses to targets and distracters by inferior temporal neurons after lesions of extrastriate areas V4 and TEO
Neuroreport
Pathways for motion analysis: cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque
J. Comp. Neurol.
Stimulus-selective properties of inferior temporal neurons in the macaque
J. Neurosci.
Form representation in monkey inferotemporal cortex is virtually unaltered by free viewing
Nat. Neurosci.
Cortical connections of inferior temporal area TEO in macaque monkeys
J. Comp. Neurol.
Responses in macaque visual area V4 following inactivation of the parvocellular and magnocellular LGN pathways
J. Neurosci.
Visual motion responses in the posterior cingulate sulcus: a comparison to V5/MT and MST
Cereb. Cortex
Columns for visual features of objects in monkey inferotemporal cortex
Nature
Dynamic and static facial expressions decoded from motion-sensitive areas in the macaque monkey
J. Neurosci.
A biologically plausible model for rapid natural image identification
Adv. Neural Inf. Process. Syst.
How do rhesus monkeys (Macaca mulatta) scan faces in a visual paired comparison task?
Anim. Cogn.
Spatially pooled contrast responses predict neural and perceptual similarity of naturalistic image categories
PLoS Comput. Biol.
Visual properties of neurons in inferotemporal cortex of the Macaque
J. Neurophysiol.
Bottom-up and top-down brain functional connectivity underlying comprehension of everyday visual action
Brain Struct. Funct.
Cited by (66)
Hierarchical organization of social action features along the lateral visual pathway
2023, Current BiologyOptogenetic stimulation of the primary visual cortex drives activity in the visual association cortex (Revision to CRNEUR-D-22-00114)
2023, Current Research in NeurobiologyFrom remembering to reconstruction: The transformative neural representation of episodic memory
2022, Progress in NeurobiologyNonhuman primate models of ischemic stroke and neurological evaluation after stroke
2022, Journal of Neuroscience Methods