Spatial updating and the maintenance of visual constancy

doi:10.1016/j.neuroscience.2008.07.079

Neuroscience

Volume 156, Issue 4, 28 October 2008, Pages 801-818

https://doi.org/10.1016/j.neuroscience.2008.07.079 Get rights and content

Abstract

Spatial updating is the means by which we keep track of the locations of objects in space even as we move. Four decades of research have shown that humans and non-human primates can take the amplitude and direction of intervening movements into account, including saccades (both head-fixed and head-free), pursuit, whole-body rotations and translations. At the neuronal level, spatial updating is thought to be maintained by receptive field locations that shift with changes in gaze, and evidence for such shifts has been shown in several cortical areas. These regions receive information about the intervening movement from several sources including motor efference copies when a voluntary movement is made and vestibular/somatosensory signals when the body is in motion. Many of these updating signals arise from brainstem regions that monitor our ongoing movements and subsequently transmit this information to the cortex via pathways that likely include the thalamus. Several issues of debate include (1) the relative contribution of extra-retinal sensory and efference copy signals to spatial updating, (2) the source of an updating signal for real life, three-dimensional motion that cannot arise from brain areas encoding only two-dimensional commands, and (3) the reference frames used by the brain to integrate updating signals from various sources. This review highlights the relevant spatial updating studies and provides a summary of the field today. We find that spatial constancy is maintained by a highly evolved neural mechanism that keeps track of our movements, transmits this information to relevant brain regions, and then uses this information to change the way in which single neurons respond. In this way, we are able to keep track of relevant objects in the outside world and interact with them in meaningful ways.

Section snippets

Definition

The term updating refers to the re-examination of a set of conditions after an event or a number of events have taken place. So, if a system has a known set of conditions at time 1, and one wishes to know the set of conditions of this same system at some later time (1+n), then one requires two pieces of information in order to update correctly: (1) the initial set of conditions of the system at time 1, and (2) information about any intervening event(s) that took place in the interim.

A familiar,

Spatial updating for saccadic eye movements

The first studies on spatial updating were conducted by Hallett and Lightstone 1976a, Hallett and Lightstone 1976b in which they designed the now classic double-step saccade task (Fig. 2). Here, subjects fixate a central target while two peripheral targets are briefly flashed in sequence (T1 followed by T2). The subject's task is to first make a saccade to the first target (T1) and then make a second saccade to the second target (T2). As in the target localization example above, the saccade to

The neural basis of spatial updating for saccadic eye movements

The neural substrate for spatial updating was first identified by a single-unit recording study examining this phenomenon in the lateral intraparietal area (LIP) of the posterior parietal cortex (Duhamel et al., 1992a; reviewed in Colby et al., 1995). Here, the receptive field of each neuron was first identified by illuminating visual stimuli at various spatial locations. The cell would respond whenever the stimulus was turned on in this particular spatial location (Fig. 3A). In a subsequent

Spatial updating during pursuit eye movements

Spatial updating has also been investigated for intervening pursuit eye movements. In these paradigms, subjects are asked to pursue a moving target until a second target is briefly flashed in the periphery. The subject's task is to make a saccade to the remembered, space-fixed location of the flashed target. But note that for a short period of time after the flash, the subject's eyes continue to pursue the moving target, even though it is no longer present. The subject can either make an

Are efference copies the source of the extra-retinal signal for spatial updating?

For saccades and pursuit, the source of the extra-retinal spatial updating signal is likely a motor efference copy [also known as corollary discharge (Sperry 1950, Von Holst and Mittlesteadt 1950)]. These signals are copies of voluntary, outgoing motor commands that are generated whenever we make a movement. For example, in order to move one's eyes, one must generate a neural command that is transmitted down to motoneurons in the brainstem that control the eye muscles. One can simply take a

Updating for self-motion

While versatile efference copies may work as an updating signal for eye movements made with the head and body fixed in space (e.g. saccades and pursuit), our everyday movements typically also involve movements of the head and body. For such movements, the vestibular system, which measures the body's inertial motion, is likely also involved. Specifically, three semicircular canals measure how the body rotates in three-dimensional space (i.e. yaw, pitch and roll), and two otolith organs (the

Pathways

Although all possible sensory and motor sources of updating signals have not yet been examined, it is becoming obvious that many originate in the brainstem, like vestibular signals and efference copies from the SC and oculomotor burst neurons. But evidence for neurons that perform spatial updating lies mainly in cortical areas including LIP (Duhamel et al 1992a, Kusunoki and Goldberg 2003, Heiser and Colby 2006), FEF (Goldberg and Bruce 1990, Umeno and Goldberg 1997, Umeno and Goldberg 2001)

Theoretical models of spatial updating

The mathematical computations that underlie spatial updating have also been described. As can be observed in Fig. 2, the correct updated ME to T2 in a double-step saccade task could theoretically be computed by subtracting the ME to T1 from the RE caused by T2 (i.e. green ME2=RE2−ME1). This formula is called vector subtraction and modeling it requires information pertaining to (1) the eye-centered representation of the visual target and (2) the metrics of the intervening movement.

Reference frames

Early behavioral evidence suggested that spatial constancy is partially maintained by a central representation of visual space in an allocentric (i.e. world-centered) reference frame (Dassonville et al 1995, Karn et al 1997; reviewed in Burgess, 2006). However, as summarized above, subsequent single unit recording experiments (e.g. Fig. 3), pointed toward a retinotopic (i.e. eye-centered) reference frame for spatial updating (Batista et al 1999, Kusunoki and Goldberg 2003, Heiser and Colby 2006

Conclusions

Spatial updating, the neural process underlying spatial constancy, is truly a complex and intriguing phenomenon. It combines RE signals with a variety of sensory and motor commands that describe the amplitude and direction of movements that occur between the time we see an object and when we decide to generate an action toward it. These intervening movements not only include eye movements such as saccades and pursuit, but also rotations of the head as well as whole-body rotations and

Acknowledgments

The authors would like to thank the following people for providing modified versions of their figures to this paper: Rebecca A. Berman, Gunnar Blohm, W. Pieter Medendorp and Min Wei. Support was provided by National Institutes of Health grant DC04260 to D.E.A.

References (124)

C. Bellebaum et al.
Post-saccadic updating of visual space in the posterior parietal cortex in humans
Behav Brain Res
(2005)
J. Blouin et al.
Failure to update the egocentric representation of the visual space through labyrinthine signal
Brain Cogn
(1995)
N. Burgess
Spatial memory: how egocentric and allocentric combine
Trends Cogn Neurosci
(2006)
C.L. Colby
Action-oriented spatial reference frames in cortex
Neuron
(1998)
P. Dassonville et al.
The use of egocentric and exocentric location cues in saccadic programming
Vision Res
(1995)
P.E. Hallett et al.
Saccadic eye movements towards stimuli triggered by prior saccades
Vision Res
(1976)
P.E. Hallett et al.
Saccadic eye movements to flashed targets
Vision Res
(1976)
W. Lang et al.
Vestibular projections to the monkey thalamus: an autoradiographic study
Brain Res
(1979)
C.S. Li et al.
Inactivation of macaque lateral intraparietal area delays initiation of the second saccade predominantly from contralesional eye positions in a double-saccade task
Exp Brain Res
(2001)
N. Li et al.
Updating visual space during motion in depth
Neuron
(2005)

E.P. Merriam et al.

Spatial updating in human parietal cortex

Neuron

(2003)

M.A. Admiraal et al.

Gaze affects pointing toward remembered visual targets after a self-initiated step

J Neurophysiol

(2004)

R.A. Andersen et al.

Multimodal representation of space in the posterior parietal cortex and its use in planning movements

Annu Rev Neurosci

(1997)

H. Aubert

Eine scheinbare bedeutende Drehung von Objekten bei Neigung des Kopfes nach rechts oder links

Virchows Arch

(1861)

M. Avillac et al.

Reference frames for representing visual and tactile locations in parietal cortex

Nat Neurol

(2005)

J.T. Baker et al.

Spatial memory following shifts of gazeI. Saccades to memorized world-fixed and gaze-fixed targets

J Neurophysiol

(2003)

A.P. Batista et al.

Reach plans in eye-centered coordinates

Science

(1999)

C. Bellebaum et al.

Altered processing of corollary discharge in thalamic lesion patients

Eur J Neurosci

(2006)

R.A. Berman et al.

Dynamic circuitry for updating spatial representationsI. Behavioral evidence for interhemispheric transfer in the split-brain macaque

J Neurophysiol

(2005)

R.A. Berman et al.

Dynamic circuitry for updating spatial representationsIII. From neurons to behavior

J Neurophysiol

(2007)

A. Berthoz et al.

Spatial memory of body linear displacement: what is being stored?

Science

(1995)

G. Blohm et al.

Smooth anticipatory eye movements alter the memorized position of flashed targets

J Vis

(2003)

G. Blohm et al.

Processing of retinal and extraretinal signals for memory-guided saccades during smooth pursuit

J Neurophysiol

(2005)

G. Blohm et al.

A model that integrates eye velocity commands to keep track of smooth eye displacements

J Comput Neurosci

(2006)

J. Bloomberg et al.

Vestibular-contingent voluntary saccades based on cognitive estimates of remembered vestibular information

Adv Otorhinolaryngol

(1988)

J. Bloomberg et al.

Adaptive modification of vestibularly perceived rotation

Exp Brain Res

(1991)

J. Blouin et al.

Visual stability with goal-directed eye and arm movements toward a target displaced during saccadic suppression

Psychol Res

(1995)

J. Blouin et al.

Control of rapid arm movements when target position is altered during saccadic suppression

J Mot Behav

(1995)

J. Blouin et al.

Updating visual space during passive and voluntary head-in-space movements

Exp Brain Res

(1998)

D. Boussaoud et al.

Gaze effects in the cerebral cortex: reference frames for space coding and action

Exp Brain Res

(1999)

J.P. Bresciani et al.

On the nature of the vestibular control of arm-reaching movements during whole-body rotations

Exp Brain Res

(2005)

P.R. Brotchie et al.

Head position signals used by parietal neurons to encode locations of visual stimuli

Nature

(1995)

C.R. Cassanello et al.

Computing vector differences using a gain field-like mechanism in monkey frontal eye field

J Physiol

(2007)

C.R. Cassanello et al.

Visual remapping by vector subtraction: analysis of multiplicative gain field models

Neural Comput

(2007)

C.L. Colby et al.

Oculocentric spatial representation in parietal cortex

Cereb Cortex

(1995)

J.R. Duhamel et al.

The updating of the representation of visual space in parietal cortex by intended eye movements

Science

(1992)

J.R. Duhamel et al.

Saccadic dysmetria in a patient with a right frontoparietal lesionThe importance of corollary discharge for accurate spatial behaviour

Brain

(1992)

M.O. Ernst et al.

Humans integrate visual and haptic information in a statistically optimal fashion

Nature

(2002)

B. Gaymard et al.

Impairment of extraretinal eye position signals after central thalamic lesions in humans

Exp Brain Res

(1994)

R.S. Gellman et al.

Eye position signals in human saccadic processing

Exp Brain Res

(1992)

S. Glasauer et al.

Noncommutative updating of perceived self-orientation in three dimensions

J Neurophysiol

(2007)

M.E. Goldberg et al.

Primate frontal eye fieldsIII. Maintenance of a spatially accurate saccade signal

J Neurophysiol

(1990)

B.L. Guthrie et al.

Corollary discharge provides accurate eye position information to the oculomotor system

Science

(1983)

M. Hayhoe et al.

Integration of form across saccadic eye movements

Perception

(1991)

W. Heide et al.

Activation of frontoparietal cortices during memorized triple-step sequences of saccadic eye movements: an fMRI study

Eur J Neurosci

(2001)

W. Heide et al.

Cortical control of double-step saccades: implications for spatial orientation

Ann Neurol

(1995)

L.M. Heiser et al.

Dynamic circuitry for updating spatial representationsII. Physiological evidence for interhemispheric transfer in area LIP of the split-brain macaque

J Neurophysiol

(2005)

L.M. Heiser et al.

Spatial updating in area LIP is independent of saccade direction

J Neurophysiol

(2006)

T.M. Herter et al.

Human head-free gaze saccades to targets flashed before gaze-pursuit are spatially accurate

J Neurophysiol

(1998)

I.P. Howard

Human visual orientation

(1982)

Cited by (67)

Dynamic representations of visual space for perception and action
2018, Cortex
Citation Excerpt :
The egocentric representation of a target in the dorsal visual stream that is used to guide actions like saccades, however, needs updating when the observer moves. This process, called visuomotor updating, is known to also reside in the dorsal stream (for reviews see Klier & Angelaki, 2008; Medendorp, 2011). According to the What & How pathway model, the dorsal stream is immune to contextual illusions, so the visuomotor updating processes should be immune as well.
The division of labor in visual processing between two anatomically relatively separate cortical pathways, a ventral and a dorsal stream, has been hotly debated in the last decades. One influential model is the What & How pathway model, suggesting that the separation is along ventral perception versus dorsal action, although the degree of functional separation between the two streams is controversial. An implication of this model is that perception and memory-guided movements are highly sensitive to visual contextual illusions, whereas visually-guided movements are largely immune to them. Here, we summarize our recent behavioral and imaging data obtained in single and double saccade paradigms that test this proposal, with a focus on the role of time in visuomotor processing and updating. We describe results showing that presentation time of the illusion affects both saccade amplitude and perceptual judgments in a similar way. We also discuss behavioral findings showing that visuomotor updating is affected by illusory context. Complementary neuroimaging data suggest a neural correlate of these findings in dorsal stream areas. Taken together, these results are suggestive of a dynamic, common visual representation that drives both perception and action, or – at least – that there is no absolute functional specialization of the two visual processing streams.
Alpha-band transcranial alternating current stimulation modulates precision, but not gain during whole-body spatial updating
2017, Neuropsychologia
Spatial updating is essential to maintain an accurate representation of our visual environment when we move. A neural mechanism that contributes to this ability is called remapping: the transfer of visual information from neural populations that code a location before the motion to those that encode it after the motion. While there is ample evidence for neural remapping in conjunction with eye movements, only recent findings suggest a role of this mechanism for whole-body motion updating, based on the observation that alpha band (10 Hz) activity is selectively reorganized during remapping. This study tested whether alpha oscillations directly contribute to whole-body motion updating using transcranial alternating current stimulation (tACS). In a double blind sham controlled design, healthy volunteers received 10 Hz tACS at an intensity of 1 mA over either the left or right posterior hemisphere during a whole-body motion updating task. Updating performance was assessed psychometrically and indices of gain and precision were obtained. No tACS-related effects on updating gain were found, irrespective of whether the remapping was across or within the hemispheres. In contrast, updating precision was enhanced when a target representation had to be internally remapped to the stimulated hemisphere, but not in other remapping conditions. Our observations suggest that alpha band oscillations do not directly affect the transfer of target representations during remapping, but increase the fidelity of the updated representation by attenuating interference of afferent information.
The Müller-Lyer illusion affects visuomotor updating in the dorsal visual stream
2015, Neuropsychologia
Citation Excerpt :
To guide our actions in the environment, our brain needs to specify relevant locations relative to our body, that is, in an egocentric reference frame. The continuous movement of our eyes and body complicates egocentric coding, but the brain has developed mechanisms to update target locations when we move, a process called visuomotor updating (for reviews see Klier and Angelaki, 2008; Medendorp, 2011). The coding and updating of (remembered) target locations is implemented in the dorsal visual stream, which is a series of anatomically connected areas from the primary visual cortex to the posterior parietal cortex (Colby and Goldberg, 1999; Goodale and Milner, 1992; Medendorp et al., 2003; Mishkin et al., 1983).
To guide our actions, the brain has developed mechanisms to code target locations in egocentric coordinates (i.e., with respect to the observer), and to update these when the observer moves. The latter mechanism, called visuomotor updating, is implemented in the dorsal visual stream of the brain. In contrast, the ventral visual stream is assumed to transform target locations into an allocentric reference frame that is highly sensitive to visual contextual illusions. Here, we tested the effect of the Müller-Lyer illusion on visuomotor updating in a double-step saccade task. Using the same paradigm in a 3T fMRI scanner, we investigated the effect of the illusion on the neural correlate of the updating process. Participants briefly viewed the Brentano version of the Müller-Lyer illusion with a target at its middle vertex, while fixating at one of the two endpoints of the illusion. Shortly after the disappearance of the stimulus, the eyes' fixation point moved to a position outside the illusion. After a delay, participants made a saccade to the remembered position of the target. The landing position of this saccade was systematically displaced in a manner congruent with the perceptual illusion, showing that visuomotor updating is affected by the illusion. fMRI results showed that the BOLD response in the occipito-parietal cortex (area V7) and the intraparietal sulcus related to planning of the saccade to the updated target was also modulated by the configuration of the illusion. This suggests that the dorsal visual stream represents perceived rather than physical locations of remembered saccade targets.
Improving spatial updating accuracy in absence of external feedback
2015, Neuroscience
Updating the position of an earth-fixed target during whole-body rotation seems to rely on cognitive processes such as the utilization of external feedback. According to perceptual learning models, improvement in performance can also occur without external feedback. The aim of this study was to assess spatial updating improvement in the absence and in the presence of external feedback. While being rotated counterclockwise (CCW), participants had to predict when their body midline had crossed the position of a memorized target. Four experimental conditions were tested: (1) Pre-test: the target was presented 30° in the CCW direction from participant’s midline. (2) Practice: the target was located 45° in the CCW direction from participant’s midline. One group received external feedback about their spatial accuracy (Mackrous and Simoneau, 2014) while the other group did not. (3) Transfer T³⁰_CCW: the target was presented 30° in the CCW direction to evaluate whether improvement in performance, during practice, generalized to other target eccentricity. (4) Transfer T³⁰_CW: the target was presented 30° in the clockwise (CW) direction and participants were rotated CW. This transfer condition evaluated whether improvement in performance generalized to the untrained rotation direction. With practice, performance improved in the absence of external feedback (p = 0.004). Nonetheless, larger improvement occurred when external feedback was provided (ps = 0.002). During T³⁰_CCW, performance remained better for the feedback than the no-feedback group (p = 0.005). However, no group difference was observed for the untrained direction (p = 0.22). We demonstrated that spatial updating improved without external feedback but less than when external feedback was given. These observations are explained by a mixture of calibration processes and supervised vestibular learning.
Vision: A moving hill for spatial updating on the fly
2015, Current Biology
Occipital and parietal cortex participate in a cortical network for transsaccadic discrimination of object shape and orientation
2023, Scientific Reports

View all citing articles on Scopus

View full text

Neuroscience Forefront ReviewSpatial updating and the maintenance of visual constancy

Abstract

Section snippets

Definition

Spatial updating for saccadic eye movements

The neural basis of spatial updating for saccadic eye movements

Spatial updating during pursuit eye movements

Are efference copies the source of the extra-retinal signal for spatial updating?

Updating for self-motion

Pathways

Theoretical models of spatial updating

Reference frames

Conclusions

Acknowledgments

Behav Brain Res

Brain Cogn

Trends Cogn Neurosci

Neuron

Vision Res

Vision Res

Vision Res

Brain Res

Exp Brain Res

Neuron

Neuron

Gaze affects pointing toward remembered visual targets after a self-initiated step

J Neurophysiol

Multimodal representation of space in the posterior parietal cortex and its use in planning movements

Annu Rev Neurosci

Eine scheinbare bedeutende Drehung von Objekten bei Neigung des Kopfes nach rechts oder links

Virchows Arch

Reference frames for representing visual and tactile locations in parietal cortex

Nat Neurol

Spatial memory following shifts of gazeI. Saccades to memorized world-fixed and gaze-fixed targets

J Neurophysiol

Reach plans in eye-centered coordinates

Science

Altered processing of corollary discharge in thalamic lesion patients

Eur J Neurosci

Dynamic circuitry for updating spatial representationsI. Behavioral evidence for interhemispheric transfer in the split-brain macaque

J Neurophysiol

Dynamic circuitry for updating spatial representationsIII. From neurons to behavior

J Neurophysiol

Spatial memory of body linear displacement: what is being stored?

Science

Smooth anticipatory eye movements alter the memorized position of flashed targets

J Vis

Processing of retinal and extraretinal signals for memory-guided saccades during smooth pursuit

J Neurophysiol

A model that integrates eye velocity commands to keep track of smooth eye displacements

J Comput Neurosci

Vestibular-contingent voluntary saccades based on cognitive estimates of remembered vestibular information

Adv Otorhinolaryngol

Adaptive modification of vestibularly perceived rotation

Exp Brain Res

Visual stability with goal-directed eye and arm movements toward a target displaced during saccadic suppression

Psychol Res

Control of rapid arm movements when target position is altered during saccadic suppression

J Mot Behav

Updating visual space during passive and voluntary head-in-space movements

Exp Brain Res

Gaze effects in the cerebral cortex: reference frames for space coding and action

Exp Brain Res

On the nature of the vestibular control of arm-reaching movements during whole-body rotations

Exp Brain Res

Head position signals used by parietal neurons to encode locations of visual stimuli

Nature

Computing vector differences using a gain field-like mechanism in monkey frontal eye field

J Physiol

Visual remapping by vector subtraction: analysis of multiplicative gain field models

Neural Comput

Oculocentric spatial representation in parietal cortex

Cereb Cortex

The updating of the representation of visual space in parietal cortex by intended eye movements

Science

Saccadic dysmetria in a patient with a right frontoparietal lesionThe importance of corollary discharge for accurate spatial behaviour

Brain

Humans integrate visual and haptic information in a statistically optimal fashion

Nature

Impairment of extraretinal eye position signals after central thalamic lesions in humans

Neuroscience Forefront Review
Spatial updating and the maintenance of visual constancy