Key Points
-
Visual processing of a feature or object is powerfully affected by its context, that is, its spatial and temporal neighbourhood. It is important to achieve a comprehensive understanding of how and why this is the case, from neural, perceptual and functional perspectives. We focus on tilt as a paradigmatic example, given the large body of historical and recent literature. We also emphasize similarities with other visual attributes and sensory domains.
-
Spatial and temporal context have often been treated separately in the literature. Nevertheless, despite quite different demands on their neural substrate (for example, memory for temporal context, but horizontal intraareal interactions for spatial context), they are closely tied, both functionally and in terms of their impact on vision.
-
Spatial and temporal context exhibit strikingly similar effects in many experimental circumstances. Psychophysical effects include perceptual biases, which are apparent in illusions and after-effects; physiological effects include suppression of the mean firing rate and changes in tuning curves.
-
Mechanistic population models of orientation tuning suggest a link between changes at the neural population level and perceptual changes. In these models, perceptual biases arise due to the 'coding catastrophe' (or decoding ambiguity): downstream mechanisms are unaware of the changes in tuning caused by contextual stimuli, and therefore err when such changes take place.
-
From a functional viewpoint, two main questions exist. First, why are these biases so similar for spatial and temporal contexts; and second, why do contextual stimuli induce perceptual biases at all? In answer to the first question, the obvious source of similarities is the shared statistical regularity in natural visual scenes, with a small patch of image in a scene being typically similar (in properties such as orientation) to patches observed simultaneously in nearby spatial regions and to patches that were recently observed.
-
The reason behind the functional benefit of contextual biases is more contentious: we discuss exciting directions in recent literature, focusing on computational frameworks of efficient coding and Bayesian inference. Natural scene statistics over space and time can inform both of the above frameworks.
-
These directions have the potential to unify our understanding of spatial and temporal contextual processing. Such unified approaches also highlight the many gaps in our current understanding and suggest future perspectives for physiological, psychophysical and computational investigations.
Abstract
No sensory stimulus is an island unto itself; rather, it can only properly be interpreted in light of the stimuli that surround it in space and time. This can result in entertaining illusions and puzzling results in psychological and neurophysiological experiments. We concentrate on perhaps the best studied test case, namely orientation or tilt, which gives rise to the notorious tilt illusion and the adaptation tilt after-effect. We review the empirical literature and discuss the computational and statistical ideas that are battling to explain these conundrums, and thereby gain favour as more general accounts of cortical processing.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$189.00 per year
only $15.75 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Clifford, C. W. & Rhodes, G. (eds) Fitting the Mind to the World Adaptation and After-Effects in High-Level Vision (Oxford University Press, 2005). This book comprehensively covers a wealth of aspects of and perspectives on adaptation, ranging from orientation to high level processing (such as of faces), and from physiology to perception and functional perspectives.
Wohlgemuth, A. On the after-effect of seen movement. Bri. J. Psychol. (Suppl.) 1, 1–117 (1911).
Adelson, E. H. in The New Cognitive Neurosciences (ed. Gazzaniga, M.) 339–351 (MIT Press, Cambridge, Massachusetts, 2000).
Eagleman, D. M., Jacobson, J. E. & Sejnowski, T. J. Perceived luminance depends on temporal context. Nature 428, 854–856 (2004).
Gibson, J. J. Adaptation, after-effect, and contrast in the perception of tilted lines. J. Exp. Psychol. 20, 553–569 (1937). Pioneering paper that introduced the tilt after-effect, a striking perceptual effect that has been studied intensively over the years.
Webster, M. A., Georgeson, M. A. & Webster, S. M. Neural adjustments to image blur. Nature Neurosci. 5, 839–840 (2002).
Webster, M. A., Kaping, D., Mizokami, Y. & Duhamel, P. Adaptation to natural facial categories. Nature 428, 557–561 (2004).
Leopold, D. A., Rhodes, G., Müller, K. M. & Jeffery, L. The dynamics of visual adaptation to faces. Proc. Biol. Sci. 272, 897–904 (2005).
Oxenham, A. J. Forward masking: adaptation or integration? J. Acoust. Soc. Am. 109, 732–741 (2001).
Wallace, M. T. et al. Unifying multisensory signals across time and space. Exp. Brain Res. 158, 252–258 (2004).
Series, P., Lorenceau, J. & Frégnac, Y. The “silent” surround of V1 receptive fields: theory and experiments. J. Physiol. Paris 97, 453–474 (2003).
Albright, T. D. & Stoner, G. R. Contextual influences on visual processing. Annu. Rev. Neurosci. 25, 339–379 (2002).
Dragoi, V. & Sur, M. in The Visual Neuroscience (eds Chalupa, L. M. & Werner, J. S.) 1654–1664 (MIT Press, 2003).
Krekelberg, B., Boynton, G. M. & van Wezel, R. J. Adaptation: from single cells to BOLD signals. Trends Neurosci. 29, 250–256 (2006).
Katz, Y., Heiss, J. E. & Lampl, I. Cross-whisker adaptation of neurons in the rat barrel cortex. J. Neurosci. 26, 13363–13372 (2006).
Wachtler, T., Sejnowski, T. J. & Albright, T. D. Representation of color stimuli in awake macaque primary visual cortex. Neuron 37, 681–691 (2003).
Li, Z. Border ownership from intracortical interactions in visual area V2. Neuron 47, 143–153 (2005).
Wenderoth, P. & Johnstone, S. The different mechanisms of the direct and indirect tilt illusions. Vision Res. 28, 301–312 (1988).
Clifford, C. W., Wenderoth, P. & Spehar, B. A functional angle on some after-effects in cortical vision. Proc. R. Soc. Lond. B Biol. Sci. 267, 1705–1710 (2000). Puts forth a functional model based on efficient coding principles for understanding both spatial and temporal perceptual context effects within a common framework.
Webster, M. A., Malkoc, G., Bilson, A. C. & Webster, S. M. Color contrast and contextual influences on color appearance. J. Vis. 2, 505–519 (2002).
Guo, K. et al. Effects on orientation perception of manipulating the spatio-temporal prior probability of stimuli. Vision Res. 44, 2349–2358 (2004).
Durant, S. & Clifford, C. W. Dynamics of the influence of segmentation cues on orientation perception. Vision Res. 46, 2934–2940 (2006).
Polat, U. & Sagi, D. Temporal asymmetry of collinear lateral interactions. Vision Res. 46, 953–960 (2006).
Felsen, G., Touryan, J. & Dan, Y. Contextual modulation of orientation tuning contributes to efficient processing of natural stimuli. Network 16, 139–149 (2005). Emphasizes that similarities in image statistics over space and time can explain experimental similarities in cortical processing (specifically, in terms of repulsive shifts in tuning curves due to spatial and temporal context).
Shepherd, A. J. Increased visual after-effects following pattern adaptation in migraine: a lack of intracortical excitation? Brain 124, 2310–2318 (2001).
Hubel, D. & Wiesel, T. Receptive fields, binocular interaction, and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962).
Seung, H. S. & Sompolinsky, H. Simple models for reading neuronal population codes. Proc. Natl Acad. Sci. USA 90, 10749–10753 (1993).
Snippe, H. P. Parameter extraction from population codes: a critical assessment. Neural Comput. 8, 511–529 (1996).
Pouget, A., Dayan, P. & Zemel, R. Information processing with population codes. Nature Rev. Neurosci. 1, 125–132 (2000).
Coppola, D. M., Purves, H. R., McCoy, A. N. & Purves, D. The distribution of oriented contours in the real world. Proc. Natl Acad. Sci. USA 95, 4002–4006 (1998).
Cavanaugh, J. R., Bair, W. & Movshon, J. A. Selectivity and spatial distribution of signals from the receptive field surround in macaque V1 neurons. J. Neurophysiol. 88, 2547–2556 (2002).
Müller, J. R., Metha, A. B., Krauskopf, J. & Lennie, P. Local signals from beyond the receptive fields of striate cortical neurons. J. Neurophysiol. 90, 822–831 (2003).
van der Smagt, M. J., Wehrhahn, C. & Albright, T. D. Contextual masking of oriented lines: interactions between surface segmentation cues. J. Neurophysiol. 94, 576–589 (2005).
Knierim, J. J. & van Essen, D. C. Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. J. Neurophysiol. 67, 961–980 (1992).
Nothdurft, H. C., Gallant, J. L. & Essen, D. C. V. Response modulation by texture surround in primate area V1: correlates of “popout” under anesthesia. Vis. Neurosci. 16, 15–34 (1999).
Li, W., Thier, P. & Wehrhahn, C. Contextual influence on orientation discrimination of humans and responses of neurons in V1 of alert monkeys. J. Neurophysiol. 83, 941–954 (2000).
Sengpiel, F., Sen, A. & Blakemore, C. Characteristics of surround inhibition in cat area 17. Exp. Brain Res. 116, 216–228 (1997).
Müller, J. R., Metha, A. B., Krauskopf, J. & Lennie, P. Rapid adaptation in visual cortex to the structure of images. Science 285, 1405–1408 (1999).
Dragoi, V., Sharma, J. & Sur, M. Adaptation-induced plasticity of orientation tuning in adult visual cortex. Neuron 28, 287–298 (2000).
Crowder, N. A. et al. Relationship between contrast adaptation and orientation tuning in V1 and V2 of cat visual cortex. J. Neurophysiol. 95, 271–283 (2006).
Levitt, J. B. & Lund, J. S. Contrast dependence of contextual effects in primate visual cortex. Nature 387, 73–76 (1997).
Kohn, A. & Movshon, J. A. Adaptation changes the direction tuning of macaque MT neurons. Nature Neurosci. 7, 764–772 (2004). Demonstrates the importance of studying systems hierarchically, reporting that population tuning changes at one level (in the primary visual cortex) can differ markedly from tuning changes at the next level (in the motion processing middle temporal area).
Gilbert, C. D. & Wiesel, T. N. The influence of contextual stimuli on the orientation selectivity of cells in primary visual cortex of the cat. Vision Res. 30, 1689–1701 (1990).
Li, C. Y., Lei, J. J. & Yao, H. S. Shift in speed selectivity of visual cortical neurons: a neural basis of perceived motion contrast. Proc. Natl Acad. Sci. USA 96, 4052–4056 (1999).
Greenlee, M. W. & Magnussen, S. Saturation of the tilt aftereffect. Vision Res. 27, 1041–1043 (1987).
Wenderoth, R. & van der Zwan, R. The effects of exposure duration and surrounding frames on direct and indirect tilt aftereffects and illusions. Percept. Psychophys. 46, 338–344 (1989).
Schrater, P. R. & Simoncelli, E. P. Local velocity representation: evidence from motion adaptation. Vision Res. 38, 3899–3912 (1998).
Mather, G., Verstraten, F. & Anstis, S. (eds) The Motion Aftereffect (MIT press, Massachusetts, USA, 1998).
Morgan, M., Chubb, C. & Solomon, J. A. Predicting the motion after-effect from sensitivity loss. Vision Res. 46, 2412–2420 (2006).
Solomon, J. A. & Morgan, M. J. Stochastic re-calibration: contextual effects on perceived tilt. Proc. Biol. Sci. 273, 2681–2686 (2006).
Mareschal, I., Sceniak, M. P. & Shapley, R. M. Contextual influences on orientation discrimination: binding local and global cues. Vision Res. 41, 1915–1930 (2001).
Dragoi, V., Sharma, J., Miller, E. K. & Sur, M. Dynamics of neuronal sensitivity in primate V1 underlying local feature discrimination. Nature Neurosci. 883–891 (2002).
Westheimer, G. & Gee, A. Opposing views on orthogonal adaptation: a response to Clifford, Arnold, Smith, and Pianta (2003). Vision Res. 43, 721–722 (2003).
Regan, D. & Beverley, K. I. Postadaptation orientation discrimination. J. Opt. Soc. Am. A 2, 147–155 (1985).
Clifford, C. W., Arnold, D. H., Smith, S. T. & Pianta, M. Opposing views on orthogonal adaptation: a reply to Westheimer and Gee (2002). Vision Res. 43, 717–719 (2003).
Clifford, C. W., Wyatt, A. M., Arnold, D. H., Smith, S. T. & Wenderoth, P. Orthogonal adaptation improves orientation discrimination. Vision Res. 41, 151–159 (2001).
Barlow, H. B., Macleod, D. I. A. & van Meeteren, A. Adaptation to gratings: no compensatory advantages found. Vision Res. 16, 1043–1045 (1976).
Jin, D. Z., Dragoi, V., Sur, M. & Seung, H. S. Tilt aftereffect and adaptation-induced changes in orientation tuning in visual cortex. J. Neurophysiol. 94, 4038–4050 (2005). Demonstrates how changes in tuning curves observed in cortical adaptation data can give rise to perceptual repulsion and attraction within a population decoding model.
Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nature Rev. Neurosci. 7, 358–366 (2006).
Fairhall, A. L., Lewen, G. D., Bialek, W. & de Ruyter Van Steveninck, R. R. Efficiency and ambiguity in an adaptive neural code. Nature 412, 787–792 (2001). Studies adaptation to variance statistics in a velocity-sensing neuron in the fly, and suggests that it might evade the coding catastrophe by reporting aspects of its state of adaptation through its long-run average firing rate.
Teich, A. F. & Qian, N. Learning and adaptation in a recurrent model of V1 orientation selectivity. J. Neurophysiol. 89, 2086–2100 (2003). Proposes a mechanistic population decoding model and draws out the perceptual implications for two different timescales of temporal context, namely adaptation and learning.
Li, Z. A saliency map in primary visual cortex. Trends Cogn. Sci. 6, 9–16 (2002).
Barlow, H. B. in Vision: Coding and Efficiency (ed. Blakemore, C.) 363–375 (Cambridge University Press, New York, USA,1990). Barlow is a pioneer in the application of computational principles to explaining neural and psychophysical phenomena; here he discusses his original notion that efficient coding ideas can explain adaptation at the synaptic and perceptual levels.
Webster, M. A., Werner, J. S. & Field, D. J. in Fitting the Mind to the World: Adaptation and Aftereffects in High-Level Vision, Advances in Visual Cognition Series (eds Clifford, C. W. & Rhodes, G. L.) 241–277 (Oxford University Press, USA, 2005). Points out that the effects of adaptation on perceptual discriminability are quite modest compared with the more striking perceptual repulsion, and suggests alternative functional frameworks with a focus on societal norms.
Field, D. J. What is the goal of sensory coding? Neural Comput. 6, 559–601 (1994).
Simoncelli, E. P. & Olshausen, B. A. Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001).
Li, Z. & Atick, J. J. Towards a theory of the striate cortex. Neural Comput. 6, 127–146 (1994).
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse factorial code. Nature 381, 607–609 (1996).
Bell, A. J. & Sejnowski, T. J. The 'independent components' of natural scenes are edge filters. Vision Res. 37, 3327–3338 (1997).
Li, Z. Theoretical understanding of the early visual processes by data compression and data selection. Network 17, 301–334 (2006). A review of theoretical frameworks for understanding early visual processing, including the notion that intracortical connections in the primary visual cortex are responsible for computing a map of the salience of regions of visual space, and that this can explain properties that are opaque to efficient coding principles.
Field, D. J., Hayes, A. & Hess, R. F. Contour integration by the human visual system: evidence for a local “association field”. Vision Res. 33, 173–193 (1993).
Elder, J. H. & Goldberg, R. M. Ecological statistics of Gestalt laws for the perceptual organization of contours. J. Vis. 2, 324–353 (2002).
Olshausen, B. A. & Field, D. J. Vision and the coding of natural images. Am. Sci. 88, 238–244 (2000).
Baddeley, R. The correlational structure of natural images and the calibration of spatial representations. Cogn. Sci. 21, 351–372 (1997).
Dong, D. W. & Atick, J. J. Statistics of natural time-varying images. Network 6 345–358 (1995).
Grzywacz, N. M. & de Juan, J. Sensory adaptation as Kalman filtering: theory and illustration with contrast adaptation. Network 14, 465–482 (2003).
Geman, S. & Geman, D. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE. Trans. Pat. Anal. Mach. Intell. 6, 721–741 (1984).
Kersten, D. Predictability and redundancy of natural images. J. Opt. Soc. Am. A 4, 2395–2400 (1987).
Schwartz, O. & Simoncelli, E. P. Natural signal statistics and sensory gain control. Nature Neurosci. 4, 819–825 (2001).
Hoyer, P. & Hyvärinen, A. A multi-layer sparse coding network learns contour coding from natural images. Vision Res. 42, 1593–1605 (2002).
Karklin, Y. & Lewicki, M. S. A hierarchical Bayesian model for learning nonlinear statistical regularities in nonstationary natural signals. Neural Comput. 17, 397–423 (2005).
Zetzsche, C. & Nuding, U. Nonlinear and higher-order approaches to the encoding of natural scenes. Network 16, 191–221 (2005).
Schwartz, O., Sejnowski, T. J. & Dayan, P. Soft mixer assignment in a hierarchical generative model of natural scene statistics. Neural Comput. 18, 2680–2718 (2006).
Hyvärinen, A., Hurri, J. & Väyrynen, J. Bubbles: a unifying framework for low-level statistical properties of natural image sequences. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1237–1252 (2003).
Sigman, M., Cecchi, G. A., Gilbert, C. D. & Magnasco, M. O. On a common circle: natural scenes and Gestalt rules. Proc. Natl Acad. Sci. USA 98, 1935–1940 (2001).
Geisler, W. S., Perry, J. S., Super, B. J. & Gallogly, D. P. Edge co-occurrence in natural images predicts contour grouping performance. Vision Res. 41, 711–724 (2001). Examines both absolute and Bayesian orientation spatial context statistics in natural images, and derives a model of contour grouping that is compared to perception.
Howe, C. Q. & Purves, D. Natural-scene geometry predicts the perception of angles and line orientation. Proc. Natl Acad. Sci. USA 102, 1228–1233 (2005). Measures image statistics of overlapping orientations in space, and suggests how these can give rise to perceptual tilt illusions.
Dragoi, V. & Sur, M. Image structure at the center of gaze during free viewing. J. Cogn. Neurosci. 18, 737–748 (2006).
Melcher, D. Spatiotopic transfer of visual-form adaptation across saccadic eye movements. Curr. Biol. 15, 1745–1748 (2005).
Burr, D. & Morrone, M. C. Eye movements: building a stable world from glance to glance. Curr. Biol. 15, R839–R840 (2005).
Barlow, H. B. in Sensory Communication (ed. Rosenblith, W. A.) 217–234 (MIT press, Massachusetts, 1961).
Attneave, F. Some informational aspects of visual perception. Psychol. Rev. 61, 183–193 (1954).
Shannon, C. A mathematical theory of communication. Bell Sys. Tech. J. 27, 379–423 (1948).
Barlow, H. B. Redundancy reduction revisited. Network 12, 241–253 (2001).
Simoncelli, E. P. Vision and the statistics of the visual environment. Curr. Opin. Neurobiol. 13, 144–149 (2003). Reviews the efficient coding hypothesis in recent literature, covering aspects of experimental testing of these principles and computational models based on efficient coding of natural images.
Barlow, H. B. & Foldiak, P. in The Computing Neuron (eds Miall, C., Durbin, R. M. & Mitchison, G. J.) 54–72 (Addison-Wesley, England, 1989).
Atick, J. J. & Redlich, A. N. Towards a theory of early visual processing. Neural Comput. 2, 308–320 (1990).
Atick, J. J., Li, Z. & Redlich, A. N. What does post-adaptation color appearance reveal about cortical color representation? Vision Res. 33, 123–129 (1993).
Dan, Y., Atick, J. J. & Reid, R. C. Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. J. Neurosci. 16, 3351–3362 (1996).
Smith, E. & Lewicki, M. Efficient auditory coding. Nature 439, 978–982 (2006).
Wainwright, M. J., Schwartz, O. & Simoncelli, E. P. in Probabilistic Models of the Brain: Perception and Neural Function (eds Rao, R., Olshausen, B. A. & Lewicki, M.) 203–222 (MIT Press, Massachusetts, USA, 2002).
Geisler, W. S. & Albrecht, D. G. Cortical neurons: isolation of contrast gain control. Vision Res. 8, 1409–1410 (1992).
Heeger, D. J. Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–198 (1992).
Vinje, W. E. & Gallant, J. L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000). Suggests that spatial context increases the efficiency of cortical neural processing.
Movellan, J. R., Wachtler, T., Albright, T. D. & Sejnowski, T. J. in Neural Information Processing Systems (eds Becker, S., Thrun, S. & Obermayer, K.) 205–212 (MIT Press, Massachusetts, USA, 2002).
Schwartz, O., Movellan, J. R., Wachtler, T., Albright, T. D. & Sejnowski, T. J. Spike count distributions, factonizability, and contextual effects in area V1. Neurocomputing 58–60 (2004).
Wainwright, M. J. Visual adaptation as optimal information transmission. Vision Res. 39, 3960–3974 (1999). Proposes that adaptation serves to optimize information transmission in an efficient coding context, using the tilt after-effect as a key example.
Foldiak, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165–170 (1990).
Bednar, J. A. & Miikkulainen, R. Tilt aftereffects in a self-organizing model of the primary visual cortex. Neural Comput. 12, 1721–1740 (2000).
Sirosh, J. & Miikkulainen, R. Topographic receptive fields and patterned lateral interaction in a self-organizing model of the primary visual cortex. Neural Comput. 9, 577–594 (1997).
Doya, K., Ishii, S., Pouget, A. & Rao, R. P. N. (eds) Bayesian Brain: Probabilistic Approaches to Neural Coding (MIT Press, Massachusetts, USA, 2007).
Yuille, A. & Bulthoff, H. H. in Bayesian Decision Theory and Psychophysics (eds Knill, D. and Richards, W.) 123–161 (Cambridge University Press, New York, USA, 1996).
Yuille, A. & Kersten, D. Vision as Bayesian inference: analysis by synthesis? Trends Cogn. Sci. 10, 301–308 (2006).
Balboa, R. M. & Grzywacz, N. M. The minimal local-asperity hypothesis of early retinal lateral inhibition. Neural Comput. 12, 1485–1517 (2000).
Grzywacz, N. M. & Balboa, R. M. A Bayesian framework for sensory adaptation. Neural Comput. 14, 543–559 (2002).
Weiss, Y., Simoncelli, E. P. & Adelson, E. H. Motion illusions as optimal percepts. Nature Neurosci. 5, 598–604 (2002).
Stocker, A. A. & Simoncelli, E. P. Noise characteristics and prior expectations in human visual speed perception. Nature Neurosci. 9, 578–585 (2006).
Kersten, D., Mamassian, P. & Yuille, A. Object perception as Bayesian inference. Annu. Rev. Psychol. 55, 271–304 (2004).
Stocker, A. A. & Simoncelli, E. P. in NIPS Advances in Neural Information Processing Systems (eds Weiss, Y., Schölkopf, B. & Platt, J.) 1291–1298 (MIT Press, Massachusetts, USA, 2006). Proposes a Bayesian model of visual adaption in terms of adjustments to the likelyhood, on the important basis that changes to the prior are likely to lead to perceptual attraction rather than repulsion.
Schwartz, O., Sejnowski, T. J. & Dayan, P. in Advances in Neural Information Processing Systems 18 (eds Weiss, Y., Schölkopf, B. & Platt, J.) 1201–1208 (MIT Press, Massachusetts, USA, 2006).
Körding, K. & Tenenbaum, J. B. in Advances in Neural Information Processing Systems (eds Schölkopf, B., Platt, J. & Hoffman, T.) 737–744 (The MIT Press, Massachussets, USA, 2006).
Over, R. Comparison of normalization theory and neural enhancement explanation of negative aftereffects. Psychol. Bull. 75, 225–243 (1971).
Andrews, D. P. Error-correcting perceptual mechanisms. Q. J. Exp. Psychol. 16, 104–115 (1964).
Mitchell, D. E. & Muir, D. W. Does the tilt after-effect occur in the oblique meridian? Vision Res. 16, 609–613 (1976).
Leopold, D. A., Bondar, I. V. & Giese, M. A. Norm-based face encoding by single neurons in the monkey inferotemporal cortex. Nature 442, 572–575 (2006).
Neisser, U. Cognitive Psychology (Prentice-Hall, New Jersey, USA, 1967).
Hinton, G. E. & Ghahramani, Z. Generative models for discovering sparse distributed representations. Philos. Trans. R. Soc. Lond. B Biol. Sci. 352, 1177–1190 (1997).
Grenander, U. & Srivastava, A. Probabibility models for clutter in natural images. IEEE. Trans. Patt. Anal. Mach. Intell. 23, 423–429 (2002).
Zhu, S. & Mumford, D. Prior learning and gibbs reaction-diffusion. IEEE. Trans. Patt. Anal. Mach. Intell. 19, 1236–1250 (1997).
Nundy, S. & Purves, D. A probabilistic explanation of brightness scaling. Proc. Natl Acad. Sci. USA 99, 14482–14487 (2002).
Gilchrist, A. L. et al. An anchoring theory of lightness perception. Psychol. Rev. 4, 795–834 (1999).
Andrews, D. P. & Mallows, C. Scale mixtures of normal distributions. J. R. Stat. Soc. 36, 99–102 (1974).
Wainwright, M. J. & Simoncelli, E. P. in Advances in Neural Information Processing Systems (eds Solla, S. A., Leen, T. K. & Müller, K. R.) 855–861 (MIT Press, Massachusetts, USA, 2000).
Portilla, J., Strela, V., Wainwright, M. & Simoncelli, E. P. Image denoising using a scale mixture of Gaussians in the wavelet domain. IEEE. Trans. Image Process. 12, 1338–1351 (2003).
Schwartz, O., Sejnowski, T. J. & Dayan, P. The tilt illusion, population decoding, and natural scene statistics. Computational and Systems Neuroscience (COSYNE) Abstract 280 (2007).
Rao, R. P. & Ballard, D. H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neurosci. 2, 79–87 (1999).
Srinivasan, M. V., Laughlin, S. B. & Dubs, A. Predictive coding: a fresh view of inhibition in the retina. Proc. R. Soc. Lond. B Biol. Sci. 216, 427–459 (1982).
Hosoya, T., Baccus, S. A. & Meister, M. Dynamic predictive coding by the retina. Nature 436, 71–77 (2005).
Mackay, D. M. in Automata Studies (eds Shannon, C. E. & McCarthy, J.) 235–251 (Princeton University Press, New Jersey, USA, 1956).
Kapadia, M. K., Ito, M., Gilbert, C. D. & Westheimer, G. Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron 15, 843–856 (1995).
Polat, U. & Sagi, D. Lateral interactions between spatial channels: suppression and facilitation revealed by lateral masking experiments. Vision Res. 33, 993–999 (1993).
Magnussen, S. & Johnsen, T. Temporal aspects of spatial adaptation. A study of the tilt aftereffect. Vision Res. 26, 661–672 (1986).
Rose, D. A square root law for adaptation to contrast? Vision Res. 32, 1781–1788 (1992).
Kanai, R. & Verstraten, F. A. Perceptual manifestations of fast neural plasticity: motion priming, rapid motion aftereffect and perceptual sensitization. Vision Res. 45, 3109–3116 (2005).
Muir, D. & Over, R. Tilt aftereffects in central and peripheral vision. J. Exp. Psychol. 85, 165–170 (1970).
Solomon, J. A., Felisberti, F. M. & Morgan, M. J. Crowding and the tilt illusion: toward a unified account. J. Vis. 4, 500–508 (2004).
Roberts, M. J. & Thielle, A. Attention and contrast modulate the influence of spatio-temporal context in orientation discrimination of human subjects. FENS abstract 3, A053.15 (2006).
Kapadia, M. K., Westheimer, G. & Gilbert, C. D. Spatial distribution of contextual interactions in primary visual cortex and in visual perception. J. Neurophysiol. 84, 2048–2062 (2000).
Li, Z. Computational design and nonlinear dynamics of a recurrent network model of the primary visual cortex. Neural Comput. 13, 1749–1780 (2001).
Pelli, D. G. & Farell, B. Why use noise? J. Opt. Soc. Am. A 16, 647–653 (1999).
Meese, T. S. & Georgeson, M. A. The tilt aftereffect in plaids and gratings: channel codes, local signs and “patchwise” transforms. Vision Res. 36, 1421–1437 (1996).
Georgeson, M. A. Human vision combines oriented filters to compute edges. Proc. Biol. Sci. 249, 235–245 (1992).
Smith, S., Wenderoth, P. & van der Zwan, R. Orientation processing mechanisms revealed by the plaid tilt illusion. Vision Res. 41, 483–494 (2001).
Foley, J. M. & Boynton, G. M. Forward pattern masking and adaptation: effects of duration, interstimulus interval, contrast, and spatial and temporal frequency. Vision Res. 33, 959–980 (1993).
Foley, J. M. & Yang, Y. D. Forward pattern masking: effects of spatial frequency and contrast. J. Opt. Soc. Am. A 8, 2026–2037 (1991).
Wehrhahn, C. & Dresp, B. Detection facilitation by collinear stimuli in humans: dependence on strength and sign of contrast. Vision Res. 38, 423–428 (1998).
Tanaka, Y. & Sagi, D. Long-lasting, long-range detection facilitation. Vision Res. 38, 2591–2599 (1998).
Zenger-Landolt, B. & Koch, C. Flanker effects in peripheral contrast discrimination–psychophysics and modeling. Vision Res. 41, 3663–3675 (2001).
Chen, C. C. & Tyler, C. W. Lateral modulation of contrast discrimination: flanker orientation effects. J. Vis. 2, 520–530 (2002).
Greenlee, M. W. & Heitger, F. The functional role of contrast adaptation. Vision Res. 28, 791–797 (1988).
Maattanen, L. M. & Koenderink, J. J. Contrast adaptation and contrast gain control. Exp. Brain Res. 87, 205–212 (1991).
Clifford, C. W. Perceptual adaptation: motion parallels orientation. Trends Cogn. Sci. 6, 136–143 (2002). Demonstrates perceptual analogies in the adaptation to orientation and motion, suggesting that common computational principles may underly contextual processing in both domains.
Cavanaugh, J. R., Bair, W. & Movshon, J. A. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J. Neurophysiol. 88, 2530–2546 (2002).
Albrecht, D. G., Farrar, S. B. & Hamilton, D. B. Spatial contrast adaptation characteristics of neurones recorded in the cat's visual cortex. J. Physiol. (Lond.) 347, 713–739 (1984).
Ohzawa, I., Sclar, G. & Freeman, R. D. Contrast gain control in the cat's visual system. J. Neurophysiol. 54, 651–667 (1985).
Kapadia, M. K., Westheimer, G. & Gilbert, C. D. Dynamics of spatial summation in primary visual cortex of alert monkeys. Proc. Natl Acad. Sci. USA 21, 12073–12078 (1999).
Sceniak, M. P., Ringach, D. L., Hawken, M. J. & Shapley, R. Contrast's effect on spatial summation by macaque V1 neurons. Nature Neurosci. 2, 733–739 (1999).
Webb, B. S., Dhruv, N. T., Solomon, S. G., Tailby, C. & Lennie, P. Early and late mechanisms of surround suppression in striate cortex of macaque. J. Neurosci. 25, 11666–11675 (2005).
Walker, G. A., Ohzawa, I. & Freeman, R. D. Disinhibition outside receptive fields in the visual cortex. J. Neurosci. 22, 5659–5668 (2002).
Carandini, M., Movshon, J. A. & Ferster, D. Pattern adaptation and cross-orientation interactions in the primary visual cortex. Neuropharmacology 37, 501–511 (1998).
Sharpee, T. O. et al. Adaptive filtering enhances information transmission in visual cortex. Nature 439, 936–942 (2006).
Fuster, J. M. Memory in the Cerebral Cortex: An Empirical Approach to Neural Networks in the Human and Nonhuman Primate (MIT Press, Massachusetts, USA, 1995).
Goldman-Rakic, P. S. Regional and cellular fractionation of working memory. Proc. Natl Acad. Sci. USA 93, 13473–13480 (1996).
Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).
Renart, A., Song, O. & Wang, X. J. Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks. Neuron 38, 473–485 (2003).
Abbott, L. F., Varela, J. A., Sen, K. & Nelson, S. B. Synaptic depression and cortical gain control. Science 275, 220–224 (1997).
Markram, H. & Tsodyks, M. V. Redistribution of synaptic efficacy between neocortical pyramidal neurones. Nature 382, 807–809 (1996).
Chance, F. S., Nelson, S. B. & Abbott, L. F. Synaptic depression and the temporal response characteristics of V1 cells. J. Neurosci. 18, 4785–4799 (1998).
Fusi, S., Drew, P. J. & Abbott, L. F. Cascade models of synaptically stored memories. Neuron 45, 599–611 (2005).
Drew, P. J. & Abbott, L. F. Models and properties of power-law adaptation in neural systems. J. Neurophysiol. 96, 826–833 (2006).
Wang, X. J., Liu, Y., Sanchez-Vives, M. V. & McCormick, D. A. Adaptation and temporal decorrelation by single neurons in the primary visual cortex. J. Neurophysiol. 89, 3279–3293 (2003).
Carandini, M. Visual cortex: fatigue and adaptation. Curr. Biol. 10, R605–R607 (2000).
Sanchez-Vives, M. V., Nowak, L. G. & McCormick, D. A. Cellular mechanisms of long-lasting adaptation in visual cortical neurons. J. Neurosci. 20, 4286–4299 (2000).
Nowak, L. G., Sanchez-Vives, M. V. & McCormick, D. A. Role of synaptic and intrinsic membrane properties in short-term receptive field dynamics in cat area 17. J. Neurosci. 25, 1866–1880 (2005).
Fitzpatrick, D. The functional organization of local circuits in visual cortex: insights from the study of tree shrew striate cortex. Cereb. Cortex 6, 329–341 (1996).
Angelucci, A. & Bullier, J. Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons? J. Physiol. Paris 97, 141–154 (2003).
Levitt, J. B. & Lund, J. S. The spatial extent over which neurons in macaque striate cortex pool visual signals. Vis. Neurosci. 19, 439–452 (2002).
Bair, W., Cavanaugh, J. R. & Movshon, J. A. Time course and time-distance relationships for surround suppression in macaque V1 neurons. J. Neurosci. 23, 7690–7601 (2003).
Stetter, M., Bartsch, H. & Obermayer, K. A mean-field model for orientation tuning, contrast saturation, and contextual effects in the primary visual cortex. Biol. Cybern. 82, 291–304 (2000).
Bressloff, P. C. & Cowan, J. D. An amplitude equation approach to contextual effects in visual cortex. Neural Comput. 14, 493–525 (2002).
Sullivan, T. J. & de Sa, V. R. A model of surround suppression through cortical feedback. Neural Netw. 19, 564–572 (2006).
Schwabe, L., Obermayer, K., Angelucci, A. & Bressloff, P. C. The role of feedback in shaping the extra-classical receptive field of cortical neurons: a recurrent network model. J. Neurosci. 26, 9117–9129 (2006).
Das, A. & Gilbert, C. D. Topography of contextual modulations mediated by short-range interactions in primary visual cortex. Nature 399, 655–661 (1999).
Okamoto, T., Watanabe, M., Aihara, K. & Kondo, S. An explanation of contextual modulation by short-range isotropic connections and orientation map geometry in the primary visual cortex. Biol. Cybern. 91, 396–407 (2004).
Wielaard, J. & Sajda, P. Extraclassical receptive field phenomena and short-range connectivity in V1. Cereb. Cortex 16, 1531–1545 (2006).
Dragoi, V., Rivadulla, C. & Sur, M. Foci of orientation plasticity in visual cortex. Nature 411, 80–86 (2001).
Gelbtuch, M. H., Calvert, J. E., Harris, J. P. & Phillipson, O. T. Modification of visual orientation illusions by drugs which influence dopamine and GABA neurones: differential effects on simultaneous and successive illusions. Psychopharmacology (Berl.) 90, 379–383 (1986).
Borst, A., Flanagin, V. L. & Sompolinsky, H. Adaptation without parameter change: dynamic gain control in motion detection. Proc. Natl Acad. Sci. USA 102, 6172–6176 (2005).
Boynton, G. M. & Finney, E. M. Orientation-specific adaptation in human visual cortex. J. Neurosci. 23, 8781–8787 (2003).
Fraser, J. A new visual illusion of direction. Brit. J. Psych. 2, 307–320 (1908).
Kayser, W., Einhauser, C. & Konig, P. Temporal correlations of orientations in natural scenes. Neurocomputing 52–54, 117–123 (2003).
Simoncelli, E. P., Freeman, W. T., Adelson, E. H. & Heeger, D. J. Shiftable multi-scale transforms. IEEE. Trans. Inform. Theory 38, 587–607 (1992).
Simoncelli, E. P. in Proc. 31st Asilomar conf. on Signals, Systems and Computers 673–678 (Pacific Grove, California, USA, 1997).
Acknowledgements
This work was funded by the Howard Hughes Medical Institute (O.S.), the Gatsby Charitable Foundation (A.H., P.D.), the Biotechnology and Biological Sciences Research Council, the Engineering and Physical Sciences Research Council and the Wellcome Trust (P.D). We are very grateful to C. Clifford, A. Kohn, A. Stocker and J. Solomon for comments on the manuscript and discussion, and to T. Sejnowski and E. Simoncelli for discussion.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Related links
Glossary
- Saccade
-
A rapid eye movement (with speeds of up to 800° per second) that brings the point of maximal visual acuity — the fovea — to the image of interest.
- Orientation tuning
-
The property of visual neurons to only respond to stimuli (images) with a certain orientation or tilt (for example, vertically orientated bars).
- Tuning curve
-
A tuning curve to a feature (such as orientation) is the curve describing the average response of a neuron as a function of the feature values.
- Rotation invariance
-
When each input angle is treated in the same way; that is, when the input rotates, the output rotates.
- Discrimination threshold
-
The smallest difference between two visual stimuli (for example, vertical versus tilted bars) that can reliably (that is, with a given probability of error) be reported by an observer.
- Population code
-
Sensory events that are encoded by neuronal populations rather than by individual neurons.
- Poisson spiking neuron
-
A simple model neuron for which the number of spikes emitted in a given time is Poisson distributed about a mean firing rate. Spikes are assumed to be independent both in time and across neurons.
- Fisher information
-
Measures how quickly the likelihood of the population responses changes with stimulus parameters, and thereby provides a decoder-independent quantification of the potential accuracy of decoding (the Cramér-Rao lower bound).
- Mean square estimation error
-
Estimation error can be quantified by the squared difference between the (population-based) estimate and its true value. The mean of this over trials is one measure of the accuracy of an estimate.
- Efficient coding
-
When information is coded in an efficient and non-redundant manner, for instance, when the outputs of neurons in the population are statistically independent.
- Bayes (Bayesian approach)
-
A statistical method that allows the use of prior information to evaluate the posterior probabilities of different hypotheses.
- Kalman filter
-
A recursive formulation that estimates the present outcome dynamically in time, based on prior information and noisy measurements.
- Markov random field
-
An undirected graphical model that represents statistical dependencies between a set of variables. The Markov property is that a variable associated with one location in the image is only directly influenced by variables associated with neighbouring locations.
- Linear (or second order) de-correlation
-
Random variables are de-correlated if the off-diagonal elements of their covariance matrix (representing the second order statistics) are equal to zero. De-correlation is in general a weaker requirement than independence, because higher order statistics may still exhibit dependencies.
- Divisive normalization
-
Strictly speaking, when (for example) sum output across a population is kept constant by dividing each response by a (trial-dependent) quantity. Looser versions model gain control mechanisms in V1 and elsewhere.
- Gain control
-
When the (for example) sum output across a population is used to adjust the gain to an appropriate level for a range of input signal levels, with higher signal levels resulting in higher gain and reduced response. Stricter versions are denoted divisive normalization.
- anti-Hebbian learning
-
A learning rule whereby whenever two units or neurons are active simultaneously, the effective connection between them becomes less excitatory or more inhibitory.
- Bayesian inference
-
Inference according to the standard laws of probability, notably including Bayes theorem. Conclusions are based on posterior distributions arising from combining observations (as probabilistic likelihoods) with prior information.
- Prior
-
A probability distribution that captures the belief or expectation about a variable, in the absence of observations or evidence. Here, priors are specified through personal or evolutionary experience of environmental statistics.
- Decision-theoretic loss function
-
The loss (or cost) associated with a particular decision about a quantity as a function of its true values. Bayesian decision theory suggests that choices should be made by minimizing expected losses under posterior distributions.
- Gibson's normalization
-
The hypothesis that replusive biases arise in the orientation domain due to a long-run prior favouring absolute cardinal axes.
- Power law synapses
-
A synaptic adaptation that is (time) scale invariant; for example, having the same response shape at multiple timescales. This is in contrast to an exponential adaptation process with a single time constant.
Rights and permissions
About this article
Cite this article
Schwartz, O., Hsu, A. & Dayan, P. Space and time in visual context. Nat Rev Neurosci 8, 522–535 (2007). https://doi.org/10.1038/nrn2155
Issue Date:
DOI: https://doi.org/10.1038/nrn2155
This article is cited by
-
Neural correlates of lateral modulation and perceptual filling-in in center-surround radial sinusoidal gratings: an fMRI study
Scientific Reports (2022)
-
Hidden by bias: how standard psychophysical procedures conceal crucial aspects of peripheral visual appearance
Scientific Reports (2021)
-
Improved fidelity of orientation perception: a learning effect dissociable from enhanced discriminability
Scientific Reports (2020)
-
Perceptual bias is reduced with longer reaction times during visual discrimination
Communications Biology (2020)
-
Dissociable neural mechanisms underlie currently-relevant, future-relevant, and discarded working memory representations
Scientific Reports (2020)
