Opinion
Memory as Perception of the Past: Compressed Time inMind and Brain

https://doi.org/10.1016/j.tics.2017.11.004Get rights and content

Trends

The brain contains time cells in many regions that code for the time since a stimulus was encountered.

Time cells show compression analogous to the compression of retinal space in visual perception.

In short-term memory, and perhaps long-term memory, memory retrieval may utilize scanning of a temporal representation, analogous to visual scanning.

In memory the recency effect – a decrease in “memory acuity” with the passage of time – parallels the decrease in acuity with eccentricity in vision.

In memory the contiguity effect – a boost in “memory acuity” for stimuli near in time to a remembered event – is accompanied by a neural jump back in time. This is analogous to the boost in acuity that comes from changing fixation in vision.

In the visual system retinal space is compressed such that acuity decreases further from the fovea. Different forms of memory may rely on a compressed representation of time, manifested as decreased accuracy for events that happened further in the past. Neurophysiologically, “time cells” show receptive fields in time. Analogous to the compression of visual space, time cells show less acuity for events further in the past. Behavioral evidence suggests memory can be accessed by scanning a compressed temporal representation, analogous to visual search. This suggests a common computational language for visual attention and memory retrieval. In this view, time functions like a scaffolding that organizes memories in much the same way that retinal space functions like a scaffolding for visual perception.

Section snippets

The Analogy between Time and Space

Memory theorists have long proposed that memory relies on a representation of past events organized along a timeline [1], [2], [3]. Indeed, it can be shown that our experience of the passage of time requires the involvment of memory [4], [5], [6]. Consider Bergson's description of a person observing a swinging pendulum (quoted at the beginning of this paper). Simultaneous information about both time points A and B is required to determine that A precedes B. Our experience of the passage of time

Neural Evidence for a Compressed Representation of Time

It has long been known that the visual system supports a compressed representation (see Glossary) of visual space. Photoreceptors in the retina respond only to light that lands on a circumscribed region of the retinal surface; this spatial organization is reflected at least through early cortical regions (e.g., [7]). Receptive fields further from the fovea are larger than receptive fields closer to the fovea (Figure 1A, [8]). Moreover, there are more cells with receptive fields centered closer

Behavioral Evidence for a Compressed Timeline

The neural evidence for compressed representations of time and space is paralleled by behavioral evidence for compressed representations. These studies use response times (RTs) to estimate the time taken to access different pieces of information, suggesting the organization of the neural representation.

Toward a Common Computational Language for Visual Attention and Memory Retrieval

If vision and memory have representational and behavioral similarities, then perhaps this means that the neural representations are accessed using a similar computational language. Operations for accessing visual space could be recycled to access time, and perhaps even other domains (such as numerosity) with similar representational compression. In this view, time functions like a scaffolding that organizes memories in much the same way that retinal space functions like a scaffolding for visual

Concluding Remarks

Recent developments in neurophysiology and cognitive psychology suggest a potentially deep analogy between the representation of retinal space in vision and the representation of time in memory. While there are still important gaps in the empirical story, this convergence would create an opportunity for a general framework for cognitive computation operating on similar representations across domains. What might such a general framework look like?

We have suggested that memory and time have

Acknowledgements

Supported by NSF IIS 1631460, ONR MURI N00014-16-1-2832, NIBIB R01EB022864, and NIMH R01MH112169. The author acknowledges helpful discussions with Nick Turk-Browne, Sam Ling, Michele Rucci, David Somers, Ian Bright, and Aude Oliva. Inder Singh helped with the preparation of Figure 2.

Glossary

Allocentric
a coordinate system that is fixed as an animal (or person) moves around in the world. Because they carry information about the animal's location relative to environmental landmarks, place cells and grid cells are believed to code for allocentric space. This is in contrast to an egocentric coordinate system, which moves with movement of the body or a retinotopic coordinate system which moves with the retina.
Compressed representation
neural representations of visual space and time do

References (109)

  • W. James

    The Principles of Psychology

    (1890)
  • C. Gallistel

    The Organization of Learning

    (1990)
  • G. Brown et al.

    A temporal ratio model of memory

    Psychol. Rev.

    (2007)
  • H. Bergson

    Time and Free Will: An Essay on the Immediate Data of Consciousness

    (1910)
  • E. Husserl

    The Phenomenology of Internal Time-consciousness

    (1966)
  • J. Mill

    Analysis of the Phenomena of the Human Mind

    (1878)
  • D. Hubel et al.

    Receptive fields, binocular interaction and functional architecture in the cat's visual cortex

    J. Physiol.

    (1962)
  • D. Hubel et al.

    Uniformity of monkey striate cortex: a parallel relationship between field size, scatter, and magnification factor

    J. Comp. Neurol.

    (1974)
  • P. Daniel et al.

    The representation of the visual field on the cerebral cortex in monkeys

    J.Physiol.

    (1961)
  • E. Schwartz

    Spatial mapping in the primate sensory projection: analytic structure and relevance to perception

    Biol. Cybern.

    (1977)
  • E. Schwartz

    Computational anatomy and functional architecture of striate cortex: a spatial mapping approach to perceptual coding

    Vision Res.

    (1980)
  • D.V. Essen et al.

    The visual field representation in striate cortex of the macaque monkey: asymmetries, anisotropies, and individual variability

    Vision Res

    (1984)
  • C. MacDonald et al.

    Hippocampal “time cells” bridge the gap in memory for discontiguous events

    Neuron

    (2011)
  • M. Howard et al.

    A unified mathematical framework for coding time, space, and sequences in the hippocampal region

    J. Neurosci.

    (2014)
  • G. Mello et al.

    A scalable population code for time in the striatum

    Curr. Biol.

    (2015)
  • S. Bolkan et al.

    Thalamic projections sustain prefrontal activity during working memory maintenance

    Nat. Neurosci.

    (2017)
  • E. Pastalkova et al.

    Internally generated cell assembly sequences in the rat hippocampus

    Science

    (2008)
  • D. Salz et al.

    Time cells in hippocampal area ca3

    J.Neurosci.

    (2016)
  • Z. Tiganj et al.

    Sequential firing codes for time in rodent mPFC

    Cereb. Cortex

    (2017)
  • B. Kraus et al.

    During running in place, grid cells integrate elapsed time and distance run

    Neuron

    (2015)
  • C. MacDonald et al.

    Distinct hippocampal time cell sequences represent odor memories immobilized rats

    J.Neurosci.

    (2013)
  • S. Terada et al.

    Temporal and rate coding for discrete event sequences in the hippocampus

    Neuron

    (2017)
  • D. Jin et al.

    Neural representation of time in cortico-basal ganglia circuits

    Proc. Natl. Acad. Sci. U. S. A.

    (2009)
  • A. Adler et al.

    Temporal convergence of dynamic cell assemblies in the striato-pallidal network

    J.Neurosci.

    (2012)
  • H. Akhlaghpour et al.

    Dissociated sequential activity and stimulus encoding in the dorsomedial striatum during spatial working memory

    eLife

    (2016)
  • M. Howard et al.

    A distributed representation of internal time

    Psychol. Rev.

    (2015)
  • G. Fechner, Elements of Psychophysics, Vol. Vol. I, Houghton Mifflin,...
  • P. Balsam et al.

    Temporal maps and informativeness in associative learning

    Trends Neurosci.

    (2009)
  • N. Chater et al.

    From universal laws of cognition to specific cognitive models

    Cogn. Sci.

    (2008)
  • J. Gibbon

    Scalar expectancy theory and weber's law in animal timing

    Psychol. Rev.

    (1977)
  • J. Gibbon et al.

    Scalar timing in memory

    Ann. N. Y. Acad. Sci.

    (1984)
  • C. McAdams et al.

    Effects of attention on orientation-tuning functions of single neurons in macaque cortical area v4

    J.Neurosci.

    (1999)
  • M. Posner et al.

    Attention and the detection of signals

    J. Exp. Psychol. Gen.

    (1980)
  • A. Treisman et al.

    A feature-integration theory of attention

    Cogn. Psychol.

    (1980)
  • M. Carrasco et al.

    The eccentricity effect: target eccentricity affects performance on conjunction searches

    Percept. Psychophys.

    (1995)
  • M. Hacker

    Speed and accuracy of recency judgments for events in short-term memory

    J. Exp. Psychol. Hum. Learn. Mem.

    (1980)
  • W. Hockley

    Analysis of response time distributions in the study of cognitive processes

    J. Exp. Psychol. Learn. Mem. Cogn.

    (1984)
  • P. Muter

    Response latencies in discriminations of recency

    J. Exp. Psychol. Hum. Learn. Mem.

    (1979)
  • B. McElree et al.

    Serial recovery processes in the recovery of order information

    J. Exp. Psychol. Gen.

    (1993)
  • B. McElree et al.

    Serial position and set size in short-term memory: the time course of recognition

    J. Exp. Psychol. Gen.

    (1989)
  • C. Donkin et al.

    A power-law model of psychological memory strength in short- and long-term recognition

    Psychol. Sci.

    (2012)
  • R. Nosofsky et al.

    Short-term memory scanning viewed as exemplar-based categorization

    Psychol. Rev.

    (2011)
  • M. Chan et al.

    Precise instructions determine participants’ memory search strategy in judgments of relative order in short lists

    Psychon. Bull. Rev.

    (2009)
  • D. Yntema et al.

    Recall as a search process

    J. Verbal Learn. Verbal Behav.

    (1963)
  • J. Hinrichs et al.

    Judgment of recency under steady-state conditions

    J. Exp. Psychol.

    (1968)
  • J. Hinrichs

    A two-process memory-strength theory for judgment of recency

    Psychol. Rev.

    (1970)
  • D.L. Hintzman

    How does repetition affect memory? evidence from judgments of recency

    Mem. Cogn.

    (2010)
  • R. Luce

    Handbook of Mathematical Psychology

    (1959)
  • M. Posner

    Orienting of attention

    Q. J. Exp. Psychol.

    (1980)
  • A. Glenberg et al.

    A two-process account of long-term serial position effects

    J. Exp. Psychol. Hum. Learn. Mem.

    (1980)

    • Cited by (50)

    • Minute-scale periodicity of neuronal firing in the human entorhinal cortex

      2023, Cell Reports

    • The tie that binds: temporal coding and adaptive emotion

      2022, Trends in Cognitive Sciences

    • Time as the fourth dimension in the hippocampus

      2021, Progress in Neurobiology
      Citation Excerpt :

      The matrix W approximates an off-center/on-surround ‘receptive field’ of time cells when time constants are densely spaced in layer I neurons. If temporal and visual/spatial information processing share similar principles of maximizing statistical independence or sparsity in perceived patterns, this type of receptive field could reflect adaptation of the mechanism of temporal information processing to statistical properties of the world (Howard, 2018). Layer I neurons are exponentially decaying persistent firing neurons, maintained by the calcium-activated non-specific (CAN) cation current.

    • Temporal Validity Change Prediction

      2024, arXiv

    • COGNITIVE NEUROSCIENCE OF TIME: NOWS, TIMELINES, AND CHRONOLOGIES

      2023, Time and Science: Volume 1-3

    • Memory out of context: Spacing effects and decontextualization in a computational model of the medial temporal lobe

      2022, bioRxiv
    View all citing articles on Scopus
    View full text
    © 2017 Elsevier Ltd. All rights reserved.