Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Constructing scenes from objects in human occipitotemporal cortex

Nature Neuroscience volume 14, pages 1323–1329 (2011)Cite this article

Subjects

Abstract

We used functional magnetic resonance imaging (fMRI) to demonstrate the existence of a mechanism in the human lateral occipital (LO) cortex that supports recognition of real-world visual scenes through parallel analysis of within-scene objects. Neural activity was recorded while subjects viewed four categories of scenes and eight categories of 'signature' objects strongly associated with the scenes in three experiments. Multivoxel patterns evoked by scenes in the LO cortex were well predicted by the average of the patterns elicited by their signature objects. By contrast, there was no relationship between scene and object patterns in the parahippocampal place area (PPA), even though this region responds strongly to scenes and is believed to be crucial for scene identification. By combining information about multiple objects within a scene, the LO cortex may support an object-based channel for scene recognition that complements the processing of global scene properties in the PPA.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental stimuli.
Figure 2: Logic of scene classification analysis.
Figure 3: Multivoxel classification of scenes using object-based predictors.
Figure 4: Classification of objects on the basis of the patterns elicited by their same-context counterpart objects (for example, the accuracy of discriminating refrigerators from bathtubs on the basis of patterns evoked by stoves and toilets).
Figure 5: Group random-effects analysis of local searchlight accuracy maps for the classification of scenes from object averages, including subjects from all three experiments.
Figure 6: Behavioral evidence for object-based scene recognition.

Similar content being viewed by others

References

  1. Potter, M.C. Meaning in visual search. Science 187, 965–966 (1975).

    Article  CAS  Google Scholar 

  2. Biederman, I., Rabinowitz, J.C., Glass, A.L. & Stacy, E.W. On the Information extracted from a glance at a scene. J. Exp. Psychol. 103, 597–600 (1974).

    Article  CAS  Google Scholar 

  3. Schyns, P.G. & Oliva, A. From blobs to boundary edges: Evidence for time- and spatial-scale-dependent scene recognition. Psychol. Sci. 5, 195–200 (1994).

    Article  Google Scholar 

  4. Renninger, L.W. & Malik, J. When is scene identification just texture recognition? Vision Res. 44, 2301–2311 (2004).

    Article  Google Scholar 

  5. Greene, M.R. & Oliva, A. Recognition of natural scenes from global properties: seeing the forest without representing the trees. Cognit. Psychol. 58, 137–176 (2009).

    Article  Google Scholar 

  6. Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature 392, 598–601 (1998).

    Article  CAS  Google Scholar 

  7. Aguirre, G.K., Zarahn, E. & D'Esposito, M. An area within human ventral cortex sensitive to “building” stimuli: evidence and implications. Neuron 21, 373–383 (1998).

    Article  CAS  Google Scholar 

  8. Park, S., Brady, T.F., Greene, M.R. & Oliva, A. Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31, 1333–1340 (2011).

    Article  CAS  Google Scholar 

  9. Malach, R. et al. Object-related activity revealed by functional magnetic-resonance-imaging in human occipital cortex. Proc. Natl. Acad. Sci. USA 92, 8135–8139 (1995).

    Article  CAS  Google Scholar 

  10. Peelen, M.V., Fei-Fei, L. & Kastner, S. Neural mechanisms of rapid natural scene categorization in human visual cortex. Nature 460, 94–97 (2009).

    Article  CAS  Google Scholar 

  11. Epstein, R., Graham, K.S. & Downing, P.E. Viewpoint-specific scene representations in human parahippocampal cortex. Neuron 37, 865–876 (2003).

    Article  CAS  Google Scholar 

  12. Haxby, J.V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).

    Article  CAS  Google Scholar 

  13. Walther, D.B., Caddigan, E., Fei-Fei, L. & Beck, D.M. Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. 29, 10573–10581 (2009).

    Article  CAS  Google Scholar 

  14. Drucker, D.M. & Aguirre, G.K. Different spatial scales of shape similarity representation in lateral and ventral LOC. Cereb. Cortex 19, 2269–2280 (2009).

    Article  Google Scholar 

  15. Haushofer, J., Livingstone, M.S. & Kanwisher, N. Multivariate patterns in object-selective cortex dissociate perceptual and physical shape similarity. PLoS Biol. 6, e187 (2008).

    Article  Google Scholar 

  16. MacEvoy, S.P. & Epstein, R.A. Decoding the representation of multiple simultaneous objects in human occipitotemporal cortex. Curr. Biol. 19, 943–947 (2009).

    Article  CAS  Google Scholar 

  17. Diana, R.A., Yonelinas, A.P. & Ranganath, C. High-resolution multi-voxel pattern analysis of category selectivity in the medial temporal lobes. Hippocampus 18, 536–541 (2008).

    Article  Google Scholar 

  18. Bar, M. & Aminoff, E. Cortical analysis of visual context. Neuron 38, 347–358 (2003).

    Article  CAS  Google Scholar 

  19. Potter, M.C., Staub, A. & O'Connor, D.H. Pictorial and conceptual representation of glimpsed pictures. J. Exp. Psychol. Hum. Percept. Perform. 30, 478–489 (2004).

    Article  Google Scholar 

  20. Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. USA 103, 3863–3868 (2006).

    Article  CAS  Google Scholar 

  21. Rogosa, D. Comparing nonparallel regression lines. Psychol. Bull. 88, 307–321 (1980).

    Article  Google Scholar 

  22. Naselaris, T., Prenger, R.J., Kay, K.N., Oliver, M. & Gallant, J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 63, 902–915 (2009).

    Article  CAS  Google Scholar 

  23. Kay, K.N., Naselaris, T., Prenger, R.J. & Gallant, J.L. Identifying natural images from human brain activity. Nature 452, 352–355 (2008).

    Article  CAS  Google Scholar 

  24. Reddy, L., Kanwisher, N.G. & VanRullen, R. Attention and biased competition in multi-voxel object representations. Proc. Natl. Acad. Sci. USA 106, 21447–21452 (2009).

    Article  CAS  Google Scholar 

  25. Zoccolan, D., Cox, D.D. & DiCarlo, J.J. Multiple object response normalization in monkey inferotemporal cortex. J. Neurosci. 25, 8150–8164 (2005).

    Article  CAS  Google Scholar 

  26. Moran, J. & Desimone, R. Selective attention gates visual processing in the extrastriate cortex. Science 229, 782–784 (1985).

    Article  CAS  Google Scholar 

  27. MacEvoy, S.P., Tucker, T.R. & Fitzpatrick, D. A precise form of divisive suppression supports population coding in the primary visual cortex. Nat. Neurosci. 12, 637–645 (2009).

    Article  CAS  Google Scholar 

  28. Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995).

    Article  CAS  Google Scholar 

  29. Reynolds, J.H. & Heeger, D.J. The normalization model of attention. Neuron 61, 168–185 (2009).

    Article  CAS  Google Scholar 

  30. Beck, D.M. & Kastner, S. Stimulus similarity modulates competitive interactions in human visual cortex. J. Vis. 7 (19), 11–12 (2007).

    Google Scholar 

  31. Beck, D.M. & Kastner, S. Stimulus context modulates competition in human extrastriate cortex. Nat. Neurosci. 8, 1110–1116 (2005).

    Article  CAS  Google Scholar 

  32. Fei-Fei, L., Iyer, A., Koch, C. & Perona, P. What do we perceive in a glance of a real-world scene? J. Vis. 7 (1), 10 (2007).

    Article  Google Scholar 

  33. Schwarzlose, R.F., Swisher, J.D., Dang, S. & Kanwisher, N. The distribution of category and location information across object-selective regions in human visual cortex. Proc. Natl. Acad. Sci. USA 105, 4447–4452 (2008).

    Article  CAS  Google Scholar 

  34. Sayres, R. & Grill-Spector, K. Relating retinotopic and object-selective responses in human lateral occipital cortex. J. Neurophysiol. 100, 249–267 (2008).

    Article  Google Scholar 

  35. Kravitz, D.J., Kriegeskorte, N. & Baker, C.I. High-level visual object representations are constrained by position. Cereb. Cortex 20, 2916–2925 (2010).

    Article  Google Scholar 

  36. Andresen, D.R., Vinberg, J. & Grill-Spector, K. The representation of object viewpoint in human visual cortex. Neuroimage 45, 522–536 (2009).

    Article  Google Scholar 

  37. Eger, E., Kell, C.A. & Kleinschmidt, A. Graded size sensitivity of object-exemplar-evoked activity patterns within human LOC subregions. J. Neurophysiol. 100, 2038–2047 (2008).

    Article  Google Scholar 

  38. Epstein, R.A. Parahippocampal and retrosplenial contributions to human spatial navigation. Trends Cogn. Sci. 12, 388–396 (2008).

    Article  Google Scholar 

  39. Kravitz, D.J., Peng, C.S. & Baker, C.I. Real-world scene representations in high-level visual cortex: it's the spaces more than the places. J. Neurosci. 31, 7322–7333 (2011).

    Article  CAS  Google Scholar 

  40. Spiridon, M. & Kanwisher, N. How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron 35, 1157–1165 (2002).

    Article  CAS  Google Scholar 

  41. Habib, M. & Sirigu, A. Pure topographical disorientation: a definition and anatomical basis. Cortex 23, 73–85 (1987).

    Article  CAS  Google Scholar 

  42. Aguirre, G.K. & D'Esposito, M. Topographical disorientation: a synthesis and taxonomy. Brain 122, 1613–1628 (1999).

    Article  Google Scholar 

  43. Aguirre, G.K. Continuous carry-over designs for fMRI. Neuroimage 35, 1480–1494 (2007).

    Article  Google Scholar 

  44. Epstein, R.A. & Higgins, J.S. Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. Cereb. Cortex 17, 1680–1693 (2007).

    Article  Google Scholar 

  45. Epstein, R.A., Parker, W.E. & Feiler, A.M. Where am I now? Distinct roles for parahippocampal and retrosplenial cortices in place recognition. J. Neurosci. 27, 6141–6149 (2007).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors wish to thank E. Ward, A. Stigliani and Z. Yang for assistance with data collection. This work was supported by US National Eye Institute grant EY-016464 to R.A.E.

Author information

Authors and Affiliations

  1. Department of Psychology, Boston College, Chestnut Hill, Massachusetts, USA

    Sean P MacEvoy

  2. Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Sean P MacEvoy & Russell A Epstein

Authors
  1. Sean P MacEvoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Russell A Epstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.P.M. and R.A.E. designed the experiments. S.P.M. collected fMRI data and R.A.E. collected behavioral data. S.P.M. analyzed data with input from R.A.E. S.P.M. and R.A.E. wrote the manuscript.

Corresponding author

Correspondence to Sean P MacEvoy.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7, Supplementary Results (PDF 380 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

MacEvoy, S., Epstein, R. Constructing scenes from objects in human occipitotemporal cortex. Nat Neurosci 14, 1323–1329 (2011). https://doi.org/10.1038/nn.2903

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2903

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing