Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

The parallel distributed processing approach to semantic cognition

Nature Reviews Neuroscience volume 4, pages 310–322 (2003)Cite this article

Key Points

  • Semantic cognition encompasses human performance based on knowledge about the properties of objects, relations among objects and word meanings. One approach to semantic cognition has arisen within the parallel distributed processing (PDP) framework, in which cognitive processes arise from interactions of neurons through synaptic connections. The knowledge that governs processing is stored in the strengths of the connections and is acquired gradually through experience, simulating conceptual development in childhood.

  • These ideas have been explored in a simulated neural network model that learns propositions about objects and their properties. The model is trained with propositions about several different plant and animal concepts, including trees, flowers, fish, birds and land animals.

  • The model contains 'hidden' units between its inputs and outputs, over which it learns internal representations that capture semantic relationships between concepts. Learning is influenced by coherent covariation of properties — that is, by co-occurrence of the same ensemble of properties (has wings, has feathers, can fly) in a number of different items (in this case, all the birds).

  • The model explains the tendency towards progressive differentiation of concepts observed in development and the reverse fine-to-coarse deterioration observed in a progressive neuropathological condition called semantic dementia. With appropriate assumptions about covariation of properties, and about the relative frequencies of concepts and of the words used to name them, the model also addresses many further findings in development, dementia and normal adult cognition.

  • Like other, similarity-based theories, the model accounts for the influence of graded category membership on semantic task performance, and for frequency and typicality effects. It also provides a means of addressing some of the criticisms of these other theories. Specifically, it indicates how some properties of objects, including causal properties, come to be more important than other properties; why some groups of items seem to form natural or coherent categories; how domain-specific patterns of generalization and differentiation might arise; and how conceptual knowledge structures might reorganize over the course of development.

  • The PDP approach might provide a mechanistic framework that can address many of the phenomena emphasized in an alternative approach based on naive domain theories specifying causal relations between objects and their properties. Some of the relevant phenomena have yet to be addressed by PDP models, leaving this as a task for the future.

Abstract

How do we know what properties something has, and which of its properties should be generalized to other objects? How is the knowledge underlying these abilities acquired, and how is it affected by brain disorders? Our approach to these issues is based on the idea that cognitive processes arise from the interactions of neurons through synaptic connections. The knowledge in such interactive and distributed processing systems is stored in the strengths of the connections and is acquired gradually through experience. Degradation of semantic knowledge occurs through degradation of the patterns of neural activity that probe the knowledge stored in the connections. Simulation models based on these ideas capture semantic cognitive processes and their development and disintegration, encompassing domain-specific patterns of generalization in young children, and the restructuring of conceptual knowledge as a function of experience.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The hierarchical propositional model of Quillian1 applied to the domain of living things, as adapted by Rumelhart60,61.
Figure 2: Evidence of conceptual disintegration in semantic dementia.
Figure 3: Our depiction of the connectionist network used by Rumelhart60,61.
Figure 4: The process of differentiation of conceptual representations.
Figure 5: Two-dimensional projection of the representations of the various concepts after learning in the Rumelhart60,61 network, and effects of damage on activations of properties of individual concepts.
Figure 6: Trajectories of learning concepts and their names in a network trained with 21 concepts in five categories (trees, flowers, birds, fish and land animals).
Figure 7: Hierarchical clustering of the 21 concepts used in the simulation capturing the coalescence of underlying conceptual structure after initial acquisition of superficial, appearance-based structure.

Similar content being viewed by others

References

  1. Quillian, M. R. in Semantic Information Processing (ed. Minsky, M.) 227–270 (MIT Press, Cambridge, Massachusetts, 1968).

    Google Scholar 

  2. Warrington, E. K. The selective impairment of semantic memory. Q. J. Exp. Psychol. 27, 635–657 (1975). Classic paper first reporting the syndrome that is now known as semantic dementia.

    Article  CAS  PubMed  Google Scholar 

  3. Snowden, J. S., Goulding, P. J. & Neary, D. Semantic dementia: a form of circumscribed cerebral atrophy. Behav. Neurol. 2, 167–182 (1989).

    Article  Google Scholar 

  4. Hodges, J. R., Graham, N. & Patterson, K. Charting the progression in semantic dementia: implications for the organisation of semantic memory. Memory 3, 463–495 (1995).

    Article  CAS  PubMed  Google Scholar 

  5. Bozeat, S., Lambon-Ralph, M. A., Patterson, K. & Hodges, J. R. When objects lose their meanings: what happens to their use? Cogn. Affect. Behav. Neurosci. (in the press).

  6. Rogers, T. T., McClelland, J. L. Patterson, K., Lambon-Ralph, M. A. & Hodges, J. R. A recurrent connectionist model of semantic dementia. Cogn. Neurosci. Soc. Annual Meeting <http://cognet.mit.edu/posters/poster.tcl?publication_id=3664> (1999).

  7. Keil, F. C. Semantic and Conceptual Development: An Ontological Perspective (Harvard Univ. Press, Cambridge, Massachusetts, 1979).

    Book  Google Scholar 

  8. Mandler, J. M. How to build a baby: II. Conceptual primitives. Psychol. Rev. 99, 587–604 (1992).

    Article  CAS  PubMed  Google Scholar 

  9. Mandler, J. M. Perceptual and conceptual processes in infancy. J. Cogn. Dev. 1, 3–36 (2000). Reviews 10 years of research on progressive differentiation of conceptual knowledge in infancy.

    Article  Google Scholar 

  10. Mandler, J. M., Bauer, P. J. & McDonough, L. Separating the sheep from the goats: differentiating global categories. Cogn. Psychol. 23, 263–298 (1991).

    Article  Google Scholar 

  11. Mandler, J. M. & McDonough, L. Concept formation in infancy. Cogn. Dev. 8, 291–318 (1993).

    Article  Google Scholar 

  12. Pauen, S. The global-to-basic shift in infants' categorical thinking: first evidence from a longitudinal study. Int. J. Behav. Dev. 26, 492–499 (2002).

    Article  Google Scholar 

  13. Rakison, D. in Early Concept and Category Development: Making Sense of the Blooming, Buzzing Confusion (eds Oakes, L. M. & Rakison, D. H.) (Oxford Univ. Press, New York, in the press).

  14. Mareschal, D. Infant object knowledge: current trends and controversies. Trends Cogn. Sci. 4, 408–416 (2000).

    Article  CAS  PubMed  Google Scholar 

  15. Quinn, P. C. in Blackwell Handbook of Childhood Cognitive Development (ed. Goswami, U.) 84–101 (Blackwell, Oxford, UK, 2002).

    Book  Google Scholar 

  16. Collins, A. M. & Quillian, M. R. Retrieval time from semantic memory. J. Verbal Learn. Verbal Behav. 8, 240–248 (1969).

    Article  Google Scholar 

  17. McCloskey, M. & Glucksberg, S. Decision processes in verifying category membership statements: implications for models of semantic memory. Cogn. Psychol. 11, 1037 (1979).

    Article  Google Scholar 

  18. Murphy, G. L. & Brownell, H. H. Category differentiation in object recognition: typicality constraints on the basic category advantage. J. Exp. Psychol. Learn. Mem. Cogn. 11, 70–84 (1985).

    Article  CAS  PubMed  Google Scholar 

  19. Rips, L. J., Shoben, E. J. & Smith, E. E. Semantic distance and the verification of semantic relations. J. Verbal Learn. Verbal Behav. 12, 1–20 (1973).

    Article  Google Scholar 

  20. Rosch, E. Cognitive representations of semantic categories. J. Exp. Psychol. Gen. 104, 192–233 (1975).

    Article  Google Scholar 

  21. Rumelhart, D. E. & Ortony, A. in Schooling and the Acquisition of Knowledge (eds Anderson, R. C., Sprio, R. J. & Montague, W. E.) 99–135 (Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1976).

    Google Scholar 

  22. Rumelhart, D. E. & Abrahamson, A. A. A model of analogical reasoning. Cogn. Psychol. 5, 1–28 (1973).

    Article  Google Scholar 

  23. Anderson, J. R. The Adaptive Character of Thought (Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1990).

    Google Scholar 

  24. Heit, E. in Rational Models of Cognition (eds Oaksford, M. & Chater, N.) 248–274 (Oxford Univ. Press, Oxford, 1998).

    Google Scholar 

  25. Heit, E. Properties of inductive reasoning. Psychon. Bull. Rev. 7, 569–592 (2000).

    Article  CAS  PubMed  Google Scholar 

  26. Medin, D. L. & Shaffer, M. M. Context theory of classification learning. Psychol. Rev. 85, 207–238 (1978).

    Article  Google Scholar 

  27. McClelland, J. L. Retrieving general and specific information from stored knowledge of specifics. Proc. Third Annu. Conf. Cogn. Sci. Soc. 170–172 (1981).

  28. Nosofsky, R. M. Attention, similiarity and the identification-categorization relationship. J. Exp. Psychol. Learn. Mem. Cogn. 115, 39–57 (1986).

    CAS  Google Scholar 

  29. Kruschke, J. K. ALCOVE: an exemplar-based connectionist model of category learning. Psychol. Rev. 99, 22–44 (1992).

    Article  CAS  PubMed  Google Scholar 

  30. Nosofsky, R. M. & Palmeri, T. J. An exemplar-based random-walk model of speeded classification learning. Psychol. Rev. 104, 266–300 (1997).

    Article  CAS  PubMed  Google Scholar 

  31. Murphy, G. L. The Big Book of Concepts (MIT Press, Cambridge, Massachusetts, 2002). An up-to-date review of the literature on concepts and semantic cognition.

    Book  Google Scholar 

  32. Rumelhart, D. E., McClelland, J. L. & the PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations (MIT Press, Cambridge, Massachusetts, 1986). Introduces the parallel distributed processing framework. Chapter 6 describes back-propagation, which is briefly presented in reference 35.

    Book  Google Scholar 

  33. Hinton, G. E. in Parallel Models of Associative Memory (eds Hinton, G. E. & Anderson, J. A.) 161–187 (Erlbaum, Hillsdale, New Jersey, 1981).

    Google Scholar 

  34. Ackley, D. H., Hinton, G. E. & Sejnowski, T. J. A learning algorithm for Boltzmann machines. Cogn. Sci. 9, 147–169 (1985).

    Article  Google Scholar 

  35. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).

    Article  Google Scholar 

  36. Williams, R. J. & Zipser, D. A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1, 270–280 (1989).

    Article  Google Scholar 

  37. Hinton, G. E. & McClelland, J. L. in Neural Information Processing Systems (ed. Anderson, D. Z.) 358–366 (American Institute of Physics, New York, 1988).

    Google Scholar 

  38. Mazzoni, P., Andersen, R. A. & Jordan, M. I. A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA 88, 4433–4437 (1991).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. O'Reilly, R. C. Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm. Neural Comput. 8, 895–938 (1996).

    Article  Google Scholar 

  40. Devlin, J. T., Gonnerman, L. M., Andersen, E. S. & Seidenberg, M. S. Category specific semantic deficits in focal and widespread brain damage: a computational account. J. Cogn. Neurosci. 10, 77–94 (1998).

    Article  CAS  PubMed  Google Scholar 

  41. Farah, M. J. & McClelland, J. L. A computational model of semantic memory impairment: modality-specificity and emergent category-specificity. J. Exp. Psychol. Gen. 120, 339–357 (1991).

    Article  CAS  PubMed  Google Scholar 

  42. Gotts, S. & Plaut, D. C. The impact of synaptic depression following brain damage: a connectionist account of 'access/refractory' and 'degraded-store' semantic impairments. Cogn. Affect. Behav. Neurosci. 2, 187–213 (2002).

    Article  PubMed  Google Scholar 

  43. Hinton, G. E. in Parallel Distributed Processing: Implications for Psychology and Neurobiology (ed. Morris, R. G. M.) 46–61 (Clarendon, Oxford, 1989).

    Google Scholar 

  44. Hinton, G. E. & Shallice, T. Lesioning a connectionist network: investigations of acquired dyslexia. Technical Report No. CRG-TR-89–3 (University of Toronto, Department of Computer Science, Toronto, Ontario, Canada, 1989).

  45. Lambon Ralph, M. A., McClelland, J. L., Patterson, K., Galton, C. J. & Hodges, J. R. No right to speak? The relationship between object naming and semantic impairment: neuropsychological evidence and a computational model. J. Cogn. Neurosci. 13, 341–356 (2001).

    Article  CAS  PubMed  Google Scholar 

  46. McRae, K., de Sa, V. R. & Seidenberg, M. S. On the nature and scope of featural representations of word meaning. J. Exp. Psychol. Gen. 126, 99–130 (1997).

    Article  CAS  PubMed  Google Scholar 

  47. Plaut, D. C. Graded modality-specific specialization in semantics: a computational account of optic aphasia. Cogn. Neuropsychol. 19, 603–639 (2002).

    Article  PubMed  Google Scholar 

  48. Plaut, D. C. & Booth, J. R. Individual and developmental differences in semantic priming: empirical and computational support for a single-mechanism account of lexical processing. Psychol. Rev. 107, 786–823 (2000).

    Article  CAS  PubMed  Google Scholar 

  49. Plaut, D. C. & Shallice, T. Deep dyslexia: a case study of connectionist neuropsychology. Cogn. Neuropsychol. 10, 377–500 (1993).

    Article  Google Scholar 

  50. Tyler, L. K., Moss, H. E., Durrant-Peatfield, M. R. & Levy, J. P. Conceptual structure and the structure of concepts: a distributed account of category-specific deficits. Brain Lang. 75, 195–231 (2000).

    Article  CAS  PubMed  Google Scholar 

  51. Tyler, L. K. & Moss, H. E. Towards a distributed account of conceptual knowledge. Trends Cogn. Sci. 5, 244–252 (2001).

    Article  PubMed  Google Scholar 

  52. Mareschal, D., French, R. M. & Quinn, P. C. A connectionist account of asymmetric category learning in early infancy. Dev. Psychol. 36, 635–645 (2000).

    Article  PubMed  Google Scholar 

  53. Mareschal, D., Plunkett, K. & Harris, P. A computational and neuropsychological account of object-oriented behaviours in infancy. Dev. Sci. 2, 306–317 (1999).

    Article  Google Scholar 

  54. McClelland, J. L. in Parallel Distribured Processing: Implications for Psychology and Neurobiology (Morris, R. G. M.) 8–45 (Oxford Univ. Press, New York, 1989).

    Google Scholar 

  55. Munakata, Y., McClelland, J. L., Johnson, M. H. & Siegler, R. S. Rethinking infant knowledge: toward an adaptive process accout of successes and failures in object permanence tasks. Psychol. Rev. 104, 686–713 (1997).

    Article  CAS  PubMed  Google Scholar 

  56. Munakata, Y. & McClelland, J. L. Connectionist models of development. Dev. Sci. (in the press).

  57. Quinn, P. C. & Johnson, M. H. The emergence of perceptual category representations in young infants: A connectionist analysis. J. Exp. Child Psychol. 66, 236–263 (1997).

    Article  CAS  PubMed  Google Scholar 

  58. Schyns, P. G. A modular neural network model of concept acquisition. Cogn. Sci. 15, 461–508 (1991).

    Article  Google Scholar 

  59. Shultz, T. R., Mareschal, D. & Schmidt, W. C. Modeling cognitive development of balance scale phenomena. Machine Learn. 16, 57–86 (1994).

    Google Scholar 

  60. Rumelhart, D. E. in An Introduction to Neural and Electronic Networks (eds Zornetzer, S. F., Davis, J. L. & Lau, C.) 405–420 (Academic, New York, 1990).

    Google Scholar 

  61. Rumelhart, D. E. & Todd, P. M. in Attention and Performance XIV: Synergies in Experimental Psychology, Artifical Intelligence, and Cognitive Neuroscience (eds Meyer, D. E. & Kornblum, S.) 3–30 (MIT Press, Cambridge, Massachusetts, 1993).

    Google Scholar 

  62. Flavell, J. The Developmental Psychology of Jean Piaget (Van Nostrand, New York, 1963).

    Book  Google Scholar 

  63. McClelland, J. L., McNaughton, B. L. & O'Reilly, R. C. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419–457 (1995). Elaborates the complementary learning system theory that is described briefly in the current article.

    Article  PubMed  Google Scholar 

  64. Quinn, P. C. & Johnson, M. H. Global-before-basic object categorization in connectionist networks and 2-month-old infants. Infancy 1, 31–46 (2000).

    Article  PubMed  Google Scholar 

  65. Bloom, P. How Children Learn the Meaning of Words (MIT Press, Cambridge, Massachusetts, 2000).

    Book  Google Scholar 

  66. McCloskey, M. & Cohen, N. J. in The Psychology of Learning and Motivation (ed. Bower, G. H.) 109–165 (Academic, New York, 1989).

    Google Scholar 

  67. Marr, D. Simple memory: a theory for archicortex. Phil. Trans. R. Soc. Lond. B 262, 23–81 (1971).

    Article  CAS  Google Scholar 

  68. Milner, P. A cell assembly theory of hippocampal amnesia. Neuropsychologia 27, 23–30 (1989).

    Article  CAS  PubMed  Google Scholar 

  69. Rolls, E. T. in An Introduction to Neural and Electronic Networks (eds Zornetzer, S. F., Davis, J. L. & Lau, C.) 73–90 (Academic, San Diego, California, 1990).

    Google Scholar 

  70. Alvarez, P. & Squire L. R. Memory consolidation and the medial temporal lobe: a simple network model. Proc. Natl Acad. Sci. USA 91, 7041–7045 (1994).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  71. Scoville, W. B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurosurg. Neurol. Psychiatry 20, 11–21 (1957).

    Article  CAS  Google Scholar 

  72. Squire, L. R. Memory and the hippocampus: a synthesis from findings with rats, monkeys and humans. Psychol. Rev. 99, 195–231 (1992). A useful overview of amnesia resulting from lesions to the medial temporal lobes.

    Article  CAS  PubMed  Google Scholar 

  73. Nadel, L. & Moscovitch, M. Memory consolidation, retrograde amnesia and the hippocampal complex. Curr. Opin. Neurobiol. 7, 217–227 (1997).

    Article  CAS  PubMed  Google Scholar 

  74. Cipolotti, L. et al. Long-term retrograde amnesia: the crucial role of the hippocampus. Neuropsychologia 39, 151–172 (2001).

    Article  CAS  PubMed  Google Scholar 

  75. Norman, K. A. & O'Reilly, R. C. Modeling hippocampal and neocortical contributions to recognition memory: a complementary learning systems approach. Psychol. Rev. (in the press).

  76. Hodges, J. R., Patterson, K., Oxbury, S. & Funnell, E. Semantic dementia: progressive fluent aphasia with temporal lobe atrophy. Brain 115, 1783–1806 (1992).

    Article  PubMed  Google Scholar 

  77. Garrard, P. & Hodges, J. R. Semantic dementia: clinical, radiological, and pathological perspectives. J. Neurol. 247, 409–422 (2000).

    Article  CAS  PubMed  Google Scholar 

  78. Rogers, T. T. & McClelland, J. L. Semantic Cognition: A Parallel Distributed Processing Approach (MIT Press, Cambridge, Massachusetts, in the press). A full presentation of the PDP approach.

  79. Mervis, C. B. in Concepts and Conceptual Development: Ecological and Intellectual Factors in Categorization (ed. Neisser, U.) 201–233 (Cambridge Univ. Press, Cambridge, 1987).

    Google Scholar 

  80. Jolicoeur, P., Gluck, M. & Kosslyn, S. M. Pictures and names: making the connection. Cogn. Psychol. 19, 31–53 (1984).

    Google Scholar 

  81. Rosch, E., Mervis, C. B., Gray, W., Johnson, D. & Boyes-Braem, P. Basic objects in natural categories. Cogn. Psychol. 8, 382–439 (1976).

    Article  Google Scholar 

  82. Brown, R. A First Language (Harvard Univ. Press, Cambridge, Massachusetts, 1973).

    Book  Google Scholar 

  83. MacWhinney, B. The CHILDES Project: Tools for Analyzing Talk (Lawrence Erlbaum Associates, Mahwah, New Jersey, 2000).

    Google Scholar 

  84. Carey, S. Conceptual Change in Childhood (MIT Press, Cambridge, Massachusetts, 1985). Seminal studies of conceptual reorganization in children.

    Google Scholar 

  85. Keil, F. C. in The Epigenesis of Mind: Essays on Biology and Cognition (eds Carey, S. & Gelman, R.) 237–256 (Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1991).

    Google Scholar 

  86. Murphy, G. L. & Medin, D. L. The role of theories in conceptual coherence. Psychol. Rev. 92, 289–316 (1985).

    Article  CAS  PubMed  Google Scholar 

  87. Gopnik, A. & Wellman, H. M. in Mapping the Mind: Domain Specificity in Cognition and Culture (eds Hirschfeld, L. A. & Gelmans, S. A.) (Cambridge Univ. Press, Cambridge, 1994). One of the most explicit attempts to articulate the nature of 'theory theory'.

    Google Scholar 

  88. Wilson, R. A. & Keil, F. C. in Explanation and Cognition (eds Keil, F. C. & Wilson, R. A.) 87–114 (MIT Press, Boston, Massachusetts, 2000).

    Google Scholar 

  89. Gelman, R. First principles organize attention to and learning about relevant data: Number and the animate/inanimate distinction as examples. Cogn. Sci. 14, 79–106 (1990).

    Article  Google Scholar 

  90. Macario, J. F. Young children's use of color in classification: Foods and canonically colored objects. Cogn. Dev. 6, 17–46 (1991).

    Article  Google Scholar 

  91. Gelman, S. A. & Markman, E. M. Categories and induction in young children. Cognition 23, 183–209 (1986).

    Article  CAS  PubMed  Google Scholar 

  92. Kuhn, T. The Structure of Scientific Revolutions (Univ. Chicago Press, Chicago, 1962).

    Google Scholar 

  93. Ahn, W. -K. Why are different features central for natural kinds and artifacts?: The role of causal status in determining feature centrality. Cognition 69, 135–178 (1998).

    Article  CAS  PubMed  Google Scholar 

  94. Ahn, W., Gelman, S., Amsterlaw, J. A., Hohenstein, J. & Kalish, C. W. Causal status effect in children's categorization. Cognition 76, B35–B43 (2000).

    Article  CAS  PubMed  Google Scholar 

  95. Gopnik, A. & Sobel, D. M. Detecting blickets: how young children use information about novel causal powers in categorization and induction. Child Dev. 71, 1205–1222 (2000).

    Article  CAS  PubMed  Google Scholar 

  96. Keil, f. Concepts, Kinds, and Cognitive Development (MIT Press, Cambridge, Massachusetts, 1989). Presents key findings relevant to the role of causal mechanisms in determining what kind of thing an object is.

    Google Scholar 

  97. Elman, J. L. Finding structure in time. Cogn. Sci. 14, 179–211 (1990).

    Article  Google Scholar 

  98. Cleeremans, A., Servan-Schreiber, D. & McClelland, J. L. Finite state automata and simple recurrent networks. Neural Comput. 1, 372–381 (1989).

    Article  Google Scholar 

  99. Rogers, T. T. & Griffin, R. in Proc. 13th Biennial Int. Conf. Infant Studies (eds Schmuckler, M. A. Trehub, S. E. & Kosarev, M. F.) 625 (International Society for Infant Studies, 2002).

    Google Scholar 

  100. St. John, M. F. The story Gestalt: a model of knowledge-intensive processes in text comprehension. Cogn. Sci. 16, 271–306 (1992).

    Article  Google Scholar 

  101. Marcus, G. F. The Algebraic Mind (MIT Press, Cambridge, Massachusetts, 2001). Critique of the parallel distributed processing framework, offering an alternative perspective on the basic nature of human cognitive and semantic processing.

    Book  Google Scholar 

  102. Carey, S. & Spelke, E. in Domain-Specific Knowledge and Conceptual Change (ed. Hirschfeld, L. A.) 169–200 (Cambridge Univ. Press, New York, 1994).

    Google Scholar 

  103. Barsalou, L. W., Simmons, W. K., Barbey, A. & Wilson, C. D. Grounding conceptual knowledge in modality-specific systems. Trends Cogn. Sci. (in the press).

  104. Damasio, A. R. The brain binds entities and events by multiregional activation from convergence zones. Neural Comput. 1, 123–132 (1989).

    Article  Google Scholar 

  105. Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L. & Ungerleider, L. G. Discrete cortical regions associated with knowledge of color and knowledge of action. Science 270, 102–105 (1995).

    Article  CAS  PubMed  Google Scholar 

  106. Kellenbach, M. L., Brett, M. & Patterson, K. Actions speak louder than functions: the importance of manipularity and action in tool representation. J. Cogn. Neurosci. 15, 30–46 (2003).

    Article  PubMed  Google Scholar 

  107. Kellenbach, M. L., Brett, M. & Patterson, K. Larger, colorful, or noisy? Attribute- and modality-specific activations during retrieval of perceptual attribut knowledge. Cogn. Affect. Behav. Neurosci. 1, 207–221 (2001).

    Article  CAS  PubMed  Google Scholar 

  108. Martin, A. & Chao, L. L. Semantic memory in the brain: structure and processes. Curr. Opin. Neurobiol. 11, 194–201 (2001). A useful contemporary review of findings on semantic representation in the brain.

    Article  CAS  PubMed  Google Scholar 

  109. Warrington, E. K. & Shallice, T. Category specific semantic impairments. Brain 107, 829–854 (1984). Seminal report of category-specific deficits in semantic knowledge.

    Article  PubMed  Google Scholar 

  110. Rogers, T. T. & Plaut, D. C. in Category Specificity in Mind and Brain (eds Forde, E. & Humphreys, G.) 251–289 (Psychology Press, East Sussex, UK, 2002).

    Google Scholar 

  111. Gonnerman, L. M., Andersen, E. S., Devlin, J. T., Kempler, D. & Seidenberg, M. S. Double dissociation of semantic categories in Alzheimer's disease. Brain Lang. 57, 254–279 (1997).

    Article  CAS  PubMed  Google Scholar 

  112. Allport, D. A. in Current Perspectives in Dysphasia (eds Newman, S. K. & Epstein, R.) 207–244 (Churchill Livingstone, Edinburgh, 1985).

    Google Scholar 

  113. Caramazza, A. & Shelton, J. R. Domain-specific knowledge systems in the brain: the animate-inanimate distinction. J. Cogn. Neurosci. 10, 1–34 (1998).

    Article  CAS  PubMed  Google Scholar 

  114. Warrington, E. K. & McCarthy, R. Categories of knowledge: further fractionation and an attempted integration. Brain 110, 1273–1296 (1987).

    Article  PubMed  Google Scholar 

  115. Jacobs, R. A. & Jordan, M. I. Computational consequences of a bias toward short connections. J. Cogn. Neurosci. 4, 323–336 (1992).

    Article  CAS  PubMed  Google Scholar 

  116. Kohonen, T. The self-organizing map. Proc. IEEE 78, 1464–1480 (1990).

    Article  Google Scholar 

  117. Snodgrass, J. G. & Vanderwart, M. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J. Exp. Psychol. Learn. Mem. Cogn. 6, 174–215 (1980). | PubMed

    Article  CAS  Google Scholar 

  118. Gelman, R. & Williams, E. M. in Handbook of Child Psychology, Volume II: Cognition, Perception and Development (eds Kuhn, D. & Siegler, R.) 575–630 (John Wiley and Sons, New York, 1997).

    Google Scholar 

  119. Plaut, D. C. Double dissociation without modularity: evidence from connectionist neuropsychology. J. Clin. Exp. Neuropsychol. 17, 291–321 (1995).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Preparation of this article was supported by the National Institute of Mental Health (USA) and the Medical Research Council (UK).

Author information

Authors and Affiliations

  1. Center for the Neural Basis of Cognition and Department of Psychology, Carnegie Mellon University, 4400 Fifth Avenue, Pittsburgh, 15213-2683, Pennsylvania, USA

    James L. McClelland

  2. Medical Research Council Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge, CB2 2EF, UK

    Timothy T. Rogers

Authors
  1. James L. McClelland
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy T. Rogers
    View author publications

    You can also search for this author in PubMed Google Scholar

Glossary

SYLLOGISM

A formal structure for deduction in argument, consisting of a major and a minor premise from which a conclusion logically follows.

SEMANTIC KNOWLEDGE

Knowledge about objects and their properties, and of relationships between and among them, including knowledge of word meanings. General encyclopaedic knowledge is sometimes also included.

SEMANTIC DEMENTIA

A degenerative neuropathological condition that results in the progressive loss of semantic knowledge as revealed through naming, description and non-verbal tests of semantic knowledge, resulting from disease of the anterior and lateral aspects of the temporal lobes.

PERCEPTUAL-TO-CONCEPTUAL SHIFT

A hypothesized developmental transition whereby infants initially categorize objects on the basis of their directly perceived visual properties, but later come to categorize them on the basis of deeper relationships.

TAXONOMIC HIERARCHY

A structured set of concepts linked together with class-inclusion relationships.

PARALLEL DISTRIBUTED PROCESSING

(PDP). A computational modelling framework in which cognitive and other mental processes arise from the interactions of simple, neuron-like units through their weighted connections. PDP models are a subset of connectionist or artificial neural network models that use distributed representations (a scheme in which the representation of an item is distributed as a pattern of activity across a pool of units also used for the representation of other items) and that treat any act of information processing as involving the simultaneous participation of many units.

INTERNAL REPRESENTATION

In a PDP network, a pattern of activity that arises across a layer of hidden units. When a network is presented with a given input, the pattern of activity arising across its hidden layer is the internal representation of that input.

HIDDEN UNITS

Units in a neural network that mediate the propagation of activity between input and output layers. The activations or target values of such units are not specified by the environment, but instead arise from the application of a learning procedure that sets the connection weights into and out of the unit.

FEEDFORWARD NETWORK

A class of neural networks wherein activation propagates only in one direction, from a set of inputs towards a set of output units, possibly through one or more layers of hidden units.

CATASTROPHIC INTERFERENCE

The loss of information previously stored in a PDP network that can occur as a result of later learning. Reducing overlap among representations or ensuring that learning is very gradual and interleaved with ongoing exposure to material already known are two ways of avoiding this problem.

BASIC LEVEL

The level of a taxonomic hierarchy at which normal participants typically identify a given object. For most concepts, the basic level is at an intermediate level of specificity, such as bird rather than animal or canary. So, when shown a photograph of a canary, people will be more likely to identify it as a bird then as a canary or an animal.

COHERENT COVARIATION

Consistent co-occurrence of a set of properties across different objects. The concept is distinct from simple correlation in that it generally refers to the co-occurrence of more than two properties. For example, having wings, having feathers, having hollow bones and being able to fly all consistently co-occur in birds.

THEORY THEORY

A class of theories that take as their main premise the proposition that human cognition is underpinned by naive domain theories. Under this view, naive theories help the learner to determine which concepts are good ones, and which properties are important for determining conceptual relations among objects; and conceptual development is likened to the process of theory change in science.

DOMAIN THEORY

Knowledge of the causal explanations that are appropriate to a particular kind of object. For example, gravity and momentum are relevant for inanimate objects, whereas beliefs and desires are relevant for human beings (and might be extended to other animals by young children).

CENTRAL PROPERTIES

Properties of an object that are understood to be most important for determining what kind of thing it is.

CAUSAL PROPERTIES

The properties of objects that give rise to predictable outcomes in event sequences in which the object is observed to participate. For example, when pressing a button on the remote control consistently precedes the TV turning on, the remote can be said to have the causal property of turning on the TV.

Rights and permissions

Reprints and permissions

About this article

Cite this article

McClelland, J., Rogers, T. The parallel distributed processing approach to semantic cognition. Nat Rev Neurosci 4, 310–322 (2003). https://doi.org/10.1038/nrn1076

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn1076

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing