Skip to main content
SpringerLink
Log in

Degenerate coding in neural systems

  • Review
  • Published:
Journal of Comparative Physiology A Aims and scope Submit manuscript

Abstract

When the dimensionality of a neural circuit is substantially larger than the dimensionality of the variable it encodes, many different degenerate network states can produce the same output. In this review I will discuss three different neural systems that are linked by this theme. The pyloric network of the lobster, the song control system of the zebra finch, and the odor encoding system of the locust, while different in design, all contain degeneracies between their internal parameters and the outputs they encode. Indeed, although the dynamics of song generation and odor identification are quite different, computationally, odor recognition can be thought of as running the song generation circuitry backwards. In both of these systems, degeneracy plays a vital role in mapping a sparse neural representation devoid of correlations onto external stimuli (odors or song structure) that are strongly correlated. I argue that degeneracy between input and output states is an inherent feature of many neural systems, which can be exploited as a fault-tolerant method of reliably learning, generating, and discriminating closely related patterns.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. More generally, if an n-dimensional linear system does not have n distinct eigenvalues, the system is said to be degenerate and contains multiple eigenvectors that are not linearly independent.

  2. A variety of terms have been used to describe these population activity patterns—neural assemblies, neural ensembles, population activity vectors, etc. Within the context of this review, state vectors are a more natural terminology to describe the dimensionality and dynamics of neural circuits.

  3. The development of more rigorous methods of identifying degeneracy is important (see Tononi et al. 1999, for an example), but at present the correlation method has been used as a reasonable first-pass at the problem.

  4. For example, the model networks had a bias in the relative strength of fast and slow synapses in a way that favored the correct pyloric burst order, as is seen in experimental data.

  5. Nonlinear syringeal dynamics can cause abrupt (< 1 ms) sound changes not directly controlled by the bird. However, for longer time scales (> ∼1 ms), there is generally a close relation between vocal muscle activity and acoustic structure.

  6. Assuming that over the course of the 1 s song, all the HVC neurons are on once.

  7. Note that a linear weighting is not strictly necessary for degeneracy; any projection of the high-dimensional RA state space (∼8000 neurons) onto the much lower-dimensional syringeal muscle state space (∼7 muscles) can produce a degenerate mapping between RA states and sound.

  8. Note that each point in the song correlation matrix refers to two sounds in the song, and the corresponding points in the neural state correlation matrix refer to the two neural states that generated those sounds [after a latency correlation for the delay between RA activation and sound production; see Leonardo and Fee (2004) for analytical details].

  9. While there are nonlinearities in the dynamics of the syrinx (Fee et al. 1998) this indicates that similar RA states can produce different sounds. It does not imply the converse (different RA states will produce similar sounds) that is the relevant variable for the present analysis.

  10. RA is not the only source of degeneracy in the song production pathway—it has been shown that very different patterns of vocal muscle activity can generate the same types of sounds (Suthers and Hartley 1996).

  11. The connection to mammalian olfaction, however, is less clear than to other vertebrate systems like the zebrafish.

  12. The mushroom body is thought to be functionally similar to the piriform cortex in mammals.

  13. KC odor selectivity is presumably even lower than 1/17, since only a limited number of odors can be tested in an experiment.

  14. There are roughly 1000 glomeruli in the locust antennal lobe.

  15. In many species, such as flies and moth, each ORN projects to a single glomerulus, the identity of which depends on the specific olfactory receptor it is expressing (Buck 1996). In locust, ORNs project to multiple glomeruli, such that there are more glomeruli than olfactory receptor types.

  16. The LFP is present only during odor-presentation, its frequency is dependent on the PN-LN dynamics and not on the odor identity.

  17. If chemically similar odors produced uncorrelated ORN patterns, PNs would simply inherit the degeneracy in the odor to ORN mapping, rather than generating it.

  18. After the 200 ms of computation leading to the onset of decorrelation.

  19. Individual RA neurons are connected to only a few syringeal muscles; similarly, PNs receive input from only a few types of ORNs.

  20. However, see Barlow (2001) for a critical reinterpretation of this original idea.

Abbreviations

LFP:

local field potential

LN:

local neuron

PN:

projection neuron

ORN:

olfactory receptor neuron

PVA:

population vector average

RA:

robust nucleus of the arcopallium

STG:

stomatogastric ganglion

References

  • Barlow H (1961) Possible principles underlying the transformation of sensory messages. MIT Press, Cambridge, MA

    Google Scholar 

  • Barlow H (2001) Redundancy reduction revisited. Network: Computation in neural systems 12:241–253

    Article  CAS  Google Scholar 

  • Bazhenov M, Stopfer M, Rabinovich M, Abarbanel D, Sejnowski T, Laurent G (2001) Model of cellular and network mechanisms for odor-evoked temporal patterning in the locust antennal lobe. Neuron 30:569–581

    Article  PubMed  CAS  Google Scholar 

  • de Bruyne M, Foster K, Carlson J (2001) Odor coding in the drosophila antenna. Neuron 30:537–552

    Article  PubMed  Google Scholar 

  • Bucher A, Prinz A, Marder E (2005) Animal-to-animal variability in motor pattern production in adults and during growth. J Neurosci 25:1611–1619

    Article  PubMed  CAS  Google Scholar 

  • Buck L (1996) Information coding in the vertebrate olfactory system. Ann Rev Neurosci 19:517–544

    Article  PubMed  CAS  Google Scholar 

  • Chi Z, Margoliash D (2001) Temporal precision and temporal drift in brain and behavior of zebra finch song. Neuron 32:899–910

    Article  PubMed  CAS  Google Scholar 

  • Coleman M, Vu E (2005) Recovery of impaired songs following unilateral but not bilateral lesions of nucleus uvaeformis of adult zebra finches. J Neurobiol 63:71–89

    Article  Google Scholar 

  • Edelman G, Gally J (2001) Degeneracy and complexity in biological systems. Proc Nat Acad Sci 98:13763–13768

    Article  PubMed  CAS  Google Scholar 

  • Fee M, Shraiman B, Pesaran B, Mitra P (1998) The role of nonlinear dynamics of the syrinx in birdsong production. Nature 395:67–71

    Article  PubMed  CAS  Google Scholar 

  • Fee M, Kozhevnikov A, Hahnloser R (2004) Neural mechanisms of vocal sequence generation in the songbird. Ann NY Acad Sci 1016:153–170

    Article  PubMed  Google Scholar 

  • Fetz EE (1992) Are movement parameters recognizably coded in the activity of single neurons? Behav Brain Sci 15:679–690

    Google Scholar 

  • Fiete I, Hahnloser R, Fee M, Seung H (2004) Temporal sparseness of the premotor drive is important for rapid learning in a neural network model of birdsong. J Neurophys 92:2274–2282

    Article  Google Scholar 

  • Fitch H, Kupin J, Kessler I, DeLucia J (2003) Relating articulation and acoustics through a sinusoidal description of vocal tract shape. Speech Commun 39:243–268

    Article  Google Scholar 

  • Friedrich R, Korsching S (1997) Combinatorial and chemotopic odorant coding in the zebrafish olfactory bulb visualized by optical imaging. Neuron 18:737–752

    Article  PubMed  CAS  Google Scholar 

  • Friedrich R, Laurent G (2001) Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity. Science 291:889–894

    Article  PubMed  CAS  Google Scholar 

  • Friedrich R, Habermann C, Laurent G (2004) Multiplexing using synchrony in the zebrafish olfactory bulb. Nat Neurosci 7:862–871

    Article  PubMed  CAS  Google Scholar 

  • Gardner T, Cecchi G, Magnasco M, Laje R, Mindlin GB (2001) Simple gestures for birdsongs. Phys Rev Lett 87:208101

    Article  PubMed  CAS  Google Scholar 

  • Georgopoulos A, Schwartz A, Kettner R (1986) Neuronal population coding of movement direction. Science 233:1416–1419

    Article  PubMed  CAS  Google Scholar 

  • Goldman M, Golowasch J, Marder E, Abbott L (2001) Global structure, robustness, and modulation of neuronal models. J Neurosci 21:5229–5238

    PubMed  CAS  Google Scholar 

  • Goller F, Cooper B (2004) Peripheral motor dynamics of song production in the zebra finch. Ann NY Acad Sci 1016:130–152

    Article  PubMed  Google Scholar 

  • Goller F, Suthers R (1996) Role of syringeal muscles in controlling the phonology of bird song. J Neurophysiol 76:287–300

    PubMed  CAS  Google Scholar 

  • Golowasch J, Abbott L, Marder E (1999) Activity-dependent regulation of ionic currents in an identified neuron of the stomatogastric ganglion of the crab cancer borealis. J Neurosci 19:RC33

    PubMed  CAS  Google Scholar 

  • Greenewalt C (1968) Bird song: acoustics and physiology. Smithonian Institution Press, Washington DC

    Google Scholar 

  • Guckenheimer J, Gueron S, Harris-Warrick R (1993) Mapping the dynamics of a bursting neuron. Philos Trans R Soc Lond B Biol Sci 341:354–359

    Google Scholar 

  • Gurney ME (1981) Hormonal control of cell form and number in the zebra finch song system. J Neurosci 1:658–673

    PubMed  CAS  Google Scholar 

  • Hahnloser RHR, Kozhevnikov AA, Fee MS (2002) An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature 419:65–70

    Article  PubMed  CAS  Google Scholar 

  • Harris-Warrick R, Marder E, Selverston A, Mouline M (1992) Dynamic biological networks: the stomatogastric nervous system. MIT, Cambridge, MA

    Google Scholar 

  • Heisenberg M, Borst A, Wagner S, Byers D (1985) Drosophila mushroom body mutants are deficient in olfactory learning. J Neuro-genet 2:1–30

    CAS  Google Scholar 

  • Hildebrand J, Shepherd G (1997) Mechanisms of olfactory discrimination: converging evidence for common principles across phyla. Ann Rev Neurosci 20:595–631

    Article  PubMed  CAS  Google Scholar 

  • Immelmann K (1969) Song development in the zebra finch and other estrilid finches. In: Hinde RA (ed) Bird Vocalizations. Cambridge University Press, New York, pp 61–74

    Google Scholar 

  • Kanerva P (1988) Sparse distributed memory. MIT Press, Cambridge

    Google Scholar 

  • Kao M, Doupe A, Brainard M (2005) Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433:638–643

    Article  PubMed  CAS  Google Scholar 

  • Konishi M (1965) The role of auditory feedback in the control of vocalization in the white crown sparrow. Z Tierpsychol 22:770–783

    PubMed  CAS  Google Scholar 

  • Konishi M (1991) Deciphering the brain’s codes. Neural Comput 3:1–18

    Article  Google Scholar 

  • Laurent G (2002) Olfactory network dynamics and the coding of multidimensional signals. Nat Rev Neurosci 3:884–895

    Article  PubMed  CAS  Google Scholar 

  • Laurent G, Davidowitz H (1994) Encoding of olfactory information with oscillating neural assemblies. Science 265:1872–1875

    Article  PubMed  Google Scholar 

  • Laurent G, Stopfer M, Friedrich R, Rabinovich M, Volkovskii A, Abarbanel H (2001) Odor encoding as an active, dynamical process: experiments, computation and theory. Ann Rev Neurosci 24:263–297

    Article  PubMed  CAS  Google Scholar 

  • Lee D, Seung H (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791

    Article  PubMed  CAS  Google Scholar 

  • Leonardo A (2002) Neural dynamics underlying complex behavior in a songbird. Caltech, PhD thesis, Pasadena

  • Leonardo A (2004) Experimental test of the birdsong error-correction model. Proc Nat Acad Sci 101:16935–16940

    Article  PubMed  CAS  Google Scholar 

  • Leonardo A, Fee M (2005) Ensemble coding of vocal control in a songbird. J Neurosci 25:652–661

    Article  PubMed  CAS  Google Scholar 

  • Leonardo A, Konishi M (1999) Decrystallization of adult birdsong by perturbation of auditory feedback. Nature 399:466–470

    Article  PubMed  CAS  Google Scholar 

  • Lewis J, Kristan W (1998) A neuronal network for computing population vectors in the leech. Nature 391:76–79

    Article  PubMed  CAS  Google Scholar 

  • Lohr B, Dooling R (1998) Detection of changes in timbre and harmonicity in complex sounds by zebra finches and budgerigars. J Comp Psych 112:36–47

    Article  CAS  Google Scholar 

  • MacLean J, Zhang Y, Johnson B, Harris-Warrick R (2003) Activity-independent homeostasis in rhythmically active neurons. Neuron 37:109–120

    Article  PubMed  CAS  Google Scholar 

  • MacLeod K, Laurent G (1996) Distinct mechanisms for synchronizing and temporal patterning of odor-encoding neural assemblies. Science 274:976–979

    Article  PubMed  CAS  Google Scholar 

  • Malnic B, Hirono J, Sato T, Buck L (1999) Combinatorial receptor codes for odors. Cell 96:713–723

    Article  PubMed  CAS  Google Scholar 

  • Marr D (1969) A theory of cerebellar cortex. J Physiol (Lond) 202:437–470

    CAS  Google Scholar 

  • Mindlin GB, Gardner TJ, Goller F, Suthers R (2003) Experimental support for a model of birdsong production. Phys Rev E 68

  • Ngai J, Chess A, Dowling M, Axel R (1993) Expression of odorant receptor genes in the catfish olfactory epithelium. Chem Senses 18:209–216

    Article  CAS  Google Scholar 

  • Nordeen K, Nordeen E (1992) Auditory feedback is necessary for the maintenance of stereotyped song in adult zebra finches. Behav Neural Biol 57:58–66

    Article  PubMed  CAS  Google Scholar 

  • Nottebohm F, Stokes T, Leonard C (1976) Central control of song in the canary. J Comp Neurology 165:457–486

    Article  CAS  Google Scholar 

  • Olshausen B, Field D (1997) Sparse coding with an overcomplete basis set: a strategy imployed by V1?. Vis Res 37:3311–3325

    Article  PubMed  CAS  Google Scholar 

  • Olveczky B, Andalman A, Fee M (2005) Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLOS (in press)

  • Perez-Orive J, Mazor O, Turner G, Cassenaer S, Wilson R, GL (2002) Oscillations and sparsening of odor representations in the mushroom body. Science 297:359–365

  • Perez-Orive J, Bazhenov M, Laurent G (2004) Intrinsic and circuit properties favor coincidence detection for decoding oscillatory input. J Neurosci 24:6037–6047

    Article  PubMed  CAS  Google Scholar 

  • Petersen R, Panzeri S, Diamond M (2001) Population coding of stimulus location in rat somatosensory cortex. Neuron 32

  • Prinz A, Bucher A, Marder E (2004) Similar network activity from disparate circuit parameters. Nat Neurosci 7:1345–1352

    Article  PubMed  CAS  Google Scholar 

  • Reich D, Mechler F, Victor J (2001) Independent and redundant information in nearby cortical neurons. Science 294:2566–2568

    Article  PubMed  CAS  Google Scholar 

  • Reichmann M, Markham R, Symons R, Rees M (1962) Experimental evidence for degeneracy of nucleotide triplet code. Nature 195:999

    Article  PubMed  Google Scholar 

  • Reiner A, Perkel D, Bruce L, Butler A, Csillag A, Kuenzel W, Medina L, Paxinos G, Shimizu T, Streidter G, Wild M, Ball G, Durand S, Guturkun O, Lee D, Mello C, Powers A, White S, Hough G, Kubikova L, Smulders T, Wada K, Dugas-Ford J, Husband S, Yamaoto K, Yu K, Saing C, Jarvis E (2004) Revised nomenclature for avian telencephalon and some related brainstem nuclei. J Comp Neurol 473:377–414

    Article  PubMed  Google Scholar 

  • Schneidman E, Bialek W, Berry M (2003) Synergy, redundancy, and independence in population codes. J Neurosci 23:11539–11553

    PubMed  CAS  Google Scholar 

  • Spiro JE, Dalva MB, Mooney R (1999) Long-range inhibition within the zebra finch song nucleus RA can coordinate the firing of multiple projection neurons. J Neurophysiol 81:3007–3020

    PubMed  CAS  Google Scholar 

  • Stopfer M, Laurent G (1999) Short-term memory in olfactory dynamics. Nature 402:664–668

    Article  PubMed  CAS  Google Scholar 

  • Stopfer M, Bhagavan S, Smith B, Laurent G (1997) Impaired odor discrimination on desynchronization of odor-encoding assemblies. Nature 390:70–74

    Article  PubMed  CAS  Google Scholar 

  • Stopfer M, Jayaraman V, Laurent G (2003) Intensity versus identity coding in an olfactory system. Neuron 39:991–1004

    Article  PubMed  CAS  Google Scholar 

  • Strotmann J, Wanner I, Helfrich T, Beck A, Meinken C, Kubick S, Breer H (1994) Olfactory neurons expressing distinct odorant receptor subtypes are spatially segregated in the nasal neuroepithelium. Cell Tissue Res 276:429–438

    PubMed  CAS  Google Scholar 

  • Sturdy CB, Wild JM, Mooney R (2003) Respiratory and telencephalic modulation of vocal motor neurons in the zebra finch. J Neurosci 23:1072–1086

    PubMed  CAS  Google Scholar 

  • Suthers R, Hartley R (1996) Motor stereotypy and diversity in songs of mimic thrushes. J Neurobiol 30:231–245

    Article  PubMed  CAS  Google Scholar 

  • Tchernichovski O, Nottebohm F, Ho C, Pesaran B, Mitra P (2000) A procedure for an automated measurement of song similarity. Anim Behav 59:1167–1176

    Article  PubMed  Google Scholar 

  • Todorov E (2004) Optimality principles in sensorimotor control. Nat Neurosci 7:907–915

    Article  PubMed  CAS  Google Scholar 

  • Tononi G, Sporns O, Edelman G (1999) Measures of degeneracy and redundancy in biological networks. Proc Nat Acad Sci 96:3257–3262

    Article  PubMed  CAS  Google Scholar 

  • Turrigiano G, Abbott L, Marder E (1994) Actiivty-depedent changes in the intrinsic properties of cultured neurons. Science 264:974–977

    Article  PubMed  CAS  Google Scholar 

  • Vicario DS (1991) Organization of the zebra finch song control system: II. Functional organization of outputs from nucleus Robustus archistriatalis. J Comp Neurol 309:486–494

    Article  PubMed  CAS  Google Scholar 

  • Vu ET, Mazurek ME, Kuo Y (1994) Identification of a forebrain motor programming network for the learned song of zebra finches. J Neurosci 14:6924–6934

    PubMed  CAS  Google Scholar 

  • Wehr M, Laurent G (1996) Odor encoding by temporal sequences of firing in oscillating neural assemblies. Nature 384:162–166

    Article  PubMed  CAS  Google Scholar 

  • Wild JM (2004) Functional neuroanatomy of the sensorimotor control of singing. Ann NY Acad Sci 1016:438–462

    Article  PubMed  Google Scholar 

  • Wild J, Goller F, Suthers R (1998) Inspiratory muscle activity during bird song. J Neurobiol 36:441–453

    Article  PubMed  CAS  Google Scholar 

  • Yu AC, Margoliash D (1996) Temporal hierarchical control of singing in birds. Science 273:1871–1875

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgements

The ideas in this paper have benefited from discussions with Gilles Laurent and Michale Fee. In addition, I thank Roian Egnor, Mark Konishi and Astrid Prinz for comments on the manuscript. Anthony Leonardo received support from the Helen Hay Whitney Foundation.

Author information

Authors and Affiliations

  1. Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA, 02138,  

    Anthony Leonardo

Authors
  1. Anthony Leonardo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anthony Leonardo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Leonardo, A. Degenerate coding in neural systems. J Comp Physiol A 191, 995–1010 (2005). https://doi.org/10.1007/s00359-005-0026-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00359-005-0026-0

Keywords

Search

Navigation