Abstract
An analytical method is introduced for evaluating the performance of neural encoding models. The method addresses a critical question that arises during the course of the development and validation of encoding models: is a given model near optimal in terms of its accuracy in predicting the stimulus-elicited responses of a neural system, or can the predictive accuracy be improved significantly by further model development? The evaluation method is based on a derivation of the minimum mean-square error between actual responses and modeled responses. It is formulated as a comparison between the mean-square error of the candidate model and the theoretical minimum mean-square error attainable through an optimal model for the system. However, no a priori information about the nature of the optimal model is required. The theoretically minimum error is determined solely from the coherence function between pairs of system responses to repeated presentations of the same dynamic stimulus. Thus, the performance of the candidate model is judged against the performance of an optimal model rather than against that of an arbitrarily assumed model.
Using this method, we evaluated a linear model for neural encoding by mechanosensory cells in the cricket cercal system. At low stimulus intensities, the best-fit linear model of encoding by single cells was found to be nearly optimal, even though the coherence between stimulus-response pairs (a commonly used measure of system linearity) was low. In this low-stimulus-intensity regime, the mean square error of the linear model was on the order of the power of the cell responses. In contrast, at higher stimulus intensities the linear model was not an accurate representation of neural encoding, even though the stimulus-response coherence was substantially higher than in the low-intensity regime.
Similar content being viewed by others
References
Atick J (1992) Could information theory provide an ecological theory of sensory processing? Network: Comput. in Neural Systems 3:213–251.
Bennet-Clark HC (1984) A particle velocity microphone for the song of small insects and other acoustic measurements. J. Exp. Biol. 108:459–463.
Bialek W, Rieke F, de Ruyter van Steveninck RR, Warland D (1991) Reading a neural code. Science 252:1854–1857.
Buño W, Bustamante J, Fuentes J (1984) White noise analysis of pace-maker response interactions and nonlinearities in slowly adapting crayfish stretch receptor. J. Physiol. 350:55–80.
Clague H, Theunissen F, Miller JP (1997) The Effects of Adaptation on Neural Coding by Primary Sensory Interneurons in the Cricket Cercal System. J. Neurophysiol. 77:207–220.
Cover TM, Thomas JA (1991) Elements of Information Theory. Wiley, New York.
DeCharms RC, Merzenich MM (1996) Primary cortical representation of sounds by the coordination of action potential timing. Nature 381:610–613.
de Ruyter van Steveninck RR, Lewen GD, Strong SP, Koberle R, Bialek W (1997) Reproducibility and variability in neural spike trains. Science 275:1805–1808.
Eggermont JJ (1993) Wiener and Volterra analyses applied to the auditory system. Hearing Research 66:177–201.
French A, Holden AV, Stein RB (1972) The estimation of the frequency response function of a mechanoreceptor. Kybernetic 11:15–23.
Gawne TJ, Richmond BJ, Optican LM (1991) Interactive effects among several stimulus parameters on the responses of striate cortical complex cells. J. Neurophysiol. 66:379–389.
Girish B (1997) How good is your model? Criteria for assessing adequacy of linear and nonlinear models for noisy systems. Master's thesis, University of California, Berkeley.
Gray CM, Engel AK, Koenig P, Singer W (1992) Synchronization of oscillatory neuronal responses in cat striate cortex: Temporal properties. Visual Neurosci. 8:337–347.
Haag J, Borst A (1997) Encoding of visual motion information and reliability in spiking and graded potential neurons. J. Neurosci. 17:4809–4819.
Kämper G, Kleindienst HU (1992) Oscillation of cricket sensory hairs in a low frequency sound field. J. Comp. Physiol. A. 167:193–199.
Kondoh Y, Arima T, Okuma J, Hasegawa Y (1991) Filter characteristics of cercal afferents in the cockroach. J. Comp. Physiol. A. 169:653–662.
Landolfa MA, Miller JP (1995) Stimulus-response properties of cricket cercal filiform receptors. J. Comp. Physiol. A. 177:749–757.
Ma SK (1981) Calculation of entropy from data of motion. J. Stat. Phys. 26:221–240.
Naka KI, Marmarelis PZ, Chan RY (1975) Morphological and functional identifications of catfish retinal neurons. III. Functional identifications. J. Neurophysiol. 33:625–642.
Osborne L (1996) Signal processing in a mechanosensory array: Dynamics of cricket cercal hairs. Doctoral thesis, University of California, Berkeley.
O'Shea M, Adams ME (1981) Pentapetide (proctolin) associated with an identified neuron. Science 213:576–569.
Panton RL (1995) Incompressible Flow (2nd ed.). Wiley, New York.
Rieke F (1992) Physical principles underlying sensory processing and computation. Doctoral thesis, University of California, Berkeley.
Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W (1996) Spikes: Exploring the Neural Code. MIT Press, Boston.
Roddey JC (1995) Stimulus encoding in the cricket cercal sensory system. Doctoral thesis, University of California, Berkeley.
Roddey JC, Jacobs GA (1996) Information theoretic analysis of dynamical encoding by filiform mechanoreceptors in the cricket cercal system. J. Neurophysiol. 75:1365–1376.
Ruderman DL, Bialek W (1994) Statistics of natural images: Scaling in the woods. Phys. Rev. Let. 73:814–817.
Sperkeijse H (1969) Rectification in the goldfish retina: Analysis by sinusoidal and auxiliary stimulation. Vision Research 9:1461–1472.
Strong SP, Koberle R, de Ruyter van Steveninck RR, Bialek W (1997) Entropy and information in neural spike trains. Preprint available as condmat/9603127 from the physics e-print archive (http://xxx.lanl.gov/archive/cond-mat).
Srinivasan MV, Laughlin SB, Dubs A (1982) Predictive coding: A fresh view of inhibition in the retina. Proc. R. Soc. Cond. B216:427–459.
Theunissen FE, Roddey JC, Stufflebeam S, Clague H, Miller JP (1996) Information theoretic analysis of dynamical encoding by four interneurons of the cricket cercal sensory system. J. Neurophysiol. 75:1345–1364.
Treves A (1997) On the perceptual structure of face space. Biosystems 40:189–196.
van Hateren JH (1992) Theoretical predictions of spatiotemporal receptive fields of fly LMC's and experimental validation. J. Comp. Physiol. A. 171:157–170.
Wehr M, Laurent G (1996) Odor encoding by temporal sequences of firing in oscillating neural assemblies. Nature 384:162–166.
Welch PD (1967) The use of the fast Fourier transform for the estimation of power spectra: A method based on time averaging over short modified periodograms. IEEE Trans. Audio Electroacoust. AU-15:70–73.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Roddey, J.C., Girish, B. & Miller, J.P. Assessing the Performance of Neural Encoding Models in the Presence of Noise. J Comput Neurosci 8, 95–112 (2000). https://doi.org/10.1023/A:1008921114108
Issue Date:
DOI: https://doi.org/10.1023/A:1008921114108