Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
Confirmatory Results

Deep learning: Using machine learning to study biological vision

View ORCID ProfileNajib J. Majaj, View ORCID ProfileDenis G. Pelli
doi: https://doi.org/10.1101/178152
Najib J. Majaj
1Center for Neural Science, New York University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Najib J. Majaj
Denis G. Pelli
1Center for Neural Science, New York University
2Department of Psychology, New York University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Denis G. Pelli
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

ABSTRACT

Today many vision-science presentations employ machine learning, especially the version called “deep learning”. Many neuroscientists use machine learning to decode neural responses. Many perception scientists try to understand how living organisms recognize objects. To them, deep neural networks offer benchmark accuracies for recognition of learned stimuli. Originally machine learning was inspired by the brain. Today, machine learning is used as a statistical tool to decode brain activity. Tomorrow, deep neural networks might become our best model of brain function. This brief overview of the use of machine learning in biological vision touches on its strengths, weaknesses, milestones, controversies, and current directions. Here, we hope to help vision scientists assess what role machine learning should play in their research.

  • GLOSSARY

    Machine learning
    is any computer algorithm that learns how to perform a task directly from examples, without a human providing explicit instructions or rules for how to do so. In one type of machine learning, called “supervised learning,” correctly labeled examples are provided to the learning algorithm, which is then “trained” (i.e. its parameters are adjusted) to be able to perform the task correctly on its own and generalize to unseen examples.
    Deep learning
    is a newly successful and popular version of machine learning that uses backprop (defined below) neural networks with multiple hidden layers. The 2012 success of AlexNet, then the best machine learning network for object recognition, was the tipping point. Deep learning is now ubiquitous in the internet. The idea is to have each layer of processing perform successively more complex computations on the data to give the full “multi-layer” network more expressive power. The drawback is that it is much harder to train multi-layer networks (Goodfellow et al. 2016). Deep learning ranges from discovering the weights of a multilayer network to parameter learning in hierarchical belief networks.
    Neural nets
    are computing systems inspired by biological neural networks that consist of individual neurons learning their connections with other neurons in order to solve tasks by considering examples.
    Supervised learning
    refers to any algorithm that accepts a set of labeled stimuli — a training set — and returns a classifier that can label stimuli similar to those in the training set.
    Unsupervised learning
    discovers structure and redundancy in data without labels. It is less widely used than supervised learning, but of great interest because labeled data are scarce while unlabeled data are plentiful.
    Cost function.
    A function that assigns a real number representing cost to a candidate solution by measuring the difference between the solution and the desired output. Solving by optimization means minimizing cost.
    Gradient descent:
    An algorithm that minimizes cost by incrementally changing the parameters in the direction of steepest descent of the cost function.
    Convexity:
    A real-valued function is called “convex” if the line segment between any two points on the graph of the function lies on or above the graph (Boyd & Vandenberghe, 2004). A problem is convex if its cost function is convex. Convexity guarantees that gradient descent will always find the global minimum.
    Generalization
    is how well a classifier performs on new, unseen examples that it did not see during training.
    Cross validation
    assesses the ability of the network to generalize, from the data that it trained on, to new data.
    Backprop,
    short for “backward propagation of errors”, is widely used to apply gradient-descent learning to multi-layer networks. It uses the chain rule from calculus to iteratively compute the gradient of the cost function for each layer.
    Hebbian learning
    and spike-timing-dependent plasticity (STDP). According to Hebb’s rule, the efficiency of a synapse increases after correlated pre- and post-synaptic activity. In other words, neurons that fire together, wire together (Löwel & Singer, 1992).
    Support Vector Machine (SVM)
    is a type of machine learning algorithm for classification. SVMs generalize well. An SVM uses the “kernel trick” to quickly learn to perform a nonlinear classification by finding a boundary in multidimensional space that separates different classes and maximizes the distance of class exemplars to the boundary (Cortes & Vapnik, 1999).
    Convolutional neural networks (ConvNets)
    have their roots in the Neocognitron (Fukushima 1980) and are inspired by the simple and complex cells described by Hubel and Wiesel (1962). ConvNets apply backprop learning to multilayer neural networks based on convolution and pooling (LeCun et al., 1989; LeCun et al., 1990; LeCun et al., 1998).
  • Copyright 
    The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.
    Back to top
    PreviousNext
    Posted August 20, 2018.
    Download PDF
    Email

    Thank you for your interest in spreading the word about bioRxiv.

    NOTE: Your email address is requested solely to identify you as the sender of this article.

    Enter multiple addresses on separate lines or separate them with commas.
    Deep learning: Using machine learning to study biological vision
    (Your Name) has forwarded a page to you from bioRxiv
    (Your Name) thought you would like to see this page from the bioRxiv website.
    CAPTCHA
    This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
    Share
    Deep learning: Using machine learning to study biological vision
    Najib J. Majaj, Denis G. Pelli
    bioRxiv 178152; doi: https://doi.org/10.1101/178152
    Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
    Citation Tools
    Deep learning: Using machine learning to study biological vision
    Najib J. Majaj, Denis G. Pelli
    bioRxiv 178152; doi: https://doi.org/10.1101/178152

    Citation Manager Formats

    • BibTeX
    • Bookends
    • EasyBib
    • EndNote (tagged)
    • EndNote 8 (xml)
    • Medlars
    • Mendeley
    • Papers
    • RefWorks Tagged
    • Ref Manager
    • RIS
    • Zotero
    • Tweet Widget
    • Facebook Like
    • Google Plus One

    Subject Area

    • Neuroscience
    Subject Areas
    All Articles
    • Animal Behavior and Cognition (3686)
    • Biochemistry (7774)
    • Bioengineering (5668)
    • Bioinformatics (21245)
    • Biophysics (10563)
    • Cancer Biology (8162)
    • Cell Biology (11915)
    • Clinical Trials (138)
    • Developmental Biology (6738)
    • Ecology (10388)
    • Epidemiology (2065)
    • Evolutionary Biology (13843)
    • Genetics (9694)
    • Genomics (13056)
    • Immunology (8123)
    • Microbiology (19956)
    • Molecular Biology (7833)
    • Neuroscience (42973)
    • Paleontology (318)
    • Pathology (1276)
    • Pharmacology and Toxicology (2256)
    • Physiology (3350)
    • Plant Biology (7208)
    • Scientific Communication and Education (1309)
    • Synthetic Biology (1999)
    • Systems Biology (5528)
    • Zoology (1126)