Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

BigNeuron: A resource to benchmark and predict best-performing algorithms for automated reconstruction of neuronal morphology

View ORCID ProfileLinus Manubens-Gil, Zhi Zhou, Hanbo Chen, Arvind Ramanathan, Xiaoxiao Liu, Yufeng Liu, Alessandro Bria, Todd Gillette, Zongcai Ruan, Jian Yang, Miroslav Radojević, Ting Zhao, Li Cheng, Lei Qu, Siqi Liu, Kristofer E. Bouchard, Lin Gu, Weidong Cai, Shuiwang Ji, Badrinath Roysam, Ching-Wei Wang, Hongchuan Yu, Amos Sironi, Daniel Maxim Iascone, Jie Zhou, Erhan Bas, Eduardo Conde-Sousa, Paulo Aguiar, Xiang Li, Yujie Li, Sumit Nanda, Yuan Wang, Leila Muresan, Pascal Fua, Bing Ye, Hai-yan He, Jochen F. Staiger, Manuel Peter, Daniel N. Cox, Michel Simonneau, Marcel Oberlaender, Gregory Jefferis, Kei Ito, Paloma Gonzalez-Bellido, Jinhyun Kim, Edwin Rubel, Hollis T. Cline, Hongkui Zeng, Aljoscha Nern, Ann-Shyn Chiang, Jianhua Yao, Jane Roskams, Rick Livesey, Janine Stevens, Tianming Liu, Chinh Dang, Yike Guo, Ning Zhong, Georgia Tourassi, Sean Hill, Michael Hawrylycz, Christof Koch, Erik Meijering, Giorgio A. Ascoli, Hanchuan Peng
doi: https://doi.org/10.1101/2022.05.10.491406
Linus Manubens-Gil
1Institute for Brain and Intelligence, Southeast University, Nanjing, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Linus Manubens-Gil
Zhi Zhou
2Microsoft Corporation, Redmond, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hanbo Chen
3Tencent AI Lab, Bellevue, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Arvind Ramanathan
4Computing, Environment & Life Sciences Directorate, Argonne National Laboratory, Lemont, IL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiaoxiao Liu
5Kaya Medical, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yufeng Liu
1Institute for Brain and Intelligence, Southeast University, Nanjing, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alessandro Bria
6University of Cassino and Southern Lazio, Cassino, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Todd Gillette
7Center for Neural Informatics, Structures, & Plasticity; Krasnow Institute for Advanced Study; George Mason University, Fairfax, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zongcai Ruan
1Institute for Brain and Intelligence, Southeast University, Nanjing, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jian Yang
8Faculty of Information Technology, Beijing University of Technology, Beijing, China
9Beijing International Collaboration Base on Brain Informatics and Wisdom Services, Beijing, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Miroslav Radojević
10Nuctech Netherlands, Rotterdam, the Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ting Zhao
11Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Li Cheng
12Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lei Qu
1Institute for Brain and Intelligence, Southeast University, Nanjing, China
13Ministry of Education Key Laboratory of Intelligent Computation & Signal Processing, Anhui University, Hefei, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Siqi Liu
14Paige AI, New York, NY, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kristofer E. Bouchard
15Scientific Data Division and Biological Systems & Engineering Division, Lawrence Berkeley National Lab, Berkeley, CA, USA
16Helen Wills Neuroscience Institute and Redwood Center for Theoretical Neuroscience, UC Berkeley, Berkeley, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lin Gu
17RIKEN AIP, Tokyo, Japan
18Research Center for Advanced Science and Technology (RCAST), The University of Tokyo, Tokyo, Japan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Weidong Cai
19School of Computer Science, University of Sydney, Sydney, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Shuiwang Ji
20Texas A&M University, College Station, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Badrinath Roysam
21Cullen College of Engineering, University of Houston, Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ching-Wei Wang
22Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hongchuan Yu
23National Centre for Computer Animation, Bournemouth University, Poole, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Amos Sironi
24PROPHESEE, Paris, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Daniel Maxim Iascone
25Department of Neuroscience, Columbia University, New York, NY, USA
26Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jie Zhou
27Department of Computer Science, Northern Illinois University, DeKalb, IL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Erhan Bas
28AWS AI, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Eduardo Conde-Sousa
29i3S, Instituto de Investigação E Inovação Em Saúde, Universidade Do Porto, Porto, Portugal
30INEB, Instituto de Engenharia Biomédica, Universidade Do Porto, Porto, Portugal
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paulo Aguiar
29i3S, Instituto de Investigação E Inovação Em Saúde, Universidade Do Porto, Porto, Portugal
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiang Li
31Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yujie Li
32Allen Institute for Brain Science, Seattle, WA, USA
55Cortical Architecture Imaging and Discovery Lab, Department of Computer Science and Bioimaging Research Center, The University of Georgia, Athens, GA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sumit Nanda
7Center for Neural Informatics, Structures, & Plasticity; Krasnow Institute for Advanced Study; George Mason University, Fairfax, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yuan Wang
33Program in Neuroscience, Department of Biomedical Sciences, Florida State University College of Medicine, Tallahassee, FL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Leila Muresan
34Cambridge Advanced Imaging Centre, University of Cambridge, Cambridge, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Pascal Fua
35Computer Vision Laboratory, EPFL, Lausanne, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bing Ye
36Life Sciences Institute and Department of Cell and Developmental Biology, University of Michigan, Ann Arbor, MI, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hai-yan He
37Department of Biology, Georgetown University, Washington, DC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jochen F. Staiger
38Institute for Neuroanatomy, University Medical Center Göttingen, Georg-August-University Göttingen, Goettingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Manuel Peter
39Department of Stem Cell and Regenerative Biology & Center for Brain Science, Harvard University, Cambridge, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Daniel N. Cox
40Neuroscience Institute, Georgia State University, Atlanta, GA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michel Simonneau
41Université Paris-Saclay, CNRS, ENS Paris-Saclay, CentraleSupélec, LuMIn, Gif-sur-Yvette, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marcel Oberlaender
42Max Planck Group: In Silico Brain Sciences, Max Planck Institute for Neurobiology of Behavior – caesar, Bonn, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gregory Jefferis
43Division of Neurobiology, MRC Laboratory of Molecular Biology, Cambridge, United Kingdom
44Department of Zoology, University of Cambridge, Cambridge, United Kingdom
11Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kei Ito
11Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
45Institute for Quantitative Biosciences, University of Tokyo, Tokyo, Japan
46Institute of Zoology, Biocenter Cologne, University of Cologne, Cologne, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paloma Gonzalez-Bellido
47Department of Ecology, Evolution, and Behavior, University of Minnesota, St. Paul, MN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jinhyun Kim
48Brain Science Institute, Korea Institute of Science and Technology (KIST), Seoul, Korea
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Edwin Rubel
49Virginia Merrill Bloedel Hearing Research Center, University of Washington, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hollis T. Cline
50The Scripps Research Institute, San Diego, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hongkui Zeng
32Allen Institute for Brain Science, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Aljoscha Nern
11Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ann-Shyn Chiang
51Brain Research Center, National Tsing Hua University, Hsinchu, Taiwan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jianhua Yao
52Tencent AI Lab, Shenzhen, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jane Roskams
53Department of Zoology, Life Sciences Institute, University of British Columbia, Vancouver, Canada
32Allen Institute for Brain Science, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rick Livesey
54UCL Great Ormond Street Institute of Child Health, Zayed Centre for Rare Disease Research, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Janine Stevens
11Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tianming Liu
55Cortical Architecture Imaging and Discovery Lab, Department of Computer Science and Bioimaging Research Center, The University of Georgia, Athens, GA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Chinh Dang
50The Scripps Research Institute, San Diego, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yike Guo
56Data Science Institute, Imperial College London, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ning Zhong
8Faculty of Information Technology, Beijing University of Technology, Beijing, China
9Beijing International Collaboration Base on Brain Informatics and Wisdom Services, Beijing, China
57Department of Life Science and Informatics, Maebashi Institute of Technology, Maebashi, Japan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Georgia Tourassi
58BSEC, ORNL, Oak Ridge, TN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sean Hill
59Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada
60Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
61Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, Ontario, Canada
62Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Hawrylycz
32Allen Institute for Brain Science, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christof Koch
32Allen Institute for Brain Science, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Erik Meijering
63School of Computer Science and Engineering, University of New South Wales, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: h@braintell.org ascoli@gmu.edu meijering@imagescience.org
Giorgio A. Ascoli
7Center for Neural Informatics, Structures, & Plasticity; Krasnow Institute for Advanced Study; George Mason University, Fairfax, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: h@braintell.org ascoli@gmu.edu meijering@imagescience.org
Hanchuan Peng
1Institute for Brain and Intelligence, Southeast University, Nanjing, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: h@braintell.org ascoli@gmu.edu meijering@imagescience.org
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

BigNeuron is an open community bench-testing platform combining the expertise of neuroscientists and computer scientists toward the goal of setting open standards for accurate and fast automatic neuron reconstruction. The project gathered a diverse set of image volumes across several species representative of the data obtained in most neuroscience laboratories interested in neuron reconstruction. Here we report generated gold standard manual annotations for a selected subset of the available imaging datasets and quantified reconstruction quality for 35 automatic reconstruction algorithms. Together with image quality features, the data were pooled in an interactive web application that allows users and developers to perform principal component analysis t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and reconstruction data, and benchmarking of automatic reconstruction algorithms in user-defined data subsets. Our results show that image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. By benchmarking automatic reconstruction algorithms, we observed that diverse algorithms can provide complementary information toward obtaining accurate results and developed a novel algorithm to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms. Finally, to aid users in predicting the most accurate automatic reconstruction results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic reconstructions.

Competing Interest Statement

The authors have declared no competing interest.

Footnotes

  • https://linusmg.shinyapps.io/BigNeuron_Gold166/

  • https://neuroxiv.net/bigneuron/

  • https://github.com/BigNeuron/BigNeuron-Wiki/wiki

  • https://github.com/lmanubens/BigNeuron

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license.
Back to top
PreviousNext
Posted May 11, 2022.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
BigNeuron: A resource to benchmark and predict best-performing algorithms for automated reconstruction of neuronal morphology
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
BigNeuron: A resource to benchmark and predict best-performing algorithms for automated reconstruction of neuronal morphology
Linus Manubens-Gil, Zhi Zhou, Hanbo Chen, Arvind Ramanathan, Xiaoxiao Liu, Yufeng Liu, Alessandro Bria, Todd Gillette, Zongcai Ruan, Jian Yang, Miroslav Radojević, Ting Zhao, Li Cheng, Lei Qu, Siqi Liu, Kristofer E. Bouchard, Lin Gu, Weidong Cai, Shuiwang Ji, Badrinath Roysam, Ching-Wei Wang, Hongchuan Yu, Amos Sironi, Daniel Maxim Iascone, Jie Zhou, Erhan Bas, Eduardo Conde-Sousa, Paulo Aguiar, Xiang Li, Yujie Li, Sumit Nanda, Yuan Wang, Leila Muresan, Pascal Fua, Bing Ye, Hai-yan He, Jochen F. Staiger, Manuel Peter, Daniel N. Cox, Michel Simonneau, Marcel Oberlaender, Gregory Jefferis, Kei Ito, Paloma Gonzalez-Bellido, Jinhyun Kim, Edwin Rubel, Hollis T. Cline, Hongkui Zeng, Aljoscha Nern, Ann-Shyn Chiang, Jianhua Yao, Jane Roskams, Rick Livesey, Janine Stevens, Tianming Liu, Chinh Dang, Yike Guo, Ning Zhong, Georgia Tourassi, Sean Hill, Michael Hawrylycz, Christof Koch, Erik Meijering, Giorgio A. Ascoli, Hanchuan Peng
bioRxiv 2022.05.10.491406; doi: https://doi.org/10.1101/2022.05.10.491406
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
BigNeuron: A resource to benchmark and predict best-performing algorithms for automated reconstruction of neuronal morphology
Linus Manubens-Gil, Zhi Zhou, Hanbo Chen, Arvind Ramanathan, Xiaoxiao Liu, Yufeng Liu, Alessandro Bria, Todd Gillette, Zongcai Ruan, Jian Yang, Miroslav Radojević, Ting Zhao, Li Cheng, Lei Qu, Siqi Liu, Kristofer E. Bouchard, Lin Gu, Weidong Cai, Shuiwang Ji, Badrinath Roysam, Ching-Wei Wang, Hongchuan Yu, Amos Sironi, Daniel Maxim Iascone, Jie Zhou, Erhan Bas, Eduardo Conde-Sousa, Paulo Aguiar, Xiang Li, Yujie Li, Sumit Nanda, Yuan Wang, Leila Muresan, Pascal Fua, Bing Ye, Hai-yan He, Jochen F. Staiger, Manuel Peter, Daniel N. Cox, Michel Simonneau, Marcel Oberlaender, Gregory Jefferis, Kei Ito, Paloma Gonzalez-Bellido, Jinhyun Kim, Edwin Rubel, Hollis T. Cline, Hongkui Zeng, Aljoscha Nern, Ann-Shyn Chiang, Jianhua Yao, Jane Roskams, Rick Livesey, Janine Stevens, Tianming Liu, Chinh Dang, Yike Guo, Ning Zhong, Georgia Tourassi, Sean Hill, Michael Hawrylycz, Christof Koch, Erik Meijering, Giorgio A. Ascoli, Hanchuan Peng
bioRxiv 2022.05.10.491406; doi: https://doi.org/10.1101/2022.05.10.491406

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Bioinformatics
Subject Areas
All Articles
  • Animal Behavior and Cognition (3514)
  • Biochemistry (7367)
  • Bioengineering (5346)
  • Bioinformatics (20324)
  • Biophysics (10045)
  • Cancer Biology (7776)
  • Cell Biology (11352)
  • Clinical Trials (138)
  • Developmental Biology (6453)
  • Ecology (9980)
  • Epidemiology (2065)
  • Evolutionary Biology (13356)
  • Genetics (9373)
  • Genomics (12612)
  • Immunology (7725)
  • Microbiology (19103)
  • Molecular Biology (7465)
  • Neuroscience (41153)
  • Paleontology (301)
  • Pathology (1235)
  • Pharmacology and Toxicology (2142)
  • Physiology (3178)
  • Plant Biology (6880)
  • Scientific Communication and Education (1276)
  • Synthetic Biology (1900)
  • Systems Biology (5328)
  • Zoology (1091)