Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Transfer learning for biomedical named entity recognition with neural networks

View ORCID ProfileJohn M Giorgi, View ORCID ProfileGary D Bader
doi: https://doi.org/10.1101/262790
John M Giorgi
1Department of Computer Science, University of Toronto, 10 Kings College Road, Toronto, Canada M5S 3G4
2The Donnelly Centre, University of Toronto, 160 College Street, Toronto, Canada M5S 3E1
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for John M Giorgi
Gary D Bader
1Department of Computer Science, University of Toronto, 10 Kings College Road, Toronto, Canada M5S 3G4
2The Donnelly Centre, University of Toronto, 160 College Street, Toronto, Canada M5S 3E1
3Department of Molecular Genetics, University of Toronto, 1 King’s College Circle, Toronto ON M5S 1A8
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Gary D Bader
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Preview PDF
Loading

Abstract

Motivation The explosive increase of biomedical literature has made information extraction an increasingly important tool for biomedical research. A fundamental task is the recognition of biomedical named entities (NER) such as genes/proteins, diseases, and species. Recently, a domain-independent method based on deep learning and statistical word embeddings, called long short-term memory network-conditional random field (LSTM-CRF), has been shown to outperform state-of-the-art entity-specific NER tools. However, this method is dependent on gold-standard corpora (GSCs) consisting of hand-labeled entities, which tend to be small but highly reliable. An alternative to GSCs are silver-standard corpora (SSCs), which are generated by harmonizing the annotations made by several automatic annotation systems. SSCs typically contain much more noise than GSCs but have the advantage of containing many more training examples. Ideally, these corpora could be combined to achieve the benefits of both, which is an opportunity for transfer learning. In this work, we analyze to what extent transfer learning improves upon state-of-the-art results for biomedical NER. We also attempt to identify the scenarios where transfer learning offers the biggest advantages.

Results We demonstrate that transferring a deep neural network (DNN) trained on a large, noisy SSC to a smaller, but more reliable GSC improves upon state-of-the-art results for biomedical NER. Compared to a state-of-the-art baseline evaluated on 17 GSCs covering four different entity classes, transfer learning results in an average reduction in error of approximately 9%. We found transfer learning to be especially beneficial for target data sets with a small number of labels (approximately 5000 or less).

Availability and implementation Source code for the LSTM-CRF is available at https://github.com/Franck-Dernoncourt/NeuroNER/ and links to the corpora are available at https://github.com/BaderLab/Transfer-Learning-BNER-Bioinformatics-2018/.

Contact johnmg{at}cs.toronto.edu

Copyright 
The copyright holder has placed this preprint in the Public Domain. It is no longer restricted by copyright. Anyone can legally share, reuse, remix, or adapt this material for any purpose without crediting the original authors.
Back to top
PreviousNext
Posted February 12, 2018.
Download PDF

Supplementary Material

Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Transfer learning for biomedical named entity recognition with neural networks
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Transfer learning for biomedical named entity recognition with neural networks
John M Giorgi, Gary D Bader
bioRxiv 262790; doi: https://doi.org/10.1101/262790
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Transfer learning for biomedical named entity recognition with neural networks
John M Giorgi, Gary D Bader
bioRxiv 262790; doi: https://doi.org/10.1101/262790

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Bioinformatics
Subject Areas
All Articles
  • Animal Behavior and Cognition (3514)
  • Biochemistry (7361)
  • Bioengineering (5341)
  • Bioinformatics (20317)
  • Biophysics (10041)
  • Cancer Biology (7773)
  • Cell Biology (11348)
  • Clinical Trials (138)
  • Developmental Biology (6448)
  • Ecology (9979)
  • Epidemiology (2065)
  • Evolutionary Biology (13354)
  • Genetics (9370)
  • Genomics (12607)
  • Immunology (7724)
  • Microbiology (19087)
  • Molecular Biology (7459)
  • Neuroscience (41132)
  • Paleontology (300)
  • Pathology (1235)
  • Pharmacology and Toxicology (2142)
  • Physiology (3176)
  • Plant Biology (6878)
  • Scientific Communication and Education (1276)
  • Synthetic Biology (1900)
  • Systems Biology (5328)
  • Zoology (1091)