Elsevier

Magnetic Resonance Imaging

Volume 57, April 2019, Pages 194-209
Magnetic Resonance Imaging

Original Contribution
Challenges in diffusion MRI tractography – Lessons learned from international benchmark competitions

https://doi.org/10.1016/j.mri.2018.11.014Get rights and content

Abstract

Diffusion MRI (dMRI) fiber tractography has become a pillar of the neuroimaging community due to its ability to noninvasively map the structural connectivity of the brain. Despite widespread use in clinical and research domains, these methods suffer from several potential drawbacks or limitations. Thus, validating the accuracy and reproducibility of techniques is critical for sound scientific conclusions and effective clinical outcomes. Towards this end, a number of international benchmark competitions, or “challenges”, has been organized by the diffusion MRI community in order to investigate the reliability of the tractography process by providing a platform to compare algorithms and results in a fair manner, and evaluate common and emerging algorithms in an effort to advance the state of the field. In this paper, we summarize the lessons from a decade of challenges in tractography, and give perspective on the past, present, and future “challenges” that the field of diffusion tractography faces.

Introduction

Diffusion magnetic resonance imaging (dMRI) fiber tractography [1,2] has become a pillar of the neuroimaging community, revealing fundamental insights into how connectivity underlies brain function, development, and cognition, and leading to a better understanding of brain dysfunction in aging, mental health disorders, and neurological disease [3]. Additionally, in the neurosurgical setting, tractography has provided clinically relevant information during pre-operative planning as well as intra-operative mapping of brain tumor resections [4]. Despite its widespread use in both clinical and research domains, the process from data acquisition to generation of 3D connectivity maps is a multi-step procedure with numerous assumptions and uncertainties that can ultimately affect the ability of tractography to faithfully represent the true axonal connections of the brain. Because of this, validating the accuracy and reproducibility of these techniques is critical for sound scientific conclusions and effective surgical outcomes. It is necessary to measure the ability of these techniques to track white matter fibers from region to region, and to also quantify the ability of dMRI to assesses the underlying fiber orientation distribution (FOD) within each voxel. Towards this end, there have been a large number of dMRI community-wide efforts, or “challenges”, which aim to investigate the reliability of the tractography process.

Publicly organized challenges are widespread in biomedical image analysis. In this domain, an algorithm or solution may be developed to address a particular challenge in the field - a challenge that is likely being, or has already been, tackled by multiple laboratories, researchers, and algorithms. However, for many problems, there is no public database and reference standard available, and results are typically reported on proprietary datasets which may vary widely due to differences in acquisition and hardware, making fair comparisons between algorithms not practical. For this reason, public challenges are organized to provide a platform to compare algorithms and results in a fair manner, and evaluate common and emerging algorithms in an effort to advance the state of the field. In biomedical image analysis, challenges have included a number of imaging modalities, anatomies, and evaluation goals. Examples include segmenting tumors and lesions from MR, CT, and PET scans [[5], [6], [7]], detecting pulmonary nodules in chest CT [8], particle tracking [9], evaluating neuronal reconstructions [10,11], estimating knee cartilage [12], motion correction on cardiac data [13], segmenting histological images [14], and segmenting a number of organs [[15], [16], [17], [18]]. A comprehensive list of past and ongoing challenges can be found at https://grand-challenge.org/all_challenges/.

In the tractography community, these challenges provide a unique opportunity for neuroscientists, computer scientists, biomedical engineers, and MRI physicists to fairly compare tractography algorithms in an unbiased format. Traditionally, most have focused on either local modeling of the fiber geometry, or on the evaluation of tractography as a whole. These challenges have provided us with valuable lessons regarding the tractography process –resulting in quantitative measures of the reliability and limitations of existing approaches – and left us with unique opportunities for advancements in brain mapping using ideas and algorithms from different disciplines, research labs, and scientific communities. In this manuscript, we aim to summarize the lessons learned from a decade of challenges in tractography, and to give perspective on the past, present, and future “challenges” that the field of diffusion tractography faces.

We begin with a brief review of “The Challenges” that have provided insight into fiber orientation reconstruction, tractography, and brain connectivity. Next, we summarize the insights and “Lessons Learned” from these studies, and conclude with a discussion on the “Opportunities and Perspectives” on open issues in the field and the future of tractography.

Section snippets

The challenges

The basic anatomy of a challenge includes (1) defining the challenge itself, i.e. the task and desired output, (2) providing a set of images for participants to apply their algorithm on, (3) creating the reference standard, or ground truth, against which all submissions are compared, (4) defining an evaluation procedure, or metrics to quantify performance, and (5) final evaluation of all submissions. In this section, we present a brief history of past tractography challenges in chronological

Lessons learned

These challenges have resulted in a number of insights into the tractography process, its interpretations, strengths, and limitations. We summarize here the main lessons learned from these challenges.

Opportunities and perspectives

We end with a discussion on perspectives on the challenges and the future of tractography, opportunities for advancements, and open issues in the field.

Acknowledgments

This work was supported by the National Institutes of Health under award numbers R01EB017230, and T32EB001628, and by the National Science Foundation under award number 1452485. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or NSF.

References (90)

  • M.A. Cote et al.

    Tractometer: towards validation of tractography pipelines

    Med Image Anal

    (2013)
  • K.G. Schilling et al.

    Anatomical accuracy of standard-practice tractography algorithms in the motor system - a histological validation in the squirrel monkey brain

    Magn Reson Imaging

    (2019)
  • B. Jeurissen et al.

    Multi-tissue constrained spherical deconvolution for improved analysis of multi-shell diffusion MRI data

    Neuroimage

    (2014)
  • J.D. Tournier et al.

    Direct estimation of the fiber orientation density function from diffusion-weighted MRI data using spherical deconvolution

    Neuroimage

    (2004)
  • F. Dell'acqua et al.

    A modified damped Richardson-Lucy algorithm to reduce isotropic background effects in spherical deconvolution

    Neuroimage

    (2010)
  • J.D. Tournier et al.

    Resolving crossing fibres using constrained spherical deconvolution: validation using diffusion-weighted imaging phantom data

    Neuroimage

    (2008)
  • R.E. Smith et al.

    SIFT: spherical-deconvolution informed filtering of tractograms

    Neuroimage

    (2013)
  • M. Axer et al.

    A novel approach to the human connectome: ultra-high resolution mapping of fiber tracts in the brain

    Neuroimage

    (2011)
  • M. Guevara et al.

    Reproducibility of superficial white matter tracts using diffusion-weighted imaging tractography

    Neuroimage

    (2017)
  • D. Raffelt et al.

    Apparent Fibre Density: a novel measure for the analysis of diffusion-weighted magnetic resonance images

    Neuroimage

    (2012)
  • J. Xu et al.

    Mapping mean axon diameter and axonal volume fraction by MRI using temporal diffusion spectroscopy

    Neuroimage

    (2014)
  • S.N. Sotiropoulos et al.

    Ball and rackets: inferring fiber fanning from diffusion-weighted MRI

    Neuroimage

    (2012)
  • H. Zhang et al.

    NODDI: practical in vivo neurite orientation dispersion and density imaging of the human brain

    Neuroimage

    (2012)
  • D. Topgaard

    Multidimensional diffusion MRI

    J Magn Reson (San Diego, Calif: 1997)

    (2017)
  • C.F. Westin et al.

    Q-space trajectory imaging for multidimensional diffusion MRI of the human brain

    Neuroimage

    (2016)
  • R. Schurr et al.

    Tractography optimization using quantitative T1 mapping in the human optic radiation

    Neuroimage

    (2018)
  • D.C. Van Essen et al.

    The Human Connectome Project: a data acquisition perspective

    Neuroimage

    (2012)
  • J. Wasserthal et al.

    TractSeg - fast and accurate white matter tract segmentation

    Neuroimage

    (2018)
  • P.F. Neher et al.

    Fiber tractography using machine learning

    Neuroimage

    (2017)
  • R. Xue et al.

    In vivo three-dimensional reconstruction of rat brain axonal projections by diffusion tensor imaging

    Magn Reson Med

    (1999)
  • T.E. Conturo et al.

    Tracking neuronal fiber pathways in the living human brain

    Proc Natl Acad Sci U S A

    (1999)
  • Warfield MSaJLaBCaMCaOCaHTaSM-PaVJaS

    3D segmentation in the clinic: a grand challenge II: MS lesion segmentation

  • B. Menze et al.

    The multimodal brain tumor image segmentation benchmark (BRATS)

    IEEE Trans Med Imaging

    (2014)
  • N. Chenouard et al.

    Objective comparison of particle tracking methods

    Nat Methods

    (2014)
  • K.M. Brown et al.

    The DIADEM data sets: representative light microscopy images of neuronal morphology to advance automation of digital reconstructions

    Neuroinformatics

    (2011)
  • Segmentation of knee images: a grand challenge

  • I. Arganda-Carreras et al.

    Crowdsourcing the creation of image segmentation algorithms for connectomics

    Front Neuroanat

    (2015)
  • P.F. Raudaschl et al.

    Evaluation of segmentation methods on head and neck CT: auto-segmentation challenge 2015

    Med Phys

    (2017)
  • O. Jimenez-Del-Toro et al.

    Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: VISCERAL anatomy benchmarks

    IEEE Trans Med Imaging

    (2016)
  • O. Bernard et al.

    Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?

    IEEE Trans Med Imaging

    (2018)
  • U. Ferizi et al.

    Diffusion MRI microstructure models with in vivo human brain Connectome data: results from a multi-group comparison

    NMR Biomed

    (2017)
  • C. Poupon et al.

    New diffusion phantoms dedicated to the study and validation of high-angular-resolution diffusion imaging (HARDI) models

    Magn Reson Med

    (2008)
  • S. Pujol et al.

    The DTI challenge: toward standardized evaluation of diffusion tensor imaging tractography for neurosurgery

    J Neuroimaging

    (2015)
  • M.J. McGirt et al.

    Association of surgically acquired motor and language deficits on overall survival after resection of glioblastoma multiforme

    Neurosurgery

    (2009)
  • A. Daducci et al.

    Quantitative comparison of reconstruction methods for intra-voxel fiber recovery from diffusion MRI

    IEEE Trans Med Imaging

    (2014)
  • Cited by (0)

    View full text