Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Variability in the analysis of a single neuroimaging dataset by many teams

View ORCID ProfileRotem Botvinik-Nezer, View ORCID ProfileFelix Holzmeister, Colin F. Camerer, Anna Dreber, Juergen Huber, Magnus Johannesson, Michael Kirchler, Roni Iwanir, Jeanette A. Mumford, Alison Adcock, Paolo Avesani, Blazej Baczkowski, Aahana Bajracharya, Leah Bakst, Sheryl Ball, Marco Barilari, Nadège Bault, Derek Beaton, Julia Beitner, Roland Benoit, Ruud Berkers, Jamil Bhanji, Bharat Biswal, Sebastian Bobadilla-Suarez, Tiago Bortolini, Katherine Bottenhorn, Alexander Bowring, Senne Braem, Hayley Brooks, Emily Brudner, Cristian Calderon, Julia Camilleri, Jaime Castrellon, Luca Cecchetti, Edna Cieslik, Zachary Cole, Olivier Collignon, Robert Cox, William Cunningham, Stefan Czoschke, Kamalaker Dadi, Charles Davis, Alberto De Luca, Mauricio Delgado, Lysia Demetriou, Jeffrey Dennison, Xin Di, Erin Dickie, Ekaterina Dobryakova, Claire Donnat, Juergen Dukart, Niall W. Duncan, Joke Durnez, Amr Eed, Simon Eickhoff, Andrew Erhart, Laura Fontanesi, G. Matthew Fricke, Adriana Galvan, Remi Gau, Sarah Genon, Tristan Glatard, Enrico Glerean, Jelle Goeman, Sergej Golowin, Carlos González-García, Krzysztof Gorgolewski, Cheryl Grady, Mikella Green, João Guassi Moreira, Olivia Guest, Shabnam Hakimi, J. Paul Hamilton, Roeland Hancock, Giacomo Handjaras, Bronson Harry, Colin Hawco, Peer Herholz, Gabrielle Herman, Stephan Heunis, Felix Hoffstaedter, Jeremy Hogeveen, Susan Holmes, Chuan-Peng Hu, Scott Huettel, Matthew Hughes, Vittorio Iacovella, Alexandru Iordan, Peder Isager, Ayse Ilkay Isik, Andrew Jahn, Matthew Johnson, Tom Johnstone, Michael Joseph, Anthony Juliano, Joseph Kable, Michalis Kassinopoulos, Cemal Koba, Xiang-Zhen Kong, Timothy Koscik, Nuri Erkut Kucukboyaci, Brice Kuhl, Sebastian Kupek, Angela Laird, Claus Lamm, Robert Langner, Nina Lauharatanahirun, Hongmi Lee, Sangil Lee, Alexander Leemans, Andrea Leo, Elise Lesage, Flora Li, Monica Li, Phui Cheng Lim, Evan Lintz, Schuyler Liphardt, Annabel Losecaat Vermeer, Bradley Love, Michael Mack, Norberto Malpica, Theo Marins, Camille Maumet, Kelsey McDonald, Joseph McGuire, Helena Melero, Adriana Méndez Leal, Benjamin Meyer, Kristin Meyer, Paul Mihai, Georgios Mitsis, Jorge Moll, Dylan Nielson, Gustav Nilsonne, Michael Notter, Emanuele Olivetti, Adrian Onicas, Paolo Papale, Kaustubh Patil, Jonathan E. Peelle, Alexandre Pérez, Doris Pischedda, Jean-Baptiste Poline, Yanina Prystauka, Shruti Ray, Patricia Reuter-Lorenz, Richard Reynolds, Emiliano Ricciardi, Jenny Rieck, Anais Rodriguez-Thompson, Anthony Romyn, Taylor Salo, Gregory Samanez-Larkin, Emilio Sanz-Morales, Margaret Schlichting, Douglas Schultz, Qiang Shen, Margaret Sheridan, Fu Shiguang, Jennifer Silvers, Kenny Skagerlund, Alec Smith, David Smith, Peter Sokol-Hessner, Simon Steinkamp, Sarah Tashjian, Bertrand Thirion, John Thorp, Gustav Tinghög, Loreen Tisdall, Steven Tompson, Claudio Toro-Serey, Juan Torre, Leonardo Tozzi, Vuong Truong, Luca Turella, Anna E. van’t Veer, Tom Verguts, Jean Vettel, Sagana Vijayarajah, Khoi Vo, Matthew Wall, Wouter D. Weeda, Susanne Weis, David White, David Wisniewski, Alba Xifra-Porxas, Emily Yearling, Sangsuk Yoon, Rui Yuan, Kenneth Yuen, Lei Zhang, Xu Zhang, Joshua Zosky, View ORCID ProfileThomas E. Nichols, View ORCID ProfileRussell A. Poldrack, View ORCID ProfileTom Schonberg
doi: https://doi.org/10.1101/843193
Rotem Botvinik-Nezer
Tel Aviv University, Israel
Dartmouth College, NH, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Rotem Botvinik-Nezer
Felix Holzmeister
University of Innsbruck, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Felix Holzmeister
Colin F. Camerer
Caltech, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anna Dreber
University of Innsbruck, Austria
Stockholm School of Economics, Sweden
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Juergen Huber
University of Innsbruck, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Magnus Johannesson
Stockholm School of Economics, Sweden
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Kirchler
University of Innsbruck, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Roni Iwanir
Tel Aviv University, Israel
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jeanette A. Mumford
University of Wisconsin - Madison, WI, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alison Adcock
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paolo Avesani
Fondazione Bruno Kessler, Italy
University of Trento, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Blazej Baczkowski
Max Planck Institute for; Human Cognitive and Brain Sciences, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Aahana Bajracharya
Washington University in St. Louis, MO, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Leah Bakst
Boston University, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sheryl Ball
Virginia Tech, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marco Barilari
Université Catholique de Louvain, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nadège Bault
University of Trento, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Derek Beaton
Rotman Research Institute, Baycrest Health Sciences, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Julia Beitner
University of Amsterdam, The Netherlands
Goethe University, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Roland Benoit
Max Planck Institute for; Human Cognitive and Brain Sciences, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ruud Berkers
Max Planck Institute for; Human Cognitive and Brain Sciences, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jamil Bhanji
Rutgers University-Newark, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bharat Biswal
New Jersey Institute of Technology, NJ, USA
University of Electronic Science and Technology of China, Sichuan, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sebastian Bobadilla-Suarez
University College London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tiago Bortolini
D’Or Institute for Research and Education, Brazil
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Katherine Bottenhorn
Florida International; University, FL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexander Bowring
University of Oxford, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Senne Braem
Vrije Universiteit Brussel, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hayley Brooks
University of Denver, CO, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emily Brudner
Rutgers University-Newark, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cristian Calderon
Ghent University, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Julia Camilleri
Forschungszentrum Jülich, Germany
Heinrich Heine University Dusseldorf, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jaime Castrellon
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Luca Cecchetti
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Edna Cieslik
Heinrich Heine University Dusseldorf, Germany
Research; Centre Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zachary Cole
University of Nebraska-Lincoln, NE, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Olivier Collignon
Université Catholique de Louvain, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Robert Cox
National Institute of Mental Health, MD, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
William Cunningham
University of Toronto, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stefan Czoschke
Goethe University, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kamalaker Dadi
Inria, Saclay, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Charles Davis
University of Connecticut, CT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alberto De Luca
University Medical Center Utrecht, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mauricio Delgado
Rutgers University-Newark, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lysia Demetriou
Imperial College London, United Kingdom
Invicro London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jeffrey Dennison
Temple University, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xin Di
New Jersey Institute of Technology, NJ, USA
University of Electronic Science and Technology of China, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Erin Dickie
University of Toronto, Canada
Centre for Addiction and Mental Health, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ekaterina Dobryakova
Kessler Foundation, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Claire Donnat
Stanford University, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Juergen Dukart
Heinrich Heine University Dusseldorf, Germany
Research; Centre Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Niall W. Duncan
Taipei Medical University, Taiwan
TMU-ShuangHo Hospital, Taiwan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joke Durnez
OpenLattice Inc, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Amr Eed
Instituto de Neurociencias, CSIC-UMH, Spain
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Simon Eickhoff
Heinrich Heine University Dusseldorf, Germany
Research; Centre Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrew Erhart
University of Denver, CO, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Laura Fontanesi
University of Basel, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
G. Matthew Fricke
The University of New Mexico, NM, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Adriana Galvan
University of California, Los Angeles, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Remi Gau
Crossmodal Perception and Plasticity laboratory, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sarah Genon
Research; Centre Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tristan Glatard
Concordia University, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Enrico Glerean
Aalto University, Finland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jelle Goeman
Leiden University Medical Center, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sergej Golowin
Taipei Medical University, Taiwan
TMU-ShuangHo Hospital, Taiwan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carlos González-García
Ghent University, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Krzysztof Gorgolewski
Stanford University, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cheryl Grady
Rotman Research Institute, Baycrest Health Sciences, Canada
University of Toronto, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mikella Green
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
João Guassi Moreira
University of California, Los Angeles, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Olivia Guest
University College London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Shabnam Hakimi
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
J. Paul Hamilton
Linköping University, Sweden
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Roeland Hancock
University of Connecticut, CT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Giacomo Handjaras
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bronson Harry
Western Sydney University, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Colin Hawco
Centre for Addiction and Mental Health, Canada
Neuranalysis Consulting, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peer Herholz
McGill University, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gabrielle Herman
Centre for Addiction and Mental Health, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephan Heunis
Eindhoven University of Technology, The Netherlands
Kempenhaeghe Epilepsy Center, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Felix Hoffstaedter
Forschungszentrum Jülich, Germany
Heinrich Heine University Dusseldorf, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jeremy Hogeveen
University of New Mexico, NM, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Susan Holmes
Stanford University, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Chuan-Peng Hu
Johannes Gutenberg University Medical Center, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Scott Huettel
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthew Hughes
Swinburne University of Technology, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Vittorio Iacovella
The University of Trento, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexandru Iordan
University of Michigan, MI, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peder Isager
Eindhoven University of Technology, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ayse Ilkay Isik
Max Planck Institute for Empirical Aesthetics, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrew Jahn
University of Michigan, MI, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthew Johnson
University of Nebraska-Lincoln, NE, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tom Johnstone
Swinburne University of Technology, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Joseph
Centre for Addiction and Mental Health, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anthony Juliano
Kessler Foundation, NJ, USA
Childrens Specialized Hospital Research Center, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joseph Kable
University of Pennsylvania, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michalis Kassinopoulos
McGill University, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cemal Koba
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiang-Zhen Kong
Max Planck Institute for; Psycholinguistics, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Timothy Koscik
University of Iowa Hospitals and Clinics, IA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nuri Erkut Kucukboyaci
Kessler Foundation, NJ, USA
Rutgers University, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Brice Kuhl
University of Oregon, OR, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sebastian Kupek
University of Innsbruck, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Angela Laird
Florida International; University, FL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Claus Lamm
University of Vienna, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Robert Langner
Heinrich Heine University Dusseldorf, Germany
Research; Centre Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nina Lauharatanahirun
University of Pennsylvania, PA, USA
US Combat Capabilities Development Command Army Research Laboratory, MD, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hongmi Lee
Johns Hopkins University, MD, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sangil Lee
University of Pennsylvania, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexander Leemans
University Medical Center Utrecht, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrea Leo
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elise Lesage
Ghent University, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Flora Li
Fralin Biomedical Research Institute at VTC, VA, USA
Nanjing Audit University, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Monica Li
University of Connecticut, CT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Phui Cheng Lim
University of Nebraska-Lincoln, NE, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Evan Lintz
University of Nebraska-Lincoln, NE, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Schuyler Liphardt
University of New Mexico, NM, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Annabel Losecaat Vermeer
University of Vienna, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bradley Love
University College London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Mack
University of Toronto, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Norberto Malpica
Universidad Rey Juan Carlos, Spain
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Theo Marins
D’Or Institute for Research and Education, Brazil
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Camille Maumet
Inria, Rennes, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kelsey McDonald
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joseph McGuire
Boston University, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Helena Melero
Universidad Rey Juan Carlos, Spain
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Adriana Méndez Leal
University of California, Los Angeles, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Benjamin Meyer
Johannes Gutenberg University Medical Center, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kristin Meyer
University of North Carolina at Chapel Hill, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul Mihai
Max Planck Institute for; Human Cognitive and Brain Sciences, Germany
Technische Universit‰t Dresden, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Georgios Mitsis
McGill University, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jorge Moll
D’Or Institute for Research and Education, Brazil
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Dylan Nielson
National Institute of Mental Health, MD, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gustav Nilsonne
Karolinska Institutet, Sweden
Stockholm University, Sweden
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Notter
Lausanne University Hospital, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emanuele Olivetti
Fondazione Bruno Kessler, Italy
University of Trento, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Adrian Onicas
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paolo Papale
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kaustubh Patil
Forschungszentrum Jülich, Germany
Heinrich Heine University Dusseldorf, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jonathan E. Peelle
Washington University in St. Louis, MO, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexandre Pérez
McGill University, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Doris Pischedda
University of Trento, Italy
University of Milano-Bicocca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jean-Baptiste Poline
McGill University, Canada
University of California, Berkeley, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yanina Prystauka
University of Connecticut, CT, USA
The Connecticut Institute for the Brain and Cognitive Sciences, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Shruti Ray
New Jersey Institute of Technology, NJ, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Patricia Reuter-Lorenz
University of Michigan, MI, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Richard Reynolds
National Institute of Mental Health, MD, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emiliano Ricciardi
IMT School for Advanced Studies Lucca, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jenny Rieck
Rotman Research Institute, Baycrest Health Sciences, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anais Rodriguez-Thompson
University of North Carolina at Chapel Hill, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anthony Romyn
University of Toronto, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Taylor Salo
Florida International; University, FL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gregory Samanez-Larkin
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emilio Sanz-Morales
Medical Image Analysis Lab, URJC, Spain
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Margaret Schlichting
University of Toronto, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Douglas Schultz
University of Nebraska-Lincoln, NE, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Qiang Shen
School of Management, Zhejiang University of Technology, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Margaret Sheridan
University of North Carolina at Chapel Hill, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Fu Shiguang
School of Management, Zhejiang University of Technology, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jennifer Silvers
University of California, Los Angeles, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kenny Skagerlund
Linköping University, Sweden
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alec Smith
Virginia Tech, VA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Smith
Temple University, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter Sokol-Hessner
University of Denver, CO, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Simon Steinkamp
Research; Centre Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sarah Tashjian
University of California, Los Angeles, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bertrand Thirion
Inria, Saclay, France
Neurospin, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John Thorp
Columbia University, NY, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gustav Tinghög
Linköping University, Sweden
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Loreen Tisdall
University of Basel, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Steven Tompson
US Combat Capabilities Development Command Army Research Laboratory, MD, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Claudio Toro-Serey
Boston University, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Juan Torre
Inria, Saclay, France
Neurospin, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Leonardo Tozzi
Stanford University, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Vuong Truong
Taipei Medical University, Taiwan
TMU-ShuangHo Hospital, Taiwan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Luca Turella
The University of Trento, Italy
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anna E. van’t Veer
Leiden University, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tom Verguts
Ghent University, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jean Vettel
US Combat Capabilities Development Command Army Research Laboratory, MD, USA
University of California, Santa Barbara, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sagana Vijayarajah
University of Toronto, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Khoi Vo
Duke; University, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthew Wall
Imperial College London, United Kingdom
Invicro LLC, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Wouter D. Weeda
Leiden University, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Susanne Weis
Heinrich Heine University Dusseldorf, Germany
Research Centre Julich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David White
Swinburne University of Technology, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Wisniewski
Ghent University, Belgium
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alba Xifra-Porxas
McGill University, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emily Yearling
University of Connecticut, CT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sangsuk Yoon
University of Dayton, OH, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rui Yuan
Stanford University, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kenneth Yuen
Johannes Gutenberg University Medical Center & Deutsches Resilienz Zentrum (DRZ), Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lei Zhang
University of Vienna, Austria
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xu Zhang
University of Connecticut, CT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joshua Zosky
University of Nebraska-Lincoln, NE, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Thomas E. Nichols
University of Oxford, United Kingdom
University of Warwick, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Thomas E. Nichols
  • For correspondence: schonberg@tauex.tau.ac.il
Russell A. Poldrack
Stanford University, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Russell A. Poldrack
  • For correspondence: schonberg@tauex.tau.ac.il
Tom Schonberg
Tel Aviv University, Israel
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Tom Schonberg
  • For correspondence: schonberg@tauex.tau.ac.il
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Preview PDF
Loading

Summary

Data analysis workflows in many scientific domains have become increasingly complex and flexible. To assess the impact of this flexibility on functional magnetic resonance imaging (fMRI) results, the same dataset was independently analyzed by 70 teams, testing nine ex-ante hypotheses. The flexibility of analytic approaches is exemplified by the fact that no two teams chose identical workflows to analyze the data. This flexibility resulted in sizeable variation in hypothesis test results, even for teams whose statistical maps were highly correlated at intermediate stages of their analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Importantly, meta-analytic approaches that aggregated information across teams yielded significant consensus in activated regions across teams. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset. Our findings show that analytic flexibility can have substantial effects on scientific conclusions, and demonstrate factors related to variability in fMRI. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for multiple analyses of the same data. Potential approaches to mitigate issues related to analytical variability are discussed.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted November 15, 2019.
Download PDF

Supplementary Material

Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Variability in the analysis of a single neuroimaging dataset by many teams
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
Share
Variability in the analysis of a single neuroimaging dataset by many teams
Rotem Botvinik-Nezer, Felix Holzmeister, Colin F. Camerer, Anna Dreber, Juergen Huber, Magnus Johannesson, Michael Kirchler, Roni Iwanir, Jeanette A. Mumford, Alison Adcock, Paolo Avesani, Blazej Baczkowski, Aahana Bajracharya, Leah Bakst, Sheryl Ball, Marco Barilari, Nadège Bault, Derek Beaton, Julia Beitner, Roland Benoit, Ruud Berkers, Jamil Bhanji, Bharat Biswal, Sebastian Bobadilla-Suarez, Tiago Bortolini, Katherine Bottenhorn, Alexander Bowring, Senne Braem, Hayley Brooks, Emily Brudner, Cristian Calderon, Julia Camilleri, Jaime Castrellon, Luca Cecchetti, Edna Cieslik, Zachary Cole, Olivier Collignon, Robert Cox, William Cunningham, Stefan Czoschke, Kamalaker Dadi, Charles Davis, Alberto De Luca, Mauricio Delgado, Lysia Demetriou, Jeffrey Dennison, Xin Di, Erin Dickie, Ekaterina Dobryakova, Claire Donnat, Juergen Dukart, Niall W. Duncan, Joke Durnez, Amr Eed, Simon Eickhoff, Andrew Erhart, Laura Fontanesi, G. Matthew Fricke, Adriana Galvan, Remi Gau, Sarah Genon, Tristan Glatard, Enrico Glerean, Jelle Goeman, Sergej Golowin, Carlos González-García, Krzysztof Gorgolewski, Cheryl Grady, Mikella Green, João Guassi Moreira, Olivia Guest, Shabnam Hakimi, J. Paul Hamilton, Roeland Hancock, Giacomo Handjaras, Bronson Harry, Colin Hawco, Peer Herholz, Gabrielle Herman, Stephan Heunis, Felix Hoffstaedter, Jeremy Hogeveen, Susan Holmes, Chuan-Peng Hu, Scott Huettel, Matthew Hughes, Vittorio Iacovella, Alexandru Iordan, Peder Isager, Ayse Ilkay Isik, Andrew Jahn, Matthew Johnson, Tom Johnstone, Michael Joseph, Anthony Juliano, Joseph Kable, Michalis Kassinopoulos, Cemal Koba, Xiang-Zhen Kong, Timothy Koscik, Nuri Erkut Kucukboyaci, Brice Kuhl, Sebastian Kupek, Angela Laird, Claus Lamm, Robert Langner, Nina Lauharatanahirun, Hongmi Lee, Sangil Lee, Alexander Leemans, Andrea Leo, Elise Lesage, Flora Li, Monica Li, Phui Cheng Lim, Evan Lintz, Schuyler Liphardt, Annabel Losecaat Vermeer, Bradley Love, Michael Mack, Norberto Malpica, Theo Marins, Camille Maumet, Kelsey McDonald, Joseph McGuire, Helena Melero, Adriana Méndez Leal, Benjamin Meyer, Kristin Meyer, Paul Mihai, Georgios Mitsis, Jorge Moll, Dylan Nielson, Gustav Nilsonne, Michael Notter, Emanuele Olivetti, Adrian Onicas, Paolo Papale, Kaustubh Patil, Jonathan E. Peelle, Alexandre Pérez, Doris Pischedda, Jean-Baptiste Poline, Yanina Prystauka, Shruti Ray, Patricia Reuter-Lorenz, Richard Reynolds, Emiliano Ricciardi, Jenny Rieck, Anais Rodriguez-Thompson, Anthony Romyn, Taylor Salo, Gregory Samanez-Larkin, Emilio Sanz-Morales, Margaret Schlichting, Douglas Schultz, Qiang Shen, Margaret Sheridan, Fu Shiguang, Jennifer Silvers, Kenny Skagerlund, Alec Smith, David Smith, Peter Sokol-Hessner, Simon Steinkamp, Sarah Tashjian, Bertrand Thirion, John Thorp, Gustav Tinghög, Loreen Tisdall, Steven Tompson, Claudio Toro-Serey, Juan Torre, Leonardo Tozzi, Vuong Truong, Luca Turella, Anna E. van’t Veer, Tom Verguts, Jean Vettel, Sagana Vijayarajah, Khoi Vo, Matthew Wall, Wouter D. Weeda, Susanne Weis, David White, David Wisniewski, Alba Xifra-Porxas, Emily Yearling, Sangsuk Yoon, Rui Yuan, Kenneth Yuen, Lei Zhang, Xu Zhang, Joshua Zosky, Thomas E. Nichols, Russell A. Poldrack, Tom Schonberg
bioRxiv 843193; doi: https://doi.org/10.1101/843193
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Variability in the analysis of a single neuroimaging dataset by many teams
Rotem Botvinik-Nezer, Felix Holzmeister, Colin F. Camerer, Anna Dreber, Juergen Huber, Magnus Johannesson, Michael Kirchler, Roni Iwanir, Jeanette A. Mumford, Alison Adcock, Paolo Avesani, Blazej Baczkowski, Aahana Bajracharya, Leah Bakst, Sheryl Ball, Marco Barilari, Nadège Bault, Derek Beaton, Julia Beitner, Roland Benoit, Ruud Berkers, Jamil Bhanji, Bharat Biswal, Sebastian Bobadilla-Suarez, Tiago Bortolini, Katherine Bottenhorn, Alexander Bowring, Senne Braem, Hayley Brooks, Emily Brudner, Cristian Calderon, Julia Camilleri, Jaime Castrellon, Luca Cecchetti, Edna Cieslik, Zachary Cole, Olivier Collignon, Robert Cox, William Cunningham, Stefan Czoschke, Kamalaker Dadi, Charles Davis, Alberto De Luca, Mauricio Delgado, Lysia Demetriou, Jeffrey Dennison, Xin Di, Erin Dickie, Ekaterina Dobryakova, Claire Donnat, Juergen Dukart, Niall W. Duncan, Joke Durnez, Amr Eed, Simon Eickhoff, Andrew Erhart, Laura Fontanesi, G. Matthew Fricke, Adriana Galvan, Remi Gau, Sarah Genon, Tristan Glatard, Enrico Glerean, Jelle Goeman, Sergej Golowin, Carlos González-García, Krzysztof Gorgolewski, Cheryl Grady, Mikella Green, João Guassi Moreira, Olivia Guest, Shabnam Hakimi, J. Paul Hamilton, Roeland Hancock, Giacomo Handjaras, Bronson Harry, Colin Hawco, Peer Herholz, Gabrielle Herman, Stephan Heunis, Felix Hoffstaedter, Jeremy Hogeveen, Susan Holmes, Chuan-Peng Hu, Scott Huettel, Matthew Hughes, Vittorio Iacovella, Alexandru Iordan, Peder Isager, Ayse Ilkay Isik, Andrew Jahn, Matthew Johnson, Tom Johnstone, Michael Joseph, Anthony Juliano, Joseph Kable, Michalis Kassinopoulos, Cemal Koba, Xiang-Zhen Kong, Timothy Koscik, Nuri Erkut Kucukboyaci, Brice Kuhl, Sebastian Kupek, Angela Laird, Claus Lamm, Robert Langner, Nina Lauharatanahirun, Hongmi Lee, Sangil Lee, Alexander Leemans, Andrea Leo, Elise Lesage, Flora Li, Monica Li, Phui Cheng Lim, Evan Lintz, Schuyler Liphardt, Annabel Losecaat Vermeer, Bradley Love, Michael Mack, Norberto Malpica, Theo Marins, Camille Maumet, Kelsey McDonald, Joseph McGuire, Helena Melero, Adriana Méndez Leal, Benjamin Meyer, Kristin Meyer, Paul Mihai, Georgios Mitsis, Jorge Moll, Dylan Nielson, Gustav Nilsonne, Michael Notter, Emanuele Olivetti, Adrian Onicas, Paolo Papale, Kaustubh Patil, Jonathan E. Peelle, Alexandre Pérez, Doris Pischedda, Jean-Baptiste Poline, Yanina Prystauka, Shruti Ray, Patricia Reuter-Lorenz, Richard Reynolds, Emiliano Ricciardi, Jenny Rieck, Anais Rodriguez-Thompson, Anthony Romyn, Taylor Salo, Gregory Samanez-Larkin, Emilio Sanz-Morales, Margaret Schlichting, Douglas Schultz, Qiang Shen, Margaret Sheridan, Fu Shiguang, Jennifer Silvers, Kenny Skagerlund, Alec Smith, David Smith, Peter Sokol-Hessner, Simon Steinkamp, Sarah Tashjian, Bertrand Thirion, John Thorp, Gustav Tinghög, Loreen Tisdall, Steven Tompson, Claudio Toro-Serey, Juan Torre, Leonardo Tozzi, Vuong Truong, Luca Turella, Anna E. van’t Veer, Tom Verguts, Jean Vettel, Sagana Vijayarajah, Khoi Vo, Matthew Wall, Wouter D. Weeda, Susanne Weis, David White, David Wisniewski, Alba Xifra-Porxas, Emily Yearling, Sangsuk Yoon, Rui Yuan, Kenneth Yuen, Lei Zhang, Xu Zhang, Joshua Zosky, Thomas E. Nichols, Russell A. Poldrack, Tom Schonberg
bioRxiv 843193; doi: https://doi.org/10.1101/843193

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (1647)
  • Biochemistry (2739)
  • Bioengineering (1908)
  • Bioinformatics (10251)
  • Biophysics (4186)
  • Cancer Biology (3219)
  • Cell Biology (4541)
  • Clinical Trials (135)
  • Developmental Biology (2843)
  • Ecology (4469)
  • Epidemiology (2041)
  • Evolutionary Biology (7231)
  • Genetics (5478)
  • Genomics (6815)
  • Immunology (2391)
  • Microbiology (7491)
  • Molecular Biology (2995)
  • Neuroscience (18598)
  • Paleontology (136)
  • Pathology (472)
  • Pharmacology and Toxicology (780)
  • Physiology (1151)
  • Plant Biology (2708)
  • Scientific Communication and Education (680)
  • Synthetic Biology (889)
  • Systems Biology (2848)
  • Zoology (468)