RT Journal Article
SR Electronic
T1 Outcomes of the 2019 EMDataResource model challenge: validation of cryo-EM models at near-atomic resolution
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2020.06.12.147033
DO 10.1101/2020.06.12.147033
A1 Catherine L. Lawson
A1 Andriy Kryshtafovych
A1 Paul D. Adams
A1 Pavel V. Afonine
A1 Matthew L. Baker
A1 Benjamin A. Barad
A1 Paul Bond
A1 Tom Burnley
A1 Renzhi Cao
A1 Jianlin Cheng
A1 Grzegorz Chojnowski
A1 Kevin Cowtan
A1 Ken A. Dill
A1 Frank DiMaio
A1 Daniel P. Farrell
A1 James S. Fraser
A1 Mark A. Herzik, Jr.
A1 Soon Wen Hoh
A1 Jie Hou
A1 Li-Wei Hung
A1 Maxim Igaev
A1 Agnel P. Joseph
A1 Daisuke Kihara
A1 Dilip Kumar
A1 Sumit Mittal
A1 Bohdan Monastyrskyy
A1 Mateusz Olek
A1 Colin M. Palmer
A1 Ardan Patwardhan
A1 Alberto Perez
A1 Jonas Pfab
A1 Grigore D. Pintilie
A1 Jane S. Richardson
A1 Peter B. Rosenthal
A1 Daipayan Sarkar
A1 Luisa U. Schäfer
A1 Michael F. Schmid
A1 Gunnar F. Schröder
A1 Mrinal Shekhar
A1 Dong Si
A1 Abishek Singharoy
A1 Genki Terashi
A1 Thomas C. Terwilliger
A1 Andrea Vaiana
A1 Liguo Wang
A1 Zhe Wang
A1 Stephanie A. Wankowicz
A1 Christopher J. Williams
A1 Martyn Winn
A1 Tianqi Wu
A1 Xiaodi Yu
A1 Kaiming Zhang
A1 Helen M. Berman
A1 Wah Chiu
YR 2020
UL http://biorxiv.org/content/early/2020/06/15/2020.06.12.147033.abstract
AB This paper describes outcomes of the 2019 Cryo-EM Map-based Model Metrics Challenge sponsored by EMDataResource (www.emdataresource.org). The goals of this challenge were (1) to assess the quality of models that can be produced using current modeling software, (2) to check the reproducibility of modeling results from different software developers and users, and (3) compare the performance of current metrics used for evaluation of models. The focus was on near-atomic resolution maps with an innovative twist: three of four target maps formed a resolution series (1.8 to 3.1 Å) from the same specimen and imaging experiment. Tools developed in previous challenges were expanded for managing, visualizing and analyzing the 63 submitted coordinate models, and several novel metrics were introduced. The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual laboratory experiments and holdings of structure data archives such as the Protein Data Bank. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived from these benchmark maps by 13 participating teams, representing both widely used and novel modeling approaches. We also evaluate the pros and cons of the commonly used metrics to assess model quality and recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed density in the cryo-EM map.Competing Interest StatementThe authors have declared no competing interest.