PT  - JOURNAL ARTICLE
AU  - Catherine L. Lawson
AU  - Andriy Kryshtafovych
AU  - Paul D. Adams
AU  - Pavel V. Afonine
AU  - Matthew L. Baker
AU  - Benjamin A. Barad
AU  - Paul Bond
AU  - Tom Burnley
AU  - Renzhi Cao
AU  - Jianlin Cheng
AU  - Grzegorz Chojnowski
AU  - Kevin Cowtan
AU  - Ken A. Dill
AU  - Frank DiMaio
AU  - Daniel P. Farrell
AU  - James S. Fraser
AU  - Mark A. Herzik, Jr.
AU  - Soon Wen Hoh
AU  - Jie Hou
AU  - Li-Wei Hung
AU  - Maxim Igaev
AU  - Agnel P. Joseph
AU  - Daisuke Kihara
AU  - Dilip Kumar
AU  - Sumit Mittal
AU  - Bohdan Monastyrskyy
AU  - Mateusz Olek
AU  - Colin M. Palmer
AU  - Ardan Patwardhan
AU  - Alberto Perez
AU  - Jonas Pfab
AU  - Grigore D. Pintilie
AU  - Jane S. Richardson
AU  - Peter B. Rosenthal
AU  - Daipayan Sarkar
AU  - Luisa U. Schäfer
AU  - Michael F. Schmid
AU  - Gunnar F. Schröder
AU  - Mrinal Shekhar
AU  - Dong Si
AU  - Abishek Singharoy
AU  - Genki Terashi
AU  - Thomas C. Terwilliger
AU  - Andrea Vaiana
AU  - Liguo Wang
AU  - Zhe Wang
AU  - Stephanie A. Wankowicz
AU  - Christopher J. Williams
AU  - Martyn Winn
AU  - Tianqi Wu
AU  - Xiaodi Yu
AU  - Kaiming Zhang
AU  - Helen M. Berman
AU  - Wah Chiu
TI  - Outcomes of the 2019 EMDataResource model challenge: validation of cryo-EM models at near-atomic resolution
AID  - 10.1101/2020.06.12.147033
DP  - 2020 Jan 01
TA  - bioRxiv
PG  - 2020.06.12.147033
4099  - http://biorxiv.org/content/early/2020/06/15/2020.06.12.147033.short
4100  - http://biorxiv.org/content/early/2020/06/15/2020.06.12.147033.full
AB  - This paper describes outcomes of the 2019 Cryo-EM Map-based Model Metrics Challenge sponsored by EMDataResource (www.emdataresource.org). The goals of this challenge were (1) to assess the quality of models that can be produced using current modeling software, (2) to check the reproducibility of modeling results from different software developers and users, and (3) compare the performance of current metrics used for evaluation of models. The focus was on near-atomic resolution maps with an innovative twist: three of four target maps formed a resolution series (1.8 to 3.1 Å) from the same specimen and imaging experiment. Tools developed in previous challenges were expanded for managing, visualizing and analyzing the 63 submitted coordinate models, and several novel metrics were introduced. The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual laboratory experiments and holdings of structure data archives such as the Protein Data Bank. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived from these benchmark maps by 13 participating teams, representing both widely used and novel modeling approaches. We also evaluate the pros and cons of the commonly used metrics to assess model quality and recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed density in the cryo-EM map.Competing Interest StatementThe authors have declared no competing interest.