RT Journal Article
SR Electronic
T1 Deep learning from multiple experts improves identification of amyloid neuropathologies
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2021.03.12.435050
DO 10.1101/2021.03.12.435050
A1 Daniel R. Wong
A1 Ziqi Tang
A1 Nicholas C. Mew
A1 Sakshi Das
A1 Justin Athey
A1 Kirsty E. McAleese
A1 Julia K. Kofler
A1 Margaret E. Flanagan
A1 Ewa Borys
A1 Charles L. White III
A1 Atul J. Butte
A1 Brittany N. Dugger
A1 Michael J. Keiser
YR 2022
UL http://biorxiv.org/content/early/2022/04/10/2021.03.12.435050.abstract
AB Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a DL approach that draws on a cohort of experts, weighs each contribution, and is robust to noisy labels. We collected 100,495 annotations on 20,099 candidate amyloid beta neuropathologies (cerebral amyloid angiopathy (CAA), and cored and diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained on a consensus-of-two strategy yielded 12.6-26% improvements by area under the precision recall curve (AUPRC) when compared to those that learned individualized annotations. This strategy surpassed individual-expert models, even when unfairly assessed on benchmarks favoring them. Moreover, ensembling over individual models was robust to hidden random annotators. In blind prospective tests of 52,555 subsequent expert-annotated images, the models labeled pathologies like their human counterparts (consensus model AUPRC=0.74 cored; 0.69 CAA). This study demonstrates a means to combine multiple ground truths into a common-ground DL model that yields consistent diagnoses informed by multiple and potentially variable expert opinion.Competing Interest StatementThe authors have declared no competing interest.DLdeep learningCAAcerebral amyloid angiopathyAβamyloid betaWSIswhole slide imagesIRBInstitutional Review BoardHSVhue saturation valueCNNconvolutional neural networkNP#neuropathologist #UG#undergraduate novice #AUROCarea under the receiver operating characteristicAUPRCarea under the precision recall curveCAMsclass activation maps