UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues

ABSTRACT

Newly developed technologies have made it feasible to routinely collect highly multiplexed (20-60 channel) images at subcellular resolution from human tissues for research and diagnostic purposes. Extracting single cell data from such images requires efficient and accurate image segmentation, a challenging problem that has recently benefited from the use of deep learning. In this paper, we demonstrate two approaches to improving tissue segmentation that are applicable to multiple deep learning frameworks. The first uses “real augmentations” that comprise defocused and saturated image data collected on the same instruments as the actual data; using real augmentation improves model accuracy to a significantly greater degree than computational augmentation (Gaussian blurring). The second involves imaging the nuclear envelope to better identify nuclear outlines. The two approaches cumulatively and substantially improve segmentation on a wide range of tissue types and provide a set of improved segmentation models. We speculate that the use of real augmentations may have applications in image processing outside of microscopy.