PT - JOURNAL ARTICLE AU - Kristen R. Maynard AU - Madhavi Tippani AU - Yoichiro Takahashi AU - BaDoi N. Phan AU - Thomas M. Hyde AU - Andrew E. Jaffe AU - Keri Martinowich TI - <em>dotdotdot</em>: an automated approach to quantify multiplex single molecule fluorescent in situ hybridization (smFISH) images in complex tissues AID - 10.1101/781559 DP - 2019 Jan 01 TA - bioRxiv PG - 781559 4099 - http://biorxiv.org/content/early/2019/09/25/781559.short 4100 - http://biorxiv.org/content/early/2019/09/25/781559.full AB - Multiplex single-molecule fluorescent in situ hybridization (smFISH) is a powerful method for validating RNA sequencing and emerging spatial transcriptomic data, but quantification remains a computational challenge. We present a framework for generating and analyzing smFISH data in complex tissues while overcoming autofluorescence and increasing multiplexing capacity. We developed dotdotdot (https://github.com/LieberInstitute/dotdotdot) as a corresponding software package to quantify RNA transcripts in single nuclei and perform differential expression analysis. We first demonstrate robustness of our platform in single mouse neurons by quantifying differential expression of activity-regulated genes. We then quantify spatial gene expression in human dorsolateral prefrontal cortex (DLPFC) using spectral imaging and dotdotdot to mask lipofuscin autofluorescence. We lastly apply machine learning to predict cell types and perform downstream cell type-specific expression analysis. In summary, we provide experimental workflows, imaging acquisition and analytic strategies for quantification and biological interpretation of smFISH data in complex tissues.