TY - JOUR T1 - A flow-based latent state generative model of neural population responses to natural images JF - bioRxiv DO - 10.1101/2021.09.09.459570 SP - 2021.09.09.459570 AU - Mohammad Bashiri AU - Edgar Y. Walker AU - Konstantin-Klemens Lurz AU - Akshay Kumar Jagadish AU - Taliah Muhammad AU - Zhiwei Ding AU - Zhuokun Ding AU - Andreas S. Tolias AU - Fabian H. Sinz Y1 - 2021/01/01 UR - http://biorxiv.org/content/early/2021/09/10/2021.09.09.459570.abstract N2 - We present a joint deep neural system identification model for two major sources of neural variability: stimulus-driven and stimulus-conditioned fluctuations. To this end, we combine (1) state-of-the-art deep networks for stimulus-driven activity and (2) a flexible, normalizing flow-based generative model to capture the stimulus-conditioned variability including noise correlations. This allows us to train the model end-to-end without the need for sophisticated probabilistic approximations associated with many latent state models for stimulus-conditioned fluctuations. We train the model on the responses of thousands of neurons from multiple areas of the mouse visual cortex to natural images. We show that our model outperforms previous state-of-the-art models in predicting the distribution of neural population responses to novel stimuli, including shared stimulus-conditioned variability. Furthermore, it successfully learns known latent factors of the population responses that are related to behavioral variables such as pupil dilation, and other factors that vary systematically with brain area or retinotopic location. Overall, our model accurately accounts for two critical sources of neural variability while avoiding several complexities associated with many existing latent state models. It thus provides a useful tool for uncovering the interplay between different factors that contribute to variability in neural activity.Competing Interest StatementThe authors have declared no competing interest. ER -