RT Journal Article
SR Electronic
T1 Towards a Foundation Model of the Mouse Visual Cortex
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2023.03.21.533548
DO 10.1101/2023.03.21.533548
A1 Eric Y. Wang
A1 Paul G. Fahey
A1 Kayla Ponder
A1 Zhuokun Ding
A1 Andersen Chang
A1 Taliah Muhammad
A1 Saumil Patel
A1 Zhiwei Ding
A1 Dat Tran
A1 Jiakun Fu
A1 Stelios Papadopoulos
A1 Katrin Franke
A1 Alexander S. Ecker
A1 Jacob Reimer
A1 Xaq Pitkow
A1 Fabian H. Sinz
A1 Andreas S. Tolias
YR 2023
UL http://biorxiv.org/content/early/2023/04/22/2023.03.21.533548.abstract
AB Understanding the brain’s perception algorithm is a highly intricate problem, as the inherent complexity of sensory inputs and the brain’s nonlinear processing make characterizing sensory representations difficult. Recent studies have shown that functional models—capable of predicting large-scale neuronal activity in response to arbitrary sensory input—can be powerful tools for characterizing neuronal representations by enabling high-throughput in silico experiments. However, accurately modeling responses to dynamic and ecologically relevant inputs like videos remains challenging, particularly when generalizing to new stimulus domains outside the training distribution. Inspired by recent breakthroughs in artificial intelligence, where foundation models—trained on vast quantities of data— have demonstrated remarkable capabilities and generalization, we developed a “foundation model” of the mouse visual cortex: a deep neural network trained on large amounts of neuronal responses to ecological videos from multiple visual cortical areas and mice. The model accurately predicted neuronal responses not only to natural videos but also to various new stimulus domains, such as coherent moving dots and noise patterns, underscoring its generalization abilities. The foundation model could also be adapted to new mice with minimal natural movie training data. We applied the foundation model to the MICrONS dataset: a study of the brain that integrates structure with function at unprecedented scale, containing nanometer-scale morphology, connectivity with >500,000,000 synapses, and function of >70,000 neurons within a ∼ 1mm3 volume spanning multiple areas of the mouse visual cortex. This accurate functional model of the MICrONS data opens the possibility for a systematic characterization of the relationship between circuit structure and function. By precisely capturing the response properties of the visual cortex and generalizing to new stimulus domains and mice, foundation models can pave the way for a deeper understanding of visual computation.Competing Interest StatementThe authors have declared no competing interest.