RT Journal Article SR Electronic T1 Reverse-engineering Recurrent Neural Network solutions to a hierarchical inference task for mice JF bioRxiv FD Cold Spring Harbor Laboratory SP 2020.06.09.142745 DO 10.1101/2020.06.09.142745 A1 Rylan Schaeffer A1 Mikail Khona A1 Leenoy Meshulam A1 International Brain Laboratory A1 Ila Rani Fiete YR 2020 UL http://biorxiv.org/content/early/2020/06/12/2020.06.09.142745.abstract AB We study how recurrent neural networks (RNNs) solve a hierarchical inference task involving two latent variables and disparate timescales separated by 1-2 orders of magnitude. The task is of interest to the International Brain Laboratory, a global collaboration of experimental and theoretical neuroscientists studying how the mammalian brain generates behavior. We make four discoveries. First, RNNs learn behavior that is quantitatively similar to ideal Bayesian baselines. Second, RNNs perform inference by learning a two-dimensional subspace defining beliefs about the latent variables. Third, the geometry of RNN dynamics reflects an induced coupling between the two separate inference processes necessary to solve the task. Fourth, we perform model compression through a novel form of knowledge distillation on hidden representations – Representations and Dynamics Distillation (RADD)– to reduce the RNN dynamics to a low-dimensional, highly interpretable model. This technique promises a useful tool for interpretability of high dimensional nonlinear dynamical systems. Altogether, this work yields predictions to guide exploration and analysis of mouse neural data and circuity.Competing Interest StatementThe authors have declared no competing interest.