TY - JOUR T1 - Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech JF - bioRxiv DO - 10.1101/2020.10.02.324392 SP - 2020.10.02.324392 AU - Shailee Jain AU - Shivangi Mahto AU - Javier S. Turek AU - Vy A. Vo AU - Amanda LeBel AU - Alexander G. Huth Y1 - 2021/01/01 UR - http://biorxiv.org/content/early/2021/02/16/2020.10.02.324392.abstract N2 - Natural language contains information at multiple timescales. To understand how the human brain represents this information, one approach is to build encoding models that predict fMRI responses to natural language using representations extracted from neural network language models (LMs). However, these LM-derived representations do not explicitly separate information at different timescales, making it difficult to interpret the encoding models. In this work we construct interpretable multi-timescale representations by forcing individual units in an LSTM LM to integrate information over specific temporal scales. This allows us to explicitly and directly map the timescale of information encoded by each individual fMRI voxel. Further, the standard fMRI encoding procedure does not account for varying temporal properties in the encoding features. We modify the procedure so that it can capture both short- and long-timescale information. This approach outperforms other encoding models, particularly for voxels that represent long-timescale information. It also provides a finer-grained map of timescale information in the human language pathway. This serves as a framework for future work investigating temporal hierarchies across artificial and biological language systems.Competing Interest StatementThe authors have declared no competing interest. ER -