RT Journal Article
SR Electronic
T1 GPT-2’s activations predict the degree of semantic comprehension in the human brain
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2021.04.20.440622
DO 10.1101/2021.04.20.440622
A1 Caucheteux, Charlotte
A1 Gramfort, Alexandre
A1 King, Jean-Rémi
YR 2021
UL http://biorxiv.org/content/early/2021/09/02/2021.04.20.440622.abstract
AB Language transformers, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of deep translation, summarization and dialogue algorithms. However, whether these models encode information that relates to human comprehension remains controversial. Here, we show that the representations of GPT-2 not only map onto the brain responses to spoken stories, but also predict the extent to which subjects understand narratives. To this end, we analyze 101 subjects recorded with functional Magnetic Resonance Imaging while listening to 70 min of short stories. We then fit a linear model to predict brain activity from GPT-2’s activations, and correlate this mapping with subjects’ comprehension scores as assessed for each story. The results show that GPT-2’s brain predictions significantly correlate with semantic comprehension. These effects are bilaterally distributed in the language network and peak with a correlation of R=0.50 in the angular gyrus. Overall, this study paves the way to model narrative comprehension in the brain through the lens of modern language algorithms.Competing Interest StatementThe authors have declared no competing interest.