RT Journal Article
SR Electronic
T1 GPT-2’s activations predict the degree of semantic comprehension in the human brain
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2021.04.20.440622
DO 10.1101/2021.04.20.440622
A1 Caucheteux, Charlotte
A1 Gramfort, Alexandre
A1 King, Jean-Rémi
YR 2021
UL http://biorxiv.org/content/early/2021/04/21/2021.04.20.440622.abstract
AB Language transformers, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of deep translation, summarization and dialogue algorithms. However, whether these models actually understand language is highly controversial. Here, we show that the representations of GPT-2 not only map onto the brain responses to spoken stories, but also predict the extent to which subjects understand the narratives. To this end, we analyze 101 subjects recorded with functional Magnetic Resonance Imaging while listening to 70 min of short stories. We then fit a linear model to predict brain activity from GPT-2 activations, and correlate this mapping with subjects’ comprehension scores as assessed for each story. The results show that GPT-2’s brain predictions significantly correlate with semantic comprehension. These effects are bilaterally distributed in the language network and peak with a correlation above 30% in the infero-frontal and medio-temporal gyri as well as in the superior frontal cortex, the planum temporale and the precuneus. Overall, this study provides an empirical framework to probe and dissect semantic comprehension in brains and deep learning algorithms.Competing Interest StatementThe authors have declared no competing interest.