Abstract
The human brain is very effective at integrating new words one by one into the composed representation of a sentence as it is read left-to-right. This raises the important question of what happens to the neural representations of words present earlier in the sentence? For example, do the strength of word representations encountered earlier on in the sentence remain constant or do they evolve as additional words are processed? Representation of words by neural activity in the brain has been the subject of several previous studies. We perform the experiment with a naturalistic task in which the subjects read simple active and passive sentences. Naturalistic studies have tended to explore words in isolation or in a very limited context (e.g., adjective-noun phrases). Representation of previously encountered words during incremental sentence reading, and how such representation evolve as more parts of a sentence are read, is a fundamental but unexplored problem – we take a first step in this direction. In particular, we examine the spatio-temporal characteristics of neural activity encoding nouns and verbs encountered in a sentence as it is read word-by-word. We use Magnetoencephalography (MEG) to passively observe neural activity, providing 1 ms temporal resolution.
Our experiments reveal that nouns and verbs read early in the sentence have a varying influence on neural activity while reading subsequent words, decreasing and increasing at particular word positions in active and passively voiced sentences, with particularly important contributions to activity in frontal and temporal cortical regions. We find the noun and verb information to be decodable from the neural activity for several seconds after sentence reading has completed. Our exploration is also the first to study the effect of question-answering task on the neural representation of the words post-sentence. We are releasing our 300 sentence MEG dataset to encourage further research in this important area.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
sharmisthaj{at}iisc.ac.in, ppt{at}iisc.ac.in, laing{at}andrew.cmu.edu, tom.mitchell{at}cs.cmu.edu
34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada.
Author names and affiliations are added to the paper header.
