Decoding of generic mental representations from functional MRI data using word embeddings

Abstract

Several different groups have demonstrated the feasibility of building forward models of functional MRI data in response to concrete stimuli such as pictures or video, and of using these models to decode or reconstruct stimuli shown while acquiring test fMRI data. In this paper, we introduce an approach for building forward models of conceptual stimuli, concrete or abstract, and for using these models to carry out decoding of semantic information from new imaging data. We show that this approach generalizes to topics not seen in training, and provides a straightforward path to decoding from more complex stimuli such as sentences or paragraphs.