Multi-stream predictions in human auditory cortex during natural music listening

Abstract

Real-world perception involves the prediction and integration of multiple dynamic objects and features in parallel, yet most research focuses on single-stream sequences. We present PolyRNN, a recurrent neural network designed to model predictions across multiple, simultaneous information streams, using polyphonic music as a case study. We recorded neurophysiological activity non invasively (MEG) and within the human cortex (intracranial EEG) while participants listened to real piano music. Musical expectations are encoded in P2- and P3-like components in auditory regions. Compared to a state-of-the-art generative music model, we demonstrate that parallelization better reflects the brain’s processing of simultaneous sequences compared to serialization. Overall, our approach enables the study of predictive processing in ecologically valid polyphonic music and provides a general framework for modeling predictions in simultaneous streams.