RT Journal Article
SR Electronic
T1 Experience replay supports non-local learning
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2020.10.20.343061
DO 10.1101/2020.10.20.343061
A1 Liu, Yunzhe
A1 Mattar, Marcelo G.
A1 Behrens, Timothy E J
A1 Daw, Nathaniel D.
A1 Dolan, Raymond J
YR 2020
UL http://biorxiv.org/content/early/2020/10/21/2020.10.20.343061.abstract
AB To make effective decisions we need to consider the relationship between actions and outcomes. They are, however, often separated by time and space. The biological mechanism capable of spanning those gaps remains unknown. One promising, albeit hypothetical, mechanism involves neural replay of non-local experience. Using a novel task, that segregates direct from indirect learning, combined with magnetoencephalography (MEG), we tested the role of neural replay in non-local learning in humans. Following reward receipt, we found significant backward replay of non-local experience, with a 160 msec state-to-state time lag, and this replay facilitated learning of action values. This backward replay, combined with behavioural evidence of non-local learning, was more pronounced in experiences that were of greater benefit for future behavior, as predicted by theories of prioritization. These findings establish rationally targeted non-local replay as a neural mechanism for solving complex credit assignment problems during learning.One Sentence Summary Reverse sequential replay is found, for the first time, to support non-local reinforcement learning in humans and is prioritized according to utility.Competing Interest StatementThe authors have declared no competing interest.