TY - JOUR T1 - Reinforcement waves as a mechanism for discontinuous learning JF - bioRxiv DO - 10.1101/2022.05.06.490910 SP - 2022.05.06.490910 AU - Gautam Reddy Y1 - 2022/01/01 UR - http://biorxiv.org/content/early/2022/05/06/2022.05.06.490910.abstract N2 - Problem-solving and reasoning involve mental exploration and navigation in sparse relational spaces. A physical analogue is spatial navigation in structured environments such as a network of burrows. Recent experiments with mice navigating a labyrinth show a sharp discontinuity during learning, corresponding to a distinct moment of ‘sudden insight’ when mice figure out long, direct paths to the goal. This discontinuity is seemingly at odds with reinforcement learning (RL), which involves a gradual build-up of a value signal during learning. Here, we show that biologically-plausible RL rules combined with persistent exploration generically exhibit discontinuous learning. In structured environments, positive feedback from learning generates a traveling ‘reinforcement wave’. The discontinuity occurs when the wave reaches the starting point. Task difficulty and the learning rule alter its profile and speed, which are determined by the nonlinear dynamics between the nose and bulk of the wave. Predictions explain existing data and motivate specific experiments to isolate the phenomenon. Additionally, we characterize the exact learning dynamics of various RL rules for a complex sequential task.Competing Interest StatementThe authors have declared no competing interest. ER -