TY - JOUR T1 - Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans JF - bioRxiv DO - 10.1101/107078 SP - 107078 AU - Julie Lee AU - Mehdi Keramati Y1 - 2017/01/01 UR - http://biorxiv.org/content/early/2017/02/24/107078.abstract N2 - Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory predicts that flexibility to changes in both reward values and transition contingencies can determine the relative influence of the two systems in reinforcement learning, but few studies have manipulated the latter. Therefore, we developed a novel two-level contingency change task in which transition contingencies between states change every few trials; MB and MF control predict different responses following these contingency changes, allowing their relative influence to be inferred. Additionally, we manipulated the rate of contingency changes in order to determine whether contingency change volatility would play a role in shifting subjects between a MB or MF strategy. We found that human subjects employed a hybrid MB/MF strategy on the task, corroborating the parallel contribution of MB and MF systems in reinforcement learning. Further, subjects did not remain at one level of MB/MF behaviour but rather displayed a shift towards more MB behavior over the first two blocks that was not attributable to the rate of contingency changes but was rather a more general effect of block order. The extent to which each subject used MB control was also related to reward earned, with a correlation between MB weight and reward rate. We demonstrate that flexibility to contingency changes can distinguish MB and MF strategies, with human subjects utilising a hybrid strategy that shifts towards more MB behavior over blocks, consequently corresponding to a higher payoff. ER -