RT Journal Article
SR Electronic
T1 Properties of decision-making tasks govern the tradeoff between model-based and model-free learning
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 730663
DO 10.1101/730663
A1 Zachary M Abzug
A1 Marc A Sommer
A1 Jeffrey M Beck
YR 2019
UL http://biorxiv.org/content/early/2019/08/14/730663.abstract
AB When decisions must be made between uncertain options, optimal behavior depends on accurate estimations of the likelihoods of different outcomes. The contextual factors that govern whether these estimations depend on model-free learning (tracking outcomes) vs. model-based learning (learning generative stimulus distributions) are poorly understood. We studied model-free and model-based learning using serial decision-making tasks in which subjects selected a rule and then used it to flexibly act on visual stimuli. A factorial approach defined a family of behavioral models that could integrate model-free and model-based strategies to predict rule selection trial-by-trial. Bayesian model selection demonstrated that the subjects strategies varied depending on lower-level task characteristics such as the identities of the rule options. In certain conditions, subjects integrated learned stimulus distributions and tracked reward rates to guide their behavior. The results thus identify tradeoffs between model-based and model-free decision strategies, and in some cases parallel utilization, depending on task context.