Humans adaptively select different computational strategies in different learning environments

Abstract

The Rescorla-Wagner rule remains the most popular tool to describe human behavior in reinforcement learning tasks. Nevertheless, it cannot fit human learning in complex environments. Previous work proposed several hierarchical extensions of this learning rule. However, it remains unclear when a flat (non-hierarchical) versus a hierarchical strategy is optimal, or when it is implemented by humans. To address this question, current work evaluates multiple models in multiple reinforcement learning environments both computationally (which approach performs best) and empirically (which approach fits human data best). We consider ten empirical datasets (N = 410) divided over three reinforcement learning environments. Consistent with the idea of the human brain as a mixture of expert system, our results demonstrate that different environments are best solved with different learning strategies; and that humans seemed to adaptively select the optimal learning strategy. Specifically, while flat learning fitted best in less complex stable learning environments, humans employed more hierarchically complex models in more complex environments.