ABSTRACT
Many events we experience are binary and probabilistic, such as the weather (rain or no rain) and the outcome of medical tests (negative or positive). Extensive research in the behavioural sciences has addressed people’s ability to learn stationary probabilities (i.e., probabilities that stay constant over time) of such events, but only recently have there been attempts to model the cognitive processes whereby people learn – and track – non-stationary probabilities. The old debate on whether learning occurs trial-by-trial or by occasional shifts between discrete hypotheses has been revived in this context. Trial-by-trial estimation models – such as the delta-rule model – have been successful in describing human learning in various contexts. It has been argued, however, that behaviour on non-stationary probability learning tasks is incompatible with trial-by-trial learning and can only be explained by models in which learning proceeds through hypothesis testing. Here, we show that this conclusion was premature. By combining two well-supported concepts from cognitive modelling – delta-rule learning and drift diffusion evidence accumulation – we reproduce all behavioural phenomena that were previously used to reject trial-by-trial learning models. Moreover, a quantitative model comparison shows that this model accounts for the data better than a model based on hypothesis testing. In the spirit of cumulative science, our results demonstrate that a combination of two well-established theories of trial-by-trial learning and evidence accumulation is sufficient to explain human learning of non-stationary probabilities.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
We have no conflicts of interest to disclose. The research was funded by the Swedish Research Council (Grant 2018-01947) and the Marcus and Amalia Wallenberg Foundation (MAW 2016.0132).
We thank Randy Gallistel and Matthew Ricci for sharing their data with us and for several helpful discussions about their experiments. We also thank Luigi Acerbi for a helpful discussion about the likelihood function.
All data analysed in this study can be found at https://osf.io/zhv2r/. This study was not preregistered.
The manuscript has been reformatted from an Original Research Article to a more narrowly focused Theoretical Note. The major changes are as follows: 1. The "Experiment" part of the paper has been removed, so that the focus is entirely on the reassessment of previous data and previous conclusions; 2. An extensive analysis has been added to verify that empirical data are not "fatal" to the delta-rule model; 3. The factorial model design has been replaced with a simpler design that focuses on just the two main competing models; 4. We added an analysis showing that combining delta-rule learning with a drift-diffusion mechanism on the prediction error adequately accounts for participants' reports of suspected changes in the generative function as well as second thoughts about those reports; 5. The "custom likelihood function" has been replaced with the proper likelihood function.
4 This experiment had 4 subjects, but we suspect that for one of them the responses were flipped between two sessions. We excluded this subject from our analyses.