RT Journal Article
SR Electronic
T1 Temporal discounting correlates with directed exploration but not with random exploration
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 783498
DO 10.1101/783498
A1 Hashem Sadeghiyeh
A1 Siyu Wang
A1 Maxwell R. Alberhasky
A1 Hannah M. Kyllo
A1 Amitai Shenhav
A1 Robert C. Wilson
YR 2019
UL http://biorxiv.org/content/early/2019/09/26/783498.abstract
AB The explore-exploit dilemma describes the trade off that occurs any time we must choose between exploring unknown options and exploiting options we know well. Implicit in this trade off is how we value future rewards — exploiting is usually better in the short term, but in the longer term the benefits of exploration can be huge. Thus, in theory there should be a tight connection between how much people value future rewards, i.e. how much they discount future rewards relative to immediate rewards, and how likely they are to explore, with less ‘temporal discounting’ associated with more exploration. By measuring individual differences in temporal discounting and correlating them with explore-exploit behavior, we tested whether this theoretical prediction holds in practice. We used the 27-item Delay-Discounting Questionnaire (Kirby et al., 1999) to estimate temporal discounting and the Horizon Task (Wilson et al. 2014) to quantify two strategies of explore-exploit behavior: directed exploration, where information drives exploration by choice, and random exploration, where behavioral variability drives exploration by chance. We find a clear correlation between temporal discounting and directed exploration, with more temporal discounting leading to less directed exploration. Conversely, we find no relationship between temporal discounting and random exploration. Unexpectedly, we find that the relationship with directed exploration appears to be driven by a correlation between temporal discounting and uncertainty seeking at short time horizons, rather than information seeking at long horizons. Taken together our results suggest a nuanced relationship between temporal discounting and explore-exploit behavior that may be mediated by multiple factors.