PT  - JOURNAL ARTICLE
AU  - Vijay Mohan K Namboodiri
TI  - What is the state space of the world for real animals?
AID  - 10.1101/2021.02.07.430001
DP  - 2021 Jan 01
TA  - bioRxiv
PG  - 2021.02.07.430001
4099  - http://biorxiv.org/content/early/2021/02/08/2021.02.07.430001.short
4100  - http://biorxiv.org/content/early/2021/02/08/2021.02.07.430001.full
AB  - A key concept in reinforcement learning (RL) is that of a state space. A state space is an abstract representation of the world using which statistical relations in the world can be described. The simplest form of RL, model free RL, is widely applied to explain animal behavior in numerous neuroscientific studies. More complex RL versions assume that animals build and store an explicit model of the world in memory. To apply these approaches to explain animal behavior, typical neuroscientific RL models make assumptions about the underlying state space formed by animals, especially regarding the representation of time. Here, we explicitly list these assumptions and show that they have several problematic implications. We propose a solution for these problems by using a continuous time Markov renewal process model of the state space. We hope that our explicit treatment results in a serious consideration of these issues when applying RL models to real animals.Competing Interest StatementThe authors have declared no competing interest.