It has recently been observed that the brain network supporting recall of episodic memories shares much in common with other cognitive functions such as episodic future thinking, navigation and theory of mind. It has been speculated that 'self-projection' is the key common process. However, in this Opinion article, we note that other functions (e.g. imagining fictitious experiences) not explicitly connected to either the self or a subjective sense of time, activate a similar brain network. Hence, we argue that the process of 'scene construction' is better able to account for the commonalities in the brain areas engaged by an extended range of disparate functions. In light of this, we re-evaluate our understanding of episodic memory, the processes underpinning it and other related cognitive functions.