Abstract
Central to the concept of the ‘cognitive map’ is that it confers behavioural flexibility, allowing animals to take efficient detours, exploit shortcuts and realise the need to back-track rather than persevere on a poorly chosen route. The neural underpinnings of such naturalistic and flexible behaviour remain unclear. During fMRI we tested human subjects on their ability to navigate to a set of goal locations in a virtual desert island riven by lava, which occasionally shifted to block selected paths (necessitating detours) or receded to open new paths (affording shortcuts). We found that during self-initiated back-tracking, activity increased in frontal regions and the dorsal anterior cingulate cortex, while activity in regions associated with the core default-mode network was suppressed. Detours activated a network of frontal regions compared to shortcuts. Activity in right dorsolateral prefrontal cortex specifically increased when participants encountered new plausible shortcuts but which in fact added to the path (false shortcuts). These results help inform current models as to how the brain supports navigation and planning in dynamic environments.
Significance Statement Adaptation to change is important for survival. Although real-world spatial environments are prone to continual change, little is known about how the brain supports navigation in dynamic environments where flexible adjustments to route plans are needed. Here, we used fMRI to examine the brain activity elicited when humans took forced detours, identified shortcuts and spontaneously back-tracked along their recent path. Both externally and internally generated changes in the route activated the fronto-parietal attention network, whereas only internally generated changes generated increased activity in the dorsal anterior cingulate cortex with a concomitant disengagement in regions associated with the default-mode network. The results provide new insights into how the brain plans and re-plans in the face of a changing environment.
Acknowledgements
We thank Mate Lengyel for advice on the experimental design. This work was supported by the Wellcome Trust (grant 094850/Z/10/Z) and James S. McDonnell Foundation to H.J.S, and the Gatsby Charitable Foundation (P.D.). The authors declare no competing financial interests. P.D. is currently on sabbatical at Uber AI Lab.
Introduction
A challenge all motile animals face is adapting to changes in an environment so that they can efficiently return to safety or find food. Adaptations include identifying novel shortcuts and minimizing the lengths of imposed detours. Tolman (1948) conceptualized this ability as arising from an internal ‘cognitive map’ (or, in control theoretic terms, an internal model) of the environment. Evidence from electrophysiological recordings in rodents and fMRI in humans has supported the view that hippocampus contains a cognitive map (Ekstrom, Spiers, Bohbot, & Rosenbaum, 2018; Epstein, Patai, Julian, & Spiers, 2017; O’Keefe & Nadel, 1978; Spiers & Barry, 2015). However, the evidentiary basis in humans and other animals of the navigational functions of the cognitive map in the face of dynamic environments is incomplete.
Early studies in rodents (Tolman & Honzik, 1930) along with more recent studies in rats and other mammals (Alvernhe, Save, & Poucet, 2011; Alvernhe, Van Cauter, Save, & Poucet, 2008; Chapuis, 1987; Chapuis, Durup, & Thinus-Blanc, 1987; Poucet, Thinus-Blanc, & Chapuis, 1983; Winocur, Moscovitch, Rosenbaum, & Sekeres, 2010) have helped characterise flexible navigation behaviour when the environmental layout changes. However, there has been relatively limited investigation of the evoked neural responses at the moment of encountering changes in the environment. By contrast, a number of functional neuroimaging studies in humans have studied the evoked responses to detours (Howard et al., 2014; Iaria, Fox, Chen, Petrides, & Barton, 2008; Maguire et al., 1998; Rauchs et al., 2008; Rosenbaum, Ziegler, Winocur, Grady, & Moscovitch, 2004; Simon & Daw, 2011; Spiers & Maguire, 2006; Xu, Evensmoen, Lehn, Pintzka, & Håberg, 2010). Rather than revealing hippocampal activity, these studies have consistently reported increased prefrontal activity. These studies report: i) increased activity in right lateral prefrontal regions when detecting changes in the environment, ii) activity in frontopolar cortex when re-planning and setting sub-goals, and iii) superior prefrontal cortical activity when processing conflict between route options (Spiers & Gilbert, 2015). Such responses are consistent with the view that the prefrontal cortex (PFC) supports flexible behaviour in response to changing affordances in the environment (Shallice, 1982; Spiers, 2008).
While some functional magnetic resonance imaging studies have examined shortcuts and detours in changing environments (Simon & Daw, 2011; Yoshida & Ishii, 2006) the paradigms deployed were not optimized to disentangle the neural response to detours and shortcuts, thus to date we lack evidence for how neural systems adjust at shortcuts and how they compare to detours. Since both detours and shortcuts change the path to the goal it is possible that both events elicit a similar neural response. Alternatively, the potential to simulate a new alternative path at detours might evoke increased activity in frontal regions associated with planning. Similarly, the process of determining that a newly encountered path is not in fact a useful route (a false shortcut), and needs to be suppressed, may draw on greater neural resources than when new path is identified as viable. Finally, some re-planning in the real-world is purely spontaneous, self-driven (Spiers and Maguire 2006) where an optimal action is to backtrack along the recent path to reach the goal. To date this behaviour has mainly been studied in ants (Wystrach, Schwarz, Baniel, & Cheng, 2013) and thus the neural correlates of such back-tracking currently remain elusive. To explore neural responses during back-tracking, detours and shorcuts we combined fMRI with a virtual reality (VR)-based environment (‘LavaWorld’) in which participants navigated a desert island containing hidden treasure with paths constrained by lava, which had the capacity to shift and open new paths (shortcuts) or close others off (detours).
Methods
Participants
Twenty-two subjects (mean age: 21.8 ± 2.3 years, range: 19-27; 14 female). Participants were administered a questionnaire regarding their navigation abilities/strategies (Santa Barbara Sense of Direction Scale; mean score = 4.9, range: 3.7-5.7). All participants had normal to corrected vision, reported no medical implant containing metal, no history of neurological or psychiatric condition, color blindness, and did not suffer from claustrophobia.
All participants gave written consent to participate to the study in accordance with the Birkbeck-UCL Centre for Neuroimaging ethics committee. Subjects were compensated with a minimum of £70 plus an additional £10 reward for good performance during the scan. One participant was excluded from the final sample because there was severe signal loss from the medial-temporal area in their functional scan.
VR environment: Lavaworld
A virtual island maze environment was created using Vizard virtual reality software (© WorldViz). The maze was a grid network, consisting of ‘sand’ areas that were walkable, and ‘lava’ areas, which were unpassable and as such were like walls in a traditional maze. However, the whole maze layout was flat, so there was visibility into the distance over both sand and lava. This allowed participants to stay oriented in the maze throughout the task. Orientation cues were provided by four unique large objects in the distance. Movement was controlled by 4 buttons: left, right, forwards and backwards. Pressing left, right or backwards moved the participant to the grid square to the left, right or behind respectively (if there was no lava in the way), and rotated the view accordingly. Similarly, pressing forward moved the participant to the next square along. See Figure 1 for a participant viewpoint at one point in the maze. Participants were tested over two days, on day one they were trained on the maze, and on day two they were tested on the maze in the MRI scanner.
Training
On the first day, participants were trained on the virtual maze (25 × 15 grid) to find goal locations. During this phase, all goal objects (20 in total, distributed across the maze) were visible at all times, and participants navigated from one to the next based on the currently displayed target object (displayed in the top-right corner of the screen). After one hour of training, subjects were given a test to establish how well they had learnt the object locations. On a blank grid, where only the lava was marked, participants had to place all the objects they remembered. They were given feedback from the experimenter, and if needed, prompts as to the missing objects. This memory-test was repeated twice more during the training, after 1.5 and 2 hours. At completion, for participants to return for the fMRI phase on the second day, they had to score at 100% accuracy in placing the objects.
Test & fMRI scan
On the test day, participants were given a brief refresher of the maze with the objects before beginning the test phase. Before scanning, participants were allowed to familiarize themselves with the scanner button pad, and the changes that would occur. This involved presenting them with a novel environment that had not been experienced on day one, and which had no objects, different distal cues and a different maze layout, to avoid any confounds or confusion with training and test mazes. Participants could then practice the task in this new environment, and accustom themselves to the controls (button pad with 4 active buttons: left, right, forward, and turn around) and to the appearance of changes to the lava.
While in the MRI scanner, participants performed the test phase of the experiment. A single trial in the test phase is defined as being informed which is the new goal object, and then finding the way to, and arriving at, it. During the test phase, two things were different from training: 1) target objects were not visible, so participants had to navigate between them based on their memories of the locations, and 2) the lava could move, blocking some paths and creating new paths. Participants were informed that this was a temporary change and that after reaching the goal the environment would revert to the baseline state. During each journey to an object, a single change event occurred in the lava layout. At the point of a change, the screen froze for 4 seconds to ensure that participants had an opportunity to detect the change and consider their path options. These changes could either be Detours (when a piece of lava was added to block the current path on the grid, thus forcing the participant to take an new longer route to their goal); Shortcuts (a piece of lava was removed and replaced with sand, allowing the participant to pick a shorter route); False Shortcuts (visually identical to Shortcuts, but choosing a route through them would increase the net distance to the goal because of the layout of the maze; False Shortcuts came in two classes: False Shortcuts Towards and False Shortcuts Away from the goal, depending on whether or not the False Shortcut seemed to lead in the general direction of the goal or if it was an opening pointing away from it, see Figure 2 for visual examples); and a condition in which the screen froze, but no lava was added/removed. Behavioral data showed that participants were much slower to respond in the last condition with no change, which may be because they were unsure if they had missed a subtle change in the environment (the lava appeared or disappeared instantaneously when they reached the change point). Thus, we focused our analysis on the other conditions. For Detours and Shortcuts, there were also two levels of change to the (optimal) new path, either 4 or 8 grid steps extra/less, respectively. See Figure 2 for example schematics of these changes. Finally, during the test phase there were also control ‘Follow’ trials which started with an arrow that indicated the direction to travel. In this case, participants were required to follow the twists and turns of the arrow until a new target object appeared, and from then onward the trial was like the ‘Navigation’ events as described above. The comparison of ‘Navigation’ vs ‘Follow’ movements allowed us to relate our results to those of previous experiments (Howard et al., 2014; Javadi et al., 2017; Patai et al., 2017).
Spatial Parameters
Spatial parameters were calculated in the same manner as in Howard et al (2014) and Javadi et al (2017). In brief, Path Distance (PD), Euclidian Distance (ED), Egocentric Goal Direction (EGD) and the number of optimal upcoming Turns were calculated at each change point. All parameters were highly correlated (p<0.001, see Table 1A), except for PD/ED and EGD. Based on our previous work (Howard et al. 2014, Patai et al., 2017), our main analysis involved using PD as an independent parametric regressor (Table 1B). We also considered a control model that included both PD and EGD, as these measures were not correlated. The other parameters were not explored independently. Spatial parameter values were rescaled between 0 and 1, where 1 is the maximum value, e.g. the greatest distance.
fMRI Scanning & Preprocessing
Scanning was conducted at the Birkbeck-UCL Centre for Neuroimaging (BUCNI) using a 1.5 Tesla Siemens Avanto MRI scanner (Siemens Medical System, Erlangen, Germany) with a 12 channel head coil. Each experimental session lasted around 60 minutes and was separated in three parts (each of approximately 15-20 minutes). Approximately 980 functional scans were acquired per session (depending on routes taken), using a gradient-echo incremental EPI sequence (TR=3400ms, TE=50ms, TA=3.315s, flip angle= 90°). The slice thickness was 2mm with a gap of 1mm, TR=85ms, TE=50ms, slice tilt = 30°. The field of view was 192 mm, and the matrix size was 64 × 64. The scan was a whole brain acquisition, with 40 slices and a repetition time of 3.4 s. A T1-weighted high-resolution structural scan was acquired after the functional scans (TR=12ms, TE=5,6ms, 1×1×1mm resolution). Ear plugs were used for noise reduction, foam padding was used to secure the head in the scanner and minimize head movements. Stimuli were projected to the back screen, a mirror was attached to the head coil and adjusted for the subjects to see full screen. All fMRI preprocessing and analysis was performed using SPM12. To achieve T1 equilibrium, the first six dummy volumes were discarded. During preprocessing, we used the new Segment (with 6 tissue classes) to optimize normalization. Otherwise, we used all default settings, and we performed slice timing correction. No participants had any abrupt motion change over 4mm.
Experimental Design and Statistical Analysis
Participants performed a total of 120 routes, with one change event occurring in each route (number of trials per condition was 17 on average, range: 11-25, depending on the different scenarios used for counterbalancing routes taken). Each route started from a previous goal and ended at the new goal object for that trial. We used repeated-measures ANOVAs to test for behavioural differences (accuracy, extra steps, back-tracking trials) between conditions. We also calculated d-prime and criterion (signal detection theory measures) to quantify the bias to take a False Shortcuts Towards instead of Away from a goal (both false alarms calculated relative to correct Shortcuts, which are hits). We recorded the response time to make the first choice after the 4 seconds elapsed, but due to the 4 second delay we do not interpret this as a traditional decision-making reaction time.
To analyse the fMRI data, we constructed multiple models, based on a priori predictions from previous work (Howard et al, 2014). Please see that Table 1B for a description of the models, events included and regression parameters. We used a standard preprocessing pipeline in SPM. A priori regions of interest were small volume corrected using anatomical masks (WFU Pick atlas [Maldjian, Laurienti, Kraft, & Burdette, 2003; Tzourio-Mazoyer et al., 2002]) and a functional mask for the dorsomedial PFC (Kaplan et al., 2017) was employed in one follow-up exploratory analysis. For completeness we also report all results at an uncorrected threshold of p < 0.001, minimum 5 contiguous voxels (Howard et al., 2014). This is provided to allow comparison to past similar datasets rather that to draw specific inferences about predicted responses. Note that we used all trials for an event type, irrespective of whether or not the participant was correct for not. We also report data for False Shortcuts comparing correct and incorrect choices in the Extended Data (Figure 3-1).
A ‘Back-tracking’ event was defined by when participants pressed the backwards button and returned to a step along the route they had just come down. Non-back-tracking events were selected from the participants’ other paths and were matched to these in the relative number of steps taken before the back-tracking happened (for example: halfway through the route). We did not use the absolute number of steps, because trials that contained a back-tracking event were often much longer, and thus the step number at which a back-tracking event occurred would in many cases already been located at (or past) a goal location in a non-back-tracking trial.
Results
Behaviour
Our primary measure of navigation was the accuracy of the whole route, in other words whether participants took the optimal path to the target. We conducted a repeated measures ANOVA to test for effects of terrain change type on participants’ accuracy in finding the correct path. We found that there was a significant effect of terrain change type (F(1,120)=17.7,p<0.001), such that Detours (+8) and False Shortcuts Towards the goal resulted in less optimal path taking (both t(1,20)<-3.6,p<0.002) compared to all other conditions (see Table 2, and Extended Data Table 1 for comprehensive t-tests).
To follow up the errors in which participants did not take the optimal path, we looked at the number of extra steps taken on a route. We found a significant effect of terrain change type (F(1,120)=8.3,p<0.001), with overall more steps off-route in the Detours (+8) and False Shortcuts Towards conditions. When quantifying the number of extra steps as a proportion of the total (new) number of optimal steps from the terrain change point onward for a given route, in fact Shortcuts resulted in the largest proportion off-route (see Table 2). Some of these extra steps were due to participants turning around, i.e., “Back-tracking”; these were again more common in the Detours (+8) condition (F(1,120)=8.5,p<0.001, compared to all t(1,20)>2.1, p<0.051, see Extended Data Table 2–6 for details). Overall, only 19% of the extra steps were such Back-tracking events (and this ratio was not significantly different between conditions, F(1,65)=2.17, p=0.068). Moreover, we calculated the ratio of correct, compared to incorrect, Back-tracking trials (“correct” is defined as a trials in which back-tracking would actually bring the participant closer to the goal), and found that overall 85% (±2% s.e.m, range: 50-100%) of Back-tracking events were correct or optimal, and occurred equally frequently across all conditions (F(1,5)=2.2,p=0.066). Thus, in the majority of Back-tracking events participants became aware that they were heading away from the goal and spontaneously decided to turn around.
fMRI Results
fMRI analyses revealed that bilateral hippocampus, posterior cingulate and retrosplenial cortex, as well as frontal areas were more active when participants were actively navigating than when they merely followed an arrow on the screen (Extended Data Table 7). Both the left and the right hippocampus were significantly more active in the navigate than the follow condition (small-volume correction, FWE p<0.05), in line with previous findings (Howard et al., 2014; Patai et al., 2017, for an overview see Spiers & Gilbert 2015).
Caudate, but not hippocampal, activity responds to changes in the path to the goal at Detours
We predicted that hippocampal activity would track the parametric change in the path distance to the goal when the structure of the environment changed. We found no evidence to support this prediction, both with specific ROIs and at a low uncorrected threshold (p < 0.005). This was also true when large Detours (+8) were directly compared with small Detours (+4). By contrast, we found that activity in the caudate nucleus bilaterally tracked the change in the path distance across all types of events (see Table 3), complementing past evidence that this region tracked the magnitude of change in the path at Detours (Howard et al., 2014).
Superior and right lateral prefrontal cortex respond to preferentially to Detours
Next we investigated the brain areas that have been found in previous studies when comparing Detours to non-Detours, including the superior frontal gyrus (SFG), the right lateral prefrontal (rlPFC) and frontopolar cortex (Spiers & Gilbert, 2015), using a combined mask of these areas. A linear contrast of terrain change type: Detours (+8) > Detour (+4) > Shortcuts (−4) > Shortcuts (−8), revealed a significant effect, with specifically the SFG and rlPFC activity scaling with the deviation from the optimal path prior to the change (Figure 3A and Table 3). False Shortcuts Towards the goal also significantly activated the rlPFC compared to Shortcuts (Figure 3B, and Table 3). By contrast False Shortcuts Away from the goal did not drive activity in rlPFC. Frontopolar cortex was not found to be modulated by Detours, Shortcuts, or False Shortcuts.
Prefrontal responses evoked during spontaneous internally generated Back-tracking events
We were interested in the brain areas that were activated by spontaneous, internally generated route changes, i.e. Back-tracking events. To investigate their neural correlates, we compared moments in routes where a back-tracking event occurred (defined as a return to a previous grid point along a single journey towards a target), to equivalent events where no back-tracking happened (equalized according to relative steps along a journey). When comparing Back-tracking to non-Back-tracking events, we again found significant SFG and rlPFC activations (Figure 4A, and Table 3). These results suggest that the prefrontal regions previously found to be involved in detour processing are also active when route changes occur due to spontaneous, internally generated events. We did not find evidence that the frontopolar cortex was responsive to Back-tracking.
Outside our ROIs drawn from Spiers and Gilbert (2015), we found that during Back-tracking, activity in a region of medial prefrontal cortex survived whole-brain FWE correction (Figure 4). This activation overlapped with the dorsal anterior cingulate area (dACC) reported by a recent study when difficult navigational decisions were required (Kaplan et al., 2017). We used a functional ROI (includes the dACC and supplementary motor area (SMA)) from Kaplan et al. (2017) to investigate whether this area was also active during terrain change points, and found that for both the linear contrast above (Detours (+8) > Detour (+4) > Shortcuts (−4) > Shortcuts (−8)) and False Shortcuts Towards > Shortcuts contrast this area was significantly active (Table 3). However when plotting the peak activation, we found that Backtracking events resulted in a significantly larger activation of the dACC than the other contrasts (both t>5.4,p<0.001; Figure 4, bottom right), pointing to a potentially unique role during the processing of internally generated route changes, which presumably occurs when participants become aware their current route plan is not effective.
Back-tracking events disengage from the putative Default-Mode Network
Because Back-tracking involved a visual change in view (180°) compared to Non-back-tracking events we explored whether similar regions would be active when participants turned left or right compared to when they did not turn. We found a significant response in the functional dACC mask when comparing Turns to Non-turns, however the peak was located in the SMA. When Back-tracking was compared to Turns the dACC was significantly more active in Back-tracking events, in which an equivalent amount of visual change is contrasted (90° difference in change of viewpoint). Conversely, Turns compared to Back-tracking resulted in significant responses in the hippocampus, anterior medial PFC and posterior cingulate cortex. These regions are overlapping with those implicated in the default-mode network (DMN) (Andrews-Hanna, Reidler, Sepulcre, Poulin, & Buckner, 2010). To explore whether the results match the default-mode network more explicitly we created an aggregate ROI including the medial PFC, the precuneus/posterior cingulate cortex, bilateral parahippocampal cortex and angular gyrus. We found that Back-tracking significantly suppressed this putative DMN compared with Turns, and this was also the case when comparing Back-tracking to Non-back-tracking and Detour events, highlighting disengagement from this network when participants spontaneously instigated a route change (Figure 5).
Control Analysis for Route Visibility
As our maze environment was open-plan, and participants could see ahead (see Figure 1, ‘Test’), it should be noted that the task could have been solved using a purely visual search of the available paths to the remembered goal location during the change period (as opposed to relying on the map of the layout of the environment from memory). In this case we would expect that at change points where visual analysis of the scene is restricted due to the heterogeneity of the maze layout from the point where they are stopped, to be lower than when they can ‘look’ ahead to check the available paths to the goal. This is particularly relevant for False Shortcuts where ideally participants would decide whether to take the opening based on pre-existing knowledge of the maze layout and the connecting paths to decide if it is a real shortcut or not. We compared False Shortcuts Towards the goal in two cases: when the upcoming path was clearly visible from the change point and when it was not, and found a significant effect in accuracy (visible: 76%(±4), not clearly visible: 53%(±5)), however we also note that in the latter instances, it was the case that taking the False Shortcut only resulted in a minor addition of steps to the path (maximum 3 steps), whereas when then alternative path was more visible, taking those false entry points would have resulted in a significant lengthening to the path to the goal (10+ steps). Due to trial numbers, we cannot directly compare visible vs non-visible by accuracy. Based on these results we cannot clearly confirm whether or not participants were solely using a cognitive map to solve the problem at change points, or if they were also engaging in visual search behaviour. Importantly, peak activity in the right lateral PFC activity seen in the contrast of False Shortcut Towards vs Shortcuts (Figure 3B), was not significantly different between ‘visible’ and ‘not clearly visible’ False Shortcuts Towards the goal (p>0.1), underscoring the notion that in order to correctly reject a false opening, participants did engage prefrontal areas.
Discussion
A core tenet of the cognitive map theory is that internal representations support flexible navigation, enabling an animal to make use of shortcuts and take efficient detours (O’Keefe & Nadel, 1978; Tolman, 1948). Despite the wide acclaim for this idea, little research, especially in humans, has been directed at the neural mechanisms of such adaptive behaviour (Spiers and Gilbert, 2015; Epstein et al., 2017). Using fMRI and a VR task involving navigation through a landscape that changed layout sporadically, we examined the neural responses to forced Detours, novel Shortcuts, False Shortcuts and Back-tracking. We found: i) superior and lateral PFC and caudate activity responded to Detours, maximally when there was a large change in the path, i) rlPFC responded when false shortcuts to the goal needed to be avoided and iii) a sub-region of anterior cingulate increased its activity during spontaneous Back-tracking in the context of de-activation of the regions associated with the default-mode network.
The role of prefrontal cortex, hippocampus and caudate in responding to detours and shortcuts
Based primarily on evidence from nine fMRI studies, Spiers & Gilbert (2015) provided preliminary predictions about how the PFC and the hippocampus might respond to forced detours and changes in the layout of an environment. Lateral PFC was suggested to provide a prediction error signal in response to changes in the path options (responding whenever an unpredicted change in the possible paths occurs). The superior and anterior PFC were speculated to support re-formulation of the route plan (responding at all events that require reconsidering the change in route plan). The hippocampus was postulated to simulate the future path the goal (responding the greater the increase in the path to the goal), drawing on rodent place cell studies (Ólafsdóttir, Barry, Saleem, Hassabis, & Spiers, 2015; Pfeiffer & Foster, 2013). Here we failed to find evidence that the hippocampus specifically encodes the change in the path distance to the future goal. One possibility is that the hippocampus simulates future possible scenes (Hassabis & Maguire, 2007), re-constructing the different locations that lie between the current location and the future goal (Javadi et al., 2017; Spiers & Barry, 2015). In the case of the current study the environment was sparse with few features to distinguish different parts of the island, which might explain why we did not observe a correlation between the hippocampus and the change in path to the goal. Notably, previous studies reporting hippocampal activity correlated with the future path to the goal used real-world stimuli with nameable landmarks located along the paths (Howard et al., 2014; Javadi et al., 2017; Patai et al., 2017).
By contrast to the hippocampus, we found that activity in lateral and superior PFC, as well as the caudate, responded maximally when there was a large change in the path to the goal. The caudate response is consistent with a prior result from Howard et al. (2014) which found that the larger the detour the more activity was elicited in the caudate nucleus. Thus, speculatively the caudate activity may relate to a signal linked to updating the transition structure in the environment at that particular location where the change occurs. An alternative is that the caudate plays an important role in updating its representations in relation to a model of the environment, consistent with this region coding a prediction error about future events (O’Doherty et al., 2004). Consistent with our caudate responses reflecting a model-based updating process, a previous fMRI study of navigation in a continually changing environment found that caudate activity correlated with parameters of a model-based representation of the environment (Simon & Daw, 2011).
The PFC responses we observed are in agreement with the predicted roles of the superior PFC supporting resolving path conflict and the rlPFC processing a prediction error signal between the predicted state of the world and the encountered layout (Spiers & Gilbert, 2015). Two types of prediction error could be processed in the current paradigm. One is the signed prediction error signal linked to the difference in the path before and after the change in the layout (+ve for detours, -ve for shortcuts). The other is an unsigned prediction error where the amount of change is coded rather than the direction of change (+ve for both detours and shortcuts). Our results show a wide network of regions including our PFC and caudate ROIs were driven in a manner consistent with the signed prediction error (maximal for +8 Detours). Our results thus align more strongly with models in which the PFC and caudate code the increase in path, and rather than being driven in a clear linear manner by the signed prediction error, the data suggest these regions might be driven in a threshold manner by large detours over the other conditions. Future research carefully varying along a broader range the amount of path change at detours will be required to explore these possibilities.
It is possible the PFC responses to Detours are driven by the presence of the physical barrier appearing to block the route. This is certainly a possibility in several past studies (e.g. Iaria et al., 2008; Maguire et al., 1998), though not all (see Howard et al., 2014) However, because rlPFC was more active for false shortcuts compared to shortcuts and these two events are visually similar (one unit of lava is removed to create a new path), it seems rlPFC is driven by planning demands rather than the visual processing of a barrier. The response is consistent with it playing a role in behavioural control: suppressing the pre-potent response to move towards the goal drawing on the observation that there is a now a barrier or that there is a new opening that is not helpful (Spiers & Gilbert, 2015).
Spontaneous Internally Driven Changes in Route - Backtracking
Across both externally driven (Detours, False Shortcuts) and internally driven (Backtracking) route re-evaluation events we found increased activation in the dorso-medial PFC, an area that has been implicated in various contexts including: hierarchical planning (Balaguer, Spiers, Hassabis, & Summerfield, 2016), high planning demand decisions (Kaplan et al., 2017), and model updating, irrespective of difficulty or simple error-related signalling (Kolling et al., 2016; O’Reilly et al., 2013). However, we found activation in the dorsal anterior cingulate cortex (dACC) was much more pronounced during Back-tracking, possibly reflecting stronger engagement of planning and error signals when participants spontaneously realized that they were on a sub-optimal path. This is further supported by the result that nearly 80% of Back-tracking events were usefully corrective, resulting in a shorter path to the goal. One previous study also found medial PFC to be correlated with a model-based estimate of behaviour relating to putative events of back-tracking in a simple maze (Yoshida & Ishii, 2006), and our peak activation for the dACC is highly proximal to the one previously reported in that study. This highlights the role of the medial PFC/dorsal anterior cingulate in situations in which re-evaluation and updating from internal monitoring are required, as is the case during introspective awareness (Critchley, Wiens, Rotshtein, Öhman, & Dolan, 2004), and reorienting and error detection (Wang, Ulbert, Schomer, Marinkovic, & Halgren, 2005)(Carter et al., 1998). It remains to be tested whether the underlying error signal comes from the medial PFC/dACC itself or alternatively this area is involved in orchestrating the reorienting of attentional and memory systems based on incoming uncertainty signals.
Additionally, when comparing Back-tracking to Non-back-tracking, Detours or Turn events, we found a disengagement from the putative default-mode network (including the medial PFC, hippocampi, posterior cingulate cortex), with an increase in right lateral-frontal activity, consistent the notion that internally generated route changes resulted in a global resetting of attention from internal to external sources (Sridharan, Levitin, & Menon, 2008). The default-mode network has been implicated during transitions between tasks/states and at task restarts (Crittenden, Mitchell, & Duncan, 2015; Fox, Snyder, Barch, Gusnard, & Raichle, 2005; Smith, Mitchell, & Duncan, 2018), when participants were ‘in the zone’ during a self-generated behaviour (Kucyi, Hove, Esterman, Hutchison, & Valera, 2017), and before an error is committed (Eichele et al., 2008). Given that Backtracking involves a spontaneous realization of an incorrect path and an update in route plans, reductions in default-mode network activity and activation of the frontal areas including the ACC imply that this behaviour is similar to scenarios in which novel stimuli or task rules activate the saliency network (Seeley et al., 2007). It will be useful in future research with navigation and such non-navigation paradigms to dissociate making novel responses from the novel responses which require an adjustment in the plan for future action.
Conclusion
Overall, our results help evaluate putative models of how the brain responds during back-tracking, shortcuts and detours. Future studies separating conflict between route options from prediction errors, search processes and updating representations will be needed to better understand how the brain adapts to changing environments. More importantly, exploring spontaneous human behaviour during naturalistic tasks such as navigation, including error monitoring and updating, could lead to more applications for cross-species comparisons of general problem-solving behaviour.
Footnotes
Conflict of Interest: None.