Behavioral evidence for memory replay of video episodes in macaque monkeys

Humans recall the past by replaying fragments of events temporally. Here, we demonstrate a similar effect in macaques. We trained six rhesus monkeys with a temporal-order judgement (TOJ) task and collected 5000 TOJ trials. In each trial, they watched a naturalistic video of about 10 s comprising two across-context clips, and after a 2-s delay, performed TOJ between two frames from the video. The monkeys apply a non-linear forward, time-compressed replay mechanism during the temporal-order judgement. In contrast with humans, such compression of replay is however not sophisticated enough to allow them to skip over irrelevant information by compressing the encoded video globally. We also reveal that the monkeys detect event contextual boundaries and such detection facilitates recall by an increased rate of information accumulation. Demonstration of a time-compressed, forward replay like pattern in the macaque monkeys provides insights into the evolution of episodic memory in our lineage. Impact Statement Macaque monkeys temporally compress past experiences and use a forward-replay mechanism during judgment of temporal-order between episodes.

slope patterns were observed irrespective of correctness. It has been shown in the humans that memory replay is not a straightforward (12). We propose two possible models with respect to whether the compression is during replay, the speed to initiate replay for Clip 2 would be sooner than the 2 3 5 endpoint of replay for Clip 1 due to the animal being able to skip over the entire for the Strict forward model when we divided the video into either 10 (P = 0.030) or  hypothetical RDMs (right). The reduction in RT indicated by an arrow between Clip 1 2 7 0 and Clip 2 is defined as "offset"; the magnitude of such "offset" is arbitrary but see  RDM (marginally insignificant r = 0.39, P = 0.069) than with the Strict forward model iterations of randomization. P values are FDR-corrected (*** denotes P < 0.001). We defined the reduced RT to initiate replay for Clip 2 as "offsets" in initiating 2 8 6 search in Clip 2 by skipping the non-informative Clip 1 ( Figure 3A). With respect to 2 8 7 the detailed differences between the two models, one may wonder whether and how 2 8 8 the "offsets" between Clip 1 and Clip 2 might influence the results. Especially for the 2 8 9 Global compression model, changes of this parameter will cause changes in the 2 9 0 RDMs. To address this concern, we simulated an array of RDMs by systemically Strict-forward model (model 6, middle in Figure 4A), and beyond (7 th to 11 th models, 2 9 4 right in Figure 4A). We then tested each individual monkeys' data with each of these as insets). These suggest that the monkeys have processed the video as a holistic   and RDMs (bottom). We systemically varied the "offset" parameter while keeping a and beyond (7 th to 11 th models, right). The numerals below the RDMs denote the human participant's data is also tested against each of these 11 hypothetical models.

2 7
The Spearman correlations decrease as a function of offset magnitude between Clip 1 3 2 8 and Clip 2 until reaching an asymptote when the offset value is around zero. This confirms the hypothetical discrepancy between the two species (see also Figure 3B). segments. Then, we transformed the vectors "offset" and "slope of RT/chosen frame 3 3 5 location" into two arrays and displayed them as three-dimensional mesh/surface plots. In Figure 5, in addition to "offsets", we showed that the correlation between model and monkeys' data also increase as the slope of RT/chosen frame location increases.

8
In contrast, the correlation values for human data seem to be driven by the "offsets" (cf. Figure 4C), and in terms of "slope" going in an opposite direction from the cross-species differences (e.g., by contrasting Figure 1C vs. 1D). Human participants data.

Context changes (event boundary) increase rate of rise in decision information
We have thus far focused on how the monkeys retrieve the order of frames when information was equated within contexts, but how contextual changes would aid temporal order judgement processes remains to be examined. It was evident that the with Ergodic Rate (LATER) model ( Figure 6A). We compared across-context and null model, which assumes there is no effect of manipulation. Using the Bayesian  accumulate information for memory decisions at a faster rate when the frames were 3 7 3 extracted from two different clips than for frames that were extracted from the suggest that the monkeys adopt a forward search for targets among linearly ordered 3 7 7 memory traces and they reach their memory decision threshold more quickly when 3 7 8 probe frames are extracted from the two different contexts (across-context condition). within-context trials respectively. In order to verify whether the effects are not attributed to the basic stimulus 3 9 6 features such as perceptual differences inherent in the across-context condition. We which the monkeys could capitalize on using contextual boundaries for TOJ judgment. The within-context GLM shows that monkey's RT was indeed significantly faster when 4 0 7 the probe frame was located earlier in the video, P < 0.001 (or in equivalent terms, the monkeys adopt a forward scanning strategy for information retrieval (Figure 7 left).
In contrast, the across-context GLM shows that there was not any significant effect of  In light of recent reports on neural correlates underlying how humans and 4 3 5 rodents reply their past experiences (11, 12, 14, 18), here, we demonstrate parallel clarifies that this replay process is conducted in a time-compressed manner.

5 2
Despite the cross-species similarity, our revelation of a critical discrepancy representations to allow components to be recombined in more ways than were  patterns in the monkey during TOJ, we are aware that replay is a neural phenomenon sequences of hippocampal place cells that reflect past and future trajectories (28-30).

7 7
However, by establishing that neither the number of intervening frames nor passage of  We observed one further interesting feature here that the monkeys are able to shown in humans (33-36) and rodents (37).

9 2
Memory replay is an elaborate mental process and our demonstration of a  The six monkeys performed the task with a significantly above chance level with 5 0 3 an overall accuracy at 67.9 ± 1.5% (mean ± SD) and with an above chance accuracy 5 0 4 for within-context trials, t5=14.35, p < 0.001. The human participants performed the 5 0 5 task on average at 92.7 ± 1.2%. routinely offered treats such as peanuts, raisins and various kinds of seeds in their home 5 1 7 cage for forage purpose. The monkeys were procured from a nationally accredited light-dark cycle and kept within temperate around 18-23°C and humidity of 60-80%.

2 3
Human subjects. Seven participants (mean age = 19.57 ± 1.13, 6 female) took part in the experiment. The participants were recruited from the undergraduate population in East China Normal University. The participants provided informed consent and were 5 2 6 compensated 400 RMB for their time. We used the 6 unique video-trials sets and TOJ frames (one unique set per monkey) correspondingly for the human subjects (subject 7 5 2 8 re-used set 1).

2 9
The experimental protocol was approved by the Institutional Animal Care and Use the "NIH Guidelines for the Care and Use of Laboratory Animals". The testing was conducted in an automated test apparatus controlled by two 5 3 7 Windows computers (OptiPlex 3020, Dell). The subject sat, head-unrestrained, in a distance between the subjects' head and the screen was kept at ~20 cm. The Co., Ltd., China) allowed the subject to be monitored while it was engaged in the task.

4 8
The entire apparatus was housed in a sound-proof experimental cubicle that was dark 5 4 9 apart from the background illumination from the touch screen.

2
A collection of documentary films on wild creature was gathered from YouTube.

3
The films were Monkey Kingdom (Disney), Monkey Planet (Episode 1 -3; BBC), (PBS Nature). In total 28 hours of footage was gathered. We applied Video Studio X8 (Core Corporation) to parse the footage by camera-cuts into smaller segments.

7
Experimenters then applied the following criteria to manually edit out ~ 2500 unique experimental condition would not be used/shown in another condition so that a 5 6 6 particular monkey will only view that video repeatedly under the same condition 5 6 7 across exposure. Task and experimental procedure 5 7 0 We combined naturalistic material with a temporal order judgement paradigm that water), and following a 2-s retention delay, two frames extracted from the video were frame that was shown earlier in the video (see Movie S1). A touch to the target frame 5 7 7 resulted in 1.5 ml water as reward, removed the foil frame, and the target frame would frames from the screen and blanked the screen for 20 s without water delivery. Since 5 8 0 the monkeys could self-start the trials, we did not set an explicit inter-trial interval.

8 1
Correction trial procedures were not used in the main test.

8 2
We collected 50 daily sessions of data. Each session contained 100 trials, giving us design. The two frames for TOJ could be extracted from same clip (Within) or distinct