Anticipatory brain activity depends on the sensory modality of the expected stimulus: insights from single-trial analyses

Perceptual anticipation is known to impact the reaction time of decisions. While anticipatory patterns have been identified in human brain activity, the single-trial neural signature of anticipation remains unexplored. Similarly, past studies have underlined an effect of pre-stimulus alpha-band activity on reaction times. Still, it remains unknown whether this activity is stimulus-specific or rather acts as a general indicator of readiness. This study aimed to determine whether human participants expected a visual or an auditory stimulus at the single-trial level in both cued and uncued trials. We show that pre-stimulus brain activity entails information about the expected upcoming stimulus, and that the information content can be extracted from single-trial brain activity. Behavioral analyses helped uncover the link between correct anticipation and shifts in decision strategy, and additionally validated the classification of uncued trials. This understanding of single-trial stimulus-specific neural signatures of anticipation can significantly impact cognitive neuroscience, human-computer interaction, and neuroergonomics research, enabling the development of real-time systems capable of predicting and adapting to an individual’s anticipated stimuli, thereby enhancing task performance and user experience in diverse applications ranging from adaptive interfaces to clinical interventions. Novelty and Significance With a focus on uncovering single-trial neural signatures of anticipation, this study aims to explore the specificity of pre-stimulus brain activity to expected visual or auditory stimuli in both cued and uncued trials, i.e. whether participants are informed about the upcoming stimulus or not. Our research seeks to contribute to the fundamental understanding of human cognition by elucidating the informational content embedded in anticipatory brain patterns. The insights gained hold promise in advancing fields like cognitive neuroscience and human-computer interaction, paving the way for innovative applications in adaptive interface design, real-time systems, and personalized interventions.


Introduction
Perceptual anticipation in decision-making shapes behavioral responses by allowing individuals to prepare for forthcoming stimuli (Poulton, 1950).Correctly anticipating a stimulus typically results in faster and more accurate responses (Petro et al., 2019;Poulton, 1950).Moreover, pre-stimulus brain states impact judgement confidence (Samaha et al., 2017) and stimuli perceptibility (Railo et al., 2021).Despite its critical role, exploring pre-stimulus brain states, particularly anticipation, remains to be developed due to methodological challenges.
Traditional analyses align brain activity to specific events, posing significant hurdles when capturing anticipatory processes that manifest at undefined time points before stimulus onset.While aligning all trials to a consistent point in time is complex with well-identified brain activity (Barthélemy et al., 2013), it becomes highly challenging to interpret complex single-trial patterns.Moreover, the unconscious nature of anticipation (Koch and Preuschoff, 2007) complicates the accessibility to the ground-truth knowledge of what was expected, as direct participant questioning might alter neural activity (Trevena and Miller, 2002).Electroencephalography (EEG) emerges as a favored neurorecording technique due to its fine temporal resolution and ability to capture conscious and unconscious cortical activities.To circumvent the challenges above, researchers have additionally employed paradigms utilizing cues to induce specific expectations and pinpoint their onset (Chavarriaga et al., 2012;Petro et al., 2019).However, using cues may only reflect idealized scenarios, as real-world decisions are often uncued, presenting a gap in ecological validity.
A neuronal component of expectation, called the Contingent Negative Variation (CNV), has been extensively studied.The CNV is an event-related potential (ERP) that emerges after a cue announcing a target stimulus is displayed.Upon several presentations of this association of stimuli, the CNV appears, with temporal jitter weakening it (Walter et al., 1964).Initially identified as a neural marker of temporal expectation, the CNV was later associated with stimulus expectation and motor preparation (Chavarriaga et al., 2012;Garipelli et al., 2011Garipelli et al., , 2009;;Khaliliardali et al., 2015Khaliliardali et al., , 2012)).In Go/NoGo tasks, anticipation was successfully classified with above-chance accuracy (Garipelli et al., 2009), revealing anticipatory brain activity patterns among half of the participants (Khaliliardali et al., 2015).While these studies focused on discerning whether participants expected to take action, decoding the semantic content of anticipation at the single-trial level, particularly its dependence on the anticipated stimulus, remains an open avenue for exploration.
Besides the CNV, pre-stimulus alpha band activity (8 − 13 ) has attracted attention for its presumed role in sensory information gating (Jensen and Mazaheri, 2010).Linked to top-down processing (Min and Herrmann, 2007), its precise influence on decision-making remains unclear.
Studies correlating alpha activity with confidence ratings, albeit not perception, in visual (Iemi et al., 2017;Samaha et al., 2017) and auditory (Wöstmann et al., 2019) tasks suggest its potential modulatory effect on higher cognitive functions.Conversely, other studies have implied plausible connections between alpha activity and sensory encoding (Lou et al., 2014) and decision formation (Barik et al., 2019).Despite these insights, whether alpha-band activity serves as a specific neural correlate of anticipation remains unknown.
This work shows the possibility of distinguishing between anticipation of visual and auditory stimuli at the single-trial level.We implemented an EEG experiment wherein participants engaged in a discrimination task between visual and auditory stimuli, cued in one experimental session, revealing the existence of anticipatory processes.The core analysis of this study involved differentiating EEG activity depending on participants' expectations (sound or image) using the CNV or the pre-stimulus alpha activity of cued trials only.Classifiers trained on cued data were subsequently applied to uncued trials, successfully deciphering the semantic content of anticipation even without explicit cues.To understand which stage of decision formation anticipation modulates and assess the classification performance on uncued trials, we related the classification performance and predictions to behavioral modelling parameters (Ratcliff, 1978).This work marks a substantial stride toward understanding the dynamics of anticipation and its influence on decision-making processes.

Materials and methods
Participants 42 participants (24 males, 18 females; 5 left-handed; aged 20 − 64, mean: 30.43 ± 10.78) with normal or corrected-to-normal hearing and vision took part in this study.The experiment was approved by the Comité d'Ethique de la Recherche Paris-Saclay, under the application number 321.
Each participant was informed about the purpose of the study and signed informed consent forms upon participation.All participants fully completed the experiment.

Experimental design
The experiment consisted of two parts, denominated "test" and "calibration" phases thereafter, which the participants completed within the same recording session.During the test phase, participants were presented randomly at each trial with either the sketch of a face ("face" trial) or a sound ("sound" trial).They were instructed to respond as fast as possible to each stimulus by pressing with their dominant hand on the right arrow of a keyboard for "face" stimuli and on the left arrow for "sound" stimuli.Each trial started with the apparition of a red cross in the middle of the screen, indicating a 1.5  rest period to the participant.The cross then became white to instruct participants to start focusing on the task and avoid parasitic movements that can blur EEG signals (blinking, jaw and head movements in particular).This baseline period lasted for 1.5 − 3 , with its duration varying randomly across trials.After that, a square visual noise clip appeared for 0.9  in the middle of the screen, frames being updated at a rate of 10 .While the last frame remained on the screen until the end of the trial, the stimulus was displayed for 200  at the end of the noise clip.
The trial terminated upon participant response or after a timeout of 2  after stimulus onset.
Trials in the calibration phase were identical to those in the test phase, with the addition that a cue appeared for 200  at the beginning of the noise clip, indicating which stimulus would be displayed.A pictogram of an eye indicated a "face" trial, and an ear icon indicated a "sound" trial.To eliminate the possibility of having created a delayed response instead of sensory anticipation, we set the cues to be inaccurate in 20% of the cases, excluding the ten first trials.We chose this catch probability according to the threshold at which oddball paradigms function, hence ensuring that the cue was still reliable while introducing uncertainty and surprise among subjects (Chavarriaga et al., 2014;Ehrlich and Cheng, 2018;Noah et al., 2020).A summary of the trial sequence for these two phases is given in Figure 1.
Participants performed 3 test and 4 calibration blocks of 60 trials each, resulting in 180 test trials and 240 calibration trials.20 training trials preceded the test phase, and 60 training trials were performed before the calibration phase.The training phase preceding the calibration phase was longer so that participants had time to learn the impact of the cue, the correct response to each target stimulus, and to avoid making mistakes in catch trials.Half of the participants performed the test blocks first, while the other group of participants started with the calibration blocks to prevent any influence of the order of the phases.Participants could take a break between each block and could decide when to continue with the next one.The experiment was performed in a dark room to enhance the sight of the visual stimulus.

Figure 1
Summary schematic of the sequence of a trial.Each trial starts with a 1.5  rest period, followed by a 1.5 − 3  baseline period, during which participants are instructed to limit movement and focus on the screen.The prestimulus period then lasts for 0.9 , a period during which visual noise (refresh rate of 10 ) is presented.In the "cued" condition, the cue is presented during the first 200  of this period.Then the stimulus, either an image of a face or a sound, is presented for 200  in addition to a single frame of visual noise which lasts until the end of the trial.Participants are instructed to report the perceived stimulus as fast and accurately as possible by pressing on the left arrow for a sound and the right arrow for a face stimulus.The trial is terminated upon participant's response or after a 2  timeout.

Stimuli
Target stimuli consisted of a sound pulse at 1000  for the sound stimulus, and a sketch of a face for the visual stimulus.The sketches were generated by Yang et al. (2020) from the Radboud Faces Database (Langner et al., 2010).Cue stimuli consisted of an open-source drawing of an eye and one of an ear, displayed at the center of a screen over a random dot square cloud.All stimuli lasted for 200 .

EEG data acquisition and pre-processing
EEG signals were recorded at 1000  using 32 active AgCl electrodes, and the actiCHamp Plus amplifier (Brain Products GmbH, Gilching, Germany).The electrodes were placed according to the 10/20 international system.Electrode  served as the reference electrode upon acquisition.
To avoid contamination by stimulus-related activity, we applied offline a non-causal finite impulse response band-pass filter with cut-off frequencies 0.1 − 35  to remove high-frequency artifacts (muscle activity in particular) and low-frequency artifacts, such as sweat.Data was subsequently epoched on the relevant periods of interest (see below).

EEG data comparisons: periods of interest
We wanted to see if we could distinguish the ERP patterns between anticipation versus baseline, anticipation and no anticipation (i.e., pre-stimulus period with and without a cue), and visual versus auditory anticipation.The pre-stimulus period was defined as the window [−400: 0]  preceding the target stimulus for both stimuli and in both cued and non-cued conditions.It can equivalently be defined as the [500: 900]  time window following the onset of the noise clip and cue. 2 comparisons were performed: anticipation vs. no anticipation (defined as cued vs. uncued trials), and visual vs. auditory anticipation.

Computing ERP components
For the initial investigation of significant differences between the different periods of interest described above, ERP components were computed by averaging trials within each category (cued trials, uncued trials, face anticipation, and sound anticipation).A comparison of the grand average was performed, averaging the signal both across trials and channels within participants.Note that only the 0.1 − 35  bandpass filter was applied for this analysis.

Classification pipelines Classifying ERPs using covariance matrices
The CNV is a candidate for discriminating the type of anticipation at the single-trial level.To use it as a feature for classification, we first applied a 1 − 4  band-pass filter on raw calibration data and epoched it from 500 to 900  after noise clip onset, corresponding to the last 400  before stimulus onset.we used it as a feature to classify single trials according to the type of anticipation (face vs. sound anticipation).Trials were then split into 10 stratified folds, using cue labels for stratification to ensure that the label distribution is preserved in each fold.For each cross-validation round, 9 of these folds were used as the train set and the remaining one as the test set, so that each fold was used once as the test set.Using the PyRiemann Python toolbox (Barachant et al., 2023;Congedo et al., 2017), extended covariance matrices (Barachant and Congedo, 2014) with XDawn components (Rivet et al., 2009) were computed for each class using the train set, selecting 8 components, and the Minimum Distance to Mean was used to assess the class of each trial.Using covariance matrices for EEG activity classification was proven efficient in improving classification accuracy on ERP data (Barachant and Congedo, 2014) and motor imagery (Barachant et al., 2013(Barachant et al., , 2012)).The classification accuracy was then computed for each test set and then averaged across test sets within participant.As pre-stimulus alpha-band activity has also been shown to be predictive of behavior in other works (Barik et al., 2019;Bode et al., 2012;Iemi et al., 2017;Lou et al., 2014;Min et al., 2008;Min and Herrmann, 2007;Petro et al., 2019;Railo et al., 2021;Samaha et al., 2017;van Dijk et al., 2008;Wöstmann et al., 2019), we repeated this analysis with a [8 − 13]  band-pass filter.

Classifying pre-stimulus activity of uncued trials
The afore-described classification pipelines are based on supervised learning, i.e. they require the prior knowledge of a ground-truth.In this study, it requires a prior knowledge of the type (visual or auditory) of anticipation.However, the ground-truth label of anticipation is unknown on trials of the test phase of the experiment, i.e. on uncued trials.To circumvent this issue, two solutions are proposed here.First, if we assume the stationarity of the anticipation phenomenon across tasks, the brain activity patterns of anticipation over uncued trials should be similar to those found on the cued trials.Therefore, classifiers trained on cued data can be applied on uncued data of the same participant.We hence re-trained the classifiers obtained on all the trials of the calibration phase of the experiment for each participant, and predicted the corresponding anticipation class using the thus trained classifier on uncued trials.
The second solution consists of analyzing the post-stimulus effects of anticipation.Indeed, challenged expectations should result in potential variations in the post-stimulus period (Kappenman and Luck, 2011;Lou et al., 2014).Consequently, detecting mismatch signals in the post-stimulus ERP should indicate the type of anticipation produced by the participant in the pre-stimulus period, hence providing "ground-truth" labels on the type of anticipation on uncued trials.We therefore trained a classifier to detect incorrect anticipations in the post-stimulus period of the cued trials, using the same pipeline as described in "Classifying ERPs using covariance matrices".The ROC curves were plotted for each participant to verify the correctness of the classification, then the classifiers were trained again over all cued trials and were subsequently used to detect incorrect anticipations on uncued trials.The anticipation labels were then obtained by combining the stimulus labels and the labels obtained from this pipeline.These labels were then used as the cue labels in the pipelines described earlier.
The different pipelines used to generate the classes of anticipation on both cued and uncued trials are summarized Figure 2. In the following, we will refer to the anticipation on cued trials as "ground" anticipation when the cue label is used as the anticipation class, or as "classifier" when the anticipation class is the one predicted by the pre-stimulus classifier.On uncued trials, we will refer to "cued-pre-stimulus" anticipation when the anticipation is the one predicted by the aforementioned classifier trained on cued trials, or to "post-stimulus" when this anticipation is the one predicted by the pre-stimulus classifier trained using the labels predicted by the post-stimulus classifier trained on cued data and subsequently applied to uncued trials.

Linking brain and behavior
One of the main assumptions of the effects of anticipation on behavior is the reduction of response times and error rates subsequent to correct anticipations.If the classification algorithm was indeed trained to distinguish the type of anticipation, we should observe that correct anticipations are more represented among shorter reaction times relative to long ones.Conversely, incorrect anticipations should result in longer reaction times.

Response time quantiles
We first split the predicted anticipations depending on their correctness and behavioral outcome: the anticipation can either match the stimulus or not, corresponding to correct and incorrect anticipations respectively, and the response can match the stimulus or not, corresponding to correct and incorrect responses respectively.In addition, the trials were split into three groups depending on their response time: for each participant, we computed the 33 ℎ and 66 ℎ percentiles of response times and hence split the trials into short, medium, and long response times.
We then studied the distribution of each of the four pairs of anticipation+response outcomes across response time quantiles.This was done separately for cued and uncued trials in order to compare the results on uncued trials to the more reliable ones over cued trials.

DDM fitting
To analyze the effects of anticipation on behavior further, we fitted diffusion decision models (DDMs) (Ratcliff, 1978;Ratcliff and McKoon, 2008;Ratcliff and Tuerlinckx, 2002) to behavioral data.
According to DDMs, sensory evidence is accumulated linearly from a starting point   until reaching a fixed decision boundary a, at which time a decision is made.
The DDM is defined by the equation: where the decision variable  varies by  in infinitesimal time .The decision state can be viewed as a particle subject to Brownian motion with drift , Gaussian white noise term ().Additionally, a non-decision time  0 is fitted to account for the biological delay of sensory encoding and motor preparation explaining the difference between the decision time and the observed response time.
Our DDM analysis aimed at identifying whether changes in anticipation resulted in changes in the accumulation rate , or rather in a bias in the starting point   .More specifically, correct anticipation resulting in faster and more accurate decisions than incorrect anticipation, one could expect that correct anticipation either increases the rate of evidence accumulation, or that the decision is initially biased towards the correct decision.
On cued data, two hypotheses were tested: -Correct anticipations should result in larger drifts or larger starting points than incorrect anticipations -Participants who displayed above-chance classification performance should also display a larger parameter difference between correct and incorrect anticipations To test these hypotheses, we fitted a DDM for each participant, fixing the boundary  = 1 and nondecision time  0 = 0.3  for all participants.This was necessary in order to compare model parameters across participants, since these parameters are interdependent with the drift and the starting point.For each participant, both the drift term and the starting point were fitted separately depending on whether ground-truth anticipation was correct or incorrect.Starting point, drift, and non-decision time variability were additionally fitted for each subject, but not further analysed.The software fast-dm (Voss and Voss, 2007) was used, using the Kolmogorov-Smirnov fitting method.
To test hypothesis 1, we performed paired-sample -tests to assess whether the drifts and starting points significantly depended on the correctness of anticipation.
To test hypothesis 2, we followed two complementary approaches.First, we performed independent sample -tests to assess whether the parameter difference was significantly greater on participants displaying above-chance classification accuracies.Second, we performed a correlation analysis to assess a possible linear dependency of classification accuracy on either of the parameter differences.
The same DDM fitting was performed on uncued data, using this time the anticipation class predicted by either classifier (i.e., using "cued-pre-stimulus"-and "post-stimulus"-based labels) to assess the correctness of anticipation.This time, we assessed whether one method provided a greater parameter difference than the other.Based on the DDM analysis of cued trials (see Results), this measure would indicate a better classification performance from one classifier or the other.

Behavioral effects
We first tested the effects of the presence of the cue and the stimulus on the response time and accuracy at the group level using a repeated measures ANOVA procedure, with "Stimulus type" (auditory, visual) and "Condition" (cued, uncued) as within-subject factors.We further tested the effect of congruent and incongruent cues and stimuli on the response time and accuracy on the cued condition only, using repeated measures ANOVA again, with "Stimulus type" and "Cue congruency" as within-subject factors.Note that the tests were done separately in each condition due to the unbalanced number of trials.

Significance of ERP differences
To test the group-level effects of anticipation on time series, we performed cluster-based permutation testing.The idea of this type of testing is explained in (Maris and Oostenveld, 2007), and the methodology is summarized here.This analysis aims to test the statistical significance of effects appearing in ERPs, all while correcting for multiple comparisons.Since the number of tests to perform is very big (901 time points, or pixels), classical corrections such as the Bonferroni correction are not adapted as they would result in meaningless reference thresholds.
-statistics of each point are computed to identify clusters of (uncorrected) significant activity.The pre-cluster threshold is set to  < 0.05.The signals are then shuffled (i.e., in the case of the comparison of cued and uncued signals, the attribution of each signal to the "cued" or "uncued" category is randomized.Similarly, the assignment to the "auditory" and "visual" categories is randomized at this stage when comparing auditory and visual anticipation) and the statistics are computed again on the permutation.The statistics of the bigger cluster thus obtained are stored, and the whole process is repeated over several iterations to form the null hypothesis distribution, against which all the clusters identified in the original signals are tested.
We performed 1000 permutations.This number allowed us to obtain stable null hypothesis distributions while remaining time-efficient.

Significance of classification performance
We compute a mean classification score from the -fold cross-validation for each participant and pipeline.The classification scores are compared to chance level that we obtained through permutation testing over 100 iterations.At each iteration, the labels are shuffled, and the classifier is re-run to compute the classification accuracy over the randomly labeled data.The null-hypothesis distribution of the chance level classification performance is thus obtained for each participant, and the classification performance is compared against this distribution.The -value of the classification accuracy against the null-hypothesis distribution is computed for each participant and further compared across all participants by performing one-sided paired samples  − tests of the accuracy against the individual chance level 95% confidence interval.The accuracy indeed needs to lie above the individual chance level to consider that the classification succeeded.

Results
We have analyzed EEG recordings from 42 participants who performed a sensory categorization task.
At each trial, participants had to decide whether the stimulus they were randomly presented with was a sound or a visual stimulus consisting of a drawing of a face.The trials were preceded by a prestimulus period of 0.9 seconds, and on some trials a cue was presented at the beginning of this prestimulus period, indicating with 80% confidence the class of the upcoming stimulus.In that case, the pre-stimulus period was called the anticipation period.
Our analyses aimed to characterize the brain activation patterns related to specific stimulus anticipation at the group level and at the single-trial level.

Behavioral results
The descriptive statistics of response times and accuracies for each condition and stimulus type are summarized on Figure 3 (A and B).We first tested the effects of the stimulus and the condition (i.e. presence or not of an informative cue) on the response times and accuracy at the group level using ANOVA.We observe a significant effect of the stimulus (  = 526,   = 462,  {1,41} = 119.783, < 0.001) and the condition (  = 470,   = 526,  {1,41} = 70.304, < 0.001) on the response time.We also note an interaction effect between stimulus and condition ( {1,41} = 30.024, < 0.001), with responses of cued visual trials faster than any other trial type ( < 0.001), and uncued visual trials significantly faster than uncued auditory trials ( = 12.236,  < 0.001), but not than cued auditory trials ( = 1.058,  = 0.293).
However, the accuracy was impacted by neither the stimulus type nor the presence of the cue (see Figure 3B).Note that there is no interaction effect between the two factors on the accuracy.These results show that the presence of the cue effectively reduced the response times (see also Figure 3A).
Then, we tested the effect of a congruent or incongruent cue and the stimulus on the response times and mean accuracy (the descriptive statistics can be found in Figure 3C and 3D) within the cued condition using ANOVA.We observed that incongruent trials had a significant effect on both the response time (  = 452,   = 558,  {1,41} = 131.890, < 0.001) and the accuracy (  = 99.1%,  = 91.1%, {1,41} = 31.140, < 0.001), and observe as previously that the stimulus type had an effect on the response times (  = 496,   = 445,  {1,41} = 67.009, < 0.001, see also Figure 3C) but not on the accuracy (Figure 3D).We observed no interaction effect on either the response times or the accuracy.These results show that the cue was considered informative by the participants.Together, these results support the emergence of anticipatory effects after the presentation of an informative but partially unreliable cue, which validates the experimental paradigm implemented here.

Anticipatory activity is distinct from temporal expectation
We first wanted to show that the anticipation activity is distinct from temporal expectation of the coming event.Therefore, we compared at the group level the brain activity in the anticipation period to the activity in the pre-stimulus period of uncued trials.
The first one corresponds to the visual-evoked potential due to the presentation of the cue and is not relevant for our next analyses because it is not related to anticipation but rather to the sensory processing of information.The second one, however, is quite similar to the Contingent Negative Variation (CNV) that appears during temporal expectations.Since this signal is significantly different from the uncued one, where little perceptual anticipation can be made, and temporal expectation should prevail, it indicates that the content of anticipation shapes the related brain activity.

Discriminating anticipation-specific neural signatures
We first tried to test whether the pre-stimulus ERP could be distinguished at the group level between visual and auditory anticipation, that is, whether the pre-stimulus ERP differed at the group level when an eye and an ear cue were presented.As shown in Figure 4B, we observed no significant difference between the two ERPs.
Qualitatively, however, we note a difference between the two grand averages, which indicates that some participants may display stronger differences between the two types of anticipation.The topographies also present some differences (Figures 4B).We note a difference over the central areas, suggesting a difference between the CNVs and indicating that the CNV holds information about the type of expectation, but also a difference in the parietal-occipital electrodes.This difference might be due to a pre-activation of sensory areas in preparation of the upcoming stimulus.

Single-trial decoding of anticipation
Decoding anticipation from the CNV is feasible on some participants on cued trials The core test of this study was to determine whether single-trial anticipation could be classified.In the context of our experiment, we hypothesized that participants would display differences in their CNV component depending on whether they expected a sound or a visual stimulus.To this aim, we filtered the signal to keep frequencies between 1 and 4 Hz, hence keeping only the lower frequency signals that characterize the CNV.As a first step, we only considered cued trials, as they allow for direct computation of the classification accuracy.
The table of classification performances is shown in Table 1.Out of the 42 participants, 25 displayed classification accuracies, as computed from a 10-fold cross-validation procedure, above their individual empirical chance level.At the group level, the classification accuracy is significantly above chance level (mean ± standard error of the mean: 57.6 ± 0.7% accuracy, 56 ± 0.1% chance 95% threshold; one-sided paired-sample  − test between classification accuracy and individual empirical chance levels: (41) = 2.188,  = 0.017, Cohen's  = 0.338).
As the CNV appears after several presentations of the cue-stimulus association, we assess the effect of the condition order on the classification performance.We observed no significant difference between the two groups in the classification accuracy (Welch -test,  = 0.341,  = 40).This result was expected since all the participants received 60 training trials prior to the cued condition.
Table 1 Classification accuracy obtained from the covariance matrices of the 1 − 4-filtered pre-stimulus activity of cued trials ("accuracy"), compared to the empirical chance level ("chance (95%)").The -value corresponds to the position of the classification accuracy relative to the empirical distribution of chance level, obtained by permutation.

Alpha-band activity for estimating the single-trial anticipation class
Following the results of past studies, we also assessed whether the alpha-band power spectral density could characterize anticipation at the single-trial level.Using the same classification pipeline as for the CNV classification, this time filtering the raw signal between 8 and 13 , we found that 10 participants showed above-chance classification accuracies, representing again a smaller proportion of all participants compared to the classification of ERPs.Moreover, the group-level accuracy lies within chance-level (mean ± standard error of the mean: 54 ± 0.8% accuracy, 56.3 ± 0.1% chance 95% threshold, one-sided paired-samples  − test: (41) = −2.741, = 0.995, Cohen's  = 0.423).
Given the better results obtained using ERP classification, the analyses performed on uncued trials are using the pipeline described in the ERP section.

Towards decoding single-trial anticipation on uncued trials
On uncued trials, we do not have the ground truth about what the participant expected.Therefore, a behavioral proxy for anticipation is necessary to check the validity of our classification.
From the behavioral analyses, we found that incorrect anticipation led to longer response times, as seen in the effect of catch trials on the response times.Therefore, we assume in the following that incorrect anticipations lead to longer correct responses, while correct anticipations lead to shorter reaction times on correct responses.
Uncued trials were classified for each participant using two pipelines.We first describe how poststimulus activity was used to infer a ground truth for the state of anticipation.We additionally applied the classifier trained on the covariance matrices of ERPs of cued trials.

Using post-stimulus brain activity to decipher anticipation
Post-stimulus activity holds information about the prior expectations about the stimulus.We therefore trained a classifier to detect when the anticipation did not match the stimulus, using the same pipeline as previously.
The classification performance on cued trials was significantly above chance level (mean AUC across participants: 0.79 ± 0.12), with trials of one participant classified under chance level.
Given the satisfactory results of the classification on cued trials, the classifiers were trained again on all the cued trials for each participant and applied to uncued data.The labels thus obtained were converted to anticipation labels, themselves used as the ground-truth labels of anticipation for classification.Applying the same pipeline as described earlier on CNV classification, we computed a classification score for each participant and then combined the -values across participants using the Fisher method.The classifiers yielded above-chance classification accuracy for 5 participants, which was not significant at the group level (one-sided paired-sample  −test, (41) = 7.064,  = 1).
Combining the  −values indicated that an effect may have nevertheless been captured by the classifiers (Fisher method,  2 = 121.1, = 0.005).
We expected a predicted correct anticipation rate close to 50% for all the participants.However, we observed that participants anticipate the trials correctly 31 ± 5% (mean±standard deviation) of the time, which is significantly different from 50% (one-sample -test, (41) = 27.6, < 0.001).
In the following, we will refer to this pre-stimulus classification pipeline as "post-stimulus"-based.

Using the CNV classifier trained on cued trials
While further qualitative assessment of the classification performance is given later in the "Brain and behavior" results section, the rate of predicted-correct anticipations can quantitatively indicate the quality of the fit, as we expect again that around 50% of the trials are correctly anticipated.We observed that 53 ± 4% of the trials are correctly anticipated by each participant on average, according to the classifiers.This value also significantly differed from 50% (one-sample -test, (41) = 4.746,  < 0.001).Note that in that case, individual chance levels could not be computed because we used pre-trained models, and have no ground for comparison to an individual chance level.
In the following, we will refer to these classification results as "cued-pre-stimulus"-based.
We observe in that the classification fails at the group level with both algorithms.This could indicate either a failure of the post-stimulus classification or a failure of this pre-stimulus classification, although it is not possible to conclude on that based on the results presented above.The next analysis, whose results are presented thereafter, compared the classification predictions to behavioral observations to substantiate these results.

Behavioral predictions
Behavior in decision-making tasks is characterized by the mean response time of correct and incorrect responses and the response accuracy.The previous analyses yielded four different anticipation determination method: on cued data, we either used the cue as the class of anticipation (the "ground" method) or the label predicted by the classifier (the "classifier" method).On uncued data, we either trained the classifiers on pre-stimulus uncued data using labels predicted on poststimulus activity (the "post-stimulus" method) or used the classifiers created during the "classifier" method trained on cued data to generate the anticipation labels of the uncued trials (the "cued-prestimulus" method).For each anticipation determination method, we therefore computed these quantities for each participant separately.The mean and standard deviation of these values across participants are presented in Table 2.
Table 2 Mean response times on correct and incorrect responses (in ) and response accuracy (in %) depending on the correctness of the anticipation and the anticipation determination method.In each cell, the mean within participant and then across participant ± the standard deviation across participants is presented.On cued trials, the "Ground" anticipation determination method corresponds to the hypothesis that the cue is the anticipation class, while the "Classifier" method means that anticipation labels are the ones returned by the CNV classifier.On uncued trials, the anticipation label is always obtained from pre-stimulus CNV classification.
The "Post-stimulus" method is the one where the classifier is trained on uncued data using the labels obtained by the post-stimulus classification, while the "Cued-pre-stimulus" is the method where the classifier was trained on the pre-stimulus activity of cued trials and then applied to uncued data.
We additionally performed repeated measures ANOVAs on the response times and response accuracy, taking Response (correct or incorrect), Anticipation (correct or incorrect) and Anticipation determination method (Ground, Classifier, Post-stimulus or Cued-pre-stimulus) as within-subject factors.On response times, we observed a significant impact of anticipation ((1,41) = 10.510, = 0.01), as well as an interaction effect of anticipation and response ((1,41) = 8.012,  = 0.02).

Distribution of anticipation and behavioral outcomes in response time quantiles
For each participant, the trials were split into three groups depending on their tercile of response times.These terciles were computed individually, so that each participant had the same number of short, medium, and long response times.We then also assessed whether trials were anticipated correctly (i.e., if the anticipation and stimulus labels matched) and if they were responded to correctly (i.e., if the stimulus and response labels matched), yielding four pairs of possible anticipation-behavior outcomes.
Figure 5 represents the distribution of anticipation-behavior outcomes in each response time tercile when the anticipation label is drawn directly from the cue label ("ground" method) (Figure 5A), on cued trials when the anticipation is predicted by the classifier (Figure 5B), on uncued trials using the post-stimulus method (Figure 5C), and on uncued trials using the cued-pre-stimulus method (Figure 5D).
On cued data (Figure 5A), we observed that, among correct responses, correct anticipations were prominently represented among shorter reaction times relative to longer ones, while on the contrary incorrect anticipations were denser on long reaction times.On the other hand, error responses tended to be more represented among longer reaction times relative to shorter reaction times regardless of whether the anticipation was correct or not.Only the proportion of incorrectlyanticipated correctly-responded trials was significantly higher in long response times than in short ( = 12.816,  < 0.001) and medium ( = 10.125,  < 0.001) ones, and correctly-anticipated incorrectly-responded trials were significantly more represented in long relative to short response time trials ( = 5.227,  < 0.001).These global trends were also found using the classifier method instead of the ground method on cued trials (Figure 5B).However, these differences were not significant at the group level.
On uncued data, the two sources of anticipation labels yielded qualitatively different results.While anticipation predicted by the post-stimulus method raised qualitatively similar results to those obtained on cued data, the trends are reversed when using anticipation predicted by the cued-prestimulus method.In particular, correct anticipations with correct responses qualitatively display longer reaction times while incorrect anticipations with correct responses tend to be shorter.Note that the post-stimulus-based classifier yielded that correctly-anticipated correctly-responded trials were significantly more represented in short than in long response times ( = 6.310,  < 0.001).The same qualitative difference can be observed on the ground-truth cued trials (Figure 5A), although not significant.
These observations lead us to the interpretation that the post-stimulus method may have performed better than the cued-pre-stimulus method.
An interesting difference that emerges between cued and uncued trials lies in the distribution of correctly-anticipated error trials.These tend to have longer reaction times in the cued condition while they are more represented among the shorter reaction times in the uncued condition (Poststimulus-based classifier (Figure 5C): short-medium:  = 5.608,  < 0.001, short-long:  = 5.720,  < 0.001; cued-pre-stimulus-based (Figure 5D): short-medium:  = 4.356,  = 0.014).While it is possible that the classification algorithms failed on uncued trials, we propose the alternative view that error trials on correct anticipations result from different mechanisms depending on the condition: in the cued condition, these errors emerge when there is doubt either on the cue that has been seen or the motor command corresponding to the matching response.In contrast, as no indication about the upcoming stimulus is provided to the participant in uncued trials, shorter reaction times and errors could emerge in a repetitive task when motivation drops.

Anticipation and diffusion-decision models
We fitted the DDM to individual behavioral data in the cued condition to assess the mechanism by which response times and response accuracy are improved upon correctly anticipating the upcoming stimulus.For this, the boundary  = 1 and non-decision time  0 = 0.3 were fixed for all participants, while the drift  and starting point   was fitted for each participant depending on whether the cue matched the stimulus or not.This resulted in 2 drifts and starting points per participant.
We first tested whether the drift or the starting point varied significantly depending on the correctness of anticipation.The drift was significantly greater on correctly anticipated trials compared to incorrectly anticipated trials (paired-sample -test: (41) = 4.322,  < 0.001, Cohen's  = 0.667).We also observed a significant effect of anticipation correctness on the starting point value, whereby the starting point was closer to the correct decision boundary when the stimulus was correctly anticipated (paired-sample -test: (41) = 2.465,  = 0.018, Cohen's  = 0.380).
We then tested whether these differences also related to the classification accuracy on cued trials.
We observed that participants that displayed above-chance classification performances had a significantly more positive difference in starting point between correctly and incorrectly-anticipated trials, compared to participants whose classification performance remained at chance-level (independent sample -test: (40) = 2.102,  = 0.042, Cohen's  = 0.668).Moreover, we noted a significant correlation between classification accuracy and difference in starting point (Pearson's : 0.346,  = 0.025).Note that neither analysis revealed a significant relation between drift difference and classification accuracy.
Applying these results to assess classification performance on uncued data of the post-stimulusbased and the cued-pre-stimulus-based pipelines, we repeated the subjective fitting, this time using uncued trials.We then computed the difference between parameters of correctly and incorrectly anticipated trials, and compared the differences within and across classification pipelines.Within pipeline, none of the pipelines returned a significant difference between parameters, which we could allocate to a poor fitting compared to cued trials.However, the comparison of these differences

Discussion
In this work, our objective was to delve into the neural underpinnings of anticipation within the brain, focusing on scrutinizing its manifestations in both cued and uncued perceptual decision contexts.Our work's core idea involved assessing the selectivity of the CNV and pre-stimulus alphaband power concerning specific perceptual anticipation.We established that the CNV contained information about the expected stimulus at the single-trial level on cued trials.We subsequently attempted single-trial decoding on uncued trials using two techniques.DDM modeling also allowed us to link anticipatory activity to shifts in decision strategy, additionally providing a method to assess the classification performance of anticipation on uncued trials.
The CNV emerged as a promising indicator, with 25 out of 42 participants displaying above-chance single-trial classification performance on cued trials.This discernment significantly contrasts with historical views associating CNV with non-specific temporal expectation (Walter et al., 1964) or estimation (Kononowicz and Penney, 2016).More recently, works have shown that it was possible to distinguish anticipation in Go/NoGo tasks at the single-trial level, that is, to distinguish anticipation between trials where a movement is required relative to when movement has to be withheld (Chavarriaga et al., 2012;Garipelli et al., 2011Garipelli et al., , 2009;;Khaliliardali et al., 2015Khaliliardali et al., , 2012)).While Chavarriaga et al. (2012) emancipated their analyses from motor preparation by requiring no motor output in one of their tasks, the authors argued that the differences are ascribable to attentional changes following an erroneous proposition.The present work goes further by showing that the CNV is specific to the anticipated type of stimulus, opening new avenues for understanding the richness of anticipatory processes.
Second, pre-stimulus alpha power has historically been considered a cornerstone in examining the pre-stimulus period.Despite significant differences in alpha-band activation between types of anticipation, the single-trial classification performance using alpha-band activity yielded comparatively less robust results than CNV-based classification.Several reasons could explain this.
First, alpha band activity was previously thought to be related to idling states (Pfurtscheller et al., 1996), thus resulting in general rather than modality-specific brain signals.This, however, contradicts more recent evidence that sensory gating (Jensen et al., 2012;Jensen and Mazaheri, 2010) and selective attention (Foxe and Snyder, 2011) are stimulus-specific.The more likely alternative is that the paradigms implemented in previous studies used stimuli at different perception levels, including stimuli close to the individual perception threshold of each observer (Barik et al., 2019;Bode et al., 2012;Samaha et al., 2017).In these instances, stimulus perception would highly depend on the network excitability, represented by the alpha-band activity level.Here, we used distinguishable stimuli.However, Samaha et al. (2017) showed that the alpha-band activity biased the confidence ratings of participants and not the behavioral performance, which would mean that regardless of their alpha activity, participants could still sense the visual information they were presented with.In the same line, Benwell et al. (2021) showed that discrimination accuracy was unaltered by fluctuations of pre-stimulus alpha power, while subjective ratings of awareness correlated negatively with alpha power.This is still compatible with our explanation, as it would mark a difference between sensed and perceived information, the latter requiring a conscious grasp of the stimulus.
A question that can arise from these results is the performance of the classifiers.Indeed, the literature often reports classification accuracies of the order of 90% in supervised learning.Here, we reported a classification accuracy of the order of 60%, which can appear relatively anecdotical.
However, it is important to note that the studies reporting such high classification accuracies observe well-aligned, stimulus-evoked post-stimulus brain activity, whose typical amplitudes is of the order of 10 μV in the case of P300.In comparison, while we tempered the difficulty of timing anticipation onset by introducing an anticipation window, it remains an endogenous process, i.e. not induced by external stimulation.Moreover, the typical amplitude of the CNV is of the order of 2μV, which is an order of magnitude smaller than well-studied stimulus-evoked activity.Remarkably, no study currently reports above-chance classification performances between CNV types.Despite the seemingly low classification performances, robust statistical analyses show that they lie well beyond chance level.
The DDM analyses revealed an association between correct anticipation and shifts of the starting point toward the correct decision boundary (similar to Bode et al. (2012)) and an augmentation in the drift rate (similar to Urai et al. (2019)).This is consistent with the theory of premature sampling (Grosjean et al., 2001;Laming, 1979aLaming, , 1979b)), whereby information is integrated before the stimulus appears, and the idea of sensory facilitation (Walz et al., 2015), whereby information is more easily integrated when expected.Premature sampling results in a starting point shift, while sensory facilitation could explain the increased drift rate.Interestingly, only the difference in starting point between correct and incorrect anticipation significantly varied between participants whose CNVs were classified with above-chance accuracy and participants with chance-level classification accuracy.In that sense, the classifier could have captured premature sampling more efficiently in those participants with good classification performance compared to those for which the classifier yielded chance-level classification accuracy.We further used this result to infer the classification performance on pre-stimulus activity of uncued trials.
The exploration extended into single-trial decoding of anticipation on uncued trials, where two distinct techniques were employed.The first method relied on classifiers trained on cued prestimulus data, while the second method involved a separate classifier trained on cued post-stimulus activity trials to discern incorrect anticipations and an additional classifier trained on uncued prestimulus data.This multifaceted approach allowed for a nuanced examination of the dynamics at play during uncued trials.Remarkably, the classification performance on uncued trials exhibited variability among participants, with 5 out of 42 showcasing above-chance accuracy with the poststimulus-based pipeline.This variation emphasizes the complexity of anticipatory processes in the absence of cues, suggesting individual differences in the ability to decode anticipation.
Classifying uncued pre-stimulus activity is particularly challenging due to the lack of ground-truth information regarding the anticipation class.The DDM analysis on these trials revealed that the difference in starting points between correct and incorrect anticipations (as predicted by either classifier) was greater when the anticipation class was predicted using a classifier trained on cued data and applied directly on uncued data ("cued-pre-stimulus" pipeline) than when it was predicted from a classifier trained on uncued data, using labels predicted from post-stimulus EEG activity ("post-stimulus" pipeline).However, upon comparing the distribution of anticipation and behavioral outcomes across response time quantiles, we observed that the "post-stimulus" pipeline yielded patterns more similar to cued trials than the "cued-pre-stimulus" pipeline, suggesting conversely a better performance of the "post-stimulus" pipeline.Note, however, that this analysis is qualitative and that some uncertainty remains, as a quantitative assessment of the quality of the classification of anticipation based on post-stimulus activity on uncued data is equally challenging and was not performed here.
Alternative explanations for the observed differences could be put forward.First, one could argue that the differences reflect different motor preparations as the CNV is indeed traditionally known for its motor component (Rockstroh et al., 1990;Rohrbaugh and Gaillard, 1983;van Boxtel and Böcker, 2004).To eliminate the motor factor from our analyses, participants had to respond to both stimuli using the same motor output: a click from their dominant hand.Second, one could believe that the differences are ascribable to the sensory processing of the different visual cues.While our analyses do not completely discard this possibility, applying the classification pipelines only to the late part of the anticipation period allows to remove the initial part of sensory processing, which was shown to evoke distinguishable brain activity patterns at the single trial level (see Barachant and Congedo, 2014;Kalafatovich et al., 2020 for example).The later part of this activity, we argue, reflects anticipation and while this anticipation is induced by the visual cue, the difference in brain activity between the two types of anticipation is independent from the initial processing stages.In addition, the uncued trials were introduced to completely discard any influence from the sensory processing of cues, and gives a qualitative assessment that the features studied do in fact reflect anticipation without any sensory processing influence.
Anticipation could emerge from a variety of factors.Among them, sequential effects have been shown to impact subsequent decisions (Abrahamyan et al., 2016;Bode et al., 2012;Palminteri et al., 2017;Urai et al., 2019;Yu and Cohen, 2008).Our experiment was designed to present the stimuli randomly to each participant at each trial.Moreover, the stimuli were well distinguishable, reducing the uncertainty.However, it has been shown that past stimuli and decisions impacted subsequent decisions even without explicit sequences in the experiment (Yu and Cohen, 2008).We did not consider such effects here, in particular not in the cued experiment, where we implicitly assumed that the effect of the cue was greater than possible sequential effects.Indeed, the presence of the cue reduces the perceived uncertainty, as underlined by the reduction of response times compared to uncued trials.We argue, however, that these effects should be considered in more uncertain contexts, for example, when attempting to classify uncued pre-stimulus periods if the stimuli are closer to the perceptual threshold.The effect of global levels of attention on subsequent behavior could also be evaluated, as drowsiness is reflected in post-stimulus EEG patterns, possibly reflecting an alteration of the evidence integration stage of decision-making (Jagannathan et al., 2022).Future studies could weigh the contribution of the specific anticipation, sequential effects, and attention on behavior, and in particular, response times, using regression analyses, for instance.
The uncovered anticipatory processes are likely implicit for the main part.Indeed, while our experiment did not include a feedback questionnaire, some participants reported deliberately ignoring the cue in the calibration phase to avoid making mistakes.However, our behavioral analysis showed that participants were systematically faster in the calibration phase compared to the test phase, regardless of the order in which they performed the blocks.The accuracy, however, did not decrease with increasing speed, although the cue was faulty 20% of the time.It indicates that participants finalized their decisions upon stimulus presentation but still considered the cue informative, as they tended to make more mistakes in catch trials, i.e., when the cue and the subsequent stimulus were mismatched.A deeper analysis, which could include a scaling of the reliability of the cue combined with behavioral measurements and subjective ratings of the usefulness of the cue, could help investigate this effect.
In conclusion, our study has successfully unveiled the neural correlates of perceptual anticipation, emphasizing the CNV's specificity to the anticipated stimulus type.Applying the DDM and exploring pre-stimulus alpha power provided a nuanced understanding of the underlying mechanisms, further enriching our comprehension of anticipatory processes.Future research directions could involve deeper modeling techniques to refine the description of anticipation's influence on behavior, emphasizing individual differences.Additionally, investigations into the interplay between specific anticipation, sequential effects, and attention on behavior could offer a more comprehensive understanding of the multifaceted nature of anticipatory processes.Our work sets the stage for further exploration in ecologically valid scenarios and opens avenues for refining the classification of anticipation in uncued trials.The multidimensional nature of anticipation, embedded in neural processes, calls for continued exploration to unravel its complexities fully.

Figure 2 :
Figure 2: Processing pipeline Different anticipation classification methods, on cued and uncued trials.We use two types of classifiers, symbolized by grey ellipsoids: a classifier trained to detect whether the anticipation is visual or auditory based on pre-stimulus EEG activity, and a classifier trained to detect whether the anticipation was correct or incorrect (i.e., whether anticipation matched the following stimulus or not) based on post-stimulus EEG activity.The final labels that were used as the anticipation class are frames in light grey, with their denomination in the article underlined in the vicinity.

Figure 3
Figure 3 Mean response times in  (A, C) and accuracy (B, D) for all participants for each stimulus type, depending on (A, B) the cued/uncued condition and (C, D) whether the cue was congruent or not in cued trials.

Figure
Figure 4A also shows the areas involved in anticipation by representing the difference of evoked

Figure 4
Figure 4 Grand average ERP traces (average across all trials, participants, and channels) with 95% bootstrapped confidence interval, and topoplots of the difference of activity at 400, 267, 133 and 0  pre-stimulus.(A) cued vs. uncued trials.The topoplots represent cued activity minus uncued activity.(B) cued trials are split on whether the cue was an eye ("visual" anticipation) or an ear ("auditory" anticipation).The topoplots represent auditory minus visual anticipatory activity.

Figure 5
Figure 5Distribution of trials within each anticipation-behavior outcome type across response time terciles, for cued (A,B) and uncued (C, D) trials.In each plot, the anticipation label was extracted differently.(A) anticipation labels correspond to cue labels ("ground" method).(B) anticipation labels are the ones predicted by the classifier ("classifier" method).(C) anticipation labels are the ones predicted from pre-stimulus activity by the classifier trained on uncued data: the training was done using post-stimulus-predicted labels to build the "ground-truth" labels of anticipation ("post-stimulus" method).(D) anticipation labels are the ones predicted from pre-stimulus activity by the classifier trained on cued data("cued-pre-stimulus" method).* :  < 0.05, * * * :  < 0.005.
across pipelines signals that the cued-pre-stimulus-based pipeline returns significantly more positive starting point differences than the post-stimulus-based pipeline (Shapiro-Wilk test:  = 0.615,  < 0.001, Wilcoxon signed-rank test:  = 346,  = 0.02, Cohen's  = 0.488).Given our analysis on cued trials, a better pipeline should display a more positive starting point difference, indicating that the cued-pre-stimulus-based pipeline could have been more accurate than the post-stimulus-based one.