No Brain Signature of Prediction Error to Unattended Visual Stimuli Despite Early Brain Encoding of the Stimuli

Prediction error is a basic component of predictive-coding theory of brain processing. According to the theory, each stage of brain processing of sensory information generates a model of the current sensory input; subsequent input is compared against the model and only if there is a mismatch, a prediction error, is further processing performed. Recently, Smout et al. [1] found that a signature of prediction error, the visual (v) mismatch negativity (MMN), for a fundamental property of visual input—its orientation—was absent without attention on the stimuli. This is remarkable because the weight of evidence for MMNs from audition and vision is that they occur without attention. To resolve this discrepancy, we conducted an experiment addressing two alternative explanations for Smout et al.’s finding: that it was from a lack of reproducibility or that participants’ visual systems did not encode the stimuli when attention was on something else. We conducted a similar experiment to Smout et al.’s. We showed 21 participants sequences of identically oriented Gabor patches, standards, and, unpredictably, otherwise identical, Gabor patches differing in orientation by ±15°, ±30°, and ±60°, deviants. To test whether participants encoded the orientation of the standards, we varied the numbers of standards preceding a deviant, allowing us to search for a decrease in activity with the number of repetitions of standards—repetition suppression. We diverted participants’ attention from the oriented stimuli with a central, letter-detection task. We reproduced Smout et al.’s finding of no vMMN without attention, strengthening their finding. We also found that our participants showed repetition suppression: they did encode the stimuli pre-attentively. We also found early processing of deviants. We discuss whether this earlier processing of deviants may be why no further processing, in the vMMN time window, occurs.

standard, visual stimuli in a so-called oddball sequence. It is greater ERP negativity from parieto-occipital (PO) recording sites for deviants than for standards, occurring between 150 ms and 300 ms after the onset of the deviant.
It occurs without adaptation, showing the genuine vMMN [9]. It is supposed to occur without attention on the deviant property of the stimuli.
Recently however, Smout et al. [1] found no ERP evidence of the genuine vMMN without attention to unexpected changes in orientation. In their attended condition, they showed each participant displays consisting of an annular Gabor patch with a central black spot. They asked the participant to look only at the spot but to pay attention to the bars of the Gabor and to press a button on rare occasions when the bars' spatial frequency increased. In their ignored condition, the same participant viewed the same displays but paid attention to the spot and pressed a button on rare occasions when its contrast decreased. The difficulty of the two tasks was equated for each participant in prior psychophysical testing.
Smout et al.'s stimuli of interest were the Gabor annuli. Participants viewed different-length sequences of displays of identical annuli-standardsfollowed by an otherwise identical annulus but with bars differing in orientation by ±20°, ±40°, ±60°, or ±80°-a deviant. On the next display, the deviant annulus was repeated, commencing a new sequence for which it now served as the standard-the so-called roving-standard paradigm [10]. To measure the genuine vMMN, Smout et al. also showed control sequences in which all possible orientations occurred in random order, from which no expectation could be established, yet which had the same overall frequency of stimuli identical to deviants, equating adaptation-the equiprobable control [11].
Smout et al. found a genuine vMMN from PO electrodes to unexpected changes in the orientation of the bars in the attended condition (between about 170 ms to about 300 ms), but not in the ignored condition (their Figure S1B).
They did find genuine, deviant-related responses in the ignored condition: a negativity from about 330-430 ms from a midline central electrode (Cz) and from about 300-470 ms from a midline frontal electrode (Fz). But these are too late to be considered vMMNs, or any sort of prediction error from such an early brain-processing feature of visual input as orientation [12]. for a similar finding in the auditory modality [13]. To resolve this discrepancy, we conducted an experiment addressing two alternative explanations for Smout et al.'s finding: that it was from a lack of reproducibility or that participants' visual systems did not encode the stimuli when attention was on something else.
We made some changes to improve our study's ability to detect the genuine vMMN and to enhance its design. Smout et al.'s stimuli were Gabor annuli with an outer diameter of 11° diameter and an inner diameter of 0.83˚; their inner grey patch interrupted the central bars over a region of about the size of the human foveola to which about 20% of the processing of the visual system is devoted [14]. To improve our chances of finding a vMMN, our   Gabor patches were double the size of Smout et al.'s outer diameter size, and our central bars were continuous except for a very small area occupied by the lines of small task-relevant fixation letters. We also omitted an attended condition, eliminating any possible confound in Smout et al.'s design from participants' moving their eyes to the bars [15], thus bringing them onto central vision. We also analyzed clusters of electrodes around the single electrodes Smout et al. reported. This was to avoid issues related to variability in placement of single electrodes [16].
To look for evidence that participants encoded the orientation of the Gabor patches, we searched for repetition suppression: attenuated brain activity to repeated stimuli [17]. For Gabor patches similar to those used by Smout et al. and by us, this attenuation appears in various components of ERPs, as late as the P2 [17] and as early as the P1 at occipital-parietal electrodes [18].
We found no genuine vMMN to ignored orientation deviants despite showing that participants encoded the orientated stimuli.

Results
Participants looked at a central, desynchronized stream of letters and pressed a key whenever an X appeared ( Figure 1A Illustration of matching equiprobable controls in which we replaced standard stimuliexcept for the ones immediately preceding a deviant from the corresponding roving standard sequence-with Gabor patches whose orientation was randomly chosen among the 12 possible orientations. All orientations appeared equally (8.3%) often. The stimulus in the sequence (also NO VMMN TO UNATTENDED ORIENTATION DEVIANT 9 framed with two black chevrons) preceding the one that would have been the deviant in the corresponding roving standard sequence had an identical orientation to that from the corresponding roving standard sequence. The control (outlined in orange) also had the same orientation as that of the deviant from the corresponding roving standard sequence. Gabor patch onset is indicated on the timeline. Stimuli appeared for 80 ms, followed by a grey interstimulus interval (ISI) screen for 280 ms (not shown).

Did our Participants Attend to the Letter-Detection Task, Thereby
Ignoring the Gabor Patches?
We checked whether our participants attended to the central, letterdetection task, thereby ignoring the Gabor patches. The mean hit rate for detecting the target X in the central letter stream was 98.2% and the mean false alarm rate was <0.01%, showing that participants paid attention to the task and did very well on it. There were no significant differences in performance of the task on oddball and control blocks (details of analyses in supplementary materials: S1).

Did we Find a vMMN to Unpredicted Orientation Changes?
We did not find a genuine vMMN to unpredicted ignored orientation changes. There is no hint of any genuine negativity in the vMMN time range to any unpredicted orientation changes from any electrode cluster (we give details of our analyses, including Bayesian statistical tests, in Table S1). We show the ERPs and the difference waves in Figure 2, formatted similarly to those of Smout Table S1). The lighter colors surrounding the difference waves give ± 1 standard error of the mean.
NO VMMN TO UNATTENDED ORIENTATION DEVIANT 12

Did we Find Repetition Suppression?
We did find repetition suppression (as did Smout et al. in their attended condition). To search for it we used PCA to identify plausible components. We found two components showing repetition suppression-the N1 and P2, most clearly for the P2 (Figure 4). We show the N1 in Figure S3. Figure

Discussion
We have shown that even though participants encoded stimuli, they failed to show a genuine vMMN to unexpected orientation changes when they attended to a central task. Absence of the vMMN is consistent with five experiments including ours [1,15,20,21], but inconsistent with four [11,[22][23][24]. There are at least three possible explanations for this empirical dilemma: 1. There really is no genuine vMMN to unexpected, unattended orientation changes and the four out of nine studies that reported it are examples of publication bias or methodological errors.
2. There is a genuine vMMN to unexpected, unattended orientation changes and the five studies that failed to find it suffered from some yet-to-be-determined methodological differences or problems.
3. The vMMN is not the appropriate sign of prediction error for isolated and well controlled changes in low-level features of visual input-features that are easily processed in V1.

No Genuine vMMN to Unexpected Orientation Changes?
Although a 44% hit rate for finding a genuine vMMN is something to

Methodological Differences or Problems?
We considered whether our experimental parameters were responsible for our results and decided not. Human observers can discriminate orientation differences between gratings of about 0.4° [25], so there is no question that even our smallest orientation difference of 15° is large enough to be perceived.
Others have shown genuine vMMNs for other unattended, more complex deviants with presentation times as short as 17 ms [26], much shorter than our 80 ms. And others have shown genuine vMMNs with ISIs as short as 210 ms [27], shorter than our 280 ms. It seems unlikely our choices of orientation change, presentation time, and ISI were responsible for our failure to find a genuine vMMN.
In a review of the literature, we noted that researchers generally failed to isolate individual low-level features, such as orientation, from other, confounding factors including eye movements that are capable of changing simple orientation changes into more complex visual changes [15]. Even Smout et al.'s study suffers from this, with very careful control of fixation in their ignored condition (the participants' task was to detect contrast changes in a small central spot, requiring them to look at it) but not in their attended condition (the participants' task was to detect spatial-frequency changes in the bars of the oriented stimuli, 0.42° away from the fixation spot). Male et al. [15] showed evidence that participants' eyes did stray away from central fixation when the task-relevant information was away from fixation, despite instructions to keep fixation central.

The vMMN is not the Appropriate Sign of Prediction Error for Lowlevel Visual Features?
It is possible that unexpected changes in low-level features, such as orientation, are processed earlier than would normally be considered to fall into the time range of the vMMN. We found a genuine, deviant-related positivity at the P1, about 90 ms after onset of the Gabor patch. We have previously reported similar early deviant-related positivity [15] for orientation and for other basic stimulus features. Others have also reported early positivities for deviants, although these do not appear to have been considered as indicators of deviance detection [21,[28][29][30][31][32][33].
Alternatively, one could argue that we, like Smout et al. and others, did not find the genuine vMMN because prediction error generation in the visual system, unlike the auditory system, requires attention. This would, however, undermine one of the most frequently cited evolutionary purposes of prediction error; to monitor one's immediate surroundings pre-attentively. Smout et al. carefully navigated this obstacle, saying only that attention enhances the processing of unexpected changes in visual input. We also prefer to leave predictive-coding theory intact and point out that our finding of earlier processing of unexpected orientation changes suggests that no later processing, in the vMMN time range, is necessary.
We hope we have convinced you that the early positivity we found is a useful lead to understanding how the absence of a vMMN without attention could be reconciled with predictive coding theory.

Methods
We showed 21 [24 in Smout et al.] participants sequences of identically oriented Gabor patches ( Figure 1A) Figure 1A). Participants looked at a central, desynchronized stream of letters and pressed a key whenever an X appeared ( Figure 1B). Standard sequences ranged in length randomly from 3 to more than 11. For an equiprobable control, we presented the same stimuli, but in a random order (see Figure 1D).

Participants
Twenty-one self-declared healthy adults (  We used 25 letters of the English alphabet (all except letter 'I'). We give an illustration of this in Figure 1.

Procedure
The participants' task was to fixate on, and attend to, the continually changing random sequence of letters in the center of the monitor and to press a key with the dominant hand whenever an X appeared. Each letter occurred for 600 ms. None of the letters was immediately repeated. If the participant responded between 0.15 and 1.2 s after target onset, the response was correct.
There were 383 letter changes during a block. On average, there were 15 targets (ranging from 7 to 25) per block.
Desynchronized from the letters, we presented our stimuli of interestconcentric Gabor patches-for 80 ms separated by a 280-ms inter-stimulusinterval (ISI).
We had two sorts of blocks:  Figure 1(C) depicts these roving standard sequences.

Equiprobable blocks built from the roving standard blocks by
replacing all its standard stimuli (outlined in green in Figure   1C)-except for the ones immediately preceding a deviant -with a Gabor patch whose orientation was randomly chosen among the 12 possible orientations Figure 1 There were six oddball blocks and six equiprobable blocks. We randomized block order afresh for each participant. Each block took less than four minutes to complete. Participants were free to take breaks between blocks.

EEG recording and analysis
We recorded the electroencephalogram (EEG) using an EGI, 129channel, dense-array HydroCel geodesic sensor net, and Net Amps 300 signal amplifier. We recorded EEG at a 500 Hz sampling rate. Impedances were below 50 kΩ as recommended by Ferree et al. [40] for the high-input impedance amplifiers. All electrodes were referenced to Cz.
We processed the EEG data offline using MATLAB 2015b (MathWorks Inc., USA), EEGLAB 14.1.1 [41], and ERPLAB 6.1.4 [42]. We re-referenced the signal of all electrodes to the common average and filtered the EEG with a low-pass 40 Hz Kaiser-windowed (beta 5.65) sinc finite impulse response (FIR) filter (order 184) followed by a high-pass 0.1 Hz Kaiser-windowed (beta 5.65) sinc FIR filter (order 9056). Epochs were 400 ms long, featuring a 50 ms pre-stimulus baseline to accommodate the short 360 ms stimulus-onsetasynchrony. We excluded epochs with amplitude changes exceeding 800 μV at any electrode We identified electrodes with unusually high deviations in EEG activity relative to the average standard deviation pooled from all electrodes using the method described by Bigdely-Shamlo et al. [43]. A robust z-score was calculated for each electrode by replacing the mean by the median and the standard deviation by the robust standard deviation (0.7413 times the interquartile range). We removed any electrode with a z-score exceeding 2.0 provided at least four others surrounded them for later interpolation.
To improve the decomposition, we performed the analysis on raw data (excluding bad electrodes) filtered by a 1 Hz high-pass (Kaiser-windowed sinc FIR filter, order 804, beta 5.65) and 40 Hz low-pass filter, segmented into epochs, but not baseline corrected [45] as suggested by Winkler et al. [46]. We simultaneously reduced the data to 32 components.
To ensure that trials where participants moved their eyes or blinked were not included in the final analysis of the data, we created vertical and horizontal EOG channels by bipolarizing data from electrodes above and below the right eye (electrodes 8 and 126) and outer canthi of both eyes (electrodes 1 and 32), respectively (as in [47]). We identified epochs containing amplitude changes exceeding ±60 μV at these EOG channels for rejection after ICA correction.
We applied the de-mixing matrix to the 0.1−40 Hz filtered data and used SASICA [48] to identify which components exhibited low autocorrelation, low focal electrode or trial activity, high correlation with vertical or horizontal EOG, or met ADJUST criteria [49]. We assessed the remaining components using criteria described by Chaumon et al. [50], classifying components based on consistent activity time-locked to stimulus onset across all trials, on topography, or on power spectrum. We removed components identified as unrelated to brain activity. We then removed epochs identified for rejection. We performed a final artifact rejection-removing epochs containing amplitude changes exceeding ±60 μV at any electrode. Finally, using spherical splines [51], we interpolated data for removed electrodes.
For exploring deviant-related responses, we averaged ERPs for the standard, deviant, and control trials, excluding epochs immediately following a deviant or control trial. We produced difference waves by subtracting ERPs to standards and ERPs to controls from ERPs to deviants. The mean number (SD) of epochs for standards were 1994 (261), 187 (26) for 15-degree deviants, 187 A vMMN component would emerge as a component that is largest (most negative) between 150-300 ms, based on previous vMMN studies using orientation deviants (e.g., 162-170 ms in [56]; 200-250 ms in [11]; 190-220 ms in [22]). A genuine vMMN component should also yield scores that are more negative for deviants compared to controls at the component's negative pole.
We conducted Bayesian analyses of variance (ANOVAs) and paired ttests to determine the likelihood of obtaining the data in addition to our traditional ANOVAs and paired t-tests. We used a medium prior (with a Cauchy prior whose width was set to 0.707) for all Bayesian analyses. For interpreting our findings, a model with the largest Bayes Factor (BF10) is the model that best explains the data; this is the favored model. All main effects and interactions in the favored model are, therefore, important for explaining the data. Evidence against the null is considered weak if a BF10 is between 1 and 3. It is positive for a BF10 between 3 and 20, strong for a BF10 between 20 and 150, and very strong given a BF10 greater than 150 [57].
We also performed Bayes Factor replication (BFr0) tests [58] on mean amplitudes between 197 and 207 ms at each PO ROI given a prior reflecting the effect size (Cohen's d = -0.73) for the genuine vMMN at the PO8 in Kimura and Takeda's [22] oddball condition for 32.7° orientation deviants.

Author contributions
Both authors contributed to the experiment's conception, methodology, and writingreview & editing. AGM conducted formal analysis, data curation, investigation, project administration, visualization, and writingoriginal draft, review, and editing. ROS contributed to project administration and supervision.

vNMN
Bayesian replication confirmed that there was neither a classic nor a genuine vMMN (complete analyses in Table S1). In fact, for the classic

Deviant-related positivity
For the deviant-related positivity (DRP), we performed a 3 × 3 × 2   found the largest deviant-minus-control difference (i.e., genuine vMMN) for 32.7° orientation deviants. Amplitudes from this time window were analyzed using Bayesian replication (results in Table S1). The lighter colors surrounding the difference waves give ± 1 standard error of the mean.
NO VMMN TO UNATTENDED ORIENTATION DEVIANT 32 found the largest deviant-minus-control difference (i.e., genuine vMMN) for 32.7° orientation deviants. Amplitudes from this time window were analyzed using Bayesian replication (results in Table S1). The lighter colors surrounding the difference waves give ± 1 standard error of the mean.
NO VMMN TO UNATTENDED ORIENTATION DEVIANT 33