Abstract
The ability to detect novelty in sensory stimuli is at the base of autonomic and goal-directed behavior. Pupil size, a proxy of the Locus Coeruleus-Norepinephrine system, is sensitive to auditory novelty. However, whether this response reliably reflects conscious processing of novelty remains contentious. Here, we characterized pupil and electrophysiological responses during conscious and subconscious processing of auditory novelty by presenting participants deviant stimuli that were below and above their discriminatory thresholds. We found higher pupil responses to subthreshold targets that were not consciously perceived as deviant stimuli. Larger pupil size and dilation rates were associated to more negative Event-Related Potential values extracted from temporal, prefrontal and anterior cingulate regions. We suggest that increased phasic responses to deviant targets that escape conscious perception reflect Norepinephrine-mediated adaptation of arousal levels in order to meet the perceptual and behavioral demands imposed by the task at hand.
Introduction
The ability to extract regularities and detect novelty in the form of violations to the statistical properties of sensory information is of paramount importance for biological organisms, as it mediates both autonomic responses and goal-directed behavior (Ranganath and Rainer, 2003; Tiitinen et al., 1994; Sohoglu and Chait, 2016). Conscious processing of novel stimuli, contrast-based saliency and arousal levels can speed up, delay, or even suppress neuronal and behavioral responses (Töllner et al., 2011, Aston-Jones and Cohen, 2005, Vasey et al., 2018). Remarkably, neural populations have the ability of fine-tuning its properties and accommodating neuronal gain-modulation thresholds in order to meet environmental or task demands (Ferguson and Cardin, 2020). Such gain-modulation adaptation is mediated by the activity of the Locus Coeruleus – Norepinephrine (LC-NE) system in response to the demands imposed by environmental and task-specific conditions (Aston-Jones and Cohen, 2005; Poe et al., 2020). Additionally, pupil size has been shown to reflect NE-mediated arousal (Aston-Jones and Cohen, 2005; Ferguson and Cardin, 2020) and, more recently, to be sensitive to the detection of auditory novelty (Quirins et al, 2018; Zhao et al., 2019). However, whether this response requires conscious processing of sensory novelty remains contentious.
A phasic increase in pupil size has been associated with subjects’ conscious processing of single auditory stimuli presentation, but not to auditory stimuli that are not consciously perceived and reported (Bala et al., 2019). Similarly, consciously reported violations of auditory regularities in tonal sequences elicit a pupil response during active-counting and passive listening, whereas violations that escape conscious perception do not elicit a pupil response (Quirins et al, 2018). This line of evidence suggests that the pupil reflects conscious processing of novel stimuli. In contrast, introducing two oddballs, each one of different saliency, suppresses the pupil response to the less salient target during passive engagement. Interestingly, requiring participants to report any detected novelty restores the pupil response to both targets (Liao et al., 2016A). Likewise, abrupt violations of auditory regularities but not sudden regularity emergence elicit an increase in pupil size during passive listening. However, asking participants to monitor any change in the auditory scene results in a pupil response to both regularity violation and regularity emergence (Zhao et al., 2019). This latter line of evidence therefore suggests that the phasic pupil response can operate independently of conscious perception and that behavioral relevance of perceived stimuli might be important in eliciting a pupil response.
Two well attested markers of conscious and subconscious processing of auditory novelty are the Mismatch Negativity (MMN) and the P3 positivity complex. Auditory stimuli that violate the predictions of the central auditory system elicit an MMN response peaking around 200 milliseconds after odd stimulus presentation. This Event-Related Potential (ERP) occurs independently of attentional state or conscious processing (Bekinschtein et al., 2009; Näätänen et al., 2007, 2019). Generators of this response have been identified in posterior superior and middle temporal and prefrontal regions (Garrido et al., 2009) and more recently, in anterior portions of the Cingulate Cortex, a region involved in error detection and the processing of surprisal (Hyman et al., 2017). The MMN response is proposed to reflect an orienting attention mechanism involved in bottom-up processing of sensory information (Näätänen et al., 2007, 2019). In turn, novel auditory events that are attended to and consciously detected elicit a positive deflection in the ERP, known as the P3 response, starting at around 300 milliseconds after the presentation of a novel stimulus (Polich, 2007; Kamp and Donchin, 2015). Multiple generators for this event have been reported within a fronto-centro-parietal network encompassing dorsomedial prefrontal regions, precentral and postcentral gyri, superior parietal and cingulate regions (Linden, 2005). The P3 response has been proposed to reflect context-updating and memory-dependent information processing mechanisms (Polich, 2007).
In this study, we investigate how the pupil responds to auditory novelty with and without conscious perception, and how such response relates to well established markers of subconscious and conscious auditory processing, namely the MMN and the P3 ERP events. For this, we implemented a novel task which allowed disentangling conscious from subconscious processing of auditory novelty by presenting deviant targets above and below each subject’s threshold for conscious discrimination. We found increased pupil responses to subthreshold deviant targets that were not consciously perceived in contrast to consciously processed suprathreshold targets. Increased pupil dilation responses were associated to more negative mean ERP values extracted from source-reconstructed temporal, prefrontal and anterior cingulate regions during the latency time period corresponding to the MMN. We suggest that an increased pupil response to deviant targets that are not consciously perceived reflects an increased demand of NE which might be necessary in order to accommodate current arousal levels to the perceptual and behavioral demands imposed by the task at hand.
Results
Subthreshold deviant targets are associated with high error rates and slower reaction times
All participants performed a staircase procedure which allowed identifying their individual discriminatory thresholds before each block of the thresholded deviant detection task. The staircase procedure allowed setting subthreshold deviant targets adaptively and objectively according to the individual hearing abilities of each participant (Figure 1A). Participants (n = 24, mean age = 25.5, range = 13) were binaurally presented sequences of narrowband sinusoidal tones and asked to decide whether the last tonal stimulus (i.e the target tone) was the same as or different from the previous standard tones (Figure 1B). The target stimulus could be either another standard tone (tgtSTD), a suprathreshold deviant (supraDEV) or a subthreshold deviant (subDEV). Because subDEV stimuli were deliberately intended to be below the threshold for conscious discrimination, subthreshold deviants were expected to be systematically judged as standard tones and should therefore be associated with high error rates. Conversely, target standards and suprathreshold deviants should be correctly and systematically identified as such, which should manifest as high accuracy rates. Asserting that participants conformed to this expected response pattern was important in order to guarantee that subthreshold deviants were subconsciously processed but not consciously perceived.
Therefore, we tested individual accuracy rates for each block and for each set of stimuli within each condition against the chance probability using a binomial distribution test (Table S2, supplementary materials). Data from blocks that failed to meet the above-chance performance criterion (i.e. tgtSTD/correct, subDEV/incorrect, supraDEV/correct) were excluded from subsequent analyses. We confirmed that both target standards and subthreshold deviant targets were associated with high hit rates, whereas subthreshold deviant targets where associated with high error rates. (Figure 1C). Next, we computed median Reaction Times (RTs) and performed group-level statistics. Due to the 1000-millisecond delay in behavioral response, we did not expect to see a significant difference in median reaction times across conditions. Interestingly, we found that the median reaction time to suprathreshold deviant targets was statistically faster than to subthreshold deviant targets at the group-level (n = 21, p < 0.05, Bonferroni-corrected, Figure 1D). No other statistical differences were found between median RT.
Increased pupil response to subthreshold deviant targets that escape conscious perception
Next, we investigated pupil responses to standard, subthreshold deviant and suprathreshold deviant targets. We computed two measures of phasic pupil change: normalized pupil size and normalized pupil rate of change. We decided to include pupil rate of change as it provides more time-resolved information about both pupil dilation and constriction compared to pupil size. As a control procedure, we inspected the nature of the relationship between pupil size and pupil rate of change during our 1000-millisecond time window of interest. We confirmed that an increased pupil size was associated with faster dilation of the pupil whereas a smaller pupil size was associated either with slower dilation or constriction of the pupil across conditions (Figure S3, supplementary materials).
If the pupil reliably reflected conscious processing of auditory novelty, we would expect to see an increased pupil response to suprathreshold targets that were consciously perceived, but not to subthreshold targets which escaped conscious perception. In contrast with this expectation, we found that subthreshold deviants were associated with increased pupil sizes compared to standard and suprathreshold targets (Figure 2A). This effect was significant between ~180 and ~540 milliseconds after target stimulus onset compared to suprathreshold deviant targets (n =21, p < 0.05, Bonferroni-corrected). Dilation rates were also statistically faster for subthreshold deviants between ~100 and ~280 milliseconds compared to target standards (n =21, p < 0.05, Bonferroni-corrected) and between ~130 and ~290 milliseconds compared to suprathreshold targets (n =21, p < 0.05, Bonferroni-corrected, Figure 2B). No statistical difference was observed between pupil responses to standards and suprathreshold deviants. These results therefore suggest that the pupil response does not reliably reflect conscious processing of novel auditory stimulus.
We also investigated whether pupil responses were related to reaction times. We found that subjects who showed bigger pupil sizes also showed slower reaction times between 0 and 900 milliseconds across conditions (tgtSTD: n = 17, rho = 0.629, p = 0.008; subDEV: n 17, 0.544, p = 0.026; supraDEV = n = 17, rho = 0.580, p = 0.016, Figure 2C). As for pupil rate of change, we observed that faster dilation rates were also associated with slower reaction times for standard (n =17, rho = 0.676, p = 0.003) and suprathreshold deviant targets (n = 17, rho = 0.561, p = 0.021, Figure 2D), but not for subthreshold targets.
ERP responses reflect subconscious processing of subthreshold targets and conscious processing suprathreshold targets
We then investigated the mean ERP response measured at the scalp level to the three types of target stimuli. If the MMN is independent of conscious perception, we would expect to observe an MMN to both subthreshold deviant targets incorrectly judged as standard tones and suprathreshold deviant targets correctly reported as such. Additionally, if the P3 reflects conscious processing of sensory novelty, we should expect to see a P3 response to consciously detected suprathreshold deviant targets, but not to subthreshold deviants. In line with these expectations, both subthreshold and suprathreshold deviants elicited an MMN neural response at electrode Cz (Figure 2A). This effect was significant between ~180 and ~195 milliseconds for subthreshold deviant targets and between ~130 and ~196 milliseconds for suprathreshold deviant targets (n = 23, p < 0.05, Bonferroni corrected). In contrast, we observed a P3 response only to suprathreshold deviant targets that were consciously perceived (Figure 2A). This effect was significant between a wider time window comprising ~220 and ~440 milliseconds after stimulus presentation (n = 23, p < 0.05, Bonferroni corrected). Our ERP results thus suggest that subthreshold deviants were subconsciously processed but escaped conscious perception, whereas suprathreshold deviant targets were both subconsciously and consciously processed.
EEG-source imaging reveals involvement of temporal, prefrontal and anterior cingulate regions
In order to inspect the cortical activation dynamics associated to conscious and subconscious processing of auditory novelty, we projected each participant EEG signals onto template cortical surfaces. We performed group analyses for the difference in means between both types of deviant targets (subDEV and supraDEV) against target standards. Unconstrained forward models using the Minimum Norm solution and source imaging using the sLORETA method on MNI/ICBM152 surface templates showed patterns of activation consistent with previously reported cortical origins of the MMN and the P3 ERP responses (Figures 3B and 3C).
For subthreshold deviant targets, we observed increased activation of left prefrontal (DLPFC), left pre and postcentral and right temporo-parietal regions at 192 milliseconds. This prefrontal activation is consistent with reports of a frontal generator for the MNN response. At 300 milliseconds, subthreshold deviants elicited increased activation of right superior and middle temporal (STG and MTG) regions, as well as the right insula, but no activation was found for central, superior parietal or dorsomedial (DMPFC) regions which are classically associated with the P3 event (Figure 3B). These cortical activation dynamics are in line with the known origins of the MMN in temporal and prefrontal regions. For suprathreshold deviant targets, we found increased activation of insular, superior and middle temporal (STG and MTG), and prefrontal regions (DLPFC) bilaterally at 192 milliseconds. At 300 milliseconds, there was increased activation of dorsomedial prefrontal regions (DMPFC), superior pre and postcentral and superior parietal areas for suprathreshold deviant targets (Figure 3C). These latter results are consistent with the known cortical generators of the P3.
Finally, we found increased activation of the ACC for both subthreshold and suprathreshold deviants against target standards. For subthreshold deviants, there was an involvement of the ACC at between ~220 and ~228 milliseconds (Figure 3D). Suprathreshold deviants also elicited activation of the ACC between ~220 and ~232 milliseconds, but there was also involvement of the ACC between ~328 and ~360 milliseconds (Figure 3E). These results confirm the involvement of temporal, prefrontal and cingulate regions during the latency period corresponding to the MMN and of dorsomedial and central regions during the time window corresponding to the P3.
Increased pupil responses are associated to more negative source-reconstructed ERP values
Finally, we investigated the relationship between the pupil response and ERPs extracted from six source-reconstructed regions of interest (ROIs). These regions were informed by the literature on the generators of the MMN and P3 and our source-imaging results, and included left and right STG, left and right DLPFC, DMPFC bilaterally and ACC. We performed correlational analyses between our measures of pupil change and the mean ERPs extracted from our ROIs for two time windows of interest corresponding to our ERP events of interest, namely the MMN response (150-220 ms) and the P3 (280-320 ms) response. If the pupil response and the P3 were related neural events reflecting context updating and memory-dependent information processing, we should see that subjects showing more positive ERP values extracted from bilateral DMPFC or ACC within 280 and 320 milliseconds would also show increased pupil responses. Alternatively, if the pupil response and the MMN were related neural events reflecting bottom-up orienting attention mechanisms, we would expect to see that subjects showing more negative ERP values extracted from the STG, the DLPFC and the ACC within 150 and 220 milliseconds would also show increased pupil responses.
We found evidence for the second scenario: for subthreshold targets, faster pupil dilation rates were associated with more negative ERP values at right STG (n = 17, rho = −0.497, p = 0.044, Figure 4A) and right DLPFC (n = 17, rho = −0.502, p = 0.041, Figure 4B) during the 150-to-220-millisecond time window, but this effect was absent between 280 and 320 milliseconds (Figure S4A and S4B, supplementary materials). Also for subthreshold targets, a faster rate of change (n = 17, rho = −0.664, p = 0.004, Figure 4C) and bigger pupil size (n = 17, rho = −0.750. p = 0.008, Figure 4D) were associated to more negative ERP values extracted from the ACC between 280 and 320 milliseconds. This effect was not observed during the 150-220-millisecond time window (Figure S4C, supplementary materials). Moreover, no effects were found for standard or suprathreshold deviant targets. These findings suggest that subconscious processing of auditory novelty is associated with both increased pupil response and more negative values in ERPs extracted from regions and time periods corresponding to the MMN and the P3.
Discussion
In this study, we investigated the pupil response during the processing of auditory novelty with and without conscious perception and how it relates to well established markers of subconscious and conscious auditory processing. Phasic changes in pupil size have been associated to a myriad of cognitive functions, including effort, saliency, arousal, attention, memory, consciousness among many others (van der Wel and van Steenbergen, 2018; Wang et al., 2014; 2018; Wainstein et al., 2017; Clewett et al, 2020). This, however, has made it difficult to arrive to an overarching account of what the pupil reflects across cognitive domains.
Some studies have provided evidence that the pupil reflects conscious processing of detected sensory deviance during the processing of auditory novelty (Bala et al., 2019; Quirins et al., 2018). However, other results suggest that the pupil response can operate independently of conscious processing under certain task conditions, thus highlighting the role of behavioral relevance of perceived stimuli (Liao et al., 2016A; Zhao et al., 2019; Alamia et al., 2019). Our results add up to the latter line of evidence. Increased pupil size and dilation rates were observed in response to subthreshold deviant tones that escaped conscious perception and for which there was not an associated P3 response. In contrast, no pupil response was observed for suprathreshold deviants that were consciously identified and that elicited a P3 response. This suggests that the pupil does not reliably reflect conscious processing of detected deviance, as certain task conditions can elicit a pupil response in the absence of conscious perception.
Other studies have previously found that the pupil is sensitive to contrast-based stimulus saliency, and that a more pronounced pupil response is associated to increased contrast between standard and deviant stimuli (Liao et al., 2016B; Wang et al., 2014). Similar modulatory effects of contrast-based saliency have been found for the MMN (Näätänen et al., 2007) and the P3 (Teixeira et al., 2010; Texeira et al., 2014). If contrast-based saliency was driving pupil responses, we should have seen a negative correlation between pupil response and MMN across conditions, and a positive correlation between pupil response and P3 for suprathreshold deviants. However, we observed a pupil response to the least salient of both deviant targets, whereas no such response was observed to the most salient one. Moreover, we did not observe linear relationships between measure of pupil change and the MMN across conditions, or between pupil and P3 for suprathreshold deviants. This suggests that under our experimental conditions, pupil response was not modulated by contrast-based saliency.
Both subthreshold deviants and suprathreshold deviants elicited an MMN response at ~200 milliseconds after target presentation. In contrast, suprathreshold deviants but not subthreshold deviants elicited a P3 response. This confirms that the staircase procedure along with our thresholded-deviant detection task effectively resulted in subconscious and conscious processing of auditory novelty, in line with other adaptations to the classical oddball task, such as the global-local paradigm (Bekinschtein et al., 2009). Moreover, our results replicate findings which demonstrate that the MMN operates independently of attentional states and conscious perception, whereas the P3 necessitates the subject’s conscious access to the target stimulus (Bekinschtein et al., 2009; Näätänen et al., 2019; Polich, 2007; Kamp and Donchin, 2015). EEG source-imaging confirmed that these signals originated from regions classically associated to the MMN and the P3, including middle temporal, superior temporal, prefrontal, dorsomedial and centro-parietal regions (Garrido et al., 2009; Linden, 2005). This reassures us that in spite of using a novel task, the events we observed indeed correspond to the classical ERP events associated to conscious and subconscious processing of auditory novelty. We also found evidence for the involvement of Cingulate and Insular regions, which is in line with more recent studies on surprisal and mismatch/error detection (Hyman et al., 2017; Citherlet et al., 2019; Han et al., 2019). Such results highlight the importance of these cortical areas during the processing of sensory novelty.
Previous studies have failed to identify a straightforward relationship between the pupil response and ERPs measured at the scalp level, particularly the P3 (Steiner and Barry, 2011; Kamp and Donchin, 2015). This is surprising because both the phasic pupil and the P3 responses are modulated by the activity of the LC-NE system (Aston-Jones and Cohen, 2005; Murphy et al., 2011, Vazey et al., 2018; Nieuwenhuis et al., 2005). Since we did not find evidence of such relationship, we also conclude that the pupil response and the P3 reflect separate neural mechanisms, even if they rely on a common neuromodulatory system. In contrast, we found evidence for a highly specific relationship between phasic pupil response and the MMN during subconscious processing of auditory novelty. A higher rate of change was associated to more negative ERP values computed from source-reconstructed signals in right STG, right DLPFC and ACC between 150-220 milliseconds. These regions and time window overlap with the known generators and latency period of the MMN. Similarly, increased pupil size and faster dilation rates were associated to more negative ERP values extracted from the ACC between 280 and 320 milliseconds, a time window associated to the P3 event. Because the MMN has been associated to Glutamatergic and not to Noradrenergic modulation (Harms et al., 2020) and because the MMN reliably reflects whereas the pupil is sensitive to, but does not reliably reflect contrast-based saliency, we suggest that the pupil response and the MMN are both involved in orienting attention processes but still reflect different neural mechanisms. However, this effect was only observed for subthreshold deviant targets that escaped conscious perception and not for consciously processed suprathreshold targets.
What then does the pupil response reflect? A growing number of studies have associated phasic changes in pupil size to the adaptation of arousal levels by the activity of the LC-NE system (Urai et al., 2017; de Gee et al., 2014; Clewett et al., 2020; Krishnamurthy et al., 2017). Our observation that increased pupil size was associated to slower reaction times across conditions is reminiscent of the far-right tail of the Yerkes-Dodson curve (Aston-Jones and Cohen, 2005) and suggests that the pupil response was driven by changes in arousal levels ensuing the presentation of target stimuli. Moreover, previous studies have demonstrated that the phasic pupil response is associated to changes in global arousal levels which are driven by task-specific conditions and decision-making processes (Urai et al., 2017; de Gee et al., 2014). We consider that our findings are most interpretable in terms of changes in global arousal as a result of phasic LC-NE activity.
The Adaptive Gain Theory proposes that the function of phasic LC-NE system activation is to facilitate changes in arousal for the optimization of behavioral performance according to specific task demands (Aston-Jones and Cohen, 2005; Poe et al., 2020). Importantly, this theory discriminates between tonic LC-NE activation, which is associated with baseline LC firing and baseline arousal, and phasic LC-NE activation, which is associated to evoked LC firing and phasic arousal in response to stimulus-driven and task-relevant decision processes (Aston-Jones and Cohen, 2005; Gilzenrat et al, 2010; Poe et al., 2020). Phasic activation of the LC would result in the adaptation of neural gain-modulation functions thanks to increased NE input, which would in turn modulate cortical excitation-inhibition balances, thus facilitating the adaptation of arousal levels to meet sensory or behavioral demands. (Aston-Jones and Cohen, 2005; Ferguson and Cardin, 2020; Batista-Brito et al., 2018).
Because pupil size reliably indicates the activation of the LC-NE neurons (Joshi et al., 2016; Varazzani et al., 2015; Murphy et al., 2014), we therefore suggest that increased pupil responses to subthreshold targets reflect a higher demand of NE in order to accommodate arousal levels to satisfy the perceptual and behavioral demands imposed by the thresholded-deviant detection task. This phasic activation of the LC-NE would presumably follos a feedback signal targeting the LC and associated to either higher uncertainty (Urai et al., 2017) or prediction error (Sales et al., 2019) during bottom-up information processing. Interestingly, a plausible neural circuit that could support this feedback mechanism comprises prefrontal and cingulate regions (Aston-Jones and Cohen, 2005). Indeed, direct bidirectional projections exist both between the PFC and the LC (Totah et al., 2020) and the ACC and the LC (Gompf et al, 2010). Temporally, this is also plausible: phasic discharges of NE are reported as fast as 100 milliseconds after LC stimulation and conduction latency to PFC is of ~60 milliseconds (Aston-Jones and Cohen, 2005; Aston-Jones et al, 1985), whereas phasic increases in pupil size resulting from LC microstimulation usually start at around 200-250 milliseconds with a mean peak latency between 450-550 milliseconds (Joshi et al., 2016).
In conclusion, we show that the pupil is sensitive to subconscious processing of auditory novelty, reflecting higher activity of the LC-NE system which is necessary for the adaptation of arousal in response to specific task demands. Due to a higher contrast-based saliency, suprathreshold deviant targets were amenable to both automatic orienting-attention mechanisms (i.e. MMN) and executive processes involved in conscious processing (i.e. P3). The desired behavioral output could therefore be obtained without significant changes in the system’s arousal levels available upon stimulus presentation. Subthreshold deviant targets, on the other hand, were below thresholds for conscious discrimination and posed a significant perceptual challenge. Although detectable by means of automatic bottom-up orienting attention mechanisms (i.e. MMN), they escaped higher-order executive processes indexed by the P3 that were required to meet the desired behavioral outcome (deviant detection). This would have resulted in an increase demand of NE to accommodate arousal levels via the adaptation of gain-modulation functions at relevant sensory and attention-mediating cortical area, with the presumed goal of lowering thresholds for conscious identification of subthreshold targets.
Methods
Participant details
Twenty-four right-handed healthy subjects with no self-reported record of auditory, neurological or neuropsychiatric disorders voluntarily agreed to participate in this study (mean age = 25.5, range = 13). All participants reported normal hearing and normal or corrected-to-normal vision. Extensive and/or formal musical training, as well as high competence in a second language were considered exclusion criteria. Participants were recruited from among the undergraduate and postgraduate community at Pontificia Universidad Católica de Chile and Universidad de Chile. All participants signed an inform consent prior to their participation in the study.
Procedures and stimuli
Participants sat 50cm away from of a screen in a dimly lit room. Participants’ brain activity was recorded using a 64-channel Biosemi EEG system and their pupil recorded using an Eyelink 1000 eye-tracking system. The eye-tracking system was calibrated at the beginning of each experimental session. Auditory stimuli were presented binaurally via special airtube earphones (ER-1 Etymotic Research) that minimize electrical interference. Stimuli comprised sequences of 150-millisecond long narrowband sinusoidal tones (Table S1) presented with an interstimulus interval of 150 milliseconds. Stimuli were set to be delivered at an intensity of 70dBs. Participants sat within a Faraday cage while performing the task. The task was programmed using the NBS (Neurobehavioral Systems) Presentation software.
Staircase procedure
Because hearing abilities vary across individuals, all participants performed a staircase procedure at the beginning of each experimental block and for each set of stimuli. A sequence of standard tones (dark circles inside dotted box, Figure 1, left) was presented against another sinusoidal tone (the target) 50 Hz above the standard tone (blue circle). Participant were asked whether the target was the same or different from the preceding tones. If the subject response was “different”, a new trial was presented where the target tone was stepped down five 5Hz (green arrow, Figure 1, left. But see table S1 for the full set of stimuli). Targets would eventually become increasingly similar to the standard tones (gradient of gray circles). When participants judged a target to be the same as the previous standard tones, the subsequent target tone was stepped up in 5Hz. Once the subject responses entered a same-different response loop (green arrows, Figure 1, middle), the algorithm would identify this region within the staircase as containing the subjects’ discriminatory threshold. After four consecutive iterations of this same-different response loop, the subject’s discriminatory was set to be the boundary separating the target tone consistently reported as “different” from the target tone consistently reported as “same” (dotted horizontal line, Figure 1, right). Finally, the three stimuli were automatically set: the standard (dark gray circle, tgtSTD), the subthreshold deviant (green circle, subDEV) and the suprathreshold deviant (blue circle, supraDEV). The subthreshold deviant was always set as the tone being two steps bellow the discriminatory threshold and the suprathreshold deviant was always set to be the STD tone plus 50Hz.
Thresholded deviant detection task
The thresholded deviant detection task comprised three blocks. For each block, the base frequency for standard tones would be either 800Hz, 1000Hz or 1200Hz. During each trial, participants listened to sequences of tonal stimuli and were instructed to decide whether the last tone (i.e. the target) was the same or different from the preceding standard tones. Participants had to make their choice by pressing one of two buttons upon appearance of a prompt on screen, 1000 milliseconds after the onset of the target stimulus. This delay in behavioral response was in order to avoid confounding effects due to the temporal and spatial overlap of motor signals and the ERP events of interest. There was no time limit for response. The number of standard tones before each target stimulus randomly varied between three and five tones. Random variability in the number of standard tones before each target was expected to minimize habituation effects. Participants were told to prioritize response accuracy over speed of response. The target stimulus could be either another standard tone (tgtSTD), a tone that was 50 Hz above the standard tone (supraDEV) or a tone that was below each participants’ discriminatory threshold as defined by the staircase procedure (subDEV). The theoretical probability for each type of target was ~33.333%.
Behavioral data analyses
Data (n = 24) was obtained using Presentation software. Default output files were preprocessed and analyzed using in-house Matlab scripts. Each participants’ performance was in each block and for each set of stimuli were tested against the chance probability using a binomial test (i.e. the probability of observing x correct or incorrect responses given a theoretical probability p for the corresponding number of trials per block n). Only data (behavioral, pupil and EEG) from experimental blocks that were above the chance probability were included in further analyzes. We failed to identify the discriminatory threshold of 3 of our participants due to problems during the staircase procedure (e.g. the participant accidentally confused buttons or did not fully understand the task, resulting in unprecise discriminatory threshold that did not reflect their actual perceptual abilities, Table S2). Reaction times below zero (i.e. before response prompt appeared on screen) were considered accidental button presses and therefore rejected from analyses. Any reaction time below and above the 0.25 and the 97.5 percentiles at the subject-level were also defined as outliers and therefore rejected. Histograms were plotted to inspect the distribution of reaction times. Because reaction times were right-skewed, approximating a gamma distribution, we computed the median reaction time and used it for subsequent statistical analyses.
EEG data preprocessing
Data (n = 23) was preprocessed using Brainstorm (Tadel et al., 2011, http://neuroimage.usc.edu/brainstorm). EEG data was filtered between 1 and 45Hz using a 7426-order FIR bandpass filter. Subsequently, data was detrended and visually inspected for noisy channels using Welch’s Power Spectrum Density (PSD). Next deleted channels were interpolated and the EEG signal was re-referenced to the average of all electrodes. Oculomotor and blink-related artifacts were removed using and Independent Component Analysis (Makeig et al., 1996) on the continuous EEG signal. Data was epoqued in trials comprising 2500ms before and 1000ms after presentation of target stimuli. Any trial where the signal exceeded 100 microvolts in amplitude was rejected from subsequent analyses. Event Related Potentials were computed as the baseline-corrected arithmetic average of all individual trials per subject, per target type. Baseline correction was applied by subtracting the mean ERP between −500 milliseconds and time zero. EEG forward models were computed using the symmetric Boundary Method BEM by the open source software OpenMEEG (Gramfort et al. 2010) on default MNI/ICMB152 cortical templates (Fonov et al., 2009) using default Brainstorm parameters. Source estimation was computed using the Minimum Norm solution and unconstrained sLORETA (Pasqual-Marqui, 2002) estimates on the preprocessed data. Matrices for the covariance of all electrodes were computed from approximately 1000ms baseline periods on each epoque. Regions of Interest were selected a priori based on previous literature on auditory mismatch processing as well as on the origins of the MMN/P3 Event-Related potentials, and were manually delimited informed by z-maps shown in figure on the ICBM152 template cortical surface (mean vertices = 188.66). These ROIs were portions of the right and left Superior Temporal Gyrus, right and left Dorsolateral Prefrontal Cortex, bilateral Dorsomedial Prefrontal Cortex and left Anterior Cingulate Cortex. Scalp EEG data from one participant was excluded from analyses due to technical issues during data acquisition.
Pupillometry
Data (n = 21) was acquired using Eyelinks’ default acquisition hardware and software at a sampling rate of 1000 Hz. Calibration procedures were carried out at the beginning of each experimental session. Pupil area, horizontal and vertical gaze positions were recorded in a dimly lit room from the right eye of each participant. Blinks and gaze artifacts were detected by Eyelinks’ default algorithms. Pupil data was preprocessed using Urai et al. (2017) pupil pipeline plus additional in-house Matlab script adaptations. Eyelink-defined and additionally detected blinks were padded by 150 milliseconds and linearly interpolated. The pupil response evoked by blinks and saccadic events was identified via deconvolution and removed using linear regression as in Knapen et al. (2016). The signal was then filtered between 0.01 Hz and 10 Hz using a second-order Butterworth filter and then down sampled to 250 Hz. Data was epoqued between 2500 milliseconds before and 1000 milliseconds after the onset of target stimuli and trials where extreme values were below and above the 0.5 and the percentiles were further rejected. Trials were subsequently baseline-normalized (z units) and the arithmetic average of the pupil size and its derivative for each target type per participant was estimated. The time window for baseline correction comprised −500 milliseconds to time zero. Pupil data from three participants was unavailable due to technical problems with output data files or trigger coding.
Statistics
All statistical analyses were implemented using custom-made Matlab scripts. Above-chance performance was tested using a binomial distribution (binomial test). For tgtSTDs and supraDEVs, the probability of observing x hits given a theoretical probability of 0.5 and n observations, where n is the number of trials per block was tested and data from blocks whose probability was lower than an alpha value of 0.05 were rejected (Table S2). For subDEVs, the probability of observing x incorrect responses given the same theoretical probability and n observations was calculated and the same rejection criterion was applied. For reaction times, we calculated the individual median reaction time per condition. We rejected subjects for which there was no subthreshold deviant data available and performed a two-tailed non-parametric 10.000-bootstrap resampling procedure to determine whether there was any statistical difference among conditions. We identified the percentiles corresponding to an alpha level of 0.05, Bonferroni-corrected and compared the against our observed median reaction times. For pupil data, we calculated the arithmetic mean pupil size and mean pupil rate of change across conditions. Data from blocks that failed to meet the above-chance performance criterion were not included. We plotted the times series data per condition with their 95% confidence intervals for the Standard Error of the Mean. Confidence intervals were calculated using a studentized 5000-bootstrap procedure. We then performed two-tailed 10000-bootstrapping timepoint by timepoint. For each timepoint, we tested the probability that the mean values came from the same distribution at an alpha level of 0.05, Bonferroni-corrected. Additionally, we set our algorithm to return only statistical effects that extended for more than 5 consecutive timepoints. For scalp ERPs, we performed the same procedure as for pupil data, but instead of performing the resampling procedure during the entire 1000 window, we performed two separate tests for our ERP events of interest (MMN and P3) thusly: a one-tailed 1000-bootstrap between 100 and 220 milliseconds and another one-tailed 10000 bootstrap between 200 and 350 milliseconds.
Correlations were conducted using the non-parametric Spearman correlation coefficient test. Subjects for which either pupil data or EEG data was missing were not included in the analyses.
Acknowledgements
This project was funded by CONICYT-FONDECYT Regular Grant N° 283 1160258 and ANID/CONICYT National Grant for Doctoral Studies N° 2018-21181786. We would also like to acknowledge Dr. Rodrigo Henriquez-Ch, Dr. Gonzalo Boncompte, Dr. Tomas Ossandón, Vicente Medel, Marcos Domic and Brice Follet for their advice, feedback and support.