Abstract
Which neural processes underlie our conscious experience? One theoretical view argues that the neural correlates of consciousness (NCC) reside in local activity in sensory cortices. Accordingly, local category-specific gamma band responses in visual cortex correlate with conscious perception. However, as most studies manipulated conscious perception by altering the amount of sensory evidence, it is possible that they reflect prerequisites or consequences of consciousness rather than the actual NCC. Here we directly address this issue by developing a new experimental paradigm in which conscious perception is modulated either by sensory evidence or by previous exposure of the images while recording intracranial EEG from the higher-order visual cortex of human epilepsy patients. A clear prediction is that neural processes directly reflecting conscious perception should be present regardless of how it comes about. In contrast, we observed that although subjective reports were modulated both by sensory evidence and by previous exposure, gamma band responses solely reflected sensory evidence. This result contradicts the proposal that local gamma band responses in the higher-order visual cortex reflect conscious perception.
Introduction
One central controversy in consciousness research is whether conscious perception arises locally from activity in sensory cortices (Zeki, 2001; Malach, 2007) or whether it requires integration across brain areas (Tononi and Koch, 2008; Melloni and Singer, 2010; Dehaene and Changeux, 2011). Empirical support for the idea that conscious perception is reflected in activity of sensory areas mostly stems from neuroimaging studies (Grill-Spector et al., 2000; Hesselmann et al., 2011), which are insensitive to the neural dynamics underlying the perceptual effects. Recently, however, pivotal support for a role of sensory cortices in conscious perception was gathered with intracranial EEG recordings, which are precise in time and space. Fisch and colleagues (2009) showed that category-specific gamma band responses (GBR) in the visual cortex correlate with conscious perception. Conscious recognition led to an ignition-like enhancement of the GBR, supporting the idea that conscious perception arises locally within sensory cortices. However, other findings suggest that conscious perception is grounded on the coherent activation of widely distributed cortical networks, whereas local GBR reflects local processing that can happen unconsciously (Melloni and Singer, 2010). This idea is consistent with the classic findings showing strong local stimulus-specific gamma band activity even under anesthesia (Gray and Singer, 1989).
This obvious discrepancy might be resolved when considering a recently uncovered theoretical problem that states that the methods commonly used for studying the neural correlates of consciousness are not specific for conscious experience but may reflect processes that precede or follow it (Aru et al., 2012; de Graaf et al., 2012). Thus, an alternative explanation for the findings of Fisch et al. (2009) is that category-specific GBRs in sensory cortices reflect local processing, which is a necessary prerequisite for but not the direct correlate of conscious perception. In other words, in a paradigm such as used by Fisch et al. (2009), it cannot be ruled out that the enhanced GBR just reflected more effective preprocessing of stimuli that enhanced the chances of these stimuli to get perceived consciously. It is possible that only for this reason, local processing correlated with conscious perception.
To directly test this conjecture, we manipulated conscious perception in two different ways: either by changing the amount of sensory evidence, which directly modulates local processing, or by previously exposing the patients to the pictures before the test phase (i.e., providing prior knowledge), which is a factor less dependent on local processing. As both factors enhance conscious perception to the same extent, a clear prediction is that if category-specific GBR in visual cortex reflect conscious perception, they should be modulated both by sensory evidence and previous exposure. Instead, we found that GBR were only affected by sensory evidence, directly contradicting the proposal that local GBR in sensory cortices represent a direct neural correlate of consciousness.
Materials and Methods
Subjects.
Six patients with pharmacoresistant epilepsy (three female; mean age, 27 years; age range, 18–35 years; 5 right-handed) undergoing invasive monitoring for localization of epileptogenic foci participated in the study. For presurgical screening, all patients had lateral and basal strip electrodes (4–16 stainless steel contacts, 4 mm diameter, 10 mm interelectrode spacing; AD-Tech) covering the occipital and temporal visual areas. Recordings were performed at the Department of Epileptology, University of Bonn, Germany. The study was approved by the local ethics committee, and written informed consent was obtained from all patients. Electrode locations were determined based on individual MRIs (Table 1; for an example, see Fig. 2). In total, 62 electrodes free of epileptic activity were examined. In addition, 24 healthy human subjects (17 female; mean age, 24 years; age range, 21–28 years; all right-handed) served as controls in a psychophysical experiment. All control subjects had normal or corrected-to-normal vision, reported no history of neurological or psychiatric disease, gave written informed consent, and received monetary compensation for their participation.
Procedure and stimuli.
Stimuli were displayed to the patients on a CRT monitor and to the healthy controls on a translucent screen (both had a 60 Hz refresh rate) onto which the stimuli were projected from a liquid crystal display projector. Stimulus presentation and response collection were controlled by Presentation software (Neurobehavioral Systems).
Stimuli consisted of 148 (for healthy controls: 276) grayscale pictures containing a single person in the foreground with diverse backgrounds, and a set of 22 catch images (for healthy controls: 54) only containing background. To limit image visibility, random noise was added parametrically (Fig. 1A) while keeping contrast constant across degradation levels. Stimuli were edited in Matlab (R2008b; MathWorks) using custom code. The noise levels yielding decreased visibility were determined in a pilot experiment and ranged from 60% to 90% in 5% steps. Stimuli were foveally presented at the center of the screen, spanned 4 × 3 degrees of visual angle (dva) in the horizontal and vertical plane (for healthy controls: 6 × 4.5 dva), and were surrounded by a gray background.
Before the experiment, each participant undertook a threshold experiment to determine two neighboring degradation levels that yielded a recognition performance ∼70% in the male/female task. To that end, 60 degraded images were briefly presented (150 ms) at two different degradation levels. The individually determined degradation levels were subsequently used in the main experiment.
The main experiment consisted of 11 experimental blocks. Each block comprised two phases: a familiarization phase and a test phase (Fig. 1A). Different sets of images were used per block. The familiarization phase aimed at establishing prior knowledge by exposing half of the pictures subsequently shown in the test phase. Pictures without noise were presented twice for 3 s and subjects were asked to memorize them. To assure attention to and to facilitate encoding of the images, subjects indicated via button press first the gender of the person on the picture (male/female task) and then guessed their age (older or younger than 30 years). Subsequently, pictures were presented without an explicit task, and subjects had to freely explore and memorize them.
In the test phase, degraded images were presented briefly, thus limiting their visibility. Two experimental factors controlled the images' visibility: sensory evidence and previous exposure (Fig. 1A). Sensory evidence was varied by presenting images at two degradation levels (high and low noise). Previous exposure was varied by either showing previously exposed or new images, for which no preexisting memory had been established. Additionally, to assess the reliability of the subjects' judgment, specifically for the subjective visibility task (see below), we included catch trials that only contained background on the higher degradation level. The low number of false alarms in the 44 catch trials (mean 1.5, SD 2.3) ensures that the subjects' visibility judgments were reliable.
Each trial consisted of a fixation cross on a gray background presented for a random duration (1.2–1.4 s) followed by the degraded image (150 ms). After 1 s, an objective and a subjective task were presented consecutively. In the objective task, subjects indicated the gender of the person in the picture (male/female judgment). In the subjective task, subjects reported whether they had seen a person in the picture. Occasionally, subjects were also asked to indicate whether the picture, now shown in the degraded fashion, had been presented in the familiarization phase. A block lasted 3–4 min intermixed with breaks between blocks. Forty-four trials per condition were acquired. In the control experiment, 108 trials per condition were acquired in 27 blocks.
To rule out any picture-specific effects in the neural measures, we pseudorandomized the pictures containing a person such that across four patients, each picture was assigned to every condition exactly one time. Although we report data from six patients, results are comparable when restricting the analysis to four subjects with a complete randomization.
Recording and analysis.
Recordings were performed with Harmonie recording software (Stellate Systems), amplified (Schwarzer), sampled at 1 kHz with an analog filter (0.1–300 Hz), referenced to linked mastoids, and stored for off-line analysis. Data were analyzed using the open source Matlab toolbox Fieldtrip (Oostenveld et al., 2011). Preprocessing included line noise and harmonics (50, 100, 150, 200, and 250 Hz) removal by means of discrete Fourier transform, high-pass filtering [0.5 Hz (24 dB/octave)], and re-referencing to a bipolar montage. Continuous EEG was segmented into 2-s-long epochs starting 1 s before the onset of the degraded stimuli. Segments were visually inspected for artifacts (e.g., epileptiform spikes), and channels near the epileptic focus without any signal or showing extensive artifacts were removed.
We analyzed GBR in two frequency ranges: low (30–70 Hz, lGBR) and high GBR (70–150 Hz, hGBR). Fisch et al. (2009) have reported that low GBR (30–70 Hz) in visual cortex correlates with conscious recognition. We thus tested the same frequency range to allow direct comparisons between studies. We extended the analysis to high GBR (70–150 Hz), as recent work suggests that gamma band responses in electrocorticography data have a broad band profile beginning at ∼50 Hz with the strongest responses at frequencies >70 Hz (Lachaux et al., 2005). For each trial, bipolar derivations were analyzed in the time–frequency domain by convolution with complex Gaussian Morlet wavelets with a bandwidth parameter f0/σf = 6, i.e., wavelets of approximately six cycles length. Time–frequency power values were normalized per trial by dividing them by the average power across the baseline (−400 to −100 ms prestimulus). Subsequently, we separately averaged the relative power over the frequency ranges 30–70 Hz (lGBR) and 70–150 Hz (hGBR) and obtained two-dimensional signals of time and mean power.
Category-specific electrodes were determined following a procedure similar to Fisch et al. (2009). An electrode was considered person-selective if significantly higher GBR were observed for pictures containing a person in the foreground (from the condition low sensory evidence and without previous exposure) than for those only containing the background (catch images) in the interval 100–500 ms poststimulus presentation. Importantly, this comparison was neither confounded by previous exposure (as none of the picture groups had been seen before) nor by sensory evidence (as both picture groups shared the same level of degradation), enabling the extraction of person-specific signatures independently of the experimental factors. Figure 2 shows the person-specific power spectra obtained by averaging raw power values in the time domain over the analyzed time window 100–500 ms poststimulus.
Statistical analysis.
Our analysis focuses on the subjective task, as our goal was to address the neural profile that correlates with subjective reports about conscious perception. The percentage of “seen” responses in the subjective task was analyzed with repeated-measures ANOVA with factors previous exposure (previously exposed vs novel) and sensory evidence (high vs low degradation). For the electrophysiological data, we restricted our analysis to the main effects (sensory evidence, previous exposure), as the behavioral results consistently revealed no interaction (see Results, below), which enabled us to increase statistical power by pooling trials.
We also directly investigated power modulations in relation to subjective visibility by contrasting trials where subjects reported having perceived a person with those in which they did not. We restricted this analysis to the condition with lower sensory evidence, as it contained comparable and relatively high amounts of both “seen” and “unseen” person trials. We extracted the same amount of “seen” and “unseen” person trials from the conditions with and without previous exposure, such that taken together, the conscious recognition contrast was independent of both experimental factors.
Statistical analysis were performed at the single electrode level (Lachaux et al., 2005; Fisch et al., 2009), whereby the single-trial time courses of the lGBR and hGBR were subjected to a nonparametric Wilcoxon rank-sum test contrasting factors: sensory evidence, previous exposure, and visibility. A population analysis was also performed whereby responses were averaged per condition, per electrode, and experimental factors were compared across the electrodes in Table 1. To limit the number of statistical tests, activity was averaged over 50 ms windows (no overlap) and compared over the period 100–500 ms after stimulus onset (8 points). Results were corrected for multiple comparisons using false discovery rate across all time windows and electrodes.
Post hoc power analysis was done with Monte Carlo simulations. Trials were randomly assigned into two groups; 30% of the mean of the original distribution was added to one group to mimic the shift due to an experimental factor followed by a Wilcoxon test between the groups. This was repeated 10,000 times. Here power corresponds to the proportion of comparisons the Wilcoxon test correctly identifies as “different.”
Results
Behavioral results
In the group of six patients, sensory evidence and previous exposure both modulated the visibility of the persons on the pictures, i.e., subjects reported to perceive the person on the degraded picture more often when pictures had either lower degradation (F(1,5) = 25.626, p = 0.004) or when subjects were previously exposed to the clear version of the picture (F(1,5) = 7.463, p = 0.041) (Fig. 1B). To assess whether sensory evidence and previous exposure enhance conscious perception of the person to a comparable degree, we contrasted the increase in “seen” responses resulting from changes in sensory evidence (percentage of perceived low degradation − percentage of perceived high degradation) with the corresponding increases resulting from previous exposure (percentage of perceived with previous exposure − percentage of perceived without previous exposure). Both factors boosted subjective reports similarly (t(5) = 1.510, p = 0.191), which is also corroborated by the absence of an interaction of these factors (p > 0.9). We confirmed these results in a large sample of healthy subjects. Here, both previous exposure (F(1,23) = 70.167, p = 1.93E−8) and sensory evidence (F(1,23) = 69.109, p = 2.21E−8) had a significant effect on subjective perception. Importantly, even with a larger sample, we did not observe an interaction between both factors (p > 0.4).
We further confirmed that previous exposure and sensory evidence similarly boost perception in an additional experiment (n = 8), in which we used a fine-grained 11-point perceptual rating scale (between “no perception of the person” and “clear perception of the person”). Even with this fine-scale perceptual task, both previous exposure (F(1,7) = 53.072, p = 1.65E−4) and sensory evidence (F(1,7) = 23.526, p = 0.002) modulated perception similarly (t test on the difference of the effects: t(7) = 0.237, p = 0.891).
The main question of the current work was whether the perceptual enhancement through sensory evidence and previous exposure is reflected in corresponding increases of GBR. If category-specific GBR correlate with conscious experience, GBR should be affected both by sensory evidence and by previous exposure, as both modulate subjective reports about conscious perception.
Category-specific GBR in the visual cortex
We restricted our analysis to electrodes on the lateral (30) and ventral (32) surface of the occipital and temporal cortex, for which extensive coverage existed. Importantly, GBR in those higher-order visual areas have been associated with conscious perception (Fisch et al., 2009). First, we determined which electrodes showed selective GBR responses to the person in the picture (Fisch et al., 2009). Person-specific enhancement of hGBR was observed on seven electrodes belonging to five different patients, which all were localized on the surface of the lateral occipital cortex and fusiform gyrus, in good agreement with the localization of the body-selective areas of the human brain (Table 1) (Peelen and Downing, 2007). In the lGBR range, only one electrode showed person-specific effects that survived the correction for multiple comparisons. However, electrodes with (corrected) effects in hGBR mostly had congruent uncorrected effects in the lGBR (Table 1). Thus, hGBR and lGBR may reflect a similar process, which is stronger at higher frequencies. To confirm this possibility, we analyzed the spectral changes over the whole gamma frequency band. Figure 2 shows that the differences between pictures containing a person and those with only background started at ∼50 Hz and continued throughout higher frequencies, suggesting that lGBR and hGBR are part of the same broadband process. Interestingly, Fisch et al. (2009) also reported effects starting at 50 Hz that continued to the highest analyzed frequency (70 Hz). As there was no visible indication of attenuation in power in Fisch et al. (2009), it is likely that higher frequencies exhibited a similar response. We thus focus on the hGBR, as the effects are stronger there. However, all effects were similar but smaller in the lGBR (Table 1).
Sensory evidence but not previous exposure modulates category-specific GBR
In line with Fisch et al. (2009), we observed a strong and sustained GBR that outlasted the stimulus presentation (150 ms) when subjects reported having perceived the person in the picture. This was observed in 70% (5/7) of the person-specific electrodes in four different patients. Figure 3A shows data from three exemplary electrodes. We then asked whether those responses are similarly modulated by sensory evidence and previous exposure, as both factors enhance subjective perception of the persons.
We observed that hGBR was significantly higher when sensory evidence was stronger (Fig. 3B). However, no difference in hGBR was observed between pictures with and without previous exposure, despite the fact that previous exposure enhanced conscious perception similarly to sensory evidence. hGBR was significantly modulated by sensory evidence in 85% (6/7) of the person-specific electrodes in four different patients, but none of them showed an effect of previous exposure (Fig. 3C). Table 1 shows that only one electrode (nr1) barely approached the uncorrected level of significance for previous exposure, while the other five electrodes with strong effects of sensory evidence exhibited uncorrected p values > 0.1. These results were confirmed by the population analysis, where we only observed effects of sensory evidence (p < 0.05, 100–450 ms poststimulus, except for 150–200 ms, where p = 0.053) but not of previous exposure (p > 0.2 for all time windows). Power analysis revealed that the mean power of our test over the seven electrodes and eight time windows was 0.94 and 0.9 for a 30% change at the uncorrected and corrected level, respectively, confirming the robustness of our methods to detect a potential effect of previous exposure.
Discussion
We first demonstrated that conscious perception is similarly enhanced by increasing the amount of sensory evidence and by previously showing the pictures. We argue that both factors indeed enhance perception rather than later memory processes, as our subjective task was about perception (“did you perceive the person on the picture”) and we replicated our results when we used an 11-point perceptual rating scale. Establishing that both factors boost subjective reports about conscious perception to the same extent allowed us to directly examine whether category-specific GBR in visual cortex reflect conscious perception or unconscious local processing. We observed a clear effect of sensory evidence on GBR, with higher amplitudes when sensory evidence led to a higher proportion of seen trials. However, although previous exposure also enhanced perception and the proportion of seen trials just like sensory evidence, it had no effect on the GBR (Fig. 3C). The possibility that sensory evidence is simply more effective can be excluded as both factors had numerically similar effects with comparable gains in conscious perception. Moreover, even in those subjects where previous exposure had a numerically stronger impact on perception than sensory evidence, GBR was exclusively modulated by the latter.
Another possibility is that previous exposure leads to reduced neural responses because of repetition suppression, masking a simultaneous enhancement of GBR. However, repetition suppression is typically observed only when the same, un-degraded image is repeated, while repetition leads to enhancement when degraded images are repeated (Müller et al., 2012). This is thought to be due to increased extraction of information. Thus, we should have observed enhanced responses as we used degraded images, which was not the case. Perception appears to be modulated equally by previous exposure and sensory evidence, but this effect manifests itself in the local GBR only in the case of sensory evidence. It is this lack of invariance of the local GBR that makes it unlikely that GBR reflect a mechanism directly responsible for conscious perception.
What could be the reasons for the discrepancy between our results and those obtained by Fisch et al. (2009) from which these authors concluded that local category-specific GBR are markers of conscious perception? It has been remarked previously that the contrastive method, i.e., the direct comparison between trials with and without conscious recognition, does not exclusively reflect neural processes directly and uniquely associated with conscious experience of the target but also those that precede or follow conscious experience (Aru et al., 2012; de Graaf et al., 2012). In fact, early studies have shown that such local stimulus-specific increases in gamma band activity are observed even under anesthesia (Gray and Singer, 1989) and might thus reflect low-level unconscious processing that precedes conscious experience. In typical experimental paradigms, differences in local unconscious processing might lead to different outcomes regarding conscious perception, and therefore, specific local responses can correlate with conscious perception without necessarily being direct correlates of conscious experience.
Numerous contemporary theories of consciousness propose that conscious experience is not related to the depth of local cortical processing but is grounded in long-range integration across brain areas (Tononi and Koch, 2008; Melloni and Singer, 2010; Dehaene and Changeux, 2011). It remains an open question whether sensory evidence and previous exposure similarly enhance interareal integration (Melloni et al., 2007; Gaillard et al., 2009). Due to limitations in the electrode coverage, we were unable to test this prediction in the current study. Until the relationship between long-range integration and conscious perception is not disputed by experimental paradigms like ours, such interareal interactions remain a viable candidate for mediating conscious perception.
By going beyond the contrastive analysis, we could directly test pivotal predictions arising from the experimental work of others (Fisch et al., 2009) and dissect neural processes that precede consciousness from those that are unique for consciousness. We hope that this result encourages researchers in the field to develop novel experimental paradigms that will help to distill the neural events proper that underlie our conscious experience.
Footnotes
This work was supported by the Max Planck Society, LOEWE Neuronale Koordination Forschungsschwerpunkt Frankfurt to L.M. and W.S., and by an Emmy-Noether grant to N.A. We thank R. Rutiku for collecting the control data and two anonymous reviewers for their insightful suggestions. We are indebted to T. Bachmann and C. M. Schwiedrzik for helpful discussions.
- Correspondence should be addressed to either Jaan Aru or Lucia Melloni, Max Planck Institute for Brain Research, Department of Neurophysiology, Deutschordenstrasse 46, 60528 Frankfurt am Main, Germany. jaan.aru{at}gmail.com or lucia.melloni{at}brain.mpg.de