Abstract
The study of unconscious processing requires a measure of conscious awareness. Awareness measures can be either subjective (based on participant’s report) or objective (based on perceptual performance). The preferred awareness measure depends on the theoretical position about consciousness, and may influence conclusions about the extent of unconscious processing and about the neural correlates of consciousness. We obtained fMRI measurements from 43 subjects while they viewed masked faces and houses that were either subjectively or objectively invisible. We show that neural representations of objectively invisible faces and houses are limited to visual (shape-related) object properties, while subjectively invisible stimuli are processed up to more abstract, categorical levels of representation. These results demonstrate that the hypothesized extent of unconscious information processing is determined by the measurement approach. Furthermore, our data show that subjective and objective approaches are associated with different neural correlates of consciousness and thus have implications for neural theories of consciousness.
Introduction
Determining the function and neural correlates of human consciousness is one of the most challenging topics in psychology and cognitive neuroscience today1–3. The scientific study of consciousness requires pitting conscious processes against comparable unconscious processes4. One powerful approach is to compare neural processing between stimuli presented outside conscious awareness and stimuli that are consciously perceived. Although all major theories of consciousness are based on the notion that stimuli can be processed unconsciously5–7, the scope and extent of unconscious processing are highly debated, with estimates ranging from low-level perceptual analysis8,9 to high-level object categorization10–12 and full-blown unconscious cognition and reasoning13,14. One important cause of this controversy relates to theoretical disagreements about how to measure consciousness and how to demonstrate absence of conscious awareness10,15.
The most intuitive approach is to simply ask participants to introspectively report their experience of a barely perceivable (e.g. masked) stimulus16. Recent studies adopting such subjective awareness measures found that visual stimuli reported as “invisible” still undergo high-level processing. For example, subjectively invisible stimuli are encoded17,18 and stored19–21 in visual cortex, enabling above-chance perceptual discrimination even after a memory delay22,23. By linking behavioral performance for subjectively invisible stimuli to brain activity, the neural correlates of “blindsight”-like performance can be recorded in normal observers24. Subjective measures assume that an observer’s decision to report awareness accurately distinguishes the presence vs. absence of conscious perceptual information. However, this assumption is at odds with research on perceptual decision-making, which shows that subjective measures do not accurately reflect perception, because they are susceptible to decision biases25,26. Participants often have a conservative bias27, such that an “invisible” response merely indicates that a stimulus was relatively difficult to see28. Thus, subjective measures may misclassify (partially) conscious stimuli as unconscious. Studies adopting purely subjective measures therefore most likely overestimate the extent of unconscious processing25,29,30.
To convincingly rule out conscious perception, objective awareness measures based on performance are required25 and crucially, these measures should target the key stimulus characteristic of interest31. For example, when contrasting neural responses evoked by masked faces and houses, participant’s performance in discriminating the two stimulus categories should not exceed what is expected by chance32. Unfortunately, this approach is also riddled with challenges. First, statistically, scientists have taken a failure to reject the null hypothesis (of chance performance, p > .05) as support for the null hypothesis, which is invalid in the standard frequentist hypothesis testing framework. A non-significant effect can be related to measurement noise and lack of statistical power rather than genuine invisibility (chance performance). In neuroimaging studies in particular, this represents a serious concern as sample sizes tend to be very small (e.g. only four33, five32,34, six35, seven36, or eight37 participants). Further, objective awareness is often only measured “offline” in a block separate from the fMRI recordings (or even outside the MRI scanner)34–41, and with considerably fewer trials34,36,38–40 and sometimes also fewer participants37,39. Given these issues, it is unknown how stimuli that are genuinely invisible according to the objective definition are represented in different cortical areas.
Here, our first goal was thus to measure the extent of unconscious processing of objectively invisible stimuli in human visual cortex. To ensure sufficient statistical power, we included data from 43 human observers who completed a large number of trials while measuring brain activity evoked by masked faces or houses using a multi-band fMRI sequence with fast acquisition time and high spatial resolution. An “online” (trial-by-trial) measure of perceptual discriminability of faces and houses served as an exhaustive31,42 measure of objective awareness, leading to equal numbers of trials to calculate objective discrimination performance and to evaluate the extent of neural processing of masked face/house stimuli. Perceptual sensitivity was analyzed with Bayesian statistics to establish genuine objective invisibility.
Compared to subjective invisibility, objective invisibility requires strong reduction of stimulus strength, e.g. by brief presentation times, low contrasts or strong masking, which reduces neural responses. Estimates of unconscious processing, both in behavior and in neural recordings, may thus critically depend on selection and implementation of method and statistics for establishing absence of conscious awareness. However, as objective and subjective awareness measures have not been compared in the same study, it is unclear whether the two approaches are merely associated with quantitative differences (e.g. subjective measures allowing for stronger neural responses to unconscious stimuli) or whether important qualitative differences exist (e.g. subjective measures allowing for unconscious processing of distinct higher-level stimulus properties).
Hence, our second goal was to directly compare the two approaches. We used fMRI to measure brain activity evoked by masked faces or houses that were visible, subjectively invisible, or objectively invisible. In the objective condition, visibility was controlled experimentally by fixing the contrast of the masks. In the subjective condition, visibility was controlled by the participant’s response (i.e. visibility became the dependent variable), and mask contrast was continuously adjusted to yield similar proportions of trials rated as subjectively visible and invisible. This approach yielded a similar number of trials for objectively and subjectively invisible stimuli, resulting in similar statistical power to detect neural effects in objective and subjective conditions. Using multivariate pattern discrimination, we tracked neural representations of faces and houses along the visual processing hierarchy, from early visual cortex to object-selective lateral occipital complex (LOC43,44), as well as in category-selective regions in the lateral occipital (occipital face area OFA45 and occipital place area OPA46) and ventrotemporal cortices (fusiform face area FFA47 and parahippocampal place area PPA47). This analysis allowed us to establish the level of representation of objectively and subjectively invisible stimuli in human visual cortex.
Results
On every trial, participants discriminated between faces and houses (objective measure, face/house) and simultaneously indicated stimulus visibility (subjective measure, visible/invisible). Participants were asked to be as accurate as possible, guessing when necessary. For the subjective measure, instructions emphasized that participants should press “visible” even when they had only a vague idea of the stimulus category, and press “invisible” only when they had absolutely no idea of the stimulus category. Thus, subjective invisibility on our binary scale corresponded to the lowest visibility level of more fine-grained visibility scales, such as the four-point Perceptual Awareness Scale (PAS)16. Faces and houses were presented for 16.7 ms, sandwiched between masks (Figure 1a). Subjective stimulus visibility was based on the participant’s response, while objective stimulus visibility was controlled by the experimenter. In the subjective condition (50% of trials) mask contrast was adjusted through an adaptive 1-up 1-down “staircase” procedure. Following a “visible” response, mask contrast was increased by 4%, whereas following an “invisible” response, contrast was lowered by 4%. This continuous adjustment was intended to yield a similar number of subjectively-visible (subj-vis) and subjectively-invisible (subj-inv) trials. This allowed us to test the effect of subjective visibility with minimal differences in mask contrast (subj-vis: mean contrast 7.8%, SD 4.4; subj-vis: mean contrast 11.9%, SD 4.3). In the objective condition (50% of trials) mask contrast was either low (2%) to achieve clear stimulus visibility (objectively visible (obj-vis) condition) or high (100%) to achieve chance-level discrimination (objectively invisible (obj-inv) condition).
Masking-efficiency experiment
We first ran a separate behavioral experiment to measure the influence of mask contrast on perceptual discriminability and to determine optimal mask contrast for objective invisibility (N = 17, Figure 1b). Face/house discriminability and subjective visibility (Figure 1c) increased similarly with decreasing masking strength (F(8, 128) = 247.15, p < .001, ηp2 = .94, BF10 = 2.07×1074, and F(8, 128) = 215.69, p < .001, ηp2 = .93, BF10 = 8.99×1069, respectively). A set of one-tailed t-tests showed significant above-chance performance for all contrasts (all p < .005, all dz > 0.75, all BF+0 > 14), except the 100% contrast (M = 0.03, SD = 0.41, t(16) = 0.26, p = .40, dz = 0.06, BF0+ = 3.26). Bayes factor (BF0+)1 indicated that the null hypothesis of chance performance was about three times more likely than the alternative hypothesis of above-chance performance, which represents “moderate” evidence for the null hypothesis48. Thus, full mask contrast was necessary to achieve objective invisibility and was therefore used in the following fMRI experiment.
Behavior
In the fMRI experiment (N = 43), face/house discriminability (d’) was high in obj-vis (M = 4.50, SD = 0.63) and in subj-vis trials (M = 3.50, SD = 0.93) and remained above chance in subj-inv (M = 1.20, SD = 0.94, t-test against chance, t(42) = 8.36, p < .001 (one-tailed), dz = 1.26, BF+0 = 8.59×109, Figure 1d). Such above-chance performance for subjectively invisible stimuli is often referred to as a blindsight-like phenomenon of unconscious stimulus processing. Importantly, in obj-inv trials discrimination performance (M = 0.02, SD = 0.24) did not differ significantly from chance (Figure 1e), with moderate evidence for the null hypothesis of chance-level discrimination (t(42) = 0.45, p = .33 (one-tailed), dz = 0.07, BF0+ = 4.13). Regarding subjective visibility, participants reported that the stimulus was visible in 94.6% (SD 5.5) of the obj-vis trials and invisible in 96.3% (SD 5.1) of the obj-inv trials. In the subj-vis and subj-inv trials mean subjective visibility was by definition 100% and 0%, respectively, because these trial categories were conditioned on the subjective visibility response.
Category-specific information in visual cortex
We tested in which subregions of visual cortex activation patterns reliably distinguished between faces and houses. To obtain a measure of category information in the neural responses, activity patterns were correlated with “benchmark” patterns for unmasked faces and houses obtained in an independent localizer scan where participants did a simple one-back task on stimuli presented in a block design. This approach is statistically powerful and is often considered to isolate perceptual representations from later cognitive processes related to the task, decision, and motor processes49,50. Category information was quantified as the difference between within- and between-category multi-voxel pattern correlations (Δr) between the independent localizer and every visibility condition in the main experiment for four separate regions of interest (ROIs, see Figure 2; for complimentary univariate analyses see Supplementary material).
First, we examined activity patterns in primary visual cortex (V1) and object-selective visual cortex (lateral occipital complex [LOC]43). There were marked differences between the regions, with as expected overall greater category information in LOC than in V1 (F(1, 42) = 260.39, p < .001, ηp2 = .86, BF10 = 4.53×1031). Category information also differed between visibility conditions (F(3, 126) = 79.03, p < .001, ηp2 = .65, BF10 = 1.58×1028) and these differences were more pronounced in LOC than in V1 (interaction, F(3, 126) = 99.86, p < .001, ηp2 = .70, BF10 = 1.48×1030). To test whether activity patterns discriminated between faces and houses with above-chance accuracy, separate one-tailed t-tests were carried out for every region and every visibility condition. V1 carried significant category information in obj-vis (t(42) = 3.44, p < .001, dz = 0.52, BF+0 = 46.73), subj-vis (t(42) = 3.97, p < .001, dz = 0.60, BF+0 = 188.02) and obj-inv (t(42) = 2.55, p = .007, dz = 0.39, BF+0 = 5.74), but not in subj-inv (t(42) = 1.12, p = .135, dz = 0.17, BF0+ = 1.97). Activity patterns in LOC discriminated between faces and houses with above-chance accuracy in all visibility conditions (obj-vis (t(42) = 16.69, p < .001, dz = 2.55, BF+0 = 2.43×1017), subj-vis (t(42) = 12.88, p < .001, dz = 1.96, BF+0 = 3.52×1013), subj-inv (t(42) = 5.21, p < .001, dz = 0.80, BF+0 = 7.22×103) and also in obj-inv (t(42) = 3.28, p = .001, dz = 0.50, BF+0 = 31.19). Thus, LOC contained category information for both subjectively and objectively invisible stimuli.
To gain insight into the level of representation in higher-level visual cortex, we examined activity patterns in posterior vs. anterior category-selective regions, which are partially overlapping with the spatially extended LOC region (Figure 2). The two more posterior areas represent more basic and “local” visual aspects of faces and houses (occipital face area OFA45 and occipital place area OPA46,51), whereas the two more anterior areas also contain “global” and abstract categorical representations (fusiform face area FFA and parahippocampal place area PPA47). Overall, anterior regions carried more category information than posterior regions (F(1, 42) = 47.09, p < .001, ηp2 = .53, BF10 = 4.53×1031). There were also significant differences between visibility conditions (F(3, 126) = 63.76, p < .001, ηp2 = .60, BF10 = 1.58×1028) and a significant interaction (F(3, 126) = 26.94, p < .001, ηp2 = .39, BF10 = 1.48×1030). Activity patterns in the posterior areas (OFA/OPA) discriminated between faces and houses in all conditions (obj-vis [t(42) = 10.55, p < .001, dz = 1.61, BF+0 = 6.99×1010], subj-vis [t(42) = 6.99, p < .001, dz = 1.07, BF+0 = 1.72×106], subj-inv [t(42) = 3.90, p < .001, dz = 0.59, BF+0 = 155.49] and obj-inv [t(42) = 2.72, p = .005, dz = 0.42, BF+0 = 8.27]). Also in the anterior areas (FFA/PPA) category information was also significant in all visibility conditions. Strong evidence for above-chance discrimination of faces and houses was obtained only for obj-vis (t(42) = 11.43, p < .001, dz = 1.74, BF+0 = 7.89×1011), subj-vis (t(42) = 9.61, p < .001, dz = 1.47, BF+0 = 4.86×109) and subj-inv (t(42) = 2.72, p = .005, dz = 0.90, BF+0 = 5.88×104). By contrast, for obj-inv there was no solid evidence for above-chance discrimination in FFA/PPA, with only weak and inconsistent statistical evidence (t(42) = 1.97, p = .028, dz = 0.30, BF+0 = 1.86).
Differences between visibility conditions
To directly test the effect of different definitions of (in)visibility, we compared activity patterns using paired t-tests (two-tailed). Both in V1 and LOC subjective visibility was associated with greater category information than subjective invisibility (V1: t(42) = 2.85, p = .007, dz = 0.43, BF10 = 5.57; LOC: t(42) = 9.68, p < .001, dz = 1.48, BF10 = 3.00×109), but this effect was larger in LOC (interaction, F(1, 42) = 62.93, p < ..001, ηp2 = .60, BF10 = 7.99×105), indicating a greater effect of subjective awareness in object-selective visual cortex than in early visual cortex. Also the way of establishing invisibility influenced the two regions differently (interaction, F(1, 42) = 9.02, p = .004, ηp2 = .18, BF10 = 11.14). While there was no significant difference in category information between subjectively and objectively invisible stimuli in V1 (t(42) = −0.73, p = .47, dz = 0.11, BF01 = 4.72), in LOC category information was significantly greater for subjectively than for objectively invisible stimuli (t(42) = 2.67, p = .011, dz = 0.41, BF10 = 3.72).
Similar differences were obtained for posterior vs. anterior category-selective regions. Although both posterior and anterior regions carried more category information for subjectively visible than subjectively invisible stimuli (OFA/OPA: t(42) = 4.24, p < .001, dz = 0.64, BF10 = 195.43; FFA/PPA: t(42) = 6.42, p < .001, dz = 0.98, BF10 = 1.49×105), this effect was larger in FFA/PPA (interaction, F(1, 42) = 11.41, p = .002, ηp2 = .21, BF10 = 5.66), indicating a greater effect of subjective awareness in anterior than in posterior category-selective regions. Critically, also the method of establishing invisibility had a different effect on the two ROIs (interaction, F(1, 42) = 20.43, p < .001, ηp2 = .33, BF10 = 28.18). In OFA/OPA there was no significant difference between subjectively and objectively invisible stimuli (t(42) = 0.79, p = .44, dz = 0.12, BF01 = 4.50), while FFA/PPA carried more category information for subjectively than for objectively invisible stimuli (t(42) = 4.23, p < .001, dz = 0.64, BF10 = 195.71).
Posterior-anterior category information gradient
Another way to follow up on the significant visibility × ROI interactions is to test differences in category information between ROIs, separately for every visibility condition (line plots in Figure 2). For obj-vis, subj-vis, and subj-inv this analysis revealed a posterior-anterior gradient, with increasing category information from V1 to LOC (all t(42) > 4.50, p < .001, dz > 0.70, BF10 > 596.40, Figure 2a), and from OFA/OPA to FFA/PPA (all t(42) = 4.45, p < .001, dz > 0.67, BF10 > 373.24, Figure 2b). For obj-inv, this gradient was not robust. Category information did not differ between V1 and LOC (t(42) > 1.70, p = .097, dz = 0.26, BF01 = 1.63) or between OFA/OPA and FFA/PPA (t(42) = 0.20, p = .842, dz = 0.03, BF01 = 5.95), indicating that processing of objectively invisible stimuli is limited to visual shape features.
Whole-brain searchlights
The ROI analyses showed that both subjective visibility (subjectively visible vs. subjectively invisible) and the method for establishing invisibility (subjectively invisible vs. objectively invisible) have the strongest effects in higher-level visual cortex. To further substantiate these findings and to test their spatial specificity, we conducted “searchlight” analyses52. These analyses revealed areas carrying significant category information across the whole brain (corrected for multiple comparisons via false discovery estimation53, p < .05) in the obj-vis, subj-vis, and subj-inv condition, but not in the obj-inv condition (Figure 3a). In the three former visibility conditions, clusters with significant category information were located in bilateral fusiform gyrus, lingual gyrus, parahippocampal gyrus, and inferior occipital gyrus – areas overlapping with the high-level visual cortical areas defined in our ROI analyses. Compared to subjectively invisible stimuli, subjectively visible stimuli were associated with greater category information in the very same regions, and in bilateral fusiform gyrus in particular, thus confirming the results from the ROI analyses (Figure 3b). Additional clusters for subjectively visible stimuli were located in bilateral inferior temporal gyrus, mid occipital gyrus, superior occipital gyrus, left superior parietal lobule, precuneus, and left amygdala. When directly contrasting subjectively invisible with objectively invisible stimuli, there was greater category information for subjectively invisible stimuli in right fusiform gyrus and in left anterior fusiform gyrus extending into left inferior temporal gyrus (Figure 3b).
Correlates of subjective awareness
As mask contrast covaried with subjective visibility, stronger category representations in the subj-vis than in the subj-inv condition could reflect differences in both stimulus strength and subjective awareness. To control for the effect of stimulus strength, we modeled brain responses in the subjective condition (separately for faces and houses) with a parametric regressor reflecting trial-by-trial mask contrast and an additional regressor for visibility (subj-vis, subj-inv). The visibility regressor was orthogonalized with respect to the mask-contrast regressor, assigning the mask-contrast regressor all shared variance, such that only variance not explained by the mask-contrast regressor was assigned to the visibility-regressor54. Note that if fluctuations in subjective awareness and in mask contrast reflected partially shared processes, such as fluctuations in arousal or attention, these would be captured by the mask-contrast regressor, rendering this analysis a conservative approach for determining the neural correlates of subjective awareness.
Multi-voxel pattern correlations between the localizer data and the face-/house patterns adjusted for mask contrast (Figure 4a) revealed strong effects of subjective visibility (one-sample t-test, two-tailed) in the more anterior areas LOC (t(42) = 3.57, p < .001, dz = 0.54, BF10 = 32.03) and FFA/PPA (t(42) = 3.08, p = .004, dz = 0.47, BF10 = 9.45), but only inconsistent effects in the more posterior areas V1 (t(42) = 1.96, p = .056, dz = 0.30, BF10 = 0.94) and OFA/OPA (t(42) = 2.21, p = .033, dz = 0.34, BF10 = 1.46). Note, however, that there were no significant differences between posterior and anterior ROIs (both t(42) < 1.37, p > .17, dz < 0.21, BF01 > 2.57), limiting claims about the spatial specificity of the effect. To determine whether subjective awareness enhanced category information specifically in occipitotemporal regions, we conducted a whole-brain searchlight analysis using the model controlling for mask contrast. Even at a more liberal statistical threshold (p < .001, uncorrected) this analysis revealed only larger clusters with greater category information in subj-vis than in subj-inv only in bilateral fusiform gyrus (Figure 4b) and in bilateral mid occipital gyrus (additional small clusters were located in superior parietal lobule). This further highlights the spatial sensitivity of our effects and the a-priori selection of ROI’s.
Correlates of blindsight-like discrimination performance
To test for brain areas involved in blindsight-like unconscious perception24, we harnessed the fact that there was considerable interindividual variability in behavioral discriminability of subjectively invisible stimuli (Figure 5a). The phenomenon of blindsight refers to above-chance behavioral performance in the absence of subjective awareness of the stimulus on which these behavioral effects are based55. For the subj-inv condition, correlations between perceptual discriminability (d’, behavioral performance) and category information in ROIs (Δr) were significant in LOC (p < .001), OFA/OPA (p = .028) and FFA/PPA (p = .001), but not in V1 (p = .88). One concern with these analyses is that mask contrast in the subj-inv condition was adjusted dynamically for each observer and thus differed between participants. Indeed, mask contrast was (negatively) correlated with behavioral performance (r(41) = −.40, p = .008, BF10 = 5.74). We therefore computed partial correlations between perceptual discriminability and the ROI data, controlling for the effect of mask contrast. As can be seen in Figure 5b, also with this analysis there were significant correlations between discrimination performance and category information in LOC (r(41) = .38, p = .013, BF10 = 4.68) and in FFA/PPA, (r(41) = .38, p = .012, BF10 = 4.89), but the correlation with OFA/OPA was not significant anymore (r(41) = .29, p = .064, BF10 = 1.43). To examine the spatial specificity of these correlations, we calculated a correlation between perceptual discriminability and the subj-inv whole-brain searchlight. Again, even at a liberal statistical threshold (p < .001, uncorrected, Figure 5c) this analysis only revealed a cluster in left fusiform gyrus, further indicating the spatial specificity of the results.
Discussion
In the absence of a gold standard for measuring consciousness, scientists are firmly divided into those who use subjective measures based on participant’s reported experience and those who use objective measures based on participant’s perceptual discrimination performance10,15. Subjective measures allow for greater stimulus strength (e.g. lower mask contrast) than objective measures and this may result in greater estimates of unconscious processing. Using fMRI we found that although activity patterns in visual cortex carried representations of both subjectively and objectively invisible faces and houses, there were quantitative and qualitative differences between the two approaches. Subjectively invisible stimuli were more strongly represented in object-selective visual cortex than objectively invisible stimuli. For subjectively invisible stimuli, similar to visible stimuli, there was a clear posterior-to-anterior gradient in visual cortex, with stronger category information in ventrotemporal cortex (FFA/PPA) than in occipital cortex (V1, OFA/OPA). For objectively invisible stimuli, category information remained virtually unchanged from early visual cortex to object- and category-selective areas. We interpret this as an indication that representations of objectively invisible stimuli are likely limited to visual (shape-related) object properties, while subjectively invisible stimuli are processed up to more abstract, categorical levels of representation. These differences in neural processing of subjectively vs. objectively invisible stimuli can account for some of the current controversies on the scope and extent of unconscious processing8,13.
Unconscious processing of objectively invisible stimuli has been notoriously difficult to demonstrate. Rendering stimuli perceptually indiscriminable requires very low stimulus strength (e.g. high mask contrast), which strongly reduces neural responses in visual cortex, especially at higher levels in the cortical hierarchy5. As a result, the use of objective measures may underestimate unconscious processing. Indeed, several previous fMRI studies failed to obtain visual cortex responses to invisible faces37,56, shapes57, and objects24,58. Here, although maximum mask contrast was required for objective invisibility, perceptually indiscriminable faces and houses still evoked distinct activity patterns in visual cortex. Category information could be decoded from early visual cortex, from object-selective LOC, from the (largely overlapping) category-selective areas in lateral occipital cortex (OFA/OPA), and, less robustly, from ventrotemporal areas (FFA/PPA), while participants were unable to discriminate these categories, thereby satisfying the most rigorous criteria for establishing absence of awareness (see also Figure S1 and accompanying supplementary results and discussion for more details on this issue). Previous fMRI studies that found visual cortex responses to masked stimuli did not convincingly rule out conscious processing because these studies had very small sample sizes32–37 and measured awareness with fewer trials34,36,38–41 (or even fewer participants37,39), often in a separate block outside the scanner34–40. With low power a non-significant effect in the awareness measure (p > .05) is not surprising and hard to interpret. Even if both the awareness measure and the fMRI data reflected the same underlying (conscious) process, failure to reject the null hypothesis is expected to occur more frequently for the measure with lower power (the awareness measure). To address these concerns, we tested a much larger sample than previous fMRI studies on unconscious processing, collected objective awareness measures on all 200 trials of the objectively invisible condition during scanning, and calculated Bayes factor to quantify the evidence favoring the null hypothesis of zero face/house discriminability vs. the alternative hypothesis of above-chance discrimination performance59. Finally, no participants or trials needed to be excluded based on high awareness scores. Such data exclusion is a common procedure that risks severely inflating estimates of unconscious processing due to regression to the mean60. On a cautionary note, however, although we adopted the highest standards for demonstrating unconscious processing with fMRI to date, our claim of complete absence of awareness depends on specific statistical assumptions such as the prior in Bayesian analyses (see also supplementary results and discussion). Notwithstanding these limitations, our findings provide more conclusive fMRI evidence for unconscious processing of objectively invisible stimuli in human visual cortex.
While representations of objectively invisible stimuli were of similar strength along the posterior-anterior axis from early visual cortex to object- and category-selective areas, subjectively invisible stimuli – similar to visible stimuli – were most strongly represented in category-selective areas in ventrotemporal cortex (FFA/PPA). These differences in neural representation may provide an account for discrepant findings on the influence of unconscious stimuli on behavior, for example in visual priming experiments12,61,62. For objectively invisible stimuli our findings predict priming effects based on visual features such as shape, but not based on semantic meaning such as category membership (e.g. gender, emotion, animacy). Indeed, while response priming from objectively invisible shapes (e.g. left-vs. right-pointing arrow63, square vs. diamond64) is well established, semantic priming from objectively invisible pictures, where visual effects are ruled out (e.g. a picture of an animal priming a word referring to an animal) is heavily debated12,61,62. When objective discrimination performance is well above chance, as for subjectively invisible stimuli in the present study, masked primes elicit robust semantic processing65. Recent studies indicate that subjectively invisible stimuli reach even higher levels of processing, including crossmodal semantic integration66 and working memory67. The present findings will resonate with the idea that subjectively invisible stimuli are processed in a way that is qualitatively similar to (clearly) visible stimuli. Studies that establish absence of awareness with subjective measures may thus be expected to conclude that unconscious processes can perform basically the same functions as unconscious processes13, thereby challenging influential theories of consciousness such as global workspace theory (GWT)66,67.
Activity patterns in LOC and FFA/PPA also predicted perceptual discrimination performance for subjectively invisible stimuli, representing a neural correlate of blindsight-like unconscious perception in healthy human observers24. For proponents of the use of subjective measures to study unconscious processes, this could be regarded as strong evidence that unconscious perception shares neural mechanisms with conscious perception, providing further support for the idea that consciousness has little functional role in human perception and cognition13. However, for proponents of objective awareness measures above-chance performance for subjectively invisible stimuli simply means that these stimuli were in fact not invisible. According to this view, differences in perceptual performance on subjectively invisible trials could reflect differences in response criteria (willingness to say “visible” to barely visible stimuli25,28–31). In our staircasing procedure, mask contrast was continuously adjusted based on subjective visibility, such that a more liberal criterion (greater willingness to say “visible”) would result in higher average mask contrast and this could account for lower performance. Patterns in LOC and FFA/PPA were better predicted by perceptual performance than by mask contrast and perceptual performance explained a significant portion of the neural variance taken into account the variance explained by mask strength. However, without a measure of sensitivity and criterion for the visibility task (e.g. by including stimulus-absent trials) our results cannot rule out criterion effects.
Nevertheless, by comparing stimuli of similar stimulus strength reported as “visible” vs. “invisible”, subjective measures have often been adopted to reveal the neural correlates of subjective awareness. Although the contrast of subjectively visible and invisible trials may be confounded with factors unrelated to awareness (e.g. vigilance, attention), many previous fMRI studies adopted this approach, often using binocular rivalry, and typically found stronger visual cortex responses for stimuli reported as visible11. Our finding of better pattern discrimination of subjectively visible stimuli in object- and category-selective visual cortex is consistent with the idea that subjective awareness is related to enhanced activity in the very same brain areas that are specialized for processing those stimuli68,69. Whether activity in this posterior occipito-temporo-parietal “hot zone”70 is sufficient for awareness, or whether awareness requires additional activity in prefrontal cortex7,71, is subject of ongoing debate. Recent studies using so-called “no-report” paradigms indicate that activity in frontal cortex may reflect post-perceptual processes related to reporting awareness rather than to content-specific perceptual awareness per se72. Although establishing absence of awareness on-line during imaging required trial-by-trial visibility and discrimination responses, our approach of correlating activity with “benchmark” patterns from an independent localizer scan where participants had a different task, reduced the influence of post-perceptual effects (also see Supplementary material)49,50. Isolating perceptual representations in this way, there was no evidence for prefrontal cortex involvement (see Figure S3 and Figure S4 and supplementary results for evidence that activity in a frontoparietal network distinguishes visible vs. invisible conditions, irrespective of stimulus category). As such, these findings support first-order theories that place the neural correlates of content-specific subjective awareness in sensory areas6,68–70.
The marked differences in neural representation of subjectively vs. objectively invisible stimuli revealed here imply that models where “unconscious processing” is seen as a unitary mechanism or concept, without further specifying how absence of awareness was established, will not provide a good fit to the full range of empirical data. Rendering stimuli subjectively or objectively invisible results in different estimates of unconscious information processing in the human brain. Only subjectively invisible, but not objectively invisible stimuli were processed along a posterior-anterior gradient, with greater category information in category-selective ventrotemporal cortex than in occipital cortex. At the same time, we provide the first conclusive evidence that even fully indiscriminable, objectively invisible stimuli can be decoded from patterns of fMRI activity in human visual cortex. Previous studies that made such claims32–37 did not convincingly establish absence of awareness during scanning and/or had very low statistical power (small sample sizes), resulting in high probability of false-positives in the fMRI data73 and of false-negatives in the awareness measure74,75. With greater statistical power we also find that subjective awareness involves enhanced representation in visual cortex but not prefrontal cortex, thus supporting first-order theories of consciousness that place the neural correlates of perceptual awareness in the “back” rather than in the “front” of the brain6,68–70.
Methods
Participants
The experiments were approved by the University of Amsterdam ethics committee. Volunteers were recruited from the University of Amsterdam participant pool. Participants were mostly students who received either course credit or a monetary compensation for their participation. All participants reported normal or corrected-to-normal vision, were naïve to the research question, and provided informed consent. In the fMRI experiment, we scanned 54 participants. Eleven participants were excluded from all data analyses: Three were excluded because they did not finish the experiment, two because their anatomical scans were corrupted, and six because of failure to follow the instructions inside the scanner (very few button presses, holding down the buttons continuously, or reporting stimulus visibility in nearly all trials of the condition with lowest categorization performance). The final sample consisted of 43 participants (24 female, mean age 22.7 years, SD 3.9, range 18–37 years). In the behavioral masking-efficiency experiment there were 18 participants, one of which was excluded due to a coding error (unbalanced experimental conditions), resulting in a final sample of 17 participants (12 female, mean age 21.8 years, SD 4.7, range 18–31 years).
Stimuli
In the scanner, stimuli were presented on a 32-inch LCD screen for MRI (1920 × 1080 pixel resolution, 120 Hz refresh rate) seen from a viewing distance of approximately 150 cm through a mirror mounted on the head coil. The experiment was programmed in Matlab using the Psychtoolbox76 functions. Stimulus presentation was synchronized with the 8.3-ms vertical refresh cycle of the screen. Stimuli were ten face photographs of neutral expression from the FACES database77, and ten house photographs (taken from the front) selected from the internet (see Supplementary material). This selection was based on informal pilot testing with a larger set of stimulus exemplars. Pilot results indicated that the selected exemplars were best matched on subjective visibility. Photographs were cropped to an oval of 240 × 336 pixels containing only the inner features of faces and houses, converted to grayscale, and the oval was assigned identical mean luminance and contrast (in RGB values M = 127.5, SD = 17.5). The remainder of the stimulus rectangle as well as the screen background were mid-gray (RGB value 127.5). Masks filled the whole stimulus rectangle and consisted of a randomly generated arrangement of overlapping rectangles and – in a lesser number – circles in various sizes and levels of gray. We created a set of 100 masks, from which one forward mask and one backward mask were randomly selected on every trial. The contrast of these masks differed between visibility conditions and varied between 2% and 100%. Throughout the experiment, all stimuli and masks were presented within a 8-pixel wide light-gray frame (248 × 344 pixels, RGB value 143).
In the objectively visible (obj-vis) condition, mask contrast was set to 2% to achieve clear visibility of the face/house stimuli. In the objectively invisible (obj-inv) condition, mask contrast was set to 100%, such that participants’ ability to discriminate between faces and houses was expected not to differ significantly different from chance (based on the results from the masking-efficiency experiment). In the subjective condition, mask contrast was adjusted through an adaptive 1-up 1-down staircase procedure: On the first trial of each run, mask contrast started at 18%; following a “visible” response, mask contrast on the next trial was increased by 4%; following an “invisible” response, mask contrast was lowered by 4% (minimum contrast 2%, maximum contrast 100%). This adjustment was intended to yield a roughly similar number of subjectively-visible (subj-vis) trials and subjectively-invisible trials (subj-inv).
Procedure
On every trial, a forward- and backward-masked face or house stimulus was presented and participants indicated stimulus category and visibility. Every trial began with 292 ms of fixation-on, followed by 392 ms fixation-off, 92 ms forward mask, 16.7 ms face/house stimulus (two screen refresh cycles), and 192 ms backward mask. Next, a response screen prompted participants to use one of four buttons to indicate stimulus category and visibility, using their left hand for the two left buttons and their right hand for the two right buttons (using a button box inside the scanner and a standard keyboard outside the scanner). The left-most button represented “face, visible”, and the other button on the left represented “face, invisible.” The right-most button represented “house, visible”, and the other button on the right “house, invisible.” Pilot experiments had shown that this compound response and response mapping was intuitive, and all participants received extensive training in using the buttons. There was a response window of 1.8 s in which participants could enter their response. The trial ended with a fixation period of variable (jittered) duration (selected from a uniform distribution between 100 and 900 ms with as many values as trials per fMRI run or as trials per behavioral experiment). The inter-stimulus interval ranged between 2.9 and 3.7 s (while brain activity was measured with a fast EPI sequence with a TR of 1.6 s, see below).
Instructions
Before beginning the experiment, participants received detailed written and verbal instructions. They were informed that they would be presented with pictures of faces and houses, and that masking would be used to degrade visibility of these stimuli, such that some of the stimuli would be visible and some would be invisible. They were instructed to indicate both stimulus category and visibility using the compound response. Participants were informed that when they had absolutely no idea of what category the stimulus represented (i.e. if they did not see anything that indicated that the picture was a face or a house) they should indicate “invisible” and take a guess. If they had some (vague) idea of what the stimulus category could be they should indicate “visible.” Thus, the “invisible” category on this two-point visibility scale corresponded to the lowest visibility level of finer-grained visibility scales, such as the popular four-point Perceptual Awareness Scale (PAS)16. Subjectively invisible trials in the present study corresponded to the lowest PAS level “no experience”, and subjectively visible trials corresponded to all higher PAS levels, including the second-lowest “brief glimpse” level. Participants were informed about the 1.8-s response window but instructions emphasized that (after practice) this would be more than enough time to provide an accurate response, such that there was no speed pressure, and that responses should be as accurate as possible.
Before entering the scanner, participants completed one practice run at a computer outside the scanner. At the beginning of the fMRI session, an anatomical scan was acquired and participants completed another practice run, which was followed by five runs of the main experiment. fMRI runs started and ended with 6.4 sec of fixation. In the following 160 trials of a run, there were 40 trials of the obj-vis condition, 40 trials of the obj-inv condition, and 80 trials of the subjective condition (here, the number of trials in the subj-vis and subj-inv condition depended on the participant’s response). Within each condition, each combination of two stimulus categories (faces, houses) and ten stimulus exemplars occurred equally often. Trial order was randomized. In total, there were 200 trials of the obj-vis condition, 200 trials of the obj-inv condition, and 400 trials of the subjective condition.
Localizer run
At the end of the fMRI session, we acquired a functional localizer scan to localize face-and house-responsive voxels in visual cortex. In the localizer, the same stimulus exemplars as in the main experiment were presented in a standard design for localizing category-selective brain areas: Faces and houses were displayed in separate 16-s blocks, where a series of 16 faces or houses was presented unmasked for 750 ms each, followed by 250 ms fixation. There were 20 face blocks and 20 house blocks in alternating order. After every four blocks there were 16 s of fixation. Within a block, stimulus exemplars were randomly ordered with the constraint that the same exemplar could only be presented two times per block. Participants were instructed to press a button when there was a repetition of an exemplar (we did not record these button presses).
fMRI acquisition
MRI data were collected using a 3 Tesla Philips Achieva MRI scanner with a 32-channel head coil. At the beginning of the fMRI session, an anatomical scan was acquired using a T1-weighted gradient-echo sequence (220 slices, voxel size = 1 mm3). Functional images were acquired using a T2*-weighted multiband EPI sequence (56 slices, flip angle 70°, TR 1600 ms, TE 30 ms, voxel size 2 mm3). During the main experiment, 348 volumes were recorded per run (for a total of 5 runs lasting 9:30 min each), and 248 volumes during the localizer run (lasting 6:30 min).
Behavioral masking-efficiency experiment
The masking-efficiency experiment was conducted to determine the required masking settings to achieve objective invisibility, i.e. a masking setting at which participants’ ability to discriminate between faces and houses was not significantly above chance. Stimuli were presented on a 24-inch LCD screen (1920 × 1080 pixel resolution, 120 Hz refresh rate) seen from a free viewing distance of approximately 80 cm. The masking-efficiency experiment was identical to the fMRI experiment, except that nine fixed masking strengths (log10-scale between 2% and 100% contrast) were used. There were 540 trials, in which each combination of two stimulus categories (faces, houses), ten stimulus exemplars, and nine masking strengths (2, 3.3, 5.3, 8.7, 14.1, 23.1, 37.6, 61.3, 100%) occurred equally often. Trial order was randomized, and there were three obligatory breaks.
Analyses
Behavioral data
Trials with no response (on average less than 3%) were excluded from analyses of behavior and fMRI. We calculated the signal detection theory (SDT) sensitivity index d’ as a measure of objective perceptual discriminability of faces and houses: “Face” responses were coded as hits in face trials and as false alarms in house trials. Hit and false alarm rates of 0 or 1 were converted to 1/(2N) and 1−1/(2N), respectively, with N being the number of trials on which the rates were based26. Finally, the z-transformed false alarm rate was then subtracted from the z-transformed hit rate to yield d’.
fMRI data pre-processing
Neuroimaging data were pre-processed using fMRIPrep 1.3.278. For the structural images, pre-processing steps included: intensity non-uniformity correction, skull stripping, surface reconstruction, spatial normalization to the ICBM 2009c Nonlinear Asymmetrical template version 2009c79 (with nonlinear registration), and brain tissue segmentation. Pre-processing steps for the functional images included: susceptibility distortion correction, co-registration to the structural image, estimation of head-motion parameters, resampling of the BOLD time-series to the template, and high-pass filtering (using a discrete cosine filter with 128 s cut-off).
fMRI data modeling
SPM12 was used to fit a general linear model (GLM) to the data from the localizer run, and to the data for each run of the main experiment. For the localizer, the GLM contained two regressors (faces, houses) and six regressors of no interest (head motion parameters). For each run of the main experiment, the GLM contained eight regressors (four visibility conditions; objectively and subjectively visible and invisible, and two stimulus categories; faces and houses), as well as six motion regressors. Regressors were convolved with a standard hemodynamic response function, as included in SPM12. The resulting beta weights for each voxel were used as the data points for the following analyses.
Multivoxel pattern analyses
Multivoxel pattern analysis (MVPA; using the CoSMoMVPA toolbox80) was used to identify brain regions carrying category-specific information in their activity patterns, i.e. activity that reliably distinguished between face and house stimuli. Activity patterns were correlated between the block-design localizer and the event-related main experiment. For every participant and for each of the four visibility conditions (obj-vis, subj-vis, subj-inv, obj-inv), betas from the localizer for one stimulus category (faces, houses) were correlated with betas from the main experiment for the same stimulus category, yielding within-category correlations, and with betas from the main experiment for the other stimulus category, yielding between-category correlations. Correlations were then Fisher z-transformed, the two within-category correlations were averaged, and the two between-category correlations were averaged. Finally, between-category correlations were subtracted from within-category correlations, yielding a correlation difference Δr for each visibility condition49,50. Positive correlation differences indicate that activity patterns carry information about stimulus category.
Region-of-interest (ROI) analyses
Pattern correlation differences were calculated for four different (bilateral) ROIs. A probabilistic atlas of retinotopic cortex81 was used to define bilateral early visual cortex (V1). A functional group atlas82 was used to define bilateral lateral occipital complex (LOC), posterior category-selective areas (face-selective occipital face area OFA and scene-/house-selective occipital place area OPA [transverse occipital sulcus]), and anterior category-selective areas (face-selective fusiform face area FFA and scene-/house-selective parahippocampal place area PPA). For every participant, the localizer data were used to select the 100 most face-responsive and the 100 most house-responsive voxels within each of these ROIs (based on a t-test comparing face- and house-responses; for similar results with other ROI definitions see Figure S3 and supplementary results).
Searchlight analyses
We additionally ran searchlight analyses52 to identify activity patterns that distinguished between categories across the whole brain. For these searchlights we repeatedly calculated pattern correlation differences in the same way as described for the ROI analyses but for a moving sphere with a radius of five voxels (524 voxels in each sphere) which was centered on every voxel in the functional images of every participant (for searchlight analyses based on the data from the main experiment only, see Supplementary material).
Controlling for mask contrast in the subjective condition
To control for trial-by-trial differences in mask contrast in the subjective condition, another GLM including a parametric regressor for mask contrast and an additional regressor for visibility, separately for faces and houses, was fit to the data from the subjective condition (all other aspects of the GLM were the same as described above). The visibility regressor was orthogonalized with respect to the mask-contrast regressor, such that the visibility regressor was assigned only the variance not explained by the mask-contrast regressor. Both the mask-contrast and the visibility regressor were mean-centered (run-wise, separately for faces and houses).
Statistics
For behavior and ROI data, we report both standard frequentist statistics and Bayes factors (BFs) calculated in JASP83 with default prior scales (Cauchy distribution, scale 0.707). When frequentist statistics indicate a significant effect, the corresponding BF is reported as a quantification of the evidence for the alternative hypothesis (BF10); when the effect is not significant, the reported BF quantifies the evidence for the null hypothesis (BF01). To demonstrate absence of awareness in our objective measure, the directional BF0+ quantifies evidence for null sensitivity compared to the alternative of above-chance performance. For multi-factorial ANOVAs, we report the inclusion BF quantifying the evidence for all models containing a particular effect compared to all models without that effect. To test for category information in ROIs the directional BF+0 quantified evidence for above-chance information compared to the alternative of zero information. For the searchlight group maps results were corrected for multiple comparisons via false discovery rate (FDR) corrections53 (p < .05).
Data availability
The data sets generated during and/or analyzed during the current study are available from the corresponding author on request.
Supplementary Material
Figure S1 shows individual stimulus exemplars used in the study (Figure S1a) and face-house discrimination performance as proportion correct in the obj-inv condition for individual participants (Figure S1b and Figure S1c).
Evidence for objective invisibility
For the interpretation of our fMRI results in our obj-inv condition it is important to test whether faces and houses were truly indiscriminable (objectively invisible). In the main manuscript, we report that discrimination performance (M = 0.02, SD = 0.24) did not differ significantly from chance, with moderate evidence for the null hypothesis of chance-level discrimination (t(42) = 0.45, p = .33 (one-tailed), dz = 0.07, BF0+ = 4.13).
Standard statistical significance testing
Another approach that has been adopted in the literature on unconscious processing is to test every participant’s performance against chance level Figure S1b shows face-house discrimination as proportion correct. One-tailed binomial tests comparing every participant’s accuracy against the chance level of 0.5 revealed (only) one participant whose performance was significantly above the chance level (p = .028). When carrying out 43 tests, some are expected to return a significant result. In fact, the distribution of p-values from the 43 binomial tests (Figure S1c) looks just as one would expect if values were randomly sampled from a binomial distribution with a probability of 0.5 success for each trial. However, as discussed in the main paper, although absence of awareness is commonly demonstrated by showing that performance did not exceed what is expected by chance (p > .05), a non-significant result simply means that the observed effect is not very surprising under the null hypothesis of no effect, but cannot be taken as support for the null hypothesis.
Furthermore, p-values increase with decreasing power and increasing variance in the awareness measure, such that a non-significant result is not particularly surprising if the awareness measure is collected in few trials and small participant samples (i.e. with low power)74,75. To address these issues, we tested a much larger sample than previous fMRI studies, measured awareness in every trial during scanning (average of 196 trials per participant to calculate d’ in the obj-inv condition), and did not exclude any participants based on their awareness data (which can severely inflate estimates of unconscious processing due to regression to the mean60). Although our fMRI study adopted the highest criteria for demonstrating unconscious processing of objectively invisible stimuli to date, approaches other than the standard frequentist significance-testing approach are required to provide evidence for absence of awareness. Two such alternative approaches are Bayesian statistics (as reported in the main paper) and equivalence tests.
Equivalence tests
Equivalence tests based on the two one-sided tests (TOST) procedure can be used to determine if an observed effect is surprisingly small under the assumption that a true effect, at least as large as a specific smallest effect size of interest (SESOI), exists84. In the context of unconscious processing, setting an SESOI represents a non-trivial, subjective call, as it requires determining the smallest d’ score that can still reasonably be considered objectively invisible. Setting the SESOI (arbitrarily) to a d’ score of 0.10 the TOST procedure (using the spreadsheet TOST calculators provided by Lakens and colleagues84) revealed that the observed effect was significant within the equivalent bounds of −0.1 and 0.1 scale points, t(42) = −2.23, p = .016. We note that when setting the SESOI to a d’ score of 0.15, the TOST would return p < .001; when setting the SESOI to a d’ score of 0.05, however, the TOST would return p = .19. These results provide more insight about the uncertainty in our obj-inv data. While we cannot reasonably claim that discrimination performance was surprisingly low if the true effect was d’ = 0.05, our data are unlikely if the true effect was d’ = 0.10 or larger.
Bayesian statistics
The Bayesian statistical framework allows estimating how likely the observed data are under the null hypothesis, compared to the alternative hypothesis, and has thus particular utility in the study of unconscious processes59. While the null hypothesis in this framework is a point estimate (zero), Bayesian t-tests also require a non-trivial, subjective call in specifying the prior distribution for the alternative hypothesis. Estimates of the probability of the null hypothesis relative to the alternative hypothesis (BF0+) depend on this prior. In the main paper, we used the default prior provided in the JASP software package (Cauchy distribution centered on zero and scale 0.707), and this resulted in “moderate” evidence for the null hypothesis (BF0+ = 4.13). A wider prior (assigning more weight to larger effect sizes) would result in higher probability of the null hypothesis (e.g. with a Cauchy distribution with a scale of one, BF0+ = 5.67). Conversely, a narrower prior (assigning more weight to smaller effect sizes) would results in lower probability of the null hypothesis (e.g. with a normal distribution with M = 0 and SD = 0.5, BF0+ = 2.34). Rather than allowing definite conclusions about absence of awareness, our results highlight the statistical uncertainty inherent in demonstrations of unconscious processing, and invite discussions about reasonable prior assumptions and about which (bands of) effect sizes represent conclusive evidence for presence vs. absence of conscious awareness.
Univariate ROI results
We also conducted standard univariate ROI analyses (same ROI definitions as for the MVPA analyses). Here, for every participant and ROI we used the localizer data to determine the 100 most face-responsive voxels and the 100 most house-selective voxels. We then calculated for every visibility condition in the main experiment the mean difference in activity evoked by faces vs. houses in the face-responsive voxels and the mean difference in activity evoked by houses vs. faces in the house-responsive voxels. These differences were then averaged to yield a mean beta difference (Δβ) reflecting how well a region’s activity in the main experiment distinguished its preferred category from the non-preferred category; Δβ was used as a univariate measure of category information. Univariate results were similar to the MVPA results reported in the main paper (compare Figure S2 with Figure 2).
Category information in visual cortex
As for the MVPA analyses, there were marked differences between the regions, with overall greater response differences in LOC than in V1 (F(1, 42) = 156.82, p < .001, ηp2 = .79, BF10 = 1.35×1022), as well as response differences between visibility conditions F(3, 126) = 56.44, p < .001, ηp2 = .57, BF10 = 2.69×1027), and these differences were more pronounced in LOC than in V1 (interaction, F(3, 126) = 66.72, p < .001, ηp2 = .61, BF10 = 2.74×1018). Separate one-tailed t-tests for every region and every visibility condition revealed significant category information in V1 for obj-vis (t(42) = 3.57, p < .001, dz = 0.55, BF+0 = 65.68), subj-vis (t(42) = 3.41, p < .001, dz = 0.52, BF+0 = 43.36), and obj-inv (t(42) = 2.04, p = .024, dz = 0.31, BF+0 = 2.09), but not for subj-inv (t(42) = 0.36, p = .36, dz = 0.06, BF0+ = 4.48). LOC shoed significant category information in all visibility conditions, obj-vis (t(42) = 22.77, p < .001, dz = 3.47, BF+0 = 2.18×1022), subj-vis (t(42) = 12.11, p < .001, dz = 1.85, BF+0 = 4.79×1012), subj-inv (t(42) = 4.86, p < .001, dz = 0.74, BF+0 = 2.43×103), and also in obj-inv (t(42) = 3.17, p = .001, dz = 0.48, BF+0 = 23.74).
Similarly, for category-selective regions, category information was more pronounced in anterior (FFA/PPA) than in posterior (OFA/OPA) areas, (F(1, 42) = 111.32, p < .001, ηp2 = .73, BF10 = 1.23×1018); there were also significant differences between visibility conditions (F(3, 126) = 92.41, p < .001, ηp2 = .69, BF10 = 5.18×1046), and a significant interaction (F(3, 126) = 41.51, p < .001, ηp2 = .50, BF10 = 4.25×109). In OFA/OPA category information was significant in all conditions, obj-vis (t(42) = 11.98, p < .001, dz = 1.82, BF+0 = 3.47×1012), subj-vis (t(42) = 7.47, p < .001, dz = 1.14, BF+0 = 7.61×106), subj-inv (t(42) = 4.30, p < .001, dz = 0.66, BF+0 = 478.01), and obj-inv (t(42) = 2.96, p = .003, dz = 0.45, BF+0 = 14.37). Also in FFA/PPA category information was significant in all visibility conditions, but as for the MVPA results, strong evidence for category information was obtained only for obj-vis (t(42) = 15.92, p < .001, dz = 2.43, BF+0 = 4.56×1016), subj-vis (t(42) = 13.22, p < .001, dz = 2.02, BF+0 = 8.10×1013), and subj-inv (t(42) = 6.04, p < .001, dz = 0.92, BF+0 = 8.95×104), but not for obj-inv (t(42) = 1.76, p = .043, dz = 0.27, BF+0 = 1.29).
Differences between visibility conditions
In both early visual cortex (V1) and object-selective visual cortex (LOC) subjective visibility was associated with greater category information than subjective invisibility (V1: t(42) = 2.46, p = .018, dz = 0.38, but BF10 = 2.39; LOC: t(42) = 8.44, p < .001, dz = 1.29, BF10 = 7.57×107), but this difference was more pronounced in LOC (interaction, F(1, 42) = 34.79, p < .001, ηp2 = .45, BF10 = 1.22×103). Also the way of establishing invisibility influenced the two areas differently (interaction, F(1, 42) = 9.83, p = .003, ηp2 = .19, BF10 = 5.59). While in V1 there was no significant difference in category information for subjectively and objectively invisible stimuli (t(42) = −1.26, p = .21, dz = 0.19, BF01 = 2.89), in LOC category information was significantly greater for subjectively than for objectively invisible stimuli (t(42) = 2.27, p = .029, dz = 0.35, but BF10 = 1.64).
In both posterior and anterior category-selective regions subjective visibility was associated with greater category information than subjective invisibility (OFA/OPA: t(42) = 4.24, p < .001, dz = 0.64, BF10 = 1.93×103; FFA/PPA: t(42) = 6.42, p < .001, dz = 0.98, BF10 = 2.73×107), this difference was more pronounced in FFA/PPA (interaction, F(1, 42) = 21.05, p < .001, ηp2 = .33, BF10 = 14.37). Also the method of establishing invisibility had a different effect on the two ROIs (interaction, F(1, 42) = 23.01, p < .001, ηp2 = .35, BF10 = 62.34). While there was no significant difference between subjectively and objectively invisible stimuli in OFA/OPA (t(42) = 0.79, p = .437, dz = 0.12, BF01 = 2.31), FFA/PPA showed greater category information for subjectively than for objectively invisible stimuli (t(42) = 4.23, p < .001, dz = 0.64, BF10 = 597.15).
Posterior-anterior category preference gradient
The univariate ROI data also showed a posterior-anterior gradient (line plots in Figure S2). For obj-vis, subj-vis, and subj-inv category information increased from V1 to LOC (all t(42) > 4.50, p < .001, dz > 0.59, BF10 > 75.25, Figure S2a), and from OFA/OPA to FFA/PPA (all t(42) > 5.11, p < .001, dz > 0.77, BF10 = 2.62×103, Figure S2b). For obj-inv, this gradient was virtually absent; category information did not differ significantly between V1 and LOC (t(42) = 0.06, p = .95, dz < 0.01, BF01 = 6.05) or between OFA/OPA and FFA/PPA (t(42) = 0.07, p = .95, dz = 0.01, BF01 = 6.04).
Different ROI definitions
For the univariate analyses and for the ROI analyses reported in the main paper we selected the 100 most face-responsive and the 100 most house-responsive voxels for each ROI, as determined by the localizer scan. To ensure that our results did not depend on this particular ROI size of 200 voxels, we repeated the MVPA correlation analyses for other ROI definitions (including between 10 [i.e. the 5 most face-responsive, and the 5 most house-responsive voxels in the localizer] and 500 voxels, in steps of 10). As can be seen in Figure S3, in all ROIs except for V1, overall pattern discrimination (category information Δr) dramatically increased with smaller ROI size. This was especially the case for the obj-vis and subj-vis conditions, and in LOC and in FFA/PPA also for subj-inv. ROI size had little influence on category information for the obj-inv condition. Most importantly, the patterns of differences between visibility conditions as reported in the main paper were robust across ROI definitions.
Main-experiment searchlights
All analyses reported in the main paper relate brain activity in the main experiment to the independent localizer scan, where participants did a simple one-back task on unmasked faces and houses presented for 750 ms each in a block design. We designed the study for this particular analysis strategy, because it is statistically powerful and isolates perceptual representations from later cognitive processes related to the task, decision, and motor processes49,50. Analyses based on the data from the main experiment alone cannot exclude such influences, for example, in these analyses button presses are confounded with the different stimulus and visibility conditions.
Category information
Accordingly, additional searchlight analyses based on the main-experiment data alone revealed category information in motor cortex, driven by the responses given to the presented categories (Figure S3). For these searchlights, the main experiment data were split into two sets (all possible splits of two and three runs). For every visibility conditions we repeatedly calculated within- and between-category pattern correlation differences for a moving sphere with a radius of five voxels (524 voxels in the sphere) which was centered on every voxel in the functional images of every participant. Correlations from all possible splits were then averaged to yield the searchlight maps.
These analyses revealed areas carrying significant category information across the whole brain (corrected for multiple comparisons via false discovery estimation, p < .05) in the obj-vis, subj-vis, and subj-inv condition, but not in the obj-inv condition (Figure S3a). In all three visibility conditions clusters with significant category information were located in bilateral fusiform gyrus, but most prominently in bilateral motor cortex (highlighted in the slices shown in Figure S3a). Such motor cortex pattern discrimination reflects the correlation between stimulus category and participant’s button presses for those conditions in which button presses (perceptual discrimination) carried information about stimulus category (when discrimination performance is above chance). Accordingly, additional frontal cortex clusters (in right inferior frontal gyrus, orbital parts of right superior and left middle frontal gyrus) revealed in the contrast between subj-inv and subj-vis (Figure S3b) are difficult to interpret. Remarkably, there was no evidence for category information in motor cortex in the obj-inv condition, indicating that in this condition (neural representations of) button presses were indeed unrelated to stimulus category, thus further supporting the notion that stimuli were fully indiscriminable.
Visibility irrespective of category
In addition to occipitotemporal areas, several previous fMRI studies reported correlates of conscious visual perception in a frontoparietal network5,11.These studies compared brain activity evoked by visible and invisible conditions, independent of stimulus category or content, assessing visibility either with objective measures (e.g. by comparing weakly vs. strongly masked stimuli40) or with subjective measures of visibility (e.g. by comparing similar or identical stimuli judged as “visible” vs. “invisible”11). To test for such frontoparietal involvement in content-independent conscious perception (“conscious access” or “report”), we conducted searchlight analyses that revealed brain areas where activity distinguished between visibility conditions, independent of object category. For these searchlights, we again split the main experiment data into two sets (all possible splits of two and three runs). Separately for the objective and for the subjective condition we repeatedly calculated within-visibility (visible/visible and invisible/invisible) and between-visibility (visible/invisible) pattern correlations for a moving sphere with a radius of five voxels (524 voxels in the sphere) which was centered on every voxel in the functional images of every participant. Correlations from all possible splits were then averaged to yield the searchlight maps.
In both the objective and subjective condition activity in many brain areas distinguished between visible and invisible trials (corrected for multiple comparisons via false discovery estimation, p < .05), including early visual cortex, fusiform and inferior temporal gyrus, parietal cortex (e.g. inferior parietal lobule, precuneus, angular gyrus, postcentral gyrus), frontal cortex (e.g. motor cortex, inferior frontal gyrus, superior frontal gyrus, middle frontal gyrus, anterior cingulate), and anterior cingulate cortex. The sagittal views of the right hemisphere in Figure S4 highlight significant pattern correlations in ventrotemporal areas, parietal cortex, and inferior frontal gyrus in both objective and subjective conditions. These results could be considered evidence for frontoparietal involvement in (content-independent) conscious perception. Note, however, that in these analyses based on the main-experiment data, post-perceptual processes (related to the task, decision, and motor processes) and stimulus strength (mask contrast) are confounded with differences in visibility.
Acknowledgments
We thank T. Beemsterboer, T. Derckx, and S. Rasche for help with data collection, and L. Snoek for implementing the standardized “fMRIPrep” pipeline. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (ERC-2016-STG_715605 to SvG), and from the German Research Foundation (DFG, KA4683/2-1 to DK).
Footnotes
References
- 1.↵
- 2.
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵