Increased region of surround stimulation enhances contextual feedback and feedforward processing in human V1

The majority of synaptic inputs to the primary visual cortex (V1) are non-feedforward, instead originating from local and anatomical feedback connections. Animal electrophysiology experiments show that feedback signals originating from higher visual areas with larger receptive fields modulate the surround receptive fields of V1 neurons. Theories of cortical processing propose various roles for feedback and feedforward processing, but systematically investigating their independent contributions to cortical processing is challenging because feedback and feedforward processes coexist even in single neurons. Capitalising on the larger receptive fields of higher visual areas compared to primary visual cortex (V1), we used an occlusion paradigm that isolates top-down influences from feedforward processing. We utilised functional magnetic resonance imaging (fMRI) and multi-voxel pattern analysis methods in humans viewing natural scene images. We parametrically measured how the availability of contextual information determines the presence of detectable feedback information in non-stimulated V1, and how feedback information interacts with feedforward processing. We show that increasing the visibility of the contextual surround increases scene-specific feedback information, and that this contextual feedback enhances feedforward information. Our findings are in line with theories that cortical feedback signals transmit internal models of predicted inputs. Significance Statement The visual system has circuit mechanisms for processing scene context. These circuits involve lateral and feedback inputs to neurons. These inputs interact with feedforward inputs and modulate neuronal responses to visual stimuli presented outside their receptive fields. Systematically investigating independent contributions of feedback and feedforward processes is challenging because they coexist even in single neurons. Here we use an occlusion paradigm to isolate feedback and lateral signals in human participants viewing natural scene images in fMRI. We show that increasing the visibility of the contextual surround increases scene-specific feedback information, which also enhances feedforward signals. Our findings are in line with theories that cortical feedback signals carry abstract internal models that combine with more detailed representations in primary visual cortex.


68
Sensory stimulation triggers a cascade of processing in a hierarchy of visual areas. This 69 feedforward processing meets recurrent activity from the previous sensory input and triggers 70 recurrent activity that will meet the next expected visual input. Recurrent processing 71 contextualises and predicts the incoming signal and updates internal models and future recurrent 72 streams. The contextualisation of feedforward information by feedback signals is essential for our 73 understanding of cortical processing (Gilbert and Li, 2011). We know from animal recordings that 74 cortical neurons are contextually modulated when their response to a feedforward stimulus 75 feature is modified by the presence of surrounding features (Sugita, 1999;Shushruth, 2011). In 76 visual cortex, this contextual information can be located far in the surround of a neuron's 77 receptive field. Consequently, contextual modulation of neurons is exerted by cortical feedback 78 and lateral inputs (Angelucci, 2002). Cortical feedback inputs, at least in non-human primate 79 cortex, arrive to discrete portions of cortical pyramidal neurons; mainly to the apical dendrites 80 that branch up to layer 1 (Douglas and Martin, 2007). Feedback inputs are therefore  (Larkum, 2013). This perspective requires techniques to probe brain processing that detect 85 neuronal inputs, advancing previous studies that mainly measure neuronal outputs (i.e. spiking 86 activity Larkum et al., 2018;Muckli et al., 2015). Functional magnetic resonance imaging (fMRI) 87 is one such technique that detects pre-and postsynaptic inputs, offering a means to measure 88 contextual feedback information to a region of cortex. 89 Understanding the nature of contextual modulation transmitted by cortical feedback and 90 lateral interaction is vital for understanding the brain in behavioural and cognitive contexts 91 (Gilbert and Sigman, 2007). This importance of cortical feedback and lateral interaction arises 92 because contextual modulations on a neuron include influences from higher-level top-down 93 processes including expectation, prior experience and goal-directed behaviour, which originate in 94 higher cortical areas (Muckli and Petro, 2013). Therefore, describing neuronal substrates of 95 cognition in brain networks including sensory areas requires us to measure not only stimulus- 2016), therefore functionally determining the brain's response to its environment (Friston, 2010; 103 Clark, 2015). 104 We used fMRI, a brain imaging measure of energy consumption, and multivoxel pattern 105 analysis (MVPA) to investigate how global natural scene features contextually modulate human 106 V1. Our approach complements non-classical receptive field studies in rodent and monkey cortex, 107 that measure spikes in response to a feedforward stimulus relative to contextual surround 108 Friday, 26 February 2021 5 stimulation. However, the proposed tuning to pre-and post-synaptic activity in apical dendrites 109 that might be detectable by fMRI allows us to capitalise on a signal that might not always be 110 available in sharp electrode electrophysiology, where the input at the apical dendrites might not 111 lead to a change in spiking output. Using partially occluded images, we parametrically vary the 112 amount of global contextual information that we provide and measure the resulting contextual 113 feedback (and lateral interaction) information to V1 both in the absence of feedforward 114 information, and when feedback is integrated with feedforward information. If global features in 115 the surround contextually modulate human V1, we hypothesized that scene information in non-  We compensated twenty-nine subjects from the University of Glasgow to participate in the 123 experiment (n = 13 males; mean age: 24.28 years, range: 19-41 years). Subjects provided informed 124 written consent and the experiment was approved by the local ethics committee at the University 125 of Glasgow (CSE01063). We excluded subjects if their data was at chance level classification 126 performance in at least one feedforward control condition (n = 5) or poorly aligned (anatomically) 127 between functional runs (n = 3, see Voxel Selection and Analysis, indicating substantial body 128 movement between scans). Below we report results from 21 subjects with stable classification in 129 feedforward control conditions (n = 10 males; mean age: 25.29 years, range 19-41 years). occluded by a white rectangle. Here we expect that the retinotopic region of V1 responding to the 136 white portion of the image receives no meaningful feedforward input and only cortical feedback 137 signals (and lateral inputs). The white rectangle was placed 0.5° of visual angle diagonally from 138 the centre of the image and spanned 11.6° × 9.2°. In the so-called 'feedforward' conditions, the 139 corresponding quadrant of the scene was shown; V1 voxels responding to the lower image 140 quadrant in this condition contain a mixture of feedforward, lateral and feedback inputs.

142
We used two natural scene images for each participant, as natural scenes induce a lot of 143 contextual associations (Bar 2004). Each scene was 600 x 480 pixels and spanned 24° × 19.2° of 144 visual angle. We did not normalize the images in terms of low-level visual features, such as 145 luminance, contrast or energy at each spatial frequency because we wanted the scenes to look as 146 natural as possible. Smith and Muckli (2010) previously showed that contextual feedback signals 147 in V1 cannot be solely attributed to these low-level visual features.

148
To investigate the contribution of surrounding contextual information on the brain activity

171
Near Surround, Inner Border. C) The activation for the contrast of (Target -Near Surround) used to map non-172 stimulated V1 is shown on the occipital cortex on one subject, with V1 in green on the inflated visualization.

173
Occluded region mapping 174 We presented subjects with three contrast-reversing checkerboards (5 Hz) twice per run.

175
The checkerboards either covered an inner rectangular part of the occluded region (Target -2.5° 176 diagonally from centre, 10.2° × 7.8° visual angle) or the border between the lower right quadrant 177 and the rest of the stimulus (Surround). There were two types of surround checkerboard stimuli 178 ( Figure 1B) -Near Surround (0.5° diagonally from fixation, 11.6° × 9.2° visual angle) and Inside Border (1.5° diagonally from fixation). The activation in the early visual areas for the (Target -180 Near Surround) contrast is shown in Figure 1C. 181

182
Task and procedure 183 We presented scenes on a uniform grey background using MRI compatible goggles 184 (NordicNeuroLab) with 800 × 600 pixel screen resolution, which corresponded to 32° × 24°

210
MRI acquisition 211 We collected MRI data using a 3T Siemens Tim Trio System with a 12-channel head coil. 212 We measured blood oxygen level dependent (BOLD) signals with an echo-planar imaging Excessive subject movement between runs is likely to affect correspondence between 228 voxels from one run to another. This could introduce noise into our analysis as we selected our would suggest a close anatomical alignment between the 4 runs. The median alignment value 235 across subjects was 98.08% and single subject values ranged from 77.85% to 99.31%. We excluded 236 data from further analysis if the alignment value was below 90%, which applied to three subjects.

237
Furthermore, we excluded any subject with chance level performance in any feedforward 238 condition in single trial analysis (significance above chance was measured using permutation not be meaningful to assess feedback classifier performance (or lack of) in such cases. This 244 excluded a further five subjects. Thus, the following analyses were performed on 21 subjects. 245 We identified the cortical representation of the occluded region using a general linear due to the one-run-out method on the four runs), to estimate the single subject mean. We then 284 bootstrapped these mean values from individual subjects 10000 times to estimate 95% confidence intervals (CIs) on the group mean. We counted classifier performances as significantly above 286 chance (50%) if the 95% CIs did not contain chance-level performance. We used a permutation 287 test (1000 samples) to compute differences between mean group classifier performances 288 (reported p values not corrected for multiple comparisons), by shuffling the observed values 289 across the conditions, and calculating the absolute differences between the conditions. If the 290 observed difference was in the top 5% of the differences distribution, we considered our 291 conditions to be significantly different from each other.

293
Our hypothesis is that the surround stimulation drives higher visual areas with larger 294 receptive fields to send a contextual feedback signal to voxels in V1 responding to the occluded 295 quadrant. We can therefore modify the surround stimulation to learn more about the nature of 296 contextual feedback.

297
Increased stimulation of the surround receptive field enhances contextual feedback 298 We have shown previously that scene features eliciting contextual feedback to non-  Expanding on these findings, we assessed the amount of surrounding scene information required 304 to elicit scene-relevant information in non-stimulated V1. We parametrically modulated the 305 availability of surround information and trained the SVM classifier to decode between the two 306 scenes using voxel patterns responding to the lower right quadrant when it was either occluded 307 (feedback and lateral, but no feedforward information) or stimulated (feedforward, feedback and 308 lateral information). SVM classification performance was used as an estimate of the amount of 309 available information in the activation pattern.
When the image was occluded, scene classification in non-stimulated voxels improved with 311 increasing availability of surrounding scene information (Figure 2A, left). Averaging across 312 experiments, classification was significantly above chance once the bubbles exceeded the smallest 313 size, except for Large Bubbles Single Trial analysis (Table 1)

384
To further test how much surround information contributes to visual processing, we 385 compared the Fully Visible scene with other feedforward conditions with a reduced scene 386 surround, as well as the feedback conditions (Figure 4). We trained the classifier on the Fully Visible scene and tested on the other conditions. In a fully visible scene both parts of the 388 information are available simultaneously and the classifier might rely more on the rich, fine-389 grained feedforward information. However, we found that Fully Visible feedforward to feedback 390 cross-classification was only possible with large amounts of scene information surrounding the 391 occluded region (Table 3). Fully Visible to Full Feedback cross-classification was above chance, 392 while Large, Medium and Small Bubbles did not reach significance in the feedback conditions. In     Full Feedback (AB analysis, Experiment 1 only).

472
After restricting voxels to the occluded region using pRF mapping, we saw that classifier 473 performance decreased in some conditions, but the pattern of the results remained the same 474 (Figure 6). Due to the low numbers of subjects in each experiment for whom we were able to 475 perform pRF mapping, we did not calculate confidence intervals for some of the mean values. accounts for the full extent of the surround modulation effects (Angelucci and Bressloff, 2006).

499
There was no meaningful feedforward stimulation in our occluded region of V1, and yet we could 500 decode two scenes using information patterns corresponding to this non-stimulated region. This 501 differential information must originate from contextual information in the scene surround.

502
Classical receptive fields are smaller than the surround, hence neurons in the occluded area in V1 503 most likely receive information about the rest of the scene via cortical feedback from higher areas.

504
Since we are measuring a population of neurons using fMRI, as opposed to single cells, it is hard 505 to estimate how widespread the effect of the surround receptive field is. V1 receives feedback from 506 many cortical areas, which have increasing receptive field sizes moving to higher and more 507 abstract processing areas (Dumoulin and Wandell, 2008). Therefore, we expect that influence from the surround might be restricted to regions close to the occluded region for feedback coming 509 from V2, for example, but transmit information from a larger area of the surrounding scene for 510 feedback originating from higher visual areas. 511 We found that larger bubbles in the surround lead to more informative feedback in the 512 occluded region. This may be because we are revealing more of the overall scene structure as we 513 increase the bubble size. Tang   in the occluded region is present even if participants never see the fully visible scene and were not 582 familiarised with it. We also found that increased exposure to the full scene did not improve 583 feedback in the conditions with reduced surround. Therefore, it appears that the contextual 584 feedback we observed arises from the scene structure available in each trial, or knowledge of 585 natural scene properties in general, but familiarity with the specific scene is not required for 586 informative feedback signals. This could be because natural scenes have predictable scene 587 statistics and much of the information they contain is redundant (e.g. Attneave, 1954;Barlow, 588 1961; Torralba and Oliva, 2003).

590
We demonstrated that cortical feedback information forms a part of early visual cortex 591 activity during visual stimulation. Using a brain imaging technique we have corroborated