Large-scale color biases in the 1 retinotopic functional architecture 2 are shared across human brains 3

Although the neural processing of chromatic and spatial features is intertwined, it is 12 unknown how consistent this spatio-chromatic coding is across diﬀerent brains. In this fMRI 13 study we predicted the color a person was seeing using a classiﬁer that has never been trained 14 on chromatic responses from that same brain, solely by taking into account: (1) chromatic 15 responses in other individuals’ brains and (2) commonalities between spatial coding in brains 16 used for training and the test brain. Fitting shared response models to fMRI responses elicited by 17 spatially deﬁned and achromatic retinotopic mapping stimuli, we transformed subject-speciﬁc 18 color responses to a common functional space. In this space we successfully decoded color 19 across observers based on activity patterns in V1-V3, hV4 and LO1. Examining classiﬁcation 20 weights, we found that systematic large-scale retinotopic biases for the diﬀerent colors may 21 explain at least partially the observed agreement of neural color coding between brains.


Introduction 24
It is an age-old question whether the subjective experience one person has of a given color matches 25 that of another person. While it is difficult to answer this directly, it is possible to answer a related 26 question: are there similarities in neural representations of colors that are shared across brains? In 27 the present study we used a method called shared response modeling to address this question. By 28 projecting each participant's neural representations into a shared neural space that was indepen-29 dent of color, and defined purely by achromatic spatial information, we implicitly also addressed 30 the question to which extent color and spatial information are coupled across brains. 31 In humans, color vision has traditionally been thought to be mediated by functionally segre- for every participant mapping voxel response spaces from individual ROIs into a 50-dimensional common space (see Materials and Methods). Note that only retinotopic mapping data were used to estimate transformation matrices. Color responses (bottom) were measured with fMRI from participants performing a luminance change detection task on red, green, and yellow ring stimuli presented at two luminance levels. Stimuli were shown for 8.5 s with ITI = 1.5 s (see Materials and Methods). Using the matrices, these fMRI color responses in every ROI and participant, which were recorded independently from the retinotopic mapping data, were mapped from individual response spaces into the common space.

90
The goal of the experiment was to examine whether representations of chromatic signals were 91 shared across human brains and whether this occurs in a spatial neural code. Specifically, our 92 hypothesis was that chromatic signals may share aspects of spatial, achromatic representations 93 that are conserved across different individuals. For this reason, we used brain responses to achro-94 matic, spatial stimuli to construct a neural space that was shared between observers  Within-subject classification of color and luminance (WSC) 99 Decoding color across different brains requires that color can be decoded within one and the same 100 individual brain. For this reason we first performed pattern classification analyses on fMRI activity Individual responses were transformed to the common space. Transformation matrices were estimated using the shared response model fit to an independent sets of retinotopic mapping data. Classification accuracies are cross-validated by leaving out the participant during classifier training whose data were used for testing (leave-one-subject-out). The normal approximation to the binomial distribution was used to convert accuracies to z-values (n = 3240). Note that prediction accuracies are represented as bars rather than individual dots (see Figure 2) because the prediction accuracies for individual participants are no longer independent. Error bars denote the 2.5th and the 97.5th percentile of the permuted null distributions. Asterisks denote accuracies significantly exceeding chance at p < .01 (permutation tests, 2000 iterations, FWE corrected across 7 ROIs). Both color and luminance could be predicted across subjects from areas V1-V3. Additionally, color could be predicted from hV4 and LO1.  This suggests that measurements of how purely spatially defined stimulation activates the brains 131 of different individuals is sufficient to predict colors and luminance from brain activity in any one 132 of them using a classifier trained on data from the remaining participants. 134 Since we observed significant BSC across brains after estimating shared responses from the retino- 135 topic mapping experiments, we were interested in a closer examination of any large-scale retino-136 topic biases mediating these effects. In short, we wanted to learn how the classification rules (i.e., 137 weights), which were learned by LDA during BSC classification and which would allow significant 138 decoding about the color that was presented, were related to retinotopic locations within a given 139 ROI. To that end, a linear classifier was fit to all participants' data after projection into the shared 140 common space. We mapped back classification weights into individual participants' voxel spaces 141 and combined them with retinotopic coordinates. We then used nearest neighbor classifiers to 142 predict from retinotopic coordinates in each voxel which class it preferred while cross-validating 143 across participants. We thus obtained a visual field map of class preference for every ROI (see   . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

Retinotopic analysis of classification weights
The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint For every ROI, an LDA classifier was trained on the common space responses from all participants. The weights were transformed back into individual response spaces such that every voxel was assigned with one weight per class. A nearest neighbor classifier was trained for every voxel to predict based on its Cartesian retinotopic coordinates which class had the highest weight. Every voxel in every ROI and participant thus obtained a predicted class label that was cross-validated across participants. These class predictions were used to quantify the relative preference for a given class as a function of retinotopic location. For each class separately, the retinotopic coordinates of all voxels preferring that class were entered into kernel density estimation to approximate this function. A second function was approximated using all voxels combined, which was subtracted from each of the class-specific functions to normalize them. (B) Results for green, red, and yellow classes (from left to right) in area V1. Colored areas represent visual field locations with a relative overrepresentation of voxels preferring that class while gray areas denote regions with a relative underrepresentation. 0°marks the right horizontal visual meridian.

of 24
. CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint  Figure 4(B). On the left, p values indicate that nearest neighbor classification of class preference was significantly above chance in all regions tested (Holm-Šidák corrected for seven ROIs). Classes showed distinctive topographies that differed particularly along visual meridians.

of 24
. CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint Interestingly, the maps show some variability across ROIs. For example, whereas in V3 the 156 yellow stimulus was preferred at parafoveal locations, this was not the case in V1, where preference 157 for yellow was more pronounced at more peripheral locations.

158
In the case of luminance classification, predicting stimulus preference from retinotopic coordi-159 nates was also significantly larger than chance (50 %) in many ROIs, with the following accuracies.   In sum, these results indicate a systematic relationship between a voxel's retinotopic spatial 170 preference and its response pattern to color and luminance that could at least partially mediate 171 between-subject classification.

172
Whole-brain searchlight analysis of shared responses 173 The ROI results showed that color could be classified significantly better than chance across differ-174 ent brains if classification was performed in a shared response space estimated from responses 175 to only achromatic and spatial stimulation. Yet although the data for these BSC analyses came 176 from different brains, they were recorded in ROIs that corresponded to the same retinotopic field 177 maps. Despite the interindividual variability in the locations of visual areas, they tend to overlap 178 anatomically in standard MNI space. 179 We therefore tested how well common shared responses could already be estimated from 180 only anatomically aligned activity patterns for subsequent BSC. Individual datasets were warped 181 to MNI space separately and SRM was fit to only local patterns of retinotopic mapping responses 182 in a searchlight analysis to estimate common space representations for every brain location. After 183 transforming individual color responses to this common space, BSC could be carried out on those 184 local patterns. This approach had the additional advantage that BSC could be applied to the whole 185 brain.

186
Applying a false discovery rate of q < .05, we found that classification accuracies in a region 187 within the early visual cortex was significantly larger than chance (one-tailed binomial test based 188 on 3240 Bernoulli trials). Figure 7 shows the location of this region relative to the hV4 group ROI.

189
This ROI comprised all the voxels that were identified as being part of individual hV4 ROIs in at 190 least 25 % of the participants. As can be seen, the voxels where classification was significantly 191 better than chance were located near the calcarine sulcus but did not overlap with the hV4 group 192 ROI.

193
In sum, the searchlight analysis suggests that while anatomical alignment of voxels was suffi-194 ciently high for BSC of color in early visual cortex this alignment was too coarse in more anterior 195 regions. 196

197
Here we tested whether chromatic processing is sufficiently similar between human brains to allow 198 decoding of color across different observers. In particular, we examined this question in a neural 199 space based on purely spatial and achromatic stimulation. We hence exploited dependencies be- . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint Figure 6. Relative class preferences for luminance discrimination. Same conventions as in Figure 4(B). except that since classification was binary, hot areas mark regions in the visual field where high luminance was preferred and gray areas mark regions where low luminance was preferred. On the left, p values indicate that nearest neighbor classification of class preference was significantly above chance in all regions tested except VO1 and LO2 (Holm-Šidák corrected for seven ROIs). In early areas V1-V3 for instance, luminance preference changed as a function of eccentricity. Note that although VO1 showed a difference between left and right visual field descriptively, preference classification in this area (and in LO2 likewise) was not significant.

of 24
. CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint Figure 7. SRM searchlight map. SRMs were fit to patterns within local spheres (3 voxel radius) of brain responses to the retinotopic mapping stimulus. After transforming every participant's dataset into the common space, classifiers were used to predict the color of the viewed image using only the information in the local activity pattern (BSC). Cross-validation was performed by leaving out data from a different participant for testing on every iteration. Classification accuracies are shown in warm colors on a standard MNI template for brain regions that survived the whole-brain significance threshold of a binomial test (q < .05, FDR corrected). Green marks all voxels falling within area hV4 in at least 25 % of the participants. The regions where BSC of color yielded accuracies significantly above chance, namely near the calcarine sulcus, were distinctly different from the location of hV4. This indicates that anatomical alignment was not sufficient beyond earliest visual areas for BSC (e.g. in hV4).
we followed up on this research by testing if integrated spatiochromatic processing is sufficiently 203 similar between human brains to allow decoding of color across different observers based on their 204 shared responses to only spatial and achromatic stimulation. 205 We found that, after taking into account how different brains respond to the same, purely spa-206 tially defined retinotopic mapping stimulus, there was a strong agreement across participants be-207 tween the fMRI activity patterns elicited by stimuli of different luminance and color. Otherwise it 208 would not be possible to linearly classify what color a person was seeing based on their brain activ-209 ity using only the shared responses to the retinotopic mapping stimulus, i.e. without training the 210 classifier on actual color responses from that particular brain. This is consistent with previous re-211 ports of integrated spatiochromatic processing. Yet crucially, it follows from this observation that 212 the relationship between the specific way that the brain represents spatial and chromatic visual 213 features is to some degree preserved across individuals.

214
Upon further exploration of the nature of this relationship we discovered systematic retino-215 topic response biases to color as well as luminance. Specifically, we observed that visual field 216 coordinates in a voxel predicted for which color or luminance level that voxel was most predictive.

217
Mapping voxel preferences across the visual field revealed response biases that correlated with 218 eccentricity in a continuous (e.g. V1 in Figure 5) or inverted-U shaped manner (e.g. V3 in Figure 6). 219 Some retinotopic biases aligned with the vertical (e.g. V2 in Figure 5) or horizontal meridian (e.g. 220 VO1 in Figure 5). 221 Response biases had been reported initially for stimulus orientation (  225 In mice, differences in the behavioral significance of stimuli in the upper visual field (above the 226 horizon) versus the lower visual field (below the horizon) may similarly explain differential opsin 227 10 of 24 . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint distributions (Baden et al., 2013; Rhim et al., 2017), which in turn have been related to differences 228 in chromatic discriminatory ability between upper and lower visual fields (Denman et al., 2018). 229 Focusing, in contrast, on the human color vision, the following paragraphs will discuss studies 230 reporting retinal, behavioral, and cortical chromatic biases in relation to the present findings.

231
Retinal spatio-chromatic biases 232 With respect to color, the retinotopic response biases may be related to the distribution of cones 233 in the retina. The fact that cone density decreases with eccentricity (Curcio et al., 1990), that S 234 cones are not found in the fovea (Roorda and Williams, 1999), or that a fourth (melanopsin) pho-235 topigment may contribute to peripheral color vision (Horiguchi et al., 2012) could thus be related 236 to differences between preferences at high versus low eccentricities, as we have found for yellow 237 stimuli in V1 for instance. However, the distributions of L and M cones (in contrast to S cones) show 238 large variability across individuals (Brainard, 2015) so that commonalities at the receptor level likely 239 cannot fully explain the observed effects.

258
Behavioral spatio-chromatic biases 259 Behaviorally, the contrast sensitivities for the chromatic red-green cone mechanism peaks at the 260 fovea and is stronger than for achromatic and blue-yellow mechanisms, yet it also shows the 261 strongest decrease as a function of eccentricity such that it falls below the blue-yellow and achro-  273 Simulation has shown, however, that cone density differences as a function of polar angle are too 274 11 of 24 . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint is concerned and that postreceptoral, possibly cortical, contributions are therefore likely (Kupers  et al., 2019). To what extent these findings generalize to differences in cone ratios and chromatic 277 processing remains an open question.

278
Regarding cortical effects, human fMRI was able to link the advantage of the lower visual field in 279 an orientation discrimination task to stronger and more extensive activation at the corresponding 280 retinal locations in V1/V2 (Liu et al., 2006). Population receptive field (pRF) model fits between 281 visual field maps representing the upper versus lower hemifield exhibited significant differences 282 in size and orientation parameters (Silson et al., 2018) as well as a systematic dependence of pRF 283 size and cortical magnification upon polar angle (Silva et al., 2018). Finally, at the anatomical level, 284 it has been shown that there is a cortical overrepresentation of the horizontal meridian in area V1 285 in humans (Benson et al., 2012). 286 Flexibility in spatio-chromatic biases 287 Recently, it has been demonstrated that the coarse-scale response biases found for orientation are 288 not fixed but can be changed by multiplying a grating stimulus with a modulator whose intensity 289 changes cyclically as a function of either polar angle or eccentricity (Roth et al., 2018). These mod-290 ulators determined whether the orientation bias was radial or tangential -a phenomenon termed  (Amano et al., 2016). 301 On the other hand there is evidence that some neural color representations are preserved 302 across tasks. Accordingly, patterns of brain activity do show some agreement between viewing

310
Participants 311 We analyzed fMRI data from N = 15 (2 male) participants aged between 22 and 35 years (mean: 25.5) 312 who took part in a previously published fMRI study about color vision (Bannert and Bartels, 2018). 313 We only selected participants from that study for which the cortical retinotopic representations of 314 the visual field were measured along both the polar and the eccentricity axis of the visual field (see 315 below). All participants had normal or corrected-to-normal visual acuity and were tested for normal  The observers had to foveate the fixation dot in the middle of the screen while paying attention 353 to the expanding color rings. Each stimulus was presented for 8.5 s at an inter-stimulus-interval 354 (ITI) of 1.5 s. There were 36 trials per imaging run for a total of 216 trials. The trial sequence was 355 pseudo-randomized across runs to ensure that each of the 6 conditions was preceded equally 356 often by every condition (Brooks, 2012). The last trial in every run was repeated at the beginning 357 of the subsequent run to ensure that the trial sequence remained counterbalanced across all runs.

358
The first trial of each run was excluded from the analysis.

359
The task was to indicate by button press the occurrence of a brief (0.3 s) luminance increment 360 or decrement of the color stimulus. Specifically, when presenting a high luminance color stimulus,  . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint half. In the two eccentricity runs the checkerboard stimulus was viewed through an annulus that   were used to unwarp the image sequence in order to take into account magnetic field distortions. 403 We corrected for differences in slice acquisition times by shifting the phase of each frequency in the were also applied to normalize functional images to MNI space.

408
The data from the retinotopic mapping experiment were motion-corrected, co-registered, and 409 slice-time-corrected with SPM8 in the same way as the data from the main experiment. We used  was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint Manuscript submitted to eLife map indicate the boundaries between visual areas. In this way, we delineated boundaries for areas 419 V1, V2, V3, hV4, VO1, LO1, and LO2. (effectively high-pass filtering the data), followed by scaling to zero mean and unit variance 432 Pattern classification 433 Pattern classification was performed in Python 3 using scikit-learn 0.19.1 (Pedregosa et al., 2012). 434 We used Linear Discriminant Analysis (LDA) to train classifiers based on a shrinkage estimate of the 435 covariance matrix (Ledoit and Wolf, 2004). In a first step, we tested how well color (and luminance) 436 information could be decoded from brain signals when the training and test data came from the 437 same participant. In the second step, we examined how well color (or luminance) patterns gener-438 alized across participants. We refer to the analyses in the first and second steps as within-subject 439 classification (WSC) and between-subject classification (BSC), respectively (Haxby et al., 2011). 440 In WSC analyses, we trained LDA classifiers to predict from vectors of neural responses from 441 which of the three color categories (or the two luminance conditions in the luminance classification) 442 they came. We obtained unbiased estimates of classification performance by cross-validating them 443 following a leave-one-run-out cross-validation procedure. Individual results were averaged across 444 all participants.

445
In BSC analyses, however, we were interested to learn how well color (or luminance) responses 446 generalized across participants. Rather than cross-validating classifiers across runs and averag-447 ing over participants we now cross-validated across participants and obtained a single value for 448 the whole group. Another difference was that vectors used for classification came from the 50-449 dimensional common space estimated through shared response modeling (see below). Since 450 shared response modeling implicitly already performs dimensionality reduction, we did not use 451 any additional feature selection such as recursive feature elimination.

452
Since color decoding was a three-way classification whereas luminance decoding was a two-way 453 classification, classification accuracies were transformed to z-values using the normal approxima-454 tion to the binomial distribution: is the number of predictions in the classification problem (i.e., Bernoulli trials), is the fraction chance. After z-transformation chance level for both classification problems was hence zero.

460
Shared response modeling 461 Our hypothesis was that the purely achromatically defined functional retinotopic architecture shared 462 across brains also contained color representations that were equally shared across participants.  We fit SRMs to data from the retinotopic mapping experiment (Figure 1). This yielded one trans- In addition to ROI analyses, we also applied SRM in a whole-brain analysis to local patterns at every 505 location in the brain normalized to MNI standard space. The purpose of this analysis was to test 506 16 of 24 . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint how specific the shared commonalities were to the exact individual locations of visual areas com-507 pared to anatomically aligned patterns of brain responses. Please note that anatomical alignment 508 here refers to the alignment of response patterns which are then subjected to SRM. We do not 509 assume that anatomically aligned voxels already exhibit similar stimulus tunings. Rather, we were 510 interested in determining the difference between modeling shared responses between retinotopi-511 cally versus anatomically aligned patterns of brain activity. 512 We used SRM in combination with a searchlight technique that analyzed only local patterns of 513 brain activity within a radius of 3 voxels ("searchlight sphere") at every location in the whole brain 514 (Kriegeskorte et al., 2006). The number of SRM components was again 50 in every searchlight 515 sphere or -if the total number of voxels within the sphere was smaller (e.g. near edges in the brain 516 mask) -the number of available voxels. LDA was then used in the same manner as in the BSC ROI 517 analyses to calculate an unbiased estimate of classification performance for every location in the 518 brain, which was then assigned to the center voxel of the sphere. The resulting brain map was 519 then spatially smoothed with a Gaussian kernel of size 6 mm FWHM. where each column in corresponds to the weight vector for that class. 537 We then checked if color preference could be predicted from retinotopic Cartesian coordinates.

538
For every voxel in all participants, we determined which class had the highest coefficient and its 539 x, y coordinates (Figure 4). We fitted nearest neighbor classifiers (considering 5 neighbors) to dis-540 tinguish between stimulus classes, leaving out all voxels from one participant at a time for cross-541 validation. We thus determined a cross-validated class prediction for every voxel indicating its 542 preferred color. Gaussian kernel density estimation (KDE, using Scott's rule to choose bandwidth) 543 was used to visualize biases in preferred color: first, KDE was applied to all voxels in the group of 544 participants. Then it was applied to only those voxels with a preference for a specific class yielding 545 one density for each class in the classification problem. The density estimated from all voxels was 546 finally subtracted from each of the class-specific densities (Figure 4 bottom). Positive values denote 547 retinotopic locations where preference for a given class was more pronounced with respect to the 548 overall voxel density. Negative values denote the opposite and are shown in gray in Figure 5. 549 Statistical inference 550 Within-subject and between-subject classification.

551
All our statistical decisions were based on permutation tests. For WSC we tested the one-tailed null 552 hypothesis that the sample average across all participants' classification accuracies was equal to 553 or below chance. Since there were three classes in the color classification problem, chance level 554 . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint was 1∕3. Likewise chance level in the luminance classification was 1∕2 as there were two luminance 555 levels. 556 We created 2000 new label assignments in the following way: For every participant the se-557 quence of training labels was shuffled with the restrictions that labels were only permuted within 558 cross-validation folds, i.e., functional runs, and that permutations were identical across ROIs. Both 559 restrictions were implemented by setting the groups and random_state arguments in scikit-learn's 560 permutation_test_score accordingly. A group average of classification accuracies was calculated 561 for each ROI and iteration. This way we obtained a null distribution of average classification accu-562 racies expected under the hypothesis that there was no relationship between labels and neural 563 activity patterns. 564 It was important that permutations were identical across ROIs because they constituted the 565 test family for which we controlled the family wise error (FWE). We formed a new null distribution 566 for all ROIs by taking the maximum of group average classification accuracies across ROIs in every 567 iteration (Nichols and Holmes, 2002). P values were computed as the fraction of permutations that 568 resulted in accuracies that were larger than or equal to the observed classification accuracy. We 569 declared results significant if p was below .05, thereby keeping the type I error probability of falsely 570 rejecting at least one null hypothesis at = 0.05. For each ROI, the lower and upper limits of the 95 571 % confidence interval (CI) were calculated parametrically using the standard error of the mean. In 572 the BSC analysis, upper CI limits were obtained by adding the difference between the mean of the 573 (uncorrected) null distribution and its 2.5th percentile to the observed accuracy. Lower CI limits 574 were obtained by subtracting the difference between the 97.5th percentile and the mean from the 575 observed accuracy.

576
The permutation test for BSC was identical to that for WSC. However, since data from all par-577 ticipants were now combined in the 50-dimensional functional common space and hence non-578 independent, classifiers were now cross-validated leaving out one participant at a time. Again, 579 labels were permuted only within cross-validation folds, which in this analysis were individual par-580 ticipants.

582
In this analysis we tested for each ROI whether we could predict from the Cartesian visual field 583 coordinates which color (or luminance level) was preferred by a voxel using a nearest neighbor 584 classifier. Voxel labels were permuted 10 3 times, separately within cross-validation folds (partici-585 pants), and p values were obtained from the null distribution of each ROI. CIs were calculated in the 586 same way as for BSC results. Since there was no correspondence between voxels from different 587 ROIs, we could not control the FWE using the same max statistic approach as in the previous ROI 588 analyses and therefore used Holm-Šidák correction instead to keep at .05.

590
The searchlight analysis yielded a brain map of cross-validated estimates of classification accu-591 racies. In order to test if classification accuracies were significantly larger than what would be 592 expected by chance, we used a one-tailed binomial test instead of a t-test because the classifica-593 tion accuracies from each leave-one-participant-out cross-validation were not independent. The . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint . CC-BY 4.0 International license available under a was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which this version posted June 2, 2020. ; https://doi.org/10.1101/2020.05.31.126532 doi: bioRxiv preprint