Choice activity stabilizes sensory representations and mediates sensorimotor associations in parietal cortex

Selecting actions which achieve desired goals often requires three-dimensional (3D) representations of the environment. Because the sensory epithelia cannot directly encode the world’s 3D spatial features, sensory signals must be converted into 3D representations. Here we investigated the relationships between the quality of 3D visual representations, choice-related activity, and motor-related activity in the parietal cortex of macaque monkeys using an eight-alternative 3D orientation discrimination task, visually guided saccade task, and laminar probe recordings. We found that choice activity was preferentially carried by caudal intraparietal area neurons with more robust 3D representations. Choice activity further stabilized the 3D representations, rather than attenuating information not directly relevant to the behavioral task (nuisance variables). An experience-dependent, sensorimotor association additionally aligned sensory and saccade direction preferences, particularly for neurons with choice activity. These findings reveal novel roles for choice activity in improving the fidelity of ecologically relevant object representations and mediating sensorimotor associations.


Introduction 48
Interactions with the environment require that sensory information be mapped to motor responses. 49 The parietal cortex is an important site of sensorimotor transformations (Rushworth et al., 1997; 50 Buneo et al., 2002;Brovelli et al., 2004;Buneo and Andersen, 2006). Parietal lesions can result 51 in deficits associated with the impaired use of sensory information to create and execute motor 52 plans, as opposed to deficits in sensory processing or action Pause et 53 al., 1989). They can also produce 3D visual processing deficits (Holmes, 1918;Holmes and 54 Horrax, 1919). Although sensorimotor transformations are often studied using two-dimensional 55 (2D) stimulus paradigms, mappings between 3D object information and motor responses are 56 essential in the natural world. Thus, parietal cortex may have a fundamental role in creating robust 57 3D representations and mapping them to specific actions. Angelaki, 2014b), and perform multisensory processing to achieve gravity-centered object 62 representations (Rosenberg and Angelaki, 2014a). In addition to carrying high-level sensory 63 representations, CIP activity correlates with the short-term memory and perceptual matching of 64 3D features (Tsutsui et al., 2003), and choices made during a binary orientation discrimination 65 task (Elmore et al., 2019). Furthermore, inactivating CIP impairs 3D feature discrimination 66

Quantifying 3D tilt sensitivity 101
Testing our hypotheses required a 3D visual discrimination task that could also be used to quantify 102 the robustness of neuronal 3D representations. To this end, we trained two monkeys to perform 103 an 8AFC orientation discrimination task under different viewing conditions that determined the 104 task difficulty (Chang et al., 2020). The task required the monkeys to report which side of a plane 105 was nearest (i.e., the plane's tilt) through a saccade to one of eight choice targets ( Figure 1A). Tilt and slant are polar coordinates describing 3D surface orientation ( Figure 1B). Tilt is 129 the angular variable (0° ≤ T < 360°) and specifies the direction that the plane is oriented in depth. 130 For example, T = 0° indicates right-near and T = 90° indicates top-near. Planes were presented 131 at 8 tilts (0° to 315°, 45° steps), corresponding to the 8 choice options. Slant is the radial variable 132 (0° ≤ S ≤ 90°) and specifies the amount of depth variation. There is no depth variation at S = 0°, 133 so tilt is undefined and there is no correct task response. Larger slants indicate greater depth 134 variation. Planes were presented at 5 slants (0° to 60°, 15° steps). Each of the 33 orientations 135 was presented at 4 distances: 37, 57, 97, and 137 cm ( Figure 1C) (Fisher, 1995). We therefore summarized sensitivity as a function of slant and distance 145 calculated over all tilts ( Figure 1D, Figure 1-figure supplement 1). Consistent with our previous 146 findings in which behavioral performance in the 8AFC tilt discrimination task was extensively 147 analyzed over a wide range of 3D poses as well as multiple visual cue conditions (Chang et al.,148 2020), we found that sensitivity decreased with distance from fixation (57 cm) and increased with 149 slant ( Figure 1E).  While the monkeys performed the task, we recorded from 437 neurons (Monkey L: N = 218; 166 Monkey F: N = 219) using laminar probes (Figure 2). Because CIP neurons can carry sensory 167 and choice-related signals (Elmore et al., 2019), we divided the responses into two time windows. 168 The "sensory only" (SO) window started at the median visual response latency (52 ms) and ended 169 at the onset of choice activity (202 ms; calculated below). The "sensory plus choice" (SPC) 170 window, which can include both sensory and choice activity, started at the onset of choice activity 171 and ended at the stimulus offset (1 s). We first analyze responses in the SO window. These possibilities can be disentangled by assessing how orientation selectivity is affected by the 184 object's distance when the fixation distance is held constant. A 3D representation is indicated if 185 the orientation selectivity (tuning curve shape) is invariant to the distance (though the gain can 186 change). In contrast, sensitivity to lower-level features is implied if the selectivity changes 187 drastically. The sensory responses of five CIP neurons that portray the range of sensory, choice-188 related, and motor-related properties we found are shown in Figure 3. 189 In an idealized 3D representation, the orientation tuning (preference and bandwidth, but 190 not gain) is independent of distance. That is, 3D pose tuning will be separable over orientation For only 40 neurons (9%) was there a statistically significant effect of vergence (p < 0.05). 222 Moreover, the significance of the tilt, slant, and distance main effects was unchanged for all but 223 11 neurons (3%) when vergence was included as a covariate. These effects are smaller than 224 those previously reported for disparity tuning in the middle temporal area (DeAngelis and Uka, of these distances, the mean slant preference was larger than at 57 cm ( Figure 3F). 236 To determine why the slant preferences were larger at distances closer and further than 237 fixation, we performed pairwise comparisons of the slant preferences of neurons with orientation 238 tuning at adjacent distances. Preferences at 37 and 57 cm were not significantly different (paired 239 t-test, p = 0.19, N = 209). For that subset of neurons, we therefore examined the distribution of 240 preferences at 57 cm and found that it was significantly different from uniform ( 2 , p = 1.5x10 -2 ). 241 In contrast, the distribution of preferences for the 111 neurons that represented 57 but not 37 cm 242 was not different from uniform (p = 0.24). Thus, neurons that represented 37 cm tended to prefer 243 larger slants. We further found that the preferences of individual neurons tended to increase from 244 57 to 97 cm (mean ΔSlant = 5.1°; paired t-test, p = 1.1x10 -3 , N = 144), as well as from 97 to 137 245 cm (mean ΔSlant = 4.3°; p = 9.7x10 -3 , N = 87). Thus, there was a slight inseparability in the 246 sensory tuning such that the slant preferences tended to increase with distance behind fixation. 247 These results are consistent with the behavioral data which showed that sensitivity fell off more 248 slowly with distance from fixation at larger compared to smaller slants ( Figure 1E). 249 We previously found that behavioral tilt sensitivity could be explained by a neuronal

Choice activity was parametrically tuned and aligned with the sensory preferences 267
We next tested each neuron for choice-related activity. An advantage of the 8AFC task was that 268 it could reveal parametric choice tuning, which is not possible with a typical binary choice task. 269 To test for choice-related activity, we only used frontoparallel plane (S = 0°) responses since tilt 270 is undefined at that orientation. To increase the statistical power, we combined responses across 271 all distances after separately z-scoring them at each distance. Responses were then grouped 272 according to the choice that was made on each trial. preferences tended to align.

296
Choice tuning curves with von Mises fits are shown for the example neurons in Figure  297 5B-F (blue curves). The high (Figures 3A, 5B), intermediate (Figures 3B, 5C), and first low 298 (Figures 3C, 5D) tolerance neurons all had choice activity. The second low tolerance neuron did 299 not have choice activity (Figures 3D, 5E). The neuron with stronger distance than orientation 300 tuning also had choice activity (Figures 3E, 5F). Across the population, the choice tuning curves 301 were well described by von Mises functions (mean r = 0.88 ± 0.10 SD, N = 201). The mean 302 concentration was = 3.52 ± 4.74 SD, and the mean half-width at half-height was 52° ± 21° SD. 303 For comparison, surface tilt tuning curves marginalized over slant and distance (SO 304 window) are shown in Figure 5B-F (black curves). The tilt and choice preferences of the example 305 neurons aligned, even for the neuron with stronger distance than orientation tuning. To quantify 306 this relationship across the population, we used the orientation preferences measured at each 307 distance to compute a principal orientation preference for each neuron (SO window; see 308 Materials and methods). We then compared the principal surface tilt and choice preferences 309 from the von Mises fits (Figure 5G). The median circular difference between the preferences was 310 We next tested if choice activity was associated with changes in 3D selectivity. To start, we 315 calculated the correlation between tuning curves in the SO and SPC windows (over all 132 poses). 316 The mean correlation was r = 0.49 ± 0.27 SD (N = 437), suggesting that tuning was similar in the 317 two time windows but not identical. We hypothesized that the changes reflected a stabilizing effect 318 of choice activity on 3D tuning, but also considered that choice might attenuate selectivity for slant 319 and distance (nuisance variables during tilt discrimination), leaving only selectivity for the task-320 To illustrate how selectivity changed after the onset of choice activity, the orientation 322 tuning at each distance is shown for the example neurons during the SPC window in Figure 6A-323 E. First consider the three neurons with strong orientation and choice tuning. The selectivity of 324 the neuron with high tolerance in the SO window changed little (Figures 3A, 6A), and its tolerance 325 slightly increased: ΔTolerance = 0.08. Strikingly, the orientation tuning of the neuron with 326 intermediate tolerance in the SO window aligned across distances in the SPC window, matching 327 the tuning at the preferred distance ( Figures 3B, 6B), and its tolerance greatly increased: 328 ΔTolerance = 0.35. For the neuron with low tolerance in the SO window, responses at non-329 preferred distances remained relatively weak in the SPC window. However, the orientation tuning 330 was now significant at each distance and the preferences were similar (Figures 3C, 6C), resulting 331 in a large increase in tolerance: ΔTolerance = 0.46. Next consider the neuron with low tolerance 332 during the SO window and no choice activity. The selectivity of this neuron changed relatively little 333 during the SPC window (Figures 3D, 6D). Lastly, the distance selective neuron continued to 334 respond most strongly to the furthest distances (Figures 3E, 6E). During the SPC window, 20 335 neurons (5%) were distance selective (two-way ANOVA, p < 0.05) but not orientation selective (p 336 The proportion of neurons with tuning decreased with distance from fixation (57 cm), consistent 346 with the pattern of behavioral sensitivity (Figure 1E). (G) Proportion of neurons with orientation 347 tuning at each possible number of distances during the SO (orange) and SPC (blue) windows.

349
Slant and distance tuning persisted in the SPC window for the example neurons, 350 suggesting that choice activity did not attenuate information that was not directly relevant to the 351 task. This finding was typical across the population. For each neuron and time window, we tested 352 for slant and distance tuning (4-way ANOVA with tilt linearized into cosine and sine components,  Figure 6F). Indeed, individual neurons often had orientation tuning at two distances 360 during the SO window but all four distances during the SPC window ( Figure 6G). 361 362

Choice activity was carried by more tolerant neurons and stabilized 3D selectivity 363
We next tested if neurons with robust 3D tuning preferentially carried choice activity. To do so, 364 we compared the SO window tolerances of neurons with and without choice activity (Figure 7A). 365 The mean tolerance was greater for neurons with choice activity (0.61) than without (0.56). 366 Although the difference was not large, it was significant (ANOVA followed by Tukey's HSD test, 367 p = 0.02), indicating that choice activity was preferentially carried by neurons with more robust 3D 368 tuning. We then tested if choice activity further stabilized 3D selectivity by assessing if tolerance 369 increased in the SPC window. The difference in mean tolerance between neurons with (0.72) and 370 without (0.60) choice activity increased in the SPC window ( Figure 7B) and was significant (p = 371 3.8x10 -9 ). The greater difference was due to an increase in tolerance between time windows for 372 neurons with choice activity (p = 3.8x10 -9 ; Figure 7C). The tolerance of neurons without choice 373 activity did not significantly change (p = 0.06). We additionally found that the SPC window To further characterize the effects of choice activity on 3D selectivity, we quantified the 390 differences in orientation preference across distance in each time window. For each neuron with 391 choice activity, we calculated a principal orientation preference using both time windows (see 392 Materials and methods). Deviations in orientation preference from the principal orientation are 393 plotted for each time window in Figure 7D,E. In these plots, the angular variable is the direction 394 that the preferred orientation at a given distance deviated from the principal orientation (∆Ψ), and 395 the radial variable is the angular deviation (∆∡). The origin indicates no difference, and points on 396 the outer ring indicate the maximal difference (90°). The deviations were not significantly different 397 from the origin in either time window (test for a specified principal axis; both p-values ≥ 0.09) 398 (Fisher et al., 1993). To assess how much the orientation preferences deviated from the principal 399 orientation, we calculated cumulative density functions for the angular deviations ( Figure 7F). 400 The mean deviation was greater in the SO (28.3°) than the SPC (20.4°) window, and the 401 cumulative densities significantly differed (Kolmogorov-Smirnov test, p = 1.4x10 -10 ). Thus, the 402 orientation preferences became more similar across distance after the onset of choice activity. 403 We repeated this analysis for neurons without choice activity, and found that the cumulative 404 densities for the two time windows were not significantly different (p = 0.91). Thus, 3D selectivity 405 stabilized in the SPC window, but only for neurons with choice activity. and Schall, 1996) ( Figure 8A). 414 We computed eight population-level time courses of saccade-related activity, relative to 415 the saccade direction that elicited the maximum response for each neuron (Figure 8B). The time 416 course of saccade-related activity had parametric tuning, with an amplitude that symmetrically 417 decreased with greater deviation from the preferred direction. Significant saccade direction 418 selectivity began 102 ms before saccade onset (ANOVA, p < 0.05). We further found that the 419 activity predicted the saccade timing. For each neuron, we labeled every trial in which a saccade 420 was made in the preferred direction according to the saccade latency in quartiles. Time courses 421 were then calculated for each quartile (Figure 8C; inset shows the latency histogram). The four 422 curves approximately intersected 15 ms before saccade onset, suggestive of a saccade initiation 423 threshold (~42 spikes/s, black circle). Moreover, the activity increased more slowly when the 424 saccade latency was longer (colored circles mark when each curve significantly deviated from 425 baseline; ANOVA, p < 0.05). For each curve, we computed the growth rate (linear slope) between 426 the start of activity and the putative saccade threshold. Consistent with frontal eye field findings 427 (Hanes and Schall, 1996), there was an inverse linear relationship between the growth rate and 428 mean saccade latency ( Figure 8D). Thus, saccade-related activity in CIP functionally correlated 429 with both the saccade direction and timing during the visually guided saccade task. The mean concentration parameter was = 4.67 ± 4.84 SD, and the mean half-width at half-451 height was 42° ± 17° SD. Saccade direction tuning curves with von Mises fits are shown for the 452 example neurons in Figure 8E-I (green curves). The first two neurons had orientation (Figures  453   3A,B, 6A,B) and choice (Figure 5B,C) tuning, as well as saccade direction tuning (Figure 8E,F). 454 The third neuron had orientation (Figures 3C, 6C) and choice (Figure 5D) tuning, but not saccade 455 direction tuning (Figure 8G). The fourth neuron, which had orientation (Figures 3D, 6D) but not 456 choice ( Figure 5E) tuning, had saccade direction tuning ( Figure 8H). Lastly, the neuron with 457 stronger distance than orientation tuning (Figures 3E, 6E) had both choice ( Figure 5F) and 458 saccade direction (Figure 8I) tuning, but the choice and saccade preferences were not aligned. 459 These various differences illustrate that choice and saccade response properties were dissociable. 460 For comparison, surface tilt tuning curves marginalized over slant and distance (SO+SPC 461 windows) are shown in Figure 8E-I (black curves). The tilt and saccade direction preferences of 462 the example neurons with strong orientation and choice tuning were well aligned (Figure 8E,F), 463 implying a sensorimotor association at the individual neuron level. The preferences were also 464 reasonably well aligned for the neuron with strong orientation but not choice tuning ( Figure 8H). 465 They were not aligned for the neuron with stronger distance than orientation tuning ( Figure 8I). 466 To quantify the sensorimotor association, we compared the principal tilt (SO+SPC windows) and 467 saccade direction preferences from the von Mises fits. Across the population, the preferences 468 aligned both for neurons without ( Figure 9A) and with ( Figure 9B) choice activity. For neurons 469 without (with) choice activity, the median circular difference between the preferences was -0.9° 470 (-1.6°) and not significantly different from 0° (circular median test, both p-values ≥ 0.61). Although 471 the preferences tended to align regardless of choice activity, the distribution was significantly 472 wider for neurons without (circular variance = 0.74) than with (0.36) choice activity (two-sample 473 concentration difference test, p = 3.2x10 -5 ) (Fisher, 1995), indicating that the sensorimotor 474 association was strongest for neurons with choice activity.  Interacting with the environment requires the creation of robust representations of 3D information 491 that the epithelia cannot directly sense, and mapping those representations to motor responses. 492 We investigated the relationships between the quality of 3D representations, choice-related 493 activity, and motor-related activity in CIP. The behavioral task required the monkeys to report the 494 tilt of a plane, regardless of its slant or distance. As such, slant and distance were nuisance 495 variables that were not of direct interest but modulated task performance. Rather than attenuating 496 selectivity for the nuisance variables, choice activity improved the fidelity of the 3D representation. 497 Thus, the low-dimensional task (report the tilt) did not reduce the dimensionality of the neuronal 498 representation of the multi-dimensional (tilt, slant, and distance) stimulus. This may enable more 499 robust 3D perception and flexible processing. For example, if choice activity attenuated selectivity 500 for features not directly relevant to the task at hand, performance would be impaired if the task 501 and relevant information unexpectedly changed. 502 The monkeys were trained to report which side of a plane was nearest, whereas a previous 503 CIP study had monkeys report which side was furthest (Elmore et al., 2019). In both studies, 504 surface tilt and choice preferences aligned. For example, given a neuron preferring a bottom-near 505 tilt, the preferred choice report tended to be the lower target in this study but the upper target in 506 the previous study. Here, we further found that many CIP neurons have saccade-related activity, 507 that saccade direction preferences aligned with the sensory and choice preferences, and that 508 choice-and saccade-related activities were dissociable. These findings together reveal a flexible, 509 experience-dependent mapping between sensory, choice-, and motor-related activity, indicating 510 that experience has ongoing effects on the functional properties of CIP neurons, as occurs in A stereoscopic representation of 3D orientation that is tolerant to distance requires an encoding 518 of relative disparity gradients that takes into account the nonlinear relationship between depth 519 and disparity. We found that the orientation selectivity of many CIP neurons was highly tolerant 520 to distance. That tolerance cannot simply reflect tuning for perspective cues (even though they 521 were independent of distance) because the vast majority of neurons had significant distance 522 tuning. Our findings thus reveal a high-level representation of 3D object pose created using 523 relative disparity computations. How might this representation be achieved? 524 Functional properties and connectivity data suggest that tuning for 3D object pose is built 525 hierarchically. Neurons in V1 represent local absolute disparities (Cumming and Parker, 1997, 526 1999. A transformation from absolute to relative disparity selectivity likely proceeds in V2 527 Showing that 3D orientation selectivity is relatively invariant to distance when the fixation distance 543 is held constant is a common criterion to conclude 3D tuning. This fundamental test was never 544 previously performed in CIP. We found that slant preferences tended to slightly increase with 545 greater distance behind fixation. Although the increase was small, the finding indicates that 546 orientation and distance tuning was not strictly separable. The behavioral data likewise showed 547 that the monkeys' sensitivity fell off more slowly with distance from fixation at larger compared to 548 smaller slants. This behavioral finding implies that larger slants were required to elicit robust 549 stimulus-selective neuronal responses at distances further from fixation, predicting the observed 550 increase in slant preference and suggesting that strictly separable tuning may not be a reasonable 551 expectation. It is unlikely that this conclusion rests upon using stimuli with fixed slants as opposed 552 to fixed disparity gradients. Although the disparity gradient signaling a given slant is distance 553 dependent (due to the nonlinear relationship between depth and disparity), using fixed disparity 554 gradients would mean that the 3D shape of the stimuli would change with distance. As such, 555 regardless of whether fixed slants or disparity gradients are used, it is unlikely that tuning will be 556 strictly separable over orientation and distance. 557 558

Potential origins of choice-related activity 559
Choice activity is traditionally associated with feedforward contributions to perception (Celebrini 560 and Newsome, 1994;Britten et al., 1996;Dodd et al., 2001;Nienborg and Cumming, 2006;Gu et 561 al., 2007). This possibility has been questioned on the basis that choice activity, attentional effects, 562 and correlated activity can be conflated (Cumming and Nienborg, 2016). Existing choice activity 563 measures likely reflect combinations of these factors (Haefner et al., 2013;Gu et al., 2014). We 564 found that surface tilt and choice preferences typically aligned. This is expected for a feedforward 565 origin of choice activity, and contrasts with the potentially non-specific effects of a feedback origin, 566 such as proposed for the ventral intraparietal area (Zaidel et al., 2017). We further found that 567 choice activity was preferentially carried by neurons which robustly represented the task-relevant 568 sensory information, and that the strength of choice activity increased with sensory tolerance. The 569 extent to which subcortical vestibular neurons resolve the gravito-inertial acceleration ambiguity 570 necessary to represent translation independent of head tilt (Dakin and Rosenberg, 2018) similarly 571 correlates with the strength of choice activity (Liu et al., 2013). Across modalities, these results 572 reveal that neurons which have resolved fundamental ambiguities about the sensory information 573 that an animal is actively discriminating preferentially carry choice activity. This may reflect a 574 feedforward origin of choice activity, such that neurons with more robust tuning for task-relevant 575 information have greater weight in the neural readout. Indeed, decoding neurons that are more 576 tolerant to nuisance variables may simplify readout since marginalizing out non-relevant features 577 would be more straightforward. 578 579

Sensorimotor associations 580
We discovered two roles for CIP beyond sensory processing. First, we showed that CIP activity 581 functionally correlates with both the direction and timing of visually guided saccades, implicating 582 the area in visuomotor control for the first time. Second, we found sensorimotor associations at 583 the single neuron level, which were strongest for neurons with choice activity (i.e., those with more 584 robust 3D selectivity). This finding may reflect that neurons with robust sensory representations 585 and stronger sensorimotor associations have greater weight in determining motor responses. 586 Associating robust sensory and motor activity at the single neuron level may help ensure 587 successful and timely interactions with the environment since the same neurons representing the 588 relevant sensory information also signal the appropriate motor response. While our findings 589 implicate CIP in visuomotor functioning, the area is also connected with prehensile areas, 590 suggesting that its sensorimotor functions may be more general. This possibility is consistent with 591 recent findings implicating human parietal regions that integrate visual orientation and saccade 592 signals in the updating of grasp plans during eye movements (Baltaretu et al., 2020). The stimuli were the same as the combined-cue stimuli in our previous work (Chang et al., 2020). The monkeys were trained to perform an 8AFC tilt discrimination task (Chang et al., 2020). Each 637 trial began with fixation of a circular target (0.3°) at the center of the screen for 300 ms. A plane 638 then appeared at the center of the screen for 1 s. The target and plane then disappeared, and 8 639 choice targets appeared at polar angles of 0° to 315° in 45° steps (11° eccentricity). The nearest 640 side of the plane was reported by making a saccade to the corresponding target for a liquid reward. 641

Analysis of behavioral data 674
Tilt discrimination performance was quantified by fitting a von Mises probability density function 675 repeated until the onset no longer changed. 711 Quantifying orientation selectivity. For each distance and time window, we tested for 712 orientation tuning (ANOVA, p < 0.05, with Bonferroni-Holm correction). The preferred tilt and slant 713 was estimated for each significant case by fitting a Bingham function (Rosenberg et al., 2013). 714 Differences in orientation preference were assessed as follows. First, we calculated the principal 715 orientation about which the measured orientation preferences clustered (Fisher et al., 1993).