Abstract
Task-related neural activity is widespread across populations of neurons during goal-directed behaviors. However, little is known about the synaptic reorganization and circuit mechanisms that lead to broad activity changes. Here we trained a limited subset of neurons in a spiking network with strong synaptic interactions to reproduce the activity of neurons in the motor cortex during a decision-making task. We found that task-related activity, resembling the neural data, emerged across the network, even in the untrained neurons. Analysis of trained networks showed that strong untrained synapses, which were independent of the task and determined the dynamical state of the network, mediated the spread of task-related activity. Optogenetic perturbations suggest that the motor cortex is strongly-coupled, supporting the applicability of the mechanism to cortical networks. Our results reveal a cortical mechanism that facilitates distributed representations of task-variables by spreading the activity from a subset of plastic neurons to the entire network through task-independent strong synapses.
Introduction
Large-scale measurements of neural activity show that learning can rapidly change the activity of many neurons, resulting in widespread changes in task-related neural activity [1, 2, 3, 4, 5]. For instance, a goal-directed behavior involving motor planning produces macroscopic neural dynamics that is coordinated by multiple cortical areas [1, 3], and neural correlates of action and choice made during a decision-making task are organized spatially across brain regions [4].
To gain insights into the circuit mechanism behind the observed widespread activity, it is critical to understand how interconnected neural circuits modulate their synaptic connections to produce the observed changes in task-related neural activity. Tracking synaptic modifications during learning [6, 7, 8, 9] and manipulating them to demonstrate a causal link with behavioral outputs [10, 11, 12, 13, 14], show that synaptic plasticity underlies learned behaviors and changes in neural activity [15, 16]. However, it is highly challenging to conduct multi-scale experiments that monitor and manipulate learning-specific synaptic changes at cellular resolution across a wide region of cortical networks, while measuring the resulting neural activity [17]. Thus, it remains unclear what aspects of the synaptic connections are modified to produce the widespread changes in task-related neural activity.
Here we investigated if task-related activity, learned locally in a dedicated subset of neurons, can spread across the network through pre-existing, task-independent, synaptic pathways. Although distributed neural activity may result from broad changes in synaptic connections across a neural network, we hypothesized that recruiting only a small subset of neurons is sufficient to generate the distributed task-related neural activity. To test this hypothesis, we used recurrent neural networks that provide a powerful data-driven approach for investigating how synaptic modifications can support the observed task-related neural activity. [18, 19, 20, 21, 22].
In typical implementations of network training, all the neurons in the network are considered to be plastic, in that the activity of every neuron is fit to the activity of experimentally recorded neurons, thereby constraining the entire network activity to the neural data [19, 20, 21]. In this study, we instead trained only a subset of neurons in a biologically plausible network to reproduce the activity of recorded neurons. The network consisted of excitatory (E) and inhibitory (I) populations of spiking neurons with sparse and strong connections [23, 24, 25]. Such pre-existing, task-independent, connections made the network settle into a cortical-like dynamical regime, where excitation and inhibition balanced each other [23, 25, 26], resulting in temporally irregular spikes and heterogeneous spike rates [27].
We applied our modeling framework to study the spread of task-related activity in the anterior lateral motor cortex (ALM) of mice performing a memory-guided decision-making task [21]. Similarly to neurons in the primate motor cortex [28, 29, 30], the activity of many neurons in ALM ramps slowly during motor preparation and is selective to future actions [21, 31, 32]. These task-related activity patterns are widely distributed across the ALM and are highly heterogeneous across neurons.
When a small number of synapses was trained to reproduce the ALM activity in a subset of neurons, we found that, surprisingly, the emerging activity in the untrained model neurons closely matched the responses of ALM neurons held out from training. In other words, the task-related ALM activity, learned in a subset of neurons, spread to other untrained neurons in the network without further training and produced activity that resembled the actual responses of ALM neurons. Analysis of the trained networks revealed that the pre-existing strong synapses between the neurons mediated the propagation of the task-related activity. The trained activity failed to spread in networks of neurons that were not strongly coupled. Optogenetic perturbation experiments of ALM activity provided additional evidence that the ALM network is strongly coupled, supporting the proposed mechanism for the spread of trained activity.
Our work provides a general circuit mechanism for spreading activity in cortical networks. It suggests that task-related activity observed in cortical regions during behavior can emerge from a subset of neurons with sparse synaptic reorganization and propagate to the rest of the network through the strong, task-independent synapses.
Results
Training strongly coupled spiking neural networks with sparse synapses
Our spiking network was based on a cortical circuit model that provided mechanistic explanation of the canonical features of cortical activity, such as temporally irregular spike trains, large trial-to-trial variability and a wide range of firing rates across neurons [23, 24, 25, 27, 33]. It consisted of excitatory (E) and inhibitory (I) neurons sparsely and randomly connected by strong synapses. This initial EI network structure, due to its random connectivity, was independent of the task to be learned. In addition, the strong excitatory and inhibitory synaptic inputs were dynamically balanced to maintain a stable network state, known as the balanced regime. As in the cortex, neurons, driven by fast fluctuating synaptic inputs, emitted spikes stochastically in this network state.
Here we describe a training scheme referred to as Subset Training, which trains a subset of neurons in the EI spiking network to generate target activity patterns, with the rest of the neurons untrained (Fig. 1A, left). After training the selected subset of neurons, we can analyze if the learning-related changes in activity spread throughout the network (Fig. 1A, right).
To model the effects of learning, a relatively small number of plastic synapses connecting to the subset of neurons was introduced to the existing EI spiking network. We implemented such network architecture by decomposing the recurrent connectivity into static and plastic components. The static component was the strongly coupled EI network connectivity that made the network operate in the balanced regime ( Fig. 1A, solid arrows). It remained unchanged throughout network training. For the plastic component, we added sparse connections, that were sparser than the EI connectivity (Fig. 1A, magenta dashed arrows). The choice of sparse plastic synapses was, in part, motivated by the synaptic connectivity found in the cortex that is sparse but functionally biased [34]. For instance, with K = 1000 static synapses, there were of the order of plastic synapses per neuron (Table 2). The plastic synapses were connected only to the trained neurons from randomly selected presynaptic neurons. During network training, a synaptic learning rule based on recursive least squares [19, 35, 36, 37] optimized the strength of plastic synapses to generate the target activity patterns in the selected subset of neurons.
This arrangement of plastic synapses, which connected only to the selected subset of neurons, allowed us to examine the role of the pre-existing recurrent connections of the EI network in spreading the trained activity to the untrained neurons, which were not targeted by the plastic synapses. In addition, due to their sparsity, the plastic synaptic inputs were substantially weaker than the strong excitatory and inhibitory synaptic inputs of the existing EI network (Fig. 1B). This allowed the network to stay in the balanced regime after training and supported robust network training, independent of the density of synaptic connections (Fig. S1; see Methods for full description of the training and details on the sparse plastic synapses).
In the trained network, the total synaptic input to each trained neuron was able to successfully follow the target patterns (Fig. 1B, left; Fig. 1C). The statistics of spiking activity of the trained network were similar to those of untrained, strongly coupled EI networks, thus consistent with the spiking activity of cortical neurons. Specifically, due to the highly fluctuating balanced input, the spike trains of each neuron were irregular and exhibited large trial-to-trial variability (Fig. 1D, Fano factor = 1.4) [23, 24, 38]. The firing rate distribution was also highly skewed and was well approximated by a log-normal distribution (trained model: Fig. 1E, neural data: Fig. S2D) [27].
Spread of trained neural activity to untrained neurons
We applied the Subset Training method to reproduce the firing rate patterns of cortical neurons recorded from the anterior lateral motor cortex (ALM) during a memory-guided decision-making task [21]. Mice learned to respond to an optogenetic stimulation of neurons in the vibrissal somatosensory cortex (vS1) by licking right when stimulated and licking left otherwise (Fig. 2A). For training networks and analysis of trained networks, we used the electrophysiological recordings in ALM of the spiking activity of putative pyramidal neurons (excitatory; Npyr = 1824) and putative fast-spiking neurons (inhibitory; Nfs = 306) when the mice responded correctly to lick-right and lick-left conditions.
We asked what aspects of the network connectivity should change to reproduce the activity of ALM neurons in a strongly coupled spiking neural network. In previous studies, in which networks were trained to generate specific patterns of neural activity, all the units in the network were trained to reproduce the target activity patterns [18, 19, 20, 21]. Here, we trained only a subset of neurons, embedded in the strongly coupled EI network, to reproduce the spiking activity of ALM neurons. By analyzing the activity of neurons in the trained network, we found that synaptic reorganization to a subset of neurons was sufficient to generate the observed ALM activity throughout the entire network, including the untrained neurons. Importantly, the spread of target activity patterns from the subset of trained neurons to the rest of neurons was not observed in a network that was not strongly coupled (Fig. S9). This suggests that the spread of trained activity to untrained neurons is a characteristic of strongly-coupled networks, but not a general outcome of recurrent networks.
The training targets were the trial-averaged peri-stimulus time histogram (PSTH) of pyramidal neurons recorded in ALM during the delay period (Fig. 2B, bottom). We trained a subset of excitatory neurons in the model network to reproduce the target activity patterns (Fig. 2C). Each trained neuron received recurrent plastic synapses from randomly selected excitatory and inhibitory neurons in the network and feedforward plastic synapses from external neurons, which accounted for the potential inputs from outside the ALM. By modifying the plastic synapses, the trained neurons reproduced two PSTHs, corresponding to lick-right and lick-left conditions, when evoked by two different stimuli. The rest of the excitatory, as well as all of the inhibitory neurons in the network, were not trained (Fig. 2C).
After training, the firing rate of trained excitatory neurons successfully reproduced the PSTHs of pyramidal neurons (Fig. 2B, top; Fig. S2A,B), even though the plastic synaptic inputs were substantially weaker than the excitatory and inhibitory inputs from the static synapses (Fig. S2C, right). We estimated the correlations between single neuron PSTHs in the model and in the data (Fig. S2C, left), as well as the similarity in their population activity (Fig. 2E, left) to asses the success of the training. For the latter, we performed Principal Component Analysis (PCA) on the PSTHs of neurons, which is a dimensionality reduction technique used for identifying a set of activity patterns that captures a large fraction of variance in the population activity. The principal components (PCs) of the PSTHs of the trained excitatory neurons closely matched the PCs of the pyramidal neurons. Moreover, the first six PCs explained close to 80% of the trained neurons’ activity, thus the recurrent network displayed low-dimensional dynamics as in the pyramidal neurons in ALM (Fig. 2E, right) [39].
Next, we examined the activity of the untrained neurons. Similarly to the trained excitatory neurons, their activity tended to ramp before go-cue and was choice-selective (Fig. 2D, top). The PCs of their PSTHs were almost identical to the trained excitatory neurons (Fig. 2F, right; Fig. S3E). This finding showed that cortical-like activity generated within the subset of excitatory neurons spread to the the rest of the network without additional synaptic reorganization to the untrained neurons.
Finally, we found that the PCs of the PSTHs of the fast-spiking ALM neurons, whose activity was not learned by the network, were almost identical to the PCs of the untrained inhibitory model neurons (Fig. 2F, right). For the lick-left trial, the higher PC modes (i.e., PC4 and PC5) of the inhibitory model neurons and fast-spiking ALM neurons (Fig. S3E, left) both mildly deviated from the corresponding PCs of trained neurons (Fig. S3C, left), but still closely resembled each other. A good agreement between the untrained model neurons and the held-out neural data supported the hypothesis that cortical-like activity learned within a subset of neurons can spread and produce cortical-like activity in the entire network. This could explain why the activity of putative fast-spiking neurons in ALM is heterogeneous, yet is very similar to the activity of putative pyramidal neurons [21].
Similarity in the neural activity of untrained model neurons and ALM neurons
To further investigate the similarities between the activities of the untrained inhibitory neurons in the trained network and the fast-spiking ALM neurons, we compared their PSTHs at the single neuron and population levels.
At the single neuron level, we identified an untrained inhibitory neuron that best matched each fast-spiking ALM neuron, based on the mean-squared-error of the PSTHs of all possible pairings between the ALM neuron and the population of inhibitory model neurons. Figure 3A shows the PSTHs of several matched pairs and their correlations for the lick-right and lick-left trials (see Fig. S4 for all the matched pairs). Evaluating the correlations of all the matched pairs showed that they were significantly higher than the spurious correlations obtained by matching the fast-spiking ALM neurons to inhibitory neurons in an untrained balanced network (Fig. 3B, left; two sample Kolmogorov-Smirnov tests; p-value < 0.0001).
To elucidate which aspects of the fast-spiking ALM activity were captured by the untrained inhibitory neurons in the trained network, we examined if certain activity patterns of the fast-spiking ALM neurons were indicative of the goodness-of-fit to the model neurons. We found that the projection of the PSTHs of the fast-spiking ALM neurons onto their first PC, a ramping mode that captured over 70% of the variance in the ALM activity (see PC1 in Fig. 2F), was a good indicator for how well the untrained model neurons could fit the fast-spiking ALM neurons (Fig. 3B, right). This analysis suggested that the ramping mode was the dominant component of the trained activity that was transferred to the untrained inhibitory neurons and shared with the fast-spiking ALM neurons.
We systematically examined the activity patterns shared by the populations of untrained inhibitory neurons and fast-spiking ALM neurons, by analyzing the shared-variance between the two population activities. The shared-variance analysis identified population vectors along which two population activities co-varied maximally and yielded population-averaged activity along those directions (shared components) and the proportion of variance explained by the shared components (shared variance; see [40] and Methods for details). The shared components (SCs) were similar to the PCs of the untrained inhibitory subnetwork and fast-spiking ALM activities (compare the SCs in Fig. 3C to the PCs in Fig. 2F), and the first four components captured most of the shared variance (Fig. 3C). In particular, consistent with the single neuron analysis shown in Fig. 3B, the first shared component was a ramping mode (SC1 in Fig. 3C).
In addition to the spiking activity patterns, we asked if functional properties, such as choice selectivity, were transferred to the untrained neurons in the trained network. It has been shown that pyramidal ALM neurons in mice display selectivity to the animal’s choice [21, 39, 41]. As expected, the excitatory model neurons, trained to reproduce the activity of pyramidal ALM neurons, also displayed choice selectivity (Fig. S5). Interestingly, we also found that the fast-spiking ALM neurons in the neural data were choice selective (Fig. 3D,E; see also Supplementary Fig. 2 in [21]). These observations led us to examine if the untrained inhibitory neurons in the trained network exhibited choice selectivity, as in the fast-spiking ALM neurons.
To this end, we analyzed the difference of the PSTHs to two trial types (lick-right versus lickleft) in all the untrained inhibitory neurons and found that they displayed choice selectivity (Fig. 3D). Moreover, the distribution of the choice selectivity of fast-spiking ALM neurons and untrained inhibitory neurons were in good agreement (Fig. 3E). This finding shows that not only the trained neural activity can propagate throughout the network, but the choice selectivity emerged in a subset of neurons can spread to the untrained parts of the network as well. In particular, this suggests an alternative mechanism for how selectivity may emerge in inhibitory neurons. In contrast to previous models that required specific connectivity between excitatory-inhibitory neurons for selective responses to emerge [42, 43], our model suggests that choice selectivity in inhibitory neurons can arise in strongly coupled networks even when the connections to the inhibitory neurons are non-specific.
Spreading of trained neural activity improves if the inhibitory subnetwork is trained
So far, we showed that the cortical-like activity originating from the excitatory neurons can spread to the untrained inhibitory neurons. Next, we asked how the spreading of trained activity may depend on the type of neurons being trained. To address this question, we considered two training scenarios where either the excitatory or the inhibitory subnetwork (but not both) was trained to generate the target activity patterns (Fig. 4A, right).
The number of fast-spiking ALM neurons recorded from the mice (Nfs = 306) was, however, too small to train the inhibitory neurons in large-scale spiking neural networks. We thus developed a method to generate synthetic neural activity that had similar low-dimensional dynamics as the ALM neurons. Briefly, we first performed principal component analysis on the PSTHs of ALM neurons to obtain the PCs (Fig. 2E, F) and the empirical distribution of each PC’s loading onto the neurons. To construct a synthetic target activity for neuron i, we sampled (1) a baseline rate ri from the firing rate distribution and (2) each PC’s loading on the neuron from the empirical distribution, conditioned on the rate ri (see Fig. S6 and Methods for details). Applying this method to the lick-left and lick-right trial types and to pyramidal and fast-spiking neuronal types, we were able to generate unlimited number of cortical-like PSTHs needed for training large-scale networks consisting of, e.g., N = 30, 000 neurons. In particular, these synthetic neural activity had statistically identical low-dimensional dynamics as the ALM neurons (Fig. S6E)
Using the synthetic neural activity as the target activity patterns, we performed the two training scenarios where we trained a subset of neurons in the excitatory or the inhibitory subnetwork to reproduce the synthetic neural activity. Following training, we compared the spiking activities of the untrained neurons in the subnetworks that were not trained.
We first observed that the PCs of synthetic neural activity was transferred to the untrained neurons when a sufficient number of neurons were trained (Fig. 4D, right). Such transfer of PCs was similar to what we found in the untrained inhibitory neurons when the excitatory neurons were trained on the activity of ALM pyramidal neurons (Fig. 2E,F). Based on the transfer of PCs and the low dimensionality of ALM activity, we used the variance explained by the first six PCs of the PSTHs of the untrained neurons to quantify the transferred cortical-like activity. In the trained neurons, the first six PCs explained 80% of the activity, regardless of the trained neuronal type (E or I) or the fraction of trained neurons (Fig. S7A). On the other hand, the cortical-like activity transferred to the untrained neurons gradually increased with the fraction of trained neurons. Moreover, the transferred activity was stronger by 20% when the inhibitory subnetwork was trained, compared to when the excitatory subnetwork was trained (Fig. 4A, left).
To understand what allowed the activity patterns of inhibitory neurons to spread better to the untrained neurons, we examined the differences in the spiking activities of the pyramidal and fast-spiking ALM neurons. The mean firing rate of each neuron was subtracted from its PSTH to remove the differences in the baseline firing rates of the pyramidal and fast-spiking ALM neurons (Fig. 4B, right). The principal component analysis of the centered PSTHs revealed that the strength of every PC was stronger in the fast-spiking neurons than in the pyramidal neurons, when the loadings on each PC were averaged over the population of neurons (Fig. 4B, left). This analysis showed that the modulation of firing rate around the mean rate was larger in the fast-spiking neurons, raising the possibility that stronger rate modulation leads to stronger activity transfer.
To test if stronger modulations in the trained activity patterns would increase the transferred activity to the untrained neurons, we adjusted the modulation strength of the synthetic inhibitory activity and trained a fixed subset of inhibitory neurons to generate target activity patterns with different levels of rate modulations. We found that, in the untrained excitatory neurons, the variance explained by the cortical-like activity increased monotonically with the modulation strength of the trained inhibitory neurons. (Fig. 4C). These results suggested that the stronger rate modulations in the fast-spiking ALM neurons enabled the trained inhibitory neurons in the model to spread their activity patterns to the untrained neurons more effectively. It also suggested that inhibitory neurons, whose baseline spiking rates are typically higher than the excitatory neurons in cortex (e.g., mean firing rates of ALM pyramidal and fast-spiking neurons were ~ 4Hz and ~ 11Hz, respectively, in our data), can support stronger rate modulations and potentially play a more significant role in spreading the trained activity patterns.
The finding that activity patterns with strong rate modulation spread better was also observed across the PCs. The lower PC modes of the ALM spiking activity showed stronger modulation than the higher PC modes, as expected, since the lower PC modes capture more variance (Fig. 4B). To quantify how well the trained PCs transferred to the untrained neurons, we computed the correlation between the PC modes of the trained and transferred activity (Fig. 4D, left). The lower PC modes (PC1 to PC3) transferred with high fidelity even when only 20% of the neurons were trained. On the other hand, the transfer of the higher PC modes (PC4 to PC6) improved gradually when the fraction of trained neurons increased. This result suggested that the lower PC modes, due to their strong modulations, can spread more robustly to the rest of the neurons, promoting low-dimensional neural dynamics across a strongly coupled network.
Taken together, our results demonstrate that trained activity patterns with stronger rate modulations, which can emerge from the fast-spiking ALM neurons or lower PC modes, have greater influence on the untrained neurons in the network.
Network mechanism for distributing trained neural activity in strongly coupled networks
In a recurrent neural network, in which neurons are highly inter-connected, it may seem obvious that task-related activity can spread from one part of the network to another part through the connections that are not optimized for the task. However, this intuition becomes less clear when the activity of a neuron is determined by integrating a large number of heterogeneous presynaptic activities, as considered in our network model and is the case in the cortical network.
A close examination of networks with a large number of connections reveals that whether the task-related activity can spread depends on the operating regime of the network. If the synaptic connections to an untrained neuron randomly sample and sum a large number of heterogeneous activity patterns, one could expect that the activity patterns will be averaged-out at the level of the synaptic input. Then, the untrained neuron will not display any activity patterns, as shown in Fig. S9. In this section, we give an intuitive explanation that this is not the case if the network is strongly coupled and operates in the balanced regime (Fig. 5). In this regime, heterogeneities of presynaptic activities can be preserved in the synaptic inputs to untrained neurons because of the strong synapses and then manifested in the post-synaptic rate of the untrained neurons due to the dynamic cancellation of the large, unmodulated components of the excitatory and inhibitory inputs. A detailed explanation, together with a mathematical analysis, is given in the Methods.
To explain the network mechanism underlying the spreading of trained activity patterns to the untrained neurons in the balanced regime, we considered a training setup where all the excitatory neurons were trained, while the inhibitory neurons were not. We chose the target activity to be 2Hz sine functions with random phases. After training, the temporal modulation of synaptic inputs to the excitatory neurons followed the target activity patterns (Fig. 5A, top). As a result, the first two PCs of the trained activities were 2Hz sine and cosine functions and were the dominant PCs of the trained activities (Fig. 5A, bottom).
The synaptic connections to an untrained neuron consisted of only the static synapses from randomly selected trained and untrained presynaptic neurons. Due to the large number of static synapses and their strong weights, the mean excitatory (Fig. 5C, ) and inhibitory (Fig. 5D, ) inputs to the untrained neuron were much larger, in absolute value, than their temporal modulations around the mean inputs (Fig. 5C, ; Fig. 5D, ). However, in the balanced regime, the large mean excitatory and inhibitory inputs dynamically canceled each other, resulting in the net mean input to the untrained neuron being around the spike-threshold (Fig. 5E, ut). Then, the spiking pattern of the untrained neuron was determined by the temporal modulation of the synaptic input around the net mean input, i.e., the sum of and .
To better understand how the trained activity patterns spread, we further examined the excitatory and inhibitory components of the temporal modulation of the synaptic input. Analysis of the input from trained excitatory neurons (Fig. 5C, ) showed that its temporal modulation was dominated by the same PCs the excitatory neurons were trained to generate (Fig. 5A). The modulated input from trained excitatory neurons then led the total input to the untrained neuron to be modulated as well (Fig. 5E, ut). As a result, the untrained neurons produced modulated activity (Fig. 5B), which provided modulated input to neurons in the network (Fig. 5D, ). The temporal modulations of the total input to the untrained neurons (Fig. 5E, bottom) and the inputs the untrained neurons provided to other neurons (Fig. 5D, right) were both dominated by the same PCs acquired from training.
One of the predictions of this network mechanism is that the loadings of the PCs, at the level of the synaptic inputs to untrained neurons, are normally distributed. This is because the summation of a large number of randomly distributed PC loadings, determining the trained activity patterns, results in a Gaussian distribution when the synaptic weights are strong (see Methods). Indeed, this was the case for the loadings of the first two PCs in the example network trained on sine functions (Fig. 5G). Then we analyzed the loadings of the dominant PC mode in the ALM data, which were the slopes of the ramping activity of the synaptic inputs. Since the synaptic inputs to ALM neurons were not available, we estimated them by finding inputs to the transfer function of the model neuron that yielded the observed firing rates of ALM neurons. We found that the statistics of these loadings were also well fitted by a Gaussian distribution, supporting that the proposed mechanism may be biologically plausible (Fig. 5H).
The same network mechanism also provides an explanation for how functional properties, such as choice selectivity, can spread from neurons trained to be choice-selective to other neurons that are not trained (Fig. 3E). It stems from the fact that the differences in the activity of the lick-left and lick-right trials in the trained neurons spread through the random connections and are realized into two different responses in the untrained neurons, producing choice selectivity in them (see Methods for details). In addition, our mathematical analysis of the network mechanism is consistent with the findings that, due to their strong temporal modulations, inhibitory activity patterns spread more effectively than the excitatory activity patterns (Fig. 4A), and lower PC modes transfer with better fidelity than the higher PC modes (Fig. 4D; see Methods).
The results of our analysis show that the spreading of trained activity to untrained neurons is a general and robust circuit mechanism for strongly-coupled networks operating in the balanced regime. It is independent of the number of presynaptic inputs per neuron. Moreover, the slopes of ramping activity in the ALM neurons displayed statistics that agreed with the model prediction, providing an evidence for the biological plausibility of the proposed mechanism.
Perturbation responses suggest that the ALM network operates in the balanced regime
We showed that when a subset of neurons was trained to reproduce the ALM activity, the task-related activity spread to the untrained neurons, which then also generated spiking activity resembling the ALM data (Figs. 2, 3). Such spreading of activity from trained to untrained neurons is a general mechanism for spreading activity in strongly coupled spiking networks (Figs. 4, 5). These results suggested that the observed task-related activity was produced in the ALM while it operated in the balanced regime. Here we used optogenetic perturbations to test if ALM activity displayed the characteristics of strongly coupled networks.
Specifically, we considered the activity modes of population of neurons responding to perturbations applied during the delay period (Fig. 6A, B). In strongly coupled networks consisting of excitatory and inhibitory populations, the projection of the population activity on the homogeneous mode (i.e. the average firing rate of the excitatory or inhibitory populations, Fig. 6A) is expected to recover rather fast from any perturbation. This is because the network dynamics of strongly coupled networks are extremely stable along the homogeneous mode [23]. To understand this phenomenon, one should consider changes to the average firing rate of the excitatory population in the network. This will result in a strong change (on the order of square root the number of inputs per neuron) to the excitatory drive to each of the neurons, which unless immediately suppressed by a strong inhibitory current, will destabilize the network. Therefore, to maintain the stability of network dynamics in strongly-coupled networks, a perturbation to the homogeneous mode is expected to decay quickly to its pre-perturbed value due to the strong and fast inhibition (a phenomenon known as ‘fast tracking’ [23, 44]).
We verified this prediction by analyzing the responses of the motor cortex to optogenetic perturbations, and compared them with the responses of the trained strongly coupled network to similar perturbations. Consistent with our prediction, following a perturbation to the activity of neurons in the strongly coupled network (Fig.2), the projection on the homogeneous mode quickly returned to the baseline (Fig. 6D, black). In contrast, the projection of the activity on the choice mode (Fig. 6B), a mode that maximally separates trial-averaged activity with respect to licking directions (see [21, 41] and Methods), returned to the baseline after the perturbation with a significantly longer recovery time (Fig. 6D, blue; Fig. 6F, paired Student t-test, p-value=0.016). The slow recovery suggested that a dynamic attractor, which formed around the target trajectory due to training, was able to retract the perturbed activity at a slow timescale along the coding mode [21, 45]. Importantly, the network was trained only on the unperturbed ALM activity. Therefore, the fast and slow responses to perturbations were not dynamical properties acquired directly from the perturbed ALM activity, but instead they emerged from the strongly coupled network, when it was trained just on the unperturbed ALM activity.
To test the model prediction and verify that the ALM network operated in a similar dynamical regime, we conducted the same analysis on single sessions of simultaneously recorded ALM neurons (Fig.6G-J). We found that the response time of the homogeneous mode in ALM was significantly faster than that of the choice mode (Fig. 6H-J, paired Student t-test, p-value=0.025). Thus, the fast recovery of the homogeneous mode of ALM network, relative to the slow recovery of the choice mode, to optogenetic perturbations suggests that the ALM network operated in the same dynamical regime as the strongly coupled network. These findings suggest that the ALM network has the potential to be endowed with a network level mechanisms for generating widespread task-related activity, with limited synaptic reorganization on only a subset of neurons during learning.
Discussion
In this study, we presented a circuit mechanism for distributing task-related activity in cortical networks. We have shown that neural activity learned by a subset of neurons can spread to the untrained parts of the network through pre-existing random connectivity, without additional training. This spread of activity occurs as long as the pre-existing random connections are strong and the network operates in the balanced regime. When a subset of neurons in the spiking network was trained to reproduce the activity of ALM neurons, the activity of untrained neurons in the network also displayed surprising similarity to the activity patterns of neurons in ALM. Single neuron activity patterns of the untrained neurons were ramping and selective to future choices, as was observed in ALM. Our work suggests that only a subset of neurons may be actively engaged in learning and the rest of the neurons are driven by the structured activity generated from the trained neurons.
Accumulating evidence show that inhibition in cortex is highly plastic (e.g. see review by [46]). We found that the fidelity of spreading the activity is higher when the inhibitory neurons are trained instead of the excitatory ones. For example, all of the excitatory neurons needs to be trained to explain 70% of the variance in the untrained inhibitory neurons, while training only 60% of the inhibitory neurons is enough to induce the same 70% variance in the untrained excitatory neurons (Fig. 4A). We speculate that this is a characteristic of the operating regime of cortical networks, in which typically the baseline spiking rates of inhibitory neurons is higher than the excitatory neurons. Inhibitory neurons can thus support stronger rate modulations (Fig. 4B), which in turn improves the fidelity of the spread (Fig. 4A, Fig. 5, Methods). Our results suggest that synaptic plasticity in inhibitory neurons can lead to wider spread of task-related activity in the motor cortex. Interestingly, this result is consistent with recent theoretical and computational studies showing that patterns of neural activity are primarily determined by inhibitory connectivity [47, 48].
In recent studies, the authors of [43, 49] argued that specific connectivity between excitatory and inhibitory neurons is necessary for choice selectivity to emerge in these two populations, based on computational models of their data. Our work suggests an alternative mechanism in which choice selectivity emerges in one population during training and spreads to the other population, without any reorganization of specific connections from the trained to the untrained populations. The network mechanism that spreads the task-related activity through random connectivity, as in our trained networks, is based on the susceptibility of neurons to modulations of synaptic inputs in strongly coupled networks (Fig. 5). This is a similar mechanism that explains how, without training or functional structure, orientation-selective neurons can emerge in primary visual cortex with a ‘salt-and-pepper’ organization [50, 51, 52].
While we studied the spreading of activity in recurrent spiking networks, a similar phenomena might generalize to other network architectures. For example, recent studies suggested that information can propagate through random connections in deep feedforward artificial networks [53, 54]. This suggests that the spreading of structured activity through unstructured connectivity is not restricted to local recurrent networks, but is applicable to other neural architectures and may provide the substrate for spreading information between brain areas.
Our study is related to and complements previous studies that trained rate-based units to reproduce neural activity in recurrent networks [19, 20, 21, 22]. In contrast to our spiking networks, each unit in these rate-based networks was paired with a target neuron activity for training, such that they could not investigate the effects of trained neurons on untrained neurons as we did here. We note that [19] (Supp. Fig. 3) examined what happened to the trained neurons when untrained neurons were included, but did not analyze the activity of untrained neurons. Furthermore, our network model consisted of spiking EI neurons operating in the balanced regime, and thus exhibited large temporal irregularity of spikes and trial-to-trial fluctuations, which is a key characteristic of cortical neurons. These properties stayed intact following learning. This contrasts with the commonly used rate-based chaotic networks that do not obey Dale’s law [55] and that suppress the trial-to-trial fluctuations that emerge due to the strong recurrent connectivity after training [35].
While our training algorithm requires only an order of square root of the pre-existing static connections to be plastic, the specific number of plastic connections may vary with the complexity of the trained neural activity. Indeed, almost three times more plastic synapses were needed to train the neurons to reproduce ALM activity, in which six PCs explained about 80% of the variance, compared to the number of plastic synapses needed to train sine waves, in which two PCs explained the same amount of the variance (Table 2). It is beyond the scope of this paper to determine exactly how the prefactor of the square root term depends on the complexity of the neural activity. However, previous studies suggest that the number of plastic connections might depend on the dimensionality (i.e., decay rate of the singular values) [56] or decorrelation time of the trained neural activity [37].
Several recent studies considered training spiking networks with dynamically balanced excitation and inhibition. In [57] the authors had to break the EI balance in order to achieve non-linear computations. With our training procedure, individual neurons can be trained to perform complex tasks, such as generating the spiking activity of cortical neurons, without leaving the balanced regime. The work by [58, 59] trained all the recurrent weights of the dynamically balanced spiking networks. To maintain strong excitatory-inhibitory activities after training, they considered weight regularizations that constrained the trained weights close to strong initial EI weights. Instead, in our training setup, the strong initial EI connections were left unchanged throughout training, thus always provided the strong excitation and inhibition. The trained neurons additionally received sparse plastic synapses, that were sparser than the initial EI connectivity. Such training scheme, which modifies only sparse synapses to trained neurons, is consistent with recent findings that build functional neural networks through weak, sparse, or low-dimensional synaptic modifications [19, 35, 60, 61, 62, 63]. It is also consistent with recent experimental evidence that hints for sparse, but functionally biased, synaptic connectivity in cortex [34].
There is an ongoing debate if cortex operates in the balanced regime [64]. Experimental evidence that are inconsistent with the balanced regime hypothesis mainly relies on data from sensory cortices. Here we present evidence that the motor cortex operates in the balanced regime by analyzing the recovery of neurons in the motor cortex to optogenetic perturbations. The presence of two recovery time scales, i.e., fast for the homogeneous mode and slow for the choice mode, is consistent with the prediction of the balance state on the ability of the homogeneous mode to rapidly track inputs, a phenomenon termed ‘fast tracking’ [23, 44]. Our analysis is different from the paradoxical effect observed in excitatory-inhibitory networks, where strong recurrent excitation must be compensated by strong feedback inhibition to maintain a stable network state [52, 65, 66, 67, 68].
We note that, with the use of moderate plastic inputs in our trained network, the plastic input was on the order of the spike-threshold. The network can thus implement non-linear computations at the individual neuron level. It can also support non-linear computations at the population level, as long as the computation is held by subpopulations, such that the overall excitatory and inhibitory population rates are unchanged after training [58, 62, 69]. Thus, the only mode that is strictly linear with the inputs to the network is the homogeneous mode. This is different from recent works that portrayed that the strict linear input-output relationship of balanced networks limits their computational power [57, 64].
To conclude, our work shows that while large changes in network dynamics can be observed during learning, attributing such changes to synaptic reorganization between neurons must be taken with care. In networks that operate in the balanced regime, in which motor cortex might operate, widespread changes in neuronal activity can be mainly a result of distributing learned activity from a dedicated subset of neurons to the rest of the network through strong but mostly unstructured connectivity.
Methods
Data analysis
Principal component analysis of population rate dynamics
To obtain the PSTHs of neurons in a trained network, we repeated the simulation of a trained network 400 times starting at random initial conditions and applied the same external stimulus to trigger the trained activity patterns. Subsequently, for each neuron, the spikes emitted over multiple trials were placed in 20ms time bins, which ranged over the Ttarget long training window, and averaged across trials to compute the instantaneous spike rates.
Given the PSTHs, , of a population of M neurons, we subtracted the mean rate of every neuron from its PSTH to remove differences in the baseline firing rates. In the following, we use the same notation ri to refer to the mean subtracted PSTH of neuron i.
We then performed principal component analysis on the population rate dynamics R = (r1,…, rM), which is an T × M matrix, to obtain the principal components and the principal values λ1,…, λT. This is equivalent to finding the eigenvectors and eigenvalues of the covariance matrix, RRT. The variance explained by the kth principal component was .
The same procedure was applied to all the principal component analyses performed in this study.
Shared variance analysis
We identified population vectors along which the population activities of inhibitory model neurons and fast-spiking ALM neurons co-varied maximally [40]. We also quantified the fraction of variance that can be explained by the projected population-averaged activities (Fig. 3).
We first computed the correlation Cij = corr(fi, gj), which an M1 × M2 matrix, between the PSTH’s of inhibitory model neurons , 1 ≤ i ≤ M1 and fast-spiking ALM neurons , 1 ≤ j ≤ M2 where M1 = 2500, M2 = 306 and T = 100. Then the singular-value decomposition C = UΣV of the correlation matrix was performed, where U is an M1 × M1 matrix and V is an M2 × M2 matrix, to obtain the left singular vectors U = (u1,…, uM1) with and the right singular vectors V = (v1,…, vM2) with .
To obtain the population-averaged activity along the singular vectors, the matrices of population rate, i.e., for the inhibitory model neurons and for the fast-spiking ALM neurons, were projected to the corresponding kth singular vectors uk and vk, respectively, to obtain the kth shared components, and . The variance explained by the kth shared component was defined as and , respectively.
Defining the choice and homogeneous modes
Trial-averaged spike rate of a neuron i, ri(t,k), were calculated for each trial, k, using 1ms bin size and were filtered with a 200ms boxcar filter.
We then analyzed the population dynamics of N simultaneously recorded neurons in a session. During each trial, the population activity of these neurons, r(t, k), drew a trajectory in the N-dimensional activity space. We identified the choice mode as N × 1 vector of trial-averaged spike rate differences of N neurons during trials with lick-right and lick-left outcomes, averaged within a 1sec window at the end of the delay epoch, before the go cue [21]: with the L2 norm, ║x║, and 〈x〉t,k which is averaging over trials and time. The term was introduced to ensure that the projection of the neural activity is independent of the number of recorded neurons and for consistency with the homogeneous mode below. Projections of the neural activity along the choice mode were:
Similarly, the projection over the homogeneous mode was given by , with 1 being a vector of ones.
If an individual neuron was not recorded during a particular trial, its weight in Eq.(1) was set to zero, and for the analysis we selected trials with at least 10 simultaneously recorded neurons.
Response of the modes to perturbations
To assess the impact of vS1 photostimulation during the delay on the homogeneous and choice modes in the ALM, we computed for each session the single-trial projections on each of the modes, PC(t, k) and PH(t, k), for correct lick-right trials both with and without the photostimulation. The trial-averaged activity was plotted for one example session in Fig. 6E,I along with the SD, after subtracting the average projection over the first 0.5 seconds of the delay period.
We used a statistical hypothesis test (Student t-test) to estimate the decay time back to the non-perturbed trajectories for the projections on the modes. Specifically, for each time bin we tested the null hypothesis that the perturbed and unperturbed trials were from the same distribution and rejected the null hypothesis with a p-value< 0.05 (orange dots in Fig. 6E,I). We only analyzed sessions in which the photostimulation resulted in a significant change in at least 10% of the time points during the photostimulation period ([-1.6,1.2]sec, 13/17 sessions). To calculate the decay time, we then used the last significant time bin within the time window of [-1.2, 0] sec for which the derivative was smaller than 10ms (dashed red lines in Fig. 6E,I). The perturbations in 2/13 sessions were biased and were not included in the analysis, leaving 11 sessions of simultaneously recorded neurons.
To calculate the decay time over all sessions (Fig. 6D) we averaged the projection in each of the 11 analyzed sessions and calculated the difference in the projection between the perturbed and unperturbed trials (Δ projection). We then took the absolute value and averaged over all sessions (Fig. 6D, mean± SEM). Finally, we estimated the decay rate by an exponential fit.
Spiking neural networks
Network connectivity
The spiking neural network consisted of randomly connected NE excitatory and NI inhibitory neurons. The recurrent synapses consisted of static weights J that remained constant throughout training and plastic weights W that were modified by the training algorithm. The static synapses connected neuron j in population β to neuron i in population α with probability pαβ = Kαβ/Nβ and synaptic weight , where Kαβ is the average number of static connections from population β to α:
The strength of plastic synapses, , was of the same order as the static weights. However, the plastic synapses connected neurons with a smaller probability: which made the plastic synapses much sparser than the static synapses [62]. Here, c is an order 1 parameter that depends on training setup.
The static and plastic connections were non-overlapping in that any two neurons in the network can have only one type of synapse.
Keeping them disjoint allowed us to maintain the initial network dynamics generated by the static synapses and, subsequently, introduce trained activity to the initial dynamics by modifying the plastic synapses.
The static recurrent synapses were strong in that the coupling strength between two connected neurons scaled as , while the average number of synaptic inputs scaled as Kαβ. This is in contrast to the weak, 1/Kαβ, coupling we considered in Fig. S9. As a result of this strong scaling, the excitatory and inhibitory synaptic inputs to a neuron from static synapses increased as , thus were much larger than the spike-threshold for a large Kαβ. However, and were dynamically canceled, and the sum (ubal) was balanced to be around the spike-threshold ([23], Fig. 1B, middle).
In contrast to the static synapses, each trained neuron received only about plastic synapses. This made the plastic synapses much sparser than the sparse static EI connectivity (e.g., with K = 1000 static synapses, there are of the order of plastic synapses per neuron). Consequently, the EI plastic inputs of the initial network were independent of Kαβ and substantially weaker than the EI balanced inputs for a large Kαβ. After training the plastic synapses, the total synaptic input (u = ubal + uplas) to each trained neuron was able to follow the target patterns (Fig. 1B, left; Fig. 1C), while the plastic input (uplas) stayed around the spike-threshold (Fig. 1B, right). With this scaling of plastic synapses, training was robust to variations in the number of synaptic connections, Kαβ. Network trainings were successful even when Kαβ was increased, such that the excitatory and inhibitory balanced inputs were tens of orders of magnitude larger than the plastic inputs (Fig. S1).
From a technical point of view, the choice to train only a very sparse number of plastic synapses made the plastic inputs to be on the order of the spike-threshold (i.e. order one, independent of the number of connections). Therefore, they could not affect the mean firing rates of the excitatory and inhibitory neurons in the network. These were fully determined by a linear equation, termed the ‘balanced equation’ [23, 70], that only involves the strength of the static connections, , and the external inputs. Alternatively, if the plastic synapses were more abundant in the network, on the order of the number of static connections, they could interfere with the the ability of the strong inhibition to balance the strong excitation for each neuron in the network. Such interference significantly limits the ability to train the spiking networks. Taking the number of plastic connections to be on the order of thus allowed to both train the networks, and keep it in the balanced regime.
Network dynamics
We used integrate-and-fire neuron to model the membrane potential dynamics of the i’th neuron: where a spike is emitted and the membrane potential is reset to vreset when the membrane potential crosses the spike-threshold vthr.
Here, is the total synaptic input to neuron i in population α that can be divided into static and plastic inputs incoming through the static and plastic synapses, respectively:
is the total external input that can be divided into constant external input, plastic external input, and the stimulus:
is a constant input associated with the initial balanced network. It scales with the number of connections, i.e., proportional to , determines the firing rate of the initial network and stays unchanged [23]. is plastic input provided to trained neurons in the recurrent network from external neurons that emit stochastic spikes with pre-determined rate patterns. The synaptic weights from the external neurons to the trained neurons were modified by the training algorithm. is the pre-determined stimulus, generated independently from the Ornstein-Ulenbeck process for each neuron, and injected to all neurons in the network to trigger the learned responses in the trained neurons.
The synaptic activity was modeled by instantaneous jump of the synaptic input due to presynaptic neuron’s spike, followed by exponential decay. Since the static and plastic synapses did not overlap, we separated the total synaptic input into static and plastic components as mentioned above:
Alternatively, the synaptic activity can be expressed as a weighted sum of filtered spike trains because the synaptic variable equations (Eq. 2) are linear in J and W: where describe the dynamics of synaptically filtered spike trains.
Each external neuron emitted spikes stochastically at a pre-defined rate that changed over time. The rate patterns, followed by the external neurons, were randomly generated from an Ornstein-Ulenbeck process with mean rate of 5 Hz. The synaptically filtered external spikes were weighted by plastic synapses WX and injected to trained neurons: where
Similarly, the external stimulus Xstim,i applied to each neuron i in the network to trigger the learned response is generated independently from the Ornstein-Ulenbeck process.
In the following section, we will use the linearity of W, WX in Eqs. 3 and 4 to derive the training algorithm that modifies plastic synaptic weights.
Network training scheme
Overview
Prior to training the network, neurons were connected by the recurrent static synapses and emitted spikes asynchronously at constant rates. This asynchronous state of the initial network has been investigated extensively in previous studies [23, 25, 70].
Starting from this asynchronous state, the goal of training was to produce structured spiking rate patterns in a subset of neurons selected from the network. Specifically, our training scheme modified the recurrent and external plastic synapses projecting to the selected neurons, so that they generated target activity patterns when evoked by a brief external stimulus. To this end, we first selected M neurons to be trained from a network consisting of N neurons, and then prepared M target functions f1(t),…, fM(t) defined on a time interval t ∈ [0, Ttarget] to be learned by the selected neurons. The plastic synapses projecting to each selected neuron i were then modified by the training algorithm such that the total synaptic input ui(t) to neuron i followed the target pattern fi(t) on the time interval t ∈ [0, Ttarget] after the training.
Initialization of plastic synapses
For each trained neuron, we randomly selected L excitatory and L inhibitory presynaptic neurons that projected plastic synapses to the trained neuron. When the excitatory subpopulation was trained, the presynaptic excitatory neurons were sampled from other trained excitatory neurons while the presynaptic inhibitory neurons were sampled from the entire inhibitory population. Similarly, when the inhibitory subpopulation was trained, the presynaptic inhibitory neurons were sampled from other trained inhibitory neurons while the presynaptic excitatory neurons were sampled from the entire excitatory population. The untrained neurons did not receive any plastic synapses.
Each trained neuron also received inputs from all the LX external neurons. The plastic weights from the external neurons to each trained neuron were trained by the learning algorithm.
Cost function
Each trained neuron i had its own private cost function defined by where is a vector of recurrent plastic synapses to neuron i from other presynaptic neurons in the network indexed by i1,…, iL. Similarly, is vector of plastic synapses to neuron i from the external neurons. The regularization of plastic weights consisted of two terms
The first term is a ridge regression that evaluates the L2-norm of the plastic weights. The second term is called ROWSUM regularization where the elements of the vector are defined to be if the presynaptic neuron ik belongs to population α and 0 otherwise [59]. The inner products and are the aggregate plastic weights to neuron i from the excitatory and inhibitory populations, respectively. Including the ROWSUM regularization allowed us to keep the aggregate excitatory and inhibitory plastic weights fixed throughout the training.
Training algorithm
We derived a synaptic update rule that modified the plastic synapses to learn the target activities. The learning rule was based on recursive least squares algorithm (RLS) that was previously applied to train the read-outs to perform tasks [35, 36] and the individual neurons to generate target activity patterns [19, 37, 45]. The derivation presented here closely follows previous papers [37, 59]. For notational simplicity, we dropped the index i in wi and other variables, e.g., fi, ui. We note that the same synaptic update rule was applied to all the trained neurons.
The gradient of the cost function with respect to the vector of full plastic weights w = (wrec, wX) was
Here we substituted the expressions uplas,t = wrec · rplas,t and in the first line to evaluate the gradient with respect to w. In the second line, we used a condensed expression to denote the synaptically filtered spike trains from all plastic inputs. The vectors 1α apply only to the recurrent plastic weights wrec and take zero elements on wX.
To derive the synaptic update rule, we computed the gradient at two consecutive time points and
Subtracting Eqs (5) and (6) yielded where with the initial value
To update Pn iteratively, we used the Woodbury matrix identity where A is invertible and N × N, U is N × T, C is invertible and T × T and V is T × N matrices. Then Pn can be calculated iteratively
External stimulus triggering target activity patterns
To trigger the target activity patterns learned by the trained neurons, a brief external stimulus (200ms long) was applied to every neuron in the network immediately before generating the activity patterns. Two different sets of stimuli were prepared to trigger the lick-left and lick-right population responses. One set of stimulus was used during and after training to trigger the lick-left response and the other set of stimulus was used for the lick-right response. The stimulus to each neuron i and trial type c = L,R was generated independently from the Ornstein-Ulenbeck process: where τ = 20ms, σ = 0.2 and ξ(t) was uncorrelated Gaussian distribution with zero mean and unit variance.
Generating sinusoidal activity patterns
For demonstrating the Subset Training method (Fig. 1) and the network mechanism for spreading trained activity (Fig. 5), neurons in the network were trained to follow sine functions with random phases. Specifically, neuron i in the network learned the target pattern fi(t) = asin(ωt + ϕi) + bi on the time interval t = [0,1]sec, where the amplitude a = 0.5, the frequency ω = 1rad/sec (Fig. 1) and 2rad/sec (Fig. 5), the phase ϕi was sampled from a uniform distribution [0, 2π], and the offset bi was the mean synaptic input to the neuron in the initial balanced network prior to training.
Generating target neural trajectories
A subset of excitatory neurons in the network learned to reproduce the PSTHs of pyramidal neurons recorded from ALM in [21]. For each pyramidal neuron, the spikes emitted across multiple experiment trials were placed in Δt = 20ms time bins that ranged over the Ttarget = 2 second delay period. The PSTHs were then smoothed by a moving average over a 300ms time window centered at each time bin. We obtained two sets of PSTHs and from M = 1824 pyramidal neurons for the lick-left and lick-right trial types. Each PSTH for neuron i and trial-type c ∈ L,R was an T = Ttarget/Δt = 100 dimensional vector defined on time points t = [−2 + Δt,…, −Δt, 0]sec, where 0 is the onset of go-cue.
Next, we converted the PSTHs to target synaptic activities to be used for training the synaptic inputs to selected neurons. For each spike rate where i = 1,…, M, c = L, R and t = –2 + Δt,…, 0, we obtained the mean synaptic input that needs to be applied to the the leaky integrate-and-fire neuron to generate the desired spike rate. To this end, we numerically solved the nonlinear rate equation where is the transfer function of the leaky integrate-and-fire neuron given mean input, m, and variance of the input, σ2 [27, 71]. We obtained the synaptic fluctuation σ from the synaptic noise in the neurons of the initial network since the slow plastic inputs did not significantly change the fast noise fluctuation. This conversion yielded two sets of target synaptic inputs and for M excitatory neurons to be trained.
We chose the parameters of the initial network connectivity such that the mean rate of the excitatory and inhibitory populations in the network was close to estimated mean rates of the ALM data (mean excitatory rate was 4.2 Hz and inhibitory rate was 11.0 Hz). To select the subset of excitatory neurons to be trained, we compared the mean firing rates of the neurons in the initial network with the firing rates of pyramidal neurons and identified the excitatory neuron whose firing rate’s was closest to the pyramidal neuron. This process was repeated until all the pyramidal neurons were matched to the excitatory neurons uniquely.
Generating target synthetic trajectories
To generate synthetic data that shared similar statistics and low-dimensional dynamics as the neural data, we performed PCA on the PSTHs of pyramidal ALM neurons to identify the principal components that explained majority of their variance. We found that D = 9 was large enough to explain over 95% of the variance. The same procedure was applied to the PSTHs of the fast-spiking ALM neurons to obtain their principal components.
We sought to construct synthetic trajectories that resembled the PSTHs of the pyramidal and fast-spiking ALM neurons (Fig. S6). To this end, we expressed the synthetic trajectory rsynth as a weighted sum of the principal components: . To find the distribution of the coefficients of the neural data, we projected the PSTHs of pyramidal neurons onto the PCs and obtained the empirical distribution of . Bootstrapping the synthetic coefficients from the empirical distribution of was performed in two steps. First, the mean firing rate of synthetic target was sampled from the empirical rate distribution to generate synthetic PSTHs that had rate distribution statistically identical to the empirical distribution (Fig. S6A, B). Next, since depended strongly on the mean firing rate of neurons, was bootstrapped from a subset of whose underlying firing rate was close to the firing rate of synthetic target (Fig. S6C). In this way, the distributions of the firing rates and PC loadings of the synthetic and neural data were almost identical (Fig. S6E).
In addition, we generated the synthetic PSTHs in pairs for the lick right and lick left trials. First, the PSTHs for the lick right and lick left conditions were generated independently. Then, we sorted the PSTH’s of each condition separately and paired them, to ensure the pairs had similar level of mean firing rates. Subsequently, we added Gaussian noise with zero mean and standard deviation equal to the difference of lick right and lick left mean firing rates, to the PSTH’s of the lick left condition. This allowed us to introduce choice selectivity to the synthetic PSTHs.
The synthetic PSTHs were then converted into target synaptic inputs following the same procedure applied to the neural PSTHs.
Mathematical analysis of the effects of trained inputs on untrained neurons
In this part of the methods we use mathematical analysis to show how random inputs from trained neurons can drive the untrained neurons to follow the trained activity, without further training, if the network operates in the balanced regime.
To simplify the analysis, we assumed that only the excitatory population was trained and the inhibitory population was not. In addition, we assumed that the target functions, fit for neuron i and t ∈ [0, Ttarget], were slower than the slow plasticity signal and that training was perfect. In this case, we can approximate the total synaptic input to a trained excitatory neuron using the fixed point equation: with the strong external input associated with the balanced network [23]. The transfer function, , was the Riccardi function [27, 71], with . The population rate was given by , with 〈x〉 denoting the average over the time and [x] the average over the neurons.
Similarly, the total synaptic input to an untrained neuron, which lacked plastic connections, followed: with .
Our goal was to analyze the synaptic drive from the trained (excitatory) neurons to untrained (inhibitory) neurons to make specific predictions about what aspects of the trained inputs allowed them to spread effectively to the untrained neurons.
Statistics of random inputs from the trained neurons to an untrained neuron
If an excitatory neuron i is successfully trained, its firing rate closely follows the target activity fit. We used a shorthand notation and expressed the firing rate of the trained neuron in the form , with the temporal modulation . We next considered the singular value decomposition of the temporal modulation: which is N × T matrix, and where U is a N × N matrix of the left singular vectors and V is T × T matrix of the right singular vectors. Here, we considered a discretized version of time with T = Ttarget/Δt, such that the matrices are of finite size. The values are the singular values (SVs) and are the elements of the spectrum of the covariance matrix of the trained excitatory neurons. For instance, if we choose the target activity to be sinusoidal functions with random phases (Fig. 5A), the covariance matrix is stationary and the right singular vectors are the Fourier modes (e.g., V1t ∝ sin(ωt), V2t ∝ cos(ωt)).
Untrained (inhibitory) neurons do not receive plastic synapses. Thus, the aggregate input from the trained neurons to an untrained neuron, , is a random summation of trained neurons’ activity. It is given by: with the average population input . The second term in Eq.(12) is the quenched disorder [44] and its variance is given by:
The last term in Eq.(12) is the temporal modulation of the aggregate trained input, . Using Eq.(11), it is given by:
Here, due to the Central Limit Theorem, the coefficients are Gaussian vectors with zero mean and unit variance in the large K limit (see Prediction 1 below), if the left singular vectors Uin’s are random variables with zero mean and unit variance. Importantly, we emphasize that it is the strong coupling (i.e., synaptic weights scale as ) that allows the coefficients ãin’s to have finite variance. This is not the case if synaptic weights are weak (see Prediction 2 below). In addition, the variance of the coefficients of temporal modulation is , which shows that the SVs, , determine the strength of temporal modulation (see Prediction 3 below).
With this, the synaptic input to an untrained neuron from the trained population can be written in the following form: with being a Gaussian random variable with zero mean and unit variance.
For example, when the target functions are sinusoidal functions with random phases (Figs. 1, 5) these temporal modulations are: where we replaced ãni in Eq.(14) with the even and odd coefficients of the cosine and sine functions, ani,bni, respectively.
Similarly, in the case of the ALM data, the dominant right singular vector is a ramping mode (Fig. 2E,F), i.e. V1t ∝ t and the temporal modulations are dominated by: with .
The recurrent untrained inputs and implications
The synaptic input to an untrained inhibitory neuron consists of a large, , and positive mean drive from the excitatory neurons. The untrained neurons will thus fire with high rates and regular spiking, unless the network operates in the balanced regime, in which the recurrent inhibition cancels most of this large excitatory drive [23]. In this case, the untrained neuron will be driven by the temporal modulations originating from the random summation of the activity of trained neurons, which are of due to the strong coupling. This input is spanned by the principal components (or, equivalently, the right singular vectors) of the trained population according to Eq.(14).
A similar analysis on the recurrent inputs from the untrained inhibitory population, , needs to be done to infer the statistics of the temporal fluctuations of the net input, , and rates, , of the untrained inhibitory neurons. This analysis needs to be done in a self-consistent way to determine the statistics of [56]. While this analysis is beyond the scope of the current paper, several observations can be made already by examining the statistics of the inputs from the trained population.
No matter what the right singular vectors (which we refer to as the PCs in the main text) are, their coefficients are expected to be Gaussian. This prediction is shown in Fig. 5F for artificial target functions of sine functions with random phases, as well as in Fig. 5G for the coefficients of the dominant ramping mode in the neural data.
The spread of activity in the network is possible only because the variance of ãit’s in Eq.(14) is finite. It is a result of the strong coupling in the network, i.e. the scaling of the synapses, which guarantees, due to the Central Limit Theorem, that the variance of the aggregate input from the trained neurons converge is finite. This is in contrast to the case of weak synapses (e.g., scaling of instead of ), where the variance of ãit converges to zero in the large K limit (Fig. S9, no spreading of trained activity in a weakly coupled network).
The strength of the transfer of the trained activity to the untrained neurons depends on the variance of the trained population through Eq.(14). As shown in Fig. 4B, in the ALM data the variance of the temporal modulations of the inhibitory neurons is larger than those of the excitatory neurons. This result suggests why the fidelity of the spread improved when the inhibitory population was trained instead of the excitatory population. It also explains why lower mode PCs of the activity can spread better in the network, as their corresponding SVs ( in Eq.(14)) are, by definition, larger than those of the higher mode PCs.
This framework provides additional insights into how excitatory neurons trained to be choice-selective can impart the learned selectivity to the untrained inhibitory neurons through nonspecific, strong synaptic connections (see Fig. 3E). To show this, one can estimate the statistics of the difference in the input to an untrained inhibitory neuron from the trained population for the lick-right and lickleft trials. For instance, if we consider the target functions to be defined by the dominant ramping mode that captures over 70% of the variance (Fig. 2E), the relevant basis function would be V1t ∝ t for t ∈ [0, Ttarget], and the selectivity of the trained inputs (SIE) yields where A and B determine the variance in the baseline inputs and ramping rates, respectively. From Eq.(15), the quenched disorder yields Gaussian variables , with a finite variance A2. From Eq.(17), the temporal modulation yields a Gaussian variables , with a finite variance B2. Because Δzi and Δã1i are random variables with finite variances, the trained inputs develop choice selectivity, which can then elicit choice selectivity in the untrained inhibitory neurons (Fig. 3D). The good agreement of the distribution of choice-selectivity in the untrained neurons in the model and the putative fast-spiking neurons in the neural data (Fig. 3D) is consistent with this prediction.
Simulation parameters
Supplementary material
Acknowledgements
We would like to thank Larry Abbott and Sandro Romani for their valuable feedback. A.F., K.S. and R.D. were supported by the Howard Hughes Medical Institute. C.M.K. and C.C.C were supported by the Intramural Research Program at the NIDDK/NIH. C.M.K. would like to thank the Visiting Scientist Program at Janelia Research Campus for their support.
References
- [1].↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].↵
- [7].↵
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].↵
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].↵
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].↵
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].↵
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].↵
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].↵
- [57].↵
- [58].↵
- [59].↵
- [60].↵
- [61].↵
- [62].↵
- [63].↵
- [64].↵
- [65].↵
- [66].↵
- [67].↵
- [68].↵
- [69].↵
- [70].↵
- [71].↵