Abstract
The primary motor cortex has been shown to coordinate movement preparation and execution through computations in approximately orthogonal subspaces. The underlying network mechanisms, and in particular the roles played by external and recurrent connectivity, are central open questions that need to be answered to understand the neural substrates of motor control. We develop a recurrent neural network model that recapitulates the temporal evolution of single-unit activity recorded from M1 of a macaque monkey during an instructed delayed-reach task. We explore the hypothesis that the observed dynamics of neural covariation with the direction of motion emerges from a synaptic connectivity structure that depends on the preferred directions of neurons in both preparatory and movement-related epochs. We constrain the strength both of synaptic connectivity and of external input parameters by using the data as well as an external input minimization cost. Our analysis suggests that the observed patterns of covariance are shaped by external inputs that are tuned to neurons’ preferred directions during movement preparation, and they are dominated by strong direction-specific recurrent connectivity during movement execution, in agreement with recent experimental findings on the relationship between motor–cortical and motor–thalamic activity, both before and during movement execution. We also demonstrate that the manner in which single-neuron tuning properties rearrange over time can explain the level of orthogonality of preparatory and movement-related subspaces. We predict that the level of orthogonality is small enough to prevent premature movement initiation during movement preparation; however, it is not zero, which allows the network to encode a stable direction of motion at the population level without direction-specific external inputs during movement execution.
Introduction
The activity of the primary motor cortex (M1) during movement preparation and execution plays a key role in the control of voluntary limb movement [1, 2, 3, 4, 5, 6, 7]. Classic studies of motor preparation were performed in a delayed-reaching task setting, showing that firing rates correlate with task-relevant parameters during the delay period, despite no movement occurring [8, 9, 10, 11, 12, 13, 14, 14, 15, 16, 17]. More recent works have shown that preparatory activity is also displayed before non-delayed movements [18], that it is involved in reach correction [19], and that when multiple reaches are executed rapidly and continuously, each upcoming reach is prepared by the motor cortical activity while the current reach is in action [20]. Preparation and execution of different reaches are thought to be processed simultaneously without interference in the motor cortex through computation along orthogonal dimensions [20]. Indeed, the preparatory and movement-related subspaces identified by linear dimensionality reduction methods are almost orthogonal [21] so that simple linear readouts that transform motor cortical activity into movement commands will not produce premature movement during the planning stage [22]. However, response patterns in these two epochs of motion are nevertheless linked, as demonstrated by the fact that a linear transformation can explain the flow of activity from the preparatory subspace to the movement subspace [21]. How this population-level strategy is implemented at the circuit level is still under investigation [23]. A related open question [24] is whether inputs from areas upstream to the primary motor cortex (such as from the thalamus and other cortical regions, here referred to as external inputs) that have been shown to be necessary to sustain movement generation [25] are specific to the type of movement being generated throughout the whole course of the motor action, or if they serve to set the initial conditions for the dynamics of the motor cortical network to evolve as shaped by recurrent connections [26, 27, 28, 29].
Here, we use a network modeling approach to explain the relationship between network recurrent connectivity, external inputs, and computations in orthogonal dimensions. Our analysis is based on recordings from M1 of a macaque monkey performing a delayed centerout reach task. One easily measurable feature of motor cortical activity during straight limb reaches is its covariance with the direction of motion (see [30, 31, 32, 33] but also [34, 35]). Recorded neurons are tuned to the direction of motion both during movement preparation and execution, but their tuning properties change over time ([36, 37, 38]). Interestingly, major changes in single neuron tuning happen when the activity flows from the preparatory to the movement-related subspaces. Yet, the direction of motion that we can decode at the population level is stable throughout the course of the motor action. To model these observations, we considered a recurrent neural network of rate-based neurons and we reduced the network dynamics to a few latent variables, or order parameters, that recapitulate the temporal evolution of neurons tuning properties. We then fitted the model to the data, by imposing that the model reproduce the observed dynamics of the order parameters, and that the energy cost associated with large external inputs is minimized. Our analysis suggests that during the delay period, information encoding the direction of movement is inherited by the network through external inputs that are tuned to the preferred directions of the neurons. During movement execution, external inputs are untuned and information regarding the movement direction is maintained via strong direction-specific recurrent connections. We will discuss how this prediction is in line with recent findings [39] showing that the co-firing patterns of pairs of motor-thalamic and motor-cortical neurons significantly correlate with the difference in the neurons’ preferred directions before, but not during, movement execution. Finally, we show how the specific way in which neurons tuning properties rearrange over time produces the observed level of orthogonality between the preparatory- and movement-related subspaces.
Results
Subjects and Task
We analyzed multi-electrode recordings from the primary motor cortex (M1) of two macaque monkeys performing a previously reported [40] instructed-delay, center-out reaching task. The monkey’s arm was on a two-link exoskeletal robotic arm, so that the position of the monkey’s hand controlled the location of a cursor projected onto a horizontal screen. The task consisted of three periods (Fig. 1): a hold period, during which the monkey was trained to hold the cursor on a center target and wait 500 ms for the instruction cue; an instruction period, where the monkey was presented with one of eight evenly spaced peripheral targets and continued to hold at the center for an additional 1,000–1,500 ms; a movement period, signalled by a go cue, were the monkey initiated the reach to the peripheral target. Successful trials where the monkeys reached the target were rewarded with a juice or water reward. The peripheral target was present on the screen throughout the whole instruction and movement periods. In line with previous studies (e.g., [21]), the preparatory and movement-related epochs were defined as two 300ms time intervals beginning, respectively, 100ms after target onset and 50ms before the start of the movement.
Correlations and tuning properties of neural activity
As previously reported in [21], the correlation coefficient between the activity of pairs of neurons during movement preparation weakly correlates with the correlation coefficient of the same two neurons during movement execution (Fig. 1.b); moreover, subspaces of neural activity dedicated to movement-preparation and execution are close to being orthogonal (Fig. 1.c). We aim to understand how computations during movement preparation and execution are linked, while correlation patterns are fundamentally reorganized between epochs. To this end, we first analyzed how neurons tuning properties relate in the two epochs of motion. Studies of motor cortex have shown that tuning to movement direction is not a time-invariant property of motor cortical neurons, but rather varies in time throughout the course of the motor action; single-neuron encoding of entire movement trajectories has been also reported (e.g., [36, 37]). We measured temporal variations in neurons preferred direction by binning time into 290ms time bins and fitting the binned trial-averaged spike counts as a function of movement direction with a cosine function. The cumulative density function of circular variances of preferred directions (Fig. 2.e) shows that the variability in preferred direction during the delay period alone and during movement execution alone is smaller than the variability during the entire duration of the task. This justifies our choice to characterize neurons tuning properties only in terms of their preferred direction during movement preparation and their preferred direction during movement execution. In each epoch of motion, pairs of neurons have positively correlated activity if their preferred direction is similar, and negatively correlated activity if their preferred direction is opposite (Fig. 2.f). Conversely, correlations of preparatory activities depend less strongly on the preferred direction during movement execution, and vice versa (Fig. 2.f). We next built a network model whose architecture is shaped by neurons tuning properties.
Two-cylinder model
We considered a recurrent network model where two distinct but correlated maps, denoted by A and B, encode the direction of movement during movement preparation and during movement execution, respectively. We will refer to the units in the network as neurons, even though each unit in the model rather represents a group of M1 neurons with similar functional properties, and the connection between two units in our model represents the effective connection between the two functionally similar groups of neurons in M1. The model is an extension of the ring model: while previously studied ring models are defined on one [41] or multiple [42] one-dimensional circular spaces, we added another dimension to the feature space, representing the degree of participation of each neuron to encoding the circular variable. Neurons are identified by four coordinates: θA ∈ [0, 2π) and θB ∈ [0, 2π), representing their preferred direction during movement preparation and during movement execution, and ηA ∈ [0, 1] and ηB ∈ [0, 1], describing how strongly the neuron participates into encoding the direction of motion in the two epochs of movement. Maps A and B are thus two cylindrical maps defined by one angular coordinate θ and one linear coordinate η. Neurons receive input currents Iext both from outside of the network - either homogeneous or anisotropic in maps A and B – and recurrent inputs Irec, mediated by direction-specific cortical interactions. The strength of synaptic connections from a pre-synaptic neuron with preferred directions (θA, θB) and participation strengths (ηA, ηB) to a post-synaptic neuron with preferred directions and participation strengths is denoted by . Since the two maps encode two non-concurrent features of the motor action, we assume that the couplings encode the maps additively [43]. In analogy with previously studied ring models [41, 44, 42], the interactions are given by: j0 represents a uniform inhibitory term; and measure the amplitude of the sym-metric connections storing map A and map B, respectively; ja measures the amplitude of asymmetric connections from map A to map B. Later in this section (also, Fig. 3), we will describe how the network can encode the direction of movement through a localized pattern of activity. The last term of (1) contributes to align location of the bump in map B with the location of the bump in map A. In the mean-field limit of a very large network, the dynamics of the activity rate is defined by: where [ ]+ is the threshold-linear (a.k.a. relu) function. We set the time constant to τ = 25ms, which is of the same order of magnitude of the membrane time constant, and we checked that for values of τ in the range 10ms − 100ms our results did not quantitatively change. The total input to a neuron is where x represents the coordinates vector x = {θA, θB, ηA, ηB} and ρ(x) is the joint probability density of the preferred directions and participation strengths, that we will specify later. The mean-field formalism allowed us to reduce the dimensionality of the system to only five order parameters: the total average activity, r0(t); the spatial modulation of the activity profile in either maps, rA(t) and rB(t); the location of the peak of the activity profile in either map: ψA and ψB. The equations governing the temporal evolution of the order parameters are derived in the Methods.
We first characterize the model by studying the fixed-points of the network dynamics when subject to a constant and homogeneous external input. If the external field is independent on θA, θB, the fixed points can be of two kinds, depending on the value of the couplings parameters. One solution is homogeneous, meaning that the activity rate is independent on θA, θB; in this case, the order parameters measuring the spatial modulation of the activity are zero: rA = 0 and rB = 0 (see Methods for details). If the coupling parameters modulating the cosine terms in the couplings (1) exceed a certain threshold, the activity rate is zero in a subset of the (θA, θB, ηA, ηB)−space. The only nonzero stationary states are localized patterns of activity (bump), which can be localized either more strongly in map A (rA > rB > 0), in map B (rB > rA > 0) or at the same level in both maps (rA = rB > 0). Since maps A and B are correlated, the location of the stationary bump of activity is constrained to be the same in the two maps: ψA = ψB ≡ ψ, with arbitrary ψ. The system can relax to a continuous manifold of fixed points parameterized by ψ that are marginally stable. In this continuous attractor regime, the system can store in memory any direction of motion ψ, as the activity is localized in absence of tuned inputs. In the space of parameters , we computed the bifurcation surface separating the homogeneous from the bump phase. The result is shown in Fig. 4.c where for different values of ja we plotted the bifurcation line in the space , that is implicitly defined by the following set of inequalities:
⟨…⟩ denoting average over the distribution of θA, θB, ηA, ηB. Moreover, j0 has to be smaller than 1 to avoid rate instability. Examples of the cross-section of the four-dimensional stationary activity profile r(θA, θB, ηA, ηB) in the marginal phase are shown in Fig. 3. The activity as a function of θA, θB at fixed values of ηA, ηB can be strictly positive, with a full cosine modulation profile (e.g., Fig. 3.c-d, for ηA = 0.4, ηB = 0.4); this is a major difference with respect to the double ring model [42], where the activity has a cosine threshold profile at least in one of the two rings.
Time-dependent activity profiles
The activity of motor cortical neurons shows no signature of being in a stationary state but instead displays complex transients. To model the data, we assumed that the neurons are responding both to recurrent inputs and to fluctuating external inputs that can be either homogeneous or tuned to θA, θB, with peak at constant location ΦA = ΦB ≡ Φ. In the limit of an infinitely large network, we derived a simplified description of the dynamics, where recurrent inputs are replaced by effective local inputs, that take into account the average effect of the many recurrent inputs the neuron is receiving. The total input in (3) is rewritten as the sum three local inputs: the first one is homogeneous, the second is tuned to map A and the third is tuned to map B: where and where we have introduced the external fields parameters C0, CA, CB, ϵA, ϵB representing, respectively, the magnitude of: an homogeneous input; an untuned preparatory input (proportional to ηA but homogeneous in θA); an untuned execution input (proportional to ηB but homogeneous in θB); a tuned preparatory input (proportional to ηA and tuned to θA); and a tuned execution input (proportional to ηB and tuned to θB). Equations (2) and (5) show that the pattern of activity is localized at a constant location Φ throughout the dynamics, while its shape changes in time. For example, the activity pattern can evolve from being strongly localized in map A to being strongly localized in map B; accordingly, the order parameters measuring the spatial modulation in either map, rA, rB, are also time dependent. In order to draw a correspondence between the model and the data, we note that the activity of each neuron in the data corresponds to the activity rate at a specific coordinate θA, θB, ηA, ηB in the model. From equations (2) and (5) we see that, if we assume that changes in the external inputs happen on a time scale larger than τ, the modulation of the activity profile in map A at each time t is centered at θA − Φ and has amplitude proportional to ηA, while the modulation of the activity profile in map B is centered at θB − Φ and has amplitude proportional to ηB. Accordingly, for each recorded neuron θA and θB represent the location of the peak of the neuron’s tuning curve computed during the preparatory and movement-related epoch, respectively, while ηA and ηB are proportional to the amplitude of the tuning curves. From the empirical distribution of the amplitude and location of the tuning curves that we measured from data, we inferred the probability density of the four coordinates ρ(θA, θB, ηA, ηB) that we used in our mean-field analysis (see Fig. 4 and Methods for details). It factorizes as follow: where the distribution of preferred directions is well fitted by: as shown in (Fig. S1), and where ρpA(ηA), ρpB(ηB) are estimated non-parametrically using kernel density estimation. Next, we notice that either of the two fields IA(t) and IB(t) modulating the amplitude of the activity are a sum of two terms (6): the strength of tuned external inputs and the strength of recurrent inputs, which is proportional to the couplings parameters . Hence, the localized pattern of activity can be sustained by either strongly tuned external inputs or by strong recurrent connections.
External inputs inferred from the dynamics of the order parameters
The dynamics of the order parameters rA, rB computed from data (Fig.5) shows that during the delay period neural activity is spatially modulated in map A but not in map B, while during movement execution the activity is strongly modulated in map B, with the degree of modulation in map A slowly decreasing to very small values. This behaviour can be reproduced by the model for different choices of recurrent connections and external inputs parameters, ranging in between the following two opposite scenarios (Fig.5):
The couplings are not direction specific and the localized activity is sustained by an external input that is tuned to θA during movement preparation and to θB during movement execution.
The bump is self-sustained via strong recurrent connections and fluctuating homogeneous external inputs allow the bump to be localized in map A during movement preparation and in map B during movement execution.
The value of the parameters that best fit the data are inferred through minimization of a cost function (Etot) composed of two terms: one is the reconstruction error of the temporal evolution of the order parameters (Erec) and the other represents an energetic cost penalizing large external inputs (Eext): where α is an hyperparameter of the fitting algorithm. To reduce the degeneracy in the results of the fit, we impose not only that the model reconstruct the value of the order parameters r0, rA, rB, but also the value of two additional parameters. We denote them by r0A and r0B and they represent the overlap between the variable ηA (or ηB) and the neural activity. The result of the fit depends on the hyperparameter α (Fig.5). If α is chosen so that the two terms αErec and Eext have equal magnitude – after the cost function is minimized – then the inferred values of the couplings parameters are above, but very close to, the bifurcation surface. In absence of tuned external inputs, the activity sustained by recurrent connections is localized in map B. The inferred value of the external fields is shown in Fig. 6.e. During the delay period, an external input tuned to map A sustains the bump of activity localized in map A and sets the direction of motion; during movement execution, the activity is driven by homogeneous external inputs and the activity localized in map B is sustained by the strong recurrent connections. The correlation between the two maps allows the information about the direction of motion to be encoded stably throughout the whole duration of the task. On the other hand, if we chose a larger value of the hyperparameter α, so to impose a better reconstruction of the recorded dynamics but penalize less for large external inputs, the solutions that we obtain are still close to the bifurcation surface but below it, and a small input tuned to θB is present during movement execution. Importantly, the same analysis applied to a second dataset recorded from a different macaque monkey performing the same task yielded qualitatively similar results (see Fig. S3).
Simulations of the dynamics of the network
The fitting procedure and the results of Fig. 5 are based on the equations governing the dynamics of the order parameters that we derived in the mean-field limit of infinitely many neurons. We next checked that a network of finite size with the same parameters that we inferred in the last section reproduces the dynamics seen in the data. We built a network of 16000 neurons by assigning to each neuron i the coordinates so to match the empirical distribution of coordinates of Fig. 3. In particular, the values of and are chosen to be equally spaced along the lines (see Fig. S5). Neuronal firing rates obeyed standard rate equations (29), were a noise term modelled as an Ornstein-Uhlenbeck process is added to the total input. Simulations of a network whose coupling parameters correspond to the solution of Fig. 6d reproduce well the dynamics of the order parameters (Fig. 7) and the location of the bump fluctuates only weakly around a stable location (Fig. S5). We analyzed the results of the simulations similarly as how we analyzed the data. In particular, we computed tuning curves during the preparatory and execution epochs and estimated the values from the location and the amplitude of the tuning functions. The reconstructed values of neurons tuning parameters are consistent with the values initially assigned to them when we built the network (Fig. S5); moreover, simulating the dynamics with additive noise produces tuning curves whose shape resembles the data (Fig. S6). Next, we added noise to the variables , to break the rotational symmetry of the model. If the network has coupling parameters above the bifurcation surface and no tuned external input is present during movement execution, the location of the bump after movement initiation starts to drift towards a few discrete attractors. When the noise is weak, the drift happen on a time scale that is much larger that the time of movement execution; the larger the level of the noise, the faster the drift. If we instead consider a solution like the one of Fig. 6.g, where tuned inputs are weak but non-zero, the location of the bump remains stable throughout the dynamics. Hence, adding stability constraints to the cost function that we used for our minimization procedure will favor network parameters that are close to, but below the bifurcation surface.
PCA subspaces dedicated to movement preparation and execution
We performed principal component analysis (PCA) on the trial averaged activity to show that the two subspaces that capture most of the variance of neural activity during movement preparation and execution are close to being orthogonal. After identifying the preparatory and movement-related principal components (PCs) (see Methods for details), we quantified the level of orthogonality of the two supspaces by the aligment index as defined in [21], that measures the percentage of variance of each epoch’s activity explained by both sets of PCs. Fig 7 shows that the alignment index is much smaller than the one computed between two subspaces drawn at random (random alignment index, explained in the Methods), both for the simulations and for the data. The level of orthogonality of the preparatory and movement-related subspaces can be understood as follows. The activity of the ring model encoding the value of a singular angular variable is two-dimensional in the Euclidean space. Similarly, the activity of the double-ring model encoding two distinct circular maps is four-dimensional. Our model is an extension of the double-ring model, where both the connectivity matrix and the external fields () are a sum of several terms, each one composed of an η-dependent term multiplying a θ-dependent term. The connectivity matrix is still rank-four, but its eigenvectors are modulated by the η variables. Although the dynamics is four-dimensional, we have shown that during movement preparation the activity is localized only in map A, while during movement execution it is predominantly localized in map B: in either epoch, we expect only two eigenvalues to explain most of the activity variance, as we indeed see from simulations in absence of additive noise (Fig. S7). The degree of orthogonality between the preparatory and movement-related subspaces is determined by the level of orthogonality between map A and map B, which can be quantified in terms of the following correlation: where brackets ⟨…⟩ denote the average with respect to the distribution of (7-8), and where we used the distribution of ηA, ηB that we inferred from data (Fig. 4) to compute the last term on the right-hand side. The correlation between maps is smaller with respect to both the case where ηA = 1, ηB = 1 for all neurons (CAB = 0.33) and the case where ηA, ηB are uniformly distributed (CAB = 0.25). Finally, the noise term added to the dynamics introduces extra random dimensions; as the dynamics gets higher dimensional, both the alignment index and the random alignment index get smaller.
To conclude, we observed that from the PCA analysis alone it is not possible to discriminate between the scenario where neurons tuning is induced by tuned external inputs from the one where it emerges from recurrent connections. Simulations of the network dynamics in the two scenarios yielded very similar results in terms of the variance captured by the first principal components (Fig. S7).
Discussion
Studies on the dynamics of motor cortical activity during delayed reach tasks have shown that the primary motor cortex employs an ‘orthogonal but linked strategy’ [22, 21] to coordinate planning and execution of movements. In this work, we explored the hypothesis that this strategy emerges as result of a specific Hebbian-like recurrent functional architecture, in which synaptic connections store information about two distinct patterns of activity that underlie movement preparation and movement execution. In a simplifying modeling setting based on recordings in Macaque monkeys performing a straight reach task, we characterized response patterns in terms of their covariance with the direction of motion. Hence, the preparatory and movement related patterns stored in the couplings are formalized in terms of two maps (A and B) in the space defined by two features of neurons tuning properties: their preferred direction and their level of participation to the population encoding. We inferred the distribution of these tuning features from data and showed that the degree of correlation between the two maps is small enough to allow for almost orthogonal subspaces, which is thought to be important for the preparatory activity not to cause premature movement; at the same time, having a non-zero correlation between maps allows the activity to flow from the preparatory to the movement-related subspaces with minimal external inputs.
By using a mean field analysis, we derived a simple description of neural dynamics in terms of a few order parameters that can be easily computed from data. Different combinations of the strength of direction-specific recurrent connections and of tuned external inputs allow the model to accurately reproduce the dynamics of the order parameters, ranging from a scenario where neurons tuning properties emerge solely from recurrent inputs to one where the motor cortex is simply integrating tuned external inputs. The addition of an external input strength minimization constraint breaks the degeneracy of the space of solutions, leading to a solution where synaptic couplings depend on the tuning properties of the pre- and post-synaptic neurons, in such a way that in the absence of a tuned input, neural activity is localized in map B. During movement preparation, an external input tuned to map A sustains a localized activity in map A, and sets the direction of motion encoded at the population level. During movement execution, movement direction is stably encoded at the population level, thanks to recurrent inputs that sustain a localized activity in map B. The correlation between maps A and B allows the activity in map B during movement execution to be localized around the same location as it was in map A during movement preparation. These results, based on a mean-field analysis valid for infinitely large networks, were confirmed by simulating the dynamics of a finite-size network of 16000 neurons, with additive noise whose level was chosen so that the dimensionality of the activity in the neural state space matched the one from data. The presence of noise did not disrupt the temporal evolution of the order parameters; on single trials, the direction of motion encoded by the network slightly diffuses around the value predicted by the mean field analysis, but remains stable for trial-averaged activity. However, our solution requires an implausible fine tuning of the recurrent connections. Heterogeneity in the connectivity causes a systematic drift of the encoded direction of motion on a typical time scales of seconds - the larger the structural noise in the couplings, the faster the drift, as has been extensively studied in the literature of continuous attractor models [45, 46, 47, 48, 49]. It has been shown that homeostatic mechanisms could compensate for the heterogeneity in cellular excitability and synaptic inputs to reduce systematic drifts of the activity [47] and that short-term synaptic facilitation in recurrent connections could also significantly improve the robustness of the model [48] – even when combined with short-term depression [49]. While a full characterization of our model in the presence of structural heterogeneity is beyond the scope of this work, we considered a second version of the optimization procedure to infer the model parameters that takes into account the stability of the encoded direction of motion with respect to perturbations in the couplings. We showed that a solution that improves the stability is one where tuned inputs are also present during movement execution. Interestingly, the inferred tuned inputs are still much weaker than the untuned ones, during movement execution. Also, the inferred direction-specific couplings are strong and amplify the weak external inputs tuned to map B, therefore still playing a major role into shaping the observed dynamics during movement execution.
Our prediction that external inputs are direction-specific during movement preparation but non-specific during movement execution agrees with several other studies on the activity of the primary motor cortex during limb movement. In particular, [28] showed that the changes in neural activity that characterize the transition from movement preparation to execution reflect when movement is made but are invariant to movement direction and type; [24] argued that external input transients signalling the motor cortical network a new movement direction are more strongly time-locked to target appearance as compared to the start of movement; finally, [39] measured the correlations in firing between motor thalamic (m-Th) and motor cortical cells, in monkeys performing the same task considered in the present work. In order to investigate if the observed cofiring patterns resulted from neurons tuning properties, the authors looked at the effect of the difference in preferred directions on cofiring patterns between couples of M1–mTh cells. Interestingly, the study reported significant correlation between cofiring patterns and difference in preferred direction solely before movement, but not during movement execution (see Appendix, Fig. S8 of ref. [39]).
Comparison with the ring model
The idea that the tuning properties of motor cortical neurons could emerge from directionspecific synaptic connections goes back to the work of Lukashin and Georgopulos [50]. However, it was with the theoretical analysis of the so called ring model [51, 41, 52, 44] that localized patterns of activity were formalized as attractor states of the dynamics in networks with strongly specific recurrent connections. Related models were later used to describe maintenance of internal representations of continuous variables in various brain regions [53, 54, 55, 56, 57, 58, 59, 43, 60, 61] and were extended to allow for storage of multiple continuous manifolds [62, 42, 63] to model the firing patterns of place cells the hippocampus of rodents exploring multiple environments. While our formalism is built on the same theoretical framework of these works, we would like to stress two main differences between our model and the ones previously considered in the literature. First, we studied the dynamic interplay between fluctuating external inputs and recurrent currents, that causes the activity to flow from the preparatory map to the movement-related one and, consequently, neurons tuning curves and order parameters to change over time, while maintaining stable encoding of the direction of motion at the population level. Moreover, we introduced an extra dimension representing the degree of participation of single neurons to the population encoding of movement direction. We have discussed how the presence of this dimension is key to having tuning curves whose shape resembles the one computed from data, and decreases the level of orthogonality between the subspaces dedicated to the preparatory and movement-related activity.
Extension to modeling richer movements
By analysing neural patterns of covariation with the direction of motion in the simple context of delayed straight reaches, we were able to model the temporal evolution of neural trajectories in low-dimensional latent spaces. However, neurons directional tuning properties have been shown to be influenced by many contextual factors that we neglected in our analysis [64, 65], to depend on the acquisition of new motor skills [66, 67, 68] and other features of movement such as the shoulder abduction/adduction angle even for similar hand kinematic profiles [34, 35]. Moreover,a large body of work [69, 35, 70, 64, 71, 72, 73] has shown that the activity in the primary motor cortex covary with many parameters of movement other than the hand kinematics – for a review, see [74]. More recent studies have also provided evidence that encoding of movement-related variables is insufficient to explain the rich dynamics of neurons in M1 [75, 76, 73, 77]. These and other works [74] argue that the largest signals in motor cortex might not ‘represent’ task-relevant variables at all, but if they do, surely not in a simple way that allows for different features to be decoded by linear decoders [77]. Based on these observations, we will now speculate on two possible ways of extending our model to more realistic scenarios.
In the current model, maps A and B each live in the low-dimensional space defined by neurons preferred directions: θA for map A and θB for map B. In a more general model, maps A and B could be parameterized by additional latent variables other than movement direction. For different tasks, the external input would have a different structure and select the dimensions that are relevant for the required pattern of kinematics and muscle activation. Neurons tuning to one single feature of the motor action would then vary across tasks in a highly non-linear manner, depending on the other latent variables relevant to that type of movement.
Another simplifying assumption allowed by the study of a delayed straight limb task is that the motor action can be clearly separated into a preparatory and movement related phase, where the direction of motion encoded at the population level is constant. We hypothesize that our simple model could also explain encoding of the direction of motion during more complex trajectories, that can be seen as a concatenation of short segments [20], where the network keeps an internal representation of the ongoing movement direction in map B while the next movement in a different direction is being prepared in map A. As suggested in [20], this kind of analysis can also provide a framework to interpret the population encoding of dynamic hand trajectories [78, 79, 36], that could emerge from the superposition of preparatory and movement-related activities relative to consecutive segments.
Author contributions
LBR and NB conceptualized the project. LBR performed the analytical calculations, implemented the computer codes and supporting algorithms. NB supervised the project. NGH provided the data. LBR and NB wrote the paper, and all authors edited the manuscript.
Methods
Analysis of the model
We studied the rate model with couplings defined by (1) with two complementary approaches. First, an analytic approximation based on mean-field arguments and valid in the limit of large network size allowed us to derive a low-dimensional description of the network dynamics in terms of a few latent variables, and to fit the model parameters to the data. Next, we checked that simulations of the dynamics of a network of N = 104 neurons reproduce the results that we derived in the limit N → ∞.
The mean-field equations are derived following the methods introduced in [41, 42]. In the limit where the number of neurons is large, the average activity of a neuron with coordinates θA, θB, ηA, ηB is described by equations (2) and (3). In the following, we will denote the integration measure by the shorthand
In order to derive a lower dimensional description of the dynamics, we rewrite (2) in terms of the average activity rate and of the second Fourier components of the activity rate modulated by ηA/B :
The phase ψA(/B) is defined so that the parameter rA(/B) is a real nonnegative number. Together with (10), the order parameters of the dynamics are thus defined by:
rA(/B) is interpreted as a measure of the spatial modulation of the activity profile, while ψA(/B) represents the position of the peak of the activity profile in map A(/B). From (2), we see that the order parameters evolve in time according to the following set of equations, where the total input (3) is rewritten as:
Stationary states for homogeneous external inputs
We first study the properties of the fixed point solution focusing on the scenario where the joint distribution (7) of the θA, θB is of the form which fits the empirical distribution of the data well for x = 2/3 (Fig. S1). For now, we leave the distributions ρpA(ηA) and ρpB(ηB) unspecified. We first consider the case where the external input to the network is a constant that is independent of θA, θB:
The stationary solutions of (2,14) are of the form: where we have defined the fields
Here, {r0, rA, rB} are solutions of the system (13) with the left hand side set to zero. The second term on the r.h.s in the last equation of (18) is obtained from (14) by observing that in the stationary state ψA = ψB if either ja > 0 or if θA and θB are correlated (as we will assume in the following). As in [41, 44], we can distinguish broad from narrow activity profiles. The term broad activity profile refers to the scenario where the activity of all of the neurons is above threshold, the dynamics is linear and the stationary state reduces to:
By inserting the above equation in (12), we find that the only solution is homogeneous over the maps θA and θB: where the notation ⟨. ⟩ represents an average over the measure dµ, i.e.
First, we notice that a nonzero homogeneous state is present if
Then, the stability of this state with respect to a small perturbation {δr0, δrA, δrB} can be studied by linearizing (13) around the stationary solution, at fixed ψA = ψB = 0. The resulting Jacobian matrix is where
The homogeneous solution (20) is stable if j0 < 1 and if the couplings parameters satisfy the system of inequalities (4). If j0 ≥ 1, the system undergoes an amplitude instability. For values of that exceed the threshold implicitly defined by (4), the dynamics is no longer linear and the activity profile at the fixed point is narrowly localized and characterized by positive stationary values of the order parameters rA, rB.
Tuned and time-dependent external inputs
We next considered the case where the system is subject to a tuned time-dependent external input. While the external inputs have time-varying magnitude, their location Φ is constant and is the same in the two maps: ΦA = ΦB ≡ Φ. Analogously to the ring model [44], the external input pins the location of the bump of activity. We assume that at the initial time the location of the bump equals the location of the external input, so that ψA(t) = ψB(t) = Φ at all times t. Although the position of the bump is stable, the direction modulation of the activity rates change in time in response to time-varying external inputs. We parameterized the external input in analogy with the recurrent input: where C0, CA, CB, ϵA, ϵB represent, respectively, the magnitude of: an homogeneous input; an input that is proportional to ηA (\ηB) but homogeneous in θA (\θB); an input that is proportional to ηA (\ηB) and tuned to θA (\θB). The total inputs can be rewritten as in (5).
Fitting the model to the data
Neurons activity rates were computed by smoothing the spike trains with a Gaussian kernel with s.d. of 25ms and averaging them across all trials with the same condition; denotes the rate of neuron i at time t for condition k, each condition corresponding to one of the 8 angular locations of the target on the screen. Since trials had highly variable length, we normalized the responses along the temporal dimension before averaging them over trials, as follows. We divided the activity into three temporal intervals: from the target onset to the go cue; from the go cue to the start of the movement; from the start of the movement to the end of the movement. For each interval, we normalized the response times to the average length of the interval across trials. We then aggregated the three intervals together. We defined the preparatory and execution epochs – denoted by A and B – as two 300ms time intervals beginning, respectively, 100ms after target onset and 50ms before the start of the movement, in line with [21]. Fig. S2 shows that our results do not change qualitatively when the lengths of the preparatory and execution intervals are increased. For each neuron i, we fitted the activity rate averaged across time within each epoch as a function of the angular position Φ of the target with a cosine function: where the parameters and represent the neuron’s preferred direction during the preparatory and execution epochs. In our the model, the direction modulation of the rates, see (6), is proportional to ηA\B, that measures how strongly the neuron participates in the two epochs of movement; hence, we defined to be proportional to the amplitude of the tuning curve:
The scatter plot of ηA, ηB (Fig. 2.d) shows an outlier with ηB ∼ 1. We checked that our results hold true if we discard that point. Fig. S1 shows that ηA and ηB are not significantly correlated with either θA, θB, nor θA−θB, while Fig. 2 shows that ηA and ηB have significant but weak mutual correlation. Based on these observations, we assumed for simplicity that ηA and ηB are independent variables. The order parameters (10,12) are computed at each time t by approximating the integrals with the sums: where N is the number of neurons and nc = 8 is the number of conditions. The angular location of the localized activity can be computed from (12) as
However, this estimate is strongly affected by the heterogeneity in the distribution of θA, θB - deviating from the rotational symmetry of the model. That is why in Fig. 3 we approximated by the k − th angular location of the target on the screen. Fig. S2 shows that computing by using either method does not affect the dynamics of the order parameters .
We assumed that the observed dynamics of the order parameters r0(r), rA(t), rB(t) obeys equations (13,14) with time-dependent external inputs of the form (21). We inferred the value of the parameters {C0(t), CA(t), CB(t), ϵA(t), ϵB(t)}t of the external fields and of the coupling matrix that allow us to reconstruct the dynamics of the order parameters r0, rA, rB computed from data; since this inference problem is undetermined, we required as further constraint that the model reconstruct the dynamics of the following two additional order parameters:
In this way, for given coupling parameters , we can uniquely identify the external fields parameters that produced the observed dynamics. Still, an equally good reconstruction of r0, rA, rB, r0A, r0B can be obtained for different choices of coupling parameters (3). Hence, we inferred the model parameters by minimizing a cost function composed of two terms: one that is proportional to the reconstruction error of the temporal evolution of the order parameters and the other that represents an energetic cost penalizing large external inputs.
Fitting the model to the data: details
The fitting procedure was divided in the following steps:
The time interval T going from the target onset till the end of the movement was binned into Δt = 5ms time bins: T = {Δt1, Δt2, … ΔtT}.
The couplings parameters were initialized to zero: . At the first time bin, the external fields parameters were initialized to zero: and the reconstructed order parameters (r0, …) were initialized to the order parameters estimated from the data :
For each time step Δti, i = 2, …, T :
We started from the reconstructed order parameters at the previous time step: and we let the dynamical system (13, 25) with external fields parameters C0, CA, CB, ϵA, ϵB evolve for Δt = 5ms to estimate the order parameters at the current time step:
We inferred the value of the external fields parameters by minimizing the reconstruction error: that quantifies the difference between the order parameters estimated from the data and the reconstructed ones; note that the dependence of the cost function E on C0, CA, CB, ϵA, ϵB is implicitly contained in the reconstructed order parameters. We minimized the cost function (26) by using an interior point method algorithm [Byrd et al, 1999] starting from the initial condition we imposed that ϵA > 0, ϵB > 0 and added a L1 regularization term to stabilize the solution.
The external fields inferred with step 3 depend on our initial choice of of the couplings parameters .
Using step 3, the value of the couplings parameters is inferred by mini-mizing the cost function composed of two terms: the reconstruction error and a term that favors small external fields:
The hyperparameter α was chosen so that the two terms have equal magnitude after minimization (see Fig. S4 for details) and a = 0.001 is a parameter rescaling the external fields. The minimization is done using a surrogate optimization algorithm [Gutmann et al., 2001; Wang et al., 2014].
The result does not depend on the choice of the time bin Δt. Also, the result weakly depends on the time constant τ in the mean field equations (e.g., 13), if τ varies on in the range: 10 − 100ms. We set τ = 25ms.
Simulations of the model
We simulated the dynamics of a finite network of N neurons. To each neuron i, we assigned the variables as follows.
In the θ A\B-space, we sampled Nθ points equally spaced along the lines in such a way that their joint distribution matches the distribution of the data (8), as shown in Fig.S5.
In the η A\B-space, we drew Nη points at random from the empirical distribution ρpA(ηA)ρpB(ηB).
For i = 1, 2, … Nθ, we assigned to a block of Nη neurons the same coordinates in the θA\B-space, and all possible coordinates in the ηA\B-space, so that the overall number of neurons is N = NθNη.
The network dynamics we simulated is defined by the following stochastic differential equation: where
. W in (29) is a Wiener process, and the parameters set the magnitude of the noise fluctuations. The level of noise is chosen so that the results of the PCA analysis (see next section) match the data. Note that by setting the noise to zero and taking the limit N → ∞ we recover the mean-field equations (2). The results of Fig. 4 are obtained from a network of N = 16000 neurons; we simulate the network dynamics for 8 location of the external input Φ and 20 trials for each of the 8 conditions, i.e. 20 instances of the noisy dynamics. The order parameters shown in Fig. 4 are computed from single trial activity, and then averaged over trials. The correlation-based analysis, instead, is obtained from trial-averaged activity.
Correlations and PCA analysis
The correlation-based analysis explained in this section was performed both on the smoothed and trial-averaged spike trains from recordings, and on the trial-averaged activity rates from simulations. To compute signal correlations, we preprocessed the data as follows - the same procedure holds both for the recordings and for the simulations. For each neuron, we normalized the activity by its standard deviation (computed across all times and all conditions); then, we mean-centered the activity across conditions. The T = 300ms long preparatory activity for all C = 8 conditions was concatenated into a N × TC matrix denoted by P, and the movement-related activity was grouped into an analogous matrix M. We obtained correlation matrices relative to preparatory and movement related activity by computing the correlations between the rows of the respective matrices. We then identified the prep-PCs and move-PCs by performing PCA separately on the matrices P and M. The degree of orthogonality between the prep- and move-subspaces was quantified by the Alignment Index A [21], measuring the amount of variance of the preparatory activity explained by the first K move-PCs: where Emov is the matrix defined by the top K move-PCs, Cprep is the covariance matrix of the preparatory activity and sprep(i) is the i-th eigenvalue of Cprep. K was set to the number of principal components needed to explain 88% of the execution activity variance. Hence, the Alignment Index ranges from 0 (orthogonal subspaces) to 1 (aligned subspaces). As random test, we computed the Random Alignment Index between two sets of K dimensions drawn at random within the space occupied by neural activity, using the Monte Carlo procedure described in [21]. We performed the same analysis on both the data (K = 12) and the model (K = 9) trial averaged activity. Since the number of recorded neurons was of the order of Ndata ∼ 102, while the simulated network was composed of Nsim ∼ 104 neurons, we computed signal correlations and performed the PCA analysis on a subset of Ndata neurons randomly sampled from the larger simulated network. We repeated this procedure 100 times and averaged the Alignment Index and the Random Alignment Index across the 100 random samples.
We also quantified the roational structure present in the data by applying the jPCA [26] dimensionality reduction technique to both the simulated activity and the recordings, and showed (Fig. S8) that the trajectories from simulations qualitatively resembled the ones from data when projected onto the dimensions that capture rotational dynamics.
Acknowledgements
We thank Jason MacLean, Alex P. Vaz, Subhadra Mokashe, Alessandro Sanzeni for helpful discussions, and Stephen H. Scott for pointing the work of [39] to our attention. This work has been supported by NIH R01NS104898.
References
- [1].↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].↵
- [7].↵
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].↵
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].↵
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].↵
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].↵
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].↵
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].↵
- [57].↵
- [58].↵
- [59].↵
- [60].↵
- [61].↵
- [62].↵
- [63].↵
- [64].↵
- [65].↵
- [66].↵
- [67].↵
- [68].↵
- [69].↵
- [70].↵
- [71].↵
- [72].↵
- [73].↵
- [74].↵
- [75].↵
- [76].↵
- [77].↵
- [78].↵
- [79].↵