Linking Neural Manifolds to Circuit Structure in Recurrent Networks

The classic view of cortical circuits composed of precisely tuned neurons hardly accounts for large-scale recordings indicating that neuronal populations are heterogeneous and exhibit activity patterns evolving on low-dimensional manifolds. Using a modelling approach, we connect these two contrasting views. Our recurrent spiking network models explicitly link the circuit structure with the low-dimensional dynamics of the population activity. Importantly, we show that different circuit models can lead to equivalent low-dimensional dynamics. Nevertheless, we design a method for retrieving the circuit structure from large-scale recordings and test it on simulated data. Our approach not only unifies cortical circuit models with established models of collective neuronal dynamics, but also paves the way for identifying elements of circuit structure from large-scale experimental recordings.


Introduction
The progress of experimental methods continuously reshapes our understanding of the link between brain activity and behaviour.While, for a long time, neuroscientists could only record one or a few neurons at the same time, the recent availability of tools such as high-density probes and twophoton calcium imaging has enabled the simultaneous recording of thousands of neurons [1].As a consequence, systems neuroscience currently offers two complementary perspectives on the structure of neuronal activity [2,3].
The first perspective focuses on the representation and processing of information by single neurons characterised by individual functional properties.As a starting point, landmark studies in neuroscience have established that the activity of single neurons can be modulated by features of sensory stimuli [4][5][6], spatial location and orientation in the environment [7][8][9][10][11], or higher-level taskrelated variables [12][13][14].The dependence of single-neuron responses to stimulus-or task-related features, usually termed tuning, can generally be summarised by a few properties, such as the centre of their receptive field.For example, in flies and mammals, the sense of spatial orientation arises from a population of head-direction cells, where each cell responds preferentially to a specific head direction (HD) [9,10,15].Cells can thus be sorted according to their functional role in the head direction circuit by placing them on a ring (Fig. 1A), where neighbouring cells have similar preferred HD [16,17].Single-neuron properties might also describe other response features besides stimulus tuning: neuronal responses can be modulated by some task-or stimulus-independent variables that are not controlled by the experimenter, such as internal brain state [18][19][20][21][22].In this view, each neuron can be assigned a "location", i.e. be embedded, in an abstract space where neighbourhood reflects the functional similarity between neurons.We call it the functional similarity space, or more briefly similarity space.Importantly, the embedding of the neurons may not correspond to their physical location in the brain [22].For instance in mammalian V1, preferred orientations are distributed across the cortical sheet, forming pinwheel-like patterns [23,24].
The second perspective takes a descriptive, population-level approach, put forward by the advent of large-scale recordings [25].In this perspective, the joint activity of N simultaneously recorded neurons, or population activity, is represented at any given time as a point in a N -dimensional state-space.Several studies have revealed that the activity of heterogeneous, large populations of neurons is often restricted to a low-dimensional space, often referred to as the "neural manifold", embedded within the high-dimensional state-space of the population activity [26][27][28][29][30][31].In the neural manifold, the momentary activity can be described by a small number D ≪ N of time-dependent variables that represent computationally relevant quantities; e.g., the animal's head direction, or the perceived sensory stimulus (Fig. 1C).The change of these variables over time reveals the lowdimensional dynamics of the population activity on a time scale from tens of milliseconds [21,[32][33][34] to tens of seconds [22] (Fig. 1D).Hence, in this view, neural computation is described by a set of trajectories, or "flow", in the low-dimensional neural manifold (Fig. 1E) that emerges from the collective dynamics of a large number of neurons [34][35][36][37][38].
To summarise, the first perspective focuses on how the functional properties of individual neurons shape neuronal activity, whereas the second perspective focuses on a population-level description of neural representations and computations [2].The concept of tuning potentially provides a direct link between the two perspectives [39].A tuning curve traditionally describes how a single neuron's response depends on a particular stimulus variable.Thus, the set of tuning curves of all the neurons in the population directly maps each possible stimulus to the corresponding population activity.If the set of all possible stimuli is parameterised, say, by a one-dimensional circular Fig. 1 Two perspectives on neuronal activity.A. The "circuit" perspective illustrated by the ring model of the head direction (HD) system.A1.The animal's HD is represented by an angle α.A2.Each neuron i in the population responds preferentially to a specific HD angle z i (horizontal axis).Neuronal activity (vertical axis) depends on the difference between the current HD α and each neuron's preferred HD angle z i .Coloured dots represent three examples neurons.A3.Each neuron's preferred HD angle determines its "location" on a one-dimensional ring (blue dots).The persistent activity of HD cells in the absence of stimulation [42,46] can be explained by a ring attractor model [16,17], where recurrent synaptic connections W ij between pairs of neurons i and j are stronger if neurons are close neighbours on the ring (e.g. the blue and green neuron) than if they are further apart (e.g. the blue and yellow neuron).B. In the "circuit" perspective, single-neuron responses r i (t) are characterised by the locations of the neurons in an abstract similarity space M of dimension d, where neighbourhood represents functional similarity.Classical circuit models postulate that recurrent connections between pairs of neurons i and j (and forward connections from external inputs) are determined by their locations z i , z j in this abstract space.C. The "neural manifold" perspective illustrated by large-scale recordings of the neuronal activity in the monkey prefrontal cortex [34].During a decision-making task, the population activity r(t) of a population of N cells lives in a two-dimensional subspace, spanned by a 'stimulus' and a 'choice' axis, which encode information about the sensory stimulus and the decision made during a trial, respectively.Connected points correspond to the population activity during a single trial, with color representing time elapsed since trial onset.D. In mouse medial entorhinal cortex, the linear projection of the population activity onto relevant axes reveals slow oscillatory population-level dynamics, internally generated by recurrent mechanisms [22].E. In the "manifold" perspective, the joint activity of N neurons, r(t) ∈ R N , lies on a low-dimensional neural manifold with intrinsic dimension D ≪ N (shown here as a curved, (D = 2)-dimensional surface).The activity is parametrised by population-level latent variables κ 1 (t), . . ., κ D (t) whose dynamics reflect neural computations.Panel (C) is inspired from [20].Panel (D) is inspired from [22].
variable (for example, the head direction in the case of HD cells), the set of all the possible population activities will in turn necessarily form a ring.More generally, low-dimensional tasks tend to yield low-dimensional neural trajectories [40,41].Yet, such a static transformation from stimulus to population activity is insufficient to explain some of the most striking features of neural manifolds.First, it has been observed that neural manifolds can persist during sleep [27,28,42], and be dissociated from sensory cues [10,43]; which implies that the neural manifold is not a mere reflection of external stimuli.Second, the low-dimensional trajectory of the population activity observed in experiments often does not correspond to the evolution of present external stimulus variables, but is rather determined by the initial condition [21], context [34], or internally-generated factors [22,30,44] (Fig. 1D).Both findings point towards underlying recurrent circuit mechanisms [26,45].
In the examples mentioned above, strong excitatory connections are found between neurons with similar preferred HD in the fly HD system [47], or between neurons with a similar orientation tuning in V1 [24].Consistent with the traditional notion of cortical columns [48], these observations support the idea that functional similarities between neurons arise from network structure [12,42,49].This idea has been the basis for many computational models in which forward and recurrent connections are determined by the neurons' respective functional properties (Fig. 1B) [50][51][52][53][54][55][56][57].However, the link between so-called circuit models and neural manifolds remains elusive [2,3].
In this paper, our aim is to connect the two perspectives by addressing the following questions.(i) Can we establish a mathematical link between the similarity space, summarising the structure of the connectivity in circuit models, and the flow of the collective dynamics in the neural manifold?(ii) Is there a one-to-one mapping between the neural manifold and the similarity space?Although the topology of the neural manifold has been found to match that of the similarity space in special cases (for example, a ring in the fly HD system) [3,47], there is no obvious reason why this should be true in general.(iii) Can we extract knowledge about the circuit structure from neuronal activity?Although many studies have focused on extracting the neural manifold from experimental data [27,28,32,33], an established method for extracting the similarity space, without explicitly referring to tuning curves or selectivity, is missing.
The results are organised as follows.First and foremost, we propose a modelling framework that provides an explicit link between circuit structure and collective dynamics in Recurrent Neural Networks (RNNs).Relying on the theory of neural fields [52] and large low-rank networks [58], our framework connects functional properties of single neurons to emergent low-dimensional dynamics in an RNN model.Maybe surprisingly, the theory predicts that there is no one-to-one relationship between the functional similarity space and the neural manifold.In particular, their respective dimensionality can be different, and the same collective dynamics can result from different circuit structures.Finally, we propose a method for extracting the similarity space from the neuronal activity, by exploiting the similarities between the activity of single neurons.We apply this method to simulated data, and show that we can retrieve the dissimilar circuit structure of different RNNs performing the same simple task.

A unifying framework based on the theory of large RNNs
In this section, we present a modelling framework that links the similarity space of single-neuron response properties with the neural manifold of population activity; the details of the derivations are presented in the Methods 1.Our framework extends recent theoretical results on large RNNs [59][60][61] to field models [52,59].Since the dynamics of large Spiking Neural Networks (SNNs) composed of Linear-Nonlinear-Poisson (LNP) neurons [62,63] converges to the dynamics of equally large RNNs in the large N limit (with 1/N synaptic weight scaling) [61], we directly consider the case of LNP spiking neurons.
In a network composed of N neurons, let the membrane potential h i of each neuron i = 1, . . ., N be a leaky integrator (with time constant τ ) of recurrent and external inputs.Each spike of a presynaptic neuron j, connected to postsynaptic neuron i, causes an instantaneous change of the postsynaptic potential h i with magnitude given by the synaptic weight 1 N W ij from neuron j to neuron i.The network also receives K external inputs I 1 (t), . . ., I K (t), with input weights to neuron i denoted by w ext i1 , . . ., w ext iK , respectively.The dynamics of the membrane potential h i thus reads: where S j (t) = s∈N δ(t − t s j ) denotes the spike train of neuron j.The instantaneous firing rate of each neuron i is given by r i (t) = ϕ(h i (t)), where ϕ is a positive, nonlinear transfer function.The set of instantaneous firing rates, r(t) = (r 1 (t), . . ., r N (t)), define the intensity of N (conditionally independent) inhomogenous Poisson processes from which the spike times {t s j } of each neuron j are drawn.
The theoretical framework relies on two assumptions.First, we assume that each neuron i can be functionally characterised by a small set of d ≪ N features, denoted z i = (z i,1 , . . ., z i,d ), which determine its abstract location in a d-dimensional functional similarity space M, where neurons close to each other have similar response properties.This assumption is rooted in the circuit perspective described in the introduction and is the starting point of many models in computational neuroscience [16,[50][51][52][53][54][55][56][57].This assumption means that both recurrent and external connections in Eq.( 1) depend on the respective "locations" of the neurons, so that Eq.( 1) can be rewritten as: where f ext 1 , . . ., f ext K are continuous functions that characterise the tuning of the neurons to the external inputs I 1 , . . ., I K as a function of their functional properties.Similarly, the connectivity kernel w describes the recurrent connectivity as a function of the neurons' functional properties.
Second, we assume that the kernel w can be written in the form: where f 1 , . . ., f D and g 1 , . . ., g D are arbitrary continuous functions over the similarity space M (Fig. 2); and D ≪ N is a fixed number, not necessarily related to the dimension d of the similarity space.Motivated by results on low-rank RNNs, this assumption is rooted in the "neural manifold" perspective: in an RNN where the recurrent connectivity has rank D, the population activity lies in a D-dimensional neural manifold [58].Models of low-rank networks can perform a wide range of computations, from the retrieval of patterns stored as fixed points (as in the Hopfield model) [64][65][66] to the approximation of arbitrary dynamical systems [38,60].We note that the matrix of recurrent connections in Eq.( 3) can be written as the outer product of two N × D matrices F and G: with entries F i,µ = f µ (z i ) and G j,µ = g µ (z j ).Therefore, we recognise the well-known structure of a connectivity with rank D (Fig. 2) [58].
Thus, the assumption of Eq.( 3) is a generalised formulation of the connectivity of large low-rank networks.This observation leads to our first result: models of large low-rank networks can generally be represented as circuit models, where each neuron is assigned a location in an underlying similarity space and the connectivity depends on the respective locations of the neurons as expressed by Eq.( 3).If the locations z 1 , . . ., z N of the neurons are drawn independently, the network dynamics of Eq.( 2) then converges to the solution of a neural field equation in the large N limit [61,70,71], where the time-dependent membrane potential of each neuron i depends on its location z i .(This convergence result actually holds for all networks with 1/N scaling of synaptic weights [72].)Therefore, we sometimes refer to a circuit model with many neurons as a "field model" in the rest of the paper.
For the ease of exposition -and without loss of generality (Methods 1) -we consider in the following that the rank D of the connectivity equals the dimensionality K of the input and that the functions (f 1 , . . ., f D ) and (f ext 1 , . . ., f ext K ) are the same.Our second theoretical result is that, if neuronal locations z i are sampled independently with probability distribution ρ(z), then, in the limit of a large number of neurons N → ∞, the potentials h 1 (t), . . ., h N (t) can be written as where the latent variables κ 1 (t), . . ., κ D (t) are the solution of Equation ( 6) tells us three things.First, it is deterministic, implying that large networks of stochastic LNP neurons follow deterministic collective dynamics [61].Second, the collective dynamics can be reduced to the nonlinear system of D ordinary differential equations given by Eq.( 6), which means that the neural manifold is at most D-dimensional.Third, Eq.( 6) expresses the dynamics of the latent variables κ µ (t), representing the coordinates of the population activity in the D-dimensional neural manifold, in terms of an integral over the d-dimensional similarity space M.This provides an explicit mathematical link between the circuit structure, summarised by the locations of the neurons in the similarity space, and the flow of the collective dynamics on the neural manifold.
It is worth noticing that the low-rank connectivity matrix of Eq.( 3) could be the result of Hebbian learning [64,66].Indeed, we can interpret each function f µ (z i ) as the probability that neuron i with tuning properties z i forms, in the experimental condition µ, a dendritic spine, and g µ (z j ) as the probability that neuron j forms a presynaptic bouton.The change of synaptic efficacy in condition µ will then be the product ∆W ij ∝ f µ (z i )g µ (z j ): across several experimental conditions, the low-rank connectivity of Eq.(3) will build up.In a memory-related brain area, the "experimental condition" could correspond to the presentation of a novel memory item; in early visual cortex, the experimental condition could correspond to elongated spatial edges observed during the critical period [73].

Circuit structure and neural manifold can mismatch: examples
In this section, we highlight the insights provided by the above framework through a few examples.
For the sake of simplicity, we assume, in these examples, that the network does not receive any external input and that the transfer function ϕ is a step function of the membrane potential h, i.e. ϕ(h) = R for h > 0 and zero otherwise.Qualitatively, results do not change for other choices of saturating transfer functions ϕ (Appendix B).
A one-dimensional circuit structure generating a two-dimensional neural manifold In a first toy example, imagine that the functional properties of each neuron i are summarised by a single variable z i ∈ R, which is normally distributed across neurons (Fig. 3A1).This determines each neuron's location in the one-dimensional similarity space M = R.If the interactions are given by W ij ∝ z i z j , this corresponds to a Hopfield network with a single Gaussian pattern.In this case, the collective dynamics have a single stable non-trivial fixed point (up to a symmetry degeneracy) [60,67].We study a variant of this model where synaptic weights are given by: where J > 0 is the interaction strength, and negative weights (i.e.inhibitory synapses) are rescaled by a factor 0 < a < 1.The weights of Eq.( 7) can be written in the form of Eq.(3), as: 3A2), meaning that the connectivity has rank D = 2.The theory predicts two-dimensional collective dynamics, with three stable fixed points spread out in the plane and one saddle point at the origin (Fig. 3B; more details in Methods 2.1).This is confirmed by simulations of a SNN with N = 2000 neurons (obeying Eq.( 1)) with weights given by Eq. (7).Thus, this model generates a neural manifold of dimension D = 2, whereas single-neuron responses depend on their abstract location in a (d = 1)-dimensional similarity space.

Two different circuit structures generating identical limit-cycle dynamics
We now turn to an example of limit-cycle dynamics, which is of practical relevance across different brain areas and species.For instance, in a task where a monkey has to grasp a lever and cycle with its hand, the activity in the motor cortex follows limit-cycle collective dynamics [30] that are likely generated by recurrent mechanisms [44].In a different context, slow, internally-generated cycling dynamics have been recently observed in the mouse medial enthorinal cortex (MEC) [22] (Fig. 1D).
We build here two models with different circuit structures that generate identical limit-cycle dynamics in a neural manifold of dimension D = 2.The first model is a ring model, in which neurons are uniformly assigned angular positions z i ∈ [0, 2π[ (Fig. 3C1) [16,51].The similarity space thus has dimension d = 1.We express the connectivity as: where J > 0 is the connectivity strength, and the angle δ ∈] − π, π] is a phase shift parameter.
For δ = 0, we get a standard ring attractor model with stationary bumps of activity [51], whereas for δ ̸ = 0, the attractor states are travelling bumps [16].Using the sum expansion of the cosine function [59], we see that the connectivity has rank D = 2, where the functions in Eq.( 3) are given by f 1 (z) = cos z, f 2 (z) = sin z and g µ (z) = Jf µ (z + δ) (for µ = 1, 2).In the second model, neurons have two-dimensional locations z i = (z i,1 , z i,2 ) ∈ R 2 normally distributed over the plane (Fig. 3C2); thus, the similarity space has dimension d = 2.As in the ring model above, the connections are maximal for an angle δ between two neurons: where R (δ) is the 2 × 2 matrix of rotation by an angle δ in the plane, and J ′ > 0 is the connectivity strength.The connectivity of Eq.( 9) has also rank 2, since it is readily expressed as a two-dimensional dot product.
Remarkably, for J = π 2 J ′ (i.e. up to a rescaling), the two models yield identical twodimensional collective dynamics (Methods 2.1).The theory predicts that for any initial condition, the two latent variables that parameterise the population activity converge to a globally attractive limit cycle with angular speed τ −1 tan(δ), which is confirmed by simulations of SNNs with N = 2000 neurons (Fig. 3D).
We highlight two aspects of this example.(i) Neither the dimension D of the neural manifold, nor the flow of the collective dynamics enable us to distinguish the ring model from the planar model.More generally, we found that a mismatch between the dimension of the similarity space and that of the neural manifold can originate from heterogeneity across the neuronal population: additional single-neuron properties, parameterising heterogeneity across neurons, can expand the dimensionality of the similarity space without affecting the collective dynamics.For instance, the above "planar" limit-cycle model can be interpreted as a ring model where each neuron is assigned an additional "radial" coordinate, independent of its angular location (Methods 2.2).Nevertheless, we present later, in the section "The footprint of circuit structure", a method that allows us to extract circuit structure from neuronal activity.(ii) For both the "ring" and "planar" models, the connectivity is invariant under the simultaneous rotation of all neurons by any angle.In turn, the collective dynamics exhibits a rotation symmetry, visible in Fig. 3D.The general link between these symmetries is explained in the next section.

Symmetries of circuit structure induce symmetries of collective dynamics
The general notion of a symmetry of a dynamical system refers to a transformation of the system that does not affect its dynamics.Our theory links circuit models and neural manifolds on the level of symmetries: symmetries of a circuit model, expressed as transformations of the neuronal locations in the similarity space, generate symmetries of the collective dynamics in the neural manifold.While this conceptual link is intriguing, its biological relevance is, at this stage, unclear.Nonetheless, in the following, we briefly present our main result and how it applies to the example models above.
Consider a simultaneous transformation of all neuronal locations, expressed as a mapping S : M → M over the similarity space that takes each neuron i from location z i to location Sz i .We call this transformation a microscopic symmetry if it commutes with the (microscopic) network dynamics of Eq.( 2).Analogously, a macroscopic symmetry is a transformation of the latent variables (κ 1 , . . ., κ D ), expressed as a mapping T : R D → R D , that commutes with the collective dynamics of Eq.( 6).In Methods 3 (and Appendix C), we show that each microscopic symmetry S of a circuit model induces a macroscopic symmetry T (S) of the associated collective dynamics in the D-dimensional neural manifold.For instance, the connectivity of the ring model, Eq.( 8), is invariant under the translation of all neurons along the ring by any arbitrary (angular) distance β.This microscopic symmetry of the ring model is the origin of the evident macroscopic rotational symmetry of its collective dynamics (Fig. 3E2).The earlier example of a one-dimensional circuit model with connectivity given by Eq.( 7) also has a microscopic symmetry, given by the "flip" S : z → −z of each neuron's position.The resulting macroscopic symmetry is the reflection in R 2 w.r.t. the axis spanned by the vector (1 − a, √ 1 − a 2 ) (Fig. 3E1).

The footprint of circuit structure
We conclude from the previous sections that, despite the link provided by the symmetries, the functional similarity space and the neural manifold can have different dimensionality, and that several circuit models can generate the same collective dynamics.Yet, even if the circuit structure provably can not be identified from the collective dynamics, can it be recovered by analysing neuronal activity?In this section, we propose an approach to this problem, that we apply on a model of context-dependent decision-making.

Single-neuron statistics reflect neuronal locations in the similarity space
Suppose that we have spiking data from an unknown circuit model network with N neurons.Using standard tools, we can extract at each time point the N -dimensional vector of the population activity, and trace out its trajectory on a D-dimensional neural manifold [27][28][29][31][32][33][34].On the other hand, in any circuit model with many neurons (i.e. a field model), the activity of each neuron i depends on its location in the functional similarity space.Thus, two neurons i and j at neighbouring locations z i ≈ z j will exhibit similar activity.
In order to extract similarity relations between neurons, we choose T different statistical measures that we apply to the spike train of each neuron: this yields an embedding of the neuron in T -dimensional space.Importantly, neighbourhood in the similarity space will lead to similar embeddings.The embeddings associated with several neurons will then reflect their relative positions in the similarity space.Consequently, if sufficiently many neurons are recorded, the d-dimensional similarity space will manifest itself as a point-cloud (with intrinsic dimension d) in the T -dimensional space of statistics.Thus, the above analysis should enable us to extract from N -dimensional spike data a d-dimensional similarity space, in which each neuron is assigned an empirical location.
In principle, neuronal embeddings could be generated using any well-chosen set of T statistical measures that summarise each neuron's activity: for instance, it could be given by each neuron's tuning to T different population-level factors; or by the "loadings" of the first T principal components (PCs) of population activity associated with each neuron [22] (as we illustrate in the next section).As a key property, these statistics should reflect functional similarities, so that neurons close to (far from) each other in the similarity space generate similar (different) outputs.
As an illustration, we first applied this method to extract the circuit structure of two different SNNs generating similar limit-cycle dynamics, with connectivity given as in the previous examples by Eqs.( 8) and ( 9).We characterised each neuron's spiking activity by its tuning to the two first PCs of the population activity, yielding a two-dimensional embedding of the whole similarity space (Methods 4 and Supp.Fig. S2).

Retrieving circuit structure in models of context-dependent decision-making
Finally, we turned to a more relevant setup for systems neuroscience, in which model networks have to process external inputs to solve a given behavioural task.We applied our method for extracting the similarity space to simulated data obtained from two SNNs which solved the same task.This illustrates: (i) how our framework, linking circuit models and collective dynamics, can be used to build transparent models that solve a given task; and (ii) how different circuit models can be characterised by extracting their similarity space from the neuronal activity.
The chosen task is a simplified instance of context-dependent decision-making [34].During a given trial, the network receives a sensory input consisting of two independent signals, named 'color' and 'motion', which can both be either positive or negative; as well as a binary context input which indicates which of the 'color' or 'motion' signal is relevant (Fig. 4A).The task consists in indicating, in the 'color' context, whether the 'color' signal is positive or negative at trial onset, and similarly for the 'motion' context.The 'choice' of the network is given by a weighted readout of its low-passed spiking activity.
We designed two SNNs with different circuit structures, composed of N = 3000 neurons each, that we will call the "blue" and the "orange" SNN, which both solve the task through similar two-dimensional collective dynamics [58] (Methods 4).For both models, population activity decays towards a baseline state between two trials, while in the 'color' context, symmetric attractive fixed points appear along a 'color'-relevant axis, and similarly in the 'motion' context (Fig. 4B).We aim to identify the circuit structure of each of the two SNNs based on their activity.
Extracting circuit structure from neuronal activity.
The two-dimensional trajectory corresponding to the population activity of each SNN across four trials was obtained by projecting its low-passed, N -dimensional spiking activity onto 'color' and 'motion'-relevant axes [34].Upon receiving the same sensory and context inputs, the low-dimensional trajectories and the choice readouts of the two SNNs were similar (Fig. 4C,E).
A 3-dimensional embedding of each neuron was then obtained by measuring, for each neuron, the associated loadings onto the first three principal components (PCs) of the population activity (Fig. 4F).For the "blue" SNN, the point-cloud of the embeddings of all the neurons in the population forms a ring; whereas for the "orange" SNN, it lies on a curved, 2-dimensional surface.For both networks, the similarity space inferred from statistical features reflects the underlying circuit structure: the "blue" SNN corresponds to a ring model, and the "orange" SNN to a model with a 2-dimensional similarity space (Methods 4).Neurons were then assigned empirical locations based on their relative positions in the inferred similarity space.A representation of the neuronal activity Fig. 4 Extracting the circuit structure of two SNNs performing the same task.A. During each trial, networks receive an external input consisting of two sensory features (named 'color' and 'motion'), each fluctuating around a positive or negative mean value, and a binary context feature.The task is to indicate the sign of the sensory feature that is specified by the context ('color' for the context being +1, and 'motion' for −1).Input, recurrent, and output synaptic weights are determined by the neurons' locations {z i } in an underlying similarity space.B. The task is solved by two-dimensional collective dynamics, where the context input generates attractors (green dots) along an axis corresponding to the relevant sensory input feature.This is implemented in two SNNs, called "blue" and "orange", of N = 3000 neurons each C. Time-course of the sensory inputs (green and red traces for the 'color' and 'motion' features) and normalised readout of the two SNNs (blue and orange traces) during 4 trials.The context input takes value +1 (respectively −1) in the 'color' (resp.'motion') context, indicated by green (resp.red) shadings, and 0 in between trials.Both networks correctly solve the task.D. Spike trains of 150 example neurons from the "blue" SNN.Spike trains of example neurons i and j are highlighted.E. Low-dimensional trajectory of the two SNNs (blue and orange traces) during 4 trials, obtained by projecting the N -dimensional low-passed spiking activity ŝ(t) onto 'color'and 'motion'-relevant axes.F. The spiking activity of each neuron is characterised by the corresponding loadings on the three first principal components (PCs) of the population activity, which determines an embedding of the neuron in three dimensions.F1.For the "blue" SNN, the point-cloud of the embeddings of all the neurons in the population forms a ring, revealing the similarity space of the circuit model.F2.In contrast, for the "orange" SNN, the set of all three-dimensional embeddings lies on a curved, two-dimensional surface.Points are coloured according to their height (loading on PC3).G1.Each neuron i in the "blue" SNN is assigned an empirical location ẑi according to its position on the ring-like point-cloud in panel F1.Coloured curves represent the neurons' momentary firing rates as a function of their empirical locations, at three different time points t 1,2,3 (coloured dots on the x-axis of panel C).G2.Each neuron i in the "orange" SNN is assigned a two-dimensional empirical location ẑi according to its position on the two-dimensional point-cloud in panel F2.Heatmaps represent the neurons' firing rates as a function of their empirical locations, at the same three time points t 1,2,3 (from left to right).Orange dots on the left-most heatmap represent all empirical locations inferred from panel F2.In panels F1-2, the percentage of variance of the population activity explained by each PC is indicated on the corresponding axis.Grey point-clouds show projections of the points onto the horizontal and a vertical plane.Details about implementation in Methods 4.2-4.3.
as a function of the empirical neuronal locations highlights its underlying spatial structure, which differs between the two networks (Fig. 4G).

Extracting a task-irrelevant circuit structure component
In the previous example, the difference between the circuit model underlying each of the two "blue" and "orange" SNNs could have been detected by inspecting the tuning of each neuron to the sensory input features.Suppose that we assign to each neuron a two-dimensional "tuning vector", that characterises its response to the two input features 'color' and 'motion'.In both models, individual neurons are tuned to mixtures of these features ("mixed selectivity" [14]).Yet for the "blue" network, the tuning of each neuron to these two features is correlated, so that the corresponding set of two-dimensional tuning vectors lies on a circle; which is not the case for the "orange" network.
Because our approach is task-agnostic, a circuit structure component that is irrelevant to the task at hand can also be extracted.As an illustration, we considered a third "pink" SNN, where the ring circuit structure of the "blue" SNN described above was modified by the addition of a task-irrelevant, Hebbian-like connectivity component (Fig. 5A).To do this, a task-irrelevant feature ξ i = ±1 was assigned to each neuron i (chosen independently across neurons with equal probability of being +1 or −1); and a term proportional to ξ i ξ j was added to the synaptic weight between pairs of neurons i and j (Methods 4).The functional similarity space of this new "pink" model now consists of two rings: each neuron i is jointly characterised by an angular position z i , and by the binary value of the additional feature ξ i .Since this additional feature is irrelevant to the task, it cannot be detected through the tuning of the neurons to the task variables (context, color or motion).If both SNNs receive identical external inputs, their respective population activity follow similar low-dimensional trajectories along task-relevant axes (Fig. 5B).
Low-dimensional embeddings of the neurons in both networks were obtained as in Fig. 4 by measuring, for each neuron, the associated loadings onto the first PCs of the population activity.The resulting point-clouds reflect the underlying circuit structure: for the "pink" SNN, the pointcloud forms two disjoint rings, revealing the topology of the similarity space (Fig. 5C); whereas one recovers the original single-ring structure for the "blue" SNN.

Discussion
To address the relationship between the functional similarity space, assumed to reflect the structure of cortical circuits, and the neural manifold, assumed to characterise computations via the collective dynamics of population activity, we formulated three specific questions in the Introduction.The first question focused on the mathematical link between the structured connectivity of a circuit model and the flow of its collective dynamics in the neural manifold.To answer this question, we proposed a modelling framework that explicitly provides this link.Our framework relies on the key idea that, in the limit of large networks, both low-rank RNNs and classical circuit models (where the connectivity is structured by the functional properties of the neurons) can be interpreted in the general framework of neural field models.In passing, we answered the question of whether "low-rank RNNs involve mechanisms similar to classical circuit models or implement truly novel solutions" [3].
The second question, addressing whether a unique relationship exists between the functional similarity space and the neural manifold, must be answered negatively: different circuit structures can yield equivalent collective dynamics.On the other hand, we proved that symmetries of the circuit structure give rise to symmetries of the collective dynamics, a result that highlights the importance of symmetries for models of computation in recurrent networks.
Fig. 5 Extracting a task-irrelevant circuit structure component from population activity.A. Two SNNs (of N = 3000 neurons each) solve the same task as depicted in Fig. 4A.The first SNN corresponds to a ring model (the "blue" SNN of Fig. 4).In the second "pink" SNN, in addition to its location z i on the ring, each neuron is assigned a task-irrelevant binary feature ξ i = ±1, distributed independently across neurons; and a task-irrelevant component Kξ i ξ j , of strength K = 0.4 > 0, is added to the synaptic weights between pairs of neurons i, j.B. Low-dimensional trajectories of the two SNNs (blue solid line for the original "blue" SNN, for which K = 0; and pink solid line for the "pink" SNN with K = 0.4) across 4 trials during which they received the same external input, obtained by projecting the N -dimensional spiking activity on two task-relevant axes (as in Fig. 4D-E).C. 3-dimensional embeddings of the neurons in the "pink" (K = 0.4) and the "blue" SNN (K = 0; small inset).For the "pink" SNN, the three-dimensional point-cloud forms two disjoint rings, revealing the similarity space of the circuit model.Points on the lower or upper ring correspond to neurons with a task-irrelevant feature ξ i = −1 (resp., ξ i = +1).The embedding of each neuron is given by the corresponding loadings on the PCs 2,3 and 4 of the population activity (PCs 1,3,4 for the "blue" SNN).Points are coloured according to their projection onto the vertical axis.Plotting conventions are as in Fig. 4F.Inset: the single-ring circuit structure of the original "blue" SNN is recovered for K = 0 (similar to Fig. 4F1).
The third question explored the possibility of extracting knowledge about the circuit structure from recordings of neuronal activity.As an answer to this question, we proposed a method for extracting the topology of the similarity space of a circuit model from the neuronal activity, and applied it on simulated data.Our method allowed us to distinguish between different circuit structures giving rise to the same collective dynamics and solving the same behavioural task.In contrast with the analysis of neuronal tuning or selectivity, this approach is task-agnostic, which allowed us to detect a circuit structure irrelevant to a given behavioural task.In conjunction with task-and behaviour-informed methods [32], the unsupervised approach we propose for characterising functional similarities could enable the discovery of intricate features that modulate neuronal responses in higher cortical areas, and could also be applied in the absence of any task on recordings of freely behaving animals [74,75].
While our modelling framework provides a link between circuit structure and low-dimensional neuronal activity, it does not incorporate important features of more realistic models of biological neural networks.First, in our model, recurrent connectivity and neuronal properties are constructed according to a homogeneous structural principle (that of field models), whereas cortical neuronal responses exhibit a large degree of trial-to-trial variability [76,77], which could be explained by chaotic or chaos-like activity due to randomness in the connectivity [58,66,78].The generalisation of our theory to the case of mixed structured and random connectivity, following [58], is left for future work.Second, the neurons in our model are linear-nonlinear-Poisson neurons, which do not account for neuronal refractoriness.Our theory can be partly extended to models of renewal neurons (such as the Spike Response Model or Integrate-and-Fire models [79,80]).Because of the reset, neuronal dynamics is no longer memory-free and requires an extended formalism [79].Yet, in the regime of asynchronous firing, each neuron's firing rate can be approximated by a non-linear function of its synaptic input [81], so that our formalism can be applied to the description of the fixed points of the asynchronous population activity.Third, cortical networks exhibit a balance between excitation and inhibition [82,83].This has been taken into account in models of balanced networks [84][85][86][87], which can include a spatial structure [88,89].Whether our formalism can be extended to derive a low-dimensional description of spatially structured balanced networks remains an open problem.
Finally, we did not apply our method to extract the functional similarity space from experimental data.Yet, in ref. [22], an approach similar to that proposed in this paper was used on recordings of the mouse medial enthorinal cortex: single-neurons were characterised by a few statistics (the PC loadings) extracted from the population activity, and assigned a location on a ring according to these statistics (similar to Supp.Fig. S2); this revealed a bump of neuronal activity slowly cycling over this virtual ring.The question of whether such a low-dimensional population structure can be generally observed in experimental recordings remains to be tested.A positive answer would mean that, even in a population where neurons exhibit mixed selectivity and seemingly heterogeneous activity, neuronal responses can be parameterised by only a few variables, making field models wellsuited for describing collective neuronal activity.More broadly, the link between neural manifolds and circuit (or field) models established in this work could potentiate the use of large-scale recordings to uncover circuit structures underlying population-level computation in large networks of neurons.We turn here to a more precise mathematical formulation.We consider a model of Recurrent Neural Network (RNN) composed of N linear-nonlinear-Poisson neurons [62].The spike train S i (t) of each neuron i = 1, . . ., N is generated by an inhomogeneous Poisson process with intensity r i (t) = ϕ(h i (t)), where h i (t) represents the neuron's membrane potential, and ϕ : R → R + is a nonlinear bounded voltage-to-rate function.The potential h i (t) is a leaky integrator of the synaptic input; which, for the ease of exposition, we first consider as coming only from recurrent connections.The dynamics of the potential thus follow: with given initial condition h i (0).Here, τ > 0 is the membrane time constant; and the synaptic weights, scaled as 1/N , are given by a matrix of rank D ∈ N fixed: where the dot denotes the dot product in R D .Each entry W ij of the low-rank connectivity matrix corresponds to the dot product of two D-dimensional "connectivity vectors", ⃗ F i = (F i,1 , . . ., F i,D ) and ⃗ G i = (G i,1 , . . ., G i,D ), associated with neurons i and j (Fig. 2).As discussed in the Results section, Eq.(1.2) is a general formulation of the constraint of a low-rank connectivity; specific models of low-rank networks are essentially built by specifying the distribution of these vectors across the population [58,60,61,[64][65][66][67][68].In what follows, we introduce a general model for the distribution of the connectivity vectors.The resulting class of RNNs encompasses most known models of meanfield low-rank networks.Moreover, for fixed rank D, as N → ∞, it corresponds to a field model over a similarity space whose dimension d is not necessarily related to the rank D of the connectivity, i.e. to the dimension of the neural manifold.
A general model of low-rank connectivity.
As we are interested in the limit of large N , let's start from the simple observation that, if one wants to add a neuron labelled i * to the network, one has to specify its connectivity, through the associated vectors ⃗ F i * and ⃗ G i * , as well as its initial condition h i * (0).It is commonly assumed that these features are independent across neurons: intuitively, it means one can add a neuron i * to the network without knowing the { ⃗ F i , ⃗ G i , h i (0)} N i=1 associated with all other neurons.However, for a given neuron i, the components of ⃗ F i and ⃗ G i might be correlated.For the new neuron i * , we thus model these features as being determined by some functional properties of the neuron, sampled from a given distribution.
Formally, we represent these functional properties by the neuron's location z i * , drawn at random according to a probability distribution ρ(dz) on a smooth manifold M of intrinsic dimension d.Given the location z i of any neuron i, the associated connectivity vectors ⃗ F i , ⃗ G i are determined by mappings f , g : M → R D : with f µ , g µ the µ-th components of f , g.This is illustrated on Fig. 2. Analogously, the initial condition is determined by a field h 0 : M → R, according to h i (0) = h 0 (z i ).
Eqs.(1.2),(1.3)define the connectivity kernel w(z, z ′ ) = f (z) • g(z ′ ), such that the recurrent connectivity is given by W ij = w(z i , z j ).The manifold M thus represents the functional similarity space, in which the neurons' locations determine the connectivity.The distribution of the neuronal locations, described by ρ(dz), defines an inner product between functions φ, ψ over M: In the following, we assume that the connectivity functions f µ , g µ (µ = 1, . . ., D), as well as the initial condition h 0 , are continuous and belong to the vector space of ρ-square-integrable functions over M.This last condition defines a Hilbert space H equipped with the ρ-inner product ⟨•, •⟩ ρ .

Corresponding field model.
As N → ∞, the RNN defined by Eqs.(1.1,1.2,1.3) is described by a field model: the membrane potential at time t of a neuron at location z is given by the neural field v t (z), which obeys the field equation with initial condition v 0 (z) = h 0 (z).It is the continuum version of Eq.(1.1) [63,70], where the Poisson noise from the spikes vanishes [61,71]; i.e., S i (t) was replaced by the expected instantaneous rate ϕ(h i (t)).Under suitable regularity and boundedness assumptions concerning the transfer function ϕ, the convergence as N → ∞ of each membrane potential h i (t) to v t (z i ) is guaranteed, with rate 1/ √ N .It follows that neurons become pairwise independent [71,90].
Analogously to a rank-D RNN [60], the dynamics of the neural field can be written as a Ddimensional system [59], as presented here in a simple fashion.The field v t , at any time t, is an element of the Hilbert space H (provided that h 0 , {f µ } µ ⊂ H).The field equation (1.5) can thus be written as an ODE in H: [91] ∂ where (•) denotes the composition of functions, and ⟨•, •⟩ ρ the ρ-inner product (Eq.(1.4)).The field v t undergoes an exponential decay, except in the linear subspace F := span{f µ } ⊂ H.It can thus be written as a linear combination of the basis functions {f µ }, whose coefficients κ µ (t) are the "latent variables": where is the projection of the initial condition h 0 onto the orthogonal complement of F in H.By projection of Eq.(1.7) onto each basis function f µ , the latent variables can be written in vector form as: where we introduced the inverse correlation matrix M , defined by (M −1 ) µν = ⟨f µ , f ν ⟩ ρ (assuming without loss of generality that the functions {f µ } are linearly independent in H).Eqs.(1.6-1.7-1.8)yield D-dimensional collective dynamics for κ, which can be written compactly: under the assumption that h ⊥ 0 = 0.This equation for the collective dynamics (and its more general expression, Eq.(A.13) in the Appendix) is the central ingredient of our theory.The derivation is given in Appendix A.
Neural manifold and attractor states.
Eq.(1.7) implies that the firing rate of each neuron i, given by r i (t) = ϕ(h i (t)) = ϕ(v t (z i )), is parametrised by the latent variables κ; and so is the population activity r(t), which is the joint vector of all N instantaneous firing rates r 1 (t), . . ., r N (t).Besides, r(t) exponentially decays towards the D-dimensional neural manifold generated by the set of all possible latent variables; where F ∈ R N ×D denotes the matrix of connectivity vectors ⃗ F i given by Eq.(1.3), and ϕ is applied element-wise.In the absence of external input, the latent variables κ(t) will converge to the set of attractor states of the collective dynamics of Eq.(1.9), which specifically depends on the considered model.In the Results, we give concrete examples, in which attractor states are either discrete fixed points or limit cycles.

Generalisations: external input and arbitrary connectivity kernel.
This derivation can be readily extended to the case of a network receiving a low-dimensional external input.We assume that the external input is composed of K features I ν (t) (ν = 1, . . ., K), and that the input weight to a neuron i from the input feature I ν is determined by a corresponding function f ext ν (z i ) of the neuron's location z i .The external input it receives can thus be written: This simply results in a drive to the collective dynamics of Eq.(1.9), while expanding their dimensionality -at most up to D + K.The corresponding general expression of the collective dynamics is given by Eq.(A.13) in Appendix A.2.
The derivation of the collective dynamics can also be generalised to the case of field models with arbitrary square-integrable interaction kernel over M 2 .In this case, the collective dynamics are not guaranteed to be finite-dimensional; but the field can always be well approximated by a low-rank model by a truncation of the dimension [59].Both generalisations are given in Appendix A.
2 Similarity space and neural manifold can mismatch

Two different circuit structures generate identical limit-cycle dynamics
We consider two circuit models, with connectivity given by Eqs.( 8)-( 9), which we recall here for convenience.The first model is a ring model, with angular locations z i uniformly distributed on the ring [0, 2π[, and rank-2 connectivity given by: with δ an arbitrary phase shift.In second "planar" model, neurons have two-dimensional locations z i = (z i,1 , z i,2 ) ∈ R 2 normally distributed over the plane; and synaptic weights are given by: 4) where R (δ) is the 2 × 2 matrix of rotation by an angle δ in the plane; and J ′ > 0.
For any initial condition, the latent variables converge to a globally attractive limit cycle with constant angular speed which only depends on τ and δ (given by θ = 1 τ tan δ).This last point generalises to arbitrary transfer function ϕ and any rotationally invariant distribution of the neuronal locations (Appendix B.2).

General interpretation
Heterogeneity as a source of dimensionality mismatch.
We observed that the dimensionality d of the similarity space can be expanded by additional singleneuron properties, that parameterise heterogeneity across the population.As a concrete example, consider the "planar" limit-cycle model above with a two-dimensional similarity space (Eq.(2.4)).In this example, neuronal locations are normally distributed in R 2 .Thus, the location z i of any neuron i can be written in polar coordinates, with an angle φ i (distributed uniformly over the ring) and a radius ξ i (distributed following the χ 2 -distribution, independent from φ i ).The connectivity of Eq.(2.4) then reads W ij = ξ i ξ j w ring (φ i , φ j ), where w ring denotes the connectivity kernel of the ring model (Eq.(2.3)).We now interpret the radius ξ i as a single-neuron property that parameterises an heterogeneity across the population, affecting the connectivity of the original ring model (for which ξ i = 1 for all neurons).Yet, this heterogeneity does not affect the collective dynamics of the model (Appendix B.2). Building on this observation, we found that additional single-neuron properties that parameterise heterogeneity across the population can be incorporated into any circuit model, thus extending the dimensionality d of the similarity space, without affecting the collective dynamics.Simple examples include the multiplication of all outgoing weights of each neuron by a random factor, that describes variations of synapse strength; and heterogeneous maximum firing rates across neurons (Appendix D).

Several universal approximators.
The mismatch between the circuit structure and the collective dynamics can also be understood from another perspective: several circuit models with different similarity spaces can approximate the same collective dynamics.We identified two types of circuit models that are "universal approximators".On the one hand, a Gaussian mixture low-rank network of rank D can approximate arbitrary collective dynamics in a D-dimensional neural manifold [60].This class of models has a functional similarity space of dimension d = 2D (Appendix E.1).On the other hand, field models with a onedimensional similarity space can in principle also approximate arbitrary collective dynamics, if the functions f µ , g µ in Eq.( 3) can be chosen arbitrarily with no requirement on their "smoothness"analogous to the universal approximation property of feed-forward neural networks (Appendix E.2).

Symmetries
A microscopic symmetry of a model can be informally defined as follows.Consider a transformation of the neuronal locations, expressed as a bijection S : M → M over the similarity space that takes each neuron i from location z i to location Sz i .There are two ways in which this mapping can be combined with the microscopic network dynamics of Eq.( 2).Given the set of all membrane potentials at initial time, one can first let the network evolve up to time t and then "move" each neuron i from location z i to Sz i .Alternatively, one can first "move" the neurons and then let the network evolve up to time t.A microscopic symmetry of the model is a transformation S of the neuronal locations for which these two orders of composition are equivalent (Fig. S1).
Analogously, a macroscopic symmetry is a transformation of the latent variables (κ 1 , . . ., κ D ) that commutes with the collective dynamics of Eq.( 6).

A symmetry is a mapping that commutes with the flow
A more precise formulation relies on the concept of flow of the dynamics.Consider that the membrane potential of each neuron is given, at initial time t = 0, by a function v 0 of its location; i.e., h i (0) = v 0 (z i ).The flow of the dynamics transforms this initial condition into the function v t that describes the membrane potentials at time t (i.e.h i (t) = v t (z i )).In other words, the flow is a mapping Φ t such that v t = Φ t [v 0 ] (Fig. S1A, left panels).On the other hand, if one "moves" each neuron from location z i to location Sz i , its membrane potential is now given by the value of the field at its "new" position; i.e. one now has h i (0) = v 0 (Sz i ).In other words, "moving" the neurons is equivalent to a composition of the field with S (Fig. S1A, top panels).As described above, the bijection S is a microscopic symmetry of the model if "moving" the neurons before or after the time evolution is equivalent Likewise, a macroscopic symmetry of the model is a transformation T : R D → R D of the latent variables (κ 1 , . . ., κ D ) that commutes with the flow of the collective dynamics of Eq.( 6) (Fig. S1B).

From microscopic to macroscopic symmetries
We show that any microscopic symmetry S of a field model generates a macroscopic symmetry T (S) of its collective dynamics.For the latter, we give an explicit expression under a reasonable additional assumption.More precisely (Theorem 1 in Appendix C): 1. Let S be (i) a microscopic symmetry of the field model, and (ii) under which the D-dimensional space F = span{f µ } D µ=1 containing the recurrent input is invariant.This is expressed by the condition: f ν • S ∈ F, for all ν = 1, . . ., D. Then, a symmetry of the collective dynamics is given by the linear mapping T (S) , defined by the D × D matrix with M the inverse correlation matrix, defined by (M −1 ) µν = ⟨f µ , f ν ⟩ ρ .2. If S is a bijection for which both the connectivity kernel w and the distribution of the locations ρ are invariant, i.e. w(Sz, Sz ′ ) = w(z, z ′ ) and ρ • S(dz) = ρ(dz), then both requirements (i) and (ii) of point 1. above are satisfied.
A fortiori, any symmetry T (S) of the collective dynamics will be a symmetry of the set of attractor states of the population activity.

Symmetry groups
The mapping T : S → T (S) defines the action on R D of any group G of microscopic symmetries of the model, which will also be a group of macroscopic symmetries.Yet, the order of composition of group elements is reversed: for any elements S, S ′ of G acting on M, the action on R D of their composition S •S ′ is given by: T (S•S ′ ) = T (S ′ ) T (S) .Therefore, a "right" group action on M becomes a "left" group action on R D (Corollary 1 in Appendix C).This will have an effect if the symmetry group G is non-commutative; like, for example, the group of rotations in R 3 .

Extracting the similarity space 4.1 Models of limit-cycle dynamics
As a first illustration of our method for extracting the similarity space from the neuronal activity, we studied two spiking models that generate similar limit-cycle dynamics.SNNs were implemented as in the examples of Fig. 3C-D, with connectivity given by Eqs.( 8) and ( 9); the only difference is that we now consider a sigmoidal transfer function ϕ1 .Since both models yield similar collective dynamics (Appendix B.2), the activity of the two networks generates similar low-dimensional trajectories, that both enter into a limit cycle in a D = 2-dimensional neural manifold (Fig. S2A,B).
To distinguish the two networks from each we characterised the spiking activity of each neuron with two statistical measures, chosen to be the neuron's tuning to each of the two first PCs of the low-passed spiking activity.This yields an two-dimensional embedding of each neuron.The point-cloud of the set of all neuronal embeddings enables us to visualise the functional similarity space (up to an isomorphism) and to distinguish between the two different models (Fig. S2C): for the first SNN, the similarity space is a one-dimensional ring (Fig. S2C1); for the second SNN, it is a two-dimensional surface (Fig. S2C2).For both models, the empirical location of the neurons in the similarity space was then obtained by parameterising their relative positions in this point-cloud.A representation of the neuronal activity as a function of the empirical neuronal locations highlights the underlying spatial structure, which differs between the two models (Fig. S2D).Finally, sorting the neurons with respect to their empirical locations makes oscillatory dynamics clearly visible (Fig. S2E), similar to a recent analysis that revealed slow oscillatory dynamics in the mouse MEC [22].We stress that sorting the neurons is equivalent to arranging them in one-dimensional space; higher-dimensional neuronal embeddings can thus be seen as a generalisation of neuron sorting (Fig. S2,D2).

Models of context-dependent decision-making
We reformulate the task in our framework.In a circuit model, the input weights, i.e., the contribution of the sensory and contextual signals to the external input received by each neuron i, are parameterised by functions of the neuron's location z i , as expressed by Eq.( 2).Similarly, the output is a one-dimensional readout of the spiking activity, with weights determined by the neurons' locations.
For both models, the contribution of the sensory and contextual input features (respectively: I 1 , 'color', I 2 , 'motion', and I c , "context") to the external input received by each neuron i is parameterised by functions f ext 1 (z i ), f ext 2 (z i ) and f c (z i ), respectively.We impose the recurrent and the sensory input to be colinear by setting f ext µ = f µ (µ = 1, 2).The total external input received by each neuron i reads: where I = (I 1 , I 2 ) is the sensory input, and J c is the strength of the context input.During each trial, I 1 (t) and I 2 (t) fluctuate around a fixed mean ± Ī, which is positive or negative with fixed magnitude Ī: i.e.I µ (t) = ± Ī + σξ µ (t), where ξ µ (t) is a Gaussian white noise and σ 2 is the variance of the fluctuations.The context input is unitless and takes binary values I c (t) = ±1 during trials; and the factor 1/τ takes care of the units.Since the maps f 1 , f 2 , f c determine the contribution of the sensory features I 1 , I 2 and context I c to the external input each neuron receives, they can be understood as encoding the neuron's selectivity (or tuning) to these features.

Collective dynamics
We assume symmetric couplings, i.e. g = Jf ; and arbitrary voltage-to-rate function ϕ.For a stationary context input I c (t) = ±1, the collective dynamics write (from their general expression, Eq.(A.13)): Fig. S2 The functional similarity space can extracted from the neuronal activity.Two SNNs of N = 2000 neurons follow similar limit-cycle collective dynamics, even though they have a different cicuit structure (Fig. 3C,D).A. Spike trains of randomly chosen example neurons in the SNN with d = 1, with two neurons i and j highlighted.At any time t, the N -dimensional population activity lies on the neural manifold.In contrast, the spike train of each neuron i depends on its location z i in the similarity space.B. The trajectory of the two SNNs is shown in the space of the first two principal components (PCs) of the low-passed spiking activity ŝ(t) (> 80% variance explained for both networks), after a rotation to superimpose the two trajectories.Both trajectories enter a limit-cycle with the same period T = 195 ms.C. The spiking activity of each neuron was characterised by its tuning to each of the two first principal components (PCs) of the population activity.C1.For the first SNN, the point-cloud formed by the embeddings of all the neurons in the population forms a ring.C2.In constrast, for the second SNN, the set of all two-dimensional embeddings lies on a two-dimensional surface.For both networks, it is consistent with the similarity space of the underlying circuit model (Fig. 3C).D1.Each neuron i of the first SNN is assigned an empirical location ẑi according to its position on the ring-like point-cloud of panel C1.Coloured curves represent the neurons' firing rates as a function of their empirical locations, at several time points (where colour from green to red represents increasing time).The time series of the firing rate of two neurons i and j are highlighted.D2.Each neuron i of the second SNN is assigned a two-dimensional empirical location ẑi according to its position in the point-cloud of panel C2.Heatmaps represent the neurons' momentary firing rates as a function of their empirical locations, at three time points t 1 , t 2 , t 3 (from left to right).Orange dots on the left-most heatmap represent all empirical locations inferred from panel C2.The whole point-cloud is displayed with a 45°rotation compared to panel C2 for convenience.E. Rasterplot of the spiking activity of the second SNN, where neurons were sorted according to the angle φi of their empirical location (panel D2, left).This makes oscillatory dynamics clearly visible, similar to the analysis performed in [22].Red bars correspond to the three time points shown in panel D2.Details about implementation in Methods 4.3.
The connectivity functions f (z), the context selectivity f c (z) and the strength parameters J, J c were chosen such that in the absence of external input (I 1 = I 2 = I c = 0), these dynamics admit no nontrivial fixed point; but in the 'color' context (I c = +1), symmetric attractive fixed points appear along the 'color' axis at κ = (±k, 0), with k > 0 given by the fixed points of Eq.(4.2) with I c = +1.Similarly, the 'motion' context (I c = −1) yields attractive fixed points along the 'motion' axis at κ = (0, ±k), as illustrated in Fig. 4B.More details about the implementation are given below.
This was achieved by setting f c |f 1 | − |f 2 |.In the 'color' context I c = +1, neurons that are more selective to the first input feature ('color'), i.e. for which |f 1 (z i )| > |f 2 (z i )|, are excited; while those that are more selective to the second input feature ('motion') are inhibited; and reciprocally in the 'motion' context At the beginning of each trial, the context input is switched on to I c = ±1, indicating the context.The population activity is then driven towards one of the fixed points by the relevant component of the sensory input (I 1 , 'color' if I c = +1, or I 2 , 'motion' if I c = −1), while ignoring the irrelevant component; and it relaxes towards the baseline state when the context input is switched off, being ready for a new trial.

Choice readout
The goal of the task is to match the output of the network with the sign of the relevant input feature, which is specified by the context.The output is a one-dimensional linear readout of the activity, with output weights f out (z i ): with ŝi (t) the low-passed spike train of neuron i.The map of output weights is set to Thus, the readout is positive for the two fixed points that correspond to a positive relevant sensory input: (+k, 0) in the 'color' context if I 1 > 0, or (0, +k) in the 'motion' context if I 2 > 0. Similarly, the readout is negative for a negative relevant sensory input.

Low-dimensional trajectory of the SNN
Low-dimensional trajectories of the SNNs in Figs.4E and 5B were obtained by projecting the lowpassed N -dimensional spiking activity onto 'color' and 'motion'-relevant axes, defined by (f µ (z i )) N i=1 (with µ = 1 for the 'color' and µ = 2 for the 'motion' axis).Since f out = f 1 + f 2 , the 'choice' readout of Eq.(4.3) is equivalent to the projection of this 2-dimensional population response on the diagonal axis (1, 1) (Fig. 4C,E).

Implementation with two circuit models
The first "blue" model in the main text is a ring model: neurons have angular positions uniformly distributed over [0, 2π[; and the connectivity is given by W ij = J cos(z i − z j ) (as in Eq.( 8), with δ = 0) which is written in rank-2 form using f (z) = (cos z, sin z).
The second "orange" model has a 2-dimensional functional similarity space: neuronal locations are uniformly distributed in the square [−1, 1] 2 (i.e., z i = (z i,1 , z i,2 ) where z i,1 and z i,2 are independent and uniformly distributed in [−1, 1]).The connectivity is the dot product of the locations: we set f (z i ) = z i , so that W ij = J z i • z j .For both models, the voltage-to-rate function was a shifted sigmoid: ϕ(x) = R(1 + tanh(x − 1))/2.In the absence of sensory external input, Eq. (4.2) admits fixed points given by where k ≥ 0 is the solution "blue" ring model: The values of the parameters J and J c were chosen so that the above equations have no solutions k ̸ = 0 for I c = 0 (corresponding to the "baseline" dynamics in Fig. 4B), but have a non-trivial solution k > 0 for |I c | = 1.Solutions were found numerically.The chosen values were J = 0.7, J c = 1.5 for both models (given the value of τ R = 10).
In Fig. 5, K was set near below the critical value K * at which the fixed points with κ 3 = 0 lose their stability.In addition, the network received a weak external input in the direction of κ 3 , of the form where ζ(t) is an Ornstein-Uhlenbeck process with mean 0 and small variance.In the original "blue" model with K = 0, the effect on the dynamics was negligible.In the model with K close to the critical value K * , the additional circuit structure dramatically enhances this weak input, leading to large fluctuations of κ 3 , and thus to a strong dependence of the membrane potentials (and thus of the firing rates) on the task-irrelevant features ξ i .The value of the critical strength (Eq.(4.9)) was computed to be K * ≈ 0.67 for I c = ±1, and K * ≈ 0.48 for I c = 0 (given the parameters τ R = 10, and J = 0.7, J c = 1.5; which yields from Eq.(4.5) k ≈ 1.29 for I c = ±1, and k = 0 for I c = 0).K was then set to 0.4.

Numerical simulations
In all simulations (Figs. 3, 4, 5, S2), the neuronal membrane time constant was set to τ = 10 ms, which defines all characteristic time scales.The maximum firing rate was R = 10/τ .In Figs. 4, 5, S2, spikes were low-passed with a time constant α = τ , and the spike train of each neuron was then z-scored by substracting its mean and dividing by its standard deviation.PCA was performed on the z-scored low-passed activity.
For the decision-making task, analyses were performed on 8 trials that covered all possible combinations of the 3 binary task variables (mean ± Ī of the 'color' and 'motion' inputs, and value ±1 of the 'context' input) (only 4 trials are shown on Figs. 4 and 5).The magnitude of the sensory inputs (see below Eq.(4.1)) was set to Ī = 0.1τ −1 , and their standard deviation to σ = Ī/5.
In Figs. 4 and 5, all three models described above ("blue", "orange" and "pink") received the same low-dimensional input, including the sensory and context features, and also the noisy input with binary random projections ξ i given by Eq.(4.10).The effect on the dynamics was negligible for models whose connectivity was uncorrelated with ξ i ("blue" and "orange" SNNs).Parameters of the O.-U. process ζ(t) were θ = 0.1τ −1 and σ = θ/5.
In Fig. S2, the tuning of each neuron i to the two first PCs was computed as the normalised difference between spike counts of the neuron during periods where each PC was positive or negative: where PCk(t) (k = 1, 2) represents the trajectory of the k-th principal component of the activity, and S i (t) the spike train of neuron i.This choice of statistics allowed one to keep information about the variance of each neuron's activity, which was varying across neurons for the SNN with d = 2.In contrast, this information was collapsed in the PC loadings associated with the z-scored population activity (due to the z-scoring).
Thus, v ⊥ t = e −t/τ h ⊥ 0 , so that Eq.(A.2) yields Eq.(1.7).It allows replacing v t in Eq.(A.5) to obtain, in vector form: which yields Eq.(1.9) if we assume h 0 ∈ F. Eventually, the expression of the latent variables κ can be obtained by projecting Eq.(A.2) on F: ⟨f ν , v t ⟩ ρ = µ ⟨f ν , f µ ⟩ ρ κ µ ; or in vector form: ⟨f , v t ⟩ ρ = Cκ, with the correlation matrix C νµ := ⟨f ν , f µ ⟩ ρ .We can assume without loss of generality that the set of functions {f µ } are linearly independent in H.If that is not the case, a new basis of F with fewer elements can always be chosen (see also the next section A.2). Thus, the matrix C has full rank D, and is thus invertible, yielding Eq.(1.8):

A.2 Including external inputs
Assume that the network receives an external input composed of K features I ν (t) (ν = 1, . . ., K), to each of which the tuning of any neuron i is determined by a corresponding function f ext ν (z i ) of its location z i .The external input received by any neuron can be written: . The total (recurrent + external) input thus now belongs to the linear subspace F ⊂ H spanned by {f µ } D µ=1 ∪ {f ext ν } K ν=1 , of dimension D ≤ D + K. Let { fα } D α=1 be a basis of F. Since any f µ ∈ F is a linear combination of the { fα }, then for each α, there exists a linear combination gα of the feature maps {g µ } such that and similarly, there exists a linear combination Ĩα of the input features {I ν } such that Without loss of generality, we can thus write the total synaptic input received at location z as: From now on, we omit the tildes (•) for readability.The field equation in the case of a lowdimensional external input can be written:

B Examples of collective dynamics B.1 Fixed points in a generalised Hopfield model
We consider here the connectivity mappings given by Eq.(2.2), which read: . Let the neurons be normally distributed over the real line, i.e. z i ∼ N (0, 1).The collective dynamics of Eq.(1.9) write: with Dz the standard Gaussian measure; and we assume for now a general transfer function ϕ.We make the following change of variable: )κ, we have: Notice that: so that Eq.(B.3) can be written, by splitting the integral over R: We now derive further the dynamics for the step transfer function ϕ(x) = R1 {x>0} .Notice that (i) for z > 0, λ 1 z > 0 ⇔ λ 1 > 0; and (ii) for z < 0, λ 2 z > 0 ⇔ λ 2 < 0. The above gives: where + ⟩, and: We can write: They are thus stable fixed points of the dynamics, while λ = 0 is a saddle point.The dynamics of the original variables κ, shown in fig.3(B), are obtained through the change of coordinates κ = (A T ) −1 λ.In particular, the three corresponding fixed points are: and (κ (+) + κ (−) ).The numerical values correspond to Fig. 3(B), where α = π/3 (i.e. a = cos α = 1/2).

B.2 Different models of limit-cycle collective dynamics
Ring model.

Gaussian distribution of neuronal locations.
Let the neurons be normally distributed in R 2 , i.e. z i ∼ N (0, 1 2 ).As discussed in Methods 2, this distribution can be parameterised in polar coordinates: each neuron is independently assigned a radius ξ i ≥ 0 and an angle φ i ∼ U([0, 2π[).The connectivity of Eq.( 9) thus writes: which can be written, as a function of a neuron's radius ξ and angle φ: where f , g are defined by Eq.(B.10).The collective dynamics of Eq.(1.9) thus write: where ρ ξ (dξ) denotes the distribution of the radii (it is the χ 2 distribution if neuronal locations are normally distributed in the plane).In the case of a step transfer function ϕ, we have ϕ(ξx) = ϕ(x) ∀ξ > 0, hence: which is identical to Eq.(B.11) if J ′ ⟨ξ⟩ = J, where ⟨ξ⟩ := ξρ ξ (dξ) (= π/2 for the χ 2distribution).Hence, the two models' collective dynamics are identical up to a rescaling of the connectivity.
Notice that the derivation is not restricted to a normal distribution over R 2 , but applies to any rotationally invariant distribution; in this case, the only difference will be the distribution of the radii ξ i , which will affect the rescaling factor ⟨ξ⟩.

Arbitrary transfer function and radial distribution
Here, we extend the above derivation of the collective dynamics for the connectivity given by Eq.(B.15) to the case of an arbitrary transfer function ϕ and rotationally invariant distribution of the neuronal locations.In this case, the collective dynamics (Eq.(1.9)) corresponding to the connectivity functions of Eq.(B.16) are: where f = (cos, sin) (Eq.(B.10)), and ρ ξ (dξ) is the distribution of the radii of neuronal locations.
For the radial coordinate (ϱ, θ) of κ, it writes: where Φ(ϱ) is defined by (after the change of variable x = cos φ): Provided that the fixed point equation for ϱ, ϱ = τ J cos(δ) Φ(ϱ), (B.25) admits a solution ϱ * , the model generates rotational dynamics that converge towards a limit cycle of radius ϱ * .Noticeably, the angular speed ω of the limit cycle does not depend on the transfer function ϕ, nor on the distribution ρ ξ of the radii, but only on the parameters τ and δ.From Eq.(B.23) and using the definition of ϱ * (Eq.(B.25)), we get

C Symmetries
We show here the results of Sec.2, relating the symmetries of a field model to those of its collective dynamics.We take an operator theory perspective.The dynamics of the field, Eq.(1.6), can be written: where □ = d dt + 1 τ is linear, and D H : H → F (where F = span{f µ }) is a nonlinear operator defined by: D On the other hand, the collective dynamics of Eq.(1.9) are described by a nonlinear operator D D acting on

Defining symmetries
In the following, let S be the set of bijections S : M → M such that φ • S ∈ H, ∀φ ∈ H.For any mapping S ∈ S, the composition operator L (S) : φ → φ • S defines a linear operator on H.We call a (microscopic) symmetry of a field model a bijection S ∈ S such that the operator D H describing the field dynamics commutes with L (S) ; i.e.D H • L (S) = L (S) • D H .It implies the equivariance of the field dynamics for L (S) : if v t solves the dynamics of Eq.(C.1), then L (S) v t is also a solution to the dynamics, with initial condition L (S) v 0 .Analogously, a macroscopic symmetry is a symmetry of the collective dynamics; i.e., a mapping T : R D → R D that commutes with D D .

Results
We show that any microscopic symmetry S ∈ S of the field model generates a macroscopic symmetry T (S) of its collective dynamics, for which we give an explicit expression -under an additional condition that is satisfied by invariance symmetries.Theorem 1.Let S ∈ S.
1.If (i) S is a symmetry of the field model, and (ii) F = span{f µ } is an invariant subspace of L (S) , then a symmetry of the collective dynamics is given by the linear operator T (S) : R D → R D , defined by the matrix where M −1 = (⟨f µ , f ν ⟩ ρ ), and (A µν ) denotes the D × D matrix with entries A µν .
2. If both the distribution ρ and the connectivity kernel w(z, z ′ ) = f (z) • g(z ′ ) are invariant for S, then the conditions (i) and (ii) of point 1 are satisfied.
The proof can be found at the end of this section.
Left and right symmetry groups.Similarly, a left (or right) symmetry group of the collective dynamics is a group with a left (resp., right) action on R D , whose action of any element is a symmetry of the collective dynamics.
Proof of Theorem 1.

Point 1.
Let L be a linear operator on H that satisfies: (i) D H L = LD H , and (ii) F is an invariant subspace of L. We have from (i): K(LD H )V = K(D H L)V (C.9)The left-hand side can be written: µσ = ν M µν ⟨f ν , f σ • S⟩ ρ .We used the linearity of L (S) in the second equality.
Let both ρ and the connectivity w be invariant for S (i.e.w(z, z ′ ) = w(Sz, Sz ′ ) ∀z, z ′ ∈ M).We first show that it implies: D H L (S) = L (S) D H .For all v ∈ H and z ∈ M, we have: where we used Lemma 3 in ( * ), and the invariance of w in ( * * ).
We now show that it implies that F is invariant under L (S) .For this, it is sufficient to show that the images of the basis functions L (S) f µ ∈ F, ∀µ = 1, . . ., D. We have: where ( * ) comes by projection of the functions of z ′ on L (S) g ν ; and G νµ := ⟨L (S) g ν , L (S) g µ ⟩ ρ = ⟨g ν , g µ ⟩ ρ from the invariance of ρ and Lemma 3.
Without loss of generality, one can assume that the functions {g µ } are linearly independent.If not verified, one can always choose a new basis of F such that this is the case.Thus, the matrix G has full rank, and is therefore invertible.The last line of Eq.(C.14) yields: i.e.L (S) f µ ∈ F, ∀µ = 1, . . ., D, which concludes the proof.
Proof of Corollary 1.
In the following, we use the shorthand notation L (Sg) = L (g) and T (Sg) = T (g) .To prove the corollary, it is sufficient to show that T (gh) = T (h) T (g) , ∀g, h ∈ G; and that T (e) = 1 R D for the identity e ∈ G.
By recalling T (S) = KL (S) V , we get: where we used Lemma 2 in the third equality, with V K = Π F (Eq.(C.6)).
Notice eventually that the action on R D of the identity element e ∈ G is the identity: we have S e = 1 M ⇒ L (e) = 1 H ⇒ T (e) = K1 H V = 1 R D according to Eq.(C.6).

Rescaling of outgoing weights
Consider a field model, obeying the collective dynamics of Eq.(1.9), in which the outgoing weights of each neuron i are rescaled by an heterogeneity parameter ξ i > 0, drawn i.i.d.across neurons according to an arbitrary distribution ρ ξ .This results in a "new" field model, where each neuron i is also characterised by its heterogeneity parameter ξ i , corresponding to an additional dimension of the similarity space.We denote each neuron's location in the "new" model zi = (z i , ξ i ) ∈ M × R + (D.1) and the "new" connectivity reads: which allows us to identify f (z i ) = f (z i ), and g(z j ) = ξ j g(z j ).The collective dynamics, Eq.(1.9), of the latent variables κ associated with this "new" model with heterogeneous connectivity write: with ⟨ξ⟩ := ξρ ξ (dξ) the mean value of the heterogeneity parameter ξ.This is identical to the dynamics of the original model (Eq.(1.9)), up to a rescaling of the latent variables by a factor ⟨ξ⟩.

Heterogeneous maximum firing rates
In a similar fashion, starting from an original field model, each neuron's firing rate can be rescaled by a parameter a i > 0 drawn according to a distribution ρ a .The firing rate of any neuron i, given its membrane potential h i , is now given by r i = a i ϕ(h i ).The transfer function φ of the "new" model thus depends on each neuron's "new" location zi = (z i , a i ): we write φ(h, zi ) = a i ϕ(h).Similar to the example above ("Rescaling of outgoing weights"), the collective dynamics of the "new" and of the original field models are identical, up to a rescaling by the mean value of the heterogeneity parameter, ⟨a⟩ = aρ a (da).

E Field models as universal approximators E.1 Gaussian mixture low-rank networks
Gaussian mixture low-rank networks are a class of low-rank RNNs recently introduced to study how the collective dynamics are shaped by the statistics of the connectivity vectors [49,60].In this model, the rank-D interaction between neurons i, j is given by m i • n j , where the connectivity vectors m i , n i ∈ R D associated with each neuron i are drawn according to the Gaussian mixture of P populations: where for each p = 1 . . ., P , ρ (p) is a multivariate Gaussian over R 2D ; and P p=1 α p = 1.Here, each neuron's location z i ∈ R 2D corresponds to the concatenation of the connectivity vectors, and one can directly identify f (m, n) = m, g(m, n) = n.
These RNNs have been shown to be universal approximators of arbitrary D-dimensional collective dynamics, if the number P of populations can be chosen arbitrarily [60].

E.2 Field models over one-dimensional space
For the sake of simplicity, we focus on the case of bounded one-dimensional space M = [0, 1], with uniform distribution ρ(dz) = dz.We furthermore restrict to the subspace of connectivity functions f µ ∈ H that vanish at the boundary, that we denote H 0 .
First of all, biases can be introduced by adding a constant latent variable κ 0 (t) = 1, associated with a "bias" function f 0 (z).Let the set of approximating functions, called AF 0 , be the set of where D µ ∈ C(R D , R) vanish for ∥⃗ κ∥ → ∞.The approximation problem consists in finding functions g and {f µ } so that each given D µ is approximated by some D ∈ AF 0 .As for feed-forward neural networks, this can be decomposed into two parts [94].First, show that approximating functions can be linearly combined: for any D(1) , D(2) ∈ AF 0 and c 1 , c 2 ∈ R, one can find D′ ∈ AF 0 such that D′ = c 1 D(1) + c 2 D(2) .Second, show that elementary functions (like half-plane indicators) can be well approximated.Arbitrary functions can then be approximated using linear combinations of elementary functions.

Fig. 2
Fig. 2 Linking circuit structure to low-rank connectivity.A postsynaptic neuron with location z i and a presynaptic neuron with location z j in the functional similarity space M (left) are mapped by two functions, f , g : M → R D , to two D-dimensional vectors ⃗ F i = f (z i ) and ⃗ G j = g(z j ), respectively.Each entry W ij of the rank-D connectivity matrix (right) is given by the dot product ⃗ F i • ⃗ G j .

Fig. 3
Fig. 3 The dimension d of the functional similarity space and the dimension D of the neural manifold do not always match.A-B.One-dimensional circuit structure generating a two-dimensional neural manifold.A1.Neuronal locations are normally distributed over the real line: the dimension of the similarity space is d = 1.A2.The connectivity of Eq.(7) is given by 2-dimensional functions f and g = Jf : the dimension of the neural manifold is D = 2.The angle of the kink is α = arccos a. B. Flow of the collective dynamics (Eq.(6)) of the circuit model in panel (A) (darker blue-shading indicates faster dynamics).The population trajectories of an SNN with N = 2000 neurons (blue dots), starting from 4 different initial conditions (black stars), follow the flow of the collective dynamics.The units of the axes are k := τ JR √ 2π .C-D.Two different circuit models generating identical limit-cyle collective dynamics.C. Schematic of the distribution of neuron locations: C1 uniform over the ring (d = 1); C2 standard Gaussian in the plane (d = 2).D. Same as panel (B), for the circuit models in panel (C), with the connectivity of Eqs.(8)-(9).The collective dynamics generate a globally attractive limit cycle.The units of the axes are ϱ * := τ JR π cos δ.E. Microscopic symmetries of the circuit structure induce macroscopic symmetries of the collective dynamics.E1.The microscopic symmetry w.r.t. a simultaneous flip of all neuron locations (flip symmetry) in panel (A) causes a reflection symmetry of the collective dynamics in panel (B).E2.The microscopic symmetry w.r.t. a simultaneous shift of all neurons along the ring (translation symmetry) in panel (C1) causes a rotation symmetry of the collective dynamics in panel (D).In panel (B), the SNN leaves the saddle point at the origin towards one of the attractors due to finite-size fluctuations and spike firing noise.The numerical parameters are: τ JR = 10, and a = 1/2 in (A-B), δ = π/10 in (C-D) (Eqs.(8)-(9)).The low-dimensional trajectory of the SNN is obtained from Eq.(1.8): κ SNN (t) := 1

Methods 1 A 22 2 3 27 4
unifying framework: field model of low-rank networks Similarity space and neural manifold can mismatch 25 Symmetries Extracting the similarity space 28 1 A unifying framework: field model of low-rank networks Network model.

Fig
Fig. S1 A symmetry is a mapping commutes with the flow.A. Heatmaps represent the field of instantaneous firing rates, ϕ(vt(z)), over a disc-like, d = 2-dimensional similarity space, at initial time (upper panels) and final time t (lower panels).A microscopic symmetry of a field model is a transformation S : M → M of the neuronal locations that commutes with the flow Φt of the field dynamics up to any time t, i.e. such that the upper and lower path are equivalent.The upper path (right-down) first composes the initial field v 0 with S, then applies Φt, yielding the final state Φt[v 0 • S].The lower path (down-right) first applies Φt, then composes the field with S, yielding the final state vt • S = Φt[v 0 ] • S. B. A macroscopic symmetry of the model is a mapping T : R D → R D of the latent variables that commutes with the flow Ψt of the collective dynamics up to any time t: the final state (purple dot) is the same in the left panel (first apply Ψt, and then T ) and in the right panel (first apply T , and then Ψt).Solid lines represent the trajectory of the latent variables in a D = 2-dimensional neural manifold, from initial time up to time t, starting from the initial condition κ(0) (left panel) or T κ(0) (right panel).

g
′ (z)ϕ ⃗ κ • ⃗ f ′ (z) + f ′ 0 (z) dz =: D′ (⃗ κ) (E.4) denote the action of a group G on M. It is called a left group action if S gh = S g • S h , i.e. S gh (z) = S g (S h (z)), ∀g, h ∈ G (and a right group action if S gh = S h • S g ).G is a symmetry group of the field model if for any g ∈ G, S g is a symmetry.It is a left symmetry group if it has a left group action on M (and respectively, a right symmetry group if it has a right group action).