Neural network model of the primary visual cortex: From functional architecture to lateral connectivity and back

Abstract

The role of intrinsic cortical dynamics is a debatable issue. A recent optical imaging study (Kenet et al., 2003) found that activity patterns similar to orientation maps (OMs), emerge in the primary visual cortex (V1) even in the absence of sensory input, suggesting an intrinsic mechanism of OM activation. To better understand these results and shed light on the intrinsic V1 processing, we suggest a neural network model in which OMs are encoded by the intrinsic lateral connections. The proposed connectivity pattern depends on the preferred orientation and, unlike previous models, on the degree of orientation selectivity of the interconnected neurons. We prove that the network has a ring attractor composed of an approximated version of the OMs. Consequently, OMs emerge spontaneously when the network is presented with an unstructured noisy input. Simulations show that the model can be applied to experimental data and generate realistic OMs. We study a variation of the model with spatially restricted connections, and show that it gives rise to states composed of several OMs. We hypothesize that these states can represent local properties of the visual scene.

1. Introduction

Processing of visual information in the mammalian primary visual cortex (V1) is thought to arise from an interplay between the pattern of external projections and intrinsic cortical dynamics. Even though the intrinsic processing has been a subject of an extensive research, its role and the mechanisms underlying it remain poorly understood. A powerful tool for understanding the intrinsic processing is the study of spontaneous cortical activity, that is, activity not evoked by visual stimulation. Even in the absence of visual stimulation, V1 exhibits a rich and complex spontaneous activity (Lampl et al., 1999; Tsodyks et al., 1999; Kenet et al., 2003; Fiser et al., 2004). This activity is thought to originate from mechanisms intrinsic to V1. Thus, spontaneous activity can reveal the underlying V1 architecture and shed light on its function.

In this study, we focus on the link between spontaneous activity and the activity evoked by visual stimulation. Since the work of Hubel and Wiesel (1959) it is believed that one of the main functions of V1 is to encode and process oriented stimuli. A major progress in studying the functional architecture of V1 was achieved by optical imaging (Blasdel and Salama, 1986; Grinvald et al., 1986). In a widely used experimental protocol (Grinvald et al., 1999), an animal is shown full field moving gratings of different orientations. Each orientation yields a single condition orientation map (OM), a two-dimensional representation of the neuronal activity across the cortical sheet evoked by the stimulus. In cats, as well in many other mammals (including primates, ferrets, tree shrews and sheep, but not rodents; see Van Hooser et al., 2005) neurons with similar orientation preference tend to be clustered. Thus, OMs are characterized by patches of high neuronal activity separated by regions of low neuronal activity. The location and shape of the patches change with the gratings’ orientation.

A recent work (Kenet et al., 2003) used optical imaging with voltage sensitive dye to study spontaneous activity in V1 of an anaesthetized cat, measured when the animal’s eyes were closed. Even though no visual stimulus was presented, some of the spontaneous cortical states were similar to the OMs of the same region. When such spontaneous OMs emerged, they spanned several hypercolumns, and were often followed by states similar to OMs of a proximal orientation. It was speculated that the spontaneous activity reflects the internal state of the brain, which might be influenced by context, attention, or perceptual memory. For example, the spontaneous state can reflect an expectation of the orientation of a forthcoming input, based on prior activity. In any case, the striking similarity between spontaneous and evoked states suggests that there exists a mechanism intrinsic to V1 capable of generating the OMs. If such mechanism exists, the evoked OMs originate from an interplay between the intrinsic mechanism and the pattern of afferent LGN input, known to have some degree of orientation tuning.

What is the intrinsic V1 mechanism underlying the spontaneous emergence of OMs? Many experimental studies suggest an important role for the pattern of lateral connectivity in V1. A large body of anatomical and electrophysiological research has shown that the V1 intracortical connectivity is correlated with orientation preference. The general trend is that long range lateral projections connect neurons with similar orientation preference (Gilbert and Wiesel, 1989; Malach et al., 1993; Weliky et al., 1995; Bosking et al., 1997; Kisvárday et al., 1997; Buzás et al., 1998). As the data is highly variable, this trend is true only on average, and the exact relation between intracortical connectivity, orientation preference, and other functional maps remains a subject of active research (Kisvárday et al., 2002; Ben-Shahar and Zucker, 2004). Direct evidence for the involvement of the lateral connections in orientation tuning comes from studies in which inactivation of GABAergic neurons altered the orientation tuning curves of remotely located neurons (Crook et al., 1998; Kisvárday et al., 2000). Developmental studies in newborn animals have shown that the overall geometry of the OMs is stable from a very early stage, and that it is resistant to manipulations of the visual input (Crair et al., 1998; Löwel et al., 1998). On the other hand, cortical development involves sharpening of the OMs with a time course that closely matches the time course for expression and refinement of long range lateral connections (Gödecke et al., 1997). Developmental experiments in which the visual input was rerouted into the auditory cortex (Sharma et al., 2000), yielded an auditory cortex that was similar, in many aspects, to V1. In particular, it exhibited both OMs and long range lateral projections that preferentially connected cells with similar orientation preference. Taken together, these developmental studies provide further support for the strong link between OMs and lateral connections.

From a theoretical point of view, V1 can be viewed as a complex, nonlinear, dynamical system. Several theoretical studies suggested that OMs are attractor states of the cortical dynamics (Somers et al., 1995; Sompolinsky and Shapley, 1997; Ernst et al., 2001). This line of thought is consistent with the spontaneous emergence of orientation maps because OMs can still be attractors of the intracortical dynamics even without the stimulus-encoding afferent input. In this family of models the pattern of lateral connectivity plays an important role in the formation of the attractor landscape, consistent with the experimental findings listed above. A well studied example of such a model is the ring model (Ben-Yishai et al., 1995; Hansel and Sompolinsky, 1998). This model considers a population of neurons in one hypercolumn in V1. Each neuron is characterized by a preferred orientation, and the overall distribution of preferred orientations is uniform over the entire range of possible orientations. The neurons receive both afferent LGN inputs and lateral recurrent inputs. When a stimulus is presented to the network via the afferent connections, the network develops an activity profile that peaks at the neuron whose preferred orientation matches the stimulus’ orientation. However, the precise shape of the stationary profile is determined, to a large extent, by recurrent connections. The ring model therefore suggests a mechanism that links the cortical response evoked by an oriented stimulus and the pattern of lateral connections.

Moreover, if the recurrent connections are strong enough, states similar to those evoked by a stimulus emerge even if a uniform input (representing lack of an external visual stimulus) is applied. This occurs because the uniform solution becomes unstable, and a continuum of states that are similar to the states evoked by the different stimulus orientations emerges as an attractor of the network’s dynamics. Collectively, these states are known as the ring attractor. Thus, the ring model suggests a mechanism that explains how states similar to OMs can emerge spontaneously. This mechanism was suggested to underlie the patterns of spontaneous activity observed in Kenet et al. (2003).

The ring model, however, cannot directly explain the generation of experimental OMs. It is inherently a one-dimensional model, proposed as a model for one hypercolumn, and cannot account for the generation of the 2-dimensional OMs. One can of course consider a straightforward 2-dimensional implementation of the ring model, by simply placing the neurons on a 2-dimensional sheet (Goldberg et al., 2004); however, the attractor states of such implementation do not necessarily match the experimental OMs. We will further discuss the limitations of the ring model in Section 2.

This leaves the following question open: what type of lateral connectivity can lead to the spontaneous emergence of the OMs? In this contribution we answer this question by suggesting a neural network model with a simple connectivity rule that supports a ring attractor; this ring attractor is composed of states similar to the experimental OMs. To construct this network we first consider an approximation to the OMs (Section 2). Then, we introduce the model and solve it analytically (Section 3). Later, we take an experimental data set, and apply to it the theory we developed in the previous section (Section 4).We continue by studying the effect of the limited spatial extent of the lateral connections (Section 5), and conclude with a discussion of our results and predictions of the models (Section 6).

2. The polar map

As a first step toward constructing the network, we will consider an approximation to the OMs that is based on the “polar map” (PM; Bonhoeffer and Grinvald, 1993). The PM is a functional map that assigns each location x, a complex number, z _x . The values of z _x are typically calculated from the OMs as follows. Let φ ₁, φ ₂,…, φ _p be p equidistant orientations presented to the animal.1 Let S ^φj _x denote the response at location x for the OM evoked by orientation φ _j. Then, z _x is defined by:

$${z_x} \equiv {r_x}{e^{i\theta x}} = {2 \over p}\sum\limits_{j = 1}^p {s_x^{{\varphi _j}}{r^{i{\varphi _j}}}} $$

where $i = \sqrt { - 1}$ Equation (1) can be viewed as a summation of p 2-dimensional vectors whose angle, φ _j , represents the orientation of the stimulus, and their length, S ^φj _x represents the magnitude of the cortical response to that orientation. This view gives rise to the standard interpretation of the angle of z _x , i.e. θ _x , as the preferred orientation at location x. The magnitude of z _x , i.e. r _x , measures the degree to which the response at location x is modulated by the stimulus’ orientation. Following other authors, we will refer to this variable as the selectivity of location x

Throughout the article we will use the data set obtained in Kenet et al. (2003) to demonstrate and validate the theory we develop. This data set contained 8 OMs obtained from an anaesthetized cat using voltage-sensitive dye optical imaging. After a two-fold downsampling of the original data set, each map contained 42 × 17 = 714 pixels, corresponding to an area of approximately 11.7 mm² (the downsampling was preformed so that we can run efficient simulations in the later stages of the analysis). The PM calculated from the 8 OMs using Eq. (1) is depicted in Fig. 1(A). The color coding represents the above polar interpretation of z _x . The color of each pixel represents the preferred orientation, θ _x at that location, and the brightness represents the selectivity, r _x . Pinwheels are represented as dark pixels, i.e. pixels where r _x is close to zero, around which all preferred orientations are represented. A closer examination reveals that when moving from a point far from the pinwheel (“linear zone”) towards the pinwheel, the selectivity is gradually reduced, resulting in a large range of selectivity values. The variability of the selectivities can be also observed in Fig. 1(B), where distribution of the values of z _x over the complex plane is depicted. In this figure each pixel in the PM is represented by a single point. The distance of each point from the origin represents the selectivity of the pixel, and the angle it makes with the real axis represents its preferred orientation. It can be seen that for each preferred orientation there is a wide range of selectivities. This variability is not captured by previous models such as the ring model, which implicitly assume a constant selectivity. As a result, these models are limited in their ability to account for realistic OMs.

Fig. 1

(A) The polar map (PM) for the data obtained in Kenet et al. (2003). The preferred orientation at each pixel is encoded by its color according to the scheme at the bottom of the figure. The selectivity is encoded by brightness (a dark pixel represents low selectivity, a bright pixel represents high selectivity; The brightness scale is linearly stretched between zero selectivity and the maximal selectivity in this PM) (B) Distribution over the complex plane of the values of z _x of the PM in (A)

Equation (1) can also be viewed as the (discrete) Fourier transform of the “tuning curve” of location x, where the term “tuning curve” here refers to the optical imaging response, S ^φj _x , as a function φ _j. Thus, z _x is simply the first Fourier coefficient of the tuning curve of location x. Since these tuning curves tend to be unimodal and centered around the pixel’s preferred orientation, we can expect the first Fourier component to contain much of the tuning curve’s energy. Therefore, the tuning curve of each location can be approximated with this single Fourier component. This approximation can be written explicitly by defining the approximated OM for orientation φ as:

$$M_x^\varphi = {r_x}\cos ({\theta _x} - \varphi )$$

and S ^φj _x ≈ M ^φj _x , based on the arguments presented above. We suggest two ways of measuring the quality of this approximation and apply them to the Kenet et al. (2003) data set. First, the similarity between each of the experimental OMs and its approximated version can be we quantified. A standard measure of similarity between OMs is their Pearson correlation coefficient. For the experimental data set, the average correlation between the 8 pairs of experimental and approximated OMs was 0.815 (min: 0.746, max: 0.887). These are high correlations that validate the approximation. Another measure for the quality of the approximation is the percent of variance explained, which we denote by γ. With r _x defined by Eq. (1), γ can be defined as:

$$\gamma = {{\sum\nolimits_x {{r_x}^2} } \over {2\sum\nolimits_x {{\sigma _x}^2} }}$$

where σ ² _x is the variance of the cortical response over all orientations at location x. The values that γ takes are between 0 and 1, with γ = 1 if and only if the approximation is precise. For the experimental data set, γ = 0.89, again validating the usage of the approximation. We obtained similar results for other data sets as well.

From the above analysis we conclude that experimental OMs can be well-approximated using the PM. For the purpose of this contribution, that is, constructing a neural network with OMs as being its attractor states, this approximation offers two advantages. First, by associating each location with only two real numbers (r _x , θ _x ), rather than the entire tuning curve, the dimensionality of the problem is greatly reduced. Second, the approximation suggests a natural way to interpolate OMs for orientations other than the ones presented during the experiment. We can thus consider a ring of approximated OMs, formally defined by assigning arbitrary values to φ in Eq. (2) Our modeling goal can be now reformulated as constructing a model whose attractor states will coincide with this ring of approximated OMs. This goal is achieved by the model we introduce in the next section.

3. Model

Let m _x (t) denote the average firing rate of the cortical minicolumn at location x at time t. Since OMs tend to be smooth functions of space, we use a continuous approximation and let x take real numbers rather than discrete values. The evolution of m _x is described by the standard rate equation (Wilson and Cowan, 1973):

$$\tau {{\dot m}_x} = - {m_x} + {\left[ {I_x^{{\rm{rec}}} + I_x^{{\rm{aff}}} - T} \right]_ + }$$

where ṁ _x is the time derivative of m _x , I ^rec _x is the input to location x due to the recurrent connections, I ^aff _x is the afferent input to that location, τ is a time constant, T is the firing threshold, [ ]₊ denotes the ramp gain function ([ξ]₊ = ξ if ξ > 0 and [ξ]₊ = 0 otherwise), and the time argument has been suppressed for brevity. The recurrent input, I ^rec _x , is defined by:

$$I_x^{{\rm{rec}}} = {1 \over A}\int {dy{W_{xy}}{m_y}} $$

The integral is taken over the whole region of V1 that is modeled, and A is the area of that region. A is used as a normalization factor that simplifies the mathematical formulations. The synaptic weights connecting locations x and y, W _xy , are defined by:

$${W_{xy}} = {J_2}{r_x}{r_y}\cos ({\theta _x} - {\theta _y}) + {J_0}$$

The information about the PM is given by the term r _x r _y cos(θ _x −θ _y ). The parameter J ₂ > 0, is a global scaling factor of this term. Since scaling the values of r is equivalent to scaling J ₂, we assume that the values of r are normalized so that the mean of r ² _x over the area is 1. The parameter J ₀ represents global excitation (if J ₀ > 0) or global inhibition (if J ₀ < 0). It should be noted that the connectivity given by Eq. (6) is similar to the connectivity of the ring model (Ben-Yishai et al., 1995). However, unlike in the ring model, the connectivity we suggest depends not only on the preferred orientations of the pre- and post- synaptic locations, but also on their selectivities.

The connectivity pattern induced by Eq. (6) is illustrated in Fig. 2(A). The connectivity matrix was calculated using the PM of the Kenet et al. (2003), and setting J ₀ = −2, J ₂ = 5, as used in subsequent simulations. The connectivity strength between one specific location (black dot) and all other locations is visualized by color coding, with shades of green representing positive weights (“excitatory” connections), shades of red representing negative weights (“inhibitory” connections), and white representing connections close to zero. Note the patchy structure of the connectivity pattern, which follows directly from the patchy nature of the PM. Moreover, the “excitatory patches” correspond to regions with a similar orientation preference; Thus, the patchy pattern in Fig. 2(A) reflects the patchy layout of long range connections in V1 found in many anatomical studies (Gilbert and Wiesel, 1989; Malach et al., 1993; Bosking et al., 1997; Kisvárday et al., 1997). However, in contrast with known anatomy, the connectivity does not depend on the distance between the pre-and post-synaptic locations. This is a simplification we make at this point to allow an analytical investigation. Later, in Section 5, we discuss a model with spatially restricted connections and show that it is similar, in many aspects, to the model we consider here. The spatially restricted connections are illustrated in Fig. 2(B), with the corresponding parameters used in subsequent simulations.

Fig. 2

Connectivity pattern of the model. (A) Connection strength between one location, marked by the black dot, and all other locations. The connection strength was calculated by Eq. (6), using the data of Kenet et al. (2003), with J ₀ = −2, J ₂ = 5. (B) Connection strength for the model with spatially restricted connections. The connection strength was calculated by Eq. (48), using the same data set, with J ₀ = 3, J ₂ = 3.5, σ = 0.6 mm

We next consider the system given by Eqs. (4)–(6) for two types of afferent input: one corresponding to spontaneous activity (Section 3.1), and the other representing the afferent input evoked by the gratings stimulus (Section 3.2).

3.1. Spontaneous activity

To model spontaneous activity, we take the afferent input to be constant for all locations, i.e., I ^aff _x . We also assume C > T, otherwise all locations are below threshold, and the network does not develop any activity.

3.1.1. Reduced dynamics

We start the analysis of the model by reducing the dynamics given in (4) to the dynamics of two order parameters. To this end, we observe that the synaptic weights defined by Eq. (6) can be written as J ₂ Re(~z _x z _y )+J ₀, with z _x , z _y defined by Eq. (1). Thus, Eq. (5) can rewritten as:

$$I_x^{{\rm{rec}}} = {J_2}{\mathop{\rm Re}\nolimits} ({{\bar z}_x}{1 \over A}\int {dy} {z_y}{m_y}) + {J_0}{1 \over A}\int {dy{m_y}} $$

This form leads to the following definitions of the order parameters μ and Z:

$$\mu = {1 \over A}\int {dx{\rm{ }}{m_x}} $$

$$Z \equiv \rho {e^{i\psi }} = {1 \over A}\int {dx{r_x}{e^{i{\theta _x}}}{m_x}} $$

These variables represent global properties of the network activity: μ is the average firing rate and Z is the inner product of the firing rates with the PM. Equation (7) can be now rewritten as:

$$I_x^{{\rm{rec}}} = {J_2}{\mathop{\rm Re}\nolimits} ({{\bar z}_x}Z) + {J_0}\mu = {J_2}\rho {r_x}\cos ({\theta _x} - \psi ) + {J_0}\mu $$

To obtain equations for the evolution of μ and Z, we differentiate Eqs. (8) and (9) with respect to time, and then express ṁ _x as a function of μ and Z using Eqs. (4) and (10). With these substitutions, we replace the spatial integration, i.e., the integration over y, with an integration over r and θ. As a result, the following equations are obtained:

$$\tau \dot \mu = - \mu + \int {{{\int {drd\theta {\rm{P}}\left( {r,\theta } \right)\left[ {{J_2}\rho r\left( {\theta - \psi } \right) + {J_0}\mu + C - T} \right]} }_{_ + }}} $$

$$\tau \dot Z = - Z + \int\!\!\!\int {drd\theta {\rm{P}}\left( {r,\theta } \right)r{e^{i\theta }}{{\left[ {{J_2}\rho r\cos \left( {\theta - \psi } \right) + {J_0}\mu + C - T} \right]}_ + }} $$

where the integrals are taken over all possible values of r and θ (i.e., r ∈ [0,∞], θ ∈ [−π, π]) and P(r, θ) is the joint probability density function of r and θ. Equations (11) and (12) are closed equations for the dynamics of μ and Z that do not depend directly on the particular values of m _x . However, in general, these equations still depend on the angle of Z, i.e., ψ. Nevertheless, if isotropy is assumed, that is, the distribution of θ is uniform and independent of the distribution of r, this dependency upon ψ is lost. The validity of these assumptions is discussed later, in Section 4. In this case, we can rewrite Eqs. (11) and (12) as:

$$\tau \dot \mu = - \mu + \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}{{\left[ {{J_2}\rho r\cos \theta + {J_0}\mu + C - T} \right]}_ + }} $$

$$\tau \dot \rho = - \rho + \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}r\cos \theta {{\left[ {{J_2}\rho r\cos \theta + {J_0}\mu + C - T} \right]}_ + }} $$

$$\tau \rho \psi = 0$$

As usual, to find the fixed points for the dynamics, we set the time derivative in Eqs. (13) – (15) to zero. We discuss uniform (with ρ = 0), and non-uniform (with ρ > 0) solutions separately.

3.1.2. Uniform solutions

In the case of a uniform solution, Eq. (14) implies that the time derivative of ρ is zero for any μ. Equation (13) implies $\mu = {\left[ {{J_0}\mu + C - T} \right]_ + }$ which can be solved for any J ₀ < 1. The solution for the uniform fixed point is therefore:

$$\mu = {{C - T} \over {1 - {J_0}}}$$

$$\rho = 0$$

The stability analysis is straightforward, as all neurons are above threshold, and the system near the fixed point is linear. We find that the fixed point is stable in the region J ₀ < 1 and J ₂ < 2.

3.1.3. Non-uniform solutions

To find the non-uniform solutions, we pull ρ out of the integrals in Eqs. (13) and (14) and rewrite them as:

$$\tau \dot \mu = - \mu + {J_2}\rho {F_0}\left( X \right)$$

)

$$\tau \dot \rho = - \rho + {J_2}\rho {F_2}\left( X \right)$$

with,

$$X = {{{J_0}\mu + C - T} \over {{J_2}\rho }}$$

and,

$${F_0}\left( X \right) = \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}{{\left[ {r\cos \theta + X} \right]}_ + }} $$

$${F_2}\left( X \right) = \int\!\!\!\int {drd\theta {{{\rm{P}}\left( r \right)} \over {2\pi }}r\cos \theta {{\left[ {r\cos \theta + X} \right]}_ + }} $$

In order to find the fixed point solutions for μ and ρ using Eqs. (18) and (19), it is necessary to know the functions F ₀ and F ₂ that in turn depend on the concrete choice of the selectivities’ distribution P(r) (Eqs. (21) and (22)). To study the properties of these functions, we first plot them for 3 different choices of P(r) (Fig. 3). In these examples, we see that the precise values that F ₀ and F ₂ take indeed depend on P(r). However, in all 3 cases, F ₀ is a non-decreasing function of X, approaching zero for large negative X, and increasing with a slope of 1 for large positive X. Similarly, in all 3 cases, F ₂ is a sigmoid-shaped function that saturates at 0 and 1/2. In fact, these observations about F ₀ and F ₂ are true in the general case. Formally, we show a set of properties of F ₀ and F ₂ that hold for any P(r). These properties are listed in Table 1, and their proof is given in the appendix. As we will show, they allow us to obtain solutions for the fixed point values of μ and ρ for an arbitrary P(r).

Fig. 3

The functions F ₀ and F ₂. (A) F ₀ plotted for 3 choices of P(r); Dashed: constant r, P(r) = δ(r − 1). Dash-dotted: exponential distribution, ${\rm{P}}(r) = {\sqrt {2e}^{ - \sqrt {2r} }}$. Solid: P(r) according to the isotropy-adjusted experimental PM (see Section 4 and Fig. 7 for details). (B) F ₂ plotted for the same distributions of r

Table 1

We first find the fixed point value of X by setting the time derivative in Eq. (19) to zero. We find that the fixed point value of X is defined by:

$$1 - {J_2}{F_2}\left( X \right) = 0$$

Property (vi) in Table 1 directly implies that when J ₂ < 2, Eq. (23) cannot be satisfied, therefore a non-uniform solution does not exist (see also Fig. 4(C)). On the other hand, properties (v), (vi), (vii) imply that when J ₂ > 2 Eq. (23) has a unique solution (Fig. 4(D)). Therefore, when J ₂ > 2 there is a unique solution for the fixed point value of X. Assuming this solution is known, the following expressions for the fixed point value of µ and ρ in terms of the fixed point value of X can be derived form Eqs. (18), (20), and (23):

$$\mu = {{C - T} \over {X - {J_0}{F_0}\left( X \right)}}{F_0}\left( X \right)$$

$$\rho = {{C - T} \over {X - {J_0}{F_0}\left( X \right)}}{F_2}\left( X \right)$$

Since μ and ρ are by definition positive, it must hold that:

$$X - {J_0}{F_0}\left( X \right) > 0$$

otherwise, the solution diverges. Again, we can use the properties in Table 1 to find the conditions under which this inequality is satisfied. Property (ii) implies that inequality (26) never holds when J ₀ > 1 (Fig. 4(A)). On the other hand, if J ₀ < 1, X − J ₀ F ₀(X) is a monotonically increasing function of X (properties (iii), (iv)), taking negative values for large negative values of X, and positive values for large positive values of X (properties (i), (ii)). Therefore, when J ₀ < 1, X − J ₀ F ₀(X) = 0 has a unique solution. We denote this solution by X ₀ (Fig. 4(B)), and the solution of Eq. (23) by X ₂ (Fig. 4(D)). Using these notations, we find that inequality (26) holds for X = X ₂ if and only if X ₀ < X ₂. Since X ₀ is a function of J ₀ and X ₂ is a function of J ₂, the condition X ₀ < X ₂, together with the conditions J ₀ < 1, J ₂ > 2, define a region in the J ₀ J ₂ plane, where the non-uniform fixed point exists. Stability analysis given in the appendix shows that when this fixed point exists, it is always stable.

Fig. 4

Solution and phase diagram of the spontaneous activity model. (A, B) The function X − J ₀ F ₀(X) (see Eq. (26)). In (A), J ₀ = 1.5, representing the case where J ₀ > 1. The function X − J ₀ F ₀(X) is negative for all X. In (B), J ₀ = −2, representing J ₀ < 1. The function X − J ₀ F ₀(X) crosses zero at a unique point, which is denoted by X ₀. (C, D) The function 1 − J ₂ F ₂(X) (see Eq. (23)). In (C) J ₂ = 1.5, representing the case where J ₂ < 2. The function 1 − J ₂ F ₂(X) is positive for all X. In (D) J ₂ = 2.5 representing J ₂ > 2. The function 1 − J ₂ F ₂(X) crosses zero at a single point, which is denoted by X ₂. (E) Phase diagram for the spontaneous activity. The border of the linear phase is given by the lines J ₀ = 1, J ₂ = 2. The line separating the marginal phase from the amplitude instability is determined by the condition X ₀ = X ₂. In all examples, F ₀ and F ₂ were calculated using the experimental P(r) (solid lines in Fig. 3)

3.1.4. Phase diagram

We turn to summarize the different regimes of the model and relate the fixed points in these regimes to the OMs. Since the voltage-sensitive dye optical imaging signal corresponds to the instantaneous post synaptic membrane potential rather than firing rate (Sterkin et al., 1999; Grinvald et al., 1999; Sharon and Grinvald, 2002; Petersen et al., 2003), we compare OMs to the total synaptic input, denoted by I ^tot = I ^rec + I ^aff. The shape of I ^tot is given by (see Eq. (10)):

$$I_x^{{\rm{tot}}} = {J_2}\rho {r_x}\cos ({\theta _x} - \psi ) + {J_0}\mu + C$$

with μ, ρ, and ψ taking their fixed point values. The different phases of the model are summarized in the phase diagram (Fig. 4(E)). In the linear phase, J ₀ < 1, J ₂ < 2, the only solution is ρ = 0, and it is stable. In this case, Eq. (27) implies that I ^tot _x is uniform and does not correspond to any OM. However, in the marginal phase, J ₀ < 1, J ₂ > 2, X ₀ < X ₂ (where X ₀ is a function of J ₀ and X ₂ is a function of J ₂), the solution with ρ = 0 is unstable, but solutions withρ > 0 are stable. In fact, there are infinitely many solutions, because the value of ψ is arbitrary (Eq. (15)). By comparing Eq. (27) to Eq. (2), we see that in this case I ^tot _x equals a scaled version of the approximated OM of orientation ψ, up to an additive constant shift. Thus, in the marginal phase, OMs of arbitrary orientations emerge as fixed points of the network dynamics. Collectively, these fixed points form a ring attractor, that is, a ring of states where each state is an attractor of the dynamics.

Outside the regions of linear phase and marginal phase no stable fixed point exists and the network develops amplitude instability. Unlike the boundaries of the linear phase, the line X ₀ = X ₂ that separates the marginal phase from the amplitude instability region depends on P(r). However, some of its properties do not depend on P(r). In particular it can be checked that it always passes through the points (J ₀ = 1, J ₂ = 2), and (J ₀ = 0, J ₂ = 4).

3.2. Evoked activity

The case of a non-uniform input is more complex than the uniform input case, because the steady state solutions are sensitive to the fine structure of the input. However, some general observations about the fixed point can still be made. Since the synaptic input I ^rec is still given by Eq. (10), it equals a scaled/shifted version of one of the approximated OMs, and the total synaptic input is the sum of that map, and the afferent input, I ^aff. The orientation of the approximated OM, and its scaling and translation parameters are still determined by ψ, ρ, and μ. Thus, the role of I ^aff reduces to selecting the values of μ, ρ, and ψ, which will put the system in a steady state.

In this section we study a simple structure of the input; we add to the uniform input we studied in the previous section a small modulation that is proportional to one of the approximated OM. Formally we define:

$$I_x^{{\rm{aff}}} = C(1 + \varepsilon {r_x}\cos ({\theta _x} - {\psi _{{\rm{aff}}}}))$$

where ψ _aff is the orientation encoded in the input and ɛ is the modulation of the orientation encoding term. With this form, each location receives maximal afferent input when the orientation of the stimulus coincides with its preferred orientation. The main motivation for choosing this particular form is that it allows a relatively simple analytical study. Using simulations, we also studied different variations of the above input, and obtained essentially the same results. An example of such variation, is given in Section 4.

3.2.1. Reduced dynamics

Rewriting the right-hand side of Eq. (28) as $C + {\mathop{\rm Re}\nolimits} ({{\bar z}_x}{Z_{{\rm{aff}}}})$ with ${Z_{{\rm{aff}}}} = C\varepsilon {e^{i{\psi _{{\rm{aff}}}}}}$, we find that:

$$I_x^{{\rm{rec}}} + I_x^{{\rm{aff}}} = {\mathop{\rm Re}\nolimits} ({{\bar z}_x}({J_2}Z + {Z_{{\rm{aff}}}})) + {J_0}\mu + C$$

where μ and Z are defined by the same equations given for the spontaneous activity (Eqs. (8) and (9)). Equation (29) leads to the following definition:

$${Z_{{\rm{tot}}}} = {\rho _{{\rm{tot}}}}{e^{i{\psi _{{\rm{tot}}}}}} = {J_2}Z + {Z_{{\rm{aff}}}}$$

and with these notations:

$$\tau {{\dot m}_x} = - {m_x} + {[{\rho _{{\rm{tot}}}}{r_x}\cos ({\theta _x} - {\psi _{{\rm{tot}}}}) + {J_0}\mu + C - T]_ + }$$

As in the case of spontaneous activity, Eq. (31) can be used to derive dynamical equations for μ and Z:

$$\tau \dot Z = - Z + {e^{i{\psi _{{\rm{tot}}}}}}\int\!\!\!\int {drd\theta {{{\rm{P}}(r)} \over {2\pi }}r{e^{i(\theta - {\psi _{{\rm{tot}}}})}}{{[{\rho _{{\rm{tot}}}}r\cos (\theta - {\psi _{{\rm{tot}}}}) + {J_0}\mu + C - T]}_ + }} $$

$$\tau \dot \mu = - \mu + \int\!\!\!\int {drd\theta {{{\rm{P}}(r)} \over {2\pi }}{{[{\rho _{{\rm{tot}}}}r\cos (\theta - {\psi _{{\rm{tot}}}}) + {J_0}\mu + C - T]}_ + }} $$

This can be rewritten as:

$$\tau \dot \mu = - \mu + {\rho _{{\rm{tot}}}}{F_0}(X)$$

$$\tau \dot \rho = - \rho + {\rho _{{\rm{tot}}}}\cos ({\psi _{{\rm{tot}}}} - \psi ){F_2}(X)$$

$$\tau \rho \dot \psi = {\rho _{{\rm{tot}}}}\sin ({\psi _{{\rm{tot}}}} - \psi ){F_2}(X)$$

with,

$$X = {{{J_0}\mu + C - T} \over {{\rho _{{\rm{tot}}}}}}$$

In this formulation, it is assumed that ρ _tot > 0. However it can be easily checked that ρ _tot = 0 is not a fixed point solution, therefore the case of the uniform solution can be disregarded.

3.2.2. Solution

At fixed point, Eq. (36) implies that sin(ψ _tot − ψ) = 0. Therefore, it must hold that either ψ _tot = ψ or ψ _tot = ψ + π. However, in the second case, Eq. (35) implies ρ = −ρ _tot F ₂(X), which cannot hold because both ρ and ρtot are positive, and F ₂(X) is non-negative (property (v) in Table 1). Thus, as follows from Eq. (30), the possible solutions are ψ _tot = ψ = ψ _aff, and ψ _tot = ψ = ψ _aff + π. We focus on the first case, which corresponds, as the analysis in the appendix shows, to the only stable fixed point. In this case, ρ _tot = J ₂ ρ + C∈. Setting $\dot \mu = 0$, $\dot \rho = 0$ we rewrite Eqs. (34), (35), (37) as:

$$\mu = ({J_2}\rho + C\varepsilon ){F_0}(X)$$

$$\rho = ({J_2}\rho + C\varepsilon ){F_2}(X)$$

$$X = {{{J_0}\mu + C - T} \over {({J_2}\rho + C\varepsilon )}}$$

Thus, a fixed point requires that this system of 3 equations for the 3 variables μ, ρ, and X has a solution with positive μ and ρ. Solving this system, we find that the fixed point value of X is determined by:

$${{1 - {J_2}{F_2}(X)} \over {X - {J_0}{F_0}(X)}} = {\rm{\gamma }}$$

where ϒ is the effective stimulus tuning (Hansel and Sompolinsky, 1998), defined by:

$${\rm{\gamma }} = {{\varepsilon C} \over {C - T}}$$

The solution for μ and ρ is given by the same equations we derived for the spontaneous activity (Eqs. (24) and (25)), and, as it was in that case, these equations imply that inequality (26) must hold. In addition, in the evoked case, because ϒ > 0, it must also hold that:

$$1 - {J_2}{F_2}(X) > 0$$

Taken together, a fixed point solution requires finding X that satisfies Eq. (41) and inequalities (26), (43). Thus, we check the existence of such a solution in the different regions of the J ₀ J ₂ plane.

As in the case of the spontaneous activity, when J ₀ > 1, inequality (26) never holds, and no fixed point exists. In the region corresponding to the linear phase of the spontaneous activity, i.e., J ₀ < 1, J ₂ < 2, inequality (43) holds for any X, and inequality (26) holds for any X > X ₀, where X ₀ is a function of J ₀ (Section 3.1.3). On the interval X > X ₀ the left hand side of (41) is a monotonically decreasing function of X, approaching ∞ at X = X ₀, and 0 at X → ∞ (Fig. 5(A)). Therefore, in this region, Eq. (41) has a unique solution for any (positive) ϒ, and a unique fixed point exists.

Fig. 5

Solution and phase diagram of the evoked activity model. (A, B) The function ${{1 - {J_2}{F_2}(X)} \over {X - {J_0}{F_0}(X)}}$ (see Eq. (41)). In (A) J ₀ = −3, J ₂ = 1.5, representing the case where J ₀ < 1 and J ₂ < 2. On the interval X > X ₀ the function takes all possible positive values, and a unique solution for X exists for every value of ϒ. In (B) J ₀ = −3, J ₂ = 3.5, representing the case where J ₀ < 1, J ₂ > 2, and X ₀ < X ₂. On the interval X ₀ < X < X ₂ the function takes all possible positive values, and a unique solution for X exists for every value of ϒ as well. (C) Phase diagram for the evoked activity. The line separating the linear phase from the non-linear phase is plotted for few values of ϒ (0.05, 0.1, 0.15, 0.2, 0.25, bottom-right to top-left). In all examples, F ₀ and F ₂ were calculated with the experimental P(r) (solid lines in Fig. 3)

In the region corresponding to the marginal phase of the activity, i.e., J ₀ < 1, J ₂ > 2, X ₀ < X ₂, where X ₀ is a function of J ₀ and X ₂ is a function of J ₂, (Section 3.1.3) inequalities (26) and (43) imply X ₀ < X < X ₂. On the interval X ₀ < X < X ₂, the left hand side of Eq. (41) is monotonically decreasing of X, approaching ∞ at X = X ₀, and taking the value of 0 at X = X ₂ (Fig. 5(B)). Therefore, in this region, Eq. (41) has a unique solution for any (positive) ϒ, and a unique fixed point exists. Finally, in the region J ₀ < 1, J ₂ > 2, X ₀ > X ₂, inequalities (26) and (43) cannot be satisfied simultaneously, therefore, no fixed point exists.

3.2.3. Phase diagram

In sum, the above analysis shows that the region where a fixed point exists for the evoked activity, coincides precisely with the regions where a fixed point exists for the spontaneous activity, i.e., the regions of the linear and marginal phases of the spontaneous activity. The stability analysis in the appendix shows that this fixed point is stable, and that any other fixed point in this region is unstable. Similar arguments to those presented for the marginal phase of the spontaneous activity imply that the total synaptic current is a scaled/shifted version of an OM. However, unlike the case of the spontaneous activity where the orientation of the OM was arbitrary, for the evoked activity, the orientation of the OM is bound to match the orientation of the afferent input.

Although I ^tot is always a scaled/shifted OM, when firing rates are taken into account, we find that two types of solutions exist. The fixed point solution for the firing rate is given by:

$${m_x} = {\rho _{{\rm{tot}}}}{[{r_x}\cos ({\theta _x} - {\psi _{{\rm{aff}}}}) + X]_ + }$$

Thus, if the selectivities (r _x ) are bounded, i.e., there exists r _max with P(r > r _max) = 0, and X is large enough, all locations are above threshold, and the system at fixed point is effectively linear. Otherwise, at least some locations are below threshold, and the system is non-linear. This distinction between linear and non linear solutions is important because linear solutions are associated with broad tuning curves, whereas non-linear solutions are associated with narrow tuning curves (Hansel and Sompolinsky, 1998). If J ₂ > 2, the system is non-linear for any ϒ, because the fixed point solution for X is smaller than X ₂, which in turn is smaller that r _max (property (vi)). However, if J ₂ < 2, the fixed point solution for X can take arbitrarily large values, and both linear and non-linear solutions are possible. The line separating the linear regime from the non linear one for the evoked case, can be obtained by plugging X = r _max in Eq. (41). Using the fact that F ₀(r _max) = r _max and F ₂(r _max) = ½ (properties (ii), (vi)) we find that:

$${J_0} - 1 = {{{J_2} - 2} \over {2{\rm{\gamma }}{r_{\max }}}}$$

Thus, we find that this line always passes through the point (J{nu0} = 1, J ₂ = 2) and its slope (where the line is viewed as a function of J ₂) is inversely proportional to ϒ and r _max. This line for a few choices of ϒ is drawn in the phase diagram of the evoked activity (Fig. 5(C)). In the limit ϒ → 0, this line becomes J ₂ = 2, i.e., the line separating the linear phase from the marginal phase of the spontaneous activity. On the other hand, when ϒ → g8 the system switches from linear solutions to non-linear ones, for any choice of J ₀ and J ₂.

3.3. Energy function

The connectivity matrix of our model is symmetric; therefore, an energy function for the model can be constructed. The energy function decreases during the evolution of the system, and the fixed points are local minima of the energy function. The appropriate energy function is given by Hopfield, (1984) and Hertz et al. (1991):

$$H = - {1 \over {2{A^2}}}\int\!\!\!\int {dxdy{W_{xy}}{R_x}{R_y} + {1 \over {2A}}\int {dxR_x^2} } - {1 \over A}\int {dx{R_x}(I_x^{{\rm{aff}}}} - T)$$

where ${R_x} = {[I_x^{{\rm{rec}}} + I_x^{{\rm{aff}}} - T]_ + }$. Using a derivation similar to the one we used to reduce the dynamics of the system to the dynamics of the order parameters, Eq. (46) can be rewritten as a function of the order parameters μ and Z:

$$H = - {1 \over 2}\rho _{{\rm{tot}}}^2({J_0}{F_0}{(X)^2} + {J_2}{F_2}{(X)^2}) + {1 \over 2}({\rho _{{\rm{tot}}}}({J_0}\mu - C + T){F_0}(X) + ({({J_2}\rho )^2} - {(\varepsilon C)^2}{F_2}(X))$$

with ρ _tot and X being functions of μ and Z, according to the definitions given for the evoked activity (Eqs. (30) and (37)), and the spontaneous activity treated as a particular case with ∈ = 0.

Usually, energy functions of neural networks are high dimensional and cannot be directly visualized. However, Eq. (47) reveals that the energy function for this system is a function of one complex and one real variable (Z and μ). In the case of J ₀ = 0 the energy is a function of Z only, and therefore can be easily visualized. Figure 6 summarizes the main results we obtained in previous sections, by showing how the energy function looks like in the different regimes of the system. In Fig. 6(A)–(C) the energy is drawn as a function of Z for spontaneous activity. In Fig. 6(A), the model is in the linear phase, and the energy function is similar to a cone. Only one local minimum exists and it is at the center of the “cone”, i.e., at Z = 0. This local minimum corresponds to the uniform solution with ρ = 0. In Fig. 6(B), the model is in the marginal phase, and the energy function is similar to a “mexican hat”. The ring of local minima corresponds to the infinite number of solutions with ρ > 0. In Fig. 6(C), no local minima exist, and the system undergoes amplitude instability. In Fig. 6(D), the case of evoked activity is depicted. The energy function is similar to the energy function of the spontaneous activity in the marginal phase, Fig. (6B), but “tilted” in the direction defined by the angle ψ _aff. Thus, the continuum of local minima is lost, and only one local minimum remains. While the afferent input we used to generate Fig. 6(D) is of the type we studied in Section 3.2, qualitatively similar results can be obtained for many other types of inputs. This is because the ring attractor, as depicted in Fig. 6(B), is singular; any small anisotropy in the afferent input will destroy its prefect symmetry, and will cause the network to converge to particular values of Z, representing specific OMs.

Fig. 6

Energy function. In the examples given in this figure we assume J ₀ = 0, and the energy, H, depends only on the complex order parameter Z. (A, B, C) The energy as a function of Z for spontaneous activity. (A) J ₂ = 1, linear phase. (B) J ₂ = 3, marginal phase. (C) J ₂ = 5, amplitude instability. (D) The energy as a function of Z for evoked activity with J ₂ = 3, ∈ = 0.04. In all examples F ₀ and F ₂ were calculated with the experimental P(r) (solid lines in Fig. 3). The values of other parameters were C = 3, T = 1. For clarity the energy was drawn only for values of Z with |Z| < 3.3

4. Simulations

To illustrate the theoretical results obtained in the previous section and validate their applicability to experimental data, we applied the model to the Kenet et al. (2003) data set. An important assumption we made in the previous section is that of isotropy of orientation preference (Section 3.1.1). The isotropy assumption requires the distribution of the values of z _x to be rotationally symmetric about the origin. The experimental distribution of z _x , which is shown in Fig. 1(B), does not seem to deviate strongly form the rotational symmetry assumption. This is consistent with some previous optical imaging studies in cats that either did not find any systematic over-representations of specific orientations (Bonhoeffer and Grinvald, 1993; Kenet et al., 2003), or found small, albeit significant in some cases, over representation of cardinal orientations (Müller et al., 2000; Yu and Shou, 2000; Dragoi et al., 2001b; Wang et al., 2003). However, there are many factors that might distort the precise distribution and mask the true extent of the anisotropy. In particular, the fact that only a small fraction of V1 is imaged can increase the anisotropy in the imaged map compared to the true one, whereas measurement noise of the optical imaging data, can artificially induce an opposite effect.

In this work, we avoid the precise characterization of the anisotropy in the cat data. Instead, we first adjusted the polar map (PM) to match the isotropy assumption, and then quantified the deviation between the experimental and the adjusted PMs. The adjustment was performed in two steps. In the first step, we adjusted the values of r _x such that for any angle θ _x the conditional distribution of r _x given θ _x will match, approximately, the marginal distribution of r _x . This was achieved by considering for every pixel x, the set of pixels ⨑ub;y⫂ub; such that $\left| {\left. {{\theta _x} - {\theta _y}} \right| \le {\pi \over {10}}} \right.$. Next, we calculated p _x , defined as the percent of pixels with r _y ≤ r _x . Then, r _x was set to the p _x -th percentile of the distribution of the original values of r _x of all the pixels. In the second step, the pixels were sorted according to the value of r _x and divided into 12 groups of approximately equal size. The values of r at each group were set to the average value at the group. Then, the pixels at each group were sorted according to θ _x , and the values of θ _x were adjusted such that difference between two adjacent θ _s was made constant. The distribution of the values of z _x after the adjustment is shown in Fig. 7(B). This distribution is rotationally symmetric, and meets the isotropy assumption.

Fig. 7

(A) PM for the data obtained in Kenet et al. (2003) after isotropy adjustment. The figure is displayed on the same brightness scale as in 1A to facilitate comparison. (B) Distribution over the complex plane of the values of z _x of the PM in (A)

The PM after isotropy adjustment is shown in Fig. 7(A) and it is very similar to the experimental PM shown in Fig. 1(A). Since our main motivation of using the PM is its ability to provide an approximation to the OMs (Eq. (2)), we quantified the distortion caused by the isotropy adjustment by calculating for every orientation the correlation between the OM approximated with the experimental PM and the OM approximated with the adjusted PM. The average correlation over all possible orientations was 0.971 (min: 0.968, max: 0.973). These high correlations indicate that the original map was not very far from meeting the isotropy assumption in the first place. The average correlation between the 8 experimental OMs and the OMs approximated with the adjusted PM was high as well (0.791; min: 0.725, max: 0.855). We therefore used the adjusted PM to construct the connectivity matrix (Eq. (6)) and tested the model in the spontaneous activity and the evoked activity regimes.

Our analysis of the spontaneous activity in the marginal phase showed that the steady state synaptic input (I ^rec + C) is a scaled/shifted version of an approximated OM. The orientation of theOMis arbitrary and therefore depends only on the initial state, i.e., m _x at t = 0. We thus performed 10000 simulations with random initial states. The simulations were performed using MATLAB®, using first order Euler method to solve Eq. (4). Each simulation was let run until convergence. The parameters of the model were fixed for all simulations, with J ₀ and J ₂ selected to be in the marginal phase (see Fig. 8 for a full list of parameters’ values).

Fig. 8

Simulations of spontaneous activity. (A) Initial state in one simulation. (B) The total synaptic input at steady state for the same simulation. (C) The experimental OM of 90°, which is very similar to the state in (B). (D) Distribution of the orientations of the approximated OMs at steady state in 10000 simulations. The number of bins was 8. The following values of parameters were used in all simulations: J ₀ = −2, J ₂ = 5, τ = 10, C = 2, T = 1. The random initial state was taken from a gaussian distribution with mean 1 and variance 0.25. Equation (4) was solved using the first order Euler method with Δt = 1. The simulation was run until t = 500, a value that was large enough to guarantee the convergence to a fixed point. Grayscale range (arbitrary units): (A) 0.2–2, (B) −9–7, (C) −2.4–2.7

An example of a single simulation is presented in Fig. 8(A)–8(C). Figure 8(A) shows the initial state. Figure. 8(B) shows the total synaptic input, I ^rec + C, at the fixed point. It is a linear transformation of the approximated OM of 90°. The highly similar experimental OM of 90° is presented in Fig. 8(C) for comparison (correlation = 0.85). The steady states of all simulations where similar to one of the experimental OMs (correlation >0.7). The high similarity between the simulated and experimental maps demonstrates that the model produced a good approximation for the experimental OM. It also implies that any violations of the approximation assumptions, (ability to approximate OMs with the PM and isotropy) did not cause any significant distortion of the OM. In Fig. 8(D) we show the distribution of the orientations of the steady state OMs for all 10000 simulations. The orientation was calculated by taking the angle of the projection of the synaptic input at the fixed point on the PM. Consistently with our theoretical analysis, this figure shows that the OM at the fixed point was arbitrary, with equal probability for all orientations.

We next tested the model with a tuned input, with ∈ = 0.1 and ψ _aff selected randomly from a uniform distribution. In order to test the model with a more complex afferent input than the one we analyzed in the previous section, we added to the afferent input gaussian noise with zero mean and a standard deviation of 0.1. Thus, noise in the afferent input was equal in amplitude to ∈. The other parameters of the simulation were identical to those of the spontaneous activity simulation, and the simulation was also performed 10000 times. The results are shown in Fig. 9. Figures 9(A)–(C) showan example of a single simulation. Figure 9(A) shows the afferent input. The orientation used to construct the input was 0°. The steady state is shown in Fig. 9(B). It is very similar to the experimental OM of 0° degrees which is shown in 9C. Note that the noise we added in the afferent input is filtered out by the recurrent connections. Finally, in Fig. 9(D), we show the distribution of the difference between the afferent input orientation and the steady state orientation. The narrow distribution around zero (SD = 2.2°) shows that the input orientation and the steady state orientation are very close to one another despite the noise. A comparison of Fig. 8(D) and Fig. 9(D) demonstrates the main difference between the spontaneous and evoked conditions in the marginal phase. In the former, the OM is chosen randomly whereas in the latter, it is chosen by the input. However, in both cases, the steady states are similar to OMs.

Fig. 9

Simulations of evoked activity. (A) Afferent input in one simulation. The input is constant, with a small modulation in the shape of the approximated OM of 0°(∈ = 0.1,ψ _aff = 0) and a random gaussian noise (mean= 0, std= 0.1). (B) The total synaptic input at steady state for the same simulation. The 0δ OM is extracted from the afferent input. (C) The experimental OM of 0δ, which is very similar to the state in (B). (D) Distribution of the difference between the orientation of the approximated OM at steady state and the orientation encoded in the afferent input for 10000 simulations. The number of bins was 100. The parameters for these simulations were identical to the parameters used for the spontaneous activity simulations. Grayscale range (arbitrary units): (A) 1.5–2.5, (B) −9–7, (C) −2.4–2.7

5. Model with spatially restricted connections

The connectivity pattern we considered so far (Eq. (6)) allowed a thorough mathematical analysis, which we presented in Section 3. However, this pattern is not realistic when large cortical areas are considered because the synaptic weights do not fall off with the distance between the pre- and postsynaptic locations. In contrast, lateral cortical projections in the cortex are spatially restricted. In the cat V1 these projections do not exceed 3.5 mm, with most axons projecting up to a distance of 0.5 mm (Gilbert and Wiesel, 1989; Kisvárday et al., 1997; Yousef et al., 2001). In this section we extend the model by imposing a restriction on the spatial extent of the lateral connections. This restriction is achieved by multiplying the synaptic strength given by Eq. (6) with a spatially isotropic gaussian of the distance between the pre- and postsynaptic locations, i.e.:

$${W_{xy}} = ({J_2}{r_x}{r_y}\cos ({\theta _x} - {\theta _y}) + {J_0}){G_\sigma }(\left\| {\left. {x - y} \right\|} \right.)$$

where σ denotes the standard deviation of the gaussian that determines the spatial range of the connections, G _σ denotes a 2-dimensional gaussian function ($G\sigma (\xi ) = {1 \over {2\pi {\sigma^2}}}{e^{ - {\xi^2}/(2{\sigma^2})}}$) and ‖x − y‖ is the distance between locations x and y. An example of the connectivity pattern induced by Eq. (48) for the Kenet et al. (2003) data set, is given in Fig. 2(B). Like the connectivity pattern induced by the spatially unrestricted model (Eq. (6), Fig. 2(A)), it exhibits a patchy structure, with patches corresponding to regions with similar orientation preference. However, now the patches are spatially restricted, and their density decays as a function of the distance between the interconnected locations. Both properties have been observed experimentally (Gilbert and Wiesel, 1989; Malach et al., 1993; Bosking et al., 1997; Kisvárday et al., 1997).

While the connectivity given by Eq. (48) is more realistic than the one given by Eq. (6), the behavior of the model is now much more complicated. This is because it depends not only on the distribution of preferred orientations and selectivities, but also on the spatial structure of the PM. One implication of this additional complexity is the loss of isotropy. While the PM can be still isotropic over the entire region simulated, each location is now connected to only a small subregion; in this subregion isotropy does not necessarily hold. One might expect that this would disrupt the formation of the ring attractor, and indeed simulations of the spatially restricted model with a uniform input yielded only one or two attractor states, depending on the model parameters (e.g., J ₂, σ). These attractor states were usually similar to OMs, with the identity of the attractor-OM also depending on the precise selection of the model parameters. A similar phenomenon, in which the ring attractor collapses into few isolated attractors, occurs in the ring model when some degree of anisotropy is introduced, i.e. when the distribution of preferred orientations is not uniform (Tsodyks and Sejnowski, 1995; Zhang, 1996; Renart et al., 2003; Goldberg et al., 2004). Thus, the spatially restricted model does not support a ring attractor in the sense we discussed so far. However, if the spatial range of the interactions is large enough, such that the recurrent input received by each location is approximately isotropic, the network has an unlimited number of states that have energy levels similar to the attractors. We therefore expected that these states could be revealed by adding a random noise to the uniform input. To explore this possibility we simulated the network with many realizations of the random noise. The random noise was taken to be a gaussian noise with small spatial correlations induced by filtering it with a 2-dimensional spatially isotropic gaussian kernel with a small standard deviation (0.1 mm) (see a similar approach in Goldberg et al. (2004)). An example of two realizations of the input is given in Fig. 10(A), 10(D). To allow comparison between the spatially restricted and unrestricted models under the same conditions, we repeated the simulations of spontaneous activity in the spatially unrestricted model (Section 4) using the noisy input we used for the spatially restricted model. The results for the unrestricted model with the noisy input were very similar to the results with the uniform input, implying that our characterization of spontaneous activity in Section 4 holds also for the noisy input.

Fig. 10

Simulations of the model with spatially restricted connectivity. (A, D) Afferent input for two simulations of spontaneous activity. (B, E) The total synaptic input at steady state for the same simulations. (C, F) Mosaics of experimental OMs that are highly similar to the steady states in (B) and (E). (G) Afferent input in a simulation of evoked activity encoding an orientation of 45δ. (H) Steady state for the same simulation. (I) The experimental OM of 45δ. The small bars on the right-hand side of the figure indicate the orientations of the OMs in (C), (F), (I). The values of the parameters were J ₀ = −3, _{J 2} = 3.5, σ = 0.6 mm, C = 2, T = 1. The noise in the input was random gaussian with zero mean, 0.1 standard deviation, and it was spatially filtered with a spatially isotropic gaussian with a standard deviation of 0.1 mm. Grayscale range (arbitrary units): (A, D, G) 1.1–2.9, (B, E) −0.8–4.6, (H) −1.3–4.3, (C, F, I) −2.4–2.7

For the spatially restricted model we constructed the lateral connectivity with Eq. (48), using the same PM we used for the simulations in Section 4. We took σ = 0.6 mm, which is approximately the diameter of a hypercolumn. It has been estimated that in cats most projections are approximately within this distance (Kisvárday et al., 1997; Schummers et al., 2002). The global parameters of the lateral connections were J ₀ = −3 and J ₂ = 3.5, which correspond to the marginal phase of the spatially unrestricted model. Any single realization of the input, such as the ones in Fig. 10(A) and (D), yielded one particular steady state solution. However, for different realizations of the input, the network had an unlimited number of attractors states. Some of these states were similar to OMs, like in the case of the spatially unrestricted model. Still, other steady states were not highly correlated with OMs. An example of two such states is given in Fig. 10(B) and (E). A close examination of these states revealed that they were mosaics of OMs, i.e., they were composed of different OMs in different regions of the modeled area. The structure of these states is demonstrated in Fig. 10(C), 10(F). For example, in Fig. 10(C), the OM of 22.5° is presented in the upper part, and the OM of 67.5° is presented in the lower part. This combination of OMs presented in Fig. 10(C) produces a state that is very similar to the steady state shown in Fig. 10(B). We conclude that with noisy afferent input the model with spatially restricted connections is qualitatively similar to the model of spontaneous activity we considered in Section 3.1, except that steady states can be mosaics of OMs, rather than a single OM.

We also tested this model in the evoked activity case, by adding a small modulation in the shape of an approximated OM to the noisy input described above (similarly to our approach in Section 4). An example of such input is presented in Fig. 10(F) where the orientation of the OM was 45°. The steady state for this simulation is presented in Fig. 10(G), and the experimental OM for 45° is presented in 10H. The steady state is similar to the OM encoded in the input, and the noise is filtered out. This was the case for all simulations. Thus, for evoked activity, the behavior of the model with spatially restricted connections is very similar to the behavior of the model with unrestricted connections.

6. Discussion

The emergence of OMs during spontaneous activity in anaesthetized cats has led to the suggestion that OMs are attractors of the intracortical network (Kenet et al., 2003). Adopting this view, we posed the following question: what type of intracortical dynamics and connectivity can lead to the formation of attractors at the OMs? In this work, we suggested a rate model endowed with a simple connectivity rule (Eq. (6)), and showed that it yields attractor states that are highly similar to OMs. Specifically, we showed analytically that in the proper parameter regime (marginal phase), and given a uniform input, the model has a ring attractor formed by approximated OMs of arbitrary orientations. This property explains the formation of OMs during spontaneous activity where the afferent input is assumed to be unstructured. We also considered the case where the activity is evoked by a visual stimulus and showed how a structured afferent input can select the OM that matches the stimulus’ orientation. The model therefore suggests that OMs are encoded in the lateral connections, and that these connections can generate OMs both when the activity is spontaneous and when it is evoked by a visual stimulus.

6.1. Selectivity

The core of our model is its connectivity pattern (Eqs. (6), (48)). This connectivity pattern depends on three properties of the interconnected sites: the selectivity, the preferred orientation, and the distance between the pre- and post- synaptic locations. The main theoretical innovation in our model is the explicit incorporation of the selectivity variable into the connectivity pattern. We treat the selectivity as a structural attribute of every cortical location, similarly to how the preferred orientation is dealt with by many models (including ours). We have shown that this approach is sufficient for explaining the correspondence between OMs and intrinsically preferred states of the cortical network. The model therefore predicts a different connectivity pattern for neurons near pinwheels and neurons in linear zones. There are some experimental evidence in favor of this view. Yousef et al. (2001) reported that connections of neurons near pinwheels have shorter lateral extent that neurons in linear zones, and that these connections are not as orientation-specific as those of neurons in linear zones. The latter result is consistent with our connectivity matrix, in which the orientation-dependent term in the equation of synaptic strength (Eq. (6)) is multiplied by the selectivity.

In the context of studying orientation selectivity, it is important to distinguish between two types tuning curves: of membrane potential tuning curves, on which we focused in this contribution, and commonly considered firing rate tuning curves. These two types of tuning curves give rise to two different measures of selectivity, and the closely related measures of tuning width. In our model, the shape of the membrane potential tuning curve is the same for all locations (a cosine shape). The different selectivity of different locations comes from different scaling of the tuning curve. As for firing rate tuning curves, the picture is more complicated. Equation (44) implies that the selectivity and width of firing rate tuning curves vary with membrane potential selectivity. The precise characteristics of the firing rate tuning curve depends on the model parameters (J ₀, J ₂), and both broad and narrow tuning curves are possible. In the linear regime, all locations have broad tuning curves. In the non-linear regime, both broad and narrow tuning curves are possible.

Several experimental studies used recordings of single neurons to study the dependence of selectivity and other tuning curve properties on the cortical location. Before we compare our model with these studies, it is important to note that the low selectivity near pinwheels seen with optical imaging, which is at the heart of our modeling approach, is difficult to interpret in terms of single neuron properties. This is because it can arise from either lack of orientation selectivity of single neurons near the pinwheel, or from averaging highly selective neurons with a highly variable orientation preference. The second option received support form extra-cellular recordings (Maldonado et al., 1997; Dragoi et al., 2001a) that found that as far as firing rates are concerned, neurons near pinwheels are as sharply tuned as neurons in linear zones. However, intracellular recordings (Schummers et al., 2002) have shown that when tuning curves of sub-threshold membrane potential are considered, the selectivity of neurons is indeed reduced with the selectivity given by the PM. Our model, in which selectivity varies both for synaptic-input and firing-rate tuning curves, is therefore consistent with membrane potential tuning curves, but not with firing rate tuning curves. As suggested by Schummers et al. (2002), this difference between the tuning of membrane potential and the tuning of firing rate might be accounted for by different parameters of the gain function (e.g., firing thresh-old) for neurons near pinwheels. This suggestion can be easily incorporated into our model. Other mechanisms were suggested, some of them, like sensitivity to the temporal structure of the membrane potential fluctuations (Volgushev et al., 2002), are beyond the framework of the rate models we discussed here.

The notion of tuning curves, is clearly associated with evoked activity. However, the principles laid down in this work can lead to the definition of “spontaneous tuning curves”. In an experimental setup, if optical imaging of spontaneous activity is performed simultaneously with single unit recordings, a spontaneous tuning curve can be defined as the average firing rate (measured electrophysiologically) as a function of the orientation encoded by the cortical state (measured by the optical imaging). It would be interesting to characterize spontaneous tuning curves and compare them with the classical, evoked, tuning curves. In one particular scenario our model predicts a qualitative difference between spontaneous and evoked tuning curves. A comparison between the phase diagrams for spontaneous and evoked activity (Figs. 4(E), 5(C)) implies that if V1 operates near the phase transition between the linear and marginal phases of spontaneous activity, (J ₂ close to 2), spontaneous tuning curves are expected to be significantly broader than the evoked ones. This prediction can be tested experimentally.

6.2. Orientation preference

Similarly to other models, our connectivity pattern suggests a more excitatory (or less inhibitory) coupling between neurons with similar orientation preference. This is in agreement with many experimental works that have shown that neurons tend to make connections with other neurons with a similar orientation preference. It has been argued that this is only a weak trend that is true only on average (Kisvárday et al., 1997). This does not necessarily contradict our model because the model considers populations of neurons and not single neurons. In addition, both in the linear and in the marginal phases, the ratio between the preferred-orientation dependent and independent components can be made arbitrarily small, by taking large negative value for J ₀.

Another challenge for our model is the fact that the dependence of the connectivity on orientation preference is primarily true for long range connections, i.e. connections between different hypercolumns. Moreover, it has been suggested that short-range connections are spatially isotropic (Ts’o et al., 1986; Das and Gilbert, 1999) (note that isotropy here refers to invariance with respect to direction on the cortical sheet, unlike the isotropy in the feature space we considered before). In contrast, our model does not distinguish between long-range and short-range connections, and in particular it does not include explicitly isotropic short range connections. We focused on the long-range type connections because isotropic connections alone, are hard to reconcile with the spontaneous emergence of OMs. This type of connectivity is invariant with respect to spatial rotation/ translation. Consequently, the spontaneous patterns it induces are also invariant with respect to rotation/translation. However, experimentally observed patterns of spontaneous activity are correlated with the OMs and do not exhibit this invariance (Kenet et al., 2003). It is also important to note that because of the dependence of the connectivity on the selectivity, our model does not exhibit a strong spatial anisotropy of the short-range connections. In the linear zones, neurons are surrounded by neurons with similar orientation preference, and therefore they exhibit roughly isotropic connections. Near pinwheels this is not true anymore; However there the connectivity depends only weakly on the orientation preference, and the global component (J ₀) dominates the connectivity, producing again a roughly isotropic connectivity. Nonetheless, an explicit incorporation of isotropic short range connections might be considered in a future work.

6.3. Spatial structure of the OMs

Two assumptions about the structure of the OMs were required for the formation of the ring attractor. The first assumption is that OMs can be well-approximated using the PM (Eq. (2)), that is, by considering at each location only the preferred orientation and selectivity. This assumption can be expressed by the condition that on average, a large fraction of the variance of the pixels’ tuning curves can be explained by its first Fourier component (Eq. (3)). We found that this condition is met in experimental data sets, and that individual OMs are highly correlated with their approximated versions. We note however, that the approximation of orientation tuning curves with cosine functions, as in Eq. (2), was criticized recently on the ground that it cannot describe the height and width of the tuning curves independently (Swindale et al., 2003). While deviations from cosine tuning curves can be functionally important, our analysis indicates that the PM provides a good first-order approximation for the OMs. The examples in Figs. 8–10 of individual experimental OMs compared with the output of the model, further support this claim.

The second assumption we made about the OMs is that the representation of orientation preference in the PM is isotropic, that is, that the distribution of preferred orientations is uniform and independent of the distribution of selectivities. For the experimental data set, we showed that orientation preference did not deviate strongly from the isotropy assumption. As discussed in Section 4, this result is consistent with some experimental studies in cats. Other studies, however, found a small bias of the distribution of preferred orientations towards the cardinal orientations. Interestingly, Kenet et al. (2003) reported a similar cardinal-orientations bias in the distribution of spontaneous OMs. It is thus conceivable that a small anisotropy in the distribution of preferred orientations indeed exists in cats’ V1, and that this anisotropy induces a significant anisotropy in spontaneous activity patterns. While our current model does not account for anisotropy, it provides a framework in which the effects of anisotropy in V1 could be addressed in future works.

6.4. Relation to the ring model

The model we presented here is strongly related to the ring model for feature selectivity (Ben-Yishai et al., 1995; Hansel and Sompolinsky, 1998). In fact, the ring model can be viewed as a special case of our model, in which all locations have the same selectivity. Therefore, these two models share some features that do not depend on the distribution of selectivities P(r), e.g., the general structure of the phase diagrams (Figs. 8(E) and 9(C)). Since in our model multiple selectivities are allowed, the attractor states can exhibit a much richer spatial structure and approximate the experimental OMs.

6.5. Lateral extent of the connections

For most of the work, the connectivity pattern that we considered (Eq. (6)), did not depend on the distance between the pre- and post- synaptic locations. In contrast, neuronal connections in the cortex have a limited lateral extent. Therefore, in Section 5, we suggested a modification of the connectivity rule in which the connections were spatially restricted. Using simulations, we showed that this version of the model has properties similar to the spatially unrestricted one, except for one important difference: the states generated during spontaneous activity are not necessarily OMs, but can be a mosaic of several OMs. Consequently, we predict that such states exist in spontaneous activity data. Preliminary analysis we of the data obtained in Kenet et al. (2003), indicates that mosaic states indeed emerge spontaneously.

It is important to note that the distinction between OM states and mosaic states is somewhat blurred. An OM state might in fact be a small part of a larger mosaic state, that would have been observed had a larger cortical area been considered. It is also possible, that what looks as an OM state is a composition of several OMs of similar orientations, that cannot be distinguished with current techniques. We can thus expect that if a large cortical region will be imaged during spontaneous activity, spontaneous states will correspond more closely to mosaic states, rather than to OM states. From a theoretical point of view, the prevalence and characteristics of mosaic states in spontaneous activity is an interesting question because mosaic states provide a signature of the effective strength and extent of the lateral connections in V1. Spontaneous states that correspond to OM states rather than to mosaic states indicate a strong lateral connectivity with large spatial extent. A further analysis, which is beyond the scope of this contribution, is required to fully characterize the behavior of the spatially restricted model, and its implications on spontaneous activity.

6.6. Dynamics

Our approach was based on identifying the attractor states of the intracortical network with the OMs. Because we used a static input and simple rate dynamics, attractor states always existed; the network always converged to a stationary activity profile. In contrast, spontaneous activity in anaesthetized cats is highly dynamic, continuously switching between states. This gap between the dynamics of spontaneous activity in cats and the dynamics of our model can be bridged by considering various mechanisms that can prevent the asymptotic convergence of the model to a fixed point. The dynamical switching between states can be induced by a dynamical external input. Specifically, Goldberg et al. (2004) suggested that spatio-temporally correlated noise coming from the LGN may drive the cortical dynamics. In the marginal phase, where the shape of the activity is dominated by recurrent connections, this type of afferent noise would cause the network to perform a random walk in the space of OMs. An analysis of experimental data reveals that signatures of its dynamics are different from those of a simple random walk along the ring attractor. Another possibility, which might be more compatible with experimental data, is that the system is in the linear phase. The observed cortical states in this case are the sum of the afferent input and a component in the shape of the OMs, induced by the recurrent connections (Goldberg et al., 2004).

The dynamical switching between states can be also induced by mechanisms intrinsic to V1. For example, firing rate adaptation or synaptic depression/facilitation can destabilize the attractor states on a slow time-scale. Including a term for firing rate adaptation in the dynamical equation (Eq. (4)), as in Hansel and Sompolinsky (1998), causes the network, in the marginal phase, to perform a limit-cycle in the space of OMs. This possibility seems too simplistic to account for the spontaneous cortical dynamics. However, a more complicated dynamical behavior can emerge if synaptic depression/facilitation is included. Such complicated behavior indeed emerges in the ring model with synaptic dynamics (Tsodyks, unpublished results), and we expect similar results for our model. It is also likely that spontaneous activity in V1 is driven by a combination of several of the mechanisms we mentioned above. Our model provides the framework in which the contribution of each of these mechanisms could be studied.

6.7. The stimulus

Throughout the paper, we assumed that the evoked OMs encode the orientation of the stimulus. This might not be the case. Basole et al. (2003) found that states highly similar to OMs, can also be evoked by texture stimuli containing short line segments. The OM to which the resulting state corresponded depended on both the orientation of the line segments and the direction in which they moved. The authors concluded that OM-like states encode the spatio-temporal properties of the stimulus, rather than its (spatial) orientation. The issue of which stimulus property is encoded by the OMs is largely avoided in our model because we did not consider in details how a particular visual stimulus is transformed into a pattern of LGN input to V1. For the types of input we considered, the LGN-V1 mapping is assumed to extract some information about features of the visual scene (e.g. orientation) and the intrinsic V1 processing acts to amplify and denoise the afferent signal. Input patterns of higher complexity may lead to a more intricate transformation by the recurrent V1 network.

In a broader context, our model can be considered as a general model for population encoding of a sensory or motor variable in the presence of a weak or noisy input. This variable can be some spatio-temporal characteristic of a sensory stimulus, but also direction of a planned movement, or direction of gaze. The variable encoded must however be periodic, otherwise the concept of a ring attractor is not relevant. In fact, the generality of the model is inherited from the ring model, which our model generalizes. Extensions of the ring model and other similar models supporting ring attractors have been considered in several systems such as spatial working-memory (Camperi and Wang, 1998; Compte et al., 2000), head direction system (Skaggs et al., 1995; Redish et al., 1996; Zhang, 1996; Xie et al., 2002; Boucheny et al., 2005), hippocampal place cells (Tsodyks and Sejnowski, 1995), general statistical inference (Pouget et al., 1998; Deneve et al., 1999) and TMS-induced perceptual suppression (Miyawaki and Okada, 2004). Some of these models build directly on the formulation of Ben-Yishai et al. (1995); others use different formulations but are similar in many aspects (Ermentrout, 1998). Our model adds to this class of models the possibility of variability in response amplitude, i.e. selectivity, of different neurons. As a result, the ring attractor is not associated with a simple spatial form of the activity pattern, like a single localized bump in the ring model, but rather includes the complex spatial structure of the OMs.

Regardless of the precise nature of the features represented by the OMs, the spatially restricted version of the model suggests how local features of the stimulus can be encoded when the stimulus properties, e.g. orientation, vary across the visual field. Since the coarse grain mapping of the visual field to the cortex is retinotopic, we can expect the activity patterns at a small cortical regions to represent the properties of the stimulus in the corresponding regions of the visual field. The states generated by the model, e.g. the mosaic states in Fig. 10, when evoked, achieve precisely this. We thus predict that states similar to the mosaic states will play important role in encoding complex stimuli whose features vary across the visual field.

¹

Appendices

Appendices

A.1. Properties of F ₀ and F ₂

In this appendix, we provide proofs for the properties of F ₀ and F ₂ given in Table 1.We provide proofs only for the weak inequalities given in that table.

A.1.1. Properties of F₀

F ₀ is defined by Eq. (21). Property (i) is trivial, as F ₀ is defined by integrating a non-negative function. The derivative of F ₀, is given by:

EquationSource% MathType!MTEF!2!1!+- % feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqr1ngB % PrgifHhDYfgasaacH8srps0lbbf9q8WrFfeuY-Hhbbf9v8qqaqFr0x % c9pk0xbba9q8WqFfea0-yr0RYxir-Jbba9q8aq0-yq-He9q8qqQ8fr % Fve9Fve9Ff0dmeaabaqaciGacaGaaeqabaWaaeaaeaaakeaatuuDJX % wAK1uy0HwmaeHbfv3ySLgzG0uy0Hgip5wzaGqbciab-ftignaaDaaa % leaacaaIWaaabaqeduuDJXwAKbYu51MyVXgaiyaacqGFYaIOaaGcca % GGOaGaamiwaiaacMcacqGH9aqpdaWfqaqaaGGaciab9TIiYlab9TIi % YdWcbaGaamOCaiGacogacaGGVbGaai4CaGqaaiaa8bcaruqtV52B0L % hCLbYqVj3CPzxyaGWbaiaa7H7acaaF+aGaaWxlaiaa8HfaaeqaaOGa % aWhiaiaa8bcaieGacaGEKbGaaONCaiaa6bcacaGEKbGaaShUdmaala % aabaGaaWhuaiaa8HcacaGEYbGaaWxkaaqaaiaa8jdacqaHapaCaaaa % aa!6A7D! <Equation ID="E1"><EquationNumber>1</EquationNumber><EquationSource Format="MATHTYPE"><![CDATA[ \mathcal{F}_0^\prime (X) = \mathop {\smallint \smallint }\limits_{r\cos \theta > - X} dr d\theta \frac{{P(r)}} {{2\pi }} $$

((49))

The integral on the right-hand side can be interpreted as the probability that r cos θ > −X, which immediately gives properties (iii) and (iv). In addition, it can be verified that:

$$\int\!\!\!\int\limits_{r\cos \theta > X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }} = 1 - \int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}} } $$

therefore EquationSource% MathType!MTEF!2!1!+- % feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqr1ngB % PrgifHhDYfgasaacH8srps0lbbf9q8WrFfeuY-Hhbbf9v8qqaqFr0x % c9pk0xbba9q8WqFfea0-yr0RYxir-Jbba9q8aq0-yq-He9q8qqQ8fr % Fve9Fve9Ff0dmeaabaqaciGacaGaaeqabaWaaeaaeaaakeaatuuDJX % wAK1uy0HwmaeHbfv3ySLgzG0uy0Hgip5wzaGqbaiab-ftignaaDaaa % leaacaaIWaaabaGaai4jaaaakiaacIcacqGHsislcaWGybGaaiykai % abg2da9iaaigdacqGHsislcqWFXeIrdaqhaaWcbaGaaGimaaqaaiaa % cEcaaaGccaGGOaGaamiwaiaacMcaaaa!4EA5! <InlineEquation ID="IE1"><EquationSource Format="MATHTYPE"><![CDATA[ \mathcal{F}_0^' ( - X) = 1 - \mathcal{F}_0^' (X) $$ . Integrating both sides of the equation from 0 to X, we obtain: ${F_0}( - X) = {F_0}(X) - X$. Therefore, as follows from property (i), F ₀(X) − X ≥ 0, which gives property (ii).

A.1.2. Properties of F₂

F ₂ is defined by Eq. (22). Its derivative is given by:

$$ \mathcal{F}_0^\prime (X) = \mathop {\smallint \smallint }\limits_{r\cos \theta > - X} dr d\theta \frac{{P(r)}} {{2\pi }}r\cos \theta $$

((51))

Equation (51) immediately implies that property (vii) holds for non-positive X, as the integrated expression is nonnegative. For positive X, it can be verified that:

$$\int\!\!\!\int\limits_{r\cos \theta > X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}r\cos \theta } = \int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}r\cos \theta } $$

Therefore, $\mathcal{F}_2^\prime (X) = \mathcal{F}_2^\prime ( - X)$ , and property (vii) holds for positive X as well. Property (vii) implies that F ₂ is non-decreasing, therefore, F ₂(−∞) and F ₂(+∞) give, respectively, lower and upper bounds for F ₂. Calculating these expressions, properties (v) and (vi) are obtained.

A.1.3. Other properties

For property (viii), we observe that the integral expression defining F ₀ (Eq. (21)) can be written as the sum of two integrals as follows:

$${F_0}(X) = \int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}r\cos \theta } + \int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}X} $$

Noting that the first term on the right-hand side is F′₂ (X) and that the second term is XF′₀ (X), property (viii) is obtained.

For property (ix), we observe that the integral expression defining F ₂ (Eq. (22)) can be written as the sum of two integrals as follows:

$${F_2}(X) = \int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}{{(r\cos \theta )}^2}} + \int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}Xr\cos \theta } $$

Noting that the second term on the right-hand side is XF′₂ (X), we obtain:

EquationSource% MathType!MTEF!2!1!+- % feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqr1ngB % PrgifHhDYfgasaacH8srps0lbbf9q8WrFfeuY-Hhbbf9v8qqaqFr0x % c9pk0xbba9q8WqFfea0-yr0RYxir-Jbba9q8aq0-yq-He9q8qqQ8fr % Fve9Fve9Ff0dmeaabaqaciGacaGaaeqabaWaaeaaeaaakeaatuuDJX % wAK1uy0HwmaeHbfv3ySLgzG0uy0Hgip5wzaGqbciab-ftignaaBaaa % leaacaaIYaaabeaakiaacIcacaWGybGaaiykaiabgkHiTiaadIfacq % WFXeIrdaqhaaWcbaGaaGOmaaqaaeXafv3ySLgzGmvETj2BSbacgaGa % e4NmGikaaOGaaiikaiaadIfacaGGPaGaeyypa0ZaaCbeaeaaiiGacq % qFRiI8cqqFRiI8aSqaaiaadkhaciGGJbGaai4BaiaacohaieaacaaF % Gaqef00BU9gD5bxzGm0BYnxA2fgaiCaacaWE4oGaaWNpaiaa81caca % aFybaabeaakiaa8bcacaaFGaacbiGaaOhzaiaa6jhacaGEGaGaaOhz % aiaa7H7adaWcaaqaaiaa8bfacaaFOaGaaONCaiaa8LcaaeaacaaFYa % GaeqiWdahaaiaacIcacaWGYbGaci4yaiaac+gacaGGZbGaaShUdiaa % 8LcadaahaaWcbeqaaiaa8jdaaaaaaa!77C0! <Equation ID="E1"><EquationNumber>1</EquationNumber><EquationSource Format="MATHTYPE"><![CDATA[ \mathcal{F}_2 (X) - X\mathcal{F}_2^\prime (X) = \mathop {\smallint \smallint }\limits_{r\cos \theta > - X} dr d\theta \frac{{P(r)}} {{2\pi }}(r\cos \theta )^2 $$

((55))

The expression on the right hand side can be checked to be a non-decreasing function of X, and calculating its value for X → ∞, gives property (ix).

For property (x), we first write the following application of the Cauchy-Schwarz inequality:

$${\left( {\int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}r\cos \theta } } \right)^2} \le \left( {\int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}} } \right)\left( {\int\!\!\!\int\limits_{r\cos \theta > - X} {drd\theta {{{\rm{P}}(r)} \over {2\pi }}{{(r\cos \theta )}^2}} } \right)$$

which reads (F′₂ (X))² ≤ F′₀(X)(F ₂(X) − XF′₂(X)). Substituting F′₂(X) with the left-hand side of property (viii), property (x) is obtained.

A.2. Stability

In this section we study the stability of the non-uniform fixed point. We discuss only the case of the tuned input model (Section 3.2.2). The stability in the case of the spontaneous activity model (Section 3.1.3) can be easily obtained by repeating similar arguments with ∈ = 0.

Let μ, ρ take their fixed point values (Eqs. (24),(25)), and let ψ = ψ _aff. We look at a small perturbation about the fixed point: (μ + δμ), (ρ + δρ), (ψ + δψ). The linearized dynamics of Eqs. (34)–(36) are given by $\tau {[\dot \delta \mu \dot \delta \rho \dot \delta \psi ]^T} = S{[\delta \mu \delta \rho \delta \psi ]^T}$ with S being the 3 × 3 Jacobian matrix given by:

$$ \begin{gathered} S = \hfill \\ \left[ \begin{gathered} J_0 \mathcal{F}_0^\prime (X) - 1 J_2 (\mathcal{F}_0 (X) - X\mathcal{F}_0^\prime (X)) 0 \hfill \\ J_0 \mathcal{F}_2^\prime (X) J_2 (\mathcal{F}_2 (X) - X\mathcal{F}_2^\prime (X)) - 1 0 \hfill \\ 0 0 - \varepsilon C\mathcal{F}_2 (X) \hfill \\ \end{gathered} \right] \hfill \\ \end{gathered} $$

((57))

This matrix implies that the evolution of δψ is decoupled from that of δμ, δρ, and that δψ decays with time. To show the decay of δμ, δρ, we study the eigenvalues of 2 × 2 upper-left sub-matrix of S, which we denote by S′. For the eigen-values of S′ to have a negative real part, it is enough to show that the determinant of S′, det S′, is positive, and that its trace, tr S′, is negative. The determinant is given by:

$$ \begin{gathered} \det S^\prime = J_2 \mathcal{F}_2^\prime (X)(X - J_0 \mathcal{F}_0 (X)) \hfill \\ + (1 - J_0 \mathcal{F}_0 (X))(1 - J_2 \mathcal{F}_2 (X)) \hfill \\ \end{gathered} $$

((58))

For the fixed point value of X it holds that X − J ₀ F ₀(X) > 0 and 1 − J ₂ F ₂(X) > 0 (inequalities (26), (43)). In addition, for any X,F′₂ (X) > 0 (property (vii), Table 1) and when J ₀ < 1, 1 − J ₀ F′₀ (X) > 0 (properties (iii),(iv), Table 1). Therefore, the determinant is positive. The trace is given by:

$$ tr S\prime = - 1 + J_0 \mathcal{F}_0^\prime (X) - 1 + J_2 \mathcal{F}_2 (X) - J_2 X\mathcal{F}_2^\prime (X) $$

((59))

The inequalities we mentioned above for the determinant imply that when X ≥ 0, the trace is negative. In addition, if J ₂ < 2, by property (ix), J ₂(F ₂(X) − XF′₂ (X)) ≤ 1 and the trace is negative as well. The last case to check is when X < 0 and J ₂ > 2. In this case, from the definitions of X ₀ and X ₂, it follows that ${J_0} = {{{X_0}} \over {{F_0}({X_0})}}$ and ${J_2} = {1 \over {{F_2}({X_2})}}$. Thus, we rewrite the trace as:

EquationSource% MathType!MTEF!2!1!+- % feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqr1ngB % PrgifHhDYfgasaacH8srps0lbbf9q8WrFfeuY-Hhbbf9v8qqaqFr0x % c9pk0xbba9q8WqFfea0-yr0RYxir-Jbba9q8aq0-yq-He9q8qqQ8fr % Fve9Fve9Ff0dmeaabaqaciGacaGaaeqabaWaaeaaeaaakqaabeqaaG % qaaiaa-rhacaWFYbGaa8hiaGqaciaa+nfarmqr1ngBPrgitLxBI9gB % aGqbaiab9jdiIkaa-1dacaWFTaGaa8Nmaiaa-TcacaGFkbWaaSbaaS % qaaiaa-jdaaeqaamrr1ngBPrwtHrhAXaqeguuDJXwAKbstHrhAG8KB % LbacgiGccqaFXeIrdaWgaaWcbaGaaGOmaaqabaGccaGGOaGaamiwai % aacMcacqGHsisldaWcaaqaaiaacIcacaWGybGaeyOeI0Iaamiwamaa % BaaaleaacaaIWaaabeaakiaacMcacaGGOaGaeWxmHy0aa0baaSqaai % aaicdaaeaacqqFYaIOaaGccaGGOaGaamiwaiaacMcaaeaacqaFXeIr % daWgaaWcbaGaaGimaaqabaGccaGGOaGaamiwamaaBaaaleaacaaIWa % GaaiykaaqabaaaaaGcbaGaa8hiaiaa-bcacaWFGaGaa8hiaiaa-bca % caWFGaGaa8hiaiaa-bcacaWFGaGaa8hiaiaa-TcadaWcaaqaaiaa+H % facaWFOaacgaGaeSxmHy0aa0baaSqaaiaa-bdaaeaacqqFYaIOaaGc % caGGOaGaamiwaiaacMcacqaFXeIrdaWgaaWcbaGaaGOmaaqabaGcca % GGOaGaamiwamaaBaaaleaacaaIYaaabeaakiaacMcacqGHsislcqaF % XeIrdaqhaaWcbaGaaGOmaaqaaiab9jdiIcaakiaacIcacaWGybGaai % ykaiab8ftignaaBaaaleaacaaIWaaabeaakiaacIcacaWGybWaaSba % aSqaaiaaicdaaeqaaOGaaiykaiaacMcaaeaacqaFXeIrdaWgaaWcba % GaaGimaaqabaGccaGGOaGaamiwamaaBaaaleaacaaIWaaabeaakiaa % cMcacqaFXeIrdaWgaaWcbaGaaGOmaaqabaGccaGGOaGaamiwamaaBa % aaleaacaaIYaaabeaakiaacMcaaaaaaaa!8FCA! <Equation ID="E1"><EquationNumber>1</EquationNumber><EquationSource Format="MATHTYPE"><![CDATA[ \begin{gathered} tr S\prime = - 2 + J_2 \mathcal{F}_2 (X) - \frac{{(X - X_0 )(\mathcal{F}_0^\prime (X)}} {{\mathcal{F}_0 (X_{0)} }} \hfill \\ + \frac{{X(\mathcal{F}_0^\prime (X)\mathcal{F}_2 (X_2 ) - \mathcal{F}_2^\prime (X)\mathcal{F}_0 (X_0 ))}} {{\mathcal{F}_0 (X_0 )\mathcal{F}_2 (X_2 )}} \hfill \\ \end{gathered} $$

((60))

Since −1 + J ₂ F ₂(X) < 0, and X > X ₀, the sum of the first three terms is negative. Therefore, to show that the entire summation is negative, it remains to show that F′₀(X)F ₂(X ₂) − F′₂(X)F ₀(X ₀) > 0. First, we note that because X ₀ < X < X ₂ it holds that:

$$ \begin{gathered} \mathcal{F}_0^\prime (X)\mathcal{F}_2 (X_{2)} - \mathcal{F}_2^\prime (X)\mathcal{F}_0 (X_0 ) \hfill \\ \geqslant \mathcal{F}_0^\prime (X)\mathcal{F}_2 (X) - \mathcal{F}_2^\prime (X)\mathcal{F}_0 (X) \hfill \\ \end{gathered} $$

((61))

By property (x), the term on the right-hand side is nonnegative. Therefore, the trace is negative.

Finally, our solution for evoked activity has shown that a fixed point with ψ = ψ _aff + π can exist. Solving for this point and obtaining the corresponding stability matrix one finds that it is very similar to the one given in Eq. (57), with the evolution of δψ decoupled form that of δμ, δρ. However in this case it is given by δψ = ∈CF ₂(X)δψ, which implies that the fixed point is unstable to perturbations in ψ.