Model-Agnostic Neural Mean Field With The Refractory SoftPlus Transfer Function

Due to the complexity of neuronal networks and the nonlinear dynamics of individual neurons, it is challenging to develop a systems-level model which is accurate enough to be useful yet tractable enough to apply. Mean-field models which extrapolate from single-neuron descriptions to large-scale models can be derived from the neuron’s transfer function, which gives its firing rate as a function of its synaptic input. However, analytically derived transfer functions are applicable only to the neurons and noise models from which they were originally derived. In recent work, approximate transfer functions have been empirically derived by fitting a sigmoidal curve, which imposes a maximum firing rate and applies only in the diffusion limit, restricting applications. In this paper, we propose an approximate transfer function called Refractory SoftPlus, which is simple yet applicable to a broad variety of neuron types. Refractory SoftPlus activation functions allow the derivation of simple empirically approximated mean-field models using simulation results, which enables prediction of the response of a network of randomly connected neurons to a time-varying external stimulus with a high degree of accuracy. These models also support an accurate approximate bifurcation analysis as a function of the level of recurrent input. Finally, the model works without assuming large presynaptic rates or small postsynaptic potential size, allowing mean-field models to be developed even for populations with large interaction terms.

The mean inter-spike interval (ISI) for an LIF neuron under the diffusion approximation has been known for a long time [29].It has a complicated functional form, even after significant simplification using the assumptions that the resting and reset potential of the neuron are identical and that the input has zero mean.Here erfi is the imaginary error function, and the notation !! refers to the double factorial: where y = (V thresh − V rest ) 2 τ Rq 2 For the LIF neuron, the transfer function is the reciprocal M (R) −1 .If T ref = 0, the transfer function is a smooth curve which is near zero at low R but asymptotically grows as R 1/2 , as shown in figure S1.Also shown in the figure is a neuron with a nonzero absolute refractory period, resulting in a transfer function which grows more slowly with R, towards an asymptotic maximum firing rate 1/T ref .
The neuron was numerically simulated at 500 different input rates r.When q varies for a fixed r, the total number N of presynaptic neurons (yielding total input rate R = N r) is changed to hold D constant so that all conditions are equivalent according to the diffusion limit.Since Poisson processes superimpose linearly, this is only a visual convenience which allows plotting firing rates for different q against the same horizontal axis.Firing rates were then calculated by dividing the total number of spiking events by the total simulation time.
Figure S1: Comparison of the theoretical transfer function M (R) −1 to simulated data for an LIF neuron with τ = 10 ms, V thresh = −50 mV, V rest = −65 mV, and refractory period T ref either zero (left) or 2 ms (right).The diffusion limit is represented by a simulation with a small PSP amplitude and large number of presynaptic neurons q = 0.1 mV and N = 100000 (blue).However, violating the diffusion approximation by changing the PSP to q = 1 mV and N = 1000 (orange) leads to significant deviation from the analytical solution (dashed line).
February 6, 2024 23/26 Although the analytical transfer function is accurate in diffusion limit, the 749 simulation in figure S1 also demonstrates a case where the analytical diffusion 750 approximation breaks down.The limiting behavior is represented by a simulation with 751 a sufficiently small PSP amplitude q = mV.Increasing the PSP to the biologically 752 reasonable value of q = 1 (orange) leads to significant deviation from the 753 analytical solution.Similar deviations also occur due to the use of a finite simulation 754 timestep, which leads to a small error even in the better of the two curves in figure S1, 755 as well as to compounded errors from both of these sources in figure 2.

757
Theoretically, the use of q in equation ( 7) is redundant, as it is equivalent to rescaling 758 β and σ 0 , but because it is derived from the diffusion coefficient D, it acts as a 759 first-order approximation to the effect of PSP amplitude.The only effect of this is to make fitting numerically easier, as the parameters vary less with q.However, it is 761 to directly model this dependence to obtain a transfer function with better 762 performance across multiple values of q.We observed empirically that q has relatively 763 little influence on the fitted values σ 0 and T ref , so we modeled α and β as having a 764 first-order dependence on q with a shared parameter γ: This substitution yields the PSP-dependent Refractory SoftPlus transfer function: If the simulation data used to fit the transfer function includes varying values of q, 767 this version of the function can be fitted instead, which achieves significantly better 768 results using only one additional parameter γ.This approach is shown in figure S2, input under a range of q and r, and a single PSP-dependent Softplus fit 771 was performed.

772
We observe an overall RMS error on par with the simulations of the single neuron, 773 which is able to account for the decrease in firing rate for large q.As in figure S1, N 774 was covaried with q as to keep the range of D = q 2 N r (equation 3) the same.This

783
If the neuron fires according to a Poisson process with rate r over a time interval T , 784 the expected number of firings in the time interval is rT , with variance rT .Therefore, 785 the variance of an estimate of the neuron's firing rate can be approximated as r/T .

786
We can derive the same result by imagining using bootstrapping to calculate the 787 variance of a binary spike raster.If the neuron fired exactly rT times within the time 788 T , which is broken up into bins of a very small duration h, then each bootstrap fold is 789 T /h samples of a Bernoulli random variable with rate rh.The sum of these variables 790 follows a binomial distribution with mean rT and variance rT (1 − rh) ≈ rT , so this 791 firing rate estimate also has mean r and variance r/T .Note that this method assumes 792 that the bins of the raster are independent, which is equivalent to the Poisson population.The expected squared error of each is its variance r i /T , so the 809 normalized RMS error ε of the entire empirical firing rate curve is given by: Although the precise value now depends on a significant number of variables which 811 are difficult to know a priori, it is at least clear that the expected RMS error will scale 812 as T −1/2 just like the standard deviation of a population of single-neuron estimates.

Figure S2 :
Figure S2: The function as a function of PSP amplitude q and presynaptic rate r (left), and the error landscape of a single PSP-dependent Refractory SoftPlus fit to this data (right).

FebruaryFigure S3 :
Figure S3: Variance the firing rate between multiple simulations (colored points) as a function of the simulation time T , compared to the approximation (solid lines) for a few different firing rates.

775
figure 4 for the convergence of fitted SoftPlus curves as a function of the amount T of

FigureFebruary
Figure S3 depicts the standard deviation of 50 firing rate estimates of single LIF