## ABSTRACT

Analysis of spike and local field potential (LFP) data is an essential part of neuroscientific research. Today there exist many open-source toolboxes for spike and LFP data analysis implementing various functionality. Here we aim to provide a practical guidance for neuroscientists in the choice of an open-source toolbox best satisfying their needs. We overview major open-source toolboxes for spike and LFP data analysis as well as toolboxes with tools for connectivity analysis, dimensionality reduction and generalized linear modeling. We focus on comparing toolboxes functionality, statistical and visualization tools, documentation and support quality. To give a better insight, we compare and illustrate functionality of the toolboxes on open-access dataset or simulated data and make corresponding MATLAB scripts publicly available.

## 1 INTRODUCTION

Analysis of spike and local field potential (LFP) data is an essential part of neuroscientific research (Brown et al., 2004; Stevenson and Kording, 2011; Mahmud and Vassanelli, 2016). There are many already implemented open-source tools and toolboxes for spike and LFP data analysis. However, ascertaining whether functionality of the toolbox fits users’ requirements is in many cases time-consuming. Often neuroscientists are even not aware that some functionality is already implemented and start writing their own scripts from scratch which takes time and is error-prone. We aim to provide a practical guidance for choosing a proper toolbox on the basis of toolbox functionality, statistical and visualization tools, programming language, availability of graphical user interface, support and documentation quality. Compared to the existing reviews (Ince et al., 2009, 2010; Ince, 2012; Mahmud and Vassanelli, 2016; Timme and Lapish, 2018), we

- include in the comparison important toolboxes and tools not covered by earlier reviews (e.g., Brain-storm, Elephant and FieldTrip);

- compare in detail common and discuss unique functionality of toolboxes;

- compare and illustrate functionality of the toolboxes on open-access datasets (Perich et al., 2018;Lawlor et al., 2018; Lowet et al., 2015) and simulated data. For readers’ convenience we make the corresponding MATLAB scripts publicly available

^{1};- overview specialized tools for dimensionality reduction and generalized linear modeling as they are widely used in neuroscientific research (Cunningham and Byron, 2014; Truccolo et al., 2005);

- provide information about documentation and support quality for the toolboxes;

- indicate bibliometric information

^{2}: while popularity among users alone does not guarantee quality, it can be an important indicator that toolbox’s functions are easy-to-use and have been tested.

### Scope

We include into our comparison major open-source^{3} toolboxes (see Table 1) for spike and LFP data processing and analysis which have a valid link for downloading, documentation, scientific paper describing toolbox’s features or corresponding method, and which were updated during the last five years. In Table 1 we provide a summary of the toolboxes we consider, we list all toolboxes with a brief description in alphabetical order in Section 7 with paper reference and downloading link.

Considered toolboxes were developed in MATLAB^{4} and Python^{5} languages which are popular in neuroscientific community.

We have not listed in Table 1 toolboxes FIND (Meier et al., 2008), infotoolbox (Magri et al., 2009) and STAtoolkit (Goldberg et al., 2009), since they are not available under the links provided by the authors (accessed on 27.03.2019); toolboxes BSMART (Cui et al., 2008), DATA-Means (Bonomini et al., 2005), MEA-tools (Egert et al., 2002), MEAbench (Wagenaar et al., 2005), sigTOOL (Lidierth, 2009), SPKTool (Liu et al., 2011), STAR (Pouzat and Chaffiol, 2009), since they have not been updated during the last five years (since 2008, 2005, 2007, 2011, 2011, 2011, and 2012, correspondingly); toolbox SigMate (Mahmud et al., 2012) since it is in beta version; and toolbox OpenElectrophy (Garcia and Fourcaud-Trocmé, 2009) which is not recommended for new users by the toolbox authors^{6}.

### Documentation/Support

We have indicated “In part” in Documentation column for Spike Viewer and SPIKY since, compared to other toolboxes from Table 1, they do not provide a description of input parameters for most of the functions. This complicates understanding of implementation details for programming-oriented users that use only a part of the toolbox functionality in their analysis workflow. gramm toolbox specifies function input parameters not in code comments but in separate documentation file^{7}. Considered version of Elephant provides only getting started tutorial, more tutorials are to be added^{8}. Chronux and SPIKY (MATLAB version) toolboxes are not uploaded to GitHub or other public version control systems, which prevents from tracking version differences and smoothly reporting bugs (Python version of SPIKY is on GitHub^{9}).

### Import/Export

We have indicated “In part” in Import/Export column for Elephant and Spike Viewer toolboxes since they require Neo-based Python package^{10,11} (Garcia et al., 2014) for the support of spike file formats (Spike2, NeuroExplorer, AlphaOmega, Blackrock, Plexon etc.). This Neo-based package is popular in neuroscientific society but requires either a separate installation or data conversion to Neo-compatible data format. Brainstorm, Chronux and FieldTrip support working with several spikes file formats (e.g. Blackrock, CED, Neuralynx, Plexon etc.)^{12} as well as working with data from MATLAB workspace or stored as .mat files. SPIKY and gramm support working with data from MATLAB workspace; SPIKY also supports working with data stored in .mat and .txt file formats.

### Compatibility

Chronux under Macintosh operating system requires recompilation of the locfit^{13} and spikesort packages. All other listed toolboxes are supported by Microsoft Windows, Macintosh and Linux operating system. Chronux, FieldTrip, gramm and SPIKY require MATLAB installation, Elephant requires Python installation, Brainstorm and Spike Viewer require neither MATLAB nor Python installation.

### Test dataset

We consider for illustration of toolboxes functionality an open-access dataset (Lawlor et al., 2018; Perich et al., 2018) and refer to this dataset further as “test dataset”. The dataset contains extracellular recordings from premotor (PMd) and primary motor (M1) cortex from a macaque monkey in a sequential reaching task where monkey controlled a computer cursor using arm movements. A visual cue specified the target location for each reach. The monkey receives a reward after making four correct reaches to the targets within the trial.

In Sections 2 and 3, we compare toolboxes for the general spike and LFP data analysis, correspondingly. In Section 4, we compare tools for the analysis of synchronization and connectivity in spike and LFP data. Each of Sections 2-4 is subdivided into two subsections: first, we compare common toolboxes functionality, then we discuss unique toolboxes functionality, i.e. functionality implemented only in one of the toolboxes under comparison. In Section 5, we compare toolboxes with specialized tools for dimensionality reduction and generalized linear modeling. Finally, we summarize the comparisons in Section 6. In Section 7, we list all the considered toolboxes in alphabetical order with links for toolbox downloading and brief descriptions. We do not consider in this review toolboxes specializing on spike sorting and modeling spiking activity. For this we refer to (Ince et al., 2010; Mahmud and Vassanelli, 2016) and web-reviews^{14,15,16} correspondingly.

## 2. TOOLBOXES FOR SPIKE DATA PROCESSING AND ANALYSIS

In Table 2 we compare major open-source toolboxes for spike data analysis, both for point-process data and for spike waveforms. Functionality related to synchronization and connectivity analysis (e.g. cross-correlation, coherence, joint peri-stimulus time histogram, spike-LFP phase-coupling and dissimilarity measures etc.) will be covered in Section 4, and functionality related to dimensionality reduction and generalized linear modeling in Section 5.

From Table 1 and 2 one can see that Brainstorm, Chronux and FieldTrip toolboxes provide more versatile functionality (see also below) than others, are highly cited, well-documented and allow import from many file formats. The Elephant toolbox has versatile functionality (seeSubsection 2.2) but it does not have built-in visualization tools (Elephant provides visualization examples in the documentation using `matlabplot` Python library). Compared to other toolboxes from Table 2,

- Brainstorm and FieldTrip include detailed documentation with tutorials and examples (documentation of other toolboxes from Table 2 has less examples/tutorials for spike data analysis) and have either a forum

^{17}or a discussion list^{18}where users can ask questions on data analysis; both toolboxes regularly hold hands-on courses^{19,20}, while other toolboxes from Table 2 provide neither forums nor courses;- Brainstorm and FieldTrip are actively developing by including new functionality;

- FieldTrip provides many descriptive and inferential statistics mostly not requiring MATLAB statistical toolbox (Brainstorm provides statistical tools

^{21}without examples for spike data analysis^{22}and these statistical functions are not part of spike data analysis functions, different to how it is often done in FieldTrip and Chronux; Spike Viewer and SPIKY do not provide statistical tools for general spike data analysis);- FieldTrip and gramm allow versatile data plots customization (color maps, line widths, smoothing, errorbars etc.); while gramm provides better and quicker general visualization tools, FieldTrip provides plotting customization specific for spike data analysis (conditions/interval/trials/channels and optimal bin size selection);

- for programming-oriented users, Chronux and FieldTrip provide, to our opinion, most convenient and well-commented data analysis pipeline with clear uniform data structure (other toolboxes from Table 2 are lacking at least one of three following components: detailed code comments with description of input/output parameters, uniform data structure throughout the analysis pipeline, modular function design allowing to easily adapt them into analysis workflow). Chronux reference documentation in the function description provides a list of functions which are called from the function and from which the function is called, this is convenient for programming-oriented users.

### 2.1 Comparing common tools: peri-stimulus time-histogram, raster plot, inter-spike interval histogram and spike sorting

In this subsection we compare most common spike data analysis functions: peri-stimulus time histogram (PSTH), raster plot, inter-spike interval histogram (ISIH) and spike sorting algorithms for toolboxes from Table 2. Regarding visualization, the gramm visualization toolbox stands out with its publication-quality graphics, which helps avoiding major post-processing. This is illustrated in Figure 1, where we compare PSTH and raster plots for test dataset produced in FieldTrip and gramm toolboxes, both of which provide most adjustable plot properties compared to other toolboxes from Table 2 (see below a detailed comparison).

We do not provide raster plots and PSTH plots for other toolboxes from Table 2 with visualization tools since

- Brainstorm does not provide PSTH plots; raster plots are available only for one unit per figure

^{23};- Chronux does not provide raster plots and allows to plot only smoothed PSTH for one unit per figure without built-in tools to adjust line width, font size, colors etc.;

- in SPIKY raster and PSTH plots are available only for one unit per figure without built-in tools to adjust line width, marker size, font style and size, colors (Kreuz et al., 2015, Figure 2) and without confidence intervals for PSTHs;

- in Spike Viewer PSTH plots are available without confidence intervals

^{24}.

Regarding statistical tools when computing PSTHs, Chronux computes PSTH for adaptive or user-defined kernel width with Poisson error or bootstrapped over trials (both with doubled standard deviation error). Elephant computes PSTH for fixed user-defined bin size without additional statistics (note that Elephant provides many kernel functions for convolutions such as rectangular, triangular, Guassian, Laplacian, exponential, alpha function etc.). FieldTrip computes PSTH for optimal (by Scott’s formula (Scott, 1979)) or user-defined bin width with variance computed across trials. Besides, FieldTrip, different to other toolboxes from Table 2, allows statistical testing on PSTHs for different conditions or subjects^{25} with a parametric statistical or a non-parametric permutation test. Brainstorm provides this functionality by calling FieldTrip functions. gramm allows to compute PSTHs with (bootstrapped) confidence intervals, standard error of the mean, standard deviation etc^{26} only for user-defined bin width. Spike Viewer and SPIKY compute PSTH only for user-defined bin width and do not compute statistics for PSTHs across trials.

In Figure 2 we compare visualization of ISIH provided by FieldTrip and Spike Viewer since other toolboxes from Table 2 do not provide ISIH visualization (Brainstorm, Chronux and Elephant compute ISIH without visualization, see details below).

Regarding statistical tools when computing ISIH, FieldTrip computes ISIH with a coefficient of variation (a ratio of the standard deviation to the mean), Shinomoto’s local variation measure (Shinomoto et al., 2005) or a shape scale for a gamma distribution fit. Chronux computes ISIH with two standard deviations away from the mean calculated using jackknife resampling. Elephant computes ISIH with a coefficient of variation. Spike Viewer does not compute statistics on ISIH.

Brainstorm and FieldTrip provide spike sorting algorithms including spike detection and extraction, i.e., using time-continuous broadband data as input. Spike sorting package is no longer provided by Chronux. Brainstorm implements supervised and unsupervised spike sorting according to the methods WaveClus (Quiroga et al., 2004), UltramegaSort2000 (Hill et al., 2011; Fee et al., 1996), KiloSort (Pachitariu et al., 2016) and Klusters (Hazan et al., 2006). FieldTrip implements k-means and Ward (for several Ward distances) sorting methods. While Chronux and FieldTrip do not provide tutorials on spike sorting, Brainstorm has a detailed tutorial^{27}.

Brainstorm provides computing and visualization of tuning curves: they are plotted with one figure per unit for selected units, conditions and time interval but without customization of font size, line width and colors, no variance statistic across trials is computed^{28}. gramm toolbox provides visualization of tuning curves including fits from MATLAB curve smoothing toolbox and user-defined functions (also in polar coordinates) with (bootstrapped) confidence intervals, standard error of the mean, standard deviation etc. As the considered gramm version is not focused on spike data analysis, firing rates averaged per condition need to be computed prior to tuning curves visualization (see example in our open MATLAB script).

### 2.2 Description of unique tools

In this subsection we discuss unique tools of toolboxes from Table 2, e.g. fitting tools, and higher order statistics (variability and spectral measures) on spike timing.

Chronux provides two unique tools: local regression package (locfit) and point-process spectrograms. locfit is based on local regression methods (Loader, 2006; Parikh, 2009; Hayden et al., 2009) and provides a set of methods for fitting functions and probability distributions to noisy data. The idea of local regression is that the estimated function is approximated by a low order polynomial in a local neighborhood of any point with polynomial coefficients estimated by the least mean squares method (Bokil et al., 2010). In (Bokil et al., 2010; Loader, 2006) local regression methods are motivated by their simplicity, non-parametric approach to kernel smoothing and by reducing the bias at the boundaries which is present in kernel smoothing methods. On the other hand, it was shown that fixed and variable kernel methods (Shimazaki and Shinomoto, 2010, Algorithm 2, Appendix A.2) as well as Abramson’s adaptive kernel method (Abramson, 1982) outperform `locfit` for simulated data examples (Shimazaki and Shinomoto, 2010).

Point-process spectrograms are usually used to illustrate rhythmic properties of otherwise stochastic spiking patterns rather than for statistical inference (Deng et al., 2013). We refer to (Hurtado et al., 2004, 2005) regarding methods to evaluate statistical significance of point-process spectral estimators and to (Jarvis and Mitra, 2001; Rivlin-Etzion et al., 2006) for a critical discussion. Chronux provides the only open-source, to our knowledge, implementation of point-process spectral estimates which is implemented according to (Jarvis and Mitra, 2001; Rivlin-Etzion et al., 2006, Section 4, Formula 11), see example of usage in our open MATLAB script.

Elephant provides several statistical measures for spike timing variability such as Fano factor, CV2 measure of inter-spike variability (Holt et al., 1996) and a measure of local variation (Shinomoto et al., 2003) which were introduced as substitutes of classical coefficient of variation to overcome its sensitivity to firing rate fluctuations between trials (Shinomoto et al., 2005).

FieldTrip allows to compute mean average spike waveform and its variance across trials, one can optionally align waveforms based on their peaks, rejects outlier waveforms and interpolate the waveforms.

## 3 TOOLBOXES FOR LFP DATA ANALYSIS

In Table 3 we compare open-source toolboxes for processing and analysis of local field potential (LFP) data. Functionality related to synchronization and connectivity analysis will be discussed in Section 4.

From Table 3 one can see that Brainstorm and FieldTrip toolboxes provide most versatile functionality for LFP data analysis. Compared to other toolboxes from Table 3,

- FieldTrip provides most flexible and versatile digital filtering (in particular, a fast and accurate line noise removal technique) and spectral analysis tools (see details in Subsection 3.1);

- Brainstorm

^{29,30}and FieldTrip^{31,32}provide detailed tutorials with guidance on parameter choice and examples for digital filtering and spectral analysis. Chronux provides examples on parameter choice for spectral analysis in manuals^{33}(Pesaran, 2008);- Brainstorm and Elephant provide fast implementation of Morlet wavelet transform (see details in Subsection 3.1);

- Brainstorm, Chronux and FieldTrip provide statistical tools for computing variance across trials and for comparing between conditions when estimating spectra; Elephant does not compute statistics on the estimated spectra;

- Brainstorm and FieldTrip allow adjustment of plot properties for spectral analysis such as baseline correction, trials and channels selection, colormaps and interactive selection of spectrogram part for further processing. Neither Chronux nor Elephant provide these options. Compared to Brainstorm, FieldTrip also allows to adjust font sizes, titles, plot limits etc.

### 3.1 Comparing common tools: filtering, detrending and spectral analysis

Digital filtering is implemented in Brainstorm, FieldTrip and Elephant toolboxes. Compared to toolboxes from Table 3, MATLAB and Python themselves provide more flexible filtering tools. Yet, it is convenient to have filtering within the toolbox pipeline. First, it allows to avoid extra conversion from toolbox’s format to MATLAB/Python and back. Second, toolboxes allow simplified setting of filter parameters for typical neuroscientific datasets and offer tutorials for their choice for non-experienced users.

Brainstorm, FieldTrip and Elephant toolboxes provide low/high/band-pass and band-stop filters for user-defined frequencies.

- Brainstorm provides Finite Impulse Response (FIR) filters with Kaiser window based on

`kaiserord`functions from MATLAB Signal Processing Toolbox (Octave-based alternatives are used if this toolbox is not available). The user can set 40 or 60 dB stopband attenuation, data are padded with zeros at edges with a half of filter order length (according to the description of the filtering`bst bandpass hfilter`function used by default);- Elephant provides Infinite Impulse Response (IIR) Butterworth filtering with adjustable order using

`scipy.signal.filtfilt`(with default padding parameters) or`scipy.signal.lfilter`standard Python functions;- FieldTrip provides the most flexible filtering tools with user-defined filter type (Butterworth IIR, window sinc FIR filter, FIR filter using either standard MATLAB

`fir1`or`firls`function from Signal Processing Toolbox or frequency-domain filter using standard`fft`and`ifft`MATLAB functions), padding type and optional parameters such as window type (Hanning, Hamming, Blackman, Kaiser), filter order and direction, transition width, passband deviation, stopband attenuation etc.^{34}. An automatic tool to deal with filter instabilities (which MATLAB 2016a, to our knowledge, does not provide) is implemented by either recursively reducing filter order or recursively splitting the filter into sequential filters.

Brainstorm, Chronux and FieldTrip also provide specific tools for line noise removal. Brainstorm reduces line noise with IIR notch filter (employing either `filtfilt` function from MATLAB Signal Processing toolbox or MATLAB `filter` function). Chronux reduces line noise using Thomson’s regression method for detecting sinusoids (Thomson, 1982). FieldTrip reduces line noise by two alternative methods: with a discrete Fourier transform (DFT) filter (by fitting a sine and cosine at user-defined line noise frequency and subsequently subtracting estimated components) or by spectrum interpolation (Mewett et al., 2004). In Figure 3 we compare 60 Hz line noise removal by Chronux, FieldTrip and Brainstorm toolboxes on the basis of an example provided by MATLAB^{35} for open-loop voltage across the input of an analog instrument in the presence of 60 Hz power-line noise. One can see that FieldTrip selectively and successfully attenuates 60 Hz while Brainstorm does not fully suppress 60 Hz, Chronux suppresses also frequencies around 62 Hz, the MATLAB solution contains some remaining oscillations in the beginning of the signal, which is also reflected in the periodogram by a slight inaccuracy around 61-62 Hz. In Figure 3 (C) we present mean squared error (MSE) between power spectrum values of the original and estimated signal except the values estimated in 0.2 Hz vicinity of 60 Hz.

Brainstorm, Chronux and FieldTrip provide detrending tools. Brainstorm removes a linear trend from the data, Chronux detrending employs local linear regression^{36}, whereas FieldTrip detrending uses a general linear model approach and removes mean and linear trend from the data (by fitting and removing an *N*th order polynomial from the data)^{37}: Brainstorm, Chronux and FieldTrip offer similar performance in terms of processing time and trend removal accuracy for a simple MATLAB example^{38} (see our open MATLAB code).

Compared to the classic Fourier transform, multitaper methods provide more convenient control of time and frequency smoothing (Percival and Walden, 1993; Mitra, 2007). Spectral decomposition with Morlet wavelets provides a convenient way of achieving a time-frequency resolution trade-off (van Vugt et al., 2007), since it is inherent to the method that wavelets are scaled in time to vary resolution in time and frequency, see (van Vugt et al., 2007) for a comparison of multitaper and wavelet methods and (Bruns, 2004) for a comparison of wavelet, Hilbert and Fourier transform. Equivalent time-frequency trade-offs can also be implemented with short-time Fourier or Hilbert methods via variable-width tapers (Bruns, 2004).

Chronux and FieldTrip provide multitaper power spectrum estimation using Thomson’s method (Thomson, 1982; Percival and Walden, 1993; Mitra and Pesaran, 1999) with Slepian sequences (Slepian and Pollak, 1961). Additionally to this, FieldTrip allows also more conventional tapers (e.g. Hamming, Hanning). In FieldTrip, the user defines frequencies and time interval of interest, width of sliding window and of frequency smoothing. In Chronux, the user defines bandwidth product and number of tapers to be used (see (Prieto et al., 2007) for a discussion of multitapers parameter choice).

Brainstorm, Elephant and FieldTrip implement complex-valued Morlet transform. FieldTrip provides time-frequency transformation using Morlet waveforms either with convolution in the time domain or with the multiplication in the frequency domain. Brainstorm and Elephant implement convolution in the time domain. FieldTrip implements Morlet wavelet transformation methods based on (Tallon-Baudry et al., 1997), the user defines the wavelet width in number of cycles and optionally wavelet length in standard deviations of the implicit Gaussian kernel. In Brainstorm the user sets the central frequency and temporal resolution. Elephant implements Morlet wavelets according to (Le Van Quyen et al., 2001; Farge, 1992), where the user sets central Morlet frequencies, size of the mother wavelet and padding type.

Different to other toolboxes from Table 3 FieldTrip also implements Fourier transform on the coefficients of the multivariate autoregressive model estimated with FieldTrip tools (see Subsection 4.1 for more details on MVAR implementation in FieldTrip).

Elephant does not compute statistics on estimated power spectrum whereas Chronux and FieldTrip compute confidence intervals and standard error, correspondingly, in a standard way or with jackknife resampling. To compare spectrum estimates for different conditions or subjects, Chronux provides a two-group test and FieldTrip performs a parametric statistical test, a non-parametric permutation test or a cluster-based permutation test (Brainstorm includes these FieldTrip statistical functions).

MATLAB R2016a, compared to Chronux, FieldTrip and Brainstorm,

- does not provide detailed tutorials for multitaper and wavelet parameters choice;

- does not have built-in tools for computing average spectrogram across trials;

- does not have built-in tools for generating multitaper spectrograms;

- uses exclusively short-time Fourier transform for standard spectrogram plotting.

In Figure 4 we compare spectrum estimation methods implemented in Brainstorm (A), Chronux (B), Elephant (C), FieldTrip (D-F) and MATLAB (G-H) for two simulated signals, *x*_{1}(*t*) and *x*_{2}(*t*).

We generate *x*_{1}(*t*) as a sum of sines and *x*_{2}(*t*) by sinusoidal frequency modulation, see Eq. 1-2. We add normally distributed pseudo-random values with zero mean to the second half of both signals:
The instantaneous frequency of the signal *x*_{2}(*t*) is defined by the following equation (Granlund, 1949):
To compare quantitatively the spectra estimated by the toolboxes we compute power spectrum values of the ideal signal by setting maximum spectrum values at theoretical frequencies of the signals *x*_{1} (8, 20, 40 and 60 Hz) and *x*_{2} (given by Eq. 4) and minimum at all other frequencies. When setting ideal power spectrum values we allow bandwidth of 1 Hz, i.e. we set the maximum power spectrum values also at neighboring frequencies. Then we compare in Figure 5 the estimated spectrum values with the ideal spectrum values using mean squared error and two-dimensional Pearson correlation coefficient as suggested in (Rankine et al., 2005).

From Figures 4-5 we conclude that

- MATLAB standard spectrogram tools are less robust with respect to noise than spectrum estimation provided by the toolboxes from Table 3 for the signal

*x*_{2}with changing frequencies;- while Brainstorm, Chronux, Elephant and FieldTrip provide equally good accuracy of spectra estimation, Brainstorm and Elephant provide the fastest computing tools (see spectra computing times in subplot titles of Figure 4).

See in our open MATLAB script an example of spectral analysis with averaging over trials for real-world LFP data (Lowet et al., 2015).

### 3.2 Description of unique tools

Compared to other toolboxes from Table 3, Chronux provides several unique features for specialized computations (Bokil et al., 2010) such as space-frequency singular value decomposition (SVD) for univariate and multivariate continuous signals: for theoretical details we refer to (Mitra and Pesaran, 1999) and for an example of possible application to (Makino et al., 2017; Prechtl et al., 1997). Space-frequency SVD can be applied to the space-time data as, for example, in (Prechtl et al., 1997), where space-frequency SVD has been applied for spectral analysis of transmembrane potentials optically recorded in pixels distributed in space. Chronux also provides computation of multitaper spectral derivatives and stationarity statistical test for continuous processes based on quadratic inverse theory.

Elephant provides computing of the current source density from LFP data using electrodes with 2D or 3D geometries.

## 4 TOOLBOXES WITH SYNCHRONIZATION AND CONNECTIVITY ANALYSIS TOOLS

In Table 4 we compare open-source toolboxes providing tools for spike-spike, field-field (LFP-LFP) or spike-field (spike-LFP) synchronization and connectivity analysis. We refer to (Blinowska, 2011; Bastos and Schoffelen, 2016) for reviews of functional connectivity analysis methods and their interpretational pitfalls (e.g. common reference, common input, volume conduction or sample size problems). We do not include in Table 4 the connectivity toolboxes ibTB (Magri et al., 2009) and Toolconnect (Pastore et al., 2016), since they are not available under the links provided by the authors (accessed on 27.03.2019). We also do not list in Table 4 the following connectivity analysis toolboxes that are not focused on spike and LFP data analysis: Inform (Moore et al., 2017), JIDT (Lizier, 2014), MVGC (Barnett and Seth, 2014), MuTe (Montalto et al., 2014), PyEntropy (Ince et al., 2009) and TrenTool (Lindner et al., 2011). TrenTool toolbox has a FieldTrip-compatible data structure.

Compared to other toolboxes from Table 4,

- Brainstorm, Elephant and FieldTrip provide most versatile set of connectivity measures: while Field-Trip provides many classic and recent pairwise connectivity and synchronization measures, Elephant provides tools for multivariate analysis of high-order correlations in spike trains (see Subsections 4.1-4.2);

- Brainstorm tutorials for connectivity measures are actively developing

^{39}; Chronux has examples for connectivity measures for real-world data in tutorial presentations; FieldTrip provides detailed tutorials on connectivity analysis for simulated and real-world data; Elephant provides examples for connectivity measures with simulated data;- Chronux and FieldTrip compute confidence intervals for connectivity measures with jackknife resampling or variance estimates across trials, correspondingly (see Subsections 4.1-4.2); Brainstorm computes significance values for common connectivity measures, Elephant does not compute statistics on common connectivity measures.

To provide a better feeling of connectivity measures, we classify in Table 5 connectivity and synchronization measures mentioned in Table 4. We indicate for which signals the measure is applicable (Input), whether the measure is directed or not (Directed), is defined in time or frequency domain (Domain) and is bi- or multivariate (Dimension).

### 4.1 Comparing common tools: correlation, cross-correlation, coherence, Granger causality, phase-amplitude coupling, phase-locking value, spike-field coherence and spike-triggered average

In this subsection we compare implementations of common synchronization and connectivity measures for toolboxes from Table 4: correlation, cross-correlation, coherence, Granger causality, phase-amplitude coupling, phase-locking value, spike-field coherence and spike-triggered average.

Brainstorm and Elephant implement correlation, a pairwise non-directional time-domain connectivity measure. Brainstorm computes Pearson correlation coefficient (or optionally covariance) between spike trains and p-value of its significance; correlation is computed equivalently to MATLAB `corrcoef` function but in a faster vectorized way for *N >* 2 input signals. Elephant computes either Pearson correlation coefficient between binned spike trains (without additional statistics), pairwise covariances between binned spike trains (without additional statistics) or spike time tiling coefficient (STTC) introduced in (Cutts and Eglen, 2014). STTC, compared to correlation index introduced in (Wong et al., 1993), is described as not dependent on signals firing rate, correctly discriminating between lack of correlation and anti-correlation etc. (Cutts and Eglen, 2014). There is also a MATLAB STTC implementation^{40}.

Cross-correlation is correlation between two signals computed for different time lags of one signal against the other. Elephant and FieldTrip implement cross-correlation, a pairwise non-directional time-domain connectivity measure. Between two binned spike trains Elephant computes cross-correlation for user-defined window with optional correction of border effect, kernel smoothing (for boxcar, Hamming, Hanning and Bartlett) and normalization. Between two LFP signals Elephant computes the standard unbiased estimator of the cross-correlation function (Stoica et al., 2005, Eq. 2.2.3) for user-defined time-lags without additional statistics across trials; note that biased estimator of the cross-correlation function is more accurate as discussed in (Stoica et al., 2005). FieldTrip computes cross-correlation between two spike channels for user-defined time lags and bin size (correlogram can optionally be debiased depending on data segment length). FieldTrip computes shuffled and unshuffled correlograms: if two channels are independent, the shuffled cross-correlogram should be the same as unshuffled.

Brainstorm, Chronux, Elephant and FieldTrip implement coherence, a frequency-domain equivalent of cross-correlation (Bastos and Schoffelen, 2016):

- Brainstorm implements coherence according to (Carter, 1987) computing also p-values of parametric significance estimation;

- Chronux computes coherence between two (binned) point-processes or LFP signals using multitaper methos, with confidence intervals or jackknife resampled error bars;

- Elephant computes coherence using Welch’s method with phase lags but without additional statistics. Computing coherence across trials is not supported in the considered version;

- FieldTrip computes coherence according to (Rosenberg et al., 1989) with variance estimate across

- trials. Additionally, FieldTrip provides computing of partial coherence according to (Rosenberg et al., 1998), partial directed coherence (Baccalá and Sameshima, 2001) and imaginary part of coherency (Nolte et al., 2004) with variance across trials. Partial directed coherence (PDC) is a directional measure. Compared to coherence, PDC is shown to reflect a frequency-domain representation of the concept of Granger causality (Baccalá and Sameshima, 2001).

Elephant does not provide built-in tools to compare coherence values between two conditions, Chronux provides a two-group test, FieldTrip provides an independent samples Z-statistic via `ft freqstatistics` function by the method described in (Maris et al., 2007), Brainstorm is using FieldTrip `ft freqstatistics` function.

Brainstorm and FieldTrip implement Geweke’s extension of the original time-domain concept of Granger causality (GC) introduced in (Granger, 1969) to the frequency domain (Geweke, 1982). GC implemented in Brainstorm and FieldTrip is a frequency-domain pairwise directional measure of connectivity. FieldTrip GC implementation is based on (Brovelli et al., 2004). The multivariate autoregressive (MVAR) model in FieldTrip uses biosig or BSMART toolboxes implementation on user choice, which are included in FieldTrip. FieldTrip computes variance of GC values across trials. Neither Brainstorm nor FieldTrip provide built-in tools/prescribed procedure to statistically compare GC values between conditions. Different to FieldTrip, Brainstorm computes as well time-resolved GC between two signals using two Wald statistics according to (Geweke, 1982) and (Hafner and Herwartz, 2008). The directed transfer function and partial directed coherence are multivariate extensions of Granger causality (Blinowska, 2011).

In Figure 6 we compare values of several connectivity measures computed in Brainstorm, Chronux and FieldTrip for simulated data with autoregressive models^{41} according to Eq. (5) (computing coherence across trials is not included in the considered Elephant version).

Brainstorm and FieldTrip implement phase-amplitude coupling (PAC), a frequency-domain pairwise non-directional measure (Canolty et al., 2006; Samiee and Baillet, 2017; Voytek et al., 2010). FieldTrip implements two types of PAC^{42}: mean vector length and modulation index according to (Tort et al., 2010). Brainstorm implements PAC according to (özkurt and Schnitzler, 2011). Both Brainstorm and FieldTrip do not compute additional statistics on PAC.

Brainstorm and FieldTrip implement phase-locking value (PLV), a frequency-domain pairwise non-directional measure (Lachaux et al., 1999). PLV checks how consistent the phase relation between the two signals is across trials. We refer to (Vinck et al., 2011; Bastos and Schoffelen, 2016) for a comparison of different phase synchronization metrics and their biases. FieldTrip computes PLV based on (Lachaux et al., 1999) with a variance estimate using jackknife resampling.

Combination of spiking activity and LFP is often used to study rhythmic neuronal synchronization since spike-LFP measures are more sensitive than spike-spike synchronization measures (Vinck et al., 2012; Chakrabarti et al., 2014). To this end Brainstorm, Chronux and Elephant implement a spike-field coherence (SFC), a frequency-domain pairwise non-directional measure. Brainstorm implements SFC according to (Fries et al., 2001) for user-defined window size around spikes without additional statistics computed. Chronux implements SFC with a multitaper approach for user-defined tapers and frequency band, computing also a confidence level of coherency and jackknife or standard error bars. Elephant implements SFC using standard Python `scipy.signal.coherence()` function, no additional statistics is computed.

One of the first steps in the analysis of spike-field coupling is computing of a spike-triggered average (STA) of LFP that is an average LFP voltage within a small window of the time around every spike. While neither Brainstorm nor Elephant compute any additional statistic on STA, Chronux computes STA with an optional kernel smoothing and calculates bootstrapped standard error on computed values and FieldTrip computes mean and variance of STA values.

### 4.2 Description of unique tools

In this subsection we describe unique tools of the toolboxes from Table 4. Elephant provides five recent statistical tools to study higher-order correlations and synchronous spiking events in parallel spike trains:

- ASSET (Analysis of Synchronous Spike EvenTs) implements the method from (Torre et al., 2016) and is an extension of the visualization method from (Schrader et al., 2008). ASSET assesses the statistical significance of simultaneous spike events (SSE) and aims to detect such events that cannot be explained on the basis of rate coding mechanisms and arise from spike correlations on shorter time scale;

- CAD (Cell Assembly Detection) implements the method from (Russo and Durstewitz, 2017) for capturing structures of higher-order correlations in massively parallel spike train recordings with arbitrary time lags and at multiple time-scale; CAD makes statistical parametric testing between each pair of neurons followed by an agglomerative recursive algorithm aiming to detect statistically precise repetitions of spikes in the data;

- CuBIC (Cumulant Based Inference of higher order Correlations) implements a statistical method (Staude et al., 2010) for detecting higher order correlations in parallel spike train recordings;

- SPADE (Spike Pattern Detection and Evaluation) implements the method from (Quaglio et al., 2017) for assessing the statistical significance of repeated occurrences of spike sequences (spatio-temporal patterns) based on recent methods in (Torre et al., 2013; Quaglio et al., 2017). SPADE aims to overcome computational and statistical limits in detecting repeated spatio-temporal patterns within massively parallel spike trains (Quaglio et al., 2017), see (Quaglio et al., 2018) for a recent review of methods for identification of spike patterns in massively parallel spike trains;

- UE (Unitary Event analysis) implements the statistical method from (Grü n et al., 1999, 2002) for analyzing excess spike correlations between simultaneously recorded neurons. This method compares the empirical spike coincidences to the expected number on the basis of firing rate of the neurons.

Elephant and SPIKY toolboxes allow to compute measures of spike train dissimilarity (also referred as measures of spike train synchrony). Elephant implements well-known time-scale dependent van Rossum (van Rossum, 2001) and (Victor and Purpura, 1996) dissimilarity distances whereas SPIKY implements three recent parameter-free time-scale independent measures: ISI-distance (Kreuz et al., 2007), SPIKY distance (Kreuz et al., 2012) and SPIKE synchronization (Quiroga et al., 2002). We refer to (Chicharro et al., 2011; Kreuz et al., 2012; Mulansky et al., 2015) for a comparison of dissimilarity measures. Note also MATLAB implementations of dissimilarity measures at J.D. Victor^{43} and T. Kreuz^{44} web-sites.

FieldTrip, compared to other toolboxes from Table 4, computes and visualizes^{45} the following classic and recent connectivity and synchronization measures:

- directed transfer function (DTF) introduced in (Kaminski and Blinowska, 1991) is a multivariate frequency-domain directional connectivity measure; FieldTrip computes it according to (Kaminskiand Blinowska, 1991) from cross-spectral density with a variance across trials. DTF, compared to GC, makes a multivariate spectral decomposition, the advantage of this approach is that interaction between all channels is taken into account (see, e.g., Figure 6 in Subsection 4.1). However pairwise measures yield more stable results since they involve fitting fewer parameters (Blinowska, 2011; Bastos andSchoffelen, 2016);

- joint peri-stimulus time histogram (JPSTH) is a pairwise time-domain non-directional measure between spike trains that allows to gain insight into temporal evolution of spike-spike correlations (Brown et al., 2004; Aertsen et al., 1987). To check whether the resulted JPSTH is caused by task-induced fluctuations of firing rate or by temporal coordination not time-locked to stimulus onset, FieldTrip also computes JPSTH with shuffling subsequent trials. We illustrate JPSTH visualization with FieldTrip tools in Figure 7;

- mutual information (MI) is a pairwise time-domain non-directional connectivity measure. FieldTrip computes MI using implementation from ibtb toolbox (Magri et al., 2009) without additional statistics;

- noise correlations (NC) is a non-directional pairwise time-domain measure that can be computed between two spike trains; NC measures whether neurons share trial-by-trial fluctuations in their firing rate; different to so called signal correlations (SC), these fluctuations are measured over repetitions of identical experimental conditions, i.e. are not driven by variable sensory or behaviorally conditions;

- phase-coupling pairwise spike-field measures compute the phases of spikes relative to the ongoing LFP with a discrete Fourier transform of an LFP segment around the spike time (Vinck et al., 2012). FieldTrip implements recent methods from (Vinck et al., 2012): angular mean of spike phases, Rayleigh p-value and pairwise-phase consistency according to the method in (Vinck et al., 2010). We refer to (Vinck et al., 2010; Bastos and Schoffelen, 2016) for a discussion and comparison of these measures;

- phase-slope index (PSI) is a directional pairwise frequency-domain measure that can be computed between two signals from their complex-valued coherency. FieldTrip computes PSI according to (Nolte et al., 2008) with variance across trials;

- pairwise phase consistency (PPC) is a directional pairwise frequency-domain measure that can be computed from the distribution of pairwise differences of the relative phases. PPC compared to PLV is not biased by sample size (Bastos and Schoffelen, 2016). FieldTrip computes PPC with leave-one-out variance estimate;

- weighted phase-lag index (WPL) introduced in (Vinck et al., 2011) is a non-directional pairwise frequency-domain measure computed from cross-spectral density between two signals. WPL was introduced to solve the problem with sensitivity of phase-lag index (Stam et al., 2007) to volume-conduction and noise (Vinck et al., 2011). FieldTrip computes WPL according to (Vinck et al., 2011) with variance across trials.

## 5 SPECIALIZED TOOLBOXES FOR DIMENSIONALITY REDUCTION AND GENER-ALIZED LINEAR MODELING

In this section we overview specialized toolboxes for dimensionality reduction (Subsection 5.1) and generalized linear modeling (Subsection 5.2). Compared to Table 1, we do not provide in the corresponding tables for specialized toolboxes information on

- Import/Export since none of the considered toolboxes supports importing/exporting from specialized spike data formats;

- GUI since only DataHigh toolbox provides GUI (see details below).

### 5.1 Toolboxes for dimensionality reduction

Dimensionality reduction of neural data allows to obtain a simplified low-dimensional representation of neural activity. In Table 6 we compare open-source toolboxes for dimensionality reduction of neural data (note also a list of dimensionality reduction software actively updating at B. Yu web-site^{46}). See examples for application of DataHigh, dPCA and TCA toolboxes in our open MATLAB script.

We have indicated “In part” in Documentation column for GPFA and TD-GPFA toolboxes since they provide usage examples and readme files with notes on parameters choice but but neither detailed manual nor tutorial, they refer to the original publication (Yu et al., 2009) for details. We have indicated “In part” in Documentation column for DCA tool since it provides neither manual nor tutorial (only example of use in MATLAB script comments). DataHigh and GPFA toolboxes are not uploaded to GitHub or any other public version control system preventing from tracking version changes and submitting bugs. DCA and TD-GPFA toolboxes have not been updated during the last 2 years.

Compared to other toolboxes from Table 6,

- DataHigh provides a user-friendly GUI illustrating algorithm steps such as choice of bin size, smoothing, components number etc.;

- dPCA is applied on trial-averaged spiking activity; dPCA breaks down the neural activity into components each of which relates to time (condition-independent component) or a single experimental condition of the task; the idea is an easier task-relevant interpretation compared to the standard PCA or ICA; the results can be summarized in a single figure (Kobak et al., 2016);

- TD-GPFA allows to extract low-dimensional latent structure from time series in the presence of delays;

- tensor-demo and tensortools allow to reduce dimensionality both across and within trials (Williamset al., 2018).

In Table 7 we outline additional dimensionality reduction tools provided by the toolboxes.

It is important to check whether input data fit model assumptions when applying dimensionality reduction methods: whether the data are allowed to be non-stationary, contain outliers, observational noise or be correlated, whether recorded activity evolves in a low-dimensional manifold, which sample size is sufficient etc. Discussing model assumptions for each of dimensionality reduction methods is beyond the scope of this paper, we refer to the original papers and to the model assumptions for applying principal component analysis (PCA) formulated in (Shlens, 2014).

### 5.2 Toolboxes for GLM analysis

Generalized linear models (GLMs) are often applied for predicting spike counts with the aim to understand which factors influence simultaneous spiking activity: whether it is predicted by the past or concurrent neural activity of the same or remote brain area or by external covariates. In Table 8 we overview major open-source toolboxes for GLM analysis. These toolboxes do not contain any general spike data analysis functions besides GLM analysis since they are either GLM tutorials or codes related to particular analysis made in the paper.

GLMcode1 and GLMcode2 codes are not uploaded to GitHub or any other version control system as they implement methods for particular analysis made in the papers (see below) are not supposed to be updated.

Note that

- Case-Studies implements (see folders Chapter 9, 10, 11 on GitHub

^{47}) basic steps of Poisson GLM fitting with history dependence to the data on sample datasets for the corresponding book (Kramer and Eden, 2016);- GLMcode1, GLMcode2 implement the code for the papers (Glaser et al., 2018) and (Lawlor et al.,2018);

- examples of use for nStat toolbox are located in

`helpfiles`folder in the corresponding GitHub repository;- spykesML tool provides comparison of GLM performance with several methods from modern machine learning approaches (including neural networks);

- NIMclass uses MATLAB optimization toolbox and contains many examples for real-world data;

- GLMspiketraintutorial is a tutorial for teaching purposes. It is not memory-efficient implemented, but it makes easy to understand the basic steps of Poisson and Gaussian GLMs fitting, analysis and comparison for spike data

^{48}. neuroGLM and GLMspiketools are more advanced tools with efficient memory implementation. Additionally to GLMspiketraintutorial, they support some advanced GLM features such as smooth temporal basis functions for spike-history filters, different time-scales for stimulus and spike-history components etc.

## 6 CONCLUSIONS

In this review we have compared major open-source toolboxes for spike and local field potentials (LFP) processing and analysis. We have compared toolboxes functionality, statistical and visualization tools, documentation and support quality. Besides summarizing information about toolboxes in comparison tables, we have discussed and illustrated particular toolboxes functionality and implementations, also in our open MATLAB code. Below we summarize the comparisons that we made for general spike and LFP analysis toolboxes and toolboxes with connectivity tools.

Each considered toolbox has its own advantages:

- Brainstorm: graphical user interface (GUI), versatile and cross-checked functionality (highly-cited), statistical tools, detailed tutorials with recommendations on parameters choice, support of many file formats, active user discussion community and regular hands-on sessions, fast Morlet wavelet transform implementation;

- Chronux: versatile and cross-checked functionality (highly-cited), statistical tools (measures of variance across trials and statistical comparing between different conditions), detailed documentation, convenient data analysis pipeline for programming-oriented users (detailed code comments and modular code design);

- Elephant: support of many file formats, versatile functionality with implementation of classic and recent methods for spike-spike connectivity and synchronization analysis, fast Morlet wavelet transform implementation;

- FieldTrip: versatile and cross-checked functionality (highly-cited), statistical tools (measures of variance across trials and statistical comparing between different conditions), detailed tutorials with recommendations on parameters choice, support of many file formats, active user discussion community and regular hands-on sessions, flexible visualization tools, convenient data analysis pipeline for programming-oriented users (detailed code comments and modular code design), versatile filtering, connectivity and synchronization analysis tools, fast and accurate line noise removal;

- gramm: quick publication-quality PSTH, raster plots and tuning curves with many easily adjustable plot properties;

- Spike Viewer: GUI, support of many file formats;

- SPIKY: GUI, implementation of recent spike train dissimilarity measures.

## 7 LIST OF TOOLBOXES AND TOOLS IN ALPHABETICAL ORDER WITH LINKS

Below all the considered toolboxes are provided with a brief description, reference to the paper where the toolbox was introduced and a link for downloading.

- Brainstorm

^{49,50}(Tadel et al., 2011) – a MATLAB toolbox for the analysis of brain recordings: MEG, EEG, fNIRS, ECoG, depth electrodes and animal invasive neurophysiology;- BSMART

^{51}(Brain-System for Multivariate AutoRegressive Time series) (Cui et al., 2008) – a MATLAB/C toolbox for spectral analysis of continuous neural data recorded from several sensors;- Case-Studies

^{52}– a MATLAB set of examples on sample datasets accompanying the corresponding book (Kramer and Eden, 2016);- Chronux

^{53}(Bokil et al., 2010) – a MATLAB package for the analysis of neural data;- DataHigh

^{54}(Cowley et al., 2013) – a MATLAB-based graphical user interface to visualize and interact with high-dimensional neural population activity;- DATA-MEAns

^{55}(Bonomini et al., 2005) – a Delphi7 tool for the classification and management of neural ensemble recordings;- DCA

^{56}(Cowley et al., 2017) (distance covariance analysis) – an implementation (MATLAB and Python) of the linear dimensionality reduction method that can identify linear and nonlinear relation-ships between multiple datasets;- dPCA

^{57}(demixed Principal Component Analysis) (Kobak et al., 2016) – a MATLAB implementation of the linear dimensionality reduction technique that automatically discovers and highlights the essential features of complex population activities;- Elephant

^{58,59}(Yegenoglu et al., 2017) – an Electrophysiology Analysis Toolkit in Python. Elephant toolbox includes functionality from earlier developed toolboxes CSDPlotter^{60}(Pettersen et al., 2006)^{and iCSD 2D61}, it is a direct successor of NeuroTools;- FieldTrip

^{62,63}(Oostenveld et al., 2011) – a MATLAB toolbox for advanced analysis of MEG, EEG, and invasive electrophysiological (spike and LFP) data;- FIND

^{64}(Meier et al., 2008) – a MATLAB toolbox for the analysis of neuronal activity;- GLMcode1 – a MATLAB code implementing data analysis for particular publication (Glaser et al.,2018) with GLM fitting to analyze factors contributing to neural activity (this code is available from the authors upon request);

- GLMcode2

^{65}(Perich et al., 2018) – a MATLAB code implementing data analysis for particular publication (Lawlor et al., 2018) with GLM fitting to estimate preferred direction for each neuron;- GLMspikestools

^{66}(Pillow et al., 2008) – a Generalized Linear Modeling tool for single and multineuron spike trains;- GLMspiketraintutorial

^{67}(Pillow et al., 2008) – a simple tutorial on Gaussian and Poisson GLMs for single and multi-neuron spike train data;- GPFA

^{68}(Gaussian-Process Factor Analysis) (Yu et al., 2009) – a MATLAB implementation of the method extracting low-dimensional latent trajectories from noisy, high-dimensional time series data. It combines linear dimensionality reduction (factor analysis) with Gaussian-process temporal smoothing in a unified probabilistic framework;- gramm

^{69,70}(Morel, 2018) – a plotting MATLAB toolbox for quick creation of complex publication-quality figures;- ibTB

^{71}(Information Breakdown Toolbox) (Magri et al., 2009)) – a C/MATLAB toolbox for fast information analysis of multiple-site LFP, EEG and spike train recordings;- Inform

^{72}(Moore et al., 2017) – a cross-platform C library for information analysis of dynamical systems;- infoToolbox

^{73}(Magri et al., 2009) – a toolbox for the fast analysis of multiple-site LFP, EEG and spike train recordings;- JIDT

^{74}(Lizier, 2014) – an information-theoretic Java toolbox for studying dynamics of complex systems;- MEAbench

^{75}(Wagenaar et al., 2005) – a C++ toolbox for multi-electrode data acquisition and online analysis;- MEA-tools

^{76}(Egert et al., 2002) – a collection of MATLAB-based tools to analyze spike and LFP data from extracellular recordings with multi-electrode arrays;- MuTe

^{77}(Montalto et al., 2014) – a MATLAB toolbox to compare established and novel estimators of the multivariate transfer entropy;- MVGC

^{78}(Multivariate Granger Causality MATLAB Toolbox) (Barnett and Seth, 2014) – a MATLAB toolbox facilitating Granger-causal analysis with multivariate multi-trial time series data;- neuroGLM

^{79}(Park et al., 2014) – an MATLAB tool, an extension of GLMspiketraintutorial allowing more advanced features of GLM modeling such as smooth basis functions for spike-history filters, memory-efficient temporal convolutions, different timescales for stimulus and spike-history components, low-rank parametrization of spatio-temporal filters, flexible handling of trial-based data;- NIMclass

^{80,81}(McFarland et al., 2013) – a MATLAB implementation of the nonlinear input model. In this model, the predicted firing rate is given as a sum over nonlinear inputs followed by a “spiking nonlinearity” function;- nStat

^{82}(neural Spike Train Analysis Toolbox) (Cajigas et al., 2012) – an object-oriented MATLAB toolbox that implements several models and algorithms for neural spike train analysis;- OpenElectrophy

^{83,84}(Garcia and Fourcaud-Trocmé, 2009) – a Python framework for analysis of intro- and exrta-cellular recordings;- PyEntropy

^{85}(Ince et al., 2009) – a Python module for estimating entropy and information theoretic quantities using a range of bias correction methods;- seqNMF

^{86}(Mackevicius et al., 2019) – a MATLAB toolbox for unsupervised discovery of temporal sequences in high-dimensional datasets with applications to neuroscience;- SigMate

^{87}(Mahmud et al., 2012) – a MATLAB toolbox for extracellular neuronal signal analysis;- sigTOOL

^{88}(Lidierth, 2009) – a MATLAB toolbox for spike data analysis;- Spike Viewer

^{89}(Prö pper and Obermayer, 2013) – a multi-platform GUI application for navigating, analyzing and visualizing electrophisiological datasets;- SPIKY

^{90,91}(Kreuz et al., 2015) – a MATLAB graphical user interface that facilitates application of time-resolved measures of spike-train synchrony to both simulated and real data;- SPKTool

^{92}(Liu et al., 2011) – a MATLAB toolbox for spikes detection, sorting and analysis;- spykesML

^{93}(Benjamin et al., 2018) – a Python toolbox with a tutorial for comparing performance of GLM with modern machine-learning methods (neural networks, random forest etc.);- STAR

^{94}(Spike Train Analysis with R) (Pouzat and Chaffiol, 2009) – an R package to analyze spike trains;- STAToolkit

^{95}(Spike Train Analysis Toolkit) (Goldberg et al., 2009) – a MATLAB package for the information theoretic analysis of spike train data;- tensor-demo

^{96}– a MATLAB and Python package (available for both languages) for fitting and visualizing canonical polyadic tensor decompositions of higher-order data arrays;- tensortools

^{97}– a Python package for fitting and visualizing canonical polyadic tensor decompositions of higher-order data arrays;- TD-GPFA

^{98}(time-delayed Gaussian-Process Factor Analysis) (Lakshmanan et al., 2015) – a MATLAB implementation of GPFA method extension that allows for a time delay between each latent variable and each neuron;- ToolConnect

^{99}(Pastore et al., 2016) – a functional connectivity C# toolbox with GUI for in vitro networks;- Trentool

^{100}(Lindner et al., 2011) – a MATLAB toolbox for the analysis of information transfer in time series data. Trentool provides user friendly routines for the estimation and statistical testing of transfer entropy in time series data.

## CONFLICT OF INTEREST STATEMENT

The authors have declared that no competing interests exist.

## AUTHOR CONTRIBUTIONS

VU performed the reported study. VU wrote and AG edited the paper. Both authors have seen and approved the final manuscript.

## FUNDING

The work was supported by grants of the Deutsche Forschungsgemeinschaft through the Collaborative Research Center 889 “Cellular Mechanisms of Sensory Processing” and the Research Unit 1847 “The Physiology of Distributed Computing Underlying Higher Brain Functions in Non-Human Primates”, and by the European Commission through the H2020 project Plan4Act (FETPROACT-16 732266), all granted to AG. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

## DATA AVAILABILITY STATEMENT

The datasets analyzed and generated for this study can be found at (Perich et al., 2018; Lowet et al., 2015) and on GitHub^{101}, correspondingly.

## Footnotes

↵

^{1}https://github.com/ValentinaUn/Testing-open-source-toolboxes↵

^{2}according to Google Scholar in March 2019 (https://scholar.google.com)↵

^{3}when the code is available under a license which allows free redistribution and the creation of derived works↵

^{7}https://github.com/piermorel/gramm/blob/master/gramm%20cheat%20sheet.pdf↵

^{8}https://elephant.readthedocs.io/en/latest/tutorial.html↵

^{9}https://github.com↵

^{12}see https://neuroimage.usc.edu/brainstorm/Introduction, Chronux folder`dataio`and http://www.fieldtriptoolbox.org/dataformat for details, correspondingly↵

^{13}One can recompile locfit by running locfit/source/compile.m↵

^{16}https://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators↵

^{21}https://neuroimage.usc.edu/brainstorm/Tutorials/Statistics↵

^{22}https://neuroimage.usc.edu/brainstorm/Tutorials/Statistics↵

^{23}https://neuroimage.usc.edu/brainstorm/e-phys/functions↵

^{25}http://www.fieldtriptoolbox.org/reference/ft_timelockstatistics/↵

^{27}https://neuroimage.usc.edu/brainstorm/e-phys/SpikeSorting?highlight=%28sorting%29↵

^{28}https://neuroimage.usc.edu/brainstorm/e-phys/functions↵

^{29}https://neuroimage.usc.edu/brainstorm/Tutorials/ArtifactsFilter↵

^{30}https://neuroimage.usc.edu/brainstorm/Tutorials/TimeFrequency↵

^{31}http://www.fieldtriptoolbox.org/example/determine_the_filter_characteristics/↵

^{32}http://www.fieldtriptoolbox.org/tutorial/timefrequencyanalysis/↵

^{33}http://chronux.org↵

^{34}http://www.fieldtriptoolbox.org/reference/ft_preprocessing↵

^{35}https://www.mathworks.com/help/signal/ug/remove-the-60-hz-hum-from-a-signal.html↵

^{36}http://chronux.org/chronuxFiles//Documentation/chronux/spectral_analysis/continuous/locdetrend.html↵

^{37}http://www.fieldtriptoolbox.org/reference/ft_preproc_detrend/↵

^{38}https://de.mathworks.com/help/matlab/data_analysis/detrending-data.html↵

^{39}https://neuroimage.usc.edu/brainstorm/Tutorials/Connectivity↵

^{40}https://github.com/Leo-GG/NeuroFun/blob/master/%2Bcorrel/calcSTTC.m↵

^{41}http://www.fieldtriptoolbox.org/tutorial/connectivity/↵

^{42}http://www.fieldtriptoolbox.org/reference/ft_crossfrequencyanalysis/↵

^{43}http://www-users.med.cornell.edu/~jdvicto/pubalgor.html↵

^{44}http://wwwold.fi.isc.cnr.it/users/thomas.kreuz/Source-Code/VanRossum.html↵

^{45}http://www.fieldtriptoolbox.org/reference/ft_connectivityplot/↵

^{47}https://github.com/Mark-Kramer/Case-Studies-Kramer-Eden↵

^{52}https://github.com/Mark-Kramer/Case-Studies-Kramer-Eden↵

^{53}http://chronux.org↵

^{54}http://users.ece.cmu.edu/~byronyu/software/DataHigh/datahigh.html↵

^{55}http://cortivis.umh.es↵

^{59}https://github.com/NeuralEnsemble/elephant/commits/master↵

^{65}https://crcns.org/data-sets/motor-cortex/pmd-1/about-pmd-1↵

^{69}https://www.mathworks.com/MATLABcentral/fileexchange/54465-gramm-complete-data-visualization-toolbox-ggplot2-r-like↵

^{71}http://www.ibtb.org↵

^{76}http://material.brainworks.uni-freiburg.de/research/meatools/↵

^{77}https://figshare.com/articles/MuTE_toolbox_to_evaluate_Multivariate_Transfer_Entropy/1005245↵

^{91}http://wwwold.fi.isc.cnr.it/users/thomas.kreuz/Source-Code/SPIKY.html↵

^{92}https://sourceforge.net/projects/spktool/files/latest/download↵

^{94}https://sites.google.com/site/spiketrainanalysiswithr/↵

^{101}https://github.com/ValentinaUn/Testing-open-source-toolboxes