Robust single neuron tracking of calcium imaging in behaving Hydra

Abstract

Measuring the activity of neuronal populations with calcium imaging can capture emergent functional properties of neuronal circuits with single cell resolution. However, the motion of freely behaving animals, together with the intermittent detectability of calcium sensors, can hinder automatic long-term monitoring of the activity of individual neurons and the subsequent statistical characterization of neuronal functional organization. We report the development and open-source implementation of a multi-step cellular tracking algorithm (Elastic Motion Correction and Concatenation or EMC²) that compensates for the intermittent disappearance of moving neurons by integrating local deformation information from detectable neurons. We demonstrate the accuracy and versatility of our algorithm using calcium imaging data from behaving Hydra, which experiences major body deformation during its contractions. We quantify the performance of our algorithm using ground truth manual tracking of neurons, along with synthetic time-lapse sequences, covering a large range of particle motions and detectability parameters. Combining automatic monitoring of single neuron activity over long time-lapse sequences in behaving animals with statistical clustering, we characterize and map neuronal ensembles in behaving Hydra. We document the existence three major non-overlapping ensembles of neurons (CB, RP1 and RP2) whose activity correlates with contractions and elongations. Our results prove that the EMC² algorithm can be used as a robust platform for neuronal tracking in behaving animals.

I- INTRODUCTION

Tracking single neuron activity in freely behaving animals can help a detailed understanding of how neural circuits integrate external information, compute, learn and control animal behavior. Calcium imaging remains the gold standard for measuring single neuron activity as it is non-invasive and allows the simultaneous measurement of hundreds to thousands of cells, with single cell resolution [1]. The monitoring of single neuron activity in freely moving animals such as rodents can be achieved with miniaturized microscopes attached to the head of animals [2]. However, technical limitations of current microscopy techniques and of mathematical analysis hinder the complete imaging and understanding of such complex brains. Therefore, an alternative strategy is to monitor single neuron activity of an entire nervous system of a smaller animal, one that can fit entirely within a microscope’s field of view, such as Caenorhabditis elegans [3], Hydra [4], Zebrafish [5] or Drosophila larvae [6]. Another advantage of simple model organisms is that they contain many fewer neurons than mammals and have a limited repertoire of behaviors [7] that could be entirely measured in a near future.

Aside from the difficulties in 3D imaging of entire nervous systems in model animals {Yang, 2017 #530}, an important bottleneck in analyzing calcium imaging data is to achieve robust and automatic tracking of individual cells over long time-lapse sequences while the animal is freely behaving. Single-cell tracking is challenging for three main reasons: First, there could be a large number of cells in a cluttered environment. Therefore, false positives and negatives during single cell detection and localization impede the association of detected neurons between successive time frames and call for more elaborate tracking algorithms. Second, neurons can remain undetectable over large periods of time because calcium sensors can be significantly brighter than background only when neurons are firing. Third, tracking methods in freely behaving animals have to handle animal motion [5] and also potential body deformations [8]. For these reasons, standard single particle tracking methods are not sufficiently robust, and tracking is usually performed manually [4] or semi-manually [9]. This limits the analysis to a few hundreds of frames, introduces operator bias and ultimately, hinders our understanding of the functional organization of nervous systems.

To track particles with intermittent detectability (neurons) in a cluttered and deforming environment, we report the development of a method and software named Elastic Motion Correction and Concatenation (EMC²). In contrast to traditional tracking methods, EMC² does not set expected priors for particle motion (diffusion and/or linear motion typically). Instead, it uses information about local motion and deformation contained in detectable and tracked particles in the neighborhood of undetectable particles. EMC² is therefore more versatile, and does not require motion priors or heuristics to close potentially long tracking gaps. For the local tracking of detectable particles, EMC² uses a probabilistic method and is therefore robust to very cluttered conditions. We validate the robustness and accuracy of EMC² with manual tracking of neurons in calcium imaging of the Hydra nervous system while the animal is freely behaving and deforming. Hydra imaging datasets represent perhaps the worst possible scenario for tracking purposes, since animals can have major changes in body size with non-isometric deformations. We also measure the performance of EMC² using simulations of fluorescence time-lapse sequences with different types of motion (confined diffusion, linear displacement and elastic deformation), and show that EMC² outperforms state-of-the art tracking algorithms.

After having integrated EMC² in the open-source and freely available platform Icy [10] (icy.bioimageanalysis.org), we monitored the activity of single neurons in fluorescent calcium imaging of behaving Hydra, and characterized the functional clustering of individual neurons into co-active ensembles [11]. Consistent with previous observations [4], we find that Hydra contains three main neuronal ensembles (CB, RP1 and RP2); after mapping the positions of individual neurons from each ensemble, we also confirmed that these ensembles are not overlapping, i.e., they do not share neurons, and that they are correlated with contraction bursts and elongation behaviors.

These results demonstrate that EMC² is an effective tracking algorithm for the long-term tracking of single neuron activity in calcium imaging of living animals. Robust tracking constitutes a prerequisite for the statistical analysis of the functional organization of neural circuits, the description of emergent computational units such as neuronal ensembles, and ultimately, for the understanding and prediction of animals’ adaptive behavior.

II- RESULTS

1- Limitations of state-of-the-art tracking algorithms

Most tracking algorithms rely on the automatic detection of particles (cells, molecules…) that are significantly brighter than the noisy background in each frame of the time-lapse sequence (see [12] and [13] for review) and, subsequently, the linking of detections between frames corresponding to the reconstruction of coherent particle trajectories (see Table 1). False positives (i.e. background signal) and negatives (i.e. missing detections) in the detection of particles, together with the influence of high particle density and stochastic dynamics has, over the last two decades, motivated the development of algorithms that go beyond naïve tracking methods that simply associate nearest-neighbor detections between consecutive time frames (see Table 1 and [14, 15] for a review of existing methods).

Table 1: A review of SPT methods in bio-imaging

One category of elaborated tracking algorithms is based on global distance minimization (GDM) between all pairs of detections in consecutive time frames. The distance measure between detections can be simply the Euclidean distance, or can additionally take into account the similarity of the intensity and/or shape of the detected particle [16]. To handle possible missing and false detections, heuristics for track termination and initiation are defined by the user [17]. GDM methods are fast and robust, but user-defined parameters hinder their applicability in very cluttered conditions [14]. Moreover, the limitations of current particle motion models (confined diffusion and linear directed motion [16-18]) prevent the robust estimation of particle positions when they remain undetectable over long periods of time, as for sparsely firing neurons in calcium imaging. This limits the ability to close gaps in the trajectories of tracked particles to a very few frames.

Probabilistic methods are an attractive alternative to GDM methods, even if their computational cost is higher. Probabilistic algorithms model both the stochastic motion of the particles and their detectability, and then compute the optimal tracking solution by maximizing the model likelihood of observed detections [14, 19, 20]. The gold standard of probabilistic association is multiple hypothesis tracking (MHT) in which one computes all the possible tracking solutions over the entire time-lapse sequence, before inferring the tracking solution that maximizes the observation’s likelihood. However, MHT is generally not computationally tractable. Approximate solutions that iteratively compute a nearly optimal solution over a limited number of frames (typically up to 5) have been proposed [14]. As probabilistic methods model the particles’ detectability, they are usually more robust than GDM methods in cluttered conditions. However, the small number of frames considered when approximating the MHT solution together with limited particle motion models (again diffusion and/or linear displacements [14, 21]), reduce the capability of probabilistic algorithms to close large tracking gaps and keep track of particles’ putative position when they remain undetectable over many frames.

2- EMC² algorithm

EMC² is a method and software to track single particles with intermittent detectability in cluttered environments. It is particularly well-suited for tracking single neuron activity with calcium imaging in a behaving animal. The EMC² algorithm can be decomposed into four main steps (Figure 1). First, bright spots (e.g. firing neurons in calcium imaging sequence) are automatically detected with a robust method based on the wavelet decomposition of the time-lapse sequence and the statistical thresholding of wavelet coefficients (Materials and Methods). Second, detected spots are linked into single particle trajectories with a state-of-the-art probabilistic algorithm, a variant of multiple hypothesis tracking (eMHT [14]), which is particularly robust in cluttered conditions. Obtained tracks correspond to trajectories of detectable particles. However, in many time-lapse sequences such as calcium imaging of neuron activity, tracks would be terminated prematurely when particles switch to an undetectable state (e.g. non-firing neuron) and new tracks would be generated when particles can be detected again (e.g. firing neurons). This would create large time-gaps in individual tracks that need to be closed to allow the accurate tracking of each particle’s identity over the whole time-lapse sequence. Thus the two last steps of our method aim to close gaps in trajectories using information about the motion and deformation of the field of view along the time-lapse sequence. We considered that tracked particles are embedded in a deformable medium (e.g. neurons within tissue) and that estimation of the deformation of the local field-of-view should allow the inference of particles’ positions even when they are undetectable.

Figure 1: Multi-step EMC² for tracking single neuron activity in calcium imaging data.

1- Time-lapse imaging (N frames) of intermittent fluorescence activity of a neuron in a deforming environment (e.g. behaving animals). 2- Fluorescent spots (neurons) that are significantly brighter than background are automatically detected with a wavelet-based algorithm. 3- Tracks of detectable neurons are robustly reconstructed using probabilistic tracking algorithm (eMHT). 4- Short tracks of detectable particles are used to compute the elastic deformation of the field of view at each time frame. Associated detections in neuron tracks are used as fiducials and the whole deformation is interpolated using a poly-harmonic thin-plate spline function. 5- After having corrected for the deformation of the field-of-view where neurons are embedded, gaps between the end- and starting-points of short tracks are closed by minimizing the global Euclidean distance between points. 6- Finally, complete single neuron tracks over the whole time-lapse sequence are obtained by applying the elastic transformation of the field-of-view to concatenated short tracks.

The third step of EMC² is therefore the computation of the elastic deformation of the field-of-view at each time using the information contained in tracks of detectable particles. For this, we used the positions of tracked particles between consecutive time frames as fiducial source and target points. We then computed the forward elastic deformation of the whole field by interpolating the deformation at any position between fiducials with a thin-plate-spline function. The thin-plate spline is a popular poly-harmonic spline whose robustness in image alignment and point-set matching has been demonstrated [22], and which has been recently applied for automatic neuron registration in time-lapse sequences [8]. The fourth and last step of our method is the iterative estimation and correction of the elastic deformation of the field-of-view, followed by the optimal concatenation of short tracks. In this last step, we used the elastic transformation computed with thin-plate splines to propagate forward the putative positions of undetectable particles, following the termination of their detectable tracks. After having corrected for the elastic deformation of the field-of-view, we then linked short tracks by minimizing the global distance between the end-points of prematurely terminated tracks with the starting-points of newly appearing tracks (Materials and Methods). Finally, single-particle tracks over the whole time-lapse sequence are obtained by applying the computed elastic transformation to the concatenated short tracks.

Contrary to gap-closing GDM approaches [17], EMC² contains only one free parameter: the maximal distance between propagated end- and starting-points of short tracks for concatenation. Moreover, it handles complex natural motions and deformations, contrary to GDM methods that only account for confined or linear motion. EMC² is therefore more robust and versatile. For the sake of reproducibility and dissemination of our method, we implemented the EMC² multi-step procedure in the bio-image analysis software suite Icy [10] (http://icy.bioimageanalysis.org/). Icy is an open-source platform that is particularly well-suited for multi-step analysis thanks to graphical programming (plugin protocols) where each step of the analysis can be implemented as a block with inputs and outputs that can be linked to the other blocks (Figure 2 and Materials and Methods). Our method builds on well-established Icy preprocessing functions for spot detection and tracking.

Figure 2: Implementation of the EMC² algorithm in Icy platform.

Time-lapse sequence of fluorescent particles is input to a multi-step, automatic protocol in Icy. A first series of blocks, highlighted in blue, detects the position of fluorescent neurons (spots) in each frame of the time-lapse sequence. Block 1 uses the wavelet transform of each image and statistical thresholding of wavelet coefficients to determine spots that are significantly brighter than background. To separate close spots that form clusters in the wavelet-based mask of the image, the thresholded sequence is convolved with a log-gaussian kernel to enhance single spots (block 2), and local maxima algorithm is applied (block 3). A second series of blocks, highlighted in red, computes single particle tracks from computed spot positions. First, the Bayesian tracking algorithm (eMHT) computes tracks of detectable particles (block 4). Due to fluctuating particle detectability, many Bayesian tracks are terminated prematurely and new track identities are created when particles can be detected again. To close detection gaps in single particle tracks, block 5 applies the EMC² algorithm. Final output of the Icy protocol is the collection of single particle tracks over the entire time-lapse sequence.

Figure 3: Testing EMC² robustness with synthetic simulations.

For each simulated type of motion (confined diffusion, linear motion and “Hydra-like” elastic deformation), we simulated the stochastic firing of neuronal ensembles and corresponding fluorescence dynamics in synthetic time-lapse sequences (see Materials and Methods for details). We then compared the performances of EMC² (blue) with TrackMate (no motion correction (red) or linear motion correction (magenta)). P-values are obtained with the Wilcoxon rank sum test over n=10 simulations in each case.

3- Validation of EMC²

a- Manual tracking in Hydra

To validate the capabilities of EMC², we first compared the results of our algorithm with manual tracking in calcium imaging sequences of neuron activity in freely behaving Hydra [4]. We used the first 250 frames of a time-lapse sequence previously acquired in a genetically-engineered animal [4] and automatically detected the active neurons (bright spots) using the multi-step detection process described in the Materials and Methods. We tracked the detected particles with the eMHT algorithm ([14], implemented in Spot tracking plugin in Icy) and obtained short Bayesian tracks (n = 784 tracks) for the detected neurons (step 4 of the Icy protocol in Fig. 2). We then manually concatenated all the corresponding short tracks, i.e. we closed gaps, and obtained complete tracks (n = 444 tracks) over the whole 250 frames. We observed that, before gap closing, tracks were significantly shorter than concatenated tracks, meaning that many tracks are indeed terminated prematurely by the undetectability of silent (non-firing) neurons. We measured the accuracy of EMC² by comparing the computed tracks with those obtained after manual association of short tracks, which we took as an approximate ground truth (see Materials and Methods). We also measured how tracks obtained with TrackMate [17] implemented in Fiji [18] matched this manual ground truth. TrackMate is a GDM method, based on optimal linear assignment between closest detections. To handle the gaps in tracking when particles are undetectable, TrackMate again uses a GDM algorithm to compute the optimal linear assignment between end- and starting-points of previously computed tracks. As a result, TrackMate uses the same type of gap closing algorithm as EMC² but without correcting for potential elastic deformation of the field-of-view. Finally, to evaluate how well the algorithm works simply to generate the initial set of short tracks, i.e. to compare the probabilistic eMHT used in EMC² with the GDM algorithm used in TrackMate, we also measured the accuracy of EMC², but without elastic motion correction before gap closing. Compared algorithms are summarized in Table 2. First, we found that EMC² (n = 453 tracks, with 410 (90.5%) matched tracks) outperformed TrackMate (n = 474 tracks, with 259 (54.6%) matched tracks) and EMC² without elastic correction (n = 514 tracks, with 279 (54.3%) matched tracks). The similar capabilities of TrackMate and EMC² without elastic motion correction indicate that Bayesian eMHT and the GDM tracking method perform similarly for local association of detectable spots, but fail at closing longer tracking gaps in deformable media. This highlights the importance of elastic motion correction before the optimal concatenation of short tracks.

Table 2: List of tested tracking algorithms in manual validation

b- Synthetic time-lapse sequences

Manual gap closing in time-lapse sequences of Hydra activity is tedious and prone to operator bias. Moreover, the ground truth, i.e. the identity of each individual neuron along the whole time-lapse sequence, is unknown. Therefore, we designed a reproducible, synthetic approach where we simulated individual neurons’ activity and animal motion with different sets of parameters.

We modeled three different types of motion and/or deformation (Materials & Methods): Confined diffusion, where blinking neurons diffuse within a confined area, Linear motion where neurons all move together in the same direction at constant velocity, and finally, using deformation fields measured in Hydra experimental data, we simulated naturalistic Hydra deformations. We further modeled the intermittent activity of neuronal ensembles with a probabilistic Poisson model. We also modeled the fluorescence dynamics of individual spikes using a parametric curve that we fitted to experimental data. Finally, using neuron positions, firing activity and fluorescence dynamics, we generated synthetic time-lapse sequences using a mixed Poisson-Gaussian noise model ([14] and Materials & Methods).

For confined diffusion, both EMC² and TrackMate gave excellent results, with track matches of 93.5% ± 1.6% (standard error, n=10 simulations) for EMC² and 92.9% ± 0.8% for TrackMate (Fig.3). The good performance of TrackMate was expected as this algorithm was initially designed to track confined endocytic spots at the cell membrane [17]. In addition to confined motion, TrackMate can also model linear motion of particles when computing the optimal gap closing between short tracks. Therefore, in linear motion simulations, we used TrackMate with linear motion correction instead of standard confined motion correction. However, even with linear correction, the performance of Trackmate (76.3% ± 0.6%) was significantly worse than EMC²’s performance (97.7 ± 0.5%). This difference is due to the different estimation methods that are used in the two tracking algorithms to estimate the direction of tracks: in TrackMate, the estimation of track directions is local, based on the last detection within each short track, whereas the estimation of track direction in EMC² uses global information provided by neighbouring short tracks and is therefore more robust. Finally, in the third case, we used the deformation field that we estimated over 250 frames within a time-lapse experimental sequence of Hydra (section 2.a and Materials & Methods). We found that Trackmate had similar performances with (matching score 69.4 ± 1.1%) or without (72.4 ± 0.9%) linear motion correction, and that both were outperformed by EMC² (92.7 ± 0.8%). Altogether these simulations show that EMC² is a robust tracking algorithm, independent of the type and complexity of particle motion, and is therefore a versatile method for single particle tracking.

4- Monitoring single neuron activity and characterizing neuronal ensembles in behaving Hydra

There is increasing experimental evidence that neurons are organized into neuronal ensembles composed of a few tens of highly coupled neurons, and that these co-active ensembles are the fundamental computational units of the brain rather than single neurons themselves [11, 23]. Using manual annotation, it has been shown that Hydra’s nervous system, one of the simplest of the animal kingdom, may be dominated by three main functional networks that extend through the entire animal [4, 24]. To confirm (or refute) these observations, we used EMC² and automatically tracked single neurons in n=13 time-lapse sequences (length 1067 ± 56 frames at 10 Hz (Materials and Methods) from 8 different animals (Table 3 and Fig. 4). These analyzed movies were significantly longer than the manually annotated one (length = 200 frames at 10 Hz [4]). Ensemble activity, corresponding to the co-firing of neurons, can be detected as significant peaks within the raster plot of single neuron activity (Fig. 4 and Materials and Methods). We detected a mean number of 18 ± 2.3 peaks per movie, corresponding to a mean rate of 1 activity peak every 59 frames (5.9 s) which corresponds well with the 4 peaks observed previously in the 200 frame movie [4]. To associate each peak with a putative neuronal ensemble, we adopted a similar approach as in [25] and measured the similarity between these events in terms of the identities of the participating neurons (Fig. 4 and Materials and Methods). We then performed k-means clustering of peak similarity, and used the Silhouette criterion [26] to determine the most likely number of neuronal ensembles causing the detected peaks of activity. We found 2 or 3 neuronal ensembles in each movie (3 neuronal ensembles were detected in 3 out of the 13 movies (23%)). We then categorized each detected ensemble into one of the previously defined ensembles [4, 24] : Contraction Burst (CB) neurons that fire during longitudinal contraction of the animal, Rhythmic Potential 1 (RP1) that fire during the longitudinal elongation of the animal, and Rhythmic Potential 2 (RP2) neurons that fire independently of RP1 and CB activity. We found CB neuronal ensembles in almost all movies (11/13 (85%) movies) and all animals (8/8 (100%)), RP1 ensembles in all movies and animals and RP2 ensembles in fewer movies (5/13 (38%)) and only 2/8 (25%) animals. The absence of detected CB ensembles in 2 movies corresponds to the observed absence of contraction cycles within these movies. On the other hand, we hypothesize that the absence of RP2 ensembles in 6/8 (75%) of the animals is due to the limited depth of the field-of-view in confocal microscopy (see Materials and Methods). Indeed, RP2 neurons lie in the thin ectoderm of the animal [4] that may not have been imaged in some animals. Finally, we classified and mapped each individual neuron in the detected ensembles (see Materials and Methods). As in [4], we found that the CB ensemble was the most important group of neurons with a mean number of 100 ± 29 neurons representing 27% ± 5.7% of the total number of neurons (a mean of 349 ± 39 neurons were tracked over >200 frames in the different movies). The RP1 ensemble, with 50 ± 12 neurons (14% ± 2.7%), was the next largest ensemble and RP2, with 49 ± 13 (11 ± 2.9%), was the third. The relative number of neurons in the different ensembles is in agreement with previous observations [4]. However, the overall size of each ensemble is smaller than the size reported previously. This is due to the fact that automatic classification of each individual neuron in an ensemble is more stringent than the manual classification that had been previously performed.

Table 3: Detailed results of statistical extraction of neuronal ensembles in Hydra (n=8 animals)

Table 4: Parameters used for synthetic simulations

Figure 4: Single neuron tracking and mapping of neuronal ensembles in behaving Hydra.

a- Calcium imaging of single neuron activity in behaving Hydra. The images and analysis of the 3^rd movie of animal 1 are given as representative examples. b- Single neuron tracks and fluorescence intensity are obtained with EMC² algorithm. c- For each neuron, spikes are extracted from fluorescence traces. Peaks of activity (highlighted with red stars) correspond to significant co-activity of individual neurons (sum of individual activities (solid red line) > statistical threshold (dashed red line), p=0.001 see Materials and Methods). Each peak putatively corresponds to the activation of one neuronal ensemble. d- Similarity between activity peaks is computed using the identities of individual neurons that are firing at each peak (see Materials and Methods). e- The optimal number of peak classes (that putatively correspond to the number of neuronal ensembles) is computed with k-means clustering of the similarity matrix. Median fluorescence trace of each neuronal ensemble and corresponding activity peaks are shown. The classification of individual neurons in each ensemble is determined based on their firing at ensemble peaks (see Materials and Methods). f- Individual neurons of each ensemble can be dynamically mapped in the original time-lapse sequence.

Altogether, these results show that EMC² is sufficiently robust to monitor single neuron activity over long periods of time in a freely behaving animal. This constitutes a fundamental prerequisite for the analysis of neurons’ functional organization and, ultimately, for our understanding of their emergent computational properties. Here, by coupling the automatic tracking of individual neurons in multiple Hydra with robust clustering analysis of neuronal activity, we were able to confirm the previous observations that the Hydra nervous system is dominated by three main non-overlapping ensembles that are involved in different animal behaviors.

III- CONCLUSION

When using calcium imaging in freely behaving animals, an important challenge is the sustained detectability of non-active neurons over potentially long periods of time in a moving and deformable environment. To tackle this issue, we have developed an algorithm, EMC², that tracks detectable particles with a state-of-the-art probabilistic tracking algorithm, and uses the information contained in reconstructed tracks about the local deformation of the field-of-view (e.g. animal) to estimate the position of undetectable particles and potentially close long tracking gaps. We validated the performance and versatility of EMC² by comparing its performance with a state-of-the-art tracking algorithm on manually tracked neurons in time-lapse calcium-imaging of Hydra and on synthetic time lapse sequences that modeled different types of motion/deformation of the field-of-view (confined diffusion, linear motion and Hydra-like elastic deformation). In all cases, EMC² showed high accuracy (>90% of reconstructed tracks matched ground truth), and outperformed state-of-the-art tracking methods.

Compared to traditional tracking approaches, composed only of particle detection and linking (see table I), our hybrid algorithm is better equipped to handle long tracking gaps and general particle motion. Recently, tracking methods based either on point-set registration [8] or artificial neural networks (autoencoders) [27] have been introduced to handle more general particle motion, not only diffusion and/or linear motion. However, these methods are designed to link two sets of particle detections in consecutive time frames and, even if they can handle some missing or false detections, they are not well suited for long tracking gaps such as those encountered in calcium imaging of neuronal activity.

After having implemented EMC² in the open-source bio-imaging platform Icy [10], we tracked single neuron activity in behaving Hydra over long time-lapse sequences (∼ 1000 frames at 10 Hz, see table 3). Using statistical clustering, we confirmed the previous observations that neural activity of Hydra is dominated by three major, non-overlapping neuronal ensembles that are involved in the animal’s repetitive contraction and elongation. At the same time, automatic clustering of neuronal activity was not able to extract other smaller ensembles of the animal’s nervous system, such as the tentacle and sub-tentacle ensembles [4] that we could observe by eye. Indeed, the coordinated activity of small ensembles is difficult to detect from individual neuronal spiking because of sparse activity and noise. Moreover, the activation of these smaller ensembles is less frequent than the activation of the three major ensembles and happened in only few movies. Therefore, the complete mapping of neuronal ensembles of Hydra, with the characterization of even the smallest neuronal ensembles with less frequent activity, will require in the near future further development of better imaging and tracking of single neuron activity over longer periods of time.

To conclude, our results show that EMC² is a robust and versatile tracking algorithm that allows monitoring and quantification of single neuron activity in freely behaving animals over long periods of time. Robust tracking of neural activity is a first step towards a better understanding of the neural code, i.e. how connected neuronal ensembles integrate information, trigger adaptive behavior and, more generally, compute the animal’s behavioral or internal states.

IV- MATERIALS AND METHODS

1- Elastic motion correction and concatenation (EMC²) of short tracks

The eMHT algorithm returns a set of N short tracks S_i, for 1 ≤ i ≤ N, of detectable particles; the i^th track ends at position at time . We iteratively estimate the putative position of the undetectable particle for any by applying the forward thin-plate-spline transformations TPS_forward(t → t + 1) estimated from to t = t₀. If denotes the starting time of short track S_j, we then compute the distance d_i,j between the different short tracks using the propagated positions of end-points: where gap_max is the user-defined maximum time gap that EMC² is allowed to close (typically a few hundreds of frames). We need to apply a maximum time gap for time-lapse sequences with particles that remain undetectable over long period of time, such as neurons in calcium-imaging sequences, due to growth of the error in the forward estimation of the putative position of undetectable particles.

Using the distances between short tracks, we then define an association cost matrix, with ϕ_i,j = d_i,j if d_i,j < d_max, and ϕ_i,j = ∞ otherwise. d_max is another parameter of our tracking algorithm that specifies the maximum distance allowed between the propagated end-point of a short track and the starting-point of another short track for their putative concatenation. Finally, among all possible associations for which the cost ϕ_i,j < ∞, the optimum set of concatenated tracks i^* → j^* among the short tracks {i, j}, 1 ≤ i, j ≤ N is the solution of the global minimization problem

This minimization problem, known as the assignment problem, is similar to the problem solved in GDM methods of tracking, where algorithms determine the optimal association between particle detections by minimizing the global distance between detections in consecutive time frames of the sequence. One of the first and most popular algorithms to solve assignment problems is the Hungarian algorithm {Kuhn, 1955 #537}. However, due to its computational load, faster algorithms have been proposed over the years. We used here the Jonker-Volgenant algorithm {Jonker, 1986 #538} implemented in the TrackMate plugin in ImageJ (see table 1).

2- Icy protocol

a- Detection of spots (e.g. neurons) in time-lapse sequences

To detect automatically the positions of fluorescent spots, corresponding to detectable particles, in each frame of the time lapse sequence, we designed a multi-step algorithm (see Fig. 2) where we first detected fluorescent spots that are significantly brighter than background with a fast and robust algorithm based on a wavelet transformation of the image and statistical thresholding of the wavelet coefficients (block number 1 in Icy protocol (Fig. 2)) [28]. These spots correspond to individual particles or clusters of particles (e.g. neurons). To separate individual particles in the detected clusters, we then multiplied the original sequence with the binary mask obtained with wavelet thresholding and convolved the result of the multiplication with a log-Gaussian transformation (block number 2). The log-Gaussian convolution ressembles the point-spread function of microscopes and thus enhances individual particles [29]. Finally, we extracted the positions of single particles by applying a local-maxima algorithm (block number 3) to the convolved sequence.

b- Single-particle tracking (EMC²)

After having detected the positions of fluorescent spots in each time frame, a second series of blocks computed the tracks of each single particle. First, block number 4 used the positions of spots and a robust Bayesian algorithm (eMHT [14]) to compute single tracks of detectable particles. Due to fluctuating detectability, many computed tracks are terminated prematurely and new tracks are created when particles are detectable again. We thus applied EMC² algorithm (block number 5) to close gaps and reconstruct single-particle tracks over the entire time lapse sequence.

3- Validation of EMC²

a- Comparison metric

To compare the tracks obtained with EMC² and other automatic tracking algorithms with ground truth tracks, we first considered the whole set of detections x_i(t), 1 ≤ i ≤ N(t), with N (t) the number of detections at time 1 ≤ t ≤ T (T being the length of the time sequence) and assigned each detection to the closest active track at time t. Therefore, for each reference track , with |Θ^r| the total number of reference tracks, and for each test track , with |Θ^t| the total number of test tracks, we obtained a set of associated detections. We then considered that a reference track; a test track matched if it shared at least 80% of common detections with the reference. Finally, for each reference track, we either obtained no test track that matched, exactly one test track that matched or more than one.

b- Synthetic motions

To validate the robustness and accuracy of EMC² in different scenarios, we simulated three classes of motions: confined diffusion, linear motion and elastic deformation. For confined diffusion, each simulated spot (e.g. fluorescent neuron) can diffuse with coefficient D = 1 pixel² per frame and is confined to a 10 pixel disk area. For linear motion, each simulated spot moves linearly at speed v = 1 pixel per frame. When a track reaches the boundary of the field-of-view (a rectangle of 200×200 pixels), it is terminated and another track is initiated at the other side of the FOV. Finally, for elastic deformation, we used the experimental tracks in Hydra to estimate iteratively (i.e. from one frame to the following one) the local deformation for each synthetic track position.

c- Firing rates of individual neurons

To model the stochastic firing rates of individual neurons (total number of neurons n_neurons), we first determined a proportion (α_stable) of stable spots, i.e. non-blinking cells, with constant fluorescent intensity. In Hydra, stable cells typically correspond to nematocytes or other cell types that also express fluorescent proteins after the genetic editing of the animal, but that don’t fire as neurons do [4]. To simulate the correlated activity patterns observed in Hydra, we then divided the (1 − α_stable)n_neurons firing neurons, with intermittent activity and detectability, into n_group ensembles. All neurons in each ensemble fire simultaneously with Poisson rate λ_group = size_group λ_individual, with λ_individual the firing rate of individual neurons and size_group = (1 − α_stable) n_neurons / n_group the number of neurons in each group. Parameters for each simulation used for the validation of EMC² algorithm are summarized in Table 3.

d- Generation of synthetic images

To generate synthetic fluorescence time-lapse sequences, we used a mixed Poisson-Gaussian model [15]. In this model, the intensity I[x, y] at pixel location [x, y] is equal to I[x, y] = U[x, y] + N(0, σ_n) where U is a random Poisson variable and N(0, σ_n) is additive white Gaussian noise with standard deviation σ_n. The intensity λ[x, y] of the Poisson variable varies spatially because it depends on the presence or not of particle spots (neurons). Therefore, λ[x, y] = P[x, y] + B with B a constant background value and P [x,y] the spots’ intensity at position [x,y]. Assuming an additive model for the intensity of the spots, where P_i[x,y] is the signal originating from the i^tϕ spot. We approximated the point-spread-function (PSF) of the microscope with a Gaussian profile. Thus, for a particle located at position , its intensity at position [x,y] is given by , with A_i the amplitude of the i^tϕ particle and σ_PSF the standard deviation of the 2D Gaussian profile of the PSF. We chose a constant amplitude for each particle A = A_i, for all 1 ≤ i ≤ n_neurons Parameters used in simulations are summarized in Table 3.

e- Fluorescence kinetics

When a neuron fires at time t₀, we model its fluorescence time course with the general kinetics equation where the numerator models a power-law exponential decay of the fluorescence (β= 1 models a standard single exponential decay), with a decay time constant τ_decay, and the denominator models a sigmoidal increase of fluorescence with median τ_rise and time constant μ. Kinetics parameters for each synthetic simulation are summarized in Table 3. These parameters were obtained by fitting n = 3075 individual spikes from 444 individual neuron tracks in an experimental time-lapse sequence (250 frames at 10 Hz) of GCAMP-labeled Hydra [4] (Supplementary Figure 1). We highlight that, for Hydra elastic simulations, we used a long decay time constant and a power index β= 2 instead of 1 for standard confined diffusion and linear motion simulations.

4- Extracting functional ensembles in behaving Hydra

a- Hydra Maintenance

Hydra were cultured using standard methods {Lenhoff, 1970 #539} in Hydra medium at 18°C in the dark. They were fed freshly hatched Artemia nauplii twice per week.

b- Imaging

Transgenic Hydra expressing GCaMP6s in the interstitial cell lineage were used and prepared for imaging studies as previously described {Dupre, 2017 #276}. Calcium imaging was performed using a custom spinning disc confocal microscope (Solamere Yokogawa CSU-X1). Samples were illuminated with a 488 nm laser (Coherent OBIS) and emission light was detected with an ICCD camera (Stanford Photonics XR-MEGA10). Images were captured with a frame rate of 10 frames per second using either a 6X objective (Navitar HRPlanApo 6X/0.3) or a 10X objective (Olympus UMPlanFl 10x/0.30 W).

c- Extracting single neuron activity

We extracted the fluorescence trace of each individual neuron using the Track Processor Intensity profile within the TrackManager plugin in Icy [10]. For each detection within the track, the extracted intensity corresponded to the mean intensity over a disk centered at the detection’s position, with a 2 pixel diameter. When detections are missing (tracking gap), the intensity was set to 0. Then, for each individual neuron, we denoised its non-zero fluorescence trace using wavelet denoising (wdenoise) in Matlab. We then automatically extracted spikes with a custom procedure where we first computed the discrete derivative of the smoothed fluorescence signal, then set to 0 all negative variations and finally, we detected significant positive variations of the signal (discrete positive derivative > quantile at 98% of all empirical positive variations) that putatively corresponded to spikes.

d- Statistical characterization of neuronal ensembles

Neuronal ensembles are groups of neurons that repeatedly fire together. Therefore, the activity of neuronal ensembles can be detected as significant co-activity peaks in the raster plot of single neuron firing. To detect significant peaks of activity, we applied the procedure described in [25], and identified as peaks times at which the sum of single neuron activity within a time step of 100 ms fell within the quantile at 0.999% obtained empirically by circularly shuffling the individual spikes in the activity raster plot.

Then, to relate detected peaks of activity to putative neuronal ensembles, we constructed a vector describing the activity of each individual neuron at the detected peaks with entries 1 if the neuron fires at that peak, and 0 otherwise. We then computed the similarity between these vectors for each of the activity peaks, using the Jaccard index:

Then, to estimate the number of neuronal ensembles underlying the detected peaks of activity, we clustered the peaks with a k-means algorithm based on their similarity for different numbers of classes (from 1 to 5 classes typically). K-means clustering was performed using the cosine distance. The optimal number of classes in the k-means clustering algorithms, and therefore the putative number of neuronal ensembles, was computed using the Silhouette index [26]. For a clustering of N peaks into k classes, the Silhouette index is given by where a_i is the average distance from the i^th peak to the other peaks in the same cluster as i, and b_i is the minimum average distance from the i^th peak to peaks in a different cluster, minimized over clusters. An advantage of the Silhouette evaluation criterion over other clustering criteria is its versatility, as it can be used with any distance (cosine distance was used here for the k-means clustering). Finally, we assigned each individual neuron to an ensemble if that neuron fired in more than 30% of the activity peaks of the identified ensemble.

Competing Financial Interest

Authors declare no competing financial interests pertaining to this study.

Author Contributions

T.L., A.F. and R.Y. conceived the project. T.L developed and implemented the method. A.H. generated the genetically modified Hydra line, and acquired confocal time-lapse sequences. T.L analyzed data and wrote the manuscript. All authors edited the manuscript. R.Y assembled and directed the team, providing guidance and funding.

Acknowledgements

This work was supported by the NSF (CRCNS 1822550) and the NINDS (R01NS110422). This material is also based upon work supported by, or in part by, the U. S. Army Research Laboratory and the U. S. Army Research Office under contract number W911NF-12-1-0594 (MURI). T.L. was partly supported by the Fondation pour la Recherche Médicale and the Philippe Foundation. MBL research was supported by competitive fellowship funds from the H. Keffer Hartline, Edward F. MacNichol, Jr. Fellowship Fund, The E. E. Just Endowed Research Fellowship Fund, Lucy B. Lemann Fellowship Fund, Frank R. Lillie Fellowship Fund Fellowship Fun, Fries Trust Research Award, Hartline MacNichol Research Award, L. & A. Colvin Summer Research Fellowship, and John M. Arnold Fellowship Research Award of the Marine Biological Laboratory in Woods Hole, MA. We thank MBL staff and members of the MBL Hydra lab for support and advice. R.Y. is an Ikerbasque Research Professor at the Donostia International Physics Center.