Abstract
There is a need in modern neuroscience for accurate and automated image processing techniques for analyzing the large volume of neuroanatomical imaging data. Even at light microscopic levels, imaging mouse brains produces individual data volumes in the TerraByte range. A fundamental task involves the detection and quantification of objects of a given type, e.g. neuronal nuclei or somata, in brain scan dataset. Traditionally this quantification has been performed by human visual inspection with high accuracy, that is not scalable. When modern automated CNN and SVM-based methods are used to solve this classification problem, they achieve accuracy levels that range between 85 – 92%. However, higher rates of precision and recall that are close to that of human performance are necessary. In this paper, we describe an unsupervised, iterative algorithm, which provides a high performance for a specific problem of detecting Green Fluorescent Protein labeled nuclei in 2D scans of mouse brains. The algorithm judiciously combines classical computer vision techniques and is focused on the complex problem of decomposing strong overlapped objects of interest. Our proposed technique uses feature detection methods on ridge lines over distance transformation of the image and an arc based iterative spatial-filling method to solve the problem. We demonstrate our results on mouse brain dataset of Gigabyte resolution and compare it with manual annotation of the brains. Our results show that an aptly designed CV algorithm with classical feature extractors when tailored to this problem of interest achieves near-ideal human-like performance. Quantitative comparative analysis, using manually annotated ground truth, reveals that our approach performs better on mouse brain scans than general purpose machine learning (including deep CNN) methods.
I. Introduction
Understanding the cytoarchitecture of the brain is an important goal in the field of neuroscience [26], [52]. In particular, the identification of locations and quantification of neurons across the brain is a vital task. To achieve this goal, modern imaging methods have allowed users to produce images at very high resolution for further analysis. Entire mouse brains can now be imaged using light microscopy at sub-micron resolution from entire mouse brains (~ 1cm3 volume), with individual data volumes in the terrabyte range. Optical microscopic techniques applied at such high resolutions, help to study the brain at a single neuron resolution [37], [43] but the large data volumes make it a significant computational and data analytics challenge.
In addition to the advancements in imaging techniques, advanced methods such as the expression of the Green Fluorescent Protein (GFP) [12] have ensured that specific types of neurons can be tagged and studied in great detail [55]. A successful implementation of GFP tagging in the identification and quantification of interneurons is described in [55]. Manually studying the distribution of these GFP expressing neurons across the entire brain is often time consuming and is susceptible to human errors over large datasets. To overcome this, an automated process of detection and quantification of neurons [5], [28] must be designed based on well grounded image processing (or computer vision) algorithms. However, the level of sophistication in the analysis techniques has not yet caught up with the significant improvements in microscopic imaging methods. In this work, we have sought to develop an automated method of detection for GFP tagged interneurons across mouse brain datasets that is made up of high resolution image scans of brain sections. Thus, the aim of our work, proposed in this paper, is to detect and count GFP tagged specific cell types from entire mouse brains.
Automated algorithms are often resource consuming and have been deployed in studying the brains of lower species such as Candida elegans [56]. Computer vision algorithms have also played a vital role in the segmentation of large-scale electron microscopy data. Kaynig et al. [27] have proposed a pipeline that provides state-of-the-art reconstruction performance while scaling to data sets in the GB-TB range. Such high performance was achieved by training a random forest classifier on interactive sparse user annotations. This classifier output was then combined with an anisotropic smoothing prior in a Conditional Random Field (CRF) framework to generate multiple segmentation hypotheses per image. These segmentations were then combined into geometrically consistent 3D objects by segmentation fusion. A more recent work in automatic neural reconstruction from petavoxel of Electron Microscopy data [53] suggested a dense Automatic Neural Annotation framework (RhoANA) to automatically align, segment and reconstruct a 1mm3 brain tissue (~ 2 peta-pixels), using a web-based tool to manually proofread the output, and ensure reconstruction correctness. The pipeline performs membrane classification and 2D segmentation using state-of-the-art deep learning (DL) techniques in order to generate membrane probability maps.
Deep learning methods have been used in the detection of cells in cancer biology. One method for mitosis detection [4] is based on deep learning which performs data aggregation using convolutional neural networks (CNN) via additional crowdsourcing layer (AggNet). Mualla et al. [39] propose a method for automatic cell detection on polypropylene substrate, suitable for microbeam irradiation. A Harris corner detector was employed to detect apparent cellular features. These featurecorners were grouped based on a dual-clustering technique according to the density of their distribution across the image. Weighted centroids were then extracted from the clusters of corners and constituted the targets for irradiation. Quelhas et al. [46] introduce a new approach for cell detection and shape estimation to detect overall convex shapes in multivariate images based on a sliding band filter (SBF). Furthermore, the parameters involved are intuitive as they are directly related to the expected cell size. Using the SBF filter, cell nuclei, the cytoplasm location and shapes were detected. In the remainder of the paper, we will refer to the terms cells and GFP nuclei interchangeably (as our target objects for detection/segmentation).
An example of a high resolution image (2D) of a brain section is given in Figure 1. This brain section along with 240 brain sections form a complete brain imaged dataset of one particular mouse brain (Mouse Brain Hua-167 [1]). The dataset consists of images with a 18K × 24K resolution at 0.46 micron per pixel scale. The given example shows the distribution of the GFP expressing nuclei with higher distribution densities clearly visible in the colliculi. Figure 1b shows the extracted image of the green channel alone for the same brain section. Figure 2a shows a zoomed in example of a small cropped region from the example section (figure 1). Centers of the GFP nuclei were marked manually by human annotators for the entire brain and this ground truth has been shown in Figure 2b. Results of the object-detection methods based on Faster R-CNN [50] and SVM [16], [40] have also been shown. As demonstrated in figures 2(c) and 2(d), although these methods yield fairly accurate results, however produce a few false alarms (detections) and misses, especially when detecting strong overlapping objects (Section V discusses performance using quantitative measures). As a scientific and diagnostic tool, there is a need for a higher level of accuracy and higher rates of precision and recall. The major reason for the moderate accuracy given by these supervised methods is that, they do not capture the various non-unique patterns of spatial layouts and structures that are formed by clusters of neurons, specifically with strong overlap. Machine learning algorithms [7], [20] have been designed to produce high accuracy invariant to pose, scale (size), background and texture of the object. However, they are not immune to conditions where large amount of overlap (dense crowd, large clutter/pile of objects, say) occur and fail to inadequately capture the required objects of interest due to occlusion and camouflage. Deep-CNNs have yielded rich dividends in example cases, ranging from IMAGNET [19], PASCAL [21] image datasets, to HMDB [29], Hollywood-II [34], MS-COCO [30] video datasets, where considerable inter-class variability exists in the data, unlike in our dataset. Moreover, none of these publicly available datasets contain objects with large amount of clutter and strong overlap, which is the case in some scans of our brain dataset.
Detection of cell centers in cases of strong overlap of objects of interest is a difficult problem, as the overlapping objects result in complex 2D structures that do not exhibit a unique appearance pattern of shape, size and overlap formation (layout). This results in loss of accuracy in the training of any shallow or deep learning (DL) methods. However, human observations can intuitively sort these overlapping objects and it becomes necessary for any automated algorithm to achieve high accuracy close to that of human annotators but over a large scale (≃ 400 GB size images) with lesser computational time (few minutes on a single high-end CPU machine). With this aim, we have proposed an improved and efficient algorithm for the unsupervised classification of GFP tagged GABAergic interneurons across the mouse brain [8]. Our proposed method is designed suitably as a congruence of image analysis algorithms, such as Distance Transformation (DT) and edge detection to quantify the boundaries of the detected cells, followed by the extraction of features along ridge lines for detecting the candidate centers of neurons. The algorithm is further enhanced by an iterative approach of region filling by circular blobs of adaptive size, using convex arcs on the edge maps for detecting overlapped cell areas. Using this method, we have obtained a high precision of 0.972 and a recall of 0.961, over one complete brain section image set and a significant part of another (~ 240 scans in first set and ~ 158 in the other). A comparative performance analysis shows that our proposed algorithm surpasses two existing deep learning CNN models [50], as well as two popular shallow learning object detection algorithms. Shallow and deep models, used in this study, are not tailor-made for a structure decomposition problem which is an essential component necessary for detecting overlapping objects (GFP tagged nuclei in our case).
II. Overview of the Proposed Method
Individual sections of the mouse brain are obtained as gigapixel resolution (RGB 18K × 24K pixels at 12-bits per channel) images, from the mouse brain of the wild type mouse (C57BL6), with green fluorescent protein (GFP) tagging of GABAergic neurons (PV+) [1]. This data is part of the mouse brain architecture project [1] at the Cold Spring Harbor Laboratory (CSHL), which aims to quantify the connections of different parts of the brain using tracer injections and cell tagging techniques at a mesoscopic scale [8]. The CRE cell lines that tag interneurons were developed by Josh Huang’s Laboratory and have been described in [55]. The histological and imaging preparation of the brain has been previously described in [45]. Manual annotation of one complete set of a brain and a selected portion of another set was performed by researchers from the Indian Institute of Technology (IIT), Madras, India, making the comparison and evaluation of algorithms feasible on the scan data. Details of the manual annotation is described later in Section V.
Figure 3 shows two examples of zoomed in sections with GFP tagged neurons with different cell distributions (as density of occurrence) - minimal overlap and strong overlap. Objects of interest (nuclei), which are visible in the green spectrum, in the strong overlapping regions appear as a dense 2D packed structure with no well formed/defined shape and size, where more than 3 cells overlap (figure 3(b)). In addition, individual nuclei also appear with varying sizes within the regions of interest. This arbitrary spatial arrangement of cells with significant overlap makes it a challenging task to detect each one of them accurately. The unsupervised computer vision (CV) approach of cell detection proposed in this paper has been designed and formulated for precisely dealing with this scenario of detecting individual cells within a packed set of strongly overlapping nuclei. Geometric constraints of the cell layout have been used to decompose a structure into overlapping circular units.
A. Basis and Justification of the algorithm designed
For our problem scenario, the task involves detecting centers of the GFP tagged nuclei. Our solution is achieved by using Distance Transformation and edge detection followed by Ridge estimation as default processes of Stage I of our pipeline. This part of the designed process accurately detects cells in cases of sparse (isolated) distribution or minimal overlap of objects of interest. In the complex case of strong overlap of multiple cells (as seen in figure 3(b)), an additional process (Stage II) is required for accurate identification of the nuclei. This solution is where our algorithm considerably outperforms the state of the art machine learning models (like Faster R-CNN [50], SVM [16]). The process we have designed uses geometric constraints with the foreground area filling to decompose a 2-D area of overlapping cells into partly occluded cell bodies. The complexity in the data lies in the degree of overlap of objects within the image and is dependent on the type of object, the microscopic and sample preparation techniques. The deep learning models, although well suited for multiple object detection are not well suited to the overlap-decomposition problem which we have implicitly addressed in our pipeline. The simplicity of method when combined with close to human performance, makes it advantageous to use over other methods including CNNs for solving this problem.
B. Brief Description of the proposed algorithm
As the GFP tagged neurons have relatively higher irradiance in green channel compared to the background neuropil, we have enhanced the 12-bit green channel using contrast enhancement as a pre-processing step to produce the salient foreground region (FGR) as shown in figures 3(c) and 3(d).
It was observed that modern machine learning (ML) and deep learning (DL) based supervised object detection algorithms fail to reach the desired level of high accuracy in the case of strong overlapping (figure 3(b)) arrangement of cells (results illustrated in supplementary section S.6). This task is analogous to the problem of packing circles of varying (but within a limited range) sizes within an arbitrary polygonal shape. Simpler versions of this problem have been addressed in the fields of Computational Geometry and Operational Research [32], [48], [25], [35], [15], [11], [58], [33], [17]. This is an NP-hard problem [14] and no prior work exists to give a direct solution for such a task, specifically in cases of overlap of circles. Hence, one has to resort to either an iterative method based on optimization with certain constraints for an approximate solution, or use image features first to initially place few cells suitably at locations with high feasibility and then fill residual gaps in an iterative manner. We chose the latter path for our method, as the FGR map (in figure 3(d)) provides rich clues to initially detect a few cells with high level of accuracy.
Our proposed method consists of two sub-processes (figure 4): Phase-I sub-process consists of steps, which enhance the foreground areas (FGR) followed by feature extraction, using the Distance Transform (DT) from edge map of FGR and ridge lines from the modulated DT (mDT) map. Peaks of mDT map and features on ridge lines correspond to candidate centers of detected objects of interest. Phase-II sub-process fills the cell areas (detected centers with edges) detected in Phase-I with the background color on FGR to obtain a residual map (RFM). Significant areas of the RFM are then iteratively filled using Hough Transforms on convex arc segments that were identified on the outer boundary edges of the objects in the RFM. The output of the cell center locations provided by the two phases are eventually merged using a union operation (spatial locations of very nearby cell centers are averaged) to create the final set of detected cell centers. Both these phases of the pipeline (unsupervised) process are described below.
III. Phase-I: DT and Ridge based method for cell center detection
Salient regions of the green channel correspond to the nuclei of interneurons. Since these are our main objects of interest, we have used the green channel alone for further processing. Steps used in Phase-I of our algorithm (see figure 4) are:
Normalization of the green channel
Contrast enhancement to get FGR
Edge detection on binary image of the FGR map
Distance Transform (DT) on edge map
Compute mDT map using DT and FGR
Ridge line detection (computed only for large areas of FGR) on DT map
Identify peaks on Ridge lines using mDT map
Detect Bifurcation Points (BF) on the ridge lines
Clustering of the peaks and BF to yield the list of putative centers of cell nuclei.
As a first step, the green channel has been enhanced using contrast enhancement (detailed in supplementary section S.1) to obtain the FGR. Figure 5(a) shows the FGR with the Ground Truth (manually annotated cell centers). The green channel has alone been thresholded to produce the binary image of FGR (also shown in figure S.1(e)). We then applied a Canny edge detection method [10] on this binary image of FGR to obtain an edge map. A distance transformation (DT) [36] on the edge map is then used as an intermediate feature which will eventually help to locate candidate positions of cells in FGR. The DT values corresponding to the background region of FGR are then suppressed to zero, leaving only the DT-map on pixels of FGR. Figure 5(b) shows binary (thresholded) map of FGR in figure 5(a), with the edge map overlayed and represented by the Magenta borders. Figure 5(c) shows the DT map extracted from the edge map of figure 5(b).
The DT map [44] results in producing an intensity peak which corresponds to the centers of detected nuclei, and this intensity decreases the further we get away from the centers and thus the minimum intensity represent the edges. However, in cases of mild overlap, where several GFP nuclei overlap, DT map does not always exhibit sharp peaks, making it difficult to detect cells using only a local maxima. Hence we have used a modulated DT (mDT) map (figure 5(d)) obtained as: where, g, d and f represent mDT-map, DT value and FGR respectively, x, y represent integer pixel coordinates. The mDT map exhibits better localization of peaks as cell centers (as shown in figure 6(a)). Regional maxima in mDT map are thus detected and represent the putative centers of the GFP nuclei (figure 6(a)). The process described so far though works well in cases where the GFP nuclei are isolated and spatially distributed with mild overlap, however produces unsatisfactory results when few cells have strong (large) overlap in arbitrary spatial arrangements. As closely packed clusters of cells do not follow any well-defined structure or geometrical arrangement, expecting the center-line (as in MAT [13]) to be the loci of peaks would be an erroneous assumption. In such cases, the maxima in mDT map may be falsely detected to signify a single center when there are actually two or more cells present in the FGR (figure 5(a)) for that location. This process of detecting peaks in mDT, thus results in “misses” of cell centers for the complex cases of strong overlap of cells.
To overcome this problem, we have designed an algorithm that detects features within the ridge lines from mDT map (as shown in figure 6(b)) of only those putative cells whose area is greater than 1.5 times the average cell area (i.e., minimal and no overlap cells are not considered) and assigns the centers of the nuclei based on the location of these features on Ridge lines. This ridge detection process is similar to the “Vessels” Algorithm [23], that has been used earlier to detect blood vessels from magnetic resonance imaging (MRI) data. The features used in our analysis are the Bifurcation (BF) points and Ridge Endings (RE). RE appear near the border of the cells and only the Bifurcation Points (BF) correspond to potential cell centers. The BF were obtained as landmark features by traversing along the ridge lines (as in [49]). An example of those feature points detected on ridge lines is shown in figure 6(c). Traversing along the ridge lines, as shown in figure 6(c), produces an additional list of centers. These centers estimated on the ridge lines and those obtained from the peaks on mDT (as shown in figure 6(d)), are combined together (as a union of two sets, and merger of nearby peaks) to yield the final set of cell centers at end of Phase-I of processing. Figure 7(a) shows the results of intermediate level processing at the end of Phase-I (DT + Ridge-based method).
IV. Phase-II: Convex-Arc Based Iterative Method for missing cell detection
Results of Phase-I, though fairly accurate, did not yield satisfactory results in cases of strong (large) overlap of cells. An example of this failure is highlighted by figure 7(b) where true detections, misses and false alarms of Phase-I are shown alongside true centers as annotated by a master annotator.
To further enhance the efficiency of the algorithm, we have introduced an iterative process (algorithm 1) of cell filling using boundary arcs on a residual foreground layer (FGR in figure 5(a)). This process is based on the hypothesis of topological/geometric constraint: in case of strong overlap, cells at the periphery of the region of overlap produce the boundary (edge) layer of FGR. Those in the inner cordon fill up the gap to produce prominent green fluorescence within the FGR. To efficiently detect the overlapping cells, an iterative process is designed to detect overlapping cells in FGR, starting with boundary layer, which is described below as algorithm 1.
This unsupervised process iteratively identifies undetected cells over strong overlapped regions; specifically over significant RFM areas, which are missed at the Phase-I of processing. A flowchart describing algorithm 1 along with a synthetic example is discussed in detail in the Supplementary section S.2.
Figure 8 demonstrates the application of the iterative arc based method for missing cell detection in the given example from figure 5(a). Figure 8(a) shows the peaks detected at Phase-I. Figure 8(b) gives residual map (RFM) obtained after suppressing the detected cell regions in Figure 8(a) with the background color. Figure 8(c) shows Inner boundary contours (IBCs) in green and Outer boundary contours (OBCs) in blue on the significant RFM after filtering out the small connected components from that in Figure 8(b). IBCs are eliminated and convex arcs are fitted (using Hough Transform) with circles on convex OBCs. Figure 8(d) shows the centers of the new cells detected at Phase-II of processing. The final list of cell centers is obtained by a merging process of the two lists of cell centers detected from Phase I: DT + Ridge-based peak feature detection (e.g. Figure 8(a)) and Phase II: Arc based iterative filling (e.g. Figure 8(d)) methods. If two or more distinct cell centers detected at Phase-I and Phase-II stages fall within a small neighborhood (dc < 2 ∗ RC, RC = average radius of a cell), they are merged and the average of the locations of the cell centers replaces the set of nearby cells. Figure 9 gives an illustration of the results of this process of merging and detecting new cells in residual foreground layer. Supplementary section S.3 contains the sequence of intermediate results of different steps of processing in sub-processes Phase-I and Phase-II. This section also contains the table of all heuristic parameters used in our model for cell detection.
V. Results and Performance Analysis
We have evaluated our algorithm on gigapixel resolution brain section images of two mouse brains. The data [1] was manually annotated by a set of annotators (n = 12; 10 naive and 2 master annotators) to obtain the groundtruth. Each gigapixel resolution image was split into tiles of 500 × 500 pixels for easier quantification by the human annotators. Each annotator was also given the classification results of the DT and ridge based method (Phase-I), and then allowed to modify, insert or delete the existing cell centers. The output of the Phase-I was provided to reduce the burden of identifying all isolated cell locations by visual search. This did not bias their judgment in the complex regions of overlapped cells. Annotators easily modified (delete, insert, change location) the detected cells using a GUI (see snapshots in section S.7 of the Supplementary Document). Providing manual annotators the output of a first-pass algorithm is “industry standard” in related problems (e.g. in EM for image segmentation). The annotation labels were cross-verified by a master annotator, which gave us confidence of its accuracy levels.
Figure 10 shows typical examples of cell detection for selected parts of mouse brain scans obtained using our proposed method. The proposed algorithm takes the cell centers detected from two phases, Phase-I: DT + Ridge-based; and Phase-II: Arc-based methods, and combines them using an iterative combination strategy. True positives, false alarms and false negatives are marked with blue, red and white color markers respectively in the outputs. Phase-I sub-process is sufficient to detect all cells accurately in sparse and minimal overlap regions. The Phase II sub-process requires 2 – 3 iterations to converge, typically in cases of strong overlapped cells. The details of training the supervised architectures are given in the next subsection, and are elaborately discussed in the Supplementary section S.4.
A. Evaluation of the proposed algorithm
The manually annotated samples were used as groundtruth to evaluate the performance of the algorithm. In addition, the performance of our proposed method was compared with other methods that can potentially be applied to solve the same problem. These are two supervised shallow learning methods: (a) SVM [16], [41] (using Gaussian Kernel); and (b) HAAR + Adaboost [57]; (c) cell detection based on the method used by Al-kofahi et al. [3]; two DL techniques: (d) Faster R-CNN [50] and (e) SegNet [6] and (f) Phase-I of our proposed approach (for comparing with Phase-II). For the supervised approaches, we have used half of the annotated data for training and the remainder of the data for testing. A Precision-Recall metric was used as the performance criteria [42]. The cell-centers that were detected within a certain distance (DR < 5 pixels) of true location in groundtruth were considered as to belonging to the same cell. Cell-centers of positively classified windows are considered as detected cell centers. The details of training the supervised architectures are elaborately discussed in the Supplementary document (section S.4 for Faster R-CNN and SegNet).
We have used a tensor flow based implementation of Faster R-CNN [50] for our experiments. Faster R-CNN [50] performs a bounding box regression on the objects in an image for detection. We have used equal number of samples for training and testing. Since we have only one object of interest i.e. GFP labeled neuron in our data, we provide a bounding box of size 25 × 25 pixels (derived empirically) around each cell. Faster R-CNN [50] uses the co-ordinates of the positive samples as the anchor boxes. Centroids of the bounding boxes obtained as output are considered as detected cell centers. We have utilized a SegNet [6] model that has been trained as per the code provided by the authors. Training samples (only positive ones) in SegNet were generated by delineating circular patches around manually annotated cell centers. The inputs for training were circular patches of 4 pixels radii around the manually annotated cell-centers. The trained SegNet map produces outputs as circular patches (sometimes overlapping with each other) on the test image, whose centers can be considered as putative cell-centers. Details of training are given in Supplementary section S.4. Our method has also been compared with that of Al-kofahi et al. [3], which, like our proposed model, is also an unsupervised method used for cell nuclei detection and segmentation. It initially extracts the foreground region using graph-cuts-based binarization, following which a multiscale Laplacian-of-Gaussian filtering combined with distance-mapbased adaptive scale selection is used to detect the seed points for segmentation. It then detects nuclei through graph-cuts-based segmentation along with alpha expansions and graph coloring.
Comparative analysis of our algorithm with the other models are tabulated in table I. Our iterative approach exhibits high precision and recall values of 0.972 and 0.961 (average over complete dataset of one mouse brain and a selected portion of another) when evaluated using the manually annotated ground truth, indicating the close to human efficiency of the algorithm. The supervised methods of cell detection based on SVM and Faster R-CNN based deep learning did not perform better than our proposed unsupervised approach. Details and justification of using a Non-Maximum Suppression (NMS) to obtain the results of all supervised methods are given in Supplementary section S.5.
Our proposed unsupervised iterative method of cell detection, provide near-ideal performance in most of the cases, with exceptions that can be attributed to the presence of noise in the dataset. In order to demonstrate the effectiveness of the algorithm over wide distribution of cell nuclei throughout the brain, we have separated the results over strong overlap and minimal overlap and sparse regions within cells of the brain, and results for both sub-processes are given in table II. As expected, the algorithm performs the best in the sparse regions with marginal degradation in performance in the minimal overlap and strong overlapping areas. Table III shows the comparative performance, on the separated data sets of overlapped, sparse and minimal overlap regions of the brain, of our proposed method compared to that of SVM [16] and Faster R-CNN [50]. Additional results (with examples) of cell detection of the competing supervised methods used for performance analysis are given in Supplementary section S.6.
The proposed iterative approach (Phase II) converges in typically 2 – 3 iterations for the complex overlap cases. For the example shown in figure 9, the algorithm took 1327 ms for complete evaluation compared with 795 ms for the Phase-I stage on an Intel core-i7 4820K processor, with 64 GB RAM. The average time taken by the proposed method for a single brain section of 18K × 24K pixels on an Intel core-i7 4820K processor based CPU with 64 GB RAM is 4 – 5 mins. Despite having doubled the number of training samples, the competing supervised methods did not exhibit better performance, indicating that evaluation of the model is sound. A 5-fold study has been performed for shallow ML models (5 different combinations of randomly selected sample sets to form the training and test partitions). For deep-CNN we have done a 2-fold study; and our unsupervised method does not require training samples; all samples are used for testing, a few are used to empirically obtain the optimal parameter values.
VI. Discussion
Our approach allows for robust and efficient estimation of the centers of GFP tagged cell nuclei with relatively high precision and recall. Our unsupervised and iterative algorithm based on Ridges and distance transform [44] combined with arc-based region filling provides the best precision value amongst the different approaches used to solve the detection problem. It is important to note that all the methods (including shallow and deep learning methods) also perform well in cases where there is marginal or no overlap (sparse distribution) between the GFP tagged interneurons. Our algorithm approaches near ideal performances in these cases, with the precision-recall values ≃ 0.97 and ≃ 0.96 respectively. The cases where the other models fail are the complex case of strong overlap of cells, where our method gave the best result with a overall precision of 0.972 and recall of 0.961. The drawback with our algorithm is the relatively longer evaluation time frame which is directly proportional to the complexity of the image to be evaluated. Our proposed algorithm is completely automated and unsupervised in its application. We have applied this algorithm to analyze brains and a web implementation of the iterative algorithm has been implemented at the mouse brain architecture portal at the Cold Spring Harbor Lab (CSHL) [1] for analyzing cell type data.
It was surprising to note that standard supervised classifiers, e.g. SVM, ADABOOST, etc. and state-of-the-art DL based detection techniques (Faster R-CNN and SegNET) did not perform satisfactorily when compared to our method. We believe that this may be because of our method of treating strong overlap decomposition explicitly which improves our performance to ~ 97%, the precision levels needed for real world application. The performance of the ML models using our method is comparable with a previous study on brains [28] where similar methodology of GFP tagging in mice were used and the detection was performed with CNN in order to achieve 90% performance. The difficulties for the network may have been due to the degree of overlap of objects in the image, which, in turn, depends on cell densities as well as microscopic methodology.
VII. Conclusion
We have developed an iterative approach (unsupervised) for efficiently detecting GFP tagged cell centers of GABAergic neurons in the mouse brain. Our results demonstrate that our algorithm, which is based on classical machine vision methods substantially outperform published CNN-based methods. The detection problem that we have solved is of practical interest to neuroscientists. The classical Machine Vision based method we have designed, directly solves the overlap decomposition problem with high precision and recall values. Recent methods of supervised object detection do not provide the level of performance required particularly in cases of strong overlap of cells, as the foreground blobs appear with a wide variety of non-unique two-dimensional patterns that are of different shapes, sizes and structures (due to varied spatial arrangements of cells in strong overlap), making the training data unreliable for efficient learning. Our unsupervised approach provides a much superior performance than recent supervised object detection algorithms such as ADABOOST [57], SVM [16], [41], SegNet [6] and Faster R-CNN [50]. Previously published results [28] on similar problems only reach 90% performance level (in contrast with > 97% of ours). The failure of both the deep and shallow nets to reach > 90% accuracy levels, is mainly attributable to the difficulties of accurately detecting overlapped nuclei. While our method may not generalize to other unrelated problems, it addresses an important problem in the growing field of computational neuroanatomy of brains. CNNs and SVMS are very powerful ML architectures but need further refinements to solve every real-world problem. In our use case the structure of the problem demands a different architecture.
With an ever increasing dataset and information about the neurons and their connectivity, it becomes important to develop automated unsupervised algorithms for efficient data analysis of relevant information [5]. We have provided one approach towards this goal which helps neuroscientists to achieve few targets with greater confidence.
VIII. Acknowledgments
The work was supported by the HN Mahabala chair professorship at Indian Institute of Technology, Madras, The CrickClay Professorship at the Cold Spring Harbor Laboratory to Dr. P. Mitra. The authors also acknowledge the support from the Mathers foundation, NIDA grant (5R01DA036400-03) and NSF EAGER grant (1450957).
References
- [1].↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].↵
- [7].↵
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].↵
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].↵
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].↵
- [37].↵
- [38].
- [39].↵
- [40].↵
- [41].↵
- [42].↵
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].↵
- [57].↵
- [58].↵