Abstract
Background Quantitative imaging of epithelial tissues prompts for bioimage analysis tools that are widely applicable and accurate. In the case of imaging 3D tissues, a common post-processing step consists in projecting the acquired 3D volume on a 2D plane mapping the tissue surface. Indeed, while segmenting the tissue cells is amenable on 2D projections, it is still very difficult and cumbersome in 3D. However, for many specimen and models used in Developmental and Cell Biology, the complex content of the image volume surrounding the epithelium in a tissue often reduces the visibility of the biological object in the projection, compromising its subsequent analysis. In addition, the projection will distort the geometry of the tissue and can lead to strong artifacts in the morphology measurement.
Results Here we introduce DProj a user-friendly tool-box built to robustly project epithelia on their 2D surface from 3D volumes, and to produce accurate morphology measurement corrected for the projection distortion, even for very curved tissues. DProj is built upon two components. LocalZProjector is a user-friendly and configurable Fiji plugin that generates 2D projections and height-maps from potentially large 3D stacks (larger than 40 GB per time-point) by only incorporating the signal of interest, despite a possibly complex image content. DeProj is a MATLAB tool that generates correct morphology measurements by combining the height-map output (such as the one offered by LocalZProjector) and the results of the cell segmentation on the 2D projection. In this paper we demonstrate DProj effectiveness over a wide range of different biological samples. We then compare its performance and accuracy against similar existing tools.
Conclusions We find that LocalZProjector performs well even in situations where the volume to project contains spurious structures. We show that it can process large images without a pre-processing step. We study the impact of geometrical distortions on morphological measurements induced by the projection. We measured very large distortions which are then corrected by DeProj, providing accurate outputs.
Background
Epithelia are key multicellular structures constituted of one or several layers of cells. Their organisation ensures the proper animal development and the good functioning of adult organs. They can be constituted of hundreds to millions of cells that are carefully regulated in number, organisation and genetic identity. The cell mechanical and biochemical properties are tightly regulated and their dynamics affect the organ development and functioning. Conversely, large scale forces at the tissue level can determine individual cell fate. For instance, during drosophila pupal development, local cell-to-cell interactions through adhesion forces ensure tissue integrity, control cell shape and cell division orientation and affect cell death probability (1, 2). In the avian embryo during gastrulation, local mechanical properties of cells are involved in the behaviour of the whole tissue and the dynamic formation of the first folding event (3). In vertebrate adult brains, neural stem cells (NSCs) are organized in a pool forming an epithelium. The coordination of their behavior, such as division or differentiation, are in part regulated via local cell-cell interactions (4) but is also regulated via large-scale coordination (5). Thus, the elucidation of biological questions related to processes involving complex epithelia now require the imaging of large tissues, while reaching single cell resolution and single cell shape quantitation.
Modern microscopy technologies can fulfill this imaging challenge and microscopes that can acquire fluorescence images of the embryo and generate volumetric, multi-channel, time-lapse datasets of a live sample are now used routinely. They offer single-cell resolution on a field-of-view large enough to encompass a significant part of the tissue studied. However, because of the large size of the data generated and the limited image signal-on-noise ratio (SNR) imposed by preservation of the tissue health in a live-imaging experiment, new challenges downstream of imaging appears which require the development of better image analysis tools.
Epithelia are a continuous layer of cells on a typically non-flat, smooth surface. When cells are labeled with a junction marker (e.g. cadherin, ZO1), the tissue resembles a manifold wrapped on a 3D surface. A simple approach to visualization and analysis is to perform a projection of the tissue 3D surface on a 2D plane. This dimensionality-reduction approach proves to be particularly convenient: First the resulting data size is considerably diminished. Second the visualization of the tissue layer content is immediate in 2D. Finally, most of the segmentation algorithms that can extract cell shapes have a better robustness and accuracy in 2D than in 3D so far, and manual corrections can still be reasonably performed in 2D.
The biological significance of the information extracted from these manifolds prompted for the development of several tools that can perform 2D projection. MorphoGraph-X (6) and ImSAnE (7) belong to a first class of tools, where a reference surface is built as a mesh mapping the sample boundary. The fluorescence intensity is then collected at or a few microns away from the boundary into the sample, in a direction perpendicular to the surface. This approach is particularly adequate for images of samples with complex, possibly closed boundaries. A second class of tools perform a projection of the 3D volume along the Z-axis of the 3D image. The resulting projection created by these tools is a 2D plane that has the same width and height as the source 3D volume. They cannot harness samples where the tissue layer folds, since there must be at most one surface Z value per (X, Y) position. They are however particularly convenient with classical confocal microscopy, for the axis of projection is the one where the point-spread-function (PSF) is the most elongated. The most simple projection technique in this class consists in taking the largest pixel value along a Z column for each (X, Y) position. As noted in (8), this maximum-intensity projection (MIP) technique is the most used by biologists. An important drawback is that this projection incorporates noise from throughout the sample, in particular inside cells, and will compromise segmentation based on the membrane signal. To address this limitation, several projection tools have been developed that aim at including in the projection only the signal coming from the tissue layer. Among them there is StackFocuser (9), PreMosa (10), Extended Depth of Field (EDF) (11), SurfCut (12), MinCostZSurface (13–15), the Smooth Manifold Extraction (SME) tool (8) and a new implementation of the latter: FastSME (16). In (17), authors also proposed an approach based on Deep-Learning for the projection along the Z-axis, but it requires a set of images along with their already computed projections for its training (see Supplementary Information for descriptions). While these approaches have been proven to work well for tissue imaging, they left open some challenges. First, bright or noisy structures outside of the cell layer might compromise the extraction of a meaningful reference surface, in turn strongly affecting the quality of the projection. Second, most of those approaches will not be working when the volume to project has a size larger than the available computer memory, and might take long when they can. Third, the tissue surface and the XY plane may have in some regions a large angle. Beyond 30°, the morphological features of cells measured on the 2D projection will be significantly altered, as noted in (12).
DProj is a toolbox that fits into this second class of tools, and builds upon them to address several challenges they left open. LocalZProjector is the first component of DProj, and is a user-friendly and widely configurable projection tool that can be tuned to detect the right reference surface. Its manoeuvrability makes it amenable to a wide range of samples and image qualities, as well as to images of large size beyond the size of the available computer memory. DeProj, its second component, can correct for the distortions induced by the projection of tissue even if they are very curved. We illustrate below these capacities using three different samples: the drosophila pupal notum, the quail embryo and the adult zebrafish telencephalon. We also compare DProj features, performance and accuracy to existing tools.
Implementation
DProj is a software toolbox for accurate morphology measurements on epithelial tissues. It is made of two components: LocalZProjector and DeProj (Figure 1).
LocalZProjector performs the projection of a curved surface in a 3D image on a 2D plane, including only the signal of interest (Figure 2a). It is an ImageJ2 (18) plugin distributed within Fiji (19) that focuses on usability and is designed to be adaptable to many different cases and image quality (Figure 2b). It can work with 3D time-lapses with multiple color channels, takes advantage of computers with multiple cores, can be used in scripts and can process images too large to fit in memory. The local Z projection processes as follow: First it extracts a reference surface that maps the epithelial layer (Figure 2c, Supplemental Figure 1). The reference surface is represented by the height-map, which consists in a single 2D image per time-point of the source image, that specifies for every (X, Y) position the Z position of the layer of interest. Second, the height-map is used to extract projections of the different channels from the 3D image, according to their relative, sometimes different, offset with this reference surface.
LocalZProjector relies on a few parameters set by the user. The height-map is determined by applying a 2D filter on each plane of the 3D source image, chosen and configured to yield a strong response for the layer of interest (either mean or standard deviation filter). To speed-up computation and reduce the effect of pixel noise, each 2D plane can be first binned and filtered with a Gaussian kernel. The height-map is then regularized using a median filter with a large window and rescaled to the original width and height. It is then used to extract a projection from the 3D image. A fixed offset can be specified separately for each channel, and is used to collect intensity in planes above or below the reference surface. Several planes, specified by a last parameter Δz, can be accumulated to generate a better projection, averaging the pixel values or taking the maximum value of these planes (Figure 2c, Supplemental Movie 1).
Once the 3D volume has been projected on a single 2D plane, many tools are available that can segment the individual cells in the tissue. They are often relying on apical signals to trace the membranes but can also work from a central seed, such as a nuclear staining, to calculate Voronoï diagrams. Several of them offer an intuitive user interface, allowing for immediate usage and user interaction. For instance, EpiTools (20) is a toolbox with MATLAB and Icy (21) components built to study the dynamics of drosophila imaginal discs. Its segmentation algorithm relies on region growing from seeds determined automatically and merged based on region areas. SEGGA (22) is a standalone application written with MATLAB proposed for the investigation of drosophila embryo germband epithelium. Recognizing that a small number of mistakes in segmentation can have major negative impact on cell tracking accuracy for long time-lapses, the authors propose several approaches to increase the robustness of segmentation and have a near perfect tracking results. Tissue-Analyzer (23) is a tissue segmentation tool, distributed along TissueMiner (24) and the combination of these two softwares offers a framework that let end-users implement their own analyses using the R software and custom commands.
But these tools operate on 2D images only, which implies that the epithelium is a flat plane and parallel to X Y. When this is not the case, any morphological measurements made on the segmentation results will be corrupted by geometrical distortions induced by the projection (Figure 3a). Indeed, almost all morphology metrics, such as area, eccentricity and orientation will be erroneous when they are measured on the 2D projection. The second component of the DProj toolbox aims at correcting these artifacts. DeProj is a MATLAB tool that combines segmentation results and height-maps to correct morphology measurements made on the 2D projection. DeProj returns corrected metrics, as if they were measured on the reference surface in the original 3D image (Figure 3b).
Results
LocalZProjector is an accurate, fast and convenient tool to generate projections of 3D manifolds on 2D planes
One of our recent study involves long-term 3D timelapses imaging of a drosophila pupal notum (2, 25). To follow the cells dynamics in the tissue, we relied on generating an accurate 2D projection of the cell monolayer (E-Cadherin signal, Ecad-GFP), taken from the epithelium surface, and at the same time collecting the RFP signal from cell nuclei a few μm below this surface (nls-RFP). This tissue amounts to a few difficulties. First, the epithelium is not flat. Imaging it requires the acquisition of about 40-60 μm thick optically-sectioned volumes. Second, a layer of extracellular matrix called the cuticle can be found apically to the epithelium. This cuticle is auto-fluorescent in the green, red and far-red spectra. Third, below the epithelial layer are large cells called fat bodies which are also highly fluorescent. In the GFP channel, the resulting 3D images display mainly two layers. The top one (low Z values) corresponds to the auto-fluorescent signal collected from the cuticle. The middle one corresponds to the cell layer, where the cell membranes build a manifold with a large curvature (Figure 2b). Below these two layers, the fat bodies generate punctate, bright structures. If not properly excluded from the projection, these unwanted structures in the imaged volume will degrade its quality and complicate or even forbid the subsequent segmentation task.
As the problem introduced here with the pupal notum is frequent in epithelium tissue imaging, we used it to validate the performance of the LocalZProjector plugin by comparing it to currently available projection tools (Table 1). We generated a ground-truth for the desired projection image by manually selecting the Z value of the cell layer at each (X, Y) location (see Supplemental Note 1). We then calculated several metrics measuring the accuracy and performance of the projection generated by the LocalZ-Projector tool, and compared the results to 7 other methods (Supplemental Note 2): the standard maximal-intensity-projection (MIP), the StackFocuser tool (9), SurfCut (12), PreMosa (10), the Extended-Depth-of-Field (EDF) tool (11), the Mininum-Cost-Z-Surface (MinCostZ) approach of (13, 14), implemented in (15) and the Smooth Manifold Extraction tool (8, 16). We first assessed how close the resulting projection was to the ground-truth projection using the root-mean square error (RMSE). This metrics proves useful in two aspects. First, because it allows for quantitatively comparing several methods and assessing how useful the projections will be in a subsequent analysis step. Second, because it offers a way to systematically search for optimal parameters for the 8 methods tested, by minimizing the RMSE over a wide range of parameters (Supplemental Note 2A). Comparing the optimal projection of each method, we find that the LocalZProjector projection is favored (Table 1, Supplemental Figure 3a and Supplemental Note 2B), because it is robust against noise and can be configured to deal with regions of high-curvature. The accuracy of the height-map output is important for the subsequent correction of the cell morphology measurements made by DeProj in the second step of the DProj process. We therefore calculated the RMSE of the height-maps compared with the ground-truth (Supplemental Note 2C), for the projection tools that can return a height-map of the cell layer (LocalZProjector, StackFocuser, PreMosa, EDF, MinCostZ and FastSME). We find again that LocalZProjector offers the height-map with the lowest RMSE (Supplemental Figure 3b). Because DProj aims at being a tool possibly used on very long time-lapse movies, the time needed to generate a projection is important, we confirmed it is fast in comparison to most of the other methods (Supplemental Figure 3c). Finally, the projection accuracy of such a dataset is relevant mainly for its use in a subsequent analysis. We chose to focus on cell segmentation, as DeProj will be used to measure accurate and unbiased cell morphology. We therefore derived a simple, fully automated segmentation workflow on the projections, and compared segmentation results against a ground-truth segmentation (Supplemental Figure 3d, Supplemental Note 3). These results exemplify the usefulness of LocalZProjector, both for accuracy and performance.
Finally, in this study, we wanted to generate projections of the E-cadherin channel but also needed to visualise the nuclei reporter of some of these cells localised a few micrometres just below the reference surface. This was easily feasible as LocalZProjector can handle multiple-channel long time-lapse images in a user-friendly manner. We compared some of the LocalZProjector features with other tools in Table 1. We also proved qualitatively that LocalZProjector can be used on wide-range of images coming from very different samples and we especially validated it on a large set of example images introduced in (16), taken from samples ranging from Neuroscience to Cell Biology and synthetic images. In Supplemental Figure 12 we present the projections obtained successfully with LocalZProjector on this dataset.
Projecting large images with LocalZProjector
In toto imaging of developing embryos allows for investigating the dynamics of tissues at a large spatial scale. For instance analyzing gastrulation in entire avian embryos, we showed that it is driven by the graded contraction of a large-scale supracellular actomyosin ring at the margin between the embryonic and extraembryonic territories (3). For this study we relied on particle image velocimetry (PIV) to measure the tissue displacement field. This technique does not require the segmentation and tracking of individual cells. However, several key mechanisms at large scales emerge from the dynamics of single cells (26). The ability to segment all the cells in a whole embryo prompts for imaging at high resolution and special microscopes (27). But such acquisition setups generate in turn very large images. Also some imaging modalities that enable imaging large specimen at high resolution, such as Light-Sheet Fluorescence Microscopy (LSFM), may bring additional distortions in the image. In order to image a quail embryo at high resolution, we relied on LSFM using an inverted selective plane illumination microscope (27, 28). While the light-sheet is held stationary at 45° of the embryo surface, the embryo is translated horizontally through the light-sheet. The 2D planes acquired for each translation are then concatenated in a 3D stack. Because the axis of the embryo translation and the light-sheet plane are at a 45° angle, the stack needs to be post-processed to remove the skew induced by this angle. The resulting is a 8669 ×2285 ×1067 image, amounting to a 42 GB file for a single time-point.
The 2D projection of such an image is very pertinent as the epithelium of interest is a smooth thin cell layer within a large 3D volume. It would also reduce the image size by 1067 and make it much more amenable to analysis and compact storage. To tackle the challenges coming with such large datasets, we use the Fiji Virtual Stack importer, which only loads one Z-slice in memory at a time. This allows for the opening and to some extent the processing of images much larger than RAM. While the MIP works well with virtual stacks, the resulting projection is corrupted by projection artifacts. Moreover, in such images the point-spread function (PSF) is not aligned with Z-axis of the image. The elongation of the PSF generates marked distortions and blurs the membrane signal in the projection, up to the point where cells cannot be outlined by eye (Figure 4a), even on a source image with little signal coming from spurious structures.
To improve the projection quality, we developed the LocalZProjector so that it can work with virtual stacks as well, granting it the ability to project images larger than the available memory amount and without having to do any preprocessing or resaving of the image. LocalZProjector works with two passes through the image data, one to compute the reference surface, one to perform the local projection (Supplemental Movie 2), ensuring that individual planes are read from disk at most twice. This still comes at a time penalty. LocaLZprojector takes 8.5 minutes per time-point, against 1.3 minutes for the MIP. But the LocalZProjector result is completely devoid of the defects observed with the MIP and is amenable to segmentation and quantification (Figure 4b).
Accurate measurements of cell morphology: DeProj
In vertebrates, adult neural stem cells (NSCs) are responsible for adult neurogenesis (29) and, in some vertebrates, regeneration post-injury (30). NSCs are organized as an epithelial-like structure lining ventricles that has to be maintained functional for very long periods of time (often over years). To understand NSC populattion homeostasis it is essential to integrate large-scale and long-term imaging of the NSC pool and the zebrafish telencephalon has recently emerged as a unique model for these studies (5, 31). To study cellular and mechanical functions of NSC over the entire over the entire dorsal telencephalon (pallium) we can image whole-mount immunostainings against ZO1 (4), highlighting apical domains of the NSCs. However, since the pallial hemispheres are highly curved we so far could not extract the geometrical parameters of many NSCs (falling in periphery, in sulci, etc.).
DeProj is a MATLAB app specifically built to address this issue. DeProj exploits the reference surface for the projected tissue to assign a Z position to each point of a cell contour, effectively “deprojecting” it on the tissue surface. The reference surface can be specified as the height-map which is the secondary output of LocaLZProjector and several others projection tools, or as a mesh that extends over the epithelium surface. The cells segmentation can be specified as a black and white mask, or as a the specialized structure coming from the tool of (32). Several morphological metrics (area, perimeter, orientation, eccentricity, …) are then computed and saved, along with the cell contour mapped on the tissue surface. The generated DeProj data object is used to store the analysis results and offer exporting facilities and several visualisations of the results. On the telecephalon image, a 3D view generated by DeProj shows the shape of the epithelium. We can see several regions where the tissue is very curved, particularly at its borders and in the sulcus separating the regions (Figure 5a). A visual representation of the cell area does not show a salient difference between a measurement made on the proper 3D epithelium surface (Figure 5b) or on the 2D projection (Figure 5c). However the histogram of the error metric ea = (1 − a2D/a3D) between these two quantities shows that for a large number of cells, using the 2D measurement induces an error greater than 20% (Figure 5d, 22% of the 3000 cells in this epithelium have an error larger than 20%). The cells with a large error are found at the epithelium border and in the sulcus (Figure 5e), which are regions where the angle between the cell apical planes and the XY plane is large too (Figure 5f). Without surprise, we find that a large slope correlates with a large error (Figure 5g). If a cell would be a square of side a, with one side making an angle θ with the XY plane, then its real area measured in 3D is a2. The 2D projection of this cell contour on the XY plane generates a rectangle of sides a and a cos θ, so that the error ea is equal to 1 cos θ for this cell. Because real cells have complex shapes and have a contour that is not necessarily contained in a plane, we find that this expression constitutes a lower bound for ea (Figure 5g, red line).
Conclusions
As can be noted in the comparative study of this work, there exists already several tools that perform projection of tissues in 2D from a 3D image. Their number demonstrates the importance of the information that can be extracted from the resulting images. This, and the still popular usage of MIP despite its shortcomings, also points out the difficulty of having a tool that can address all types and qualities of images to project, despite the similarity in tissues staining and shape. Yet the quality of projection dictates the subsequent step in analysis, as demonstrated in Supplemental Figure 3d. Some of these tools have the advantage of being parameter-free (8, 13–16). LocalZProjector takes another approach and requires several parameters to be tuned. In turn, this configuration step allows it to work even with difficult 3D images containing spurious structures and confers it a greater adaptability. It is also to our knowledge the only one that can process large images without pre-processing.
DeProj allows for the correction of geometrical distortions caused by the projection on morphological measurements. These artifacts remain often overlooked, despite possibly compromising the measurements accuracy when the local angle of the tissue with the XY plane is large. It works as the final step in our bioimage analysis pipeline and combine the cell segmentation results with the original shape of the sample, such as a height map, to yield various corrected tissue visualizations and accurate morphological measurements on cells. Typically microscopists prepare samples in such a way that the orientation of the tissue is favorable for imaging, with its main orientation parallel to the XY plane. Yet, we found that in a tissue like the pallium, the slope can exceed 40° in the regions of interest. But even a more moderate slope yields dramatic errors on measurements taken directly on the 2D projection (Figure 5g). DeProj offers robustness against these distortions, and makes it possible to accurately access the morphology of cells in highly curved samples while taking advantage of a simplified 2D dimensionality to segment the tissue morphology.
Availability and requirements
The project homepages contain the source code, installation instructions, documentations and extra implementation details. For DeProj, the homepage also contains 3 example scripts and data to reproduce some panels in Figures 1, 2 and 5.
Local Z Projector
Project name: LocalZProjector.
Project homepage: https://gitlab.pasteur.fr/iah-public/localzprojector
Operating systems: Platform independent.
Programming language: Java.
Other requirements: Runs from Fiji (19).
License: BSD 3
Any restrictions to use by non-academics: None.
DeProj
Project name: DeProj.
Project homepage: https://gitlab.pasteur.fr/iah-public/DeProj
Operating systems: Platform independent.
Programming language: MATLAB.
Other requirements: at least MATLAB R2019b.
License: BSD 3
Any restrictions to use by non-academics: None.
Data availability
The data of this study is available upon reasonable request.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
SH, LV and JYT wrote the code. SH, LV, LM, ND, PC, EE and JYT performed the experiments. SH, LV and JYT wrote the article with contributions from all authors.
Funding
This work was supported by the Laboratoire d’Excellence REVIVE (Investissement d’Avenir; ANR-10-LABX-73), France BioImaging (Investissement d’Avenir; ANR-10–INSB–04) and the Région Île-de-France in the framework of DIM ELICIT. L. V. is supported by a Post-doctoral grant “Aide au 224 Retour en France” from the Fondation pour la Recherche Médicale (ARF20170938651) and a Marie Sklodowska-Curie postdoctoral fellowship (MechDeath, 789573). Work in the R. L. laboratory is supported by the Institut Pasteur (G5 starting package), a ERC starting grant (CoSpaDD, #758457), the “Cercle FSER” and the CNRS (UMR 3738). Work in the J. G. laboratory was supported by the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013)/ERC Grant Agreement no. 337635. Work in the L. B-C. laboratory was supported by the Ligue Nationale Contre le Cancer and the European Research Council (AdG 322936). L. M. was recipient of a PhD student fellowship from LabEX REVIVE and the Fondation pour la Recherche Médicale. P. C. is sponsored by the Institut Pasteur and the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska Curie (665807).