Simultaneous multicolor fluorescence imaging using duplication-based PSF engineering

We present a novel way to enhance the information content of fluorescence imaging by encoding information in the microscope point-spread function (PSF). In contrast to methods that modify the shape of the PSF itself, our approach encodes the additional information via the creation of faithful copies of the original PSF. Our method is compatible with operation over a broad wavelength range, other PSF engineering modalities, and existing analysis tools. We demonstrate this approach by developing the ‘Circulator’, an optical add-on made of off-the-shelf components that encodes the emission band of the fluorophore into the PSF. The device creates four copies of the original PSF at well-defined relative orientations and positions, where the presence or intensity of each spot indicates the emission color of the fluorophore. Using this device, simultaneous multicolor super-resolution and single-molecule microscopy can be performed using the full field of view of a single camera, resulting in a pronounced increase in experimental throughput. We demonstrate our approach in simultaneous three-color super-resolution imaging with single-molecule localization microscopy (SMLM) and super-resolution optical fluctuation imaging (SOFI).

Fluorescence microscopy is a key technique in the life and materials sciences. A crucial challenge resides in maximizing the information content that can be obtained from the imaging, by taking advantage both of the multiparameter nature of the fluorescence as well as high sample throughput. Driven by this need, several approaches have been developed that increase the information content of fluorescence images. One of these approaches is point-spread function (PSF) engineering, which modifies the pattern with which the fluorophores appear on the detector, such that their shapes provide additional information. In sufficient optically-sparse samples, PSF engineering is a highly attractive approach because it does not require compromises in the field of view or in the temporal resolution of imaging. Commonly-used examples increase the axial resolution of imaging via the introduction of astigmatism or more elaborate modifications [1,2,3,4,5], though the methodology has also been applied to perform simultaneous dual-color imaging using complex phase shaping elements [6], or to determine the emission spectrum of individual dyes at the cost of a reduction in field of view [7]. Recent work has also shown that the color bands of individual emitters can be determined directly by leveraging the optical imperfections of the instrument [8], though with limited accuracy and the requirement for advanced analysis algorithms that must be trained and validated for every microscope and dye combination.
While PSF engineering is a highly attractive method, several challenges complicate its general use.
Implementations in which the PSF can adopt a continuum of shapes require extensive data analysis that must efficiently and accurately discern among many similar shapes (figure 1a). Furthermore, approaches based on phase manipulation require either adaptive optics such as spatial light modulators, which come with their own limitations, or custom optical elements that must be carefully designed and manufactured for specific wavelength ranges and instruments. This requires also complicates the combination of multiple PSF-engineering techniques to encode multiple types of information at once (e.g. color and depth). These issues limit the information content that can be retrieved and fundamentally arise from the need to modify the PSF shape itself. Avoiding such a direct approach would also eliminate these restrictions, though the absence of altering the PSF intuitively appears to be incompatible with the entire concept of PSF engineering. However, we reasoned that these limitations can be avoided if one instead 'simply' creates copies of the original PSF, in such a way that the number, relative orientation and intensities of these copies provides the desired information (Figure 1a). Such an approach poses several advantages: spectrally broad operation is supported because duplicating PSFs preserves the shape of the wavefronts and also does not reduce the sharpness of the original PSF; it does not interfere with other types of PSF engineering; it does not require new instrument-specific analysis tools; and it preserves the field of view and imaging throughput.
We decided to explore this concept via the construction of a novel optical device, called the Circulator (Figure 1b and c). The Circulator uses PSF duplication to encode the emission color by converting the native PSF, usually but not necessarily consisting of a single focused emission spot, into a pattern that consists of up to four different copies, each located at known distances and orientations relative to the position of the fluorophore. The emission color of each fluorophore then determines the different intensities of the four PSF copies, such that visual or algorithmic inspection of these spots readily allows this color to be determined while retaining the object geometry across the full field of view.
Our choice to encode the emission band of the fluorophores reflects the profound importance of multicolor imaging in fluorescence microscopy. In conventional imaging, this is commonly achieved by exchanging optical filters to sequentially acquire different wavelength ranges. However, this severely impedes sample throughput if the individual acquisitions take considerable time, as is the case for single-molecule methodologies such as single-molecule localization microscopy (SMLM). Alternatively, dichroic image splitters can be used to generate images in several wavelength ranges on distinct sensor areas of the camera. This results in a significant reduction of the field of view when committing to a single sensor or can get tedious and costly when choosing a multi-sensor approach (several cameras) in terms of mechanical instability, alignment requirements, registration and synchronization overhead, especially when imaging at higher spatial resolutions [9]. Yet another strategy is to use temporal modulation of the light sources [10], which leads to a measurement slowdown since the fluorescence modulation of each emitter must now be captured over multiple acquired images, or the introduction of a diffractive or dispersive optical element [7], which requires bright emitters and reduces the field of view. Our approach, in contrast, does not suffer from these limitations.
The Circulator is a self-contained add-on module with the overall optical layout shown in figure 1b and c. The emission light coming from the microscope is collimated prior to propagating towards a polarizing beam splitter (PBS). An achromatic quarter-wave retardation plate (QWP) at 45°orientation in front of the PBS leads to an isotropic polarization-insensitive detection and an equal split of the incident light between the two optical arms. To compensate for non-idealities, such as unknown S-P redardation for example, an additional half-wave retardation plate may be required. In each of these optical arms, the emission first passes through a QWP at 45°orientation and is then reflected by either a dichroic mirror or a subsequent broadband mirror depending on its wavelength. The reflected light then passes the QWP and PBS again. The double-pass through the QWP rotates the polarization of the light by 90°, such that it is transmitted by the PBS towards the focusing lens and is imaged onto the camera sensor.
By virtue of the Circulator, the imaging PSF now consists of up to four different faithful copies of the input PSF, two defined by the dichroic mirrors and two more by the broadband mirrors, where the relative placements of these spots is defined by the alignment of the four mirrors. The relative intensities of each of these spots depends on the wavelength of the emission and the reflectivity of the dichroic mirrors. The inset in Figure 1b shows measured PSFs for differently-colored dyes, in which one dichroic mirror reflects green and orange light, and the other reflects only green light.
This optical design has a number of advantages. The planar optics minimize aberrations and distortions. The small mutual spot displacements allow operation across the entire design spectrum of the PBS and the lenses. Regardless of its wavelength, the light is reflected twice in each arm: once at the PBS splitter interface and once either by the dichroic mirror or the broadband mirror. Optical losses are small and dominated by  the PBS, which typically absorbs 5-8% per passage. Because it only creates displaced copies of the PSF, the Circulator can be stacked with other PSF engineering approaches. If combined with astigmatism-based encoding of the axial emitter position, for example, each of the four spots will also describe the z-position of the emitter. Furthermore, our Circulator maintains the full field of view of the camera except for a small region at the edge where one or more of the PSF copies lay outside the camera frame, eliminating the need for registration or synchronization of multiple cameras. Finally, the PBS, QWPs and mirrors could be assembled as a single cemented optical element, ensuring highly robust and alignment-free operation.
We applied the Circulator to simultaneous three-color single-molecule localization microscopy, using DNA-PAINT probes to multiplex the different targets. We identified Abberior Star 488, Cy3B, and Atto643 as suitable dyes for the green, orange, and red emission channels respectively. Figure 2 shows an example three-color PAINT image obtained on a COS cell in which the microtubuli, vimentin, and clathrin were stained, as well as magnified expansions. A movie of the recorded data is available in the Supplementary Information as SI movie 1. As the observed PSFs consist of unmodified copies of the input PSF, the resulting images can either be analyzed using existing (unmodified) analysis programs that detect the native microscope PSF, and then be post-processed using conceptually straightforward software that connects emitter positions with the correct relative lobe positions, or using a more dedicated analysis that directly recovers the different PSF patterns. Using this approach, we were able to acquire three color channels simultaneously, speeding up the overall imaging acquisition by a factor of three. Furthermore, the different images do not have to be aligned or co-registered since the fluorophores are imaged onto the same camera. A possible disadvantage of the Circulator is that the increased number of PSF spots reduces the maximal emitter density allowed before the overlap between the emission spots starts reducing the analysis accuracy. However, even in a naive implementation the information content and throughput are still increased, simply because the number of spots is doubled but three different dyes can be distinguished. Furthermore, practical samples are often heterogeneous, with distinct regions of higher and lower fluorophore density, such that the copies from denser regions often end up in sparser regions of the sample where they can be more easily resolved. Finally, the additional spots are created in a few well-defined geometries, which should allow the mitigation of the increased density by sophisticated analysis such as those based on machine learning.
We then combined our Circulator with superresolution optical fluctuation imaging (SOFI) [11] in order to deliver multicolor high-resolution images at much higher emitter densities, though with a reduced resolution improvement. SOFI relies on the statistical analysis of many images (hundreds to thousands) acquired from a sample labeled with emitters that show fluorescence dynamics. SOFI is supported by a full analytical model that describes the full imaging process [12] and has also been applied with DNA-PAINT [13]. Furthermore, the SOFI analysis can be extended to selectively extract emitters with a particular PSF shape from a set of different PSF shapes, by appropriate selection of the cross-cumulants used in the SOFI calculation [14]. This approach can readily be applied to imaging data acquired using the Circulator (Figure 3a).
We applied this approach by imaging the DNA-PAINT samples at high concentrations of the DNAbound fluorophores, and applying a SOFI calculation in which the cross-cumulants were selected to respond to the PSF patterns of the different emitter colors. A movie of the raw data is provided in the Supplementary Information as SI movie 2. An average image of the acquired images readily shows the image duplications created by the PSF copies (Figure 3b). From this single dataset, the pattern-matched SOFI analysis readily retrieves the three different high-resolution images showing the distinct emitters (Figure 3c). While the resolution of the SOFI images is lower than that of SMLM, the high emitter densities of this data made it possible to acquire these images approximately 50 times faster.
In conclusion, we presented a new approach for PSF engineering in wide-field fluorescence microscopy that creates faithful copies of the native PSF instead of realizing continuous PSF shape changes. This new approach supports operation over a broad color spectrum without sophisticated optical elements. It is compatible with other PSF engineering approaches and does not require the development of new image analysis algorithms. We then demonstrated our approach by implementing the Circulator, an add-on module that encodes the emission color of the emitter using only off-the-shelf components. The Circulator enabled the simultaneous acquisition of three emitter species in imaging experiments, where the emitters are either sufficiently sparse or show fluorescence dynamics suitable for SOFI, while retaining the full field of view of the camera.
The Circulator can be adapted to a range of dyes and experiments by adjusting the dichroic mirrors that are used. In principle, the design could also be used not just to classify emitters according to their emission spectrum, but also to estimate the emission spectra of the emitters via a ratiometric analysis of the intensity of the different emission spots. Such an approach may estimate the local environment of solvatochromic dyes, for instance, or separate an increased number of dye molecules. While we reported on SMLM and SOFI in this contribution, we expect the Circulator to be useful in any technique that involves sparse (singlemolecule) emitters with different emission wavelengths, such as smFRET, single-particle tracking, and various in-situ 'omics' approaches, such as smFISH or MERFISH, and derived methods ensuring sparseness by the spatial distribution of the labeled structures. Like many optical instruments, our device could also be used in other optical imaging disciplines, such as astronomy. Overall, we expect that the Circulator can readily increase the information content and throughput for these techniques, at very modest cost and as a compact module that fits on existing microscopes.

Data analysis
The SMLM and SOFI analysis were performed with in-house code using Igor Pro (WaveMetrics Inc.). The source code is available upon reasonable request.