Open access peer-reviewed chapter

Super‐Resolution Confocal Microscopy Through Pixel Reassignment

Written By

Longchao Chen, Yuling Wang and Wei Song

Submitted: October 27th, 2015 Reviewed: March 18th, 2016 Published: September 21st, 2016

DOI: 10.5772/63192

Chapter metrics overview

1,887 Chapter Downloads

View Full Metrics


Confocal microscopy has gained great popularity in the observation of biological microstructures and dynamic processes. Its resolution enhancement comes from shrinking the pinhole size, which, however, degrades imaging signal‐to‐noise ratio (SNR) severely. Recently developed super‐resolution method based on the pixel reassignment technique is capable of achieving a factor of 2 resolution improvement and further reaching twofold improvement by deconvolution, compared with the optical diffraction limit. More importantly, the approach allows better imaging SNR when its lateral resolution is similar to the standard confocal microscopy. Pixel reassignment can be realized both computationally and optically, but the optical realization demonstrates much faster acquisition of super‐resolution imaging. In this chapter, the development and advancement of super‐resolution confocal microscopy through the pixel realignment method are summarized, and its capabilities of imaging biological structures and interactions are represented.


  • super resolution
  • confocal microscopy
  • pixel reassignment
  • computational realization
  • optical realization

1. Introduction

Better understanding of biological processes at the cellular and subcellular level is closely dependent on the direct visualization of the cellular microstructures. Among the various microscopic techniques, fluorescence microscopy takes advantage of the abilities to observe in real‐time the molecular specificities in living biological samples down to the cellular and/or subcellular scale, and thus has found broad applications in the investigations of cell biology and neuroscience. However, the spatial resolution of conventional microscopy is optically diffraction‐limited, restricting its lateral resolution to be ∼250 nm and axial resolution to be ∼600 nm (primarily determined by the numerical aperture of microscopic objective), respectively. As a result, it is very challenging to resolve the subcellular structures by the conventional microscopic technologies because their microstructures are comparable to (even finer than) the diffraction‐limited resolution.

Fortunately, a number of novel fluorescence microscopic techniques with super‐resolution capability have been established to break down the optical diffraction limitation in recent years, allowing the observation of many cellular and subcellular structures that are always not resolvable by the conventional fluorescence microscopy. For example, by sharpening the point‐spread function of the microscope with the suppression of the fluorescence emission on the rim of a focused laser spot, stimulated emission depletion (STED) microscopy breaks the optical diffraction limitation and achieves resolution as high as ∼30 nm [1]. Localization‐based techniques, such as stochastic optical reconstruction microscopy (STORM) and photoactivated localization microscopy (PALM), enable imaging at a resolution of ∼20 nm [2, 3]. Structured illumination microscopy (SIM) applies spatially structured light illumination for shifting the high spatial frequency to the low‐frequency range, which thus can be collected by microscopy [4]. These methods achieve an order of magnitude improvement in spatial resolution over the conventional fluorescence microscopy. Therefore, the super‐resolution microscopic technology opens up new windows for observing the previously unresolved cellular structures and provides great potentials for elucidating biological processes at the subcellular and molecular scale [4].

Among these high‐resolution fluorescence microscopic techniques, confocal microscopy, the first super‐resolution imaging technique, is one of the most widely used imaging approaches with moderately enhanced spatial resolution. Utilizing a focused laser as an excitation source in combination with a pinhole in front of the detector for blocking out out‐of‐focus signals, confocal microscopy is able to improve the spatial resolution by a factor of 2 in principle. However, instead of its super‐resolution capability, the sectioning capability is more impressed because the spatial resolution with a factor of 2 improvement is hardly accessible in the standard confocal microscopy. The resolution of confocal microscopy relies on the pinhole diameter, that is, higher resolution comes from the smaller sized pinhole filter. Such a small pinhole rejects the unwanted out‐of‐focus light, while parts of the desired in‐focus emission are filtered out simultaneously. As a result, the signal‐to‐noise ratio (SNR) is drastically decreased as the pinhole size shrinks, which, in turn, practically deteriorates the spatial resolution. Instead, the fluorescence efficiency within the biological samples is often weak, so a relatively large pinhole diameter is typically chosen concerning the imaging SNR. Therefore, the standard confocal microscopy is practically unable to provide super‐resolution imaging.

In order to achieve spatial resolution improvement and better imaging SNR simultaneously in confocal microscopy, light/fluorescence signals should be detected with a nearly closed pinhole array instead of a single pinhole [5]. The images acquired by each pinhole within the array have the same resolution but different SNR levels [6]. To overcome this limitation, a method applying the pixel reassignment technique is proposed by reasonably summing the signals from each nearly closed pinhole together, which enables simultaneous improvement of resolution and SNR. In this chapter, we present the state‐of‐the‐art super‐resolution techniques based on the pixel reassignment. Section 2 gives the principle of pixel reassignment firstly, and then two different operations realizing the pixel reassignment. Also, some representative super‐resolution images in biological specimens are summarized in this section. At last, some advances in super‐resolution confocal microscopy through the pixel reassignment will be discussed.


2. Super resolution by pixel reassignment

The concept of pixel reassignment is firstly proposed more than two decades ago to solve the drawbacks in standard confocal microscopy [5]. As we know, the reduction of the pinhole diameter down to zero allows the finest lateral resolution in confocal microscopy in theory, which, however, generates fluorescent images with a very low SNR due to the dramatically degraded light collection efficiency. Although the pinhole size can be adjusted to one Airy unit for better imaging SNR, the lateral resolution is sacrificed. Instead of a single pinhole, a pinhole array is used for the light detection, followed by a reconstruction algorithm for the image formation. As a result, the standard confocal microscopy with the pixel reassignment operation is capable of enhancing its lateral resolution simultaneously with higher imaging SNR.

2.1. Principle of pixel reassignment

Figure 1.

Schematic diagram illustrating the principles of pixel reassignment. (a) One‐dimensional representation of pixel reassignment. Two pinholes (left and right) within an array displaces by a distance of ‘a’ from the excitation focus, which detect light signals mostly originated from the location of the peak of the product of PSFdet (x-a) and PSFex (x). In the case that PSFdet and PSFex are identical (i.e. neglecting the Stokes shift), the maximum in PSFeff occurs at the position with a distance of a/2 from the excitation focus. Thus, the detected light signals from the displaced pinholes are reassigned to the well‐aligned pinhole that is at the center of the excitation focus and the original detection spot. (b) Pixel realignment operation. Top panel shows the excitation foci (blue circles) created by scanning illuminating laser across the sample, where four excitation foci are with the distance of D and diameter of a. Bottom: Two pixel realignment operations for increasing the image resolution. Lower left panel represents twofold reduction of the foci without altering their distance. Lower right panel displays the increase of the foci distance to 2D, while maintaining all foci sizes. These two implementations produce an equivalent imaging reconstruction, with only different global scaling factor.

Pixel reassignment demonstrates great potentials for improving both lateral resolution and imaging SNR. Instead of summing the signals directly as the conventional imaging technologies, each signal is reassigned to a particular location where the signal most probably comes. Figure 1(a) gives the principle of the pixel reassignment in terms of excitation and detection point‐spread function (PSF) [7]. The excitation PSF (PSFex, labeled by blue line) represents the distribution of the corresponding excitation focus. At a displaced pinhole, detection PSF (PSFdet, labeled by green line) is centered on the detection axis with a distributed probability of signal detection around that pinhole. The effective PSF (PSFeff, labeled by red line) is contributed from the overlap (multiplication) of PSFdet and PSFex. The well‐aligned pinhole is coaxial with the excitation focus, realizing the maximal signal detection probability. As the pinhole detector is far away from the axis of the excitation focus, the signal acquisition probability decreases because of their less overlying; consequently, these nearly closed pinhole detectors induce lower‐SNR image.

In the pixel reassignment implementation, a camera (similar with a pinhole array), rather than a point detector, is commonly employed because its individual pixels are considered as infinitely narrow pinhole. Neglecting Stocks shift in single‐photon fluorescence and assuming identical PSFdet and PSFex, a maximal probability of signal acquisition (i.e. PSFeff) is at the midway of the peaks of PSFdet and PSFex. Figure 1(b) gives two methods for the pixel reassignment operation, either twofold local contraction of the excitation focus without altering the distance between them (panel in lower left of Figure 1(b)), or twofold increasing the distance between the foci while maintaining their original size (panel in lower left of Figure 1(b)) [8]. By reassigning the signals from all pixels within the detector array (i.e. all displaced pinholes as shown in Figure 1(a)) to the particular location, a sharper and higher‐SNR image is eventually achieved.

Pixel reassignment technique is able to improve the resolution to a factor of 2 without sacrificing SNR, and the resolution can be further improved by deconvolution algorithm up to a factor of 2 [9, 10]. Although the spatial resolution of the pixel reassignment technique is still lower compared with other super‐resolution methods, such as STED and STORM [13], it overcomes some of their shortcomings. This technique inherits all advantages of the standard confocal microscopy, including high‐speed imaging rate, acceptable excitation intensity, optical sectioning capability, and a broad choice of fluorescent dyes and/or proteins, making it a readily accessible technology in a variety of biological investigations.

The pixel reassignment can be considered as an alternative method of SIM, theoretically achieving the same spatial resolution improvement compare with standard SIM through point‐like illumination feature. In contrast, the technique demonstrates better feasibility over the standard SIM, that is, the pixel reassignment operation can be easily implemented both computationally and experimentally (optical system adaptation). Unlike computational mode that is always time‐consuming in raw data processing, the pixel reassignment realized with optical means is capable of obtaining super‐resolution images with fast imaging acquisition. More details on these two different methods for realizing the pixel reassignment are represented as below.

2.2. Computational realization of pixel reassignment

2.2.1. Image scanning microscopy

Image scanning microscopy (ISM), proposed by C. Müller and J. Enderlein in 2009, is a super‐resolution microscopic technique based on the pixel reassignment [11]. This system is modified from a standard confocal microscopy that replaces the point detector (normally a photomultiplier tube) with an Electron multiplying CCD (EMCCD) camera (labeled 9) as shown in Figure 2(a). The camera takes an image of each spatial position of the scanning focus, and then an algorithm of the pixel reassignment processing is utilized by summing the raw images to reconstruct an ISM image, which improves the resolution from 244 nm to 198 nm laterally.

Figure 2.

Super‐resolution image scanning microscopy (ISM) with computational pixel reassignment. (a) The schematic diagram of ISM system. Fluorescence excitation (1); a super‐continuum white light laser equipped with an acousto‐optic tunable filter; nonpolarizing beam splitter cube (2); dichroic mirror (3); piezo scanning mirror (4); 4f telescope configuration (5); microscope objective (6); beam diagnostic camera (7); confocal aperture with 200 μm diameter (8); EMCCD camera for fluorescence detection (9). (b) Super‐resolution imaging fluorescent beads with 100‐nm diameter. Left panel: Confocal microscopy image; middle panel: ISM image; right panel: Fourier‐weighted ISM image. Scale bar: 1 μm. (c) Linear cross‐sectional distribution along the horizontal axis of an individual bead image in (b). Adapted with permission from reference [11].

Further, deconvolution function is used to improve its lateral resolution up to 150 nm, 1.63‐fold better than the image from raw data, as shown in Figure 2(b) and (c), respectively. Note that the pinhole in ISM (labeled 8) filters the out‐of‐focus light signals, maintaining the optical sectioning capability as the standard confocal microscopy. In this work, the realization of the lateral resolution improvement up to 198 nm does not entirely rely on the pinhole because of its relatively large diameter, which, however, gives a high imaging SNR. Therefore, with the computational pixel realignment ISM is able to provide images with optimization of both spatial resolution and imaging SNR.

2.2.2. Multifocal structured illumination microscopy

Figure 3.

The schematic of multifocal structured illumination microscopy (MSIM). Lasers with 561 and 488 nm serve as illumination sources. Both laser outputs are combined with a dichroic (DC). After beam expanding, both lasers are directed onto a digital micromirror device (DMD). The resulting pattern is de‐expanded by a pair of lenses, and is subsequently delivered by the tube lens and microscopic objective inside the microscope (not shown) into the samples. Mechanical shutters (SH) placed in front of the laser output are used for switching illumination on or off. Adapted with permission from reference [9].

ISM demonstrates multiple advantages, including the optical sectioning capability as the standard confocal microscopy, the enhanced lateral resolution, and the high fluorescence collection efficiency [11]. However, it is subjected to slow frame rate due to the EMCCD camera (imaging acquisition of 10 ms with each scanning position), and is time‐consuming for visualizing the three‐dimensional (3D) microstructures.

In order to speed up the imaging acquisition, Shroff et al. developed multifocal structured illumination microscopy (MSIM) by using a sparse lattice of excitation foci (similar to swept‐field or spinning disk confocal microscopy) in 2011 [9]. As shown in Figure 3, MSIM applies a digital micromirror device (DMD) for generating the sparse lattice illumination patterns. After a series of reconstruction steps (open‐source software), MSIM enables 3D subdiffractive imaging with resolution doubling, indicating a lateral resolution at 145 nm and an axial resolution at 400 nm. Moreover, it provides the capability of significantly fast imaging acquisition at one 2D image per second.

For super‐resolution MSIM, the data acquisition and processing are implemented as below (please refer to Figure 4 for detailed procedures). First, the sample is excited with a sparse, multifocal excitation pattern. Second, the resulting fluorescence image is recorded with a camera, and then the digital pinholes around each fluorescent focus are applied for rejecting the out‐of‐focus emission. Afterwards, the pixel reassignment with 2× scaling is used to process the resulting image. Repeat the above procedures for the entire imaging region fully illuminated. Eventually, a super‐resolution image with 2‐fold resolution improvement is obtained through the digital summation of all such pinholed and scaled images. Twofold resolution improvement is further achieved with deconvolution.

Figure 4.

Super‐resolution MSIM realization. Top left figure represents a wide‐field image produced with a uniformly illuminated pattern onto sample. Right panel provides the reconstructed procedure for the first, tenth, and final raw images of a 120‐frame sequence. Lower left figure displays the super‐resolution MSIM image by deconvolving the summed image. Adapted with permission from Ref [9].

The resolution improvement of MSIM is demonstrated by imaging antibody‐labeled microtubules in human osteosarcoma (U2OS) cells embedded in Fluoromount as shown in Figure 5. Compared to the wide‐field images, the multifocal‐excited, pinholed, scaled, and summed (MPSS) images have both higher resolution and better contrast (Figure 5(b)). In Figure 5(d), the full‐width at half maximum (FWHM) of light intensity of microtubules is estimated at about 145 nm in MSIM images, giving a twofold resolution enhancement compared with the image from wide‐field microscopy (∼299 nm). Moreover, the frame rate of acquiring an image with field of view at 48 × 49 μm is up to 1 Hz in MSIM, indicating more than 6500‐fold faster acquisition over the ISM technology [11].

Figure 5.

Resolution doubling of MSIM by imaging antibody‐labeled microtubules in human osteosarcoma (U2OS) cells. (a) MSIM imaging microtubules labeled with Alexa Fluor 488 in a fixed cell. MSIM image is formed from 224 raw images taking ∼1 s total acquisition time with 4.5 ms for each image. Scale bar: 5 μm. (b) Magnified images from the boxed region in (a). Top panel showing a wide‐field image, middle panel showing an MPSS image, and bottom panel showing an MPSS and deconvolved (MSIM) image. Scale bars: 5 μm. (c) Close‐up images of the boxed regions in (b). Scale bars: 1 μm. (d) Intensity profiles along the colored lines in (b), giving FWHM values at 299 nm in wide‐field microscopy, 224 nm in MPSS, and 145 nm in MSIM, respectively. Adapted with permission from reference [9].

2.3. Optical realization of pixel reassignment

The pixel reassignment implemented by the computational means is capable of doubling the resolution than wide‐field imaging [9, 11]. The limitation, however, is that the methods are fundamentally time‐consuming compared to the standard conventional microscopy because a large number of raw images are essentially acquired and processed. Recently, optically realized pixel reassignment has been developed to overcome the limitations by adapting the optical imaging system instead of digital data‐processing operations, which produces images with comparable improvement in the spatial resolution [8, 10, 12].

2.3.1. Instant structured illumination microscopy

Instant structured illumination microscopy (ISIM) is developed by Shroff et al. in 2013 that is analogous to MSIM, while its pixel reassignment process operates optically instead of the digital computation procedures [10]. As shown in Figure 6, the DMD used in MSIM is replaced with a converging microlens array. As a result, a multifocal excitation pattern is generated in ISIM. Correspondingly, a matched pinhole array is added to physically reject the out‐of‐focus emissions. With this modification, the optical pixel reassignment is realized based on the matched microlens array for twofold local contraction of each fluorescent focus. The fluorescence emission pattern is imaged onto a camera by galvanometer scanning. Eventually, the pinholed and scaled images are optically summed, enabling 2‐fold resolution enhancement.

Figure 6.

Principles of implementing instant structured illumination for super‐resolution realization. A multifocal excitation pattern is produced with a converging microlens array. For fluorescence detection, a pinhole array that matches the microlens array rejects the out‐of‐focus fluorescence signals. Afterwards, a second, matched microlens array allows a twofold local contraction of each pinholed fluorescence emission. A galvanometer serves as raster scanning of multifocal excitation and summation of multifocal emission, which thus produces a super‐resolution image during each camera exposure. Adapted with permission from reference [10].

ISIM demonstrates 3D super‐resolution imaging with a lateral resolution of 145 nm and an axial resolution of 350 nm, nearly comparable with MSIM. Moreover, the 100 Hz frame rate comes from the optical operation of pixel realignment in ISIM, allowing super‐resolution real‐time imaging (almost 100‐fold faster than MSIM). Taking into account the data processing duration, the speed‐up factor exceeds 10000. In addition, the low illumination power in ISIM (∼5–50 W/cm2) mitigates photobleaching. As a result, ISIM can perform imaging over tens of time points without obvious photobleaching or photodamage. In Figure 7, the rapid growth (∼3.5 μm/s) of endoplasmic reticulum (ER) is monitored by ISIM even though less than 140 ms in the formation and growth of new ER tubules. The biological processes blur in previously developed technologies, such as MSIM and ISM [9, 11]. The capabilities make ISIM a powerful tool for time‐lapse super‐resolution imaging in living biological samples.

Figure 7.

ISIM demonstrates high frame rate of imaging endoplasmic reticulum (ER) at 100 Hz. (a) The first image from 200 time points. ER labeled with GFP‐Sec61A within MRL‐TR‐transformed human lung fibroblasts. Scale bar: 10 μm. (b) Magnification of image with the large white box in (a). White arrows point out the growth process of an ER tubule; blue arrows represent the remodeling of an ER tubule. Scale bar: 5 μm. (c) Magnification of the image with the small white box in (a), displaying the dynamic formation of a new tubule within 140 ms. Scale bar: 200 nm. Adapted with permission from reference [10].

2.3.2. Re‐scan confocal microscopy

Rescan confocal microscopy (RCM) is another optical realization of the pixel reassignment technique, proposed by Luca et al. in 2013 [12].Compared with ISIM, it is more easily accessible to build an RCM because this system can be readily modified from a standard confocal microscopy as shown in Figure 8. The optical pixel reassignment in RCM is realized as below. The focal length of the lenses L2 and L3 is adapted for twofold local contraction of the fluorescent focus spot. Alternatively, the final fluorescence image is twofold magnified while maintaining the original fluorescence foci size.

Figure 8.

The schematic of rescan confocal microscopy (RCM). Unit 1: A standard confocal microscopy with a set of scanning mirrors for scanning the excitation light and de‐scanning the emission light. Unit 2: A re‐scanning configuration for ‘writing’ the light that passes the pinhole onto the CCD‐camera. Although the pinhole is in a relatively large diameter, the resolution is 2 times improved, which thus gives much more photo‐efficient advantage compared to conventional confocal microscopes with the similar resolution. Adapted with permission from reference [12].

This process is accomplished by reasonably changing the angular amplitude of the rescanner. The ratio of angular amplitude of the two scanners, expressed by the sweep factor M, changes the properties of the rescan microscope. For M = 1 the microscope has the same lateral resolution with a wide‐field microscope, defined by the well‐known optical diffraction limit; it achieves the super resolution for M = 2. The rescanner is used to deliver the fluorescence emission onto the camera pixels. The camera is in the exposure status for optical summation of the fluorescent focus during rescanning.

The lateral resolution improvement of RCM is quantified by imaging 100‐nm fluorescent beads. FWHM is found to reduce from 245 nm (15 nm) in wide‐field imaging to 170 nm (±10 nm) in RCM imaging, indicating an improvement by a factor of 2 without deconvolution. Also, the resolution improvement is concluded by visualizing fluorescently labeled microtubules of HUVEC cell in Figure 9(a)–(f). To demonstrate the capability of RCM for monitoring dynamics, the time‐lapse imaging of living HeLa cells expressing EB3‐GFP with the growing end of microtubules is observed by RCM. As shown in Figure 9(g), RCM is able to track the fast dynamics (0.5 μm/s) with multiple advantages of improved resolution, high sensitivity, and sufficient imaging rate (1 fps).

Figure 9.

Fluorescently labeled microtubules in HUVEC cells imaged by RCM with sweep‐factor M=1 (a), which gives an image with resolution of a wide‐field fluorescence microscope determined by the diffraction limit. In double‐sweep mode (sweep‐factor M=2) (b) RCM gives resolution improvement by a factor of 2. Junctions of microtubules (c, e) and parallel microtubules (d, f) are unresolved with wide‐field resolution (c, d), but distinguished by RCM in double sweep mode (e, f). (g) Screenshots from an RCM time lapse series of living HeLa cells at M=2 demonstrate the monitoring of fast dynamic structures (0.5 μm/s). Scale bars: 1 μm. Adapted with permission from reference [12].

2.3.3. Two‐photon instant structured illumination microscopy

RCM improves resolution by a factor of 2 compared with wide‐field imaging while possessing optical sectioning capabilities as the traditional confocal microscope [8]. Two‐photon excitation offers better optical sectioning capability based on the nonlinear effect. Infrared excitation light minimizes the optical scattering in the tissue, and the fluorescent signals come only from two‐photon absorption. These advantages effectively increase the penetration depth and simultaneously suppress the background signal, making the two‐photon excitation technique an ideal imaging tool for the thick samples.

Two‐photon instant structured illumination microscopy (2P ISIM) is a combination of RCM and two‐photon excitation technique, presented by Shroff et al. in 2014, as shown in Figure 10(a) [8]. Similarly, an additional scanning component is introduced in 2P ISIM for the optical realization of pixel reassignment. In Figure 10(b)–(d), 2P ISIM provides better resolution than the diffraction‐limited two‐photon excitation mode by imaging the microtubules. Applying the deconvolution, the lateral resolution is further improved in Figure 10(c). 2P ISIM is quantified by ∼150 nm in the lateral resolution and by ∼400 nm in the axial resolution, respectively, with 100‐nm diameter fluorescent beads as imaging targets. A factor of 2 (with deconvolution) resolution enhancement is obtained compared with the conventional two‐photon wide‐field imaging (∼311 nm).

Figure 10.

Schematic diagram of two‐photon instant structured illumination microscopy (2P ISIM) and its imaging capabilities. (a) Pulsed femtosecond laser (2PE) serves as a two‐photon excitation source (labeled by red line). Fluorescence (labeled with green line) is collected and delivered onto a camera. HWP: half‐wave plate; POL: polarizer; EXC 2D GALVO: galvanometric mirror for scanning the excitation laser; DC: dichroic mirror; IX‐70: microscope part housing objective and sample (not shown); EM 2D GALVO: galvanometric mirror for rescanning the fluorescence emission. (b)–(d) Resolution enhancement of 2P ISIM. (b) 2P ISIM image of immunolabeled microtubules in a fixed U2OS human osteosarcoma cell after deconvolution processing. (c) Magnified view of the yellow rectangular region in (b), indicating the resolution improvement in deconvolved 2P ISIM compared with both 2P wide‐field microscopy (2P WF) and 2P ISIM. (d) Fluorescence intensity profiles of microtubules highlighted with green, red, and blue lines in (c). Scale bar: 10 μm in (b) and 3 μm in (c). Adapted with permission from reference [8].

To demonstrate the enhanced penetration ability of 2P ISIM in living thick samples, embryos of transgenic Caenorhabditis elegans expressing GFP‐H2B are imaged in Figure 11. Both imaging resolution and contrast severely degrade at depths of more than ∼15 μm from the coverslip surface in 1P illumination due to strong scattering in deep tissue (Figure 11(a), (b)). The degradation is not compensated by increasing of the exposure time, which, however, mainly leads to high background noise. Two‐photon excitation of 2P ISIM effectively suppresses the out‐of‐focus emission. Thus, the subnuclear chromatin structures are clearly observed up to the depth of ∼30 μm in Figure 11(c), (d), where the fluorescence signals slightly reduce as the depth increases.

Figure 11.

Enhanced penetration ability in 2P ISIM. (a, b) 1P ISIM images of a nematode embryo expressing GFP‐H2B in nuclei. (a) Cross sections of the worm embryo at different axial positions. Scale bar: 10 μm. (b) Magnifications of the yellow rectangular regions in (a). Scale bar: 3 μm. The degradation in imaging contrast is observed as the depths increase. (c, d) 2P ISIM visualizes the subnuclear chromatin structure throughout nematode embryos. (c) Cross sections at the representative axial position. Scale bar: 10 μm. (d) Magnifications of yellow rectangular regions in (c), indicating better resolution, higher contrast, and larger imaging depth compared with 1P ISIM. Scale bar: 2 μm. Adapted with permission from reference [8]


3. Conclusion

In this chapter, we represent the super‐resolution confocal microscopy (and two‐photon microscopy) realized through the pixel reassignment methods computationally and optically. These demonstrate multiple advantages of resolution improvement, high fluorescence collection efficiency, optical sectioning capability, and fast imaging acquisition, which thus is able to investigate biological structures and processes at the cellular and even macromolecular level with 3D spatial scale. Additionally, because the method is directly established based on the standard confocal microscopy and/or two‐photon microscopy, it mitigates the requirements in fluorescent probes and/or labeling methods that are always indispensable in some super‐resolution fluorescence microscopic technologies, such as STORM and PALM [2, 3].

More importantly, the development of these techniques is not limited in the laboratorial stage. In 2015, the first commercial setup, LSM 800, is established by Carl Zeiss [13], which, in principle, is based on ISM but replaces the EMCCD camera with a 32‐channel linear GaAsP‐PMT array (i.e. Airyscan detector as shown in Figure 12). The highest imaging speed of LSM 800 with 512×512 pixels is up to 8 Hz, tremendous faster than ISM. Therefore, we expect that the super‐resolution microscopy based on the pixel reassignment technique has great potentials for boosting imaging acquisition speed, and therefore further provides better understanding in intracellular molecular interactions and dynamic processes within living biological specimens.

Figure 12.

Schematic diagram of Airyscan detector in LSM 800. In brief, a hexagonal microlens array (a) collects incident light, which is in direct connection with the ends (b) of a fiber bundle (c). The other ends (d) of the fibers are in contact with a linear GaAsP‐PMT array (e) serving as a detector. Thus, an area detector is created, onto which the Airy disk is imaged via a zoom optic configuration. Note that the single detector element, replacing the classical pinhole, acts as the separate pinholes in Airyscan detection. Adapted with permission from reference [13].

In addition to the issue of imaging acquisition speed, multicolor fluorescence microscopy is desired for investigating the interactions between different structures or biomolecules via labeling them with distinct colors. The possible interactions can be revealed by the co‐localization of the different dyes and/or proteins. The standard fluorescence microscopy, however, might give inaccurate co‐localization due to the diffraction‐limited resolution. In combination with the pixel reassignment, the multicolor imaging technique is anticipated to provide a high‐resolution imaging of the biological interaction within live cells.

In MSIM and ISIM based on the pixel reassignment approach [9, 10], both super‐resolution imaging capability and color differentiation have been demonstrated, which have the advantages of easily configured optical system and weak cross‐talk effect between the different colors. Switching laser lines for the excitation of different fluorophores might induce spatial mismatch in the images. Therefore, it is more preferable for simultaneously exciting all fluorophores and synchronously collecting their fluorescence signals. Multiple detectors with appropriate dichroic mirrors and emission filters can be used to collect the different fluorescence signals with different detection channels. Alternatively, an imaging spectrometer can be applied to record the spectral feature of these fluorophores.

Synchronous imaging decreases the fluorescence photobleaching probability due to low light exposure, benefiting to long‐term monitoring of living samples. However, cross‐talk of the different fluorophores always occurs because of the broad and overlapping excitation and emission bands of fluorophores. Although the cross‐talk effects can be removed by selecting dyes with appropriately wide and non‐overlapping emission spectra, the dyes are often inaccessible, which thus restricts its application in multicolor imaging. Linear spectral unmixing analysis is a solution to eliminate the cross‐talk effect in spectral imaging [14]. The spectrum of the mixed fluorescent signal is expressed as a linear integration of the component dye spectra [15], and therefore the concentration or intensity of the fluorescence from each dye can be precisely analyzed. Based on the data analysis, both spatial mismatch and cross‐talk effect are mitigated in multicolor imaging of live cells.

Figure 13.

Multicolor RCM reveals the cellular microstructures labeled with different dyes. (a) Simultaneous RCM imaging of nucleus and lysosomes labeled with SYTO 82 and LysoTracker Red in a live bEnd.3 cell, respectively. Based on the linear spectral unmixing analysis, nucleus (c) and lysosomes (d) are differentiated according to their corresponding spectral features (e), respectively. (b) Overlaid image of the RCM images from (c) and (d). Scale bar: 5 μm.

In Figure 13, we establish a multicolor RCM with simultaneous excitation of different fluorophores and synchronous collection of their fluorescence. Linear spectral unmixing analysis is implemented for the spectral differentiation of the live cells stained with different dyes. SYTO 82‐labeled nucleus and LysoTracker Red‐stained lysosomes within live bEnd.3 cells are imaged by RCM with a spectrometer as the spectral detector. The nucleus and lysosomes are captured simultaneously, followed by the linear spectral unmixing analysis based on the known spectral features of these two dyes (severely overlapping as shown in Figure 13(e)). Figure 13(b)–(d) gives a clear separation of the two kinds of subcellular organelles. This approach is very powerful in investigation of the dynamic interactions of the subcellular structures.



This work was supported in part by the National Natural Science Foundation of China grant Nos. 61505238 and 11504042, Daqing Normal University Youth Foundation No. 12ZR12, Daqing Normal University doctor Foundation No. 15ZR03 doctor Foundation, Natural Science Foundation Project of Heilongjiang Province Nos. A200506 and QC2015066, Science and Technology Research Project of Heilongjiang Province Education Department No. 12543002, and Guidance of Science and Technology Plan Projects of Daqing City No. szdfy‐2015‐59.


  1. 1. V. Westphal and S. W. Hell, “Nanoscale resolution in the focal plane of an optical microscope,” Phys. Rev. Lett. 94(14), 143903 (2005).
  2. 2. M. Bates, B. Huang, G. T. Dempsey and X. Zhuang, “Multicolor super‐resolution imaging with photo‐switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
  3. 3. M. J. Rust, M. Bates and X. Zhuang, “Sub‐diffraction‐limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
  4. 4. B. Huang, M. Bates and X. Zhuang, “Super resolution fluorescence microscopy,” Annu. Rev. Biochem. 78, 993 (2009).
  5. 5. C. Sheppard, “Super‐resolution in confocal imaging,” Optik 80(2), 53–54 (1988).
  6. 6. I. J. Cox, C. J. Sheppard and T. Wilson, “Improvement in resolution by nearly confocal microscopy,” Appl. Opt. 21(5), 778–781 (1982).
  7. 7. J. McGregor, C. Mitchell and N. Hartell, “Post‐processing strategies in image scanning microscopy,” Methods 88, 28–36 (2015).
  8. 8. P. W. Winter, A. G. York, D. Dalle Nogare, M. Ingaramo, R. Christensen, A. Chitnis, G. H. Patterson and H. Shroff, “Two‐photon instant structured illumination microscopy improves the depth penetration of super‐resolution imaging in thick scattering samples,” Optica 1(3), 181–191 (2014).
  9. 9. A. G. York, S. H. Parekh, D. Dalle Nogare, R. S. Fischer, K. Temprine, M. Mione, A. B. Chitnis, C. A. Combs and H. Shroff, “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy,” Nat. Methods 9(7), 749–754 (2012).
  10. 10. A. G. York, P. Chandris, D. Dalle Nogare, J. Head, P. Wawrzusin, R. S. Fischer, A. Chitnis and H. Shroff, “Instant super‐resolution imaging in live cells and embryos via analog image processing,” Nat. Methods 10(11), 1122–1126 (2013).
  11. 11. C. B. Müller and J. Enderlein, “Image scanning microscopy,” Phys. Rev. Lett. 104(19), 198101 (2010).
  12. 12. G. M. De Luca, R. M. Breedijk, R. A. Brandt, C. H. Zeelenberg, B. E. de Jong, W. Timmermans, L. N. Azar, R. A. Hoebe, S. Stallinga and E. M. Manders, “Re‐scan confocal microscopy: scanning twice for better resolution,” Biomed. Opt. Express. 4(11), 2644–2656 (2013).
  13. 13. J. Huff, “The Airyscan detector from ZEISS: confocal imaging with improved signal‐to‐noise ratio and super‐resolution,” Nat. Methods 12(12), (2015).
  14. 14. T. Haraguchi, T. Shimi, T. Koujin, N. Hashiguchi and Y. Hiraoka, “Spectral imaging fluorescence microscopy,” Genes. Cells 7(9), 881–887 (2002).
  15. 15. H. Tsurui, H. Nishimura, S. Hattori, S. Hirose, K. Okumura and T. Shirai, “Seven‐color fluorescence imaging of tissue samples based on Fourier spectroscopy and singular value decomposition,” J. Histochem. Cytochem. 48(5), 653–662 (2000).

Written By

Longchao Chen, Yuling Wang and Wei Song

Submitted: October 27th, 2015 Reviewed: March 18th, 2016 Published: September 21st, 2016