Open Access is an initiative that aims to make scientific research freely available to all. To date our community has made over 100 million downloads. It’s based on principles of collaboration, unobstructed discovery, and, most importantly, scientific progression. As PhD students, we found it difficult to access the research we needed, so we decided to create a new Open Access publisher that levels the playing field for scientists across the world. How? By making research easy to access, and puts the academic needs of the researchers before the business interests of publishers.
We are a community of more than 103,000 authors and editors from 3,291 institutions spanning 160 countries, including Nobel Prize winners and some of the world’s most-cited researchers. Publishing on IntechOpen allows authors to earn citations and find new collaborators, meaning more people see your work not only from your own field of study, but from other related fields too.
The resolution of light microscopy was thought to be limited to 250–300 nanometers based on the work of Ernest Abbe. This Abbe diffraction limit was believed to be insurmountable until the invention of Super-resolution microscopic techniques in the late 20th century. These techniques remove this limit and have provided unprecedented detail of cellular structures and dynamics down to several nanometers. An emerging goal in this field is to quantitatively measure individual molecules. Measurement of single-molecule dynamics, such as diffusion coefficients and complex stoichiometries, can be accomplished using fluorescence fluctuation techniques to reveal nanosecond-to-microsecond temporal reactions. These powerful complimentary experimental approaches are made possible by sensitive low-light photodetectors. In this chapter, an overview of the principles of super-resolution and single-molecule microscopies are provided. The different types of photodetectors employed in these techniques are explained. In addition, the advantages and disadvantages for these detectors are discussed, as well as the development of next generation detectors. Finally, example super-resolution and single-molecule cellular studies that take advantage of these detector technologies are presented.
chapter and author info
Robert T. Youker*
Department of Biology, Western Carolina University, Cullowhee, North Carolina, USA
Fluorescence microscopy has been used by biomedical scientists to uncover fundamental processes in cells, tissues, and organisms for ~100 years. The discovery of genetically-encoded fluorescent proteins (e.g. GFP) allowed visualization of molecular dynamics in real-time in biological cells and organisms. A plethora of fluorescent proteins and dyes have been engineered over the last 25 years thus advancing the capabilities of fluorescence microscopy in biomedical research fields. However, a major roadblock in fluorescence microscopy was the diffraction of light that limited conventional light microscopes operating in the visible spectrum to a maximum lateral resolution of approximately 250 nm. Recently, several super-resolution techniques (e.g. PALM, STED, STORM) have been developed to “side-step” this resolution limit and “push” the boundary of optical resolution into the Nano-dimension. Most of these techniques rely on the capture of light emitted from single molecules to obtain increased resolution. Fluorescence fluctuation spectroscopy techniques (e.g. FCS, PCH, N&B) also rely on low-light measurements from few to single molecules. These techniques would not have been possible in cells and organisms without advancements in fluorescent dye chemistry and, importantly, low-light photodetectors.
In this chapter, principles of fluorescence microscopy [Section 2], an overview of super-resolution microscopy (a.k.a. fluorescence nanoscopy), and fluorescence fluctuation techniques (FFTs) will be discussed [sections 3 & 4]. In addition, the different types of detector technologies used in these techniques will be explained [Section 5]. Finally, examples of single-molecule fluorescence experiments employing the discussed photodetectors are presented [Section 6].
2. Principles of fluorescence microscopy
Today, fluorescence microscopy is widely used by scientists all over the world because the high signal-to-noise ratio (SNR) allows for the acquisition of high contrast images of specimens. The high SNR is made possible in part because of bright fluorophore-labeled molecules in the specimen. Labeling can be achieved by chemical attachment of dyes, or genetically-encoded fluorescent proteins such as GFP, or its variants (for a review see ). Two commonly employed fluorescence microscope setups are widefield (epifluorescence) and confocal. In-focus and out-of-focus light is collected in widefield fluorescence microscopy, and the out-of-focus light can degrade image quality. In confocal microscopy, out-of-focus light is rejected through an adjustable pinhole placed in front of the detector allowing optical sectioning of the specimen and improved image quality, especially for thick specimens.
The resolution of fluorescence microscopes, or generally all light microscopes, is limited by the diffraction properties and wavelength of light. Resolution of a light microscope is typically quantified by measuring the point spread function (PSF). The PSF is a mathematical description of how blurry a point-like emitter, such as a single fluorescent molecule, would look after being diffracted through a microscope. The full width at half maximum (FWHM) of the PSF is a convenient measurement to characterize resolution. Resolution is the ability to detect two adjacent objects in a sample and is not the ability to detect small structures. However, in biological specimens many subcellular structures are closer together or smaller than the FWHM of the PSF leading to a less than true representation of the specimen being imaged. Ernst Abbe first described the diffraction properties of light in 1873 and determined that resolution is dependent on both the wavelength of light used and the numerical aperture (NA), or light collecting ability, of the objective used in the microscope . A simplified equation based on his work is where d is the minimum distance to distinguish two point sources. For example, if green light at 500 nm was collected with a 1.0 NA objective then the resolution limit in this case would be d = 500/2 x 1 = 250, or 250 nm. Lord Rayleigh further defined resolution as the distance where the center of the diffraction pattern of image one overlaps with the first minima of the diffraction pattern of image two, and his equation (Rayleigh resolution criterion) is widely used today and is [2, 3]. For light microscopes, the resolution is roughly half the wavelength of light being used and equates to 150–200 nm. This diffraction limit made it difficult to image small sub-cellular structures such as endocytic vesicles that were about 60–80 nm in size. This resolution “wall” was thought to be insurmountable until new illumination techniques and photo-activatable proteins/dyes led to the development of super-resolution microscopy.
3. Review of super-resolution microscopy
This section provides an overview of several common super-resolution microscopic methods. For an in-depth treatment of the various techniques the reader is directed to these reviews [4, 5, 6]. Super-Resolution Microscopy (SRM) refers to microscopic techniques that can circumvent the Abbe diffraction limit and “break” the resolution barrier. SRM can be divided into near-field and far-field optical techniques. In near-field scanning optical microscopy (NSOM, or SNOM) no objective lens is used and instead a glass fiber with an aperture diameter smaller than the excitation light wavelength is used to create an evanescent field. The fiber is scanned over the sample at a short distance and the image resolution is only limited by the aperture dimensions. Lateral resolutions of less than 20 nm and vertical resolutions of 2–5 nm have been reported . The major drawbacks of NSOM are the very short working distance needed, long scan times, and very shallow depth of field, thus limiting NSOM to mostly surface studies [7, 8]. In contrast, far field techniques do not suffer from these limitations and have been more widely adopted in SRM studies of biological specimens.
There are two fundamentally different approaches employed for far-field SRM. One involves manipulation of the PSF geometry, or so-called PSF engineering, by controlling the excitation beam geometry, and the other is probe-based employing Photo-switchable dyes/proteins. PSF engineering approaches include stimulated emission depletion (STED), ground state depletion (GSD), and saturated structural illumination (SSIM) microscopies. Photo-activated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM) techniques comprise the probe-based super-resolution techniques. The goal in both techniques is to minimize the signal overlap of the objects and thus improve object discrimination in the image, thereby lowering the resolution limit.
In STED, the excited fluorescent molecules in the sample are exposed to a second-high intensity beam of light that upon saturation de-excites the molecules back to the ground state and thus turns them “off”. This second beam of light is annular, or doughnut shaped (generated by phase plate), and the fluorescent molecules in the center remain excited. The resolution of this approach is dependent on the spot size of the remaining excited molecules, and resolutions of 29–60 nm have been achieved with this approach [9, 10]. It is very important to make sure the depletion laser wavelength does not overlap with the excitation wavelength of the dye, or the resolution will be degraded significantly . In addition, the type of dye and specimen mounting medium can both affect image quality . STED microscopy has been used to image protein tangles in Alzheimer’s brain slices, electron transport complexes in mitochondria, and myelin sheets in oligodendrocytes [13, 14, 15]. GSD microscopy is similar to STED except the excited molecules are not de-excited. Instead they are optically “shelved” by driving the molecules into a long-lived, non-fluorescent dark state (i.e. triple state). The increased resolution is achieved using an annular shaped beam the same as STED. GSD microscopy requires special fluorophores that have a long-lived triplet state; alternatively removal of oxygen from the specimen can be used to modulate the dye’s photo-physical properties [16, 17].
Coarse interference patterns, called moiré fringes, occur during illumination of a fluorescent sample. These fringes are not transferred to the image plane during conventional fluorescence microscopy, leading to degraded image resolution. In SSIM, optical gratings are used to create a “bar-code” pattern of high spatial frequency light that is passed over the sample, and images are collected at different angles of illumination. This process allows “Capture” of the moiré fringes and high resolution images can be acquired with the aid of computer algorithms. A two-fold increase in resolution beyond the diffraction limit can be attained and up to eight-fold resolution increase in three dimensions (axial or z-axis) [18, 19, 20, 21].
PSF engineering techniques try to limit image overlap of objects through alterations in the geometry of the exciting light. A second approach is to use low levels of excitation light to active a few fluorescence molecules at a time, thereby preventing overlapping signals. The PSF from each molecule can then be fitted mathematically to determine its center. This process is repeated thousands of times to locate nearly-all fluorescent molecules, leading to a point-by-point construction of the super-resolution image. This approach was called pointillism, an analogy to the painting technique, and is used by PALM and STORM techniques [22, 23]. The photoactivatable variant of GFP (pa-GFP) was used in the original PALM experiments, whereas photo-switchable dye-pairs (i.e. Cy3-Cy5), or proteins (i.e. EosFP), were used in STORM. The use of a single photo-switchable dye with long triplet state in place of a dye pair has been developed and is called direct STORM (dSTORM). Images routinely taken with PALM and STORM techniques can have resolutions of 20–30 nm in the lateral axis.
Multi-color (i.e. two or more fluorescent molecules) SRM has been realized for many of the above-mentioned techniques [24, 25, 26]. Improved axial resolution has been accomplished for PALM through the implementation of double plane, or biplane (BP) detection, hence the name BP-PALM . The insertion of an astigmatic lens leads to axial resolution improvement for STORM because it allows for precise 3D localization of the molecules, and this technique has been called 3D STORM . Importantly, the performance of pointillism techniques (2D or 3D versions) is highly dependent on the density of fluorophore labeling and the nature of the biological structure. For example, imaging of smaller and filamentous objects tends to work better than densely packed and asymmetric objects  .
The use of two or more opposing objectives can be used with conventional or super-resolution microscopes to improve axial resolution. For example, in 4Pi microscopy two precisely aligned objectives are used to illuminate and detect the sample form opposite sides. This allows for coherent superposition of light onto the detector and thus constructive interference that reduces the axial resolution to 100 nm or less . This type of interferometric microscope configuration has been applied to STED (called isoSTED) , SSIM (called I5S) , and PALM (called iPALM) . These technically demanding techniques require precise piezoelectric controlled alignment, careful specimen preparation, and as such are employed usually by microscopy specialists.
The type of SRM employed depends on proper matching of the experimental requirements with the technical capabilities of the SR technique. For example, PALM and STORM may not be suitable for live cell imaging of embryos during development due to photo-toxicity from the large number of images required, but SSIM, or 4Pi microscopy, is more suitable. Both SSIM and STED techniques do not require extensive image processing (unlike PALM and STORM), and what you see is what you get. Currently, there is great interest in development of dyes/proteins that will allow long-term (i.e. hours) SRM experiments in live samples.
4. Review of fluorescence fluctuation techniques (FFTs)
Fluorescence fluctuation techniques (FFTs) are a group of spectroscopic/microscopic approaches that extract information about the dynamics (e.g. size, diffusion rate, binding partners) of fluorescently-labeled molecules from the small variations encoded in the emitted fluorescence signal (See Figure 1). This section provides an overview of FFTs, but for an in-depth explanation of the theory of the techniques the reader is directed to these reviews [33, 34].
The original FFT called fluorescence correlation spectroscopy was developed by Magde, Elson, and Webb in the 1970s, and measurements were initially performed in cuvettes in vitro [35, 36]. Cellular measurements were not possible until the invention of the confocal microscope in the 1990s. The reason for this is explained below. FCS measurements work on the principle that fluorescent molecules diffuse by Brownian or directed motion through the microscope illumination volume leading to changes, or fluctuation in the signal being recorded. Plotting the autocorrelation function G(τ) of this signal versus time leads to the FCS auto-correlation curve. The time at which there is half-maximal decay of the FCS curve is the diffusion time of the molecule. The diffusion rate can be calculated once the waist dimension of the illuminated volume is determined . The inverse of the y-axis (G(τ)) on the FCS curve corresponds to the average number of molecules diffusing through the microscope illuminated volume. The fluorescent molecule concentration must be nanomolar to micromolar to prevent averaging out of fluctuations; otherwise the FCS technique will not work. The detection volume of wide-field fluorescence microscopes is incompatible with FCS because the light from hundreds of molecules is registered, but a confocal microscope has a very small detection volume (~ 1 fL) due to the adjustable pinhole, thus allowing the measurement of small numbers of molecules. Very sensitive detectors such as avalanche photodiodes (APDs, discussed below) are needed to record the signals from these few molecules.
The FCS technique can be applied to two different fluorescently-labeled molecules and this is called fluorescence cross correlation spectroscopy (FCCS). In FCCS, the auto-correlation curves from two different fluorescently labeled molecules are compared to each other, cross-correlation curve, to determine if the molecules are diffusing in a complex [37, 38]. FCS and FCCS techniques perform well if the molecules investigated undergo isotropic diffusion (diffusion rate is constant), but they are not well suited for molecules with anisotropic behavior (diffusion varies in time and space). The approach of raster image correlation spectroscopy (RICS) is better suited for the measurement of anisotropic dynamics and can measure large regions of a cell compared to spot measurements (FCS and FCCS) that are fixed in one location [39, 40].
In RICS, the light source is scanned across the sample allowing the measurement of multiple adjacent and nonadjacent volumes thus providing a more complete description of the diffusion dynamics of the molecules . The appropriate scan speed and pixel size are critical for RICS to be successful . If the scan speed is too fast compared to the molecule then the molecule will “appear” to be immobile. If the scan speed is too slow then the opposite problem emerges and the molecule will diffuse away before being registered by the detector. For example, a scan speed of 25 μs per pixel and pixel size of ~0.05 μm are usually sufficient to measure the diffusion of a 25 kDa protein by RICS .
As stated earlier, the average number of molecules can be determined from the amplitude of the FCS curve (Go ≈ 1/N), and changes in the amplitude of the curve have been used to infer protein-ligand binding and protein–protein dimerization . For example, as the protein dimerizes the concentration of individual monomers decreases, leading to a doubling of the amplitude. This relationship holds true if the concentrations of the different species are constant throughout the measurement. This type of analysis where intensity (1st moment) and variance (2nd moment) are used to determine protein size and number from FCS data is called moment analysis [44, 45]. This type of analysis breaks down in complex systems where there are multiple species of varying concentrations that are common in biological specimens [46, 47]. However, photon counting histogram (PCH) analysis does not suffer the same drawbacks as moment analysis. In PCH analysis, the fluorescence signal, recorded as photon counts, is separated into different bins and plotted as a function of frequency to generate a histogram plot. The amplitude and shape of the histogram plot is influenced by the microscope PSF, detector noise, and fluctuation in molecule numbers. Careful fitting of the histogram returns the first and second moments for the entire photon distribution. This allows measurements of complex samples and determination of the counts per second molecule (cpsm) or molecular brightness. The molecular brightness can provide information on the size of molecules with proper calibration .
An imaging analogue to PCH analysis is called number and brightness (N&B) analysis. In N&B analysis the average molecular brightness and number of fluorescent molecules can be calculated from individual pixels of a raster-scanned image using moment analysis . N&B analysis has been used to study the assembly/disassembly of focal adhesion complexes in cells, the formation of protein oligomers during vesicular transport, and many other cellular processes [48, 49, 50, 51, 52]. Like RICS, an appropriate scan speed (pixel dwell time) is required to “capture” the fluctuations of the molecules. Analogue detectors can be used for N&B acquisition but photon counting detectors are highly recommended because no adjustment is needed for digital-to-analogue conversion.
Importantly, the brightness values obtained by these FFTs are dependent on the illumination intensity used and the detector settings. For example, doubling the laser power used to excite the molecules will lead to a doubling of molecular brightness, assuming all other factors are equal. Therefore, it is imperative that all measurements be performed with the same settings for proper comparison of specimens. The molecular stoichiometries obtained by FFTs are many times minimum estimates because unlabelled molecules are mixed with labeled molecules in the biological specimen. Ideally, all molecules under investigation should be labeled in the specimen. New genetic approaches such as CRISPR make it possible to replace endogenous proteins with fluorescently-tagged versions in the genome of the cell or organism . Finally, complete characterization of the detectors used for data acquisition is required to account for detector artifacts, such as after-pulsing and dead-time, during FFT analyses.
5. Detector technologies used in SRM & FFTs
5.1. Brief history of photodetectors
The first detectors for microscopes were the human eye as seen in the 17th century handwritten drawings of van Leeuwenhoek’s animalcules . In the 1800s, light sensitive silver halide emulsions on either copper plates or nitrate film were used to record images. Photographic film dominated many aspects of microscopic imaging until the mid-20th century. Photodetectors are devices that record changes in light intensities (photons) and then create an electrical or optical output. Modern photodetector technologies began with the invention of photomultiplier tubes (PMTs) in the 1930s [55, 56, 57] followed by the invention of avalanche photodiodes (APDs) in the 1960s . Advances in semiconductor materials and integrated circuit technologies led to creation of focal plane array (FPA) detectors, such as charge-coupled devices (CCDs) in the 1970s, and to the present day scientific grade complementary metal oxide semiconductors (sCMOS) [59, 60, 61, 62]. Different detector technologies have advantages/disadvantages, and there are several criteria used to evaluate them such as quantum efficiency (QE), readout noise, dark current levels, and SNR (Table 1).
Typical characteristics of detectors used in single molecule fluorescence experiments.
not commercially available.
kHz across total surface.
after 30 minutes’ storage in dark & supply voltage 1 x 10^6.
This section provides an overview of basic photodetector technologies with an emphasis on detectors commonly used in SRM & FFTs (e.g. APDs, CCD cameras, sCMOS). For an in-depth technical explanation of photodetectors the reader is referred to the textbook Optical Systems Engineering by Kasunic .
5.2. Photomultiplier tubes (PMTs)
The working principle of PMT is based on the photoelectric effect. PMTs are composed of vacuum tubes consisting of a cathode, multiple dynodes, and an anode (Figure 2). Incident photons are absorbed by the cathode ejecting primary electrons (~3 eV) that are accelerated by an electrostatic field toward a series of dynodes. Additional numbers of electrons are ejected (5–10 electrons) because each subsequent dynode is held at a more positive voltage potential (~100 eV), leading to an amplification of the signal. The electron current is then detected by an external electrical circuit. PMTs usually have 10–14 dynodes with a cathode-to-anode voltage gap of ~1 kV and current gain of 106 to 108. The composition of the photocathode determines the PMTs spectral response, quantum efficiency (QE), and dark current levels. Cathodes made of multi-alkali semiconductor material (e.g. GaAs, or GaAsP) have maximal sensitivity at 400 nm, providing good spectral response in the UV and blue regions but a rapid drop-off in response in the green and red region of the visible spectrum. PMTS have large gain and low noise but suffer from low QE (20–40%) compared to other technologies (e.g. APDs). Importantly, single photons can be detected with PMTs, but discrimination of single versus multiple photons is difficult. Avalanche photodiodes have better QE and better sensitivity in the green and red region of the visible spectrum compared to PMTs .
5.3. Avalanche photodiodes (APDs)
Avalanche photodiodes (APDs) can be thought of as solid-state versions of PMTs with the exception that there is no photocathode and thus utilize primary photoelectrons more efficiently than PMTs. APDs are composed of three semiconductor layers called p-layer, i-layer, and n-layer (a.k.a p-i-n, Figure 2). The n-layer has extra electrons: whereas the p-layer is electron poor and has “holes”. APDs are like p-i-n photodetectors except there is an extra p-layer between the i-layer and n-layer to create a p-i-p-n orientation where the second p-layer is much thicker than the other layers. A negative voltage (reverse bias) is applied across the junction, and absorption of a photon creates an electron–hole pair that is accelerated through the thicker p-layer (acts as gain layer) where conductive electrons impact non-conductive electrons thus making them conductive. This impact ionization can cascade as occurs with PMTs, leading to a large amplification of electrons called an avalanche. Average gain for APDs is 100 times, which is too low for single-photon detection, but operation of the APD in Geiger mode above breakdown voltage allows for single photon avalanche detection (SPAD) [65, 66]. The i-layer in APDs allows for better photon absorption, shorter photoelectron diffusion time, lower capacitance, and faster response. Researchers have wanted the sensitivity of APDs combined with the gain and dynamic range of PMTs. This motivation led to the development of hybrid photodetectors.
5.4. Hybrid photodetectors
Hybrid photodetectors are a combination of PMT-APD technologies and were developed in the 1990s for high energy physics experiments [67, 68]. Hybrid detectors have a photocathode, electron multiplication component, and output terminal housed inside a vacuum tube. The difference from PMTs is the electron multiplication is performed by a silicon avalanche diode (AD), instead of dynodes. The silicon diode contains a thin p-layer facing the photocathode followed by a thicker middle silicon layer and finally a p-n junction that is attached to the bias terminal. Photoelectrons ejected from the cathode are accelerated toward the AD by a very large voltage difference compared to PMTs (~8 kV). The electrons are multiplied in the AD first through electron bombardment and then by avalanche gain. Total gain can be greater than 100 times that is considerably lower than PMTs, but hybrid detectors have other benefits that make-up for the low gain. Hybrid detectors have better SNR compared to PMTs because the first gain step can be up to 1000 times (one photoelectron yields 1000 secondary electrons). This higher SNR allows for discrimination between one photon and multiple photons. The response time is faster for hybrid detectors compared to PMTs, and there is virtually no after-pulsing (false detection of photon). Hybrid detectors are well suited for fluorescence applications where after-pulsing can cause artifacts such as in time correlated single photon counting (TCSPC) and FCS. Hybrid detectors outperform SPADs and PMTs for FCS and other single-molecule fluorescence experiments . Unfortunately, these point-like detectors discussed above are inefficient for imaging large regions. However, arrays of photodetectors are inherently suited to large field imaging. One popular array photodetector is the charged coupled device (CCD).
5.5. Charged coupled devices (CCDs)
Charge-coupled device (CCD) cameras have completely replaced photographic film cameras for scientific experiments, and are routinely used for microscopic imaging. A CCD is an array of photosensitive elements attached to a silica semiconductor substrate [62, 70]. Each element is composed of a metal oxide semiconductor (MOS) capacitor consisting of surface gate electrode, aluminum or polysilicon, deposited on top of charge carrying channels that are insulated by silicon dioxide (Figure 2). Incident photons “strike” individual MOS elements generating electron–hole pairs, leading to an accumulation of charge in the potential well below the MOS. Application of an external voltage controls the movement and release of the built-up charge in each photosensitive element or pixel. The architecture of the electrodes required for the charge readout acts as a shift register for charge transfer, and there are several different register types employed in CCDs [63, 71]. The three main CCD types are full-frame, frame-transfer, and interline transfer architectures. Full-frame CCDs utilize the entire sensor array for light collection providing maximum efficiency. However, a mechanical shutter is needed to stop exposure and allow transfer of the built-up charge limiting image acquisition rates. Frame-transfer CCDs have half of the photosensitive surface covered by an opaque mask that acts as a photoelectron storage space. Charge is transferred to the masked area allowing the next exposure to commence while the first is being processed. This setup allows for faster frame rates but half the sensor is not available for image acquisition meaning a larger chip is needed compared to a full-frame, thus adding to cost.
Interline CCDs have alternating rows of pixels that are masked with an opaque material (e.g. aluminum) thus allowing acquisition and charge transfer to occur simultaneously. The charge in the unmasked rows is transferred to the masked row allowing for a second round of exposure during readout of the previous first exposure. This dramatically speeds up acquisition rates at the expense of reduced sensor surface. The addition of micro-lenses to the interline CCD design focuses more light onto the pixels increasing efficiency, from 50–75%, of collected light [72, 73]. A second added benefit of using micro-lenses is that it extends the spectral sensitivity of the CCD into the blue and UV light range that is ideal for imaging with GFP. One of the most effective strategies to increase QE is back illumination of the sensor where the wiring is behind the photocathode layer leading to less light scattering and up to 90% QE .
For all three architectures (full-frame, frame-rate, interline) the charge readout is fed into a CCD output amplifier, and then an analogue-to-digital converter (ADC). The stored charge in each sensor pixel is linearly proportional to the light absorbed up to the full well capacity (FWC). The FWC determines the maximum signal a pixel can record and is a major factor affecting a CCDs dynamic range. Traditionally, CCDs were composed of square sensor pixels arranged in a rectangular pattern with a 4:3 aspect ratio. Common CCD image sensors range in size from 6 to 16 mm (diagonal measurement). Many current scientific grade CCD cameras employ square sensor arrays to better match the microscope field of view (Table 2). Addition of an electron multiplication register between the shift register and output amplifier can increase the signal from the image sensor, and are called electron multiplication charge-coupled device (EMCCD). This modification of the CCD improves the SNR and increases the QE to 95%, or greater in most cases (Figure 2). EMCCD cameras have replaced CCD cameras for many imaging applications, including SRM (Table 3). Drawbacks to the electron multiplication are gain instability, performance decay with age, and potentially increased dark current . Finally, an intensifying screen can be put in front of the CCD sensor (ICCD) to increase sensitivity to single-photon detection . The intensifying screen is composed of a photocathode, micro-channel plate (MCP), and a phosphor screen (Figure 2). The photons strike the photocathode generating photoelectrons that are amplified by the MCP, a plate with angled tubes that creates a “shower” of electrons like dynodes in PMT. Secondary electrons from the MCP strike the phosphor screen creating photons that are read by the CCD sensor. Importantly, ICCD sensors provide gate-ability (100’s of picosecond temporal resolution) in addition to enhanced sensitivity over EMCCDs . A drawback to ICCD and EMCCD cameras are their cost (~$30,000–40,000), but advances in semi-conductor fabrication have led to smaller and more cost-effective photodetectors, such as complementary metal oxide semiconductors (CMOS).
5.6. Complementary metal oxide semiconductors (CMOS)
CMOS sensors, like CCDs, are arrays of photosensitive pixels, but are smaller in size due to advanced manufacturing techniques . Specifically, the readout and processing circuitry are miniaturized, and incorporated into each pixel creating an “active-pixel sensor” (APS). The miniaturized transistors are fabricated using a complementary MOS (CMOS) technology (Figure 2). This miniaturization comes with a price, most notably higher noise compared to CCDs and the APS takes up pixel area affecting light absorption. An advantage of CMOS is that each pixel can be read out randomly and no electrons are lost by charge transfer across a row. CMOS chips consume less power and are more suitable for low-price products like cell phone cameras. The lower performance of early generation CMOS sensors, compared to CCDs, prohibited their use for scientific applications. Recently, higher performance CMOS sensors have been fabricated and are called scientific grade CMOS (sCMOS) . These new sCMOS sensors were introduced in 2010 and were marketed as low noise, high QE (55–70%), and fast frame rate (>100 fps) cameras (Table 4). There is improved image resolution due to the smaller pixel size of sCMOS sensors compared to EMCCDs. However, distortions can occur in images due to rolling shutters that are used (i.e. each row captured at different time). The sCMOS camera noise is not Gaussian distributed and the improved resolution is at the expense of decreased sensitivity compared to EMCCDs . Which camera technology (EMCCD, or sCMOS) is best suited for low-light microscopy experiments?
A 2012 application note by the camera manufacturer ANDOR found no performance difference between the tested EMCCD (ANDOR iXon3) and sCMOS (ANDOR Neo) cameras using a spinning disk confocal setup where the emission light from the sample was equally split between the two cameras . However, the authors are quite right to raise the important caveat that the samples imaged contained near-perfect labeling, and were fluorescently very bright and stable. These artificial conditions are near ideal and do not reflect the typical 3D spinning disk imaging experiments. Typical signals are much lower and on the order of 0–20 photons/pixel . The authors agree with other researchers in the field that there exists a SNR cross-over point where the sCMOS will outperform the EMCCD . In this experiment, the cross-over point was somewhere between 40 and 100 photons/pixel. A second study found the cross-over point to be greater than 4 photons/pixel or 180 photons/pixel when comparing the ANDOR iXon DU897BV to the Hamamatsu ORCA-Flash 4.0, or ORCA-Flash 2.8 sCMOS cameras, respectively . Image artifacts (streaking lines) were seen at specific illumination intensities for the sCMOS but not the EMCCD cameras.
The brightness of the fluorescent sample is highly dependent on the microscope setup. Therefore, the SNR cross-over point must be determined empirically for each experimental situation (typical ranges observed are ~4–200 photon/pixel). EMCCD cameras have inherent excess noise due to the amplification process that contributes to about 50% of the total noise. This prevents manufacture of a shot-noise limited EMCCD detector . In contrast, sCMOS camera performance (increased QE and reduced noise) can be improved through hardware and software optimization . It is predicted that sCMOS would be the camera of choice when greater than 50 photons/pixel is reached . Currently, EMCCDs are better suited to measure small fluorescence changes in live cells with high spatial resolution compared to sCMOS . In addition, EMCCDs superior imaging capability for low light samples outweigh the benefits of sCMOS for spinning disk confocal microscopy at this time . This hotly debated comparison between EMCCD and sCMOS camera performance is not expected to slow down soon. In fact, implementation of in-camera signal-processing algorithms are being introduced to enhance both EMCCD and sCMOS camera technologies (see Section 5.7), and could re-ignite the debate.
5.7. Next generation photodetectors
Technological advances have brought the performance of EMCCD and CMOS cameras closer to point-like detectors such as APDs and Hybrid detectors. The sensitivity and readout noise are still generally better for point-like detectors, but these types of detectors cannot “count” photon numbers unless external electronics and software are used to bin the photons (~1 nano-sec to 10 msec). Importantly, there are inherent throughput limitations for point-like detectors in contrast to array detectors. One solution around this problem is parallelization of the point-like detector (i.e. an array of point-like detectors). Factors that must be considered for parallelization include: parallel excitation, parallel detection, excitation and detection alignment, and data processing . Each light source must be sufficiently separate to prevent crosstalk during multiple-spot excitation and the spot separation must be a few diameters apart as a general rule of thumb . An eight pixel custom linear SPAD array and a 32 x 32 CMOS SPAD array were used recently to perform parallel FCS measurements on a fluorescent dye in solution with quasi-diffraction limited spots [88, 89, 90]. Custom liquid crystal light modulators, or micro-lens are required to direct and separate the multiple PSFs. These results with highly-parallel arrays are encouraging but these parallelized detectors have lower sensitivity and larger dark counts thus leading to higher background counts for FCS measurements compared to individual detectors . Recently, a frame summing/filtering scheme called “smart-aggregation” was introduced to increase SPAD array performance . This approach promises to “push” SPAD camera performance beyond EMCCD and CMOS cameras, but significant technological advances are still required.
Bright photo-stable dyes that emit in the red and near-infrared range of the visible spectrum are commonly used in imaging studies involving animals. This is due to their favorable excitation using two-photon light sources and reduced scattering of emitted light in thick animal tissues . The wide use of these dyes has led to the development of red enhanced SPAD (RE-SPAD). Traditional SPADs have a QE of 15% at 800 nm wavelengths but newer RE-SPADs have a QE of 40–60% [93, 94]. Recently, a parallelization of RE-SPADs has been fabricated and tested . A new type of SPAD architecture has been implemented to enhance the electrical isolation between individual SPAD elements to reduce crosstalk . These RE-SPAD array detectors are still in their infancy and further characterization of detector parameters (temporal resolution, dark count rate, after-pulsing, etc.) is necessary before commercialization and mass production are a reality.
SPADs have sub-nanosecond time resolution but are inefficient imaging detectors because scanning is required for image formation. In contrast, wide-field detectors that have pico-second response, such as time-gated ICCDs, are photon-inefficient due to the lens coupling of the intensifier screen to the CCD sensor. Recently, several research groups have developed single-photon wide-field detectors with high temporal and spatial resolutions thus attempting to combine the best attributes of both SPADs and ICCDs [86, 90, 95]. One such detector called H33D (pronounced “HEED”) is composed of S20 multi-alkali cathode, triple MCP stack, and cross-delay line anode . A front-end field programmable gate array (FGPA) is used for time-stamping and synchronization of photon events. The H33D detector has 18% QE at 400 nm diminishing to 3% at 630 nm. Temporal resolution was 100 picosecond FWHM using a red diode laser. Fluorescence lifetimes of dyes in solution, colloidal quantum dots and quantum dot labeled receptors on the surface of cells have been measured using this high temporal and spatial single photon detector [95, 96, 97]. Several other groups have fabricated large-area photon counting detectors with a similar architecture .
There are multiple sources of noise (dark, read, shot) that affect the SNR collected by the detector. Cooling of the detector sensor can reduce dark noise and read noise can be minimized through thoughtful electronic design and sensor performance optimization. Photon shot noise is an intrinsic property of light and has a Poisson statistical distribution. Shot noise varies with the square root of the signal (shot noise = √signal) and thus increasing light levels leads to improving SNR. However, the low light levels encountered with SRM and FFTs measurements can lead to shot noise dominating the signal. Increased exposure time, frame averaging, and increased excitation light intensity have been used to circumvent this low SNR problem. A fourth method that is more compatible with SRM is the use of de-noising algorithms to dynamically analysis the acquired image and filter out the noise. Spatial and temporal filtering of images and video has been extensively studied [99, 100, 101, 102, 103, 104]. One very popular spatial filter is the averaging-based non-local means (NLM) filter . It is important to choose an appropriate algorithm and parameter settings that do not introduce artifacts (e.g. aliasing, blurring or ringing).
Manufacturers are starting to add real-time de-noising algorithms to the firmware of their cameras for enhanced SNR and improved performance. The Prime™ family of cameras from Photometrics employs an algorithm developed at INRIA (NLM with patch based refinement) and optimized at the Curie Institute . This algorithm called Prime-Enhance allows up to an 8-fold decreased exposure time while retaining a high SNR (due to reduction of photon shot noise effects) thus leading to reduced photo-toxicity in live cell experiments [101, 105]. The Prime-Enhance algorithm is also purported to not introduce common processing artifacts such as aliasing, blurring or ringing.
6. Examples of photodetector used in SRM & FFTs
A popular photodetector for probe-based SRM is EMCCD cameras. These camera-types have been used to study membrane protein dynamics in plant cells, assembly of HIV virus particles, and viral protein receptor interactions just to name a few applications [106, 107, 108, 109]. New sCMOS cameras are inherently faster than EMCCDs, allow high-throughput capabilities, and are starting to be used for some SRM applications [83, 98]. Currently, EMCCD cameras are employed for rapid high resolution live cell imaging . While, sCMOS cameras are employed for slower cellular dynamic studies, or fixed cell super-resolution imaging . For fluorescence fluctuation studies, APD and hybrid detectors are commonly employed because of their sensitivity, efficiency, and faster response compared to PMTs. Usually APDs are used for FCS and PCH measurements of fluorescently-labeled molecules in tissue culture cells. For example, oligomerization (protein–protein association) was shown to be important for the trafficking of a membrane receptor (p75) to the apical surface of epithelial cells . These studies used APDs to make FCS, PCH, and N&B measurements of wildtype and mutant receptors. The wildtype receptor formed higher-order oligomers in the Golgi membrane and dimers at the plasma membrane (Figure 3). In contrast, the mutant proteins that could not traffic to the apical membrane did not form higher-order oligomers at the Golgi (Figure 3).
Hybrid detectors have been employed to measure the formation of lipid rafts in a cell model of Fabry’s disease . In Fabry’s disease, lysosomal function is disrupted due to reduced activity of a specific enzyme (alpha-galactosidase A) that leads to accumulation of neutral glycosphingolipids such as globotriaosylceramide (Gb3). N&B analysis was performed on wildtype and alpha-galactosidase deficient cells, which act as a model for Fabry’s disease. Antibody-induced clustering of a model lipid raft protein was increased in the mutant compared to control cells . These results suggested that accumulated Gb3 may alter lipid raft protein interactions in membranes of alpha-galactosidase deficient cells. These two examples are just a snippet of the many experiments that have employed a variety of low-light photodetectors.
7. Summary and concluding remarks
In the past seventy-five years, advances in micro-circuitry and semiconductor materials have led to dramatic advances in photodetector QE, sensitivity, and resolution. EMCCD and sCMOS cameras are the detectors of choice for probe-based SRM. In contrast, APD and hybrid detectors are becoming more common for use in PSF-engineered SRM and FFTs. Manufacturing attempts to combine the best attributes of point-like detectors with array detectors (CMOS and SPAD arrays) has been met with moderate success. These parallelized photodetectors are still in their early stages and are not routinely used. Shot noise and read noise are a problem especially as pixel densities have increased in array detectors. De-noising algorithms are being used to combat shot noise and increase SNR for low-light applications. Finally, A detector suitable for SRM and single molecule fluorescence experiments must have high sensitivity, high temporal resolution, and low readout noise. The researcher should compare the SNR, dark count, read noise, and frame rate to determine which detector type best fits their experimental needs. No single photodetector technology is suitable for all techniques and some researchers choose to outfit their microscopes with multiple camera technologies (e.g. EMCCD and sCMOS) to allow greatest flexibility when imaging. Advances in hardware and software promises to enhance detector technologies and push the boundaries of single-molecule detection even further in the coming years.
I apologize to authors whose research I was unable to cite due to page constraints. My sincere thanks to Dr. Greg Adkison and Dr. Haibing Teng for proof-reading the manuscript. The author is grateful for start-up funds from the Department of Biology and the College of Arts & Sciences Dean’s Office at Western Carolina University.
Robert T. Youker (December 20th 2017). Detectors for Super-Resolution & Single-Molecule Fluorescence Microscopies, Photon Counting - Fundamentals and Applications, Nikolay Britun and Anton Nikiforov, IntechOpen, DOI: 10.5772/intechopen.71943. Available from:
Introductory Chapter: Plasma Chemistry for Better CO2 Conversion
By Nikolay Britun and Tiago Silva
We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.