Four different lenses with their calculated beam sizes, as well as axial depth range and lateral FOV.
We report that high lateral resolution and high image quality optical coherence tomography (OCT) imaging can be achieved by the multi-frame superresolution technique. With serial sets of slightly lateral shifted low resolution C-scans, our multi-frame superresolution processing of these special sets at each depth layer can reconstruct a higher resolution and quality lateral image. Layer by layer repeat processing yields an overall high lateral resolution and quality 3D image. In theory, the superresolution with a subsequent deconvolution processing could break the diffraction limit as well as suppress the background noise. In experiment, about three times lateral resolution improvement has been verified from 24.8 to 7.81 μm and from 7.81 to 2.19 μm with the sample arm optics of 0.015 and 0.05 numerical apertures, respectively, as well as the image quality doubling in dB unit. The improved lateral resolution for 3D imaging of microstructures has been observed. We also demonstrated that the improved lateral resolution and image quality could further help various machine vision algorithms sensitive to resolution and noise. In combination with our previous work, an ultra-wide field-of-view and high resolution OCT has been implemented for static non-medical applications. For in vivo 3D OCT imaging, high quality 3D subsurface live fingerprint images have been obtained within a short scan time, showing beautiful and clear distribution of eccrine sweat glands and internal fingerprint layer, overcoming traditional 2D fingerprint reader and benefiting important biometric security applications.
- optical coherence tomography
- lateral resolution
- 3D imaging
- fingerprint identification
Optical coherence tomography (OCT) [1, 2] is an advanced non-contact 3D imaging technique, providing subsurface cross-sectional tomographic images. It offers deeper penetration depth  and larger scan area  than confocal microscope imaging  as well as higher resolution  than ultrasonic imaging . It is thus widely utilized in 3D imaging of eyes [7, 8], skins [9, 10, 11, 12], blood vessels , cartilages , and numerous biomedical applications.
With non-contact and non-invasive advantages, OCT has significant medical applications. There is also a huge potential of OCT for many non-biomedical applications that demands non-destructive testing and evaluations in micron scale resolutions . For example, there is no preparatory steps for OCT sample imaging, instead of gold coating for SEM imaging; no coupling media as required for ultrasound imaging; no special safety precautions like X-ray. Also, the near infrared light source in OCT usually has no photo reactions with most materials, very safe for quality testing of damage in silica , glass-fiber reinforced polymer samples , strained polymer samples , microstructures [19, 20, 21, 22, 23], papers , oil paintings , film coatings , fastener flushness , and so on. Besides, the successful detection of embedded and hidden structures is another potential of OCT for security applications, such as 3D fingerprint identification defending against spoofing attack with fake fingerprints [28, 29, 30, 31]. However, compared with other imaging techniques such as microscopy and confocal microscopy, the low lateral resolution and high speckle noise restrict the OCT becoming a competitive imaging tool in some non-biomedical areas highly relying on
OCT imaging has two distinct resolutions namely axial resolution in the depth direction and lateral resolution in the
Adaptive optics (AO), an astronomical telescope technique, has been adopted in OCT systems to correct aberration wave front and thus improves the lateral resolution . Except the high cost and a very limited FOV (maximum 1 × 1 mm2) [40, 41], AO technique in principle is to recover the original lateral resolution of OCT, which however is blurred by human eyes. Thus, it is not suitable for non-ophthalmic imaging like skins due to scattering blurring. A virtually structured detection (VSD) method  was reported to improve the lateral resolution by adding an electro-optic phase modulator (EOPM) in the reference arm. The EOPM shifts the light phase with multiple of π/2, and then the VSD algorithm fuses four phase shifted A-scans to one, achieving resolution doubling. It is a time consuming (taking ∼40 s for each image frame) technique which is infeasible for
Background noise is another factor degrading the resolution and image quality of OCT systems. Different from white noise, the structure related speckle noise in OCT imaging is difficult to be suppressed by the multi-frame averaging [48, 49]. Szkulmowski et al.  introduced an interesting averaging algorithm with multiple shifted B-scans to remove the speckle noise. However, this approach introduces new ghost patterns in
Lateral sampling rate  in scan-based OCT imaging is termed the scan matrix density. According to Nyquist-Shannon sampling theorem, the sampling frequency should double the sample frequency at least. Although increasing the scan matrix density could improve the lateral resolution, this method is at the expense of longer scan time and not suitable for time sensitive applications, such as
In this book chapter, we report an effective multi-frame superresolution technique to significantly improve the lateral resolution and image quality of OCT without adoption of extra hardware and higher
2. OCT system and superresolution principle
2.1 SD-OCT and lateral resolution limit
Our spectral domain optical coherence tomography (SD-OCT) (BIOptoscan OS-186, New Span Opto-Technology) is one kind of the most popular OCT systems in ophthalmic clinic applications, as schematically shown in Figure 1. It sends a broadband light from the SLD to a 2 × 2 optical fiber coupler. The SLD has a center wavelength of 860 nm and spectral bandwidth of 100 nm (IPSDW0822–0314, InPhenix). One split beam is sent to the reference arm that is focused to a mirror and then reflected back to the fiber coupler. The other split beam in the sample arm is focused to the measurement sample and laterally scanned by a pair of galvanometer scanner mirror. The scattering signals from different depth layers of the sample collected by lens are sent back to the fiber coupler to interfere with the return beam from the reference arm, generating spectral interference patterns that are imaged by the optical spectrometer for computer signal processing. Each scattering depth would result in a near sinusoidal interference pattern in the frequency domain. The final spectral image looks complex due to mixing of all interference patterns with different periods from all sample depth layers. A fast Fourier transform processing of the mixed interference pattern in the frequency domain can beautifully retrieve a series interface layers inside the sample within the depth range of the SD-OCT, set by the combination of sample arm optics
As discussed earlier, the focused beam spot size at full width half maximum (FWHM) of the SD-OCT imaging system is mainly limited by the
Given our collimated beam diameter of ∼3 mm, the corresponding focusing spot size and axial depth range are listed in Table 1 for a few common lenses’ focal lengths. Here, before reaching spectrometer limitation, the axial depth range is set by the axial
|Focal length (mm)||Beam spot size (μm)||Axial depth range (mm)||Lateral FOV (μm)|
|19||4.03||0.31||500 × 500|
|30||6.36||0.78||1400 × 1400|
|100||21.21||8.64*(2.86)||12,000 × 12,000|
Here, is the focal length of the sample arm lens and depends on both the radius of the lens and the acceptable maximum off-axis scanning angle of the galvanometer scanners.
Typically, the acceptable maximum galvanometer scanning angle could be quite large. However, due to off-axis focused beam aberration and Petzval field curvature , the FOV is quite limited. The displacement of an image point at height on the Petzval surface from the paraxial image plane is given by :
Here, and are the indices and focal lengths of the thin lenses forming the system. This equation implies that the Petzval surface is an unaltered value by changes in position or shapes of lenses and stops, but inversely proportional to the focal lengths. Detailed illustration and explanation could refer to our previous work . Usually the field curvature from high
In order to quantify the influence of the field curvature to the focusing performance, we simulate the off-axis focusing degradation of three common lenses (Thorlabs, AC254-030-B, AC127-019-B, and AC254-100-B) by ZEMAX software, with two of them shown in Figure 2. We see that the focal spot size of 30 mm focal length lens remains almost unchanged when off-axis distance is less than 300 μm. With off-axis distance larger than 800 μm, the focusing degradation becomes obvious. At off-axis distance of 1000 μm, the diameter of focal spot size (measured at FWHM of the peak) is ∼16% larger than that at the center position. The increased off-axis focused beam spot size would significantly degrade the OCT lateral resolution. With 1400 × 1400 μm2 areal scan imaging (700 μm off-axis distance), the lateral resolution at the image edge is considered acceptable.
For 19 mm focal length lens (Thorlabs, AC127-019-B) simulated in Figure 2(B), a 500 μm off-axis distance would lead to ∼39% larger in focal spot size. Thus, the optimized single C-scan FOV has to be limited to 500 × 500 μm2 area (250 μm off-axis distance), only one eighth of the 30 mm focal length lens. For 100 mm focal length lens (AC254–100-B), the image quality is usually acceptable in a 1.2 × 1.2 cm2  (not shown here), but losing lateral resolution due to large spot size as we discussed above. In practice, unavoidable spherical and coma aberrations would further degrade the image quality.
Clearly, smaller focused beam spot size would improve the lateral resolution but at the expense of the
2.2 Improving lateral resolution by high density scanning
For a large FOV image with short scan time, the focused beam spot scan matrix is usually set to one by one without spot positional overlapping, as illustrated in Figure 3(b), like spatially separated pixel array in an image sensor. The presence of spot spacing results in under sampling and loss of spatial image features. Without demanding smaller focused beam spot size for preserving a long
In order to avoid FOV reduction, a larger C-scan matrix may be applied to the sample sacrificing the scan time as shown in Figure 3(d), which however is unacceptable for time sensitive
2.3 Improving lateral resolution by the multi-frame superresolution for
The multi-frame superresolution is an image processing technique, studying image degradation models (such as optical blur, motion effect, down-sampling, and additive noise) and then recovering the high resolution image from multiple low quality images based on the superresolution algorithm and the sub-pixel information differences among these images, overcoming the resolution limit of the hardware. Figure 5 illustrates how these effects result in a low quality image during the conventional camera image acquisition. To recover the high resolution image, a series reversed methods such as up-sampling, motion/pixel shift estimation and compensation, deconvolution, and denoising are applied.
In a SD-OCT system, the main degradations of an ideal lateral image is the optical blurring caused by the lateral PSF of OCT optics and the down-sampling due to the sparse scan matrix and the large focused beam spot size. Each isolated focused beam spot is treated as a pixel in conventional camera imaging. The motion effect can be generally ignored when imaging non-biomedical samples such as microstructures [19, 37] since both the sample arm and the samples are stable. The motion effect should be considered for imaging of
Figure 6 illustrates how to apply multi-frame superresolution technique to SD-OCT imaging. A reference 5 × 5 pixel image in one depth layer of a C-scan with no scan spot overlapping is shown in Figure 6(a). As we introduced above, each pixel represents a focused scan beam spot on the sample. To satisfy multi-frame superresolution requirement, we intentionally introduce slight differences in a series of control voltage matrix of scanners, creating a sequence of C-scans with sub-spot-spacing shifts (equivalent to sub-pixel shifts) in the -lateral plane, as illustrated in Figure 6(b)–(d). The superresolution processing of the four lateral images but with sub-spot-spacing shifts reconstructs a higher pixel resolution image of that depth layer, exhibited in Figure 6(e).
The image resolution covers two concepts: one is pixel resolution which is equivalent to dots per inch or sampling rate in conventional terminology and the other is spatial resolution which is defined as the smallest discernible detail in an image , one example as Rayleigh criterion. Figure 6 is obviously a result of lateral pixel resolution improvement leading to lateral spatial resolution improvement by the increment of sampling rate. In theory, simply reducing the step size of the sub-spot-spacing shift and increasing non-identical image sets for multi-frame superresolution processing can continuously improve the lateral pixel resolution. However, there is a spatial resolution limit owing to optical diffraction limit, system noises, stability of interference pattern and so on. In other words, when the sampling rate is high enough, further increment would not be helpful to lateral resolution improvement. Thus, finding an effective relationship among the lateral resolution improvement, the sub-spot-spacing shift, and the number of image frames would be critical to identify a desired resolution improvement without unnecessary image frames and associated excess acquisition time. Without particularly indicated, the lateral resolution improvement discussed in this book chapter represents lateral spatial resolution improvement.
Figure 6 illustrates four C-scans having multi-directional 1/2-sub-spot-spacing shifts. For easier explanation, the four shifts are simplified as four blocks in Figure 7(b), showing the shifts directions and space relative to the first non-shift C-scan as reference 0. Mathematically speaking, the three shifts should be represented as (0.5, 0), (0, 0.5), and (0.5, 0.5) in and coordinates. Compared with the traditional four-frame shift strategy in Figure 7(b), we experimentally found that more shifts (gray ones) in Figure 7(c) and (d) in addition to red shifts lead to better image quality. The gray shifts in Figure 7 provide more information for superresolution processing, suppressing background noises in OCT imaging. By using 1/4-spot-spacing shift as in Figure 7(d) red points, the superresolution technique can improve the lateral pixel resolution by 16 times in principle. Similarly, a series 1/8-spot-spacing shift C-scans (not shown) can improve the lateral pixel resolution by 64 times. Simplifying the shift strategy introduced later in the chapter, we name the Figure 7(c) as 1/2-spot-spacing shift step and maximum 1/2-spot-shift, and Figure 7(d) as 1/4-spot-spacing shift step and maximum 3/4-spot-shift, and so on.
Considering an ideal high quality lateral image degraded by a pure translational motion with space invariant blur and additional noise as , one of the acquired low resolution lateral image at a selected depth layer in a C-scan is modeled as
Here, is the motion operator due to sub-spot-spacing shifts among multiple C-scans discussed above. is the PSF of the sample arm optics, blurring the image. is the convolution operator. is the discretizing down-sampling operator due to the sparse scan matrix and finite spot size.
According to the image degradation model in Eq. (5), we can recover the high resolution image with a series slight shifted ’s by mathematical processing. Generally speaking, the recovering processing is minimizing the errors between the model and all the measurement values. We estimate the approximate high resolution image in a minimum norm problem  as
Here, is the th input low resolution image. is the motion operator for the th low resolution image. is the down-sample operator which can be simply determined as , or by how many times the sampling rate improvement (such as 8, 4 or 2 times) and the total input frame number captured. is the optical blur operator or PSF. The noise is an additive term and can be suppressed by multi-frame superresolution processing, which thus is not included in Eq. (6). Besides, we define as the image convolved with a PSF, due to the complexity of the deconvolution problem in OCT imaging system. We would solve the deconvolution problem later . Rewriting Eq. (6)  we have
Eq. (7) is a minimization of norm problem that can be separated into two steps: reconstruct a non-deconvolved high resolution image from a series of low resolution image frames and then find a proper PSF to eliminate optical blur and recover the expected image from .
If , it is a norm problem, or a least-absolute problem. If , it is a norm problem, or a least-square problem. norm is robust to outliers but may penalize the high frequency signals. In most OCT applications presented in this chapter, we notice that the background noises are usually temporal noise along with structure related speckle noise without significant outliers. Both the temporal noise and the speckle noise can be suppressed by processing with adjacent pixels  and the average of multiple lateral images. Therefore, we applied a kind of norm called normalized convolution (NC) algorithms introduced by Knutsson et al.  and Pham et al.  to process the designated shifted images in Figure 7 to improve the lateral resolution of our SD-OCT system.
We select the NC algorithm [56, 57] instead of other steepest descent algorithms because it considers the relation of a center pixel with neighborhood encompassing N pixels (for example, the radius of 4 pixels). And the final value of each output pixel is optimally solved  by adjacent ones, effectively reduce the structure related speckle noise. In experiment section, through shifted C-scans and the NC algorithm, we demonstrated that our superresolution technique can significantly reduce the background noises in final lateral and 3D images. Besides, due to the shift compensation for all low resolution frames, our method avoids ghost patterns observed in output images. Additionally, this kind interpolated method has good tolerance to the incomplete input frames lack of some shifts. For example, even lack of I3 in Figure 6, we still can estimate the output image according the neighborhood pixels in incomplete input images.
After the interpolation algorithm, the next step is to find a proper PSF to recover the expected image from . There are numerous reports on various deconvolution methods to improve OCT image resolution [45, 46, 47]. Lucy-Richardson deconvolution [47, 59, 60] with a proper Gaussian PSF appears to be a widely accepted solution for recovering blurred images,
where is the estimate of the undistorted image in th iteration. The deconvolution process starts with . The original input image is obtained from Eq. (7). is the lateral PSF of the system. The Gaussian PSF is a common selection [45, 46, 47] owing to the focused beam spot lateral profile following a certain Gaussian distribution. However, the spot profile may not keep the circular symmetry for off-axis scanning. Considering the scattering inside a sample, the focused beam may not retain near Gaussian distribution. Thus the blind deconvolution [61, 62] might be a better solution, which uses maximum a posteriori probability (MAP) algorithm to automatically estimate the irregular PSF in the input image and then deblur it, avoiding the limitation of the regular PSF and exhibiting better performance in the final image. In this book chapter, we applied the blind deconvolution method introduced by Krishnan et al. . In theory, the resolution limit of an optical system is determined by diffraction limit , which is related to the PSF. Thus, it is possible to break the diffraction limit and further improve the spatial resolution of optical systems through deconvolution with a correct PSF. Although the deconvolved from a Gaussian or estimated PSF would show obvious resolution improvement to , these methods may lead to some ringing artifacts and reduce the output image quality. Also, the deconvolution methods are usually sensitive to the noise floor which further restricts their applications, explained later in the experiment Section 3. In this chapter, we thus focus on the first step to reconstruct a high quality image , but also provide deconvolved images for readers to compare.
2.4 Estimating the unknown shifts to improve lateral resolution by multi-frame superresolution for
The above superresolution processing is suitable for SD-OCT imaging of static samples such as microstructures where the sub-spot-spacing shifts are intendedly set. For
In the -axis, the height shifts among multiple C-scans can be corrected by some obvious features, like comparing the top positions of multiple 3D images. While in the lateral -plane, without any simple indicators, an advanced shift estimator is desired. To improve the estimation accuracy, we firstly average multiple lateral images along the -axis to enhance the contrast of key features in the -plane. Then a popular image registration algorithm—multi-modal volume registration  is applied to estimate the shifts among these averaged lateral images. According to the registration algorithm, we seek to maximize the mutual information between the reference image and test image v:
Here, is a transformation from the reference image to the test image. is the test image associated with the reference image after transformed with . We treat as a random variable over coordinate locations in and . The best transformation can be estimated by algorithms [64, 65, 66] to maximize the mutual information between and .
This is considered as the motion operator in Eq. (4) for the th low resolution
2.5 SD-OCT image acquisition and superresolution processing
The SD-OCT image acquisition is a lateral spot scanning image acquisition process where the depth tomographic information in -axis is obtained intrinsically for each scan point. Our superresolution processing is to analyze and improve the lateral resolution in -plane. Thus, we need to transfer -axis information of numerous points to multiple -plane layers. First, we perform the SD-OCT C-scan, acquiring multiple B-scan images in the direction at different (see Figure 8(a)—left). These B-scans can be arrayed in sequence to generate a 3D matrix as shown in Figure 8(a)—middle. We then retrieve a sequence of 2D -images at different depth as shown in Figure 8(a)—right, for later processing. The lateral resolution improvement is to use several -images at an identical position (see Figure 8(b)—middle) but from slightly lateral shifted C-scans (A, B, C, etc. from Figure 8(b)—left) to perform multi-frame superresolution processing, yielding a higher lateral resolution image as in Figure 8(b)—right. Repeat the process in Figure 8(b) layer by layer for all depth layers can yield a higher lateral resolution 3D image in the whole space, not shown.
3. Experiments and results
3.1 Lateral resolution, image quality, and efficiency improvement
We compare the performance of our superresolution technique with designated shifts to other traditional methods, such as high density scan and multiple frames averaging, in three aspects: lateral spatial resolution, image quality, and scan time.
Lateral spatial resolution: as we mentioned in Section 2, spatial resolution represents the ability to distinguish the smallest discernible detail in the object, such as closed line pairs, which is an important indicator to all imaging systems. A standard negative resolution targets (R3L3S1N—Negative 1951 USAF Test Target, Thorlabs), as partly shown in Figure 9, is used to evaluate the resolution improvement. This resolution target provides 10 groups (−2 to +7) with 6 elements per group, offering a highest resolution of 2.19 μm. Considering our beam spot size in Table 1, group 4–5 and 6–7 are suitable for resolution testing of our OCT system with 100 and 30 mm focal length lenses, respectively. The resolution (the gap between two lines, the same as the width of 1 line) of group 4–7 is listed in Table 2.
The resolution target is with a negative clear tone glass pattern. The chrome area appears dark because of blocking the backlight illumination while the transparent patterns are bright. Usually the SD-OCT system is more sensitive to reflectivity enhancement than reduction, and thus a resolution target with sudden reflection reduction is better for judging the resolution limit of the system. Successfully imaging and distinguishing these fine patterns is an effective way to demonstrate both the high lateral resolution and high sensitivity of our technique.
Here, is the standard deviation (STD) of the background noise. Higher PSNR means higher image quality and lower noise. Usually, an acceptable image quality should be with PSNR >20 dB. The DR is defined as :
Here, is root mean square (RMS) of dark noise. Higher DR means we can distinguish more details in both dark and bright areas of an image. For an OCT system, we expect to extract more information of deep layers, imaging weak structure signal from the noise.
Scan time: in order to compare the scan time of different methods in a simple way, we take the scan time of 64 × 64 matrix as unit 1 (∼0.18 s) for reference. Higher density 128 × 128 scan takes 4 units. Superresolution with 9 shifted low density C-scans of 64 × 64 takes scan time of 9 units. In experiment, we buffer the scan data and perform the fast Fourier transform subsequently to ensure the shortest scan time. Shorter scan time is very important for
|Density of line pairs (lp/mm)||Width of 1 line (μm)||Density of line pairs (lp/mm)||Width of 1 line (μm)||Density of line pairs (lp/mm)||Width of 1 line (μm)||Density of line pairs (lp/mm)||Width of 1 line (μm)|
Using a 100 mm focal length lens with a of 0.015 we performed SD-OCT imaging of the resolution target shown in Figure 9(a). In Figures 10 and 11, a set of OCT lateral images are compared, which were acquired by different scan matrixes and processing methods but with the same FOV of ∼1 × 1 mm2. All the images were taken in the same experiment with the same focusing condition and light source power. The output images were uniformly set as 8-bit gray TIFF format for comparison.
In Figure 10, the scan matrix is given in the first column (such as 64 × 64) and the corresponding scan time (taking 64 × 64 scan time as unit 1) in the second column. Column 3 is the OCT lateral image of the resolution target. Column 4 shows the enlarged image of the blue area of the column 3, comparing the barely distinguishable element (the first row) and the indistinguishable element (the last row). Here, we simplify Group i Element j on the resolution target as GiEj. The background noise image of the red region in column 3 is enlarged in column 5 with detailed noise statistics (STD, PSNR, RMS and DR values). The comparisons on lateral resolution, image quality, and scan time in Figures 10 and 11 are summarized in Table 3.
|Scan timea||High density scanning||Multi-frame superresolution|
|Scan matrixb||Spot spacing||Lateral resolution (μm)||PSNRc (dB)||DRc (dB)||Low resolution C-scans||Shift||Lateral resolution (μm)||PSNRc (dB)||DRc (dB)|
|1280||10242 × 5||1/16||9.84||23.39||15.53||—||—||—||—||—|
Except the scan density, further increasing lateral resolution should consider suppressing the background noise. We applied the traditional multi-frame averaging approach to average five of 1024 × 1024 scanned lateral images, resulting in an improved image in (G) showing visibility of G5E5 of 9.84 μm line width while G5E6 still indistinguishable as the profile in (L) left. Averaging more frames such as 10 would further reduce the noise but cannot improve the lateral solution to G5E6 (not shown here). Also, 10-frame lateral averaging takes too much scan time, unacceptable in a practical OCT 3D imaging.
Compared with high scan density and multi-frame averaging methods, our superresolution processing with designedly shifted low resolution C-scans can effectively improve the lateral resolution. Figure 10(H) shows our superresolution processed image with 961 input low resolution shifted C-scans (1/16-spot-spacing step and maximum 15/16-spot-shift). It is a 31 × 31 shifted scan matrix similar as the 7 × 7 matrix in Figure 7(d). From the enlarged resolution image in column 4, we can distinguish G5E6 of 8.77 μm line width which is also verified in (L) right. Besides, in order to verify the effectiveness of the superresolution algorithm, we up sampled the 961 input low resolution images to the same image size as (H) by bicubic interpolation, and then averaged them with shift compensation. Although the output image (K) has the same image size of (H), the spatial resolution is terrible, barely observing 12.40 μm line width pattern (G5E3), worse than both the high density scan and the multi-frame averaging. This comparison demonstrates that the lateral resolution improvement is from both sub-spot-spacing shifted information and the superresolution algorithm, not only more data collection.
After reconstructing the non-deconvolved high resolution image (H) from a series of low resolution images, further lateral resolution improvement should be achieved by Lucy-Richardson deconvolution processing of image (H) with an optimized Gaussian PSF in (I) or by blind deconvolution processing shown in (J) as we discussed in Section 2, both clearly exhibiting G6E1 of 7.81 μm line width without additional hardware configuration. Also, the superresolution with deconvolution processing obviously enhances the contrast of the resolution element. All three lines in G5E6 in (I) and (J) are much clearer than in (H), indicating the effectiveness of deconvolution methods. The optimized Gaussian PSF was selected by iteration changing of Gaussian parameters to achieve the best output image. Different from the Lucy-Richardson deconvolution with a manually selected Gaussian PSF, the blind deconvolution can automatically estimate an optimized irregular PSF and thus deblurred the image in (H) with less ringing artifacts, although it still introduces some additional noise to the background. Thus, for the following deconvolution processing, we mainly use the blind deconvolution algorithm. We also attached the Gaussian PSF or estimated irregular PSF at the right bottom of the deconvoluted image.
Compared with original C-scan (A), our superresolution technique improves the lateral resolution from 25 to 8.77 μm (H) (without deconvolution processing) and to 7.81 μm (with deconvolution processing, in (I) and (J)), a factor of ∼3 times improvement. According to the above discussion, we can summarize that for lateral resolution improvement, the superresolution technique with shifted low density C-scans is better than multi-frame averaging of several high density C-scans, which is better than one set simple high density C-scan. The superresolution with deconvolution processing will further improve the lateral resolution.
According to Rayleigh criterion , the resolution limit of an optical system is restricted to half of the focused spot size. Our present beam spot radius was measured as ∼10.5 μm, similar to the 9.84 μm line width of G5E5 in Figure 10(G). This agrees well with the theory of diffraction limit. We can actually observe the 8.77 μm line width pattern of G5E6 in (H), which is slightly better than the spot radius due to increase of pixel density, reduction of noise, and enhancement of image contrast by our superresolution technique.
The resolution of an optical system is physically restricted by the diffraction limited, or PSF in other words. Dense patterns cannot be distinguished are due to finite spot size blurring. The digital deconvolution processing with a proper PSF can break the diffraction limit for resolution and sharpness improvement. Superresolution processing with Lucy-Richardson deconvolution using an optimized Gaussian PSF in Figure 10(I) or with blind deconvolution in (J) can clearly exhibit the G5E6 line width of 8.77 μm with higher image contrast and further show the next group element G6E1 with 7.81 μm line width, both breaking the diffraction limit and significantly improving the lateral resolution.
Through five-frame averaging, the PSNR in (G) is improved to 23.39 dB, better than the value 17.50 dB in (E). A 10-frame averaging can further reach 27.75 dB (not shown) but it is still lower than 30 dB and doubling the scan time of five-frame averaging. The superresolution processed image in (H) can achieve 31.50 dB PSNR, almost doubling the dB values of the high density scan results in (E) and (F). Although all the images in Figure 10 have the same 8-bit gray range between 0 and 255, we recognize that the superresolution processed image shows better contrast and thus looks brighter. That is because the higher DR and lower background noise in (H), (I) and (J) makes brighter appearance to human eye observation. The DR value of (H) also doubles the values of (E) and (F) in dB unit. The superresolution with deconvolution processed images in (I) and (J) decrease a little in the image quality as compared to (H), because of the increased background noise by the deconvolution processing. Here, we simply summarize the image quality comparison that the superresolution processing is better than the multi-frame averaging which is better than high density scan. The superresolution with deconvolution improves the image resolution but slightly reduces the image quality.
Besides, we noticed that (K) has the best PSNR and DR value among all the images of Figure 10, which comes from averaging the 961 up-sampled low resolution images (the same input images as (H)) with shift compensation. The STD value is only 1.63, exhibiting very smooth background without obvious noises. If only focusing on the image quality values, we may be misled that the average of up-sampled images can provide better background noise suppression than the superresolution technique. However, this method penalizes the high frequency signal, resulting in a poor resolution of 12.4 μm, even worse than the high density scan in (E), which is not an acceptable method.
To reduce the scan time, we compare a list of superresolution processed images in Figure 11 with much fewer input C-scans than Figure 10(H). Also, there are two different shift strategies applied in this experiment similar as Figure 7(b) and (c) to demonstrate the additional gray shifts in Figure 7(c) are needed for higher lateral resolution and image quality. Although the red shifts are enough for sampling rate improvement by superresolution processing, those additional gray shifts could contribute to image noise reduction, the lateral spatial resolution and overall image quality improvement. In Figure 11(A), the pattern G5E3 is indistinguishable, processed with Figure 7(b) scan strategy. While with more shifts as Figure 7(c) strategy, the pattern of G5E3 in Figure 11(B) is clearly visible and we can further partly distinguish the G5E4 pattern. Similarly, the G5E5 in Figure 11(D) is not obvious with red shifts only in Figure 7(d). After including the additional gray shift patterns, the G5E5 pattern in Figure 11(E) becomes visible. Thus, these gray shifts can effectively improve the lateral resolution as well as reduce the background noise by about 20–70% in STD and RMS, overcoming the reconstructed image with red shifts only.
The superresolution processing with Lucy-Richardson or blind deconvolution has demonstrated its contribution to the resolution improvement again, shown in Figure 11(C) and (F) as compared to (B) and (E), respectively. The deconvolution also introduces some degradation to image quality, increasing the background noise similar as in Figure 10(I) and (J). It is important to note that the superresolution with deconvolution does not spend any extra scan time.
Comparing with Figure 10(B)–(E), the results in Figure 11 clearly show the advantage of our multi-frame superresolution processing with less scan time while offering much better lateral resolution and image quality. Reducing from 961 to 49 shifted C-scans, it only takes 9.8 s to see the 9.84 μm line width pattern in Figure 11(F), while the 1024 × 1024 high density scan in Figure 10(E) takes about 51 seconds to barely observe the 11.05 μm line width pattern with lower image quality. Similarly, Figure 11(A)–(E) provide higher lateral resolution and better image quality with shorter scan time than Figure 10(B)–(E). Clearly, our superresolution technique has demonstrated its superior performance in lateral resolution and image quality improvement with shorter scan time.
Based on the above experiments, the lateral resolution and image quality vs. scan time are summarized in Table 3. Obviously, the multi-frame superresolution technique can achieve much better lateral resolution and image quality with less scan time than high density scanning and multi-frame averaging.
Except the experiment with 100 mm focal length lens above, we also checked the performance of a 30 mm focal length lens, which focuses the collimated beam to the diameter of ∼6 μm, very suitable to image the patterns group 6–7 in the resolution target of Figure 9(b). Figure 12(A)–(D) exhibits the original, the high density scanned, the average of multiple high density scans, and our superresolution with deconvolution processed images, respectively. The original low density scan (64 × 64) cannot distinguish any pattern, except the G6E1 with line width of 7.81 μm in Figure 12(A). With extremely higher scan density of 2048 × 2048 (taking 1024 scan time units) or averaging of five 1024 × 1024 scanned images (1280 time units), the 3.10 μm (G7E3) and the 2.76 μm (G7E4) become barely visible as shown in Figure 12(B) and (C). After the multi-frame superresolution with blind deconvolution processing of 961 shifted low resolution images (similar as Figure 12(A), with 1/16-spot-spacing shift step and maximum 15/16-spot-shift), we can see the 2.19 μm patterns (G7E6) as in Figure 12(D). The lateral resolution has been significantly improved from 7.81 μm (the original sparse scan in Figure 12(A)) to 2.47 μm (superresolution processing without deconvolution, not shown) and to 2.19 μm (superresolution processing with blind deconvolution in Figure 12(D)), about 3–3.5 times improvement. Compared with other methods like the high density scan and the multi-frame averaging, our superresolution technique exhibits superior advantage in lateral resolution improvement again. Our technique also shows the apparently better image quality than other methods: PSNR and DR of 103.7 and 137.9% (without deconvolution) and 65.2 and 106.3% (with deconvolution, Figure 12(D)) higher than high density scan (Figure 12(B)) in dB unit; PSNR and DR of 50.9 and 60.5% (without deconvolution) and 22.4 and 39.2% (with deconvolution, Figure 12(D)) higher than the multi-frame averaging (Figure 12(C)) in dB unit. Similar as the experiment of using 100 mm focal length lens, the use of 30 mm focal length lens demonstrates again that our superresolution technique can offer higher lateral resolution and better image quality with less scan time than the high density C-scan and the multi-frame averaging method.
The present Lucy-Richardson deconvolution with a Gaussian PSF or the blind deconvolution with an estimated PSF however have some problems: Although the deconvolution effectively improves the lateral resolution, it introduces some artifacts in Figures 10(I), (J), 11(C), (F) and 12(D), which may not be acceptable for some critical applications. The artifacts are from both imperfect PSF selection and the discrete Fourier transform. And they cannot be avoided in the advanced blind deconvolution.
To our observation, the deconvolution methods are sensitive to the noise level. If background noise is as low as Figure 10(K), the deconvolution processing will not introduce obvious artifacts (not shown here, referring to our previous work ). However, the method in Figure 10(K) is harmful to the spatial resolution. Practically, it is difficult to obtain a penetrated lateral image with so smooth background as well as maintaining high resolution due to various scattering mediums in the samples.
When a focused beam penetrating into a sample, the scattering would alter the cross-section profile of the beam. The optimized lateral PSF thus may be different in different samples and at different depth layers . Even with advanced blind deconvolution, the ground true PSF  of the system at that depth layer is still difficult to find. We also could notice that the estimated PSFs in Figures 10(J), 11(F) and 12(D) are different.
Considering the above issues, we would not apply the deconvolution processing to the following OCT experiments of functional samples. However, it is important to point out that the superresolution technique with deconvolution processing can break the diffraction limit, improve the sampling rate and suppress the background noise together to significantly improve the lateral resolution and image quality.
3.2 Improved lateral resolution imaging of microstructure samples
Thus far, we have successfully demonstrated the effective lateral resolution and image quality improvement by the multi-frame superresolution processing with shifted low resolution C-scans. This processing can offer better image quality with less scan time than high density C-scan images and is especially suitable for imaging micron scale fine structures [19, 37, 71, 72, 73, 74].
We examined 3D imaging of a microstructure sample in Figure 13, in which the particle size is about 3 μm. The two left images of Figure 13(A) and (B) are the original sparse scan lateral SVP images of the same sample using 30 and 19 mm focal length lens with 1300 × 1300 and 500 × 500 μm2 lateral FOV, respectively. Even with 19 mm focal length lens and ∼4 μm focused spot size, the microstructures are still invisible. After superresolution processing of 225 low resolution shifted frames with 1/8-spot-spacing step and maximum 7/8-spot-shift (using 15 × 15 shift matrix with arrangement similar to Figure 7(d)), we are able to observe clearly those microstructures and wavy surface caused by the imperfect fabrication in exposure and developing. This wavy surface is difficult to be seen in microscope imaging without topographic imaging capability. As our previous report, the multiple-frame superresolution processing can improve the lateral resolution of our SD-OCT with 19 mm focal length lens by ∼3 times, achieving 1–2 μm . Although 19 mm focal length lens can provide better image resolution than 30 mm focal length lens due to smaller focal spot, it sacrifices the lateral FOV and axial
Except for better human vision, the lateral resolution and image quality improvements further benefit various machine vision algorithms, providing more details for feature detection. Our previous work has reported the superresolution assisted image stitching for achieving an ultra-wide lateral FOV. Taking Figure 14 as an example, we scanned a multi-layer microfluidic sample by the high density scan and our multi-frame superresolution with shifted C-scans introduced above. All the structures are visible in Figure 14(A) left, however with a lot of speckle noises. Applying the advanced SURF  feature detection algorithm to the left two adjacent SVP images, there are no correct feature pairs found between them. And the incorrect matching information fails the following image stitching, overlapping two left images as Figure 14(A) right. Actually, there is only 30% shared region for the left two images. This failure is because most machine vision algorithms are not robust to periodic structures and noisy background. After superresolution processing, the image quality is significantly improved as in Figure 14(B) left, although with the same pixel resolution. The improved images offer much more correct feature pairs, supporting the following image stitching algorithm to reconstruct a wide lateral FOV image successfully at right. This comparison demonstrates the superresolution technique would be an effective pre-processing for subsequent machine vision algorithms.
As we discussed in Section 2.1, each lens has its lateral FOV limitation due to Petzval field curvature. For example, 1400 × 1400 μm2 optimized lateral FOV for 30 mm focal length lens guarantees the overall high resolution for the whole C-scan region. However, this lateral FOV is obviously not enough to image a large sample with centimeter scale sizes. To overcome this drawback, we scan 6 nearby partial overlapped regions of a microstructure sample by a 30 mm focal length lens. Each local C-scan covers a FOV of 1300 × 1300 μm2 and is enhanced by the superresolution processing. One of the SVP images is shown in Figure 13
Again, we stitched 10 close-by C-scans with 500 × 500 μm2 FOV, imaged by a 19 mm focal length lens, to reproduce a 2.10 × 1.15 mm2 wide FOV 3D image in Figure 16. Due to short focal length lens, this figure stitched by more C-scans only covers 1/3 area of Figure 15, although with higher lateral resolution of <2 μm . The top view, selected layers, and selected B-scans are exhibited in Figure 16(A)–(C), respectively. The wide FOV images could be enlarged for stitching performance and image quality checking by readers. In principle, there is no limitation on lateral FOV enlargement by this image stitching technique while maintaining the needed high lateral resolution SD-OCT imaging by the multi-frame superresolution processing. While, for fully review the microstructure sample, the 19 mm lens need to image more than 30 adjacent regions due to small lateral FOV, spending at least 5 times more scan time than using a 30 mm focal length lens, thus it is only suitable for ultra-high lateral resolution imaging.
3.3 Improved lateral resolution imaging of
in vivo3D fingerprint
The previous section has successfully demonstrated the superresolution processing enhanced 3D imaging for static samples. Actually, this quick and high quality 3D imaging technique is very suitable for time sensitive security applications such as
As discussed above, OCT has great potential in security applications, such as
The layer by layer superresolution processing also improves the B-scan image quality. Figure 18(A) shows an original low resolution fingerprint B-scan image in the same C-scan of Figure 17(A). We only observe the external fingerprint pattern but with very blurred images of the eccrine sweat glands and the internal fingerprint structures. The two right side images are enlarged areas pointed by the yellow and blue arrows. The yellow rectangle image shows a blurred eccrine sweat gland but we cannot distinguish the helical structure. The blue square image does not exhibit any eccrine sweat glands. After the same superresolution processing as the Figure 17(B), we extracted one B-scan image shown in Figure 18(B) from the final high quality 3D image (Figure 19(A)) at the same position of Figure 18(A). In Figure 18(B), the helical structure of the eccrine sweat gland marked by the yellow arrow is clearly visible and enlarged at the right side. The three eccrine sweat glands have different intensity because their centers are not in the same B-scan plane. The superresolution processed B-scan exhibits excellent image quality with 49.9% PSNR and 50.6% DR improvement in dB unit. The improvement from Figure 18(A) and (B) can be clearly visualized. After separating the multi-layer fingerprint 3D image (Figure 19(A)) into three layers: external fingerprint layer, eccrine sweat glands layer and internal fingerprint layer, the curved layer images are shown in Figure 19(B)–(D), respectively. The distribution of eccrine sweat glands in the whole scan area are beautifully displayed in Figure 19(C). The application of colormap brings the gland distribution clearer than the gray scale mapping in Figure 17. The 3D fingerprint structure is shown in both Figure 19(B) and (D). Our superresolution enhanced SD-OCT successfully reconstructs the high quality
In conclusion, a high lateral resolution and high image quality SD-OCT 3D imaging has been achieved by the multi-frame superresolution technique, with shorter scan time than traditional methods. Through adjusting the matrix of control voltages to the galvanometer scanners, we intendedly introduce designed sub-spot-spacing shifts to low resolution C-scans for static sample imaging. After the multi-frame superresolution processing of these shifted C-scan images, about 3 times lateral resolution improvement has been demonstrated by imaging a standard resolution target, from 25 to 7.81 μm and from 7.81 to 2.19 μm with sample arm lens
We present that Lucy-Richardson deconvolution with an optimized Gaussian PSF and the advanced blind deconvolution may potentially break the diffraction limit to further improve the lateral resolution of OCT systems. Although the PSF is highly dependent on samples and depth layers as well as the deconvolutions are sensitive to noise levels, we show the conceptual significance of our superresolution with the following deconvolution in lateral resolution improvement.
Although the present study depends on a SD-OCT system, the superresolution technique is able to work with other scan based OCT imaging system including time domain OCT and swept source OCT, benefiting various medical and non-medical OCT imaging applications.
We thank New Span Opto-Technology for providing the SD-OCT system.
Conflict of interest
The authors declare no conflict of interest.