Abstract
Fourier filtering for image denoising consists in masking parts of the Fourier spectrum of an image and using inverse Fourier transform of the masked image to obtain the denoised one. In cases of directional noise, this process can induce artifacts, mainly because of the spatial coherence that exists in the theoretical noise-free image. Moreover, it can lead to loss of low-frequency content that is important in applications such as fringe projection technique, which aims at measuring 3D elevations of a surface. A method based on the principle of Fourier spectrum cloning for the denoising of images is proposed in this chapter. This method improves the PSNR and the SSIM ratio in comparison with spectrum masking denoising. The method will be detailed first, and then examples of image denoising in two different applications will be presented.
Keywords
- Fourier transform
- fringe projection
- image denoising
- spectrum cloning
- periodic noise
1. Introduction
Fourier filtering is one of the main techniques used for the denoising of images corrupted by periodic noise. Most of image processing denoising algorithms tend to consider statistically defined noises, such as Gaussian, Poisson, or speckle noises [1]. However, in a relatively high number of cases, noises encountered in images are quasiperiodic and directional. These noises can be viewed as first-order, structured noises. Quasiperiodic noises are essentially due to AC perturbations or acquisition and reconstruction process errors in the case of three-dimensional reconstruction images. This type of noise is usually removed using filtering in the Fourier domain [2]. The Fourier transform is intrinsically well adapted, because it decomposes a signal on a basis of sine and cosine function which have an infinite support. The principle of Fourier filtering is usually the same: the Fourier spectrum exhibits some peaks that correspond to the frequencies of the noise, and the denoising operation consists in masking the part of the spectrum that contains the peaks after having detected them with the eye or simple or more complex algorithms. The removal of the Fourier spectrum peaks has a major drawback: the abrupt removal of all the Fourier coefficients induces artifacts and missing spatial frequencies in the reconstructed images.
The idea developed in this chapter consists in cloning the values of the module of the spectrum around the removed part and to use a combination of these values to fill the removed part. The observed result is a reduced noise with fewer oscillations due to the phase content rupture. The underlying hypothesis is that the phase of an image is very coherent on high amplitudes because it carries most of the information linked to the structure of the image [3]. Thus, creating a hole in the spectrum harms the phase of the image and the continuity of information. Moreover, the structure of an image is related to the content of this image, but the phase of a periodic noise, which is related to an acquisition or a reconstruction process, is statistically different. In other words, it means that the phase content of the theoretical noise-free image in the Fourier domain is relatively self-consistent [4], but not consistent with that of the noise.
Images based on reconstruction principles often exhibit periodic noises. A good example of this is fringe projection technique images. These images result in the projection of a sinusoidal pattern with an angle on a surface. Then, the image is observed perpendicular to this surface, with a digital camera. Variations of topography induce a phase shift of the sinusoidal pattern, and a phase unwrapping operation allows the three-dimensional reconstruction of the surface.
The chapter is organized as follows. First, some recalls about denoising using Fourier transform are given in one and two dimensions. Second, the principle of spectrum cloning is introduced as an extension of Fourier denoising. In this case too, it is proposed in one and two dimensions. Then, a section presents results on a synthetic example consisting of the Lena image with a periodic noise added. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) measure are used to highlight the possible improvement of the spectrum cloning versus the classical Fourier filtering. The following section deals with an example of denoising on fringe projection technique images. This type of images greatly benefit from Fourier spectrum cloning due to the whole process of image formation.
2. Recalls on Fourier denoising
In this section, we recall the basic principles of Fourier denoising in one and two dimensions. These principles rely on the assumption of additive noise. The model for additive noise is
where
This model implies that the noise is a function that does not depend on the signal intensity and as a consequence that it is possible to remove it with a simple subtraction if it is fully characterized. In the case of periodic noise, the exact expression of the noise is not known, but it is well localized in the frequency domain. That is why Fourier denoising using spectrum manipulation is efficient for this type of signals.
2.1 One-dimensional Fourier denoising
Consider the Fourier transform of the signal
As the Fourier transform of a sum of functions is the sum of the individual Fourier transforms, and considering Eq. (1), one can write:
where
One can theoretically recover
However, in most cases the expression of the noise is usually not known, so
with
where
2.2 Two-dimensional Fourier denoising
In the two-dimensional case, we consider a two-dimensional spatial function
The Fourier transform of
The same principle applies, and we can also obtain an approximation
with
where
3. Fourier spectrum cloning
The main drawback of Fourier denoising using spectrum subtraction is that the whole spectrum is removed. Indeed, the operation leaves a hole in the spectrum, which can cause oscillations in the inverse Fourier transform process. In fact, one can easily understand the phenomenon considering the inverse Fourier transform of a rectangular “hole”:
where
As can be seen in Figure 1, the Fourier transform of such an inverse rectangular pulse is a
3.1 One-dimensional Fourier spectrum cloning
The purpose of Fourier spectrum cloning is to use the values of spectrum surrounding the masked parts to create a synthetic replacement. More precisely, the part of the spectrum that has been removed is replaced by a mean of surrounding parts. In order to obtain the new approximation
with
where
This operation utilizes the information of the signal itself to create false spectrum content. Indeed, it takes into account the nature of the signal which has its own regularity to construct the replacement part of the spectrum.
3.2 Two-dimensional Fourier spectrum cloning
The same principle can apply in two dimensions. To obtain an approximation
where
and
As the cloning operation actually creates information instead of the missing part of the spectrum, it can be desirable to add a setting parameter
Practically, this principle should be the best for isotropic or orthotropic periodic noise. However, in real applications, the Fourier spectrum does not present well-localized peaks but more singular lines crossing the zero frequency point and the peaks. As a consequence, it is sometimes better to clone the whole line containing the targeted frequency range.
4. Results on the Lena image
In this section, we present results on a test signal consisting of the Lena image with a sinusoidal noise added. The Lena image can be considered as a natural image because it has been acquired with a camera and then digitalized. In this example the image range is
One can observe Figure 2 that the Fourier transforms of the noise and the noisy Lena exhibit the vertical lines mentioned in the previous section.
Increasing values of
4.1 Peak signal-to-noise ratio (PSNR)
The PSNR ratio is defined as
where
where
As one can observe Figure 3, the optimum value for
4.2 Structural similarity (SSIM) measure
The default SSIM [5] between two images
where
Figure 4 shows that an optimum is reached for
4.3 Visual assessment
An example of denoising operation on the Lena image with parameter
5. Results on real applications
In this section, we focus on one application, the fringe projection technique, which benefits highly from the Fourier spectrum cloning denoising.
The fringe projection method has already been described by many authors (see, e.g., [7, 8, 9, 10]). Basically, a periodic pattern of white and black lines is projected on an object; the light is diffused by the object and captured by a CCD video camera. The deformation of the fringes depends on the shape of the illuminated object. In order to observe this deformation, the angle between the projected fringes and the observed diffused light must not be null. The result is a 3D map that can be viewed as an image, with
5.1 Fringe projection technique basics
5.1.1. Optical setup
The fringe projection setup for shape measurement is based on a pocket projector (3M© MPRO 110), 800 × 600, and a Imaging Source CCD camera, 1280 × 960, 8 bits. This solution is adapted to fields of investigation from 10 × 7 to 200 × 150 mm2 (see Figure 7).
5.1.2. Basic principle
Light intensities on an object illuminated by a set of fringes can be described by a periodic function
where
In this expression, the sensitivity is proportional to the angle
5.1.3. Phase extraction
Then, phase extraction is a classical topic in optics applied to mechanics. Considering Eq. (13), extraction of the phase from intensity map(s) can be done from a single image, using a category of methods known as spatial phase shifting [12], but better results are usually obtained using temporal phase shifting techniques. The choice only depends on the situation: if temporal effects are expected, spatial phase shifting is more appropriate, because it only requires one image [13]. If not, temporal phase shifting technique should be preferred for its higher spatial resolution. The
Here, only temporal phase shifting is described: a set of
This shape measurement setup shows interesting metrological performances compared to the classical techniques (line projection, stereovision): the spatial resolution is 1 pixel (8–156 μm, depending on the field of view), and the resolution is
The signal-to-noise ratio (SNR) being usually high, no further signal processing is required; but some considerations on the quality of the images must be done. If the illumination is not controlled, then the sine wave is distorted. Another noise source lies in the phase shift: a drift would add noise, as demonstrated by Cordero [16]. The consequence in both cases is that parasitic harmonics enter in the shape field. Surrel proposed an algorithm robust to phase drift [14]; Kemao published a procedure to characterize the intensity period and remove most of the harmonics [17], but he used a strong assumption on intensity modeling that is not always completed. As a matter of fact, it is commonly admitted that a careful tuning is the best solution.
5.1.4. Experimental test: Case 1. Digitalization of a bas-relief
Arts have already been an important field of applications of fringe projection. For example, the support stability of the Mona Lisa paint has been evaluated by [18], but wider projects of heritage object recording should be contemplated [19]. In this specific case, obviously, no surface preparation is possible before scanning, and illumination is an issue. Here, we illustrate a possible drawback with a bas-relief that has to be scanned and duplicated. A time-shifting approach was used in order to get the better spatial resolution, but an image turned to be corrupted, resulting in a phase shift drift.
Figure 8 shows a photograph of the bas-relief (a) as well as the basic shape reconstruction (b). Parasitic fringes are clearly visible because of its structure, even if the intensity is very low compared to the heights in the field. In this first example, even if some noise remains in the final image, the global shape is not affected. The final objective being the duplication, it is better to refine some parts of the 3D model by hands after a first denoising operation that does not introduce structure errors.
5.1.5. Experimental test: Case 2. Skin characterization
The skin is a challenging topic for topology reconstruction. Skin structure is multi-scale, with a global shape containing wrinkles and fine lines. Each scale has its own topological properties, in particular the orientation, and experts would like to separate wrinkles and fine lines because the dermatologic treatment associated to each is different.
Besides these characteristics that are followed as a marker of cosmetic efficiency, it is important to note that the light diffusion of the skin is not perfect for fringe projection. Moreover, it depends on many parameters that should be considered as natural (melatonin concentration, skin moisture, tobacco, etc.) or interventional (cleaning procedure, cosmetic treatment). Then, it is difficult to change the skin surface for the sake of better experimental conditions, and the physicist has to adapt the signal processing to these conditions.
A particular point in skin texture analysis is the global amplitude of the shape variations. On a square-centimeter area, elevation variations are typically close to
We propose here two illustrations from Lorica™ replica of the skin taken on the forearm or on the forehead. On the basic reconstruction, it is possible to distinguish some periodic lines almost horizontal. These lines can be associated with the fringes considering their orientation and wavelength. A classical way of removing noise in this case is to use a Gaussian low-pass filter. Here, it has been set to
Qualitatively, both filters remove the targeted parasitic lines. The Gaussian filtered image seems blurry, as it could be expected, while the Fourier spectrum cloning (FSC) filter seems to respect the image sharpness. Quantitatively, three basic topographic data are extracted for the whole image: a mean roughness indicator (RMS roughness,
6. Conclusions
In this chapter, we proposed the Fourier spectrum cloning principle. After some recalls about Fourier denoising, we gave the basis of Fourier spectrum cloning with a tuning parameter
Fringe projection has been chosen as a first application field. The analysis of the skin microreliefs (wrinkles, fine lines) requires an optimal system and a good post-processing, the signal-to-noise ratio being limited. The potential of FSC filter is clearly outlined: a periodic noise can be removed and make the image easier to interpret, without major changes in the topographical characteristics. Anyway, in this application, only one frequency band has been removed, and a multiple choice could be necessary in practice; interactions between various filtering processes would have to be studied then.
As a conclusion, this chapter aims at presenting a simple concept and giving some results and interpretations. Many refinements can be implemented in the future, in order to improve these results obtained with the simplistic application of the cloning principle. Actually, the construction of the synthetic replacement part of the spectrum could be synthesized considering different parameters such as border effects or statistical measures on the spectrum. Further research will address these different paths.
Acknowledgments
This work has been partially funded by the French National Research Agency via the LBSMI project ANR15-CE19-0002.
The authors would like to thank G. Boyer, Laboratoires Expanscience, and Nicolas Curt, Sainbiose, for their contribution to the experimental data.
References
- 1.
Huang TS. Advances in Computer Vision & Image Processing. Greenwich, CT, USA: JAI Press; 1988. ISBN:0-89232-754-5 - 2.
Gonzalez RC, Wintz P. Digital Image Processing. 2nd ed. Upper Saddle River, NJ, USA: Prentice-Hall; 2002 - 3.
Oppenheim AV, Lim JS. The importance of phase in signals. Proceedings of the IEEE. 1981; 69 (5):529-541 - 4.
Wichmann FA, Braun DI, Gegenfurtner KR. Phase noise and the classification of natural images. Vision Research. 2006; 46 (8–9):1520-1529 - 5.
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing. 2004; 13 (4):600-612 - 6.
Hore A, Ziou D. Image quality metrics: PSNR vs. SSIM. In: 2010 20th International Conference on Pattern Recognition (ICPR). IEEE; 2010. pp. 2366-2369 - 7.
Sciammarella CA, Lamberti L, Sciammarella FM. High-accuracy contouring using projection moiré. Optical Engineering. 2005; 44 (9):093605 - 8.
Huang PS, Zhang C, Chiang FP. High-speed 3-D shape measurement based on digital fringe projection. Optical Engineering. 2003; 42 :163-168 - 9.
Gigliotti M, Molimard J, Jacquemin F, Vautrin A. On the nonlinear deformations of thin unsymmetric 0/90 composite plates under hygrothermal loads. Composites Part A: Applied Science and Manufacturing. 2006; 37 (4):624-629 - 10.
Lagarde JM, Rouvrais C, Black D, Diridollou S, Gall Y. Skin topography measurement by interference fringe projection: A technical validation. Skin Research and Technology. 2001; 7 (2):112-121 - 11.
Molimard J, Navarro L. Uncertainty on fringe projection technique: A Monte-Carlo-based approach. Optics and Lasers in Engineering. 2013; 51 :840-847 - 12.
Badulescu C, Bornert M, Dupré JC, Equis S, Grédiac M, Molimard J, et al. Demodulation of spatial carrier images: Performance analysis of several algorithms. Experimental Mechanics. 2013; 53 :1357-1370 - 13.
Wang P, Drapier S, Molimard J, Vautrin A, Minni J-C. Characterization of liquid resin infusion (LRI) filling by fringe pattern projection and in situ thermocouples. Composites Part A: Applied Science and Manufacturing. 2010; 41 :36-44 - 14.
Surrel Y. Additive noise effect in digital phase detection. Applied Optics. 1997; 36 (1):271-276 - 15.
Molimard J, Surrel Y. Grid method, Moiré and deflectometry. In: Grediac M, Hild F, editors. Full-Field Measurements and Identification in Solid Mechanics. UK, USA: Wiley; 2012. ISBN: 978-1-84821-294-7 - 16.
Cordero R, Molimard J, Martinez A, Labbé F. Uncertainty analysis of temporal phase-stepping algorithms for interferometry. Optics Communications. 2007; 275 :144-155 - 17.
Pan B, Kemao Q, Huang L, Asundi A. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Optics Letters. 2009; 34 (4):416-418 - 18.
Brémand F, Doumalin P, Dupré JC, Hesser F, Valle V. Optical techniques for relief study of Mona Lisa’s wooden support. In: Gdoutos EE, editor. Experimental Analysis of Nano and Engineering Materials and Structures. Dordrecht: Springer; 2007 - 19.
Przybilla H-J, Peipe J. 3D modelling of heritage objects by fringe projection and laser scanning systems. In: Stylianidis E, Patias P, Santana Quintero M, editors. CIPA Heritage Documentation—Best Practices and Applications, Series 1, 2007 & 2009. Greece: Ziti Publications; 2011