Open access peer-reviewed chapter

Directional Denoising Using Fourier Spectrum Cloning

By Laurent Navarro and Jérôme Molimard

Submitted: October 3rd 2018Reviewed: February 27th 2019Published: March 29th 2019

DOI: 10.5772/intechopen.85519

Downloaded: 166

Abstract

Fourier filtering for image denoising consists in masking parts of the Fourier spectrum of an image and using inverse Fourier transform of the masked image to obtain the denoised one. In cases of directional noise, this process can induce artifacts, mainly because of the spatial coherence that exists in the theoretical noise-free image. Moreover, it can lead to loss of low-frequency content that is important in applications such as fringe projection technique, which aims at measuring 3D elevations of a surface. A method based on the principle of Fourier spectrum cloning for the denoising of images is proposed in this chapter. This method improves the PSNR and the SSIM ratio in comparison with spectrum masking denoising. The method will be detailed first, and then examples of image denoising in two different applications will be presented.

Keywords

  • Fourier transform
  • fringe projection
  • image denoising
  • spectrum cloning
  • periodic noise

1. Introduction

Fourier filtering is one of the main techniques used for the denoising of images corrupted by periodic noise. Most of image processing denoising algorithms tend to consider statistically defined noises, such as Gaussian, Poisson, or speckle noises [1]. However, in a relatively high number of cases, noises encountered in images are quasiperiodic and directional. These noises can be viewed as first-order, structured noises. Quasiperiodic noises are essentially due to AC perturbations or acquisition and reconstruction process errors in the case of three-dimensional reconstruction images. This type of noise is usually removed using filtering in the Fourier domain [2]. The Fourier transform is intrinsically well adapted, because it decomposes a signal on a basis of sine and cosine function which have an infinite support. The principle of Fourier filtering is usually the same: the Fourier spectrum exhibits some peaks that correspond to the frequencies of the noise, and the denoising operation consists in masking the part of the spectrum that contains the peaks after having detected them with the eye or simple or more complex algorithms. The removal of the Fourier spectrum peaks has a major drawback: the abrupt removal of all the Fourier coefficients induces artifacts and missing spatial frequencies in the reconstructed images.

The idea developed in this chapter consists in cloning the values of the module of the spectrum around the removed part and to use a combination of these values to fill the removed part. The observed result is a reduced noise with fewer oscillations due to the phase content rupture. The underlying hypothesis is that the phase of an image is very coherent on high amplitudes because it carries most of the information linked to the structure of the image [3]. Thus, creating a hole in the spectrum harms the phase of the image and the continuity of information. Moreover, the structure of an image is related to the content of this image, but the phase of a periodic noise, which is related to an acquisition or a reconstruction process, is statistically different. In other words, it means that the phase content of the theoretical noise-free image in the Fourier domain is relatively self-consistent [4], but not consistent with that of the noise.

Images based on reconstruction principles often exhibit periodic noises. A good example of this is fringe projection technique images. These images result in the projection of a sinusoidal pattern with an angle on a surface. Then, the image is observed perpendicular to this surface, with a digital camera. Variations of topography induce a phase shift of the sinusoidal pattern, and a phase unwrapping operation allows the three-dimensional reconstruction of the surface.

The chapter is organized as follows. First, some recalls about denoising using Fourier transform are given in one and two dimensions. Second, the principle of spectrum cloning is introduced as an extension of Fourier denoising. In this case too, it is proposed in one and two dimensions. Then, a section presents results on a synthetic example consisting of the Lena image with a periodic noise added. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) measure are used to highlight the possible improvement of the spectrum cloning versus the classical Fourier filtering. The following section deals with an example of denoising on fringe projection technique images. This type of images greatly benefit from Fourier spectrum cloning due to the whole process of image formation.

2. Recalls on Fourier denoising

In this section, we recall the basic principles of Fourier denoising in one and two dimensions. These principles rely on the assumption of additive noise. The model for additive noise is

gt=ft+ntE1

where tis the time, gtis the observed signal, ftis the theoretical noise-free signal, and ntis the noise.

This model implies that the noise is a function that does not depend on the signal intensity and as a consequence that it is possible to remove it with a simple subtraction if it is fully characterized. In the case of periodic noise, the exact expression of the noise is not known, but it is well localized in the frequency domain. That is why Fourier denoising using spectrum manipulation is efficient for this type of signals.

2.1 One-dimensional Fourier denoising

Consider the Fourier transform of the signal gt:

Gf=ft+nte2iπftdtE2

As the Fourier transform of a sum of functions is the sum of the individual Fourier transforms, and considering Eq. (1), one can write:

Gf=Ff+NfE3

where Ffand Nfare, respectively, the Fourier transforms of ftand nt.

One can theoretically recover ftusing the inverse Fourier transform and a subtraction if the exact expression of the noise ntis known:

ft=GfNfe2iπftdfE4

However, in most cases the expression of the noise is usually not known, so ftcannot be recovered exactly. In order to obtain an approximation ftof ft, it is common to set a part of Gfto zero:

ft=Gffe2iπftdfE5

with

Gff=Gfifff1f20ifff1f2

where f1and f2are frequency bounds, with f1>f2, and Gffis the filtered spectrum of Gf.

2.2 Two-dimensional Fourier denoising

In the two-dimensional case, we consider a two-dimensional spatial function fxyR2, with xand yas the spatial coordinates. The additive model for the noise is the same, in two dimensions.

The Fourier transform of fxyis

Fuv=fxye2vx+uydxdyE6

The same principle applies, and we can also obtain an approximation fxyof fxywith

fxy=Gfuve2vx+uydxdyE7

with

Gfuv=Gxyifx2+y2r1r220ifx2+y2r12r22

where r1and r2are the radii of the part of the two-dimensional spectrum set to zero, with r1<r2, and Gfxyis the filtered spectrum of Gf. It is important to note that in two dimensions it is better to define a two-dimensional torus for the masked part or more adequately two-dimensional torus sectors to take into account the symmetry of the Fourier transform. Actually, in most applications this principle is not applied, and ellipses are used to mask the spectrum peaks.

3. Fourier spectrum cloning

The main drawback of Fourier denoising using spectrum subtraction is that the whole spectrum is removed. Indeed, the operation leaves a hole in the spectrum, which can cause oscillations in the inverse Fourier transform process. In fact, one can easily understand the phenomenon considering the inverse Fourier transform of a rectangular “hole”:

βfeéiπftdf=1πsinctE8

where

βf=0iff<11iff>1

As can be seen in Figure 1, the Fourier transform of such an inverse rectangular pulse is a sincfunction which oscillates to the infinity. In addition, the masked part of the spectrum implies that all frequencies in the f1f2or c1c2ranges will not be present in the signal at all.

Figure 1.

(Left) Inverse rectangular pulse in frequency. (Right) Fourier transform of an inverse rectangular pulse in time.

3.1 One-dimensional Fourier spectrum cloning

The purpose of Fourier spectrum cloning is to use the values of spectrum surrounding the masked parts to create a synthetic replacement. More precisely, the part of the spectrum that has been removed is replaced by a mean of surrounding parts. In order to obtain the new approximation ftof ft, one can write

ft=Gfcfe2iπftdfE9

with

Gfcf=Gfifff1f2Gff2+f1+Gf+f2f12ifff1f2

where Gfcfis the filtered and cloned spectrum of Gf. This expression implies that the previous part of the spectrum that was set to zero is now the mean of the part before and the part after the removed portion.

This operation utilizes the information of the signal itself to create false spectrum content. Indeed, it takes into account the nature of the signal which has its own regularity to construct the replacement part of the spectrum.

3.2 Two-dimensional Fourier spectrum cloning

The same principle can apply in two dimensions. To obtain an approximation fxyof fxy, one can write

fxy=Gfcuve2vx+uydxdyE10

where

Gfcuv=Gxyifx2+y2r12r22Gxxr2+xr1yyr2+yr1+Gxxr1+xr2yyr1+yr22ifx2+y2r12r22

and

xri=ri2+y2and yri=ri2+x2, for i=12.

As the cloning operation actually creates information instead of the missing part of the spectrum, it can be desirable to add a setting parameter αthat will be optimized for the best results. Thus, Gfcxybecomes

Gfcuv=Gxyifx2+y2r1r22αGxxr2+xr1yyr2+yr1+Gxxr1+xr2yyr1+yr22ifx2+y2r12r22

Practically, this principle should be the best for isotropic or orthotropic periodic noise. However, in real applications, the Fourier spectrum does not present well-localized peaks but more singular lines crossing the zero frequency point and the peaks. As a consequence, it is sometimes better to clone the whole line containing the targeted frequency range.

4. Results on the Lena image

In this section, we present results on a test signal consisting of the Lena image with a sinusoidal noise added. The Lena image can be considered as a natural image because it has been acquired with a camera and then digitalized. In this example the image range is 0255,and the noise has an amplitude of 50.

One can observe Figure 2 that the Fourier transforms of the noise and the noisy Lena exhibit the vertical lines mentioned in the previous section.

Figure 2.

(a) Lena image and (b) its Fourier spectrum. (c) Noisy Lena image and its (d) Fourier spectrum. (e) Noise and its (f) Fourier spectrum.

Increasing values of αranging from 0to 1have been tested in order to observe the evolution of two classical image quality measurement indices. The first one is the peak signal-to-noise (PSNR) ratio. Even if the validity of this metric for human quality perception is discussed today, it remains interesting for specific applications such as images resulting from reconstruction processes. Indeed, these kinds of images, such as fringe projection images, contain geometrical information. As the PSNR is based on the calculus of the mean squared error (MSE), it makes sense to use it on this type of images. The second metric we used is the structural similarity (SSIM) measure. The SSIM has been developed for video quality assessment. It is based on the structure of the images, contrary to the PSNR which is pixel-based. This makes it closer to the human vision which is more attached to the structures contained in images.

4.1 Peak signal-to-noise ratio (PSNR)

The PSNR ratio is defined as

PSNR=10Log10maximageMSEE11

where maximageis the maximum possible value in the images and with

MSE=1m.ni=0m1j=0n1fijgij2

where m.nis the total number of pixels in the image and fand gare the images between those the PSNR is calculated.

As one can observe Figure 3, the optimum value for αis around 0.8, with a PSNR of 36.2. The value α=0corresponds to the classical case of simple suppression of a part of the spectrum. This example highlights, in this specific case, the differences between classical spectrum masking and the spectrum cloning method in terms of PSNR.

Figure 3.

PSNR values between noise-free Lena and denoised Lena as a function of α .

4.2 Structural similarity (SSIM) measure

The default SSIM [5] between two images fand gis defined by

SSIMfg=2μfμg+c12σfg+c2μf2+μg2+c1σf2+σg2+c2E12

where μf, μg, σf, σg, and σfgare the local means, standard deviations, and cross-covariance for images fand g. c1=k1L2and c2=k2L2with Lthe dynamic range of the images and k1=0.01and k2=0.03.

Figure 4 shows that an optimum is reached for α=0.55. This value is lower than the value found for the PSNR. This can be explained by the fact that these two metrics do not focus on the same properties of the image quality after denoising. The authors of [6] discussed this particular point by measuring sensitivities of the two metrics to different types of degradation: PSNR is more sensitive to additive Gaussian noise, whereas SSIM is more sensitive to jpeg compression.

Figure 4.

SSIM values between noise-free Lena and denoised Lena as a function of α .

4.3 Visual assessment

An example of denoising operation on the Lena image with parameter α=0.8gives the result in Figure 5 and with α=0.55in Figure 6. One can observe that even if the noise has been well removed, some artifacts remain in the image. In the two cases, this does not affect the readability of the image, but the artifacts present in the α=0.55case appear more natural to the eye.

Figure 5.

(a) Lena image and (b) its Fourier spectrum. (c) Denoised Lena image with α = 0.8 and its (d) Fourier spectrum.

Figure 6.

(a) Lena image and (b) its Fourier spectrum. (c) Denoised Lena image with α = 0.55 and its (d) Fourier spectrum.

5. Results on real applications

In this section, we focus on one application, the fringe projection technique, which benefits highly from the Fourier spectrum cloning denoising.

The fringe projection method has already been described by many authors (see, e.g., [7, 8, 9, 10]). Basically, a periodic pattern of white and black lines is projected on an object; the light is diffused by the object and captured by a CCD video camera. The deformation of the fringes depends on the shape of the illuminated object. In order to observe this deformation, the angle between the projected fringes and the observed diffused light must not be null. The result is a 3D map that can be viewed as an image, with zelevation corresponding to the gray levels.

5.1 Fringe projection technique basics

5.1.1. Optical setup

The fringe projection setup for shape measurement is based on a pocket projector (3M© MPRO 110), 800 × 600, and a Imaging Source CCD camera, 1280 × 960, 8 bits. This solution is adapted to fields of investigation from 10 × 7 to 200 × 150 mm2 (see Figure 7).

Figure 7.

Experimental fringe projection setup.

5.1.2. Basic principle

Light intensities on an object illuminated by a set of fringes can be described by a periodic function Ili, with a perturbation corresponding to the object shape:

Ilixy=I0xy1+γxycos2πpxyy+ΦxyE13

where I0is the average intensity and γis the fringe contrast. These values should be constant over the whole map, but some low-frequency variations due to illumination inhomogeneities or diffusivity changes on top of the surface can occur. Consequently, both average intensity and contrast have to be considered as local quantities, typically calculated over few fringe periods, and can be denoted I0xyand γxy. The fringe period pis the distance between two light peaks on a flat surface. Of course, due to perspective effects, this pitch can change over the map. Last, the object is responsible for a phase shift Φxyat each point of the field that can be written as

Φxy=2πtanθxypxyzxyE14

In this expression, the sensitivity is proportional to the angle θbetween the CCD video camera and the video projector and to the fringe density 1/p. As expressed by the equation, the sensitivity varies over the field because usual video projector and the CCD camera commonly use divergent lens. Here, pinhole model parameters were identified through an identification procedure. Readers should refer to [11] for detailed explanations.

5.1.3. Phase extraction

Then, phase extraction is a classical topic in optics applied to mechanics. Considering Eq. (13), extraction of the phase from intensity map(s) can be done from a single image, using a category of methods known as spatial phase shifting [12], but better results are usually obtained using temporal phase shifting techniques. The choice only depends on the situation: if temporal effects are expected, spatial phase shifting is more appropriate, because it only requires one image [13]. If not, temporal phase shifting technique should be preferred for its higher spatial resolution. The Photomechanix toolbox, developed in the laboratory, has genuine implementation of both techniques, as prescribed by Surrel ([14, 15]).

Here, only temporal phase shifting is described: a set of n×qfringe patterns with a known phase shift q/2πis projected successively on the surface, first and last fringe pattern being shifted by a n×2π, nphase. Then, the intensity variation at each point (i.e., each camera pixel) corresponds to a sine wave function with an initial phase shift. The phase is evaluated using the Fourier transform:

Φrs=arctan2πk=1nqsink2πqIkrsk=1nqcosk2πqIkrsE15

This shape measurement setup shows interesting metrological performances compared to the classical techniques (line projection, stereovision): the spatial resolution is 1 pixel (8–156 μm, depending on the field of view), and the resolution is σ=1/100th fringe, i.e., 10 μm at best. This capacity is very important for high-frequency phenomena monitoring: a skin submitted to mechanical load, metal instability under forming process, etc.

The signal-to-noise ratio (SNR) being usually high, no further signal processing is required; but some considerations on the quality of the images must be done. If the illumination is not controlled, then the sine wave is distorted. Another noise source lies in the phase shift: a drift would add noise, as demonstrated by Cordero [16]. The consequence in both cases is that parasitic harmonics enter in the shape field. Surrel proposed an algorithm robust to phase drift [14]; Kemao published a procedure to characterize the intensity period and remove most of the harmonics [17], but he used a strong assumption on intensity modeling that is not always completed. As a matter of fact, it is commonly admitted that a careful tuning is the best solution.

5.1.4. Experimental test: Case 1. Digitalization of a bas-relief

Arts have already been an important field of applications of fringe projection. For example, the support stability of the Mona Lisa paint has been evaluated by [18], but wider projects of heritage object recording should be contemplated [19]. In this specific case, obviously, no surface preparation is possible before scanning, and illumination is an issue. Here, we illustrate a possible drawback with a bas-relief that has to be scanned and duplicated. A time-shifting approach was used in order to get the better spatial resolution, but an image turned to be corrupted, resulting in a phase shift drift.

Figure 8 shows a photograph of the bas-relief (a) as well as the basic shape reconstruction (b). Parasitic fringes are clearly visible because of its structure, even if the intensity is very low compared to the heights in the field. In this first example, even if some noise remains in the final image, the global shape is not affected. The final objective being the duplication, it is better to refine some parts of the 3D model by hands after a first denoising operation that does not introduce structure errors.

Figure 8.

Bas-relief intensity image (a) and shape (b). The classical Gaussian low-pass (σ = 10 px) (c) filter shows better parasitic fringe removal but loses details compared to the FSC filter (μ = 110, δ = 22) (d).

5.1.5. Experimental test: Case 2. Skin characterization

The skin is a challenging topic for topology reconstruction. Skin structure is multi-scale, with a global shape containing wrinkles and fine lines. Each scale has its own topological properties, in particular the orientation, and experts would like to separate wrinkles and fine lines because the dermatologic treatment associated to each is different.

Besides these characteristics that are followed as a marker of cosmetic efficiency, it is important to note that the light diffusion of the skin is not perfect for fringe projection. Moreover, it depends on many parameters that should be considered as natural (melatonin concentration, skin moisture, tobacco, etc.) or interventional (cleaning procedure, cosmetic treatment). Then, it is difficult to change the skin surface for the sake of better experimental conditions, and the physicist has to adapt the signal processing to these conditions.

A particular point in skin texture analysis is the global amplitude of the shape variations. On a square-centimeter area, elevation variations are typically close to 1mm, but the wrinkle roughness should be as small as σRMS10μm. Consequently, in some cases, the roughness might be only tentimes higher than the noise, and if this noise has an organized texture, it interferes in the human perception of the surface topology.

We propose here two illustrations from Lorica™ replica of the skin taken on the forearm or on the forehead. On the basic reconstruction, it is possible to distinguish some periodic lines almost horizontal. These lines can be associated with the fringes considering their orientation and wavelength. A classical way of removing noise in this case is to use a Gaussian low-pass filter. Here, it has been set to σ=10 px (pixels) according to the user’s practice.

Qualitatively, both filters remove the targeted parasitic lines. The Gaussian filtered image seems blurry, as it could be expected, while the Fourier spectrum cloning (FSC) filter seems to respect the image sharpness. Quantitatively, three basic topographic data are extracted for the whole image: a mean roughness indicator (RMS roughness, σRMS), a pic-to-valley indicator (skewness, Sk), and a shape indicator (kurtosis, Kt). In this situation, it is worth recording that there is no ground truth. It can be observed after all that σRMSand Skare dramatically changed with Gaussian filter and not with the FSC filter (Figures 9 and 10).

Figure 9.

Intensity image of the skin taken at the forearm (a) and texture reconstruction of the skin after high-pass filtering to remove the global shape σRMS = 20.1 μm, Sk = −0.231, Kt = 3.274 (b). The classical Gaussian low-pass (σ = 10 px) σRMS = 18.0 μm, Sk = −0.169, Kt = 3.253 (c) filter shows good parasitic fringe removal but loses details compared to the FSC filter (μ = 48, δ = 10) σRMS = 20.0 μm, Sk = −0.225, Kt = 3.287 (d).

Figure 10.

Intensity image of the skin taken at the forehead (a) and texture reconstruction of the skin after high-pass filtering to remove the global shape σRMS = 38.5 μm, Sk = −0.013, Kt = 3.411 (b). The classical Gaussian low-pass (σ = 10 px) σRMS = 37.6 μm, Sk = −0.008, Kt = 3.364 (c) filter shows good parasitic fringe removal but loses details compared to the FSC filter (μ = 42, δ = 12) σRMS = 38.5 μm, Sk = −0.013, Kt = 3.410 (d).

6. Conclusions

In this chapter, we proposed the Fourier spectrum cloning principle. After some recalls about Fourier denoising, we gave the basis of Fourier spectrum cloning with a tuning parameter α01allowing to choose the amount of cloned spectrum. We proposed to measure the performances of the algorithm on the Lena image without optimization to observe the benefits of the method. To do so we used the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) which are good metrics for measuring differences between images.

Fringe projection has been chosen as a first application field. The analysis of the skin microreliefs (wrinkles, fine lines) requires an optimal system and a good post-processing, the signal-to-noise ratio being limited. The potential of FSC filter is clearly outlined: a periodic noise can be removed and make the image easier to interpret, without major changes in the topographical characteristics. Anyway, in this application, only one frequency band has been removed, and a multiple choice could be necessary in practice; interactions between various filtering processes would have to be studied then.

As a conclusion, this chapter aims at presenting a simple concept and giving some results and interpretations. Many refinements can be implemented in the future, in order to improve these results obtained with the simplistic application of the cloning principle. Actually, the construction of the synthetic replacement part of the spectrum could be synthesized considering different parameters such as border effects or statistical measures on the spectrum. Further research will address these different paths.

Acknowledgments

This work has been partially funded by the French National Research Agency via the LBSMI project ANR15-CE19-0002.

The authors would like to thank G. Boyer, Laboratoires Expanscience, and Nicolas Curt, Sainbiose, for their contribution to the experimental data.

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Laurent Navarro and Jérôme Molimard (March 29th 2019). Directional Denoising Using Fourier Spectrum Cloning, Fourier Transforms - Century of Digitalization and Increasing Expectations, Goran S. Nikoli? and Dragana Z. Markovi?-Nikoli?, IntechOpen, DOI: 10.5772/intechopen.85519. Available from:

chapter statistics

166total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

Related Content

This Book

Next chapter

Analysis of Financial Time Series in Frequency Domain Using Neural Networks

By Stefan Nikolić and Goran Nikolić

Related Book

First chapter

Fourier Transform Infrared Imaging Spectroscopy in Biomedicine – Important Things to Consider When Planning a New Experiment

By Rieppo J., Rieppo L., Saarakkala S. and Jurvelin J.S.

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us