Directional Denoising Using Fourier Spectrum Cloning

Abstract

Fourier filtering for image denoising consists in masking parts of the Fourier spectrum of an image and using inverse Fourier transform of the masked image to obtain the denoised one. In cases of directional noise, this process can induce artifacts, mainly because of the spatial coherence that exists in the theoretical noise-free image. Moreover, it can lead to loss of low-frequency content that is important in applications such as fringe projection technique, which aims at measuring 3D elevations of a surface. A method based on the principle of Fourier spectrum cloning for the denoising of images is proposed in this chapter. This method improves the PSNR and the SSIM ratio in comparison with spectrum masking denoising. The method will be detailed first, and then examples of image denoising in two different applications will be presented.

1. Introduction

Fourier filtering is one of the main techniques used for the denoising of images corrupted by periodic noise. Most of image processing denoising algorithms tend to consider statistically defined noises, such as Gaussian, Poisson, or speckle noises [1]. However, in a relatively high number of cases, noises encountered in images are quasiperiodic and directional. These noises can be viewed as first-order, structured noises. Quasiperiodic noises are essentially due to AC perturbations or acquisition and reconstruction process errors in the case of three-dimensional reconstruction images. This type of noise is usually removed using filtering in the Fourier domain [2]. The Fourier transform is intrinsically well adapted, because it decomposes a signal on a basis of sine and cosine function which have an infinite support. The principle of Fourier filtering is usually the same: the Fourier spectrum exhibits some peaks that correspond to the frequencies of the noise, and the denoising operation consists in masking the part of the spectrum that contains the peaks after having detected them with the eye or simple or more complex algorithms. The removal of the Fourier spectrum peaks has a major drawback: the abrupt removal of all the Fourier coefficients induces artifacts and missing spatial frequencies in the reconstructed images.

The idea developed in this chapter consists in cloning the values of the module of the spectrum around the removed part and to use a combination of these values to fill the removed part. The observed result is a reduced noise with fewer oscillations due to the phase content rupture. The underlying hypothesis is that the phase of an image is very coherent on high amplitudes because it carries most of the information linked to the structure of the image [3]. Thus, creating a hole in the spectrum harms the phase of the image and the continuity of information. Moreover, the structure of an image is related to the content of this image, but the phase of a periodic noise, which is related to an acquisition or a reconstruction process, is statistically different. In other words, it means that the phase content of the theoretical noise-free image in the Fourier domain is relatively self-consistent [4], but not consistent with that of the noise.

Images based on reconstruction principles often exhibit periodic noises. A good example of this is fringe projection technique images. These images result in the projection of a sinusoidal pattern with an angle on a surface. Then, the image is observed perpendicular to this surface, with a digital camera. Variations of topography induce a phase shift of the sinusoidal pattern, and a phase unwrapping operation allows the three-dimensional reconstruction of the surface.

The chapter is organized as follows. First, some recalls about denoising using Fourier transform are given in one and two dimensions. Second, the principle of spectrum cloning is introduced as an extension of Fourier denoising. In this case too, it is proposed in one and two dimensions. Then, a section presents results on a synthetic example consisting of the Lena image with a periodic noise added. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) measure are used to highlight the possible improvement of the spectrum cloning versus the classical Fourier filtering. The following section deals with an example of denoising on fringe projection technique images. This type of images greatly benefit from Fourier spectrum cloning due to the whole process of image formation.

2. Recalls on Fourier denoising

In this section, we recall the basic principles of Fourier denoising in one and two dimensions. These principles rely on the assumption of additive noise. The model for additive noise is

$$g\left(t\right)=f\left(t\right)+n\left(t\right)$$ E1

where $t$ is the time, $g\left(t\right)$ is the observed signal, $f\left(t\right)$ is the theoretical noise-free signal, and $n\left(t\right)$ is the noise.

This model implies that the noise is a function that does not depend on the signal intensity and as a consequence that it is possible to remove it with a simple subtraction if it is fully characterized. In the case of periodic noise, the exact expression of the noise is not known, but it is well localized in the frequency domain. That is why Fourier denoising using spectrum manipulation is efficient for this type of signals.

2.1 One-dimensional Fourier denoising

Consider the Fourier transform of the signal $\mathit{g}\left(\mathit{t}\right)$ :

$$G\left(f\right)=\underset{-\infty }{\overset{\infty }{\int }}\left(f\left(t\right)+n\left(t\right)\right)e^{-2\mathit{\text{iπft}}}\mathit{dt}$$ E2

As the Fourier transform of a sum of functions is the sum of the individual Fourier transforms, and considering Eq. (1), one can write:

$$G\left(f\right)=F\left(f\right)+N\left(f\right)$$ E3

where $F\left(f\right)$ and $N\left(f\right)$ are, respectively, the Fourier transforms of $f\left(t\right)$ and $n\left(t\right)$ .

One can theoretically recover $f\left(t\right)$ using the inverse Fourier transform and a subtraction if the exact expression of the noise $n\left(t\right)$ is known:

$$f\left(t\right)=\underset{-\infty }{\overset{\infty }{\int }}\left(G\left(f\right)-N\left(f\right)\right)e^{2\mathit{\text{iπft}}}\mathit{df}$$ E4

However, in most cases the expression of the noise is usually not known, so $f\left(t\right)$ cannot be recovered exactly. In order to obtain an approximation $f\left(t\right)$ of $f\left(t\right)$ , it is common to set a part of $G\left(f\right)$ to zero:

$$\stackrel{\sim }{f}\left(t\right)=\underset{-\infty }{\overset{\infty }{\int }}{G}_f\left(f\right)e^{2\mathit{\text{iπft}}}\mathit{df}$$ E5

with

$${\mathit{G}}_{\mathit{f}}\left(\mathit{f}\right)=\left\{\begin{array}{l}\mathit{G}\left(\mathit{f}\right)\mathit{\text{if}}\mathit{f}\in \left(-\infty ,{\mathit{f}}_1\right)\cup \left({\mathit{f}}_2,\infty \right)\\ 0\mathit{\text{if}}\mathit{f}\in \left[{\mathit{f}}_1,{\mathit{f}}_2\right]\end{array}\right.$$

where $f_1$ and $f_2$ are frequency bounds, with $f_1>f_2$ , and $G_f\left(f\right)$ is the filtered spectrum of $G\left(f\right)$ .

2.2 Two-dimensional Fourier denoising

In the two-dimensional case, we consider a two-dimensional spatial function $f\left(x,y\right)\in R^2$ , with $x$ and $y$ as the spatial coordinates. The additive model for the noise is the same, in two dimensions.

The Fourier transform of $f\left(x,y\right)$ is

$$F\left(u,v\right)=\underset{-\infty }{\overset{\infty }{\int }}\underset{-\infty }{\overset{\infty }{\int }}f\left(x,y\right)e^{-2\mathit{i\pi }\left(\mathit{vx}+\mathit{uy}\right)}\mathit{dx}\mathit{dy}$$ E6

The same principle applies, and we can also obtain an approximation $\stackrel{\sim }{f}\left(x,y\right)$ of $f\left(x,y\right)$ with

$$\stackrel{\sim }{f}\left(x,y\right)=\underset{-\infty }{\overset{\infty }{\int }}\underset{-\infty }{\overset{\infty }{\int }}{G}_f\left(u,v\right)e^{2\mathit{i\pi }\left(\mathit{vx}+\mathit{uy}\right)}\mathit{dx}\mathit{dy}$$ E7

with

$${\mathit{G}}_{\mathit{f}}\left(\mathit{u},\mathit{v}\right)=\left\{\begin{array}{l}\mathit{G}\left(\mathit{x},\mathit{y}\right)\mathit{\text{if}}\left({\mathit{x}}^2+{\mathit{y}}^2\right)\in \left(-\infty ,{\mathit{r}}_1\right)\cup \left({\mathit{r}}_2^2,\infty \right)\\ 0\mathit{\text{if}}\left({\mathit{x}}^2+{\mathit{y}}^2\right)\in \left[{\mathit{r}}_1^2,{\mathit{r}}_2^2\right]\end{array}\right.$$

where $r_1$ and $r_2$ are the radii of the part of the two-dimensional spectrum set to zero, with $r1<r2$ , and $G_f\left(x,y\right)$ is the filtered spectrum of $G\left(f\right)$ . It is important to note that in two dimensions it is better to define a two-dimensional torus for the masked part or more adequately two-dimensional torus sectors to take into account the symmetry of the Fourier transform. Actually, in most applications this principle is not applied, and ellipses are used to mask the spectrum peaks.

3. Fourier spectrum cloning

The main drawback of Fourier denoising using spectrum subtraction is that the whole spectrum is removed. Indeed, the operation leaves a hole in the spectrum, which can cause oscillations in the inverse Fourier transform process. In fact, one can easily understand the phenomenon considering the inverse Fourier transform of a rectangular “hole”:

$$\underset{-\infty }{\overset{\infty }{\int }}\beta \left(f\right)e^{é\mathit{\text{iπft}}}\mathit{df}=-\frac{1}{\pi }\mathit{sinc}\left(t\right)$$ E8

where

$$\mathit{\beta }\left(\mathit{f}\right)=\left\{\begin{array}{c}0\mathit{\text{if}}\left|\mathit{f}\right|<1\\ 1\mathit{\text{if}}\left|\mathit{f}\right|>1\end{array}\right.$$

As can be seen in Figure 1, the Fourier transform of such an inverse rectangular pulse is a $\mathit{sinc}$ function which oscillates to the infinity. In addition, the masked part of the spectrum implies that all frequencies in the $\left[f_1,f_2\right]$ or $\left[c_1,c_2\right]$ ranges will not be present in the signal at all.

Figure 1.

(Left) Inverse rectangular pulse in frequency. (Right) Fourier transform of an inverse rectangular pulse in time.

3.1 One-dimensional Fourier spectrum cloning

The purpose of Fourier spectrum cloning is to use the values of spectrum surrounding the masked parts to create a synthetic replacement. More precisely, the part of the spectrum that has been removed is replaced by a mean of surrounding parts. In order to obtain the new approximation $\stackrel{\sim }{f}\left(t\right)$ of $f\left(t\right)$ , one can write

$$\stackrel{\sim }{f}\left(t\right)=\underset{-\infty }{\overset{\infty }{\int }}{G}_{\mathit{fc}}\left(f\right)e^{2\mathit{\text{iπft}}}\mathit{df}$$ E9

with

$${\mathit{G}}_{\mathrm{fc}}\left(\mathit{f}\right)=\left\{\begin{array}{c}\mathit{G}\left(\mathit{f}\right)\mathit{\text{if}}\mathit{f}\in \left(-\infty ,{\mathit{f}}_1\right)\cup \left({\mathit{f}}_2,\infty \right)\\ \frac{\mathit{G}\left(\mathit{f}-{\mathit{f}}_2+{\mathit{f}}_1\right)+\mathit{G}\left(\mathit{f}+{\mathit{f}}_2-{\mathit{f}}_1\right)}{2}\mathit{\text{if}}\mathit{f}\in \left[{\mathit{f}}_1,{\mathit{f}}_2\right]\end{array}\right.$$

where $G_{\mathit{fc}}\left(f\right)$ is the filtered and cloned spectrum of $G\left(f\right)$ . This expression implies that the previous part of the spectrum that was set to zero is now the mean of the part before and the part after the removed portion.

This operation utilizes the information of the signal itself to create false spectrum content. Indeed, it takes into account the nature of the signal which has its own regularity to construct the replacement part of the spectrum.

3.2 Two-dimensional Fourier spectrum cloning

The same principle can apply in two dimensions. To obtain an approximation $\stackrel{\sim }{f}\left(x,y\right)$ of $f\left(x,y\right)$ , one can write

$$\stackrel{\sim }{f}\left(x,y\right)=\underset{-\infty }{\overset{\infty }{\int }}\underset{-\infty }{\overset{\infty }{\int }}{G}_{\mathit{fc}}\left(u,v\right)e^{2\mathit{i\pi }\left(\mathit{vx}+\mathit{uy}\right)}\mathit{dx}\mathit{dy}$$ E10

where

$${\mathit{G}}_{\mathit{fc}}\left(\mathit{u},\mathit{v}\right)=\left\{\begin{array}{c}\mathit{G}\left(\mathit{x},\mathit{y}\right)\mathit{\text{if}}\left({\mathit{x}}^2+{\mathit{y}}^2\right)\in \left(-\infty ,{\mathit{r}}_1^2\right)\cup \left({\mathit{r}}_2^2,\infty \right)\\ \frac{\mathit{G}\left(\mathit{x}-{\mathit{x}}_{{\mathit{r}}_2}+{\mathit{x}}_{{\mathit{r}}_1},\mathit{y}-{\mathit{y}}_{{\mathit{r}}_2}+{\mathit{y}}_{{\mathit{r}}_1}\right)+\mathit{G}\left(\mathit{x}-{\mathit{x}}_{{\mathit{r}}_1}+{\mathit{x}}_{{\mathit{r}}_2},\mathit{y}-{\mathit{y}}_{{\mathit{r}}_1}+{\mathit{y}}_{{\mathit{r}}_2}\right)}{2}\mathit{\text{if}}\left({\mathit{x}}^2+{\mathit{y}}^2\right)\in \left[{\mathit{r}}_1^2,{\mathit{r}}_2^2\right]\end{array}\right.$$

and

$x_{r_i}=\sqrt{r_i^2+y^2}$ and $y_{r_i}=\sqrt{r_i^2+x^2}$ , for $i=\left\{1,2\right\}$ .

As the cloning operation actually creates information instead of the missing part of the spectrum, it can be desirable to add a setting parameter $\alpha$ that will be optimized for the best results. Thus, $G_{\mathit{fc}}\left(x,y\right)$ becomes

$${\mathit{G}}_{\mathit{fc}}\left(\mathit{u},\mathit{v}\right)=\left\{\begin{array}{c}\mathit{G}\left(\mathit{x},\mathit{y}\right)\mathit{\text{if}}\left({\mathit{x}}^2+{\mathit{y}}^2\right)\in \left(-\infty ,{\mathit{r}}_1\right)\cup \left({\mathit{r}}_2^2,\infty \right)\\ \mathit{\alpha }\frac{\mathit{G}\left(\mathit{x}-{\mathit{x}}_{{\mathit{r}}_2}+{\mathit{x}}_{{\mathit{r}}_1},\mathit{y}-{\mathit{y}}_{{\mathit{r}}_2}+{\mathit{y}}_{{\mathit{r}}_1}\right)+\mathit{G}\left(\mathit{x}-{\mathit{x}}_{{\mathit{r}}_1}+{\mathit{x}}_{{\mathit{r}}_2},\mathit{y}-{\mathit{y}}_{{\mathit{r}}_1}+{\mathit{y}}_{{\mathit{r}}_2}\right)}{2}\mathit{\text{if}}\left({\mathit{x}}^2+{\mathit{y}}^2\right)\in \left[{\mathit{r}}_1^2,{\mathit{r}}_2^2\right]\end{array}\right.$$

Practically, this principle should be the best for isotropic or orthotropic periodic noise. However, in real applications, the Fourier spectrum does not present well-localized peaks but more singular lines crossing the zero frequency point and the peaks. As a consequence, it is sometimes better to clone the whole line containing the targeted frequency range.

4. Results on the Lena image

In this section, we present results on a test signal consisting of the Lena image with a sinusoidal noise added. The Lena image can be considered as a natural image because it has been acquired with a camera and then digitalized. In this example the image range is $\left[0,255\right],$ and the noise has an amplitude of $50$ .

One can observe Figure 2 that the Fourier transforms of the noise and the noisy Lena exhibit the vertical lines mentioned in the previous section.

Figure 2.

(a) Lena image and (b) its Fourier spectrum. (c) Noisy Lena image and its (d) Fourier spectrum. (e) Noise and its (f) Fourier spectrum.

Increasing values of $\alpha$ ranging from $0$ to $1$ have been tested in order to observe the evolution of two classical image quality measurement indices. The first one is the peak signal-to-noise (PSNR) ratio. Even if the validity of this metric for human quality perception is discussed today, it remains interesting for specific applications such as images resulting from reconstruction processes. Indeed, these kinds of images, such as fringe projection images, contain geometrical information. As the PSNR is based on the calculus of the mean squared error (MSE), it makes sense to use it on this type of images. The second metric we used is the structural similarity (SSIM) measure. The SSIM has been developed for video quality assessment. It is based on the structure of the images, contrary to the PSNR which is pixel-based. This makes it closer to the human vision which is more attached to the structures contained in images.

4.1 Peak signal-to-noise ratio (PSNR)

The PSNR ratio is defined as

$$\mathit{\text{PSNR}}=10\mathit{Lo}{g}_{10}\left(\frac{\mathit{max}\left(\mathit{\text{image}}\right)}{\mathit{MSE}}\right)$$ E11

where $\mathit{max}\left(\mathit{\text{image}}\right)$ is the maximum possible value in the images and with

$$\mathit{MSE}=\frac{1}{\mathit{m}.\mathit{n}}\sum _{\mathit{i}=0}^{\mathit{m}-1}\sum _{\mathit{j}=0}^{\mathit{n}-1}{\left[\mathit{f}\left(\mathit{i},\mathit{j}\right)-\mathit{g}\left(\mathit{i},\mathit{j}\right)\right]}^2$$

where $m.n$ is the total number of pixels in the image and $f$ and $g$ are the images between those the PSNR is calculated.

As one can observe Figure 3, the optimum value for $\alpha$ is around $0.8$ , with a PSNR of $36.2$ . The value $\alpha =0$ corresponds to the classical case of simple suppression of a part of the spectrum. This example highlights, in this specific case, the differences between classical spectrum masking and the spectrum cloning method in terms of PSNR.

Figure 3.

PSNR values between noise-free Lena and denoised Lena as a function of $\alpha$ .

4.2 Structural similarity (SSIM) measure

The default SSIM [5] between two images $f$ and $g$ is defined by

$$\mathit{\text{SSIM}}\left(f,g\right)=\frac{\left(2{\mu }_f{\mu }_g+c_1\right)\left(2{\sigma }_{\mathit{fg}}+c_2\right)}{\left({\mu }_f^2+{\mu }_g^2+c_1\right)\left({\sigma }_f^2+{\sigma }_g^2+c_2\right)}$$ E12

where ${\mu }_f$ , ${\mu }_g$ , ${\sigma }_f$ , ${\sigma }_g$ , and ${\sigma }_{\mathit{fg}}$ are the local means, standard deviations, and cross-covariance for images $f$ and $g$ . $c_1={\left(k_{1}L\right)}^2$ and $c_2={\left(k_{2}L\right)}^2$ with $L$ the dynamic range of the images and $k1=0.01$ and $k2=0.03$ .

Figure 4 shows that an optimum is reached for $\alpha =0.55$ . This value is lower than the value found for the PSNR. This can be explained by the fact that these two metrics do not focus on the same properties of the image quality after denoising. The authors of [6] discussed this particular point by measuring sensitivities of the two metrics to different types of degradation: PSNR is more sensitive to additive Gaussian noise, whereas SSIM is more sensitive to jpeg compression.

Figure 4.

SSIM values between noise-free Lena and denoised Lena as a function of $\alpha$ .

4.3 Visual assessment

An example of denoising operation on the Lena image with parameter $\alpha =0.8$ gives the result in Figure 5 and with $\alpha =0.55$ in Figure 6. One can observe that even if the noise has been well removed, some artifacts remain in the image. In the two cases, this does not affect the readability of the image, but the artifacts present in the $\alpha =0.55$ case appear more natural to the eye.

Figure 5.

(a) Lena image and (b) its Fourier spectrum. (c) Denoised Lena image with $\alpha =0.8$ and its (d) Fourier spectrum.

Figure 6.

(a) Lena image and (b) its Fourier spectrum. (c) Denoised Lena image with $\alpha =0.55$ and its (d) Fourier spectrum.

5. Results on real applications

In this section, we focus on one application, the fringe projection technique, which benefits highly from the Fourier spectrum cloning denoising.

The fringe projection method has already been described by many authors (see, e.g., [7, 8, 9, 10]). Basically, a periodic pattern of white and black lines is projected on an object; the light is diffused by the object and captured by a CCD video camera. The deformation of the fringes depends on the shape of the illuminated object. In order to observe this deformation, the angle between the projected fringes and the observed diffused light must not be null. The result is a 3D map that can be viewed as an image, with $z$ elevation corresponding to the gray levels.

5.1 Fringe projection technique basics

5.1.1. Optical setup

The fringe projection setup for shape measurement is based on a pocket projector (3M^© MPRO 110), 800 × 600, and a Imaging Source CCD camera, 1280 × 960, 8 bits. This solution is adapted to fields of investigation from 10 × 7 to 200 × 150 mm² (see Figure 7).

Figure 7.

Experimental fringe projection setup.

5.1.2. Basic principle

Light intensities on an object illuminated by a set of fringes can be described by a periodic function $I_{\mathit{li}}$ , with a perturbation corresponding to the object shape:

$$I_{\mathit{li}}\left(x,y\right)=I_0\left(x,y\right)\left[1+\gamma \left(x,y\right)\mathit{cos}\left(\frac{2\pi }{p\left(x,y\right)}y+\Phi \left(x,y\right)\right)\right]$$ E13

where $I_0$ is the average intensity and $\gamma$ is the fringe contrast. These values should be constant over the whole map, but some low-frequency variations due to illumination inhomogeneities or diffusivity changes on top of the surface can occur. Consequently, both average intensity and contrast have to be considered as local quantities, typically calculated over few fringe periods, and can be denoted $I_0\left(x,y\right)$ and $\gamma \left(x,y\right)$ . The fringe period $p$ is the distance between two light peaks on a flat surface. Of course, due to perspective effects, this pitch can change over the map. Last, the object is responsible for a phase shift $\Phi \left(x,y\right)$ at each point of the field that can be written as

$$\Phi \left(x,y\right)=2\pi \frac{\text{tan}\left(\theta \left(x,y\right)\right)}{p\left(x,y\right)}z\left(x,y\right)$$ E14

In this expression, the sensitivity is proportional to the angle $\theta$ between the CCD video camera and the video projector and to the fringe density $1/p$ . As expressed by the equation, the sensitivity varies over the field because usual video projector and the CCD camera commonly use divergent lens. Here, pinhole model parameters were identified through an identification procedure. Readers should refer to [11] for detailed explanations.

5.1.3. Phase extraction

Then, phase extraction is a classical topic in optics applied to mechanics. Considering Eq. (13), extraction of the phase from intensity map(s) can be done from a single image, using a category of methods known as spatial phase shifting [12], but better results are usually obtained using temporal phase shifting techniques. The choice only depends on the situation: if temporal effects are expected, spatial phase shifting is more appropriate, because it only requires one image [13]. If not, temporal phase shifting technique should be preferred for its higher spatial resolution. The Photomechanix toolbox, developed in the laboratory, has genuine implementation of both techniques, as prescribed by Surrel ([14, 15]).

Here, only temporal phase shifting is described: a set of $n\times q$ fringe patterns with a known phase shift $q/2\pi$ is projected successively on the surface, first and last fringe pattern being shifted by a $n\times 2\pi$ , $n\in \mathbb{Z}$ phase. Then, the intensity variation at each point (i.e., each camera pixel) corresponds to a sine wave function with an initial phase shift. The phase is evaluated using the Fourier transform:

$$\Phi \left(r,s\right)={\text{arctan}}_{2\pi }\left(\frac{{\sum }_{k=1}^{\mathit{nq}}\text{sin}\left(k\frac{2\pi }{q}\right)I_k\left(r,s\right)}{{\sum }_{k=1}^{\mathit{nq}}\text{cos}\left(k\frac{2\pi }{q}\right)I_k\left(r,s\right)}\right)$$ E15

This shape measurement setup shows interesting metrological performances compared to the classical techniques (line projection, stereovision): the spatial resolution is 1 pixel (8–156 μm, depending on the field of view), and the resolution is $\sigma =1/100$ th fringe, i.e., 10 μm at best. This capacity is very important for high-frequency phenomena monitoring: a skin submitted to mechanical load, metal instability under forming process, etc.

The signal-to-noise ratio (SNR) being usually high, no further signal processing is required; but some considerations on the quality of the images must be done. If the illumination is not controlled, then the sine wave is distorted. Another noise source lies in the phase shift: a drift would add noise, as demonstrated by Cordero [16]. The consequence in both cases is that parasitic harmonics enter in the shape field. Surrel proposed an algorithm robust to phase drift [14]; Kemao published a procedure to characterize the intensity period and remove most of the harmonics [17], but he used a strong assumption on intensity modeling that is not always completed. As a matter of fact, it is commonly admitted that a careful tuning is the best solution.

5.1.4. Experimental test: Case 1. Digitalization of a bas-relief

Arts have already been an important field of applications of fringe projection. For example, the support stability of the Mona Lisa paint has been evaluated by [18], but wider projects of heritage object recording should be contemplated [19]. In this specific case, obviously, no surface preparation is possible before scanning, and illumination is an issue. Here, we illustrate a possible drawback with a bas-relief that has to be scanned and duplicated. A time-shifting approach was used in order to get the better spatial resolution, but an image turned to be corrupted, resulting in a phase shift drift.

Figure 8 shows a photograph of the bas-relief (a) as well as the basic shape reconstruction (b). Parasitic fringes are clearly visible because of its structure, even if the intensity is very low compared to the heights in the field. In this first example, even if some noise remains in the final image, the global shape is not affected. The final objective being the duplication, it is better to refine some parts of the 3D model by hands after a first denoising operation that does not introduce structure errors.

Figure 8.

Bas-relief intensity image (a) and shape (b). The classical Gaussian low-pass (σ = 10 px) (c) filter shows better parasitic fringe removal but loses details compared to the FSC filter (μ = 110, δ = 22) (d).

5.1.5. Experimental test: Case 2. Skin characterization

The skin is a challenging topic for topology reconstruction. Skin structure is multi-scale, with a global shape containing wrinkles and fine lines. Each scale has its own topological properties, in particular the orientation, and experts would like to separate wrinkles and fine lines because the dermatologic treatment associated to each is different.

Besides these characteristics that are followed as a marker of cosmetic efficiency, it is important to note that the light diffusion of the skin is not perfect for fringe projection. Moreover, it depends on many parameters that should be considered as natural (melatonin concentration, skin moisture, tobacco, etc.) or interventional (cleaning procedure, cosmetic treatment). Then, it is difficult to change the skin surface for the sake of better experimental conditions, and the physicist has to adapt the signal processing to these conditions.

A particular point in skin texture analysis is the global amplitude of the shape variations. On a square-centimeter area, elevation variations are typically close to $\mp 1$ mm, but the wrinkle roughness should be as small as $\mathit{\text{σRMS}}\approx 10$ μm. Consequently, in some cases, the roughness might be only $t\mathrm{en}$ times higher than the noise, and if this noise has an organized texture, it interferes in the human perception of the surface topology.

We propose here two illustrations from Lorica™ replica of the skin taken on the forearm or on the forehead. On the basic reconstruction, it is possible to distinguish some periodic lines almost horizontal. These lines can be associated with the fringes considering their orientation and wavelength. A classical way of removing noise in this case is to use a Gaussian low-pass filter. Here, it has been set to $\sigma =10$  px (pixels) according to the user’s practice.

Qualitatively, both filters remove the targeted parasitic lines. The Gaussian filtered image seems blurry, as it could be expected, while the Fourier spectrum cloning (FSC) filter seems to respect the image sharpness. Quantitatively, three basic topographic data are extracted for the whole image: a mean roughness indicator (RMS roughness, $\mathit{\text{σRMS}}$ ), a pic-to-valley indicator (skewness, $\mathit{Sk}$ ), and a shape indicator (kurtosis, $\mathit{Kt}$ ). In this situation, it is worth recording that there is no ground truth. It can be observed after all that $\mathit{\text{σRMS}}$ and $\mathit{Sk}$ are dramatically changed with Gaussian filter and not with the FSC filter (Figures 9 and 10).

Figure 9.

Intensity image of the skin taken at the forearm (a) and texture reconstruction of the skin after high-pass filtering to remove the global shape σ_RMS = 20.1 μm, Sk = −0.231, Kt = 3.274 (b). The classical Gaussian low-pass (σ = 10 px) σ_RMS = 18.0 μm, Sk = −0.169, Kt = 3.253 (c) filter shows good parasitic fringe removal but loses details compared to the FSC filter (μ = 48, δ = 10) σ_RMS = 20.0 μm, Sk = −0.225, Kt = 3.287 (d).

Figure 10.

Intensity image of the skin taken at the forehead (a) and texture reconstruction of the skin after high-pass filtering to remove the global shape σ_RMS = 38.5 μm, Sk = −0.013, Kt = 3.411 (b). The classical Gaussian low-pass (σ = 10 px) σ_RMS = 37.6 μm, Sk = −0.008, Kt = 3.364 (c) filter shows good parasitic fringe removal but loses details compared to the FSC filter (μ = 42, δ = 12) σ_RMS = 38.5 μm, Sk = −0.013, Kt = 3.410 (d).

6. Conclusions

In this chapter, we proposed the Fourier spectrum cloning principle. After some recalls about Fourier denoising, we gave the basis of Fourier spectrum cloning with a tuning parameter $\alpha \in \left[0,1\right]$ allowing to choose the amount of cloned spectrum. We proposed to measure the performances of the algorithm on the Lena image without optimization to observe the benefits of the method. To do so we used the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) which are good metrics for measuring differences between images.

Fringe projection has been chosen as a first application field. The analysis of the skin microreliefs (wrinkles, fine lines) requires an optimal system and a good post-processing, the signal-to-noise ratio being limited. The potential of FSC filter is clearly outlined: a periodic noise can be removed and make the image easier to interpret, without major changes in the topographical characteristics. Anyway, in this application, only one frequency band has been removed, and a multiple choice could be necessary in practice; interactions between various filtering processes would have to be studied then.

As a conclusion, this chapter aims at presenting a simple concept and giving some results and interpretations. Many refinements can be implemented in the future, in order to improve these results obtained with the simplistic application of the cloning principle. Actually, the construction of the synthetic replacement part of the spectrum could be synthesized considering different parameters such as border effects or statistical measures on the spectrum. Further research will address these different paths.

Acknowledgments

This work has been partially funded by the French National Research Agency via the LBSMI project ANR15-CE19-0002.

The authors would like to thank G. Boyer, Laboratoires Expanscience, and Nicolas Curt, Sainbiose, for their contribution to the experimental data.