Blind Image Restoration for a Microscanning Imaging System

Image restoration is an important topic in the area of image processing because its techniques are useful to recover images degraded during a capturing process (Bovik, 2005). There are a wide range of degradations in real world, such as blurring (i.e. camera motion capture process), nonuniform illumination, cloud environments (fog, clouds, smoke), noise (white noise, impulse noise, etc.), and damaged elements in imaging array sensors (Gonzalez & Woods, 2008; Hautiere & Aubert, 2005; Jain, 1989; Narasimhan & Nayar, 2002).


Introduction
Image restoration is an important topic in the area of image processing because its techniques are useful to recover images degraded during a capturing process (Bovik, 2005).There are a wide range of degradations in real world, such as blurring (i.e.camera motion capture process), nonuniform illumination, cloud environments (fog, clouds, smoke), noise (white noise, impulse noise, etc.), and damaged elements in imaging array sensors (Gonzalez & Woods, 2008;Hautiere & Aubert, 2005;Jain, 1989;Narasimhan & Nayar, 2002).
Common restoration methods are based on a priori knowledge of the degradation process.They usually use the degradation model and a single observed scene to carry out restoration (Banham & Katsaggelos, 1997;Kundur & Hatzinakos, 1996).There are also methods of restoration based on unknown functions of degradation referred to as blind methods (Jain,1989).Although there are numerous algorithms, the process of restoration is still open problem.
Image restoration methods described in this chapter belong to the class of blind adaptive methods.The techniques use camera microscanning (Shi et al., 2006) for the restoration of images degraded with nonuniform additive, nonuniform multiplicative interferences, and sensor noise.
The spatial nonuniform additive interference is present in infrared focal-plane array sensors (IFPA), because each photodetector has a variation in its photoresponse as intrinsic result of the IFPA's fabrication stage (Hayat et al., 1999;Ratliff et al., 2002).On the other hand, the nonuniform illumination may be characterized as multiplicative interference.Nonuniform illumination limits the performance of others algorithms of image processing such as pattern recognition (Lee & Kim, 2009).
Microscanning is a technique to acquire time-sequential images of the same scene with a slight shifting between the scene and camera.Recently, restoration methods for different models of observed scenes using three images anisotropically captured with a microscanning imaging system were investigated (López-Martínez & Kober, 2008, 2010;López-Martínez et al., 2010).In order to carry out restoration, we consider three degraded images captured with a microscanning imaging system.Next, an explicit system of equations is derived and solved.

Image restoration methods for a microscanning system
Microscanning acquires multiple images of the same scene by slight shifting image acquisition system (Shi et al., 2006).Microscanning can be implemented either with a controlled movement of a sensor array that captures images or with a controlled motion of a light source, for example in the case of nonuniform illumination.
Microscanning is usually used for supper-resolution (Milanfar, 2011).However, controlled camera microscanning can be also used for image restoration, if the original image and interferences are spatially displaced relatively each other during the microscanning process.At least three observed images should be captured.The first image is taken without any displacement, the second one is captured with shift of one pixel down, and finally the third one is obtained with shift of one pixel to the right.In practice, microscanning may be implemented using a piezoelectric actuator to precision positioning of a sensor array.In the case of nonuniform illumination, controlled motion performs a light source.
Let us introduce some useful notation and definitions.Let   (, ) , 1 ,2 , . ., t sij tT  be a set of observed images, where t is the index of time-sequential images captured during microscanning, T is the number of observed images captured around the origin with a small displacement of a camera, and (i, j) are the pixel coordinates.Without loss of generality, suppose that each image has the size of M N  pixels.Let   (, ) ai j and   (, ) bi j denote an original image, an additive interference, and a multiplicative interference, respectively.Assume that these images are time-invariant during the capture process.Let

  (, )
t nij be a time-varying zero-mean white Gaussian noise.

Additive degradation model
An example of spatially nonuniform additive interference is IFPA with a low gain variation (Hayat et al., 1999;Ratliff et al., 2002).IFPA sensor is a mosaic of photodetectors placed at the focal plane of an imaging system.It is known that the performance of IFPA sensors is affected by the presence of fixed-pattern noise (spatially nonuniform noise).The nonuniform noise occurs because each detector has the photoresponse slightly different from that of its neighbors.
When image degradation is caused by additive nonuniform interference and additive noise, the observed scene can be described as With a help of the technique of microscanning, two frames with vertical and horizontal displacements of one pixel can be obtained as follows: The additive interference and the original image are spatially displaced by the microscanning.Let us compute gradient matrices as follows: and .
(, 1 ) (, 2 ) (, 2 ) (, 3 ) (, 1 ) (,)0 We want to minimize the additive noise variance contained in these matrices.So, the objective function to be minimized using the least-squares approach (Kay, 1993), is given as where the first terms takes into account the noise information present in the most part of the image, and the last two terms are inserted to the objective function in order to take into account the noise information in the bottom row and the right column of the image, respectively.To solve the minimization problem, we differentiate the objective function with respect to elements of the image   (, ) f i j and set derivatives equal to zero.The minimization of the objective function leads to a linear system of equations.In matrix-vector notation the linear system is given by where matrix A has the size MNM N  , x is a vector version of   The matrix A is sparse, and it is calculated as 13 323 323 323 31 0000 000 00 0 00 0 000 0000 where the matrices 1 and The rank of the matrix A is 1 MN  , therefore the original image can be restored if one pixel of the image is a priori assigned to a constant, for instance the last pixel of the image is set to zero.So, the matrix A associated to the lineal system has the size 1 1 MN MN   , and it www.intechopen.combecomes symmetric and positive-definite.In this case there exists a unique solution.After solving the linear system, the obtained image is point-wise processed to have the same mean value (assumed to be known) with original image.To solve the linear system, iterative conjugate gradient method is used (Golub & Van Loan, 1996).
The computational complexity of the algorithm is given by the execution order of conjugate gradient and the size of an image to be restored.It is estimated as   kp  operations, where p is the number of nonzero entries in the matrix associated to lineal system, and k is the number of iterations required for solving the system of equations.Without loss of generality, we assume that M N and kq M  where q depends on precision of the solution.The computational complexity of the method can be estimated as Impulse noise is caused by sensor failures in a camera or transmission through a noisy channel.The proposed method is able to interpolate implicitly the pixel values corrupted with impulsive noise based on the information contained in neighboring pixels.This is because, during the microscanning the information of each pixel of the original image is captured in three different observed images.If one of sensors is damaged, partial information about the pixel intensity of the original image could be available in the other observed images.

Multiplicative degradation model
A typical example of multiplicative interference is nonuniform illumination.When an input image degraded by a multiplicative nonuniform interference and additive noise, the observed scene can be described as Suppose that microscanning is able to separate the original image and the interference, and then the shifted frames are obtained as follows: Now we define two quotient matrices using spatial information between rows and columns of the images, Hence, the matrices   (1,1) (1,1) The multiplicative interference in the matrices   (, ) hi j and   (, ) vi j is eliminated when the observed images have no additive noise.However, for small standard deviation of noise the matrices are close to the correspondent quotient matrices constructed with the original image and its shifted versions.In a similar manner, the objective function can be written as .
Since it is difficult to solve the system of nonlinear equation then a logarithm transformation to the system of nonlinear equations is applied.In this way the system can be converted to the linear system as follows: In a similar manner, the iterative conjugate gradient method can be used for solving this linear system.Finally, the restored image is obtained by applying the exponential function to the solution of the linear system.Since this method is based on the conjugate gradient method, its computational complexity is close to that of for the additive degradation.Additional expenses are required for logarithm and exponential transformations.

Computer simulation results
In this section computer simulation results for restoration of images degraded by additive and multiplicative interferences are presented.The root mean square error (RMSE) criterion is used for comparison of quality of restoration.Additionally, a subjective visual criterion is used.The RMSE is given by ,, where   (, ) fij is the original image and   is the restored image.The size of images used in our experiments is 256 256  pixels.The intensity values are in the range of [0,255].The experiments were performed using a laptop computer (Intel Core 2 Duo 2.26 GHz with 2GB of RAM).To guarantee statistically correct results, 30 statistical trials of each experiment for different realizations of the random noise process were performed.The conjugate gradient method is used to solve the linear system.The convergence criterion is when the residual value drops below 10 10  .The subjective visual criterion is defined as an enhanced difference between original and restored images.This enhanced difference (Kober, 2001) is defined as follows: where 1  and 2  are predetermined constants.In our experiments we set 1 4   and 1 1   for additive and multiplicative models, respectively, and 2 128   . A pixel is displayed as gray if there is no error between the original image and the restored image.For maximum error, the pixel is displayed either black or white (with intensity values of 0 and 255, respectively).
The linear minimum mean square error method is a popular technique in image restoration.In the case of stationary processes and in the absence of any blur, the method takes a simplified form of the Wiener smoothing filtering (Jain, 1989)

304
method does not need any information about the degradation function.When observed images are degraded by multiplicative interference, first we apply a logarithm function to the degraded images to convert the multiplicative interference to additive one (ignoring the sensor noise).Then the empiric Wiener filtering with known parameters is utilized.Finally, the exponential function is applied to the Wiener restored image to obtain the output image.

Restoration of noisy image degraded with additive interference
Figs. 1(a), 1(b), and 1(c) show a test original image, a nonuniform additive interference, and the original image degraded with the interference and a zero-mean white Gaussian noise with a standard deviation of 2, respectively.The mean value and standard deviation of the interference are 118.8 and 59.7, respectively.Fig. 2(a) shows restored images with the proposed method.Fig. 3 shows the performance in terms of the RMSE of the proposed methods using three images (Am3), and the Wiener filter versus the standard deviation of additive noise.It can be seen that the performance of the proposed method is much better than that of the Wiener filtering with known parameters.It happens because the additive interference is spatially inhomogeneous, and therefore, it cannot be considered as a realization of a stationary process and correctly used in the filtering.The time required to restore the image with the proposed method is approximately 46 sec.The iterative conjugate gradient algorithm requires about 1070 iterations.Note that outside of damaged elements the restoration performance of the proposed method is good.At the location of the damaged elements the method carries out a smooth interpolation using information containing in neighboring pixels.

Restoration of noisy image degraded with nonuniform illumination
where  is the slant angle,  is the tilt angle, and r is the magnitude of the vector from pointlight source to surface.These parameters define the position of the light source with respect to the surface origin.In our simulations, we set 5

 
1.1,1.5, 2 r  . Table 1 shows the values taken by the illumination function in our experiments.The mean value and standard deviation of the original image are 112.3 and 50, respectively, with maximum and minimum values of 237 and 17, respectively.Fig. 6 shows a test original image.Figs. 8, 10, and 12 show the performance in terms of the RMSE of the proposed methods using three images (Mm3) and the Wiener filter versus the standard deviation of additive noise with different parameters of illumination.One can observe that the proposed method is useful when the standard deviation of additive noise is low.Note that the performance of the proposed method is essentially better than that of the Wiener filter, which is designed with known parameters.Time required to restore the image using the Mm3 is approximately 51 sec.In this case, the iterative conjugate gradient algorithm requires about 1070 iterations.

Experimental results
Here we present experimental results with a real-life image degraded by a multiplicative interference.The observed images were obtained as follows.A test image was displayed on a LCD screen.A printed transparency was placed between a camera and the screen in order to simulate a multiplicative degradation.Microscanning was performed by shifting the test image on the screen.Finally, the three observed images were captured with the camera.The observed images have the size of 256 × 256 pixels.First, the observed images were passed through the logarithmic transformation.Next, the proposed method was utilized to obtain a resulting image.Finally, the exponential transformation was applied to the resulting image to restore the original image.The original image, multiplicative degradation, and one of the observed images taken by a camera are shown in Figs.13(a), 13(b), and 13(c), respectively.The restored image is presented in Fig. 13(d).
Since in the experiment the level of additive noise is low, the quality of the restoration with the proposed method is very good.

Conclusion
In this chapter we presented methods for restoration of images degraded with additive and multiplicative interferences, and corrupted by sensor noise.Using three observed images taken with a microscanning imaging system, an explicit system of equations for additive and multiplicative signal models was derived.The restored image is a solution of the system.With the help of computer simulations we demonstrated the performance of the proposed method in terms of restoration accuracy and execution time.The performance of the proposed method is essentially better than that of the Wiener filter, which is designed with known parameters Figs. 1(a), 1(b), and 1(c) show a test original image, a nonuniform additive interference, and the original image degraded with the interference and a zero-mean white Gaussian noise with a standard deviation of 2, respectively.The mean value and standard deviation of the interference are 118.8 and 59.7, respectively.Fig. 2(a) shows restored images with the proposed method.Fig. 2(b) shows enhanced difference between the original image and the restored image.

Fig. 3 .
Fig. 3. Performance of the proposed method for additive degradation: RMSE versus a standard deviation of additive noise.

Fig. 7 .
Fig. 7. Nonuniform illumination correction with the proposed method: (a) observed image degraded with multiplicative interference and white noise with a standard deviation of 1, (b) multiplicative interference with 1.1 r  , (c) restored image, (d) enhanced difference between the original image and the restored image.

FigFig. 11 .
Fig. 10.Performance of the proposed method when 5    , 245    , and 1.5 r  : RMSE versus a standard deviation of additive noise.

Fig. 12 .
Fig. 12. Performance of the proposed method for multiplicative degradation with parameters of illumination of 5    , 245    , and 2.0 r  : RMSE versus a standard deviation of additive noise.
is the power spectral density of additive interference.It is assumed that all degradation parameters for the Wiener filter are exactly known.Note that the proposed NoisePu v Nonuniform illumination is an example of multiplicative interference.In our experiments we use the Lambertian model of illumination, which reflects light equally in all directions.

Table 1 .
Values of the illumination function.