Transformations of Image Filters for Machine Vision Using Complex-Valued Neural Networks

Neural networks expanded to complex domains have recently been studied in the field of computational intelligence. Complex-valued neural networks are effective for learning the relationships between complex inputs and outputs, and applications to complex analysis and complex image processing have been studied (Hirose, 2006). In addition, the effectiveness of the computational complexity and the number of training data has been confirmed when learning mappings in two-dimensional space (Nitta, 1997). Also, a method for complex-valued network inversion to produce an inverse mapping was proposed as a related technique using a complex-valued neural network (Ogawa, 2009). We can obtain forward mappings and inverse mappings in complex domains using these methods.


144
Complex-valued neural networks are effective for processing data in a coordinate system where the phase rotates or for learning relationships in the frequency domain. Applications of complex-valued neural networks include adaptive design of patch antennas (Du et al., 2002), radar image processing (Hara & Hirose, 2004), and traffic-dependent optimal control of traffic signals (Nishikawa & Kuroe, 2004).

Complex-valued multilayer neural networks
In this study, complex-valued multilayer neural networks are used for filter transformation based on their capacity for mapping. Complex-valued multilayer neural networks are an extension of the usual multilayer neural networks to complex regions. This method determines the relationships between complex inputs and outputs using complex neurons and complex weights. They are typically composed of an input layer, some hidden layers, and the output layers. All functions can be realized in complex-valued multilayer neural networks if there is at least one hidden layer and a normal multilayer network. In this study, we used a neural network with three layers, namely, input layer, hidden layer, and output layer, for simulation.
In this study, we considered a multilayer neural network based on an error backpropagation learning method. This model learns complex input-output relationships using complex weights and complex neurons. Complex-valued neural networks are classified on the basis of their architecture and the type of neurons found in these networks. For instance, one type of complex-valued neural networks is based on the transfer function of the neuron, while another consists of real-type neurons. Here, we consider a neuron that independently applies a sigmoid function to the real and imaginary parts of the weighted sum of inputs. This neuron independently applies a complex sigmoid function to each real part and imaginary part, which can be defined as where i and s = s R + is I indicate the imaginary unit and the weighted sum of the neuron input, respectively. The architecture of the complex-valued sigmoid neuron is shown in Fig.  1. The neuron transforms the weighted sum s = s R + is I of the input x n = x nR + ix nI and the weight w n = w nR + iw nI to the output f c (s)=f(s R )+if(s I ) using the sigmoid function of equation (1). In this network, complex-valued neurons are used for the hidden layer and the output layer.
Complex-valued multilayer neural networks are usually used in two phases of learning and estimation, as shown in Fig. 2. In the learning phase, we provide the training input and output, and model the forward relation using the error backpropagation algorithm. During the estimation phase, we obtain the output for a given input by fixing the weights obtained in the leaning phase.
Next, we explain learning in a complex-valued neural network using the error backpropagation algorithm, which is enhanced for complex regions. Here, we consider a three-layer network with an input layer, a hidden layer, and an output layer. The output error E = E R + iE I is defined by the squared error as www.intechopen.com where y' r = y' rR + iy' rI and y r = y rR + iy rI are the r-th tutorial output and the network output, respectively. First, we formulate the weight update procedure between the hidden and output layers. The error signal rr R r I i δδ δ =+from the output layer is calculated by rR  rR  rR  rR  rI  rI  rI  rI  rI   yy  y  y  yy  y  y  11 , 11 Also, the gradient of the output error for the weight w rk = w rkR + iw rkI between the hidden and output layer is expressed by where v k = v kR + iv kI indicates the input from the k-th hidden neuron. Therefore, the weights are updated by where ε t denotes a training gain. In this way, complex weights are updated between the hidden and output layers. Next, we formulate the weight update procedure between the input and hidden layers.
Also, the gradient of the output error for the weight w km = w kmR + iw kmI between the input and hidden layer is expressed by RI  R  I  kR mR  kI mI  kR mI  kI mR  kmR  kmR  kmI  kmI   EE  E  E  xx  x  x  ww  w where x m = x mR + ix mI indicates the input from the m-th input neuron. Therefore, the weights are updated by where ε t is a training gain. In this way, the complex weights between the input and hidden layers are updated. The input-output relationship is learned by correcting each complex weight according to the above equations. In general, the afore-mentioned weight correction is repeated until a prescribed error value or repetition number. The principle of weight correction is based on the output error, as shown in Fig. 3.
The output is estimated from a given input using the learned complex-valued multilayer neural network. In the network studied, the output corresponding to the given input can be www.intechopen.com estimated by fixing the weights obtained during learning with a given input and obtaining the output.

Complex-valued network inversion
The inverse problem determines the inner mechanism or cause of an observed phenomenon. The concept of the inverse problem is shown in Fig. 4

www.intechopen.com
Transformations of Image Filters for Machine Vision Using Complex-Valued Neural Networks 147 result is determined from a given cause by using a certain fixed mathematical model in the forward problem (Groetsch, 1993). We consider the estimation process with the neural network from the viewpoint of the direction of the problem. A multilayer neural network is a normal solution for the forward problem because it estimates the output from the input based on the forward relationship obtained by training. In contrast, an inverse problem can be solved using a multilayer neural network inversely with the forward relationship obtained during training.
A normal multilayer neural network can be used to solve the forward problem. Given a normal multilayer network where training is completed, the input-output relationship is given by y = f (w, x), where x, y, and f are the input vector, the output vector, and the function defined by the interlayer weights w of the network, respectively. Given the input vector x, the network calculates the output vector y. Linden and Kindermann proposed a method for network inversion (Linden & Kindermann, 1989). Using this method, we can determine the observed output data y with f fixed after finding the forward relationship f by training. Then, the input x can be updated according to the calculated input correction signal, based on the duality of the weights and input. The input is actually estimated from the output by the iterative updating of the input based on the output error. In this way, the inverse problem for estimating input x from output y is solved with a multilayer neural network by using the forward relationship inversely. Further, network inversion has been applied to image restoration (Valova et al., 1995) and the inverse kinematics of robot arms (Ogawa & Kanada, 2010).
The network is used in two phases, forward training and inverse estimation, to solve the inverse problem by network inversion, as shown in Fig. 5. During the training phase, we provide the training input x and the training output y and calculate the output error E. Then, the weight w is updated by where ε t is the training gain, because the output error is due to maladjustments of the weights. The forward relationship is obtained by repeating this update procedure. This procedure is based on the usual backpropagation method. During the inverse estimation phase, we fixed the relationship obtained during training, given the random input x and the test output y, and we calculated the output error E. The input x is then updated by where ε e denotes the input update gain, because the output error is due to the error of the input. The input is estimated from the output by repeating this update procedure. The principle of input correction is shown in Fig. 6.
A complex-valued network inversion is an extension of the principle of the network inversion to a complex number. In a complex-valued network inversion, the network inversion technique extended to a complex number is applied to a learned complex-valued multilayer neural network to solve inverse problems. As a result, the complex input is estimated from a complex output, which is given to a learned complex-valued neural network. The network learns the input and output relationship using the error backpropagation algorithm extended to the complex region, which is explained in the previous section.
During the inverse estimation phase, the input is estimated from the given output. Thus, the provided initial random input is repeatedly updated by the output error, which is backpropagated to the input via the fixed weights. To provide an initial random input where y r = y rR + iy rI and y' r = y' rR + iy' rI indicate the network output and tutorial output, respectively. The error signals from the output and hidden layers are also formulated as where v k = v kR + iv kI indicates the input from the k-th hidden neuron to the output neurons.
Equations (12) and (13) are similar to equations (3) and (6) where x m = x mR + ix mI indicates the input from the m-th input neuron to the hidden neurons.
The error signal δ m = δ mR + iδ mI indicates a demand for input correction to the m-th input neuron, so the complex inputs are iteratively corrected as where ε e is the inverse estimation gain. When the error reaches the target, the input correction is terminated and the obtained complex input becomes a solution. As a result, the complex input can be inversely estimated from the complex output by using the complex weight distribution obtained during training. This is similar to correcting the weights or the input iteratively during training and inverse estimation. However, the inverse estimation uses iterative corrections for a given pattern, which differs from training by repeated correction of plural patterns.

Learning of complex-mapping by complex-valued neural networks
A multilayer neural network can learn the nonlinear mapping relationships between inputs and outputs. A complex-valued multilayer neural network and a complex-valued network inversion can estimate, respectively, the forward mapping and the inverse mapping between an input and output. In this study, we examined the forward and inverse www.intechopen.com estimation problems using these networks. In the complex-mapping problem, it is important that learning is related to the mapping of points on a complex plane. A complex-valued neural network has the particular advantage of learning a map with a coordinate system that rotates. We consider the use of complex-valued neural networks by replacing the mapping with a transformation of the rotation with a mapping on a complex plane.
We consider the problem of mapping a point on a complex plane using a network with an input and an output. The network learns the input/output relationship using a given complex learning pattern for input and output learning. During the forward estimation of a complex-valued multilayer neural network, we provide the input to obtain the output by fixing the weights obtained in learning. During the inverse estimation of the complexvalued network inversion, we provide a complex random input pattern and iteratively correct the input from a given complex output pattern by fixing the weights obtained in learning.
Various kinds of information can be expressed by assigning meanings to the coordinates in complex mapping problems. In this study, we examined the allocation of the values of image filters to a complex plane and performed various conversions. We can generally implement filters that achieve non-symmetric filtering and direction-related filtering. These are effective tools in machine vision. In this study, we considered the conversion of an image filter using complex-valued neural networks.
We examined the linear transformation of expansion/compression and rotation, and general projective conversion, which are important methods for transforming image filters. First, we considered a figure g(x, y) in two-dimensional space, where each value of the coordinates (x, y) is moved to coordinates (x', y') repeatedly. Generally, a linear transform is expressed by using a matrix expression. Various linear transforms can be executed by choosing the four parameters a, b, c, and d. Here, we set the parameters b = c = 0 as to consider the transform of expansion/compression shown in Fig. 7 (a). The parameters a and d indicate the rate of expansion/compression in the x-direction and y-direction, respectively. Expansion is indicated if these parameters are larger than 1.0, whereas compression is indicated if they are lower than 1.0. The transform of rotation is given as where θ denotes the counterclockwise angle around the origin, as shown in Fig. 7 (b). These are linear transforms, and they are realized by simple parameter estimation; however, we need to learn and estimate these transforms to illustrate the procedure for complex-valued , which includes nine parameters a ij . An example of a projective transform is shown in Fig. 7 (c). In this study, we consider these three transforms as geometric transforms on a complex plane, and we use them for estimation with complex-valued neural networks.

Image filters
Image filters are an important tool in the fields of bio-inspired systems, robot vision, and image processing. Relevant information is extracted or unnecessary information is removed from a given image with various image filters. In robot vision, it is useful if the image filter can transform or adaptively map when making a necessary filter. In this section, we propose to adaptively map an image filter using a neural network to make an appropriate filter. Complex-valued neural networks are known to effectively map using less training data. Various filters can be made by applying the Gaussian function using the above-mentioned function, which basically provides a low-pass filter. For instance, a high-pass filter is produced by assuming Huv 1( , ) − . Moreover, a filter of the lateral inhibition type can be produced based on the difference of two Gaussian functions with different standard deviations. This is called the DOG function and it can be used as an edge enhancement filter.
Moreover, an orientation selectivity filter can be produced from the rotated elliptic function by compressing and rotating a Gaussian function. This filter can extract a line segment or filter an image in a specific direction. A filter with various features can be produced by variously compressing and rotating the Gaussian filter. Thus, linear transformations and nonlinear transformations can be produced by learning using neural networks. As an example, we consider a nonlinear conversion, such as a projective transform. Fig. 8 shows examples of the plot of 2-D filter functions related to Gaussian functions: a low-pass filter, a high-pass filter, a lateral inhibition filter based on the difference of Gaussian functions, and an orientation selectivity filter that is obtained using an elliptical Gaussian function.

Simulation
We conducted a simulation of a forward and inverse transform of the filter using forward and inverse mappings with a complex-valued multilayer neural network and a complexvalued network inversion. First, we examined the geometric conversions namely, the expansion/compression, rotation, and projective conversions, on a complex plane to demonstrate the learning procedure and the estimation of the complex mapping. The result was a transformation of the geometric figure, the movement vector on the complex plane, and a conversion of the filter. In addition, we filtered the image with the transformed filter and showed the effect of the filter conversion with complex-valued neural networks.
We use a network with an input and an output, and we considered the problem of mapping one point to another on a complex plane. We provided complex input and output patterns and made the network learn the input/output relationship during learning. Then, we examined the forward estimation of the multilayer neural network and the inverse estimation using the network inversion of the learned network. Thus, we obtained an output from a given input by using the learned relation as it was and obtained the input from the given output by using the learned relation inversely. The network parameters and network architecture are presented in Table 1 and Fig. 9, respectively. The following section shows the results of the expansion/compression conversion, the rotational transformation, and the projective conversion.

Expansion/compression transform
We examined the learning, forward estimation, and inverse estimation of the expansion/compression transform on a complex plane with the complex-valued neural networks. Both symmetric and non-symmetric transforms along the vertical and horizontal axes are important for transforming an image filter such as the orientation selectivity filter. Thus, symmetric transforms and non-symmetric transforms are examined here. As the learning data, we provided 22 pairs of data that satisfied xa x ′ = , y dy ′ = , y x = , y x =− , where a and d are constants, and (x, y) and (x', y') are points before and after the transform, respectively. We prepared the 11 points x = (-1.0, -0.8, ..., 1.0).
First, we examined the learning, forward estimation, and inverse estimation of the symmetric expansion/compression transform by setting parameters a = d = 0.5. The forward mapping vector by forward estimation and the inverse mapping vector by inverse estimation are shown in Figs. 10 (a) and (b), respectively. Based on these results, it was confirmed that the transform vectors were correctly obtained. In Fig. 11, we show the results of the forward and inverse estimation of the expansion/compression transform of a circle, whose radius and center are 0.5 and the origin, respectively. Based on these results, it was confirmed that the forward and inverse estimations were correctly conducted. In addition, Fig. 12 shows the results of the forward and inverse estimation of the expansion/compression of the Gaussian function. Based on these results, it was found that the expansion/compression is correctly conducted. Therefore, it was shown that a complexvalued neural network can realize the forward and inverse transforms of expansion/compression. Next, we examined the learning, forward estimation, and inverse estimation of an asymmetric expansion/compression transform by setting parameters a = 1.0, d = 0.2. Fig. 13 shows the forward mapping vector and the inverse mapping vector obtained by learning. Based on these results, it was found that the expansion/compression was applied only along the y-axis. Fig. 14 shows the results of the forward and inverse estimation of the expansion/compression transform of a circle, whose radius and center are 0.5 and the origin, respectively. Based on these results, it was found that the forward and inverse estimations of the expansion/compression were correctly shown to be an oval distribution of the estimated points. In addition, Fig. 15 shows the results of the forward and inverse estimation of the expansion/compression of the Gaussian function. Therefore, it was shown that a complex-valued neural network can realize the forward and inverse transforms of the asymmetric expansion/compression.

Rotation transform
We examined the learning, forward estimation, and inverse estimation of the rotation transform on a complex plane with complex-valued neural networks. As learning data, we provided 22 pairs of data that satisfied yx y sin cos θθ ′ =+ , y x = , y x =− , where (x, y) and (x', y') are the points before and after the transform, respectively. We prepared the 11 points x = (-0.5, -0.4, ..., 0.5).
We examined the learning, forward estimation, and inverse estimation of the rotation transform by setting parameter θ = 45°. Fig. 16 shows the forward mapping vector and the inverse mapping vector obtained by learning. Based on these results, it was confirmed that the transform vectors were correctly obtained. Fig. 17 shows the results of the forward and inverse estimation of the rotation transform of an ellipse whose major axis, minor axis, and center were 0.5, 0.125, and the origin, respectively. Based on these results, it was confirmed www.intechopen.com  www.intechopen.com that the forward and inverse estimations were correctly conducted. In addition, Fig. 18 shows the results of the forward and inverse estimation of the rotation of an elliptic Gaussian function. Based on these results, it was found that the rotation was correctly conducted. Therefore, it was shown that a complex-valued neural network can realize the forward and inverse transforms of the rotation.

Projective transform
We examined the learning, forward estimation, and inverse estimation of a projective transform on a complex plane with complex-valued neural networks. As learning data, we prepared 22 pair of data on y x = or y x =− that satisfied equation (19), where (x, y) and (x', y') were the points before and after the transform, respectively. We prepared the 11 points x = (-0.5, -0.4, ..., 0.5).
First, we examined the learning, forward estimation, and inverse estimation of a projective transform by setting the parameters a 11 = 3, a 12 = 0, a 13 = 0, a 21 = 0, a 22 = 3, a 23 = 0, a 31 = 0, a 32 = 6, and a 33 = 6. Fig. 19 shows the forward mapping vector and the inverse mapping vector obtained by learning. Based on these results, it was confirmed that the transform vectors were correctly obtained. Fig. 20 shows the results of the forward and inverse estimation of the rotation transform of an ellipse whose major axis, minor axis, and center were 0.5, 0.125, and the origin, respectively. Based on these results, it was confirmed that the forward and inverse estimations were correctly conducted. In addition, Fig. 21 shows the results of the forward and inverse estimation of the rotation of an elliptic Gaussian function. Based on these results, it was found that the rotation was correctly conducted. Therefore, it was shown that a complex-valued neural network can realize the forward and inverse transforms of a projective transform.

Filtering images using transformed filters
We performed image filtering using the transformed filter estimated in the previous simulations and examined the results. We used four transformed filters: an expanded Gaussian low-pass filter with forward estimation, an expanded Gaussian low-pass filter with inverse estimation, a rotated elliptic Gaussian filter with forward estimation, and a rotated elliptic Gaussian filter with inverse estimation. The transformed image spectrum I' (u, v) can be described as the product of the transfer function H (u, v) and the original image spectrum I (u, v); that is, The standard image on which the filtering was performed is shown in Fig. 22(a). Fig. 22(b) shows simple line segments that indicate the orientation selectivity of the rotated elliptic www.intechopen.com Gaussian filter. The results for the standard image are shown in Fig. 23. Fig. 24 shows a magnification of the upper-left part of Fig. 23. Part (a) in Figs. 23 and 24 shows the standard image filtered using the expanded Gaussian low-pass filter with forward estimation; part (b) shows the case for expanded Gaussian low-pass filter with inverse estimation. In Fig. 23 (c) and (d), a striped pattern flowing from upper left to lower right and from upper right to lower left, respectively, is observed; this indicates that orientation selection was performed. To clearly demonstrate the effects of the orientation selectivity, the simple line segments were filtered by transformed elliptic Gaussian filter, as shown in Fig. 25. It was found that the line segments at 45° and -45° were selectively extracted, as shown in Fig. 25 (a) and (b), respectively. This result confirmed that directional selectivity had been achieved.
The above results confirm that the low-pass filter and orientation selection filter can be correctly realized by transforming image filters using a neural network. We can obtain a low-pass filter and an orientation selection filter having arbitrary characteristics by appropriate scaling and rotation of a Gaussian filter. An advantage of the neural network based method is that arbitrary filter characteristics can be obtained by learning of the input/output relation.
The advantage of the transformation of the image filter by a neural network is the adaptive composition of the filter by learning. As shown in the above simulation, the use of a neural network enables the easy realization of not only linear transformations, such as scaling and rotation, but also a nonlinear transformation, namely, projective transformation. Because neural networks can obtain a transformation only by a mapping between input and output, any transformation is realizable if input and output are known. Moreover, the use of neural networks enables the realization of an inverse filter by a network inversion method, which can be applied to areas such as image restoration and bidirectional image processing. The advantage of transformation by complex-valued neural networks is that the mapping on a plane, especially one that has undergone rotation, can be expressed well. Moreover, complex-valued neural networks afford lower computational complexity than conventional real-valued neural networks because, in the former, two-dimensional information can be expressed by a neuron.
In machine vision, the advantage of filter transformation by the proposed neural network is as follows. Neural networks are often used as part of the recognition system in a machine vision system. The filter transformation proposed in this work corresponds to the preprocessing stage in machine vision. Because neural network based pre-processing is compatible with a neural network based recognition system, we will be able to integrate the pre-processing and recognition system when designing a machine vision system. Furthermore, the model of a recognition neural network, such as the neocognitron (Fukushima, 1988), includes an orientation selection cell. Therefore, we expect that the proposed method can be directly applied to such a model.
The experimental results indicate that the proposed filter afforded most of the expected advantages. The disadvantage of the proposed neural network based method is that its computational complexity is slightly high. However, it is expected that this can be resolved by further refinement of the computer, system, and calculation method.
www.intechopen.com  Fi ltered Im age Fi ltered Im age

Conclusion
In this study, we showed the mapping ability of a complex-valued neural network and proposed its use in the transform of an image filter. We introduced a complex-valued multilayer neural network for solving forward problems and a complex-valued network inversion for solving inverse problems and explained each principle of operation in detail. Moreover, to demonstrate the mapping ability, we handled complex mapping problems, i.e., the linear geometric transforms, namely, expansion/compression, rotation, and projective transforms. We confirmed the estimation procedure by simulation. In addition, we applied this mapping capacity to the transform of an image filter, and we demonstrated the geometric transform of various filters, such as the Gaussian filter. Moreover, we examined image filtering with a filter that was transformed using a complex-valued neural network and confirmed its effectiveness.
As the future work, we will apply complex-valued neural networks to nonlinear transforms of various image filters. Moreover, we will introduce processing with complex-valued neural networks into an actual robot vision system based on the results of this study. Recently, the algorithms for the processing of the visual information have greatly evolved, providing efficient and effective solutions to cope with the variability and the complexity of real-world environments. These achievements yield to the development of Machine Vision systems that overcome the typical industrial applications, where the environments are controlled and the tasks are very specific, towards the use of innovative solutions to face with everyday needs of people. The Human-Centric Machine Vision can help to solve the problems raised by the needs of our society, e.g. security and safety, health care, medical imaging, and human machine interface. In such applications it is necessary to handle changing, unpredictable and complex situations, and to take care of the presence of humans.