Application of ICA in Watermarking

Data embedding in an image may be carried out in different domains, including spatial and transform domains. Early image watermarking schemes operated directly in spatial domain, which were mostly associated with poor robustness properties. Accordingly, different transform domains have been studied in the last decade to improve the efficiency and the robustness of watermarking methods (Bounkong et al., 2003; Cox et al., 1997; Langelaar et al., 1997; M.Wang et al., 1998). One of the most effective transform in this area is ICA transform.


Introduction
Data embedding in an image may be carried out in different domains, including spatial and transform domains. Early image watermarking schemes operated directly in spatial domain, which were mostly associated with poor robustness properties. Accordingly, different transform domains have been studied in the last decade to improve the efficiency and the robustness of watermarking methods (Bounkong et al., 2003;Cox et al., 1997;Langelaar et al., 1997;M.Wang et al., 1998). One of the most effective transform in this area is ICA transform.
Independent Component Analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals (Comon, 1994). The ICA is typically known as a method for Blind Source Separation (BSS) and can be used in watermarking. It is studied in (Bounkong et al., 2003) that the ICA allows the maximization of the information content and minimization of the induced distortion by decomposing the original signal into statistically independent sources used for data embedding.
The idea of applying ICA to image watermarking has been presented in quite a handful of studies, such as in the works of (Bounkong et al., 2003;Gonzalez-Serrano et al., 2001;Hajisami et al., 2011;Shen et al., 2003;Yu et al., 2002;Zhang & Rajan, 2002). The similarity between ICA and watermarking schemes and the blind separation ability of ICA are the reasons that make ICA an attractive approach for watermarking (Nguyen et al., 2008).
Watermarking methods can be categorized into three major groups: blind, semi-blind, and non-blind (Lu, 2004). In the blind methods, there is no need for the original signal or the watermark for watermark extraction. In semi-blind methods, some features of the original signal are to be known a priori, where the original signal should be available for extracting the watermark in non-blind methods.
Firstly, in this chapter we investigate the problem of decomposition of a signal into multiple scales with a different point of view. More accurately, we propose an algorithm that contains two steps. At the first step, we decompose our signal by the use of a blocking method in which we divide the original signal into the blocks of the same size. By putting the corresponding components of each block into a vector, we can extract a number of observation signals from the original signal. At the second step, we apply a linear transform on these extracted signals. In addition, we need to find a suitable transform to analyze the original signal into multiples scales. Therefore, we see our problem as a blind source separation (BSS) problem in which the above extracted signals from different blocks are the observations in the source separation problem. Indeed, by the use of our blocking technique the extracted signals contain adjacent components of the original signal which are similar to each other, because of the fact that neighboring components of an ordinary signal are so close to each other in the sense of magnitude. Hence, by extracting the independent components of these observations by the use of ICA, one can expect that one of the resulting sources will be an approximation of the original signal while the others, will stand for details. In addition, this method of decomposing, which is called MRICA, has the advantage that it results in statistically independent components which may have applications in some signal processing areas such as watermarking (Hajisami & Ghaemmaghami, Oct. 2010).
It is reported in (Bounkong et al., 2003) that in the context of watermarking, ICA allows the maximization of the information content and minimization of the induced distortion by decomposing the cover signal into statistically independent components. Embedding information in one of these independent components minimizes the emerging cross-channel interference. In fact, for a broad class of attacks and fixed capacity values, one can show that distortion is minimized when the message is embedded in statistically independent components. Information theoretical analysis also shows that the information hiding capacity of statistically independent components is maximal (Moulin & O'Sullivan, 2003). Also as we mentioned above, MRICA can decompose the original signal into approximation and details that are statistically independent. Hence, we can exploit MRICA to improve the watermarking schemes.
This chapter is organized as follows. In the next section, some preliminary issues around the subject of BSS and ICA will be provided. Following by that, in Section 3, we will introduce MRICA and its multi-scale decomposition property. After that, in Section 4 and Section 5,two watermarking schemes are presented based on MRICA. Finally, The conclusion is drawn in Section 6.

Blind source separation and independent component analysis
In the BSS, a set of mixtures of different source signals is available and the goal is to separate the source signals, when we have no information about the mixing system or the source signals (hence the name blind). The mixing and separating systems are shown in Fig. 1 that can be represented mathematically as: in which s(t)=[s 1 (t),...,s N (t)] T is the vector of sources that are mixed by the mixing matrix A and create the observations vector x(t)=[ x 1 (t),...,x N (t)] T .L e ta l s oA be a the square matrix (N × N) of full column rank that means number of sources are equal to the number of observations and observations are linearly independent. The goal is to achieve the separating matrix B such that the y(t)=[ y 1 (t),...,y N (t)] T is an estimation of the sources. The ICA, as a method for the BSS, exploits the assumption of source independence and estimates B such that the outputs y i 's are statistically independent. It has been shown (Comon, 1994) that this leads to retrieving the source signals provided that there are at most one Gaussian source.

Multi Resolution by Independent Component Analysis (MRICA)
In this section we propose a new idea for multi-scale decomposition based on ICA called MRICA. Our method has two steps: 1) blocking the original signal and extracting our observation signals. 2) decomposing the original signal by a linear transform. Henceforth, we describe the motivation of our idea. Suppose that s 1 (t) and s 2 (t) are two independent signals which s 1 (t) has much more energy than s 2 (t). Also, suppose that x 1 (t) and x 2 (t) are two linear mixture of s 1 (t) and s 2 (t) which are presented as: In this case the shape of x 1 (t) and x 2 (t) is completely similar to the s 1 (t) (the signal with the more energy). Now, if we consider x 1 (t) and x 2 (t) as observations of the ICA algorithm, outputs will consist of two parts: 1) s 1 (t) that is the signal with the more energy and is similar to the mixtures of x 1 (t) and x 2 (t),and2)s 2 (t) that is the signal with lower energy. Therefore, we expect that if we extract two similar signals from the x(t) and consider them as x 1 (t) and x 2 (t), by applying ICA algorithm to these two signals, we must have two signals in the output, as one of them is the approximation signal and must be similar to x 1 (t) and x 2 (t) and the other one is the detail signal.
Generally, for decomposing the one-dimensional signal into k level approximation and details, it is sufficient to divide it into blocks of length k and consider the corresponding components of the blocks as an observation of the ICA algorithm. On the other hand for decomposing the two-dimensional signal into k 2 level of approximation and details it is sufficient to divide it into blocks of size k × k and consider the corresponding components of these blocks as an observation of the ICA algorithm. Procedure of blocking for one-dimensional and two-dimensional signals is shown in Fig. 2, in which x j (i) is ith sample of jth observation. Therefore, we will get k and k 2 observation signals for one-dimensional and two dimensional signals, respectively. Fig. 3 and Fig. 4 show the observation signals which are obtained from the blocking process. Then, by applying the ICA into these observation signals, for one-dimensional signals we can get one approximation signal and k − 1 detail signals and for two-dimensional signals we can get one approximation signal and k 2 − 1 detail signals. Hence, a new transform (MRICA), which is able to decompose signals into statistically independent approximation and details, is available.
To show the performance of the MRICA, a sinusoidal wave which is added to the white gaussian noise with zero mean and variance of 0.01, shown in Fig. 5, is supposed. Odd and even samples of this noisy signal are depicted in Fig. 6. By applying the ICA algorithm to these signals, we can decompose the noisy signal into the approximation and the detail signals as shown in Fig. 7. Moreover, to demonstrate the performance of the MRICA for two-dimensional signals, we consider the Lena image which is shown in Fig. 8. If we suppose k = 3, then 9 observation signals will be obtained, which are exhibited in Fig. 9. Next, by applying the ICA to these 9 images, the Lena image can be decomposed into one approximation and 8 detail signals, which are depicted in Fig. 10.

First proposed watermarking algorithm based on MRICA
In this section, the main idea is to employ the MRICA properties in order to improve the robustness, imperceptibility, and embedding rate of the watermarking. In this method, we divide the original image into blocks of size k × k and consider the corresponding components of these blocks as an observation signal, so we will have k 2 observation signals. Then we apply the ICA to these observation signals to obtain k 2 independent signals that build our ICA bases (As we previously mentioned in Section 3). In other words, if I is an intensity image of size n × m, we divide I into blocks D i,j of size k × k,wherei = 1, ··· , n/k and j = 1, ··· , m/k,then 31 Application of ICA in Watermarking www.intechopen.com  we place entries of each block on vector x l of size k 2 × 1, where l is the index of block number l = 1, ··· , nm/k 2 . The ICA problem consists of finding a k 2 × k 2 matrix B,as: such that entries of y l are statistically independent. Now, by placing each vector x l on the lth column of matrix X of size k 2 × nm/k 2 , we can obtain matrix Y,as: The rows of Y are statistically independent and are taken as our ICA bases. From (Lewicki et al., 1999) we know the ICA basis with highest energy has more information of image (see Fig. 13(a)), so it is expected to achieve higher robustness if we embed in this basis. Also, as mentioned in Section 1, maximization of the information content and minimization of the induced distortion will be attained by embedding information across independent sources obtained from the original signal through the decomposition process. Therefore, in our proposed method, we embed in the ICA basis of the highest energy: where IC H is the ICA basis of the highest energy, W is the watermark and α denotes the embedding strength. The watermarking process outlined above can be summarized in the following algorithm: 1. Divide the original image into blocks of size k × k and compute the components x l . 2. Compute the ICA components y l of the image. 3. Compute the ICA bases by constructing matrix Y. 4. Embed the watermark in the ICA basis of highest energy, as described in (5). 5. Restore the matrix X = B −1 Y. 6. Quantize entries of matrix X to obtain integer elements. 7. Restore the blocks from columns of X. 8. Restore the image from the blocks.
We take the watermarking problem as a BSS problem, where the original image and the watermark are the statistically independent sources to be separated. Accordingly, the watermarked image is assumed to be the observation in the BSS model that undergoes the ICA based extracting process.
The extraction algorithm can be described as: 1. Divide both the original image and the watermarked one into blocks of size k × k and compute the components x l .
2. Use the ICA to obtain matrices Y and Y ′ of both watermarked and original images.

Condition of blind extraction
In this part, we will show that the watermark extraction in the proposed scheme can be treated as blind, if multiple copies of an image contain the watermark at different strengths. In this situation, IC W 's that are obtained from (5) become linearly independent. Assuming we have got N copies of an image to watermark, the embedding is carried out as: To extract the watermark, it is sufficient to have two different copies of the watermarked images and follow the procedure given below.

Experimental results
In this section, we experimentally study the robustness of the suggested method against adding noise, resizing, lowpass filtering, multiple marks embedding, JPEG compression, gray-scale reduction, and cropping parts of the image. The results of these experiments show that this method is robust against the above attacks. Moreover, we show superiority of the proposed method over some well-known embedding methods, by comparing our results to those of the methods given in (Cox et al., 1997;Langelaar et al., 1997;M.Wang et al., 1998) that embed the watermark in different domains. It is to be noted that we have used FastICA algorithm (Hyvärinen, 1999) in our simulations.

Simulation setup
In our simulation, we have used a database of 200 natural images as the original images and 50 various logos as the watermarks. Fig. 11 illustrates a sample of a binary watermark image (Sharif university logo) of size 128 × 128 and original image (cameraman) of size 256 × 256.
To embed the watermark, first we divide the original signal into blocks of size 2 × 2, so, four observation signals will be obtained, as shown in Fig. 12. Then, by applying the ICA to these signals, one approximation and tree detail signals will be acquired which are our ICA bases (see Fig. 13). In Fig. 14(a), the watermarked image that is created by (5) for α = 3 255 is shown. Figure 14(b) represents the extracted watermark from the watermarked image using the ICA.
To measure the quality of the watermarked image, we use Peak Signal-to-Noise Ratio (PSNR). The PSNR between an image X and its perturbed versionX is defined as: PSNR = 20 log 10 255 where M × N is the size of the two images. In the watermarked image that is shown in Fig. 14(a), PSNR is equal to 52.87dB, whereas the PSNR in the methods of (Cox et al., 1997), (Langelaar et al., 1997) and (M. Wang et al., 1998)  In our experiments over the given original and watermark databases, we had BER = 0.004, as the average error rate.

Robustness against different attacks
In this section, we study the performance of the suggested method against different types of attacks.

Experiment 1 (Noise addition):
In this experiment, we added a Gaussian noise of zero mean and variance 0.25 and a Salt & Pepper noise of density 0.5% to the watermarked image. It was observed that FastICA could still extract the watermark as shown in Fig. 15(b) and 16(b). This is because, after adding the Gaussian noise, Equation (5) changes to IC W = IC H + αW + n, where n denotes the Gaussian noise. In this case, the two sources are IC H and αW + n and, following the extraction process, we retrieve αW + n as the watermark. In case of additive  Experiment 2 (Lowpass filtering): we applied a lowpass filter to the watermarked image by averaging each pixel with its neighbors. The result of this filtering process is illustrated in Fig. 17(a). Our extraction algorithm was quite successful to detect the watermark, as shown in Fig. 17(b). We scaled down watermarked image by factor 2 using the bilinear method. To examine the extraction performance in this case, we used the resized version of the original image due to the ICA requirement. However, because we might not be aware of the resizing procedure employed by the attacker, we used the bicubic method to resize the original image. Our mark extraction method was again found successful in all such resizing attacks applied to the images in our database. An example is shown in Fig. 18. In order to study the performance of our method when another watermark is embedded in the genuine watermarked image, we added another watermark randomly selected from our watermark database. An example is shown in Fig. 19(a) that is the second watermark embedded into the watermarked cameraman image. It is observed from Fig. 19(b) that the original watermark can still be retrieved from the attacked image. This is because in this case, Equation (5) Fig. 19. Exhibition of robustness against multiple marks embedding Experiment 5 (Cropping): Here, we cropped 25% of the image, and then applied our method to extract the watermark. Fig. 20 illustrates performance of the method in this case, where the instantaneous mixture model still holds for remainder pixels. Experiment 6 (Gray-scale reduction): In this experiment the gray-scale of watermarked image is reduced from 256 down to 64. In this case, the pixel value of new image is almost 1/4 times of the older one. Because the ICA is not sensitive to multiplying the observation by a constant, the watermark can still be retrieved, as illustrated in Fig. 21(b).

Experiment 7 (JPEG compression):
In the last experiment, we JPEG compressed the watermarked image by the quality factor of 80%. The result of our watermark retrieval method is displayed in Fig. 22(c) for the case of the cameraman. Results of a brief comparison made with two other well-known watermarking methods (Langelaar et al., 1997;M.Wang et al., 1998) are shown in Fig.22(b) against different JPEG quality factors.

Second proposed watermarking algorithm based on MRICA
In this part, a blind method for image watermarking is proposed which is robust against different type of attacks including noise addition, gray-scale reduction, cropping, and JPEG compression.
As mentioned in the Section. 3, MRICA is able to decompose the original signal into the approximation and the details that are statistically independent. In MRICA, detail signals with less energy are not valuable parts of the original signal and replacing them with the watermark will not have tangible impact on the quality of the original signal. Therefore, we decompose the original image into k 2 independent signals by means of MRICA. Accordingly, we will have one approximation and k 2 − 1 details. In order to embed the watermark, we eliminate the detail with the lowest energy and replace it with the watermark and then we convert the watermarked image into spatial domain. It should be noted that after converting the watermarked image into spatial domain, it is necessary to quantize the pixel values to obtain integer elements as image format. Also to extract the watermark, we decompose the watermarked image into k 2 independent signals by means of MRICA and extract the signal with lowest energy that is the watermark. Detailed procedures are explained as follows: Suppose I is an intensity image of size n × m, we divide I into blocks D i,j of size k × k,where i = 1, ··· , n/k and j = 1, ··· , m/k. Next, we construct matrix Y, as explained in Section 4. The rows of matrix Y (approximation and detail signals) are statistically independent and are taken as our ICA bases. According to what we mentioned in Section. 1, the detail signal with lowest energy is not valuable part of image. Moreover, maximization of the information content and minimization of the induced distortion will be attained by embedding information across independent signals obtained from the original signal through the decomposition process. Therefore, in our proposed method, we replace the secret massage with the ICA basis of lowest energy: where IC L is the ICA basis of the lowest energy, W is watermark and α denotes the embedding strength. The embedding process outlined above can be summarized as follows: 1. Divide the original image into blocks of size k × k and compute the components x l .
2. Compute the ICA components y l of the image.
3. Compute the ICA basis (approximation and detail signals) by constructing matrix Y.
4. Replace the watermark with the ICA basis of lowest energy, as described in (9).
6. Quantize entries of matrix X to obtain integer elements.
7. Restore the blocks from columns of X.
8. Restore the image from blocks.
In order to extract the watermark, it is sufficient to apply MRICA to the image and get k 2 approximation and detail signals then the watermark is obvious between the detail signals.

Experimental results
In this section, we experimentally study the robustness of the suggested method against adding noise, gray-scale reduction, cropping parts of the image, and JPEG compression.
The results of these experiments show that this method is robust against the above attacks. Moreover, we will show superiority of the MRICA over some well-known wavelet transforms.

Simulation setup
In our simulation, we have used a database of 200 natural images as the original images and 50 various binary logos as the watermark. Fig. 23 illustrates a sample of a binary watermark image (Sharif university logo) of size 128 × 128 and an original image (picture of ship) of size 256 × 256. To embed the watermark, first we divide the original signal into blocks of size 2 × 2, so 4 observation signals will be obtained, as shown in Fig. 24. Then by applying the ICA algorithm to these observations, one approximation and three detail signals can be obtained, shown in Fig. 25. After that, we replace the watermark with the detail signal of lowest energy (Fig. 25(d)). Finally, we convert the image into spatial domain and quantize pixel values to obtain integer elements. In Fig. 26(a), the watermarked image that is created by (9) for α = 4 255 is shown. To measure the quality of the watermarked image, we use Peak Signal-to-Noise Ratio (PSNR). In the watermarked image that is shown in Fig. 26(a), PSNR 41 Application of ICA in Watermarking www.intechopen.com is equal to 41.87dB, whereas the PSNR in the methods of (Cox et al., 1997), (Langelaar et al., 1997) and (M. Wang et al., 1998) are equal to 38.4dB, 36.7dB and 34.2dB, respectively. After extraction process, the watermark will be obtained as shown in Fig. 26(b). In our experiments over the given original and watermark databases, we had BER = 0.007, as the average error rate. Moreover,in Figures 26(c) and 26(d), MRICA has been compared with Haar and db5 wavelet transforms as embedding is carried out by replacing the watermark with diagonal detail coefficients matrix of these wavelet transforms. BER = 0.233 and BER = 0.164 are obtained for Haar and db5 wavelet transforms, respectively.

Robustness against different attacks
In this section, we study the performance of the suggested method against different types of attacks.
Experiment 1 (Noise addition): In this experiment, we added a Gaussian noise of zero mean and variance 0.25 and a Salt & Pepper noise of density 0.5% to the watermarked image. It was observed that MRICA could still extract the watermark as shown in Fig. 27(b) and 28(b).
Experiment 2 (Gray-scale reduction): In this experiment, the gray-scale of the watermarked image is reduced from 256 down to 64. In this case, the pixel value of new image is almost 1/4 times of the older one. Because ICA is not sensitive to multiplying the observation by a constant, the watermark still can be retrieved, as illustrated in Fig. 29(b). Moreover, Fig. 29(c) and 29(d) exhibit that MRICA is more successful than Haar and db5 wavelet transforms. Experiment 3 (Cropping): Here, we cropped 25% of the watermarked image, and then applied our method to extract the watermark. Fig. 30 illustrates performance of the method in this case, where the instantaneous mixture model still holds for remainder pixels.

Experiment 4 (JPEG compression):
In the last experiment, we JPEG compressed the watermarked image by the quality factor of 80%. The result of our watermark retrieval method is displayed in Fig. 31(b). Moreover, Fig. 31(c) and 31(d) demonstrate performance improvement of MRICA compared with Haar and db5 wavelet transforms. In addition, results of a brief comparison made with two other well-known methods (Langelaar et al., 1997;M.Wang et al., 1998) are shown in Fig.32 against different JPEG quality factors.

Conclusion
In this chapter, a new basis, which is based on ICA, for watermarking was introduced. For constructing the ICA basis, at first a new method for multi-scale decomposition called MRICA was presented. For MRICA, we divided the original image into blocks of same size. Then, we considered the corresponding components of these blocks as the observation signals. After that, by applying the ICA algorithm to these observation signals, we projected the original signal into a basis with its components as statistically independent as possible. Next, two watermarking algorithms were proposed in which data embedding was carried out in the ICA basis and the MRICA was used for watermark extraction. Experimental results showed that the MRICA outperforms wavelet transform in our watermarking schemes. Also, it was shown that our watermarking schemes has better performance than some well-known methods (Cox et al., 1997;Langelaar et al., 1997;M.Wang et al., 1998) and is robust against various attacks, including noise addition, gray-scale reduction, cropping parts of image, and JPEG compression.