Image Fusion Based on Color Transfer Technique

1. Introduction

In 2001, with the nonlinear lαβ space (Ruderman et al., 1998), Reinhard et al. (Reinhard et al., 2001) introduced a method to transfer colors between two color images. The goal of their work was to make a synthetic image take on another image’s look and feel. Applying Reinhard’s statistical color transfer strategy, Toet (Toet, 2003) subsequently developed a color-transfer-based image fusion algorithm. With an appropriate daylight color image as the target image, the method can produce a natural appearing “daytime-like” color fused image and significantly improve observer performance. Therefore, the Toet’s approach has received considerable attention in recent years (Li & Wang, 2007b; Li et al., 2010a; Li et al., 2010b; Li et al., 2005; Tsagaris & Anastassopoulos, 2005; Toet & Hogervorst, 2008; Toet & Hogervorst, 2009; Wang et al., 2007; Zheng & Essock, 2008).

Although Toet’s work implies that Reinhard’s lαβ color transfer method can be successfully applied to image fusion, it is difficult to develop a fast color image fusion algorithm based on this color transfer technique. The main reason is that it is restricted to the nonlinear lαβ space (See Appendix A). This color space is logarithmic, the transformation between RGB and lαβ spaces must be transmitted through LMS and logLMS spaces. This therefore increases the system’s storage requirements and computational time. On the other hand, since the dynamic range of the achromatic component in lαβ space is very different from that of a normal grayscale image, it becomes inconvenient to enhance the luminance contrast of the final color fused image in lαβ space by using conventional methods, such as directly using a high contrast grayscale fused image to replace the luminance component of the color fused image.

To eliminate the limitations mentioned above, we (Li et al., 2010a) employed YC_BC_R space to implement Reinhard’s color transfer scheme and applied the YC_BC_R color transfer technique to the fusion of infrared and visible images. Through a series of mathematical derivation and proof, we (Li et al., 2010b) further presented a fast color-transfer-based image fusion algorithm. Our experiments demonstrate the performance of the fast color-transfer-based image fusion method is superior to other related color image fusion methods, including the Toet’s approach.

The rest of this chapter is organized as follows. Section 2 reviews Reinhard’s lαβ color transfer method. Section 3 outlines our YC_BC_R color transfer method. Section 4 describes two basic image fusion methods based on the YC_BC_R color transfer technique. Section 5 introduces our fast color-transfer-based image fusion method. Experimental results and associated discussions are provided in Section 6. Finally, in Section 7, conclusions are drawn and future works are suggested.

2. lαβ color transfer method

Studies by Reinhard et al. (Reinhard et al., 2001) have found that straightforward color statistics can capture some important subjective notions of style and appearance in images. Their work shows how to transfer them from one image to another to copy some of the atmosphere and mood of a good picture. With the pixel data represented in lαβ space, Reinhard et al. transfer color statistics from the target image to the source image by applying a linear map to each axis separately:

$${\theta }_s^{*}=({\sigma }_t^{\theta }/{\sigma }_s^{\theta })({\theta }_s-{\mu }_s^{\theta })+{\mu }_t^{\theta }{,}^{}{}^{}\text{fo}{\text{r}}^{}\theta =l,\alpha ,\beta$$ E1

where the indexes ‘s’ and ‘t’ refer to the source and target images respectively. $$\theta$$ denotes an individual color component of an image in lαβ space. $({\mu }_s^{\theta },{\sigma }_s^{\theta })$ and $({\mu }_t^{\theta },{\sigma }_t^{\theta })$ are the means and standard deviations of the source and target images, respectively, both over the channel $$\theta$$ . Following this step, the resulting source image data is converted back to RGB space for display. After the transform described in (Eq. 1), the first and second order statistics of color distribution of the source image conform to those of the target color image. This color statistics match procedure is quite simple but can successfully transfer one image’s color characteristics to anther.

3. YC_BC_R color transfer method

Instead of using the nonlinear lαβ space, our proposed YC_BC_R color transfer method (Li et al., 2010a) transfers the color distribution of the target image to the source image in the linear YC_BC_R color space. The forward and backward YC_BC_R transformations (Neelamani et al., 2006; Skodras et al., 2001) are achieved by means of (Eq. 2) and (Eq. 3), respectively.

$$\left[\begin{array}{c}Y\\ C_B\\ C_R\end{array}\right]=\left[\begin{array}{ccc}0.2990& 0.5870& 0.1140\\ -0.1687& -0.3313& 0.5000\\ 0.5000& -0.4187& -0.0813\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]$$ E2

$$\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{c}Y\\ C_B\\ C_R\end{array}\right]$$ E3

where Y denote the luminance, C_B and C_R are two chromatic channels, which correspond to the color difference model (Poynton, 2003; Jack, 2001). As shown in (Eq. 4), C_B and C_R stand for blue and red color difference channels, respectively.

$$\begin{array}{l}{C}_B=\frac{0.5}{0.886}(B-Y)\\ C_R=\frac{0.5}{0.701}(R-Y)\end{array}$$ E4

Since the YC_BC_R transformation is linear, its computational complexity is far lower than that of the lαβ conversion. Let N be the number of the columns and rows of the input RGB image. The lαβ transformation requires a total of 22N² additions, 34N² multiplications, 3N² logarithm and 3N² exponential computations. In contrast, the YC_BC_R transformation avoids the logarithm and exponential operations. From (Eq. 2) and (Eq. 3), we can observe that only 10N² additions and 13N² multiplications are required for its implementation. Obviously, the simplicity of the YC_BC_R transformation enables a more efficient implementation of color transfer between images. Like the Reinhard’s scheme, the result’s quality of the YC_BC_R color transfer method also depends on the images’ similarity in composition.

The YC_BC_R transformation can be extended into a general formalism defined in (Eq. 5) and (Eq. 6). For color transfer, using color spaces conforming to this general YC_BC_R space framework, such as YUV space (Pratt, 2001), can produce same recoloring results as using YC_BC_R space.

$$\left[\begin{array}{c}\tilde{Y}\\ {\tilde{C}}_B\\ {\tilde{C}}_R\end{array}\right]=\left[\begin{array}{ccc}0.2990x& 0.5870x& 0.1140x\\ -0.1687y& -0.3313y& 0.5000y\\ 0.5000z& -0.4187z& -0.0813z\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]$$ E5

$$\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}1.0000x^{-1}& 0.0000& 1.4020z^{-1}\\ 1.0000x^{-1}& -0.3441y^{-1}& -0.7141z^{-1}\\ 1.0000x^{-1}& 1.7720y^{-1}& 0.0000\end{array}\right]\left(\left[\begin{array}{c}\tilde{Y}\\ {\tilde{C}}_B\\ {\tilde{C}}_R\end{array}\right]-\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\right)$$ E6

where x, y, z, c₁, c₂ and c₃ are constants, and x, y and z are nonzero. This fact can be proved by the following proposition.

Proposition 1: Let ${[{\tilde{R}}_s^{*},{\tilde{G}}_s^{*},{\tilde{B}}_s^{*}]}^T$ and ${[R_s^{*},G_s^{*},B_s^{*}]}^T$ be the recolored source images obtained by the ${\text{<mover accent="true"> <mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ and YC_BC_R based color transfer methods, respectively. Suppose the two cases use the same target image, then for a fixed source image,

$${[{\tilde{R}}_s^{*},{\tilde{G}}_s^{*},{\tilde{B}}_s^{*}]}^T={[R_s^{*},G_s^{*},B_s^{*}]}^T$$ E7

Proof: See Appendix B.

The ${\text{<mover accent="true"> <mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$

transformation specified in (Eq. 5) and (Eq. 6) can be regarded as an extension of the YC_BC_R transformation. If x = y = z = 1, and c₁ = c₂ = c₃ = 0, then the ${\text{<mover accent="true"> <mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ transformation becomes equivalent to the YC_BC_R transformation. If x = 1, y =0.872, z = 1.23, and c1 = c₂ = c₃ = 0, then the ${\text{<mover accent="true"> <mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ transformation is equivalent to the YUV transformation as follows.

$$\left[\begin{array}{c}Y\\ U\\ V\end{array}\right]=\left[\begin{array}{ccc}0.2990& 0.5870& 0.1140\\ -0.1471& -0.2888& 0.4359\\ 0.6148& -0.5148& -0.1000\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]$$ E8

$$\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}1.0000& 0.0000& 1.1403\\ 1.0000& -0.3947& -0.5808\\ 1.0000& 2.0325& 0.0000\end{array}\right]\left[\begin{array}{c}Y\\ U\\ V\end{array}\right]$$ E9

4. Basic fusion method based on YC_BC_R color transfer technique

With the YC_BC_R color transfer method, it is very easy to develop color image fusion methods for merging infrared and visible images. We introduced two basic fusion approaches, one named StaCT (Standard Color-Transfer-Based Fusion Method), the other called CECT (Contrast Enhanced Version of Color-Transfer-Based Fusion Method) (Li et al., 2010b). These two basic fusion methods have to produce a source false color fused image, we employed the NRL (Naval Research Laboratory) scheme (Scribner et al., 1998; McDanie et al., 1998) to generate the source color fused image. The NRL algorithm is very simple and fast, and greatly useful for developing fast image fusion method.

4.1. StaCT method

The StaCT method uses the YC_BC_R color transfer method to directly recolor the source false color fused image, its whole steps are as follows.

1. Construct the source false color fused image [R_f, G_f, B_f]^T by using the NRL scheme:

$${[R_f,G_f,B_f]}^T={[IR,Vis,Vis]}^T$$ E10

where IR and Vis represent the input infrared and visible images, respectively.

2. Convert the image [R_f, G_f, B_f]^T to YC_BC_R space and produce the source YC_BC_R components $${{\text{[Y}}_{\text{s}}{\text{,C}}_{\text{B}\text{,s}}{\text{,C}}_{\text{R}\text{,s}}\text{]}}^{\text{T}}$$ :

$$\left[\begin{array}{c}{Y}_s\\ C_{B,s}\\ C_{R,s}\end{array}\right]=\left[\begin{array}{c}{Y}_f\\ C_{B,f}\\ C_{R,f}\end{array}\right]=\left[\begin{array}{ccc}0.2990& 0.5870& 0.1140\\ -0.1687& -0.3313& 0.5000\\ 0.5000& -0.4187& -0.0813\end{array}\right]\left[\begin{array}{c}{R}_f\\ G_f\\ B_f\end{array}\right]$$ E11

3. Stretch the source YC_BC_R components to make their statistics match the color statistics of the target daylight color image:

$$\begin{array}{l}{Y}_c=\frac{{\sigma }_t^Y}{{\sigma }_s^Y}(Y_s-{\mu }_s^Y)+{\mu }_t^Y\\ C_{B,c}=\frac{{\sigma }_t^{C_B}}{{\sigma }_s^{C_B}}(C_{B,s}-{\mu }_s^{C_B})+{\mu }_t^{C_B}\\ C_{R,c}=\frac{{\sigma }_t^{C_R}}{{\sigma }_s^{C_R}}(C_{R,s}-{\mu }_s^{C_R})+{\mu }_t^{C_R}\end{array}$$ E12

where Y_c, C_B,c and C_R,c are the three color components of the final color fused image [R_c, G_c, B_c]^T in YC_BC_R space. The indexes ‘s’ and ‘t’ stand for the source and target images, respectively. $({\mu }_s^{\theta },{\sigma }_s^{\theta })$ and $({\mu }_t^{\theta },{\sigma }_t^{\theta })$ , where $$\theta =\text{Y}{\text{,C}}_{\text{B}}{\text{,C}}_{\text{R}}$$ , are the means and standard deviations of the source and target images, respectively, both over the channel $$\theta$$ .

Transform the result back into RGB representation and get the final color fused image [R _c , G _c , B _c ] ^T :

$$\left[\begin{array}{c}{R}_c\\ G_c\\ B_c\end{array}\right]=\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{c}{Y}_c\\ C_{B,c}\\ C_{R,c}\end{array}\right]$$ E13

The Toet’s algorithm (Toet, 2003) produces the color fused imagery by using lαβ space. The main difference between the StaCT method and the Toet’s method lies in the color space conversion of images. As explained above, the linear YC_BC_R transformation has lower computational complexity in comparison with the nonlinear lαβ transformation. Hence, the StaCT method offers a more efficient and simpler way to produce a color fused image compared to the Toet’s method.

4.2. CECT method

The luminance contrast reduction issue, as described by Toet (Toet, 2003), may arise when the contrast of an image detail varies strongly among the different bands. In some conditions a detail may be represented with opposite contrast in different spectral bands. The combination of the individual image bands into a single color image may therefore significantly reduce the luminance contrast of an image detail. As a result, a detail that is noticeable in the individual image bands may be much less visible in the final color representation, due to the lack of luminance contrast. An appropriate grayscale fused image obtained by combining the individual sensor images can preserve all relevant contrast details of the individual bands. Hence, we can enhance the luminance contrast of the final color fused image by replacing the luminance component of the source false color fused image with the grayscale fused image. Based on this strategy, we proposed the CECT method (Li et al., 2010b). Its whole steps to perform are the following.

1. Construct the source false color fused image [R_f, G_f, B_f]^T by using the NRL scheme given in (Eq. 10).

2. Convert the image [R_f, G_f, B_f]^T to YC_BC_R space as in (Eq. 11).

3. Replace the luminance component Y_f by the grayscale fused image F, which as well as the C_B,f and C_R,f components are directly used as the source YC_BC_R components:

$${[Y_s,C_{B,s},C_{R,s}]}^T={[F,C_{B,f},C_{R,f}]}^T$$ E14

4. Stretch the source YC_BC_R components to make their statistics match the color statistics of the target daylight color image as shown in (Eq. 12).

5. Transform the result back into RGB representation as in (Eq. 13).

The color spaces conforming to the ${\text{<mover accent="true"> <mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ framework defined in (Eq. 5) and (Eq. 6) can also be used in the CECT method. When the constant x is positive, using any form of ${\text{<mover accent="true"> <mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"> <mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ space can yield the same color fused image as in YC_BC_R space. This result can be proved by the following proposition.

Proposition 2: Let $${[{\tilde{R}}_c,{\tilde{G}}_c,{\tilde{B}}_c]}^T$$ and ${[R_c,G_c,B_c]}^T$ be the color fused images obtained by the $$\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover>}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$$ and YC_BC_R based CECT methods, respectively. Suppose the two cases use the same grayscale fusion method and target image. In addition, assume x defined in the $\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover>}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ transform is positive, then for a fixed pair of input images,

$${[{\tilde{R}}_c,{\tilde{G}}_c,{\tilde{B}}_c]}^T={[R_c,G_c,B_c]}^T$$ E15

The proof of this proposition is similar to the one of Proposition 1. Hence, we do not prove it here.

The achromatic component in lαβ space cannot be used directly for the luminance replacement process because its dynamic range is very different from that of the grayscale fused image. Therefore, Toet (Toet, 2003) suggested using HSV (Hue, Saturation, and Value) color space, in which the value component (V) has the same amplitude range as the grayscale fused image, to address this problem. The final high contrast color fused image is constructed by converting the recolored fused image into HSV space, replacing the value component by the high contrast grayscale fused image and then transforming back to RGB space. One can see that three color spaces (RGB, lαβ and HSV) are employed and four color space transformations (RGB to lαβ, lαβ to RGB, RGB to HSV, and HSV to RGB) are needed in the above manner. This is not an efficient way relative to the CECT method. Taking advantage of the linear YC_BC_R space, the CECT method only uses two color spaces (RGB and YC_BC_R) and requires two color space conversions (RGB to YC_BC_R and YC_BC_R to RGB) during the whole procedure to obtain the high contrast output image.

5. Fast fusion method based on YC_BC_R color transfer technique

For real-time image fusion system, faster methods are always desirable. We proposed a fast algorithm by mathematically optimizing the CECT approach’s architecture, which is named AOCT (Architecture-Optimized Version of Color-Transfer-Based Fusion Method) (Li et al., 2010b). Fig. 1 illustrates the fusion architecture of the AOCT approach, and its whole steps are as follows.

Figure 1.

Fusion architecture of the AOCT method

1. Use [F, Vis－IR, IR－Vis]^T to form the source YC_BC_R components:

$$\left[\begin{array}{c}{Y}_s\\ C_{B,s}\\ C_{R,s}\end{array}\right]=\left[\begin{array}{c}F\\ Vis-IR\\ IR-Vis\end{array}\right]$$ E16

Steps 2 and 3 are the same as the steps 4 and 5 of the CECT method. Thus,

2. Perform color statistics matching as in (Eq. 12).

3. Perform the inverse YC_BC_R transform to obtain the final color fused image as in (Eq. 13).

We have demonstrated that the AOCT method has the same performance as the CECT method (Li et al., 2010b). The PA (Pixel Averaging) and MR (Multiresolution) fusion schemes are adopted to produce the grayscale fused image in the AOCT method. The AOCT approach using the PA grayscale fusion algorithm is named P-AOCT. The AOCT approach using the MR grayscale fusion algorithm is called M-AOCT.

Similar to the CECT approach, the AOCT method also allows us to choose other color spaces conforming to the ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ framework in (Eq. 5) and (Eq. 6). When the constants x, y and z are positive, using any form of ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ space can produce the same color fusion result as in YC_BC_R space. This conclusion can be proved by the following proposition.

Proposition 3: Let $${[{\tilde{R}}_c,{\tilde{G}}_c,{\tilde{B}}_c]}^T$$ and ${[R_c,G_c,B_c]}^T$ be the color fused images obtained by the ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ based and YC_BC_R based AOCT methods, respectively. Suppose the two cases use the same grayscale fusion method and target image. In addition, assume the constants x, y and z defined in the ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ transformation are positive, then for a fixed pair of input images,

$${[{\tilde{R}}_c,{\tilde{G}}_c,{\tilde{B}}_c]}^T={[R_c,G_c,B_c]}^T$$ E17

Proof: See Appendix C.

In contrast, the construction process of the source chromatic components (C_B,s and C_R,s) in the AOCT method is quite simpler than that in the CECT method. After performing the YC_BC_R transform and replacing the luminance component in the CECT method (by inserting (Eq. 10) and (Eq. 11) in (Eq. 14)), we arrive at

$$\left[\begin{array}{c}{Y}_s\\ C_{B,s}\\ C_{R,s}\end{array}\right]=\left[\begin{array}{c}F\\ C_{B,f}\\ C_{R,f}\end{array}\right]=\left[\begin{array}{c}F\\ -0.1687IR-0.3313Vis+0.5Vis\\ 0.5IR-0.4187Vis-0.0813Vis\end{array}\right]$$ E18

Let N be the number of the columns and rows of the input images. According to (Eq. 18), the CECT method requires 4N² additions and 6N² multiplications to obtain the source chromatic components. The AOCT method skips the YC_BC_R forward transformation and directly uses the simple difference signals of the input images (that is, Vis－IR and IR－Vis) to form the source chromatic components. Equation (Eq. 16) states that the AOCT method only requires 2N² additions in the construction process. Therefore, the AOCT method is faster and easier to implement compared to the CECT method.

The following strategies can help the AOCT method to be implemented in an extremely fast and memory efficient way.

$$F=0.5(IR+Vis)$$ E19

We have proved that the source luminance component $Y_s$ can be directly achieved by the following solution when the P-AOCT method is adopted (Li et al., 2010b).

$$Y_s=IR+Vis$$ E20

$$C_{R,s}=-C_{B,s}=-(Vis-IR)$$ E21

From this by applying the mean and standard deviation properties, we can derive that

$$\begin{array}{l}{\mu }_s^{C_R}=-{\mu }_s^{C_B}\\ {\sigma }_s^{C_R}={\sigma }_s^{C_B}\end{array}$$ E22

Equations (Eq. 21) and (Eq. 22) imply that, after computing $C_{B,s}$ , ${\mu }_s^{C_B}$ , and ${\sigma }_s^{C_B}$ , there is no need to recalculate $C_{R,s}$ , ${\mu }_s^{C_R}$ , and ${\sigma }_s^{C_R}$ . A more efficient way is to obtain them directly from the negative polarity $C_{B,s}$ , the negative polarity ${\mu }_s^{C_B}$ , and ${\sigma }_s^{C_B}$ , respectively.

3. From (Eq. 12), we can see that only six target color statistical parameters ( ${\mu }_t^Y$ , $${\mu }_t^{C_B}$$ , $${\mu }_t^{C_R}$$ , ${\sigma }_t^Y$ , $${\sigma }_t^{C_B}$$ and $${\sigma }_t^{C_R}$$ ) are required in color statistics matching process. Hence, in practice, there is no real need to store target images. A system that is equipped with a look-up table of color statistical parameters for different types of backgrounds is sufficient to enable users to adjust the color to their specific needs.

6. Experimental results

6.1. Color transfer between landscape photographs

Figure 2.

Color transfer between landscape photographs. Left: the different source images. Second column: the corresponding target images. Third column: the results of the lαβ color transfer method. Right: the results of the YC_BC_R color transfer method. Top to bottom: trees, sunset, and mountains image pairs. (All the source and target images courtesy of © Ian Britton - FreeFoto.com.)

To demonstrate the potential of the YC_BC_R color transfer method, we applied it to some landscape photographs of scenes, including trees, sunset, and mountains. The lαβ color transfer scheme (Reinhard et al., 2001) was selected to serve as a comparison. The source image of each scene is shown in the left column of Fig. 2. The second left column represents their respective target images. Note that each source image contains similar composition as the corresponding target image. The third and right columns illustrate the results of applying the lαβ and YC_BC_R color transfer methods respectively to the source image at the left column with the corresponding image in the second column as the target. lαβ space works very well on the task of recoloring these natural color photographs. This provides some validation for Reinhard et al.’s work (Reinhard et al., 2001). The images produced by using YC_BC_R space look a little lighter than in lαβ space, but the results are also pleasing and reasonable. This indicates that the YC_BC_R color transfer method can be applied to many image types.

6.2. Comparison among different color image fusion methods

Fig. 3. show some examples of color image fusion methods related to color transfer technique, inculding the NRL (Scribner et al., 1998; McDanie et al., 1998), Toet’s (Toet, 2003), improved Toet’s, StaCT, P-AOCT, M-AOCT schemes. The CECT approach has the same fusion performance as the AOCT method (Li et al., 2010b), thus, we do not discuss the CECT approach in our experiments.

Figure 3.

Fusion results of applying different color image fusion methods to the UN Camp images of a nighttime scene representing a building, a path, heather, and a person walking along the fence. (a) The input 3-5 μm midwave infrared image. (b) The input visible image. (c) The target image. (d) The result of the NRL method. (e)-(i) are the results of the Toet’s method, the improved Toet’s method, the StaCT method, the P-AOCT method and the M-AOCT method respectively, all with (c) as the target image. (The target image courtesy of Alexander Toet.)

As described above, the Toet’s method (Toet, 2003) produces a color fused image by using the lαβ color transfer approach to recolor a source false color fused image. In his study, the source false color fused image is generated by mapping the infrared image and the visible image respectively to the green and blue channels of an RGB image, the red channel is set to zero or black. Similar to the Toet approach, the improved Toet’s method also employs lαβ space to perform color transfer, the difference is that the improved Toet’s method uses the NRL scheme to yield the source false color fused image.

In this study, we employ a discrete wavelet transform based fusion scheme (Li et al., 2007a) to achieve the grayscale fused image in the M-AOCT method, and use a 4-level wavelet transform with ‘5-3’ biorthogonal filters (Cohen et al., 1992; Daubechies, 1992) to implement the MR decompositions of the input images. The approximation images at the highest level (low resolution) are combined by taking the average of them. The detail images at each high frequency band of each decomposition level are fused by a weighted average fusion rule based on the local amplitude ratio. This weighted average fusion rule can achieve a fused image with less ‘ringing’ artifacts compared to the widely used maximum selection fusion rule (Zhang & Blum, 1999; Piella, 2003).

The original registered infrared (3-5 μm) and visible test images are supplied by Toet at TNO Human Factors and available online at www.imagefusion.org. Since the corresponding daylight color photographs are not available for these images, we adopt an arbitrary color image as target.

Fig. 3 corresponds to a nighttime scene, which represents a building, a path, heather, and a person walking along the fence. The input infrared (3-5 μm) and visible images are shown in Fig. 3(a) and (b), respectively. Fig. 3(c) shows an arbitrary daytime color image with similar color distribution as the source scene. Fig. 3(d) shows the false color fused image constructed by the NRL method. Most salient features from the inputs are clearly visible in this image, but the color appearance is rather unnatural. Fig. 3(e) shows the fusion result obtained by the Toet’s method with Fig. 3(c) as target. The image has a comparatively natural color appearance. But the color characteristics are just roughly close to those of the target image. It doesn’t preserve the trees’ green colors in the target image very well. Note that some heather appears yellow. Moreover, there emerges excessive saturation phenomenon in the resulting color fused image, and thus produces a glaring white, which severely hides some salient detail information contained in the input images, such as the textures of the road. In addition, some image details, such as the poles of the fence and the outlines of the building, are represented with low contrast in Fig. 3(e). Fig. 3(f) shows the fused image produced by the improved Toet’s method with Fig. 3(c) as target. The image takes the target image’s color characteristics and has high contrast, but the excessive saturation phenomenon doesn’t disappear, some salient details, such as the textures of the road, are hiden by the glaring white. Fig. 3(g) shows the fused image generated by the StaCT method with Fig. 3(c) as target. The image does not only gain the natural color appearance of the target image, but contains most salient information from the input images, moreover, avoids the excessive saturation phenomenon. Fig. 3(h) shows the fused image produced by the P-AOCT method with Fig. 3(c) as target. The image has better visual quality in comparison with the result obtained by the StaCT scheme as shown in Fig. 3(g). The visibility of image details in Fig. 3(h), such as the poles of the fence and the outlines of the person, is evidently higher than in the StaCT algorithm. Fig. 3(i) illustrates the fused image achieved by the M-AOCT method with Fig. 3(c) as target. Clearly, the M-AOCT approach has the best overall performance.

The M-AOCT method employs the MR fusion scheme, the Toet’s and improved Toet’s methods utilize the lαβ color transfer strategy. Therefore, the computational complexities of these three methods are all higher than that of the P-AOCT method. In fact, the StaCT method has the same complexity as the P-CECT method (the CECT approach using the PA fusion algorithm to implement grayscale fusion). Thus, according to the algorithm analysis in Section 5, we can confirm that the computational cost of the P-AOCT method is lower than that of the StaCT method. Hence, in the five color-transfer-based image fusion algorithms (the Toet’s, improved Toet’s, StaCT, P-AOCT, and M-AOCT methods), the P-AOCT method has the lowest implementation complexity.

Although it can be seen from Fig. 3 that the salient information from the input images is represented with higher contrast in the fused image obtained by the M-AOCT method than those produced by other approaches, this superior performance of the M-AOCT method comes at a cost of increased computational complexity of the fusion process. In contrast, the P-AOCT method is much faster and easier to implement. Moreover, this algorithm can provide a visually pleasing color fused image with available contrast, and the salient information contained in the input images is also represented quite well. Hence, if there is no special requirement, in practice, it is not necessary to utilize the M-AOCT method to merge images, the low computational-cost P-AOCT method is generally sufficient to fulfill user needs.

6.3. Choice of target image

For the AOCT method, the actual choice of the target image is not critical. The method can provide a natural looking color fused image as long as the color distribution of the target image is to some extent similar to that of the source scene.

Figure 4.

Fusion results of applying the P-AOCT method with different target images to the Dune images of a nighttime scene representing a person walking over a dune area covered with semi-shrubs and sandy paths. Top left: the input 3-5 μm midwave infrared image. Top right: the input visible image. Middle: four different target images. Bottom: the corresponding fusion results corresponding to each target image. (The two left target images courtesy of www.pics4learning.com, and the two right target images courtesy of www.bigfoto.com.)

To demonstrate this fact, we applied the P-AOCT method to merging the Dune images with different target images. The original registered infrared (3-5 μm) and visible test images are shown at the top left and right of Fig. 4 respectively. In this scene, a person is walking over a dune area covered with semi-shrubs and sandy paths. These images are also supplied by Toet at TNO Human Factors and available online at www.imagefusion.org. The second row of Fig. 4 represents the different target images. The third row illustrates the results of applying the P-AOCT approach respectively to the input images at the top row with the corresponding image in the second row as the target. The second left column of Fig. 4 shows an interesting example where a color photograph representing a grassland area with an elk in it was adopted as the target image. The content of the target image in this case is quite dissimilar to that of the source scene, but the target image has similar color distribution as the source scene. As a result, the corresponding color fused images still have a fairly natural appearance. Another special example depicted in the right column of Fig. 4 shows that the P-AOCT method fails when the color compositions of the target image and the source scene are too dissimilar. In this case, the target image also displays a dune like scene, but has more bright blue sky in the background. Consequently, the appearance of the resulting fused images is quite unnatural and the sandy paths are represented in unreasonable blue.

From the above examples, we can see that the depicted scenes of the target and source images don’t have to be identical, as long as their color distributions resemble each other to some extent. In practice, surveillance systems usually register a fixed scene, a daylight color image of the same scene that is being monitored could be used as an optimal target image.

7. Conclusion

In this chapter, we introduced color transfer techniques and some typical image fusion methods based on color transfer technique. More importantly, we presented a fast image fusion algorithm based on YC_BC_R color transfer technique, named AOCT. Depending on the PA and MR grayscale image fusion schemes, we developed two solutions, namely the P-AOCT and M-AOCT methods, to fulfill different user needs. The P-AOCT method answers to a need of easy implementation and speed of use. The M-AOCT method answers to the high quality need of the fused products. Experimental results demonstrate that the AOCT method can effectively produce a natural appearing “daytime-like” color fused image with good contrast. Even the low-complexity P-AOCT method can provide a satisfactory result.

Another important contribution of this chapter is that we have mathematically proved some useful propositions about color-transfer-based image fusion. These theories clarify that other color spaces, which are founded on the basis of luminance, blue and red color difference components, such as YUV space, can be used as an alternative to YC_BC_R space in the image fusion approaches based on YC_BC_R color transfer technique, including the StaCT, CECT, and AOCT methods.

Currently, the AOCT method only supports the fusion of imagery from two sensors. Future research effort will be the extension of the AOCT method to accept imagery from three or four spectral bands, e.g. visible, short-wave infrared, middle-wave infrared, and long-wave infrared bands. In addition, designing quantitative measure for color image fusion performance is another worthwhile and challenging research topic.

8. Appendix

A. RGB to lαβ transform

In this Appendix we present the RGB to lαβ transform (Ruderman et al., 1998). This transform is derived from a principal component transform of a large ensemble of hyperspectral images that represents a good cross-section of natural scenes. The resulting data representation is compact and symmetrical, and provides automatic decorrelation to higher than second order.

The actual transform is as follows. First the RGB tristimulus values are converted to LMS space by

$$\left[\begin{array}{c}L\\ M\\ S\end{array}\right]=\left[\begin{array}{ccc}0.3811& 0.5783& 0.0402\\ 0.1967& 0.7244& 0.0782\\ 0.0241& 0.1288& 0.8444\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]$$ E23

The data in this color space shows a great deal of skew, which is largely eliminated by taking a logarithmic transform:

$$\left[\begin{array}{c}L\\ M\\ S\end{array}\right]=\left[\begin{array}{c}\log L\\ \log M\\ \log S\end{array}\right]$$ E24

The inverse transform from LMS cone space back to RGB space is as follows. First, the LMS pixel values are raised to the power ten to go back to linear LMS space.

$$\left[\begin{array}{c}L\\ M\\ S\end{array}\right]=\left[\begin{array}{c}{10}^L\\ {10}^{{\rm M}}\\ {10}^S\end{array}\right]$$ E25

Then, the data can be converted from LMS to RGB using the inverse transform of (Eq. 23):

$$\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}4.4679& -3.5873& 0.1193\\ -1.2186& 2.3809& -0.1624\\ 0.0497& -0.2439& 1.2045\end{array}\right]\left[\begin{array}{c}L\\ M\\ S\end{array}\right]$$ E26

Ruderman et al. (Ruderman et al., 1998) presented the following simple transform to decorrelate the axes in the LMS space:

$$\left[\begin{array}{c}l\\ \alpha \\ \beta \end{array}\right]=\left[\begin{array}{ccc}1/\sqrt{3}& 0& 0\\ 0& 1/\sqrt{6}& 0\\ 0& 0& 1/\sqrt{2}\end{array}\right]\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& -2\\ 1& -1& 0\end{array}\right]\left[\begin{array}{c}L\\ M\\ S\end{array}\right]$$ E27

If we think of the L channel as red, the M as green, and S as blue, we see that this is a variant of a color opponent model:

$$\begin{array}{l}\text{Achromatic}\propto R+G+B\\ \text{Yellow-blue}\propto R+G-B\\ \text{Red-green}\propto R-G\end{array}$$ E28

After processing the color signals in the lαβ space the inverse transform of (Eq. 27) can be used to return to the LMS space:

$$\left[\begin{array}{c}L\\ M\\ S\end{array}\right]=\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& -1\\ 1& -2& 0\end{array}\right]\left[\begin{array}{ccc}1/\sqrt{3}& 0& 0\\ 0& 1/\sqrt{6}& 0\\ 0& 0& 1/\sqrt{2}\end{array}\right]\left[\begin{array}{c}l\\ \alpha \\ \beta \end{array}\right]$$ E29

B. Proof of Proposition 1

From (Eq. 5) we derive that

$$\begin{array}{l}\left[\begin{array}{c}\tilde{Y}\\ {\tilde{C}}_B\\ {\tilde{C}}_R\end{array}\right]=\left[\begin{array}{ccc}x& 0& 0\\ 0& y& 0\\ 0& 0& z\end{array}\right]\left[\begin{array}{ccc}0.2990& 0.5870& 0.1140\\ -0.1687& -0.3313& 0.5000\\ 0.5000& -0.4187& -0.0813\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\\ =\left[\begin{array}{ccc}x& 0& 0\\ 0& y& 0\\ 0& 0& z\end{array}\right]\left[\begin{array}{c}Y\\ C_B\\ C_R\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]=\left[\begin{array}{c}xY\\ y{C}_B\\ z{C}_R\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\end{array}$$ E30

Hence, we can prove that the original source and target images respectively satisfy

$$\left[\begin{array}{c}{\tilde{Y}}_s\\ {\tilde{C}}_{B,s}\\ {\tilde{C}}_{R,s}\end{array}\right]=\left[\begin{array}{c}x{Y}_s\\ y{C}_{B,s}\\ z{C}_{R,s}\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]and\left[\begin{array}{c}{\tilde{Y}}_t\\ {\tilde{C}}_{B,t}\\ {\tilde{C}}_{R,t}\end{array}\right]=\left[\begin{array}{c}x{Y}_t\\ y{C}_{B,t}\\ z{C}_{R,t}\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]$$ E31

where ${[{\tilde{Y}}_s,{\tilde{C}}_{B,s},{\tilde{C}}_{R,s}]}^T$ and ${[Y_s,C_{B,s},C_{R,s}]}^T$ are the three components of the source image in ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ and YC_BC_R spaces, respectively. ${[{\tilde{Y}}_t,{\tilde{C}}_{B,t},{\tilde{C}}_{R,t}]}^T$ and $${[Y_t,C_{B,t},C_{R,t}]}^T$$ represent the three components of the target image in ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ and YC_BC_R spaces, respectively. We know that the mean and standard deviation respectively have the following properties:

$$\mu (\lambda X+c)=\lambda \mu (X)+c\begin{array}{cc}& \end{array}and\begin{array}{cc}& \end{array}\sigma (\lambda X+c)=\left|\lambda \right|\sigma (X)$$ E32

where $\lambda$ and $$c$$ are constants, $\lambda ,c\in ℝ$ , X is a random variable. Thus for the original source and target images, the means of their achromatic components in ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ and YC_BC_R spaces respectively satisfy

$${\mu }_s^{\tilde{Y}}=x{\mu }_s^Y+c_1\begin{array}{cc}& \end{array}and\begin{array}{cc}& \end{array}{\mu }_t^{\tilde{Y}}=x{\mu }_t^Y+c_1$$ E33

The corresponding standard deviations respectively satisfy

$${\sigma }_s^{\tilde{Y}}=\left|x\right|{\sigma }_s^Y\begin{array}{cc}& \end{array}and\begin{array}{cc}& \end{array}{\sigma }_t^{\tilde{Y}}=\left|x\right|{\sigma }_t^Y$$ E34

Let ${[{\tilde{Y}}_s^{*},{\tilde{C}}_{B,s}^{*},{\tilde{C}}_{R,s}^{*}]}^T$ be the three components of ${[{\tilde{R}}_s^{*},{\tilde{G}}_s^{*},{\tilde{B}}_s^{*}]}^T$ in ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ space, ${[Y_s^{*},C_{B,s}^{*},C_{R,s}^{*}]}^T$ be the three components of ${[R_s^{*},G_s^{*},B_s^{*}]}^T$ in YC_BC_R space. By inserting (Eq. 33), (Eq. 34) and $${\tilde{Y}}_s=x{Y}_s+c_1$$ in

$${\tilde{Y}}_s^{*}=({\sigma }_t^{\tilde{Y}}/{\sigma }_s^{\tilde{Y}})({\tilde{Y}}_s-{\mu }_s^{\tilde{Y}})+{\mu }_t^{\tilde{Y}}$$ E35

we can derive the relationship between ${\tilde{Y}}_s^{*}$ and $Y_s^{*}$ :

$${\tilde{Y}}_s^{*}=x{Y}_s^{*}+c_1$$ E36

In a similar way, we can obtain

$${\tilde{C}}_{B,s}^{*}=y{C}_{B,s}^{*}+c_2\begin{array}{cc}& \end{array}and\begin{array}{cc}& \end{array}{\tilde{C}}_{R,s}^{*}=z{C}_{R,s}^{*}+c_3$$ E37

From (Eq. 6) we have

$$\left[\begin{array}{c}{\tilde{R}}_s^{*}\\ {\tilde{G}}_s^{*}\\ {\tilde{B}}_s^{*}\end{array}\right]=\left[\begin{array}{ccc}1.0000x^{-1}& 0.0000& 1.4020z^{-1}\\ 1.0000x^{-1}& -0.3441y^{-1}& -0.7141z^{-1}\\ 1.0000x^{-1}& 1.7720y^{-1}& 0.0000\end{array}\right]\left(\left[\begin{array}{c}{\tilde{Y}}_s^{*}\\ {\tilde{C}}_{B,s}^{*}\\ {\tilde{C}}_{R,s}^{*}\end{array}\right]-\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\right)$$ E38

By inserting (Eq. 36) and (Eq. 37) in (Eq. 38), we derive that

$$\begin{array}{c}\left[\begin{array}{c}{\tilde{R}}_s^{*}\\ {\tilde{G}}_s^{*}\\ {\tilde{B}}_s^{*}\end{array}\right]=\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{ccc}{x}^{-1}& 0& 0\\ 0& y^{-1}& 0\\ 0& 0& z^{-1}\end{array}\right]\left(\left[\begin{array}{c}x{Y}_s^{*}+c_1\\ y{C}_{B,s}^{*}+c_2\\ z{C}_{R,s}^{*}+c_3\end{array}\right]-\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\right)\\ =\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{ccc}{x}^{-1}& 0& 0\\ 0& y^{-1}& 0\\ 0& 0& z^{-1}\end{array}\right]\left[\begin{array}{c}x{Y}_s^{*}\\ y{C}_{B,s}^{*}\\ z{C}_{R,s}^{*}\end{array}\right]\\ =\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{c}{Y}_s^{*}\\ C_{B,s}^{*}\\ C_{R,s}^{*}\end{array}\right]=\left[\begin{array}{c}{R}_s^{*}\\ G_s^{*}\\ B_s^{*}\end{array}\right]\end{array}$$ E39

This completes the proof.

C. Proof of Proposition 3

Let ${[{\tilde{Y}}_s,{\tilde{C}}_{B,s},{\tilde{C}}_{R,s}]}^T$ and ${[Y_s,C_{B,s},C_{R,s}]}^T$ be the source color components in the ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ based and YC_BC_R based AOCT methods, respectively. Thus, from the given condition we have

$$\left[\begin{array}{c}{\tilde{Y}}_s\\ {\tilde{C}}_{B,s}\\ {\tilde{C}}_{R,s}\end{array}\right]=\left[\begin{array}{c}{Y}_s\\ C_{B,s}\\ C_{R,s}\end{array}\right]=\left[\begin{array}{c}F\\ Vis-IR\\ IR-Vis\end{array}\right]$$ E40

Let ${[{\tilde{Y}}_t,{\tilde{C}}_{B,t},{\tilde{C}}_{R,t}]}^T$ and $${[Y_t,C_{B,t},C_{R,t}]}^T$$ be the three components of the target image in ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ and YC_BC_R spaces, respectively. We know from (Eq. 30) that the target image satisfies

$$\left[\begin{array}{c}{\tilde{Y}}_t\\ {\tilde{C}}_{B,t}\\ {\tilde{C}}_{R,t}\end{array}\right]=\left[\begin{array}{c}x{Y}_t\\ y{C}_{B,t}\\ z{C}_{R,t}\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]$$ E41

Then, under the given assumption that x, y and z > 0, using the mean and standard deviation properties, from (Eq. 40) and (Eq. 41) we can derive

$$\left[\begin{array}{c}{\mu }_s^{\tilde{Y}}\\ {\mu }_s^{{\tilde{C}}_B}\\ {\mu }_s^{{\tilde{C}}_R}\end{array}\right]=\left[\begin{array}{c}{\mu }_s^Y\\ {\mu }_s^{C_B}\\ {\mu }_s^{C_R}\end{array}\right],\left[\begin{array}{c}{\sigma }_s^{\tilde{Y}}\\ {\sigma }_s^{{\tilde{C}}_B}\\ {\sigma }_s^{{\tilde{C}}_R}\end{array}\right]=\left[\begin{array}{c}{\sigma }_s^Y\\ {\sigma }_s^{C_B}\\ {\sigma }_s^{C_R}\end{array}\right],\left[\begin{array}{c}{\mu }_t^{\tilde{Y}}\\ {\mu }_t^{{\tilde{C}}_B}\\ {\mu }_t^{{\tilde{C}}_R}\end{array}\right]=\left[\begin{array}{c}x{\mu }_t^Y\\ y{\mu }_t^{C_B}\\ z{\mu }_t^{C_R}\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right],\left[\begin{array}{c}{\sigma }_t^{\tilde{Y}}\\ {\sigma }_t^{{\tilde{C}}_B}\\ {\sigma }_t^{{\tilde{C}}_R}\end{array}\right]=\left[\begin{array}{c}x{\sigma }_t^Y\\ y{\sigma }_t^{C_B}\\ z{\sigma }_t^{C_R}\end{array}\right]$$ E42

Let ${[{\tilde{Y}}_c,{\tilde{C}}_{B,c},{\tilde{C}}_{R,c}]}^T$ be the three components of $${[{\tilde{R}}_c,{\tilde{G}}_c,{\tilde{B}}_c]}^T$$ in ${\text{<mover accent="true"><mi>Y</mi> <mo>˜</mo> </mover> <mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{B}}{\text{<mover accent="true"><mi>C</mi> <mo>˜</mo> </mover>}}_{\text{R}}$ space, $${[Y_c,C_{B,c},C_{R,c}]}^T$$ be the three components of ${[R_c,G_c,B_c]}^T$ in YC_BC_R space, respectively. By inserting (Eq. 40), (Eq. 41) and (Eq. 42) in (Eq. 12), we deduce that

$$\left[\begin{array}{c}{\tilde{Y}}_c\\ {\tilde{C}}_{B,c}\\ {\tilde{C}}_{R,c}\end{array}\right]=\left[\begin{array}{c}x{Y}_c\\ y{C}_{B,c}\\ z{C}_{R,c}\end{array}\right]+\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]$$ E43

From (Eq. 6) we have

$$\left[\begin{array}{c}{\tilde{R}}_c\\ {\tilde{G}}_c\\ {\tilde{B}}_c\end{array}\right]=\left[\begin{array}{ccc}1.0000x^{-1}& 0.0000y^{-1}& 1.4020z^{-1}\\ 1.0000x^{-1}& -0.3441y^{-1}& -0.7141z^{-1}\\ 1.0000x^{-1}& 1.7720y^{-1}& 0.0000z^{-1}\end{array}\right]\left(\left[\begin{array}{c}{\tilde{Y}}_c\\ {\tilde{C}}_{B,c}\\ {\tilde{C}}_{R,c}\end{array}\right]-\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\right)$$ E44

Thus, by inserting (Eq. 43) in (Eq. 44), we can derive that

$$\begin{array}{c}\left[\begin{array}{c}{\tilde{R}}_c\\ {\tilde{G}}_c\\ {\tilde{B}}_c\end{array}\right]=\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{ccc}{x}^{-1}& 0& 0\\ 0& y^{-1}& 0\\ 0& 0& z^{-1}\end{array}\right]\left(\left[\begin{array}{c}x{Y}_c+c_1\\ y{C}_{B,c}+c_2\\ z{C}_{R,c}+c_3\end{array}\right]-\left[\begin{array}{c}{c}_1\\ c_2\\ c_3\end{array}\right]\right)\\ =\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{ccc}{x}^{-1}& 0& 0\\ 0& y^{-1}& 0\\ 0& 0& z^{-1}\end{array}\right]\left[\begin{array}{c}x{Y}_c\\ y{C}_{B,c}\\ z{C}_{R,c}\end{array}\right]\\ =\left[\begin{array}{ccc}1.0000& 0.0000& 1.4020\\ 1.0000& -0.3441& -0.7141\\ 1.0000& 1.7720& 0.0000\end{array}\right]\left[\begin{array}{c}{Y}_c\\ C_{B,c}\\ C_{R,c}\end{array}\right]=\left[\begin{array}{c}{R}_c\\ G_c\\ B_c\end{array}\right]\end{array}$$ E45

This finishes the proof.

Acknowledgments

The author thanks everyone who contributed images to this chapter. The author also thanks TNO Human Factors, ImageFusion.org, Pics4Learning.com, BigFoto.com, and FreeFoto.com for test images acquisition.