Non Separable Two Dimensional Discrete Wavelet Transform for Image Signals

1. Introduction

Over the past few decades, a considerable number of studies have been conducted on two dimensional (2D) discrete wavelet transforms (DWT) for image or video signals. Ever since the JPEG 2000 has been adopted as an international standard for digital cinema applications, there has been a renewal of interest in hardware and software implementation of a lifting DWT, especially in attaining high throughput and low latency processing for high resolution video signals [1, 2].

Intermediate memory utilization has been studied introducing a line memory based implementation [3]. A lifting factorization has been proposed to reduce auxiliary buffers to increase throughput for boundary processing in the block based DWT [4]. Parallel and pipeline techniques in the folded architecture have been studied to increase hardware utilization, and to reduce the critical path latency [5, 6]. However, in the lifting DWT architecture, overall delay of its output signal is curial to the number of lifting steps inside the DWT.

In this chapter, we discuss on constructing a ‘non-separable’ 2D lifting DWT with reduced number of lifting steps on the condition that the DWT has full compatibility with the ‘separable’ 2D DWT in JPEG 2000. One of straightforward approaches to reduce the latency of the DWT is utilization of 2D memory accessing (not a line memory). Its transfer function is factorized into non-separable (NS) 2D transfer functions. So far, quite a few NS factorization techniques have been proposed [7, 14]. The residual correlation of the Haar transform was utilized by a NS lifting structure [7]. The Walsh Hadamard transform was composed of a NS lossless transform [8], and applied to construct a lossless discrete cosine transform (DCT) [9]. Morphological operations were applied to construct an adaptive prediction [10]. Filter coefficients were optimized to reduce the aliasing effect [11]. However, these transforms are not compatible with the DWT defined by the JPEG 2000 international standard.

In this chapter, we describe a family of NS 2D lifting DWTs compatible with DWTs defined by JPEG 2000 [12, 14]. One of them is compatible with the 5/3 DWT developed for lossless coding [12]. The other is compatible with the 9/7 DWT developed for lossy coding [13]. It is composed of single NS function structurally equivalent to [12]. For further reduction of the lifting steps, we also describe another structure composed of double NS functions [14]. The NS 2D DWT family summarized in this chapter has less lifting steps than the standard separable 2D DWT set, and therefore it contributes to reduce latency of DWT for faster coding.

This chapter is organized as follows. Standard 'separable' 2D DWT and its latency due to the total number of lifting steps are discussed, and a low latency 'non-separable' 2D DWT is introduced for 5/3 DWT in section 2. The discussion is expanded to 9/7 DWT in section 3. In each section, it is confirmed that the total number of lifting steps is reduced by the 'non-separable' DWT without changing relation between input and output of the 'separable' DWT. Furthermore, structures to implement 'lossless' coding are described for not only 5/3 DWT but also for 9/7 DWT. Performance of the DWTs is investigated and compared in respect of lossless coding and lossy coding in section 4. Implementation issue under finite word length of signal values is also discussed. Conclusions are summarized in section 5. References are listed in section 6.

2. The 5/3 DWT and Reduction of its Latency

JPEG 2000 defines two types of one dimensional (1D) DWTs. One is 5/3 DWT and the other is 9/7 DWT. Each of them is applied to a 2D input image signal, vertically and horizontally. This processing is referred to 'separable' 2D structure. In this section, we point out the latency problem due to the total number of lifting steps of the DWT, and introduce a 'non separable' 2D structure with reduced number of lifting steps for 5/3 DWT.

2.1. One Dimensional 5/3 DWT defined by JPEG 2000

Fig.1 illustrates a pair of forward and backward (inverse) transform of the one dimensional (1D) 5/3 DWT. Its forward transform splits the input signal X into two frequency band signals L and H with down samplers ↓2, a shifter z ⁺¹ and FIR filters H₁ and H₂. The input signal X is given as a sequence x_n, n $$\in$$ {0,1, $$\cdots$$ , N-1} with length N. The band signals L and H are also given as sequences l_m and h_m, m $$\in$$ {0,1, $$\cdots$$ , M-1}, respectively. Both of them have the length M=N/2. Using the z transform, these signals are expressed as

$$X(z)={\displaystyle \sum _{n=0}^{N-1}}{x}_n{z}^{-n},L(z)={\displaystyle \sum _{m=0}^{M-1}}{l}_m{z}^{-m},H(z)={\displaystyle \sum _{m=0}^{M-1}}{h}_m{z}^{-m}$$ E1

Relation between input and output of the forward transform is expressed as

$$\left[\begin{array}{c}L(z)\\ H(z)\end{array}\right]=\left[\begin{array}{cc}1& H_2(z)\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ H_1(z)& 1\end{array}\right]\left[\begin{array}{c}{X}_e(z)\\ X_o(z)\end{array}\right]$$ E2

where

$$\left[\begin{array}{c}{X}_e(z)\\ X_o(z)\end{array}\right]=\left[\begin{array}{c}\downarrow 2[X(z)]\\ \downarrow 2[X(z)z]\end{array}\right]=\downarrow 2\left[\left[\begin{array}{c}1\\ z\end{array}\right]X(z)\right]$$ E3

The backward (inverse) transform synthesizes the two band signals L and H into the signal X' by

$$X\text{'}(z)=[1z^{-1}]\left[\begin{array}{c}\uparrow 2[X_e^{\text{'}}(z)]\\ \uparrow 2[X_o^{\text{'}}(z)]\end{array}\right]=[1z^{-1}]\uparrow 2\left[\begin{array}{c}{X}_e^{\text{'}}(z)\\ X_o^{\text{'}}(z)\end{array}\right]$$ E4

where

$$\left[\begin{array}{c}{X}_e^{\text{'}}(z)\\ X_o^{\text{'}}(z)\end{array}\right]=\left[\begin{array}{cc}1& 0\\ -H_1(z)& 1\end{array}\right]\left[\begin{array}{cc}1& -H_2(z)\\ 0& 1\end{array}\right]\left[\begin{array}{c}L(z)\\ H(z)\end{array}\right]$$ E5

In the equations (3) and (4), down sampling and up sampling are defined as

$$\left[\begin{array}{c}\downarrow 2[W(z)]\\ \uparrow 2[W(z)]\end{array}\right]=\left[\begin{array}{cc}1/2& 0\\ 0& 1\end{array}\right]\left[\begin{array}{c}W(z^{1/2})+W(-z^{1/2})\\ W(z^2)\end{array}\right]$$ E6

respectively for an arbitrary signal W(z). In Fig.1, the FIR filters H₁ and H₂ are given as

$$\left[\begin{array}{c}{\text{H}}_1\\ {\text{H}}_2\end{array}\right]=\left[\begin{array}{c}{H}_1(z)\\ H_2(z)\end{array}\right]=\left[\begin{array}{cc}-1/2& 0\\ 0& 1/4\end{array}\right]\left[\begin{array}{c}(1+z^{+1})\\ (1+z^{-1})\end{array}\right]$$ E7

for 5/3 DWT defined by the JPEG 2000 international standard.

2.2. Separable 2D 5/3 DWT of JPEG 2000 and its Latency

Fig.2 illustrates extension of the 1D DWT to 2D image signal. The 1D DWT is applied vertically and horizontally. In this case, an input signal is denoted as

$$X(z_1,z_2)={\displaystyle \sum _{n_1=0}^{N_1-1}}{\displaystyle \sum _{n_2=0}^{N_2-1}}{x}_{n_1,n_2}{z}_1^{-n_1}{z}_2^{-n_2}$$ E8

Down sampling and up sampling are defined as

$$\left[\begin{array}{c}\downarrow 2_{z1}[W(z_1,z_2)]\\ \downarrow 2_{z2}[W(z_1,z_2)]\end{array}\right]=\left[\begin{array}{cc}1/2& 0\\ 0& 1/2\end{array}\right]\left[\begin{array}{c}W(z_1^{1/2},z_2)+W(-z_1^{1/2},z_2)\\ W(z_1,z_2^{1/2})+W(z_1,-z_2^{1/2})\end{array}\right]$$ E9

and

$$\left[\begin{array}{c}\uparrow 2_{z1}[W(z_1,z_2)]\\ \uparrow 2_{z2}[W(z_1,z_2)]\end{array}\right]=\left[\begin{array}{c}W(z_1^2,z_2)\\ W(z_1,z_2^2)\end{array}\right]$$ E10

respectively for an arbitrary 2D signal W(z₁,z₂). The FIR filters H₁ and H₂ are given as

$$\left[\begin{array}{c}{\text{H}}_1\\ {\text{H}}_2\end{array}\right]=\left[\begin{array}{c}{H}_1(z_1)\\ H_2(z_1)\end{array}\right]=\left[\begin{array}{cc}-1/2& 0\\ 0& 1/4\end{array}\right]\left[\begin{array}{c}(1+z_1^{+1})\\ (1+z_1^{-1})\end{array}\right]$$ E11

$$\left[\begin{array}{c}{\text{H}}_{\text{1}}^{\text{*}}\\ {\text{H}}_{\text{2}}^{\text{*}}\end{array}\right]=\left[\begin{array}{c}{H}_1(z_2)\\ H_2(z_2)\end{array}\right]=\left[\begin{array}{cc}-1/2& 0\\ 0& 1/4\end{array}\right]\left[\begin{array}{c}(1+z_2^{+1})\\ (1+z_2^{-1})\end{array}\right]$$ E12

for Fig.2, instead of (7) for Fig.1.

The structure in Fig.2 has 4 lifting steps in total. It should be noted that a lifting step must wait for a calculation result from the previous lifting step. It causes delay and it is essentially inevitable. Therefore the total number of lifting steps (= latency) should be reduced for faster coding of JPEG 2000.

Figure 1.

One dimensional 5/3 DWT defined by JPEG 2000.

The procedure described above can be expressed in matrix form. Since Fig.2 can be expressed as Fig.3, relation between input vector X and output vector Y is denoted as

$$Y=(L_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23})(L_{{\text{H}}_2,{\text{H}}_1}{P}_{23})X$$ E13

for

$$X={[\begin{array}{cccc}{X}_{11}& X_{12}& X_{21}& X_{22}\end{array}]}^T, Y={[\begin{array}{cccc}LL& LH& HL& HH\end{array}]}^T$$ E14

and

$$\begin{array}{r}{P}_{23}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 1\end{array}\right]L_{p,q}=\left[\begin{array}{cc}\left[\begin{array}{cc}1& p\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ q& 1\end{array}\right]& \left[\begin{array}{cc}0& 0\\ 0& 0\end{array}\right]\\ \left[\begin{array}{cc}0& 0\\ 0& 0\end{array}\right]& \left[\begin{array}{cc}1& p\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ q& 1\end{array}\right]\end{array}\right]\\ forp,q\in \{{\text{H}}_1,{\text{H}}_2,{\text{H}}_{\text{1}}^{\text{*}},{\text{H}}_{\text{2}}^{\text{*}}\}\end{array}$$ E15

Fig.4 illustrates that each of the lifting step performs interpolation from neighboring pixels. Each step must wait for calculation result of the previous step. It causes delay. Our purpose in this chapter is to reduce the total number of lifting steps so that the latency is lowered.

Figure 2.

Separable 2D 5/3 DWT defined by JPEG 2000.

2.3. Non Separable 2D 5/3 DWT for Low latency JPEG 2000 Coding

In this subsection, we reduce the latency using 'non separable' structure without changing relation between X and Y in (13). Fig.5 illustrates a theorem we used in this chapter to construct a non-separable DWT. It is expressed as

Theorem 1;

$$Y=N_{d,c,b,a}X$$ E16

for

$$X={[\begin{array}{cccc}{x}_1& x_2& x_3& x_4\end{array}]}^T, Y={[\begin{array}{cccc}{y}_1& y_2& y_3& y_4\end{array}]}^T$$ E17

where

$$N_{d,c,b,a}=\left[\begin{array}{cccc}1& d& b& -bd\\ c& 1& 0& b\\ a& 0& 1& d\\ ac& a& c& 1\end{array}\right]$$ E18

for arbitrary value of a, b, c and d. These values can be either scalars or transfer functions. Therefore, substituting

$$L_{d,c}{P}_{23}{L}_{b,a}{P}_{23}=N_{d,c,b,a}$$ E19

with

$$[\begin{array}{cccc}a& b& c& d\end{array}]=[\begin{array}{cccc}{\text{H}}_1& {\text{H}}_2& {\text{H}}_1^{*}& {\text{H}}_2^{*}\end{array}]$$ E20

into (13), we have

$$Y=N_{{\text{H}}_2^{*},{\text{H}}_1^{*},{\text{H}}_2,{\text{H}}_1}X$$ E21

for X and Y in (14).

Finally, the non-separable 2D 5/3 DWT is constructed as illustrated in Fig.6. It has 3 lifting steps in total. The total number of lifting steps (= latency) is reduced from 4 (100%) to 3 (75%) as summarized in table 1 (separable lossy 5/3). Signal processing of each lifting step is equivalent to the interpolation illustrated in Fig.7. In the 2nd step, two interpolations can be simultaneously performed with parallel processing. Note that the non-separable 2D DWT requires 2D memory accessing.

2.4. Introduction of Rounding Operation for Lossless Coding

In Fig.1, the output signal X' is equal to the input signal X as far as all the sample values of the band signals L and H are stored with long enough word length. However, in data compression of JPEG 2000, all the sample values of the band signals are quantized into integers before they are encoded with an entropy coder EBCOT. Therefore the output signal X' has some loss, namely X'-X $$\ne$$ 0. It is referred to 'lossy' coding.

Figure 3.

Separable 2D 5/3 DWT for matrix expression (5/3 Sep).

Figure 4.

Interpretation of separable 2D 5/3 DWT as interpolation.

Figure 5.

Theorem 1.

However, introducing rounding operations in each lifting step, all the DWTs mentioned above become 'lossless'. In this case, a rounding operation is inserted before addition and subtraction in Fig.1 as illustrated in Fig.8. It means

$$\{\begin{array}{l}{y}^{*}=x+Round[x_0+x_1+x_2]\\ x\text{'}=y^{*}-Round[x_0+x_1+x_2]\end{array}$$ E22

which guarantees 'lossless' reconstruction of the input value, namely x'-x=0. In this structure for lossless coding, comparing '5/3 Sep' in Fig.3 and '5/3 Ns1' in Fig.6, the total number of rounding operation is reduced from 8 (100%) to 4 (50%) as summarized in table 2. It contributes to increasing coding efficiency.

Figure 6.

Non Separable 2D 5/3 DWT (5/3 Ns1).

Figure 7.

Interpretation of non-separable 2D 5/3 DWT as interpolation.

Figure 8.

Rounding operations for lossless coding.

3. The 9/7 DWT and Reduction of its Latency

In the previous section, it was indicated that replacing the normal 'separable' structure by the 'non-separable' structure reduces the total number of lifting steps. It contributes to faster processing of DWT in JPEG 2000 for both of lossy coding and lossless coding. It was also indicated that it reduces total number of rounding operations in DWT for lossless coding. All the discussions above are limited to 5/3 DWT. In this section, we expand our discussion to 9/7 DWT for not only lossy coding, but also for lossless coding.

3.1. Separable 2D 9/7 DWT of JPEG 2000 and its Latency

JPEG 2000 defines another type of DWT referred to 9/7 DWT for lossy coding. It can be expanded to lossless coding as described in subsection 3.4. Comparing to 5/3 DWT in Fig.1, 9/7 DWT has two more lifting steps and a scaling pair. Filter coefficients are also different from (7). They are given as

$$\left[\begin{array}{c}{H}_1(z)\\ H_2(z)\end{array}\right]=\left[\begin{array}{cc}\alpha & 0\\ 0& \beta \end{array}\right]\left[\begin{array}{c}(1+z^{+1})\\ (1+z^{-1})\end{array}\right],\left[\begin{array}{c}{H}_3(z)\\ H_4(z)\end{array}\right]=\left[\begin{array}{cc}\gamma & 0\\ 0& \delta \end{array}\right]\left[\begin{array}{c}(1+z^{+1})\\ (1+z^{-1})\end{array}\right]$$ E23

and

$$\{\begin{array}{l}\alpha =-\text{1}\text{.586134342059924}\cdots ,\beta =-\text{0}\text{.052980118572961}\cdots \\ \chi =+\text{0}\text{.882911075530934}\cdots ,\delta =+\text{0}\text{.443506852043971}\cdots \\ k=+\text{1}\text{.230174104914001}\cdots \end{array}$$ E24

for 9/7 DWT of JPEG 2000. Fig.9 illustrates the separable 2D 9/7 DWT. In the figure, filters are denoted as

$$\left[\begin{array}{c}{\text{H}}_1\\ {\text{H}}_2\end{array}\begin{array}{c}{\text{H}}_3\\ {\text{H}}_4\end{array}\right]=\left[\begin{array}{c}{H}_1(z_1)\\ H_2(z_1)\end{array}\begin{array}{c}{H}_3(z_1)\\ H_4(z_1)\end{array}\right]$$ E25

$$\left[\begin{array}{c}{\text{H}}_{\text{1}}^{\text{*}}\\ {\text{H}}_{\text{2}}^{\text{*}}\end{array}\begin{array}{c}{\text{H}}_{\text{3}}^{\text{*}}\\ {\text{H}}_{\text{4}}^{\text{*}}\end{array}\right]=\left[\begin{array}{c}{H}_1(z_2)\\ H_2(z_2)\end{array}\begin{array}{c}{H}_3(z_2)\\ H_4(z_2)\end{array}\right]$$ E26

It should be noted that this structure has 8 lifting steps.

Fig.10 also illustrates the separable 2D 9/7 DWT for matrix representation. Similarly to (13), it is expressed as

$$Y=(J_k{L}_{{\text{H}}_4^{*},{\text{H}}_3^{*}}{L}_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23})\cdot (J_k{L}_{{\text{H}}_4,{\text{H}}_3}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23})X$$ E27

for

$$X={[\begin{array}{cccc}{X}_{11}& X_{12}& X_{21}& X_{22}\end{array}]}^T,Y={[\begin{array}{cccc}LL& LH& HL& HH\end{array}]}^T$$ E28

and

$$\begin{array}{l}{P}_{23}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 1\end{array}\right],\left[\begin{array}{c}{L}_{p,q}\\ J_k\end{array}\right]=\left[\begin{array}{c}diag[\begin{array}{cc}{M}_{p,q}& M_{p,q}\end{array}]\\ diag[\begin{array}{cc}{K}_k& K_k\end{array}]\end{array}\right]\\ forp,q\in \{{\text{H}}_r,{\text{H}}_{\text{r}}^{\text{*}}\},r\in \{\text{1},\text{2},\text{3},\text{4}\}\end{array}$$ E29

In (29), a scaling pair K_k and filter a matrix K_p,q are defined as

$$K_k=\left[\begin{array}{cc}{k}^{-1}& 0\\ 0& k\end{array}\right],M_{p,q}=\left[\begin{array}{cc}1& p\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ q& 1\end{array}\right]$$ E30

Figure 9.

Separable 2D 9/7 DWT in JPEG 2000.

3.2. Single Non Separable 2D 9/7 DWT for Low latency JPEG 2000 coding

In this subsection, we reduce the latency using 'non separable' structure without changing relation between X and Y in (27), using the theorem 1 in (16)-(18) illustrated in Fig.5. Starting from Fig.10, unify the four scaling pairs {k ^-1, k} to only one pair {k ^-2, k²} as illustrated in Fig.11. It is denoted as

$$\begin{array}{l}(J_k{L}_{{\text{H}}_4^{*},{\text{H}}_3^{*}}{L}_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23})(J_k{L}_{{\text{H}}_4,{\text{H}}_3}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23})\\ =J_k^{*}\cdot L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}{L}_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23}{L}_{{\text{H}}_4,{\text{H}}_3}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23}\\ =J_k^{*}\cdot L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}(L_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23}{L}_{{\text{H}}_4,{\text{H}}_3}{P}_{23})P_{23}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23}\end{array}$$ E31

where

$$J_k^{*}=diag[\begin{array}{cccc}{k}^{-2}& 1& 1& k^2{}^{}\end{array}]$$ E32

Next, applying the theorem 1, we have the single non-separable 2D DWT as illustrated in Fig.12. It is denoted as

$$\begin{array}{l}{J}_k^{*}\cdot L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}(L_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23}{L}_{{\text{H}}_4,{\text{H}}_3}{P}_{23})P_{23}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23}\\ =J_k^{*}\cdot L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}(N_{{\text{H}}_2^{*},{\text{H}}_1^{*},{\text{H}}_4,{\text{H}}_3})P_{23}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23}\end{array}$$ E33

As a result, the total number of lifting steps (= latency) is reduced from 8 (100%) to 7 (88%) as summarized in table 1 (non-separable lossy 9/7).

Figure 10.

Separable 2D 9/7 DWT for matrix expression.

3.3. Double Non Separable 9/7 DWT for Low latency JPEG 2000 Coding

In the previous subsection, a part of the separable structure is replaced by a non-separable structure. In this subsection, we reduce one more lifting step using one more non-separable structure. Starting from equation (31) illustrated in Fig. 11, we apply

Theorem 2;

$$L_{{\text{H}}_s^{*},{\text{H}}_r^{*}}{P}_{23}{L}_{{\text{H}}_q,{\text{H}}_p}{P}_{23}=P_{23}{L}_{{\text{H}}_q,{\text{H}}_p}{P}_{23}{L}_{{\text{H}}_s^{*},{\text{H}}_r^{*}}$$ E34

Namely, (31) becomes

$$\begin{array}{l}{J}_k^{*}\cdot L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}(L_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23}{L}_{{\text{H}}_4,{\text{H}}_3}{P}_{23})P_{23}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23}\\ =J_k^{*}\cdot L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}(P_{23}{L}_{{\text{H}}_4,{\text{H}}_3}{P}_{23}{L}_{{\text{H}}_2^{*},{\text{H}}_1^{*}})P_{23}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23}\end{array}$$ E35

as illustrated in Fig.13. Then the theorem 1 can be applied twice as

$$\begin{array}{l}{J}_k^{*}\cdot (L_{{\text{H}}_4^{*},{\text{H}}_3^{*}}{P}_{23}{L}_{{\text{H}}_4,{\text{H}}_3}{P}_{23})(L_{{\text{H}}_2^{*},{\text{H}}_1^{*}}{P}_{23}{L}_{{\text{H}}_2,{\text{H}}_1}{P}_{23})\\ =J_k^{*}\cdot N_{{\text{H}}_4^{*},{\text{H}}_3^{*},{\text{H}}_4,{\text{H}}_3}{N}_{{\text{H}}_2^{*},{\text{H}}_1^{*},{\text{H}}_2,{\text{H}}_1}\end{array}$$ E36

Figure 11.

Derivation of single non separable 2D 9/7 DWT (step 1/2).

Figure 12.

Derivation of single non separable 2D 9/7 DWT (step 2/2).

and finally, we have the double non-separable 2D DWT as illustrated in Fig.14. The total number of the lifting steps is reduced from 8 (100%) to 6 (75 %). This reduction rate is the same for the multi stage octave decomposition with DWTs.

Figure 13.

Derivation of double non separable 2D 9/7 DWT (step 1/2).

Figure 14.

Derivation of double non separable 2D 9/7 DWT (step 2/2).

3.4. Lifting Implementation of Scaling for Lossless Coding

Due to the scaling pair {k ^-2, k²}, the DWT in Fig.14 can't be lossless, and therefore it is utilized for lossy coding. However, as explained in subsection 2.4, it becomes lossless when all the scaling pairs are implemented in lifting form with rounding operations in Fig.8. For example, the scaling pair K_k in equation (30) is factorized into lifting steps as

$$K_k^{(L)}=\left[\begin{array}{cc}1& s_4\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ s_3& 1\end{array}\right]\left[\begin{array}{cc}1& s_2\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ s_1& 1\end{array}\right]$$ E37

for

$$\left[\begin{array}{cc}{s}_1& s_3\\ s_2& s_4\end{array}\right]=\left[\begin{array}{cc}k\cdot s_1& 0\\ 0& {(k\cdot s_1)}^{-1}\end{array}\right]\left[\begin{array}{cc}{k}^{-1}& -1\\ 1-k& 1-k^{-1}\end{array}\right]$$ E38

Similarly, the scaling pair in equation (32) is also factorized as

$$J_k^{*(L)}=\left[\begin{array}{cccc}1& 0& 0& t_4\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ t_3& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& t_2\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ t_1& 0& 0& 1\end{array}\right]$$ E39

for

$$\left[\begin{array}{cc}{t}_1& t_3\\ t_2& t_4\end{array}\right]=\left[\begin{array}{cc}{k}^2{t}_1& 0\\ 0& {(k^2{t}_1)}^{-1}\end{array}\right]\left[\begin{array}{cc}{k}^{-2}& -1\\ 1-k^2& 1-k^{-2}\end{array}\right]$$ E40

as illustrated in Fig.15. In the equation above, t₁ can be set to 1 [15].

Figure 15.

Lifting implementation of scaling pairs.

Figure 16.

Separable 2D 9/7 DWT for lossless coding (9/7 Sep).

Fig.16, Fig.17 and Fig.18 illustrate 2D 9/7 DWTs for lossless coding. As summarized in table 1, it is indicated that the total number of lifting steps is reduced from 16 (100%) in Fig.16 to 11 (69%) in Fig.17 and 10 (63%) in Fig.18. Furthermore, the total number of rounding operations is also reduced from 32 (100%) in Fig.16 to 16 (50%) in Fig.17 and 12 (38%) as summarized in table 2.

Figure 17.

Single non separable 2D 9/7 DWT for lossless coding (9/7 Ns1).

Figure 18.

Double non separable 2D 9/7 DWT for lossless coding (9/7 Ns2).

		lossy		lossless
5/3	9/7	5/3	9/7
separable		4 (100%)	8 (100%)	4 (100%)	16 (100%)
non; separable	single	3 ( 75%)	7 ( 88%)	3 ( 75%)	11 ( 69%)
double	---	6 ( 75%)	---	10 ( 63%)

Table 1.

Total number of lifting steps

4. Performance Evaluation

In this section, all the DWTs summarized in table 3 are compared in respect of lossless coding performance first. Lossy coding performance is evaluated next and a problem due to finite word length implementation is pointed out. This problem is avoided by compensating word length at the minimum cost.

		lossless
5/3	9/7
separable		8 (100%)	32 (100%)
non; separable	single	4 ( 50%)	16 ( 50%)
double	---	12 ( 38%)

Table 2.

Total number of rounding operations

		lossless
5/3	9/7
separable		5/3 Sep (Fig.3)	9/7 Sep (Fig.16)
non; separable	single	5/3 Ns1 (Fig.6)	9/7 Ns1 (Fig.17)
double	---	9/7 Ns2 (Fig.18)

Table 3.

DWTs discussed in this chapter

4.1 Lossless Coding Performance

Table 4 summarizes lossless coding performance of the DWTs in table 3 at different number of stages in octave decomposition. The EBCOT is applied as an entropy coder without quantization or bit truncation. Results were evaluated in bit rate (= average code length per pixel) in [bpp]. Fig.19 illustrates the bit rate averaged over images. It indicates that '5/3 Ns1' is the best followed by '5/3 Sep'. The difference between them is only 0.01 to 0.02 [bpp]. Among 9/7 DWTs, '9/7 Ns1' is the best followed by '9/7 Sep'. The difference is 0.03 to 0.04 [bpp]. As a result of this experiment, it was found that there is no significant difference in lossless coding performance.

Image	DWT	Number of Stages
1	2	3	4	5	6
Couple	5/3 Sep	4.74	4.65	4.63	4.62	4.62	4.62
5/3 Ns1	4.73	4.64	4.62	4.61	4.61	4.61
9/7 Sep	4.91	4.83	4.81	4.80	4.80	4.80
9/7 Ns1	4.89	4.80	4.79	4.78	4.78	4.77
9/7 Ns2	4.93	4.84	4.82	4.81	4.81	4.81
Boat	5/3 Sep	4.78	4.70	4.69	4.69	4.69	4.69
5/3 Ns1	4.77	4.69	4.69	4.68	4.68	4.68
9/7 Sep	4.87	4.80	4.80	4.79	4.79	4.79
9/7 Ns1	4.85	4.78	4.77	4.77	4.77	4.77
9/7 Ns2	4.87	4.80	4.80	4.79	4.79	4.79
Lena	5/3 Sep	5.06	4.97	4.95	4.95	4.95	4.95
5/3 Ns1	5.05	4.96	4.94	4.94	4.94	4.94
9/7 Sep	5.19	5.09	5.07	5.07	5.07	5.07
9/7 Ns1	5.17	5.06	5.05	5.04	5.05	5.05
9/7 Ns2	5.18	5.07	5.06	5.05	5.06	5.06
average	5/3 Sep	4.86	4.77	4.76	4.75	4.75	4.75
5/3 Ns1	4.85	4.76	4.75	4.74	4.74	4.74
9/7 Sep	4.99	4.91	4.89	4.89	4.89	4.89
9/7 Ns1	4.97	4.88	4.87	4.86	4.87	4.86
9/7 Ns2	4.99	4.90	4.89	4.88	4.89	4.89

Table 4.

Bit rate for each image in lossless coding [bpp].

Figure 19.

Bit rate averaged over images in lossless coding [bpp].

4.2. Lossy Coding Performance

Fig.20 indicates rate distortion curves of the DWTs in table 3 for an input image 'Lena'. Five-stage octave decomposition of DWT is applied. Transformed coefficients are quantized with the optimum bit allocation and EBCOT is applied as an entropy coder. In the figure, PSNR is calculated as

$$Q=-10{\log }_{10}\frac{1}{255{}^2{N}_1{N}_2}{\displaystyle \sum _{n_1=0}^{N_1-1}}{\displaystyle \sum _{n_2=0}^{N_2-1}}{D}_{n_1,n_2}^2$$ E41

where

$$D_{n_1,n_2}=Y_{n_1,n_2}-X_{n_1,n_2}$$ E42

From an input image X_n1,n2, a reconstructed image Y_n1,n2 is generated through the forward transform of the 5/3 (or 9/7) DWTs in table 3, and the backward transform of the standard 5/3 (or 9/7) DWT defined by JPEG 2000. This is to investigate compatibility between the non-separable DWTs for lossless coding, and the separable DWTs in JPEG 2000 for lossy coding.

As indicated in Fig.20, there is no difference among '9/7 Sep', '9/7 Ns1' and '9/7 Ns2'. All of them have the same rate-distortion curve. There is also no difference between '5/3 Sep' and '5/3 Ns1'. It indicates that the non-separable DWTs in table 3 have perfect compatibility with the standard DWTs defined by JPEG 2000. Note that this is true under long enough word length. In this experiment, word length of signals F_s of both of the forward and the backward transform is set to 64 [bit].

Figure 20.

Rare distortion curves at F_s=64 [bit] word length of signals.

4.3. Finite Word Length Implementation

Fig.21 indicates rate distortion curves for the same image but word length of signals in the forward transform is shortened just after each of multiplications. Signal values are multiplied by 2^-Fs, floored to integers and then multiplied by 2^Fs. As a result, all the signals have the word length F_s [bit] in fraction. According to the figure, it was observed that '9/7 Ns1' is slightly worse than '9/7 Sep', and '9/7 Ns2' is much worse. It was found that the NS DWTs have quality deterioration problem at high bit rates in lossy coding, even though they have less lifting steps.

Figure 21.

Rare distortion curves at F_s=2 [bit] word length of signals.

To cope with this problem, word length is compensated for '9/7 Ns2' at the minimum cost of word length. In case of finite word length implementation, the distortion D_n1,n2 in (42) contains two kinds of errors; a) quantization noise q for rate control in lossy coding and b) truncation noise c due to finite word length expression of signals inside the forward transform. Namely, D_n1,n2 =q+c. Assuming that q and c are uncorrelated and both of them has zero mean, variance of the distortion is approximated as

$$Var[D]=\{\begin{array}{l}Var[q](qc)\\ Var[c](qc)\end{array}$$ E43

where Var denotes variance. This implies that PSNR in (41) becomes

$$Q=\{\begin{array}{l}6.02R+D_0(qc)\\ C(qc)\end{array}$$ E44

where R denotes the bit rate and D₀ is related to the coding gain [16].

It means that finite word length noise c is negligible at lower bit rates comparing to the quantization noise q in respect of L₂ norm. However, variance of c dominates over that of q at high bit rates. Therefore the quality deterioration problem can be avoided by increasing the word length F_s. We utilize the fact that C (compatibility) is a monotonically increasing function of F_s. Their relation is approximately described as

$$C=\left[\begin{array}{cc}{p}_0& p_1\end{array}\right]\left[\begin{array}{cc}1& F_s\end{array}\right]{}^T$$ E45

with parameters p₀ and p₁. We compensate F_s at the minimum cost of word length by ΔF_s so that

$$\left[\begin{array}{cc}{p}_0& p_1\end{array}\right]\left[\begin{array}{cc}1& F_s+\Delta F_s\end{array}\right]{}^T\ge \left[\begin{array}{cc}p{\text{'}}_0& p{\text{'}}_1\end{array}\right]\left[\begin{array}{cc}1& F_s\end{array}\right]{}^T$$ E46

is satisfied where {p₀, p₁} are parameters of the corresponding NS DWT, and {p'₀, p'₁} are those of the separable DWT. As a result, the minimum word length for compensation is clarified as

$$\begin{array}{l}\Delta F_s\ge a+b{F}_s\cong a,\\ a=(p{\text{'}}_0-p_0)/p_1,b=p{\text{'}}_1/p_1-1.\end{array}$$ E47

Figure 22.

Compatibility versus word length.

	5/3 Sep	5/3 Ns1	9/7 Sep	9/7 Ns1	9/7 Ns2
	48.78	47.23	40.13	39.11	35.31
	6.27	6.24	6.01	6.01	5.99

Table 5.

Parameters in the rate distortion curves.

Fig.22 indicates experimentally measured relations between the compatibility C and the word length F_s. Table 5 summarizes the parameters p₀ and p₁ calculated from this figure. Table 6 summarizes two parameters a and b in (47) which were calculated from p₀ and p₁. It indicates that F_s of '9/7 Ns1' and '9/7 Ns2' should be compensated by more than 0.17 and 0.81 [bit], respectively so that these NS DWTs have the compatibility greater than that of '9/7 Sep'. Similarly, it also indicates that '5/3 Ns1' should be compensated by more than 0.25 [bit]. As a result, the minimum word length for compensation is found to be 1 bit at maximum as summarized in table 7.

Fig.23 illustrates rate distortion curves for the compensated NS DWTs. It is confirmed that the deterioration problem observed in Fig.21 is recovered to the same level of the standard separable DWTs of JPEG 2000. It means that the finite word length problem peculiar to the non-separable 2D DWTs can be perfectly compensated by adding only 1 bit word length, in case of implementation with very short word length, i.e. F_s=2 [bit].

	5/3 Sep	5/3 Ns1	9/7 Sep	9/7 Ns1	9/7 Ns2
a	0	0.248	0	0.170	0.805
b	0	0.0048	0	0.0000	0.0033

Table 6.

Parameters for word length compensation.

	5/3 Sep	5/3 Ns1	9/7 Sep	9/7 Ns1	9/7 Ns2
ΔFS	0.000	0.248	0.000	0.170	0.805
$$\lceil \Delta F_s\rceil$$	0	1	0	1	1

Table 7.

The minimum word length for compensation.

Figure 23.

Rare distortion curves in lossy coding mode with F_s=2+ΔF_s

5. Conclusions

In this chapter, 'separable' 2D DWTs defined by JPEG 2000 and its latency due to the total number of lifting steps were discussed. To reduce the latency, a 'non-separable' 2D DWTs were introduced for both of 5/3 DWT and 9/7 DWT. It was confirmed that the total number of lifting steps is reduced by the 'non-separable' DWT maintaining good compatibility with the 'separable' DWT. Performance of these DWTs were evaluated in lossless coding mode, and no significant difference was observed. A problem in finite word length implementation in lossy coding mode was discussed. It was found that only one bit compensation guarantees good compatibility with the 'separable' DWTs.

In the future, execution time of the DWTs on hardware or software platform should be investigated.