## 1. Introduction

Video-coding system has generally utilized block-based transform-coding skills to shrink the data rates by joining quantization and entropy coding. Among some block-based transforms, the discrete cosine transform (DCT) [1] and integer transforms have extensively been used to still image and video-coding specifications, such as JPEG [2], MPEG-1/2 [3, 4], MPEG-4 [5], H.264/AVC [6, 7], AVS [8, 9], VC-1 [10], VP8 [11], and HEVC [12]. Because integer transforms perform the low complexity and effective coding performance, the advanced video coding (AVC) in ITU-T H.264 [6, 7, 13, 14], which is also known as MPEG-4 part 10, applies integer transforms for transform process. The 4 × 4 and 8 × 8 transforms in [13, 14] were calculated exactly to prevent non-adaptation issues of inverse transforms for high-quality moving visual images. The VC-1 specification [10, 15, 16] employed 4 × 4 and 8 × 8 integer transforms, and it was developed by Microsoft Corporation and standardized by the Society of Motion Picture and Television Engineers (SMPTE). The 8 × 8 integer transform is utilized to obtain the high-coding performance in the Audio Video Coding Standard (AVS) for China [8, 9]. In [11], the VP8 video-coding standard was developed for Internet browser applications. The Joint Collaborative Team on Video Coding proposed the high-efficiency video coding (HEVC) specification [12]. By HEVC, the compression efficiency was greatly better than that achieved using the H.264/AVC high-profile-coding specification.

To support the single-standard H.264/AVC video coding, several transform architectures in [17–24] have been developed to approach the multiple transform modes in H.264. To support the single-standard H.265/HEVC video coding, several transform architectures in [25–32] have been developed to approach the multiple transform modes in HEVC. Besides, supporting multiple-standard functions in video coding has been an important issue in multimedia applications recently, such as H.264/AVC, MPEG-1/2/4, VC-1, AVS, and VP8 standards, and several transform architectures in [33–41] have also been developed to complete the multiple transform functions. Owing to the growth of multistandard video-coding applications, how to achieve low-computational complexities and implement by hardware-sharing-based cost-effective architectures simultaneously are interesting research topics for the VLSI design of video codecs.

## 2. Matrix decomposition preprocessing for fast algorithm and hardware-sharing-based designs

Based on the resemblance property, the 8 × 8 inverse integer transforms [41] in H.264/AVC, AVS, VC-1, VP8, MPEG-1/2/4, and HEVC specifications are revealed in Eq. (1), and **Table 1** depicts the coefficient values in the transforms.

(1) |

Transform sizes | VC-1 | AVS | VP8 | MPEG-1/2/4 | H.264/AVC | HEVC |
---|---|---|---|---|---|---|

4 × 4 | √ | √ | √ | N/A | √ | √ |

8 × 8 | √ | √ | N/A | √ | √ | √ |

16 × 16 | N/A | N/A | N/A | N/A | N/A | √ |

32 × 32 | N/A | N/A | N/A | N/A | N/A | √ |

In Eq. (1), it is decomposed by Eq. (2) as

In Eq. (2), *A*_{0} is divided into two modules, *U*_{4 × 4} and *D*_{4 × 4}, where

Thus

and *C*_{8×8} becomes

In (3), “⊕ “ is the direct sum operator, and the two diagonal blocks *U*_{4 × 4} and *D*_{4 × 4} are processing in parallel. To cut down the computational operations and achieve effective hardware shares, the upper diagonal matrix *U*_{4 × 4} and the down diagonal matrix *D*_{4 × 4} are further decomposed into the cascaded multiplication form or the addition form of sparse matrices. After matrix factorizations, the chosen sparse matrices have the coefficients which are 1, −1, 0, or an integer, and an integer value can equal the combination of powers of two. Besides, zero factors in the chosen sparse matrices could be factorized as many as possible [42].

By Eq. (1), for VC-1 the values of the coefficient set {*a, b, c, d, e, f, g*} are {12, 16, 15, 9, 4, 16, 6}, and those for AVS are {8, 10, 9, 6, 2, 10, 4}. Next, those for MPEG-1/2/4 are {362, 502, 426, 284, 100, 473, 196}, and those for H.264/AVC are {8, 12, 10, 6, 3, 8, 4}. Finally, those for HEVC are {64, 89, 75, 50, 18, 83, 36}.

The general 4 × 4 inverse integer transform matrices [41] can be presented in Eq. (5) as

By Eq. (5), for VC-1 the values of the coefficient set {h, i, j} are {17, 22, 10}, and those for VP8 are {128, 167, 70}. Next, those for AVS-M are {2, 3, 1}, and those for H.264/AVC are {1, 1, 0.5}. Finally, those for HEVC are {64, 83, 36}.

## 3. Case study [32]: single-standard multiple-mode transform design

### 3.1. Hardware-sharing based 32 × 32 integer core transform for HEVC

The one-dimensional (1D) 32 × 32 inverse core transform for HEVC is described in [30]. By the symmetrical property, the 32 × 32 inverse core transform is presented as

where *I*_{16×16} is a 16 × 16 identity matrix. In Eq. (6), *P _{A}* is the butterfly-like postprocessing, and

*C*

_{A1}is the sparse matrix. By swapping each column of

*C*

_{A1}, it becomes

By Eqs. (6) and (7), *H*_{i32} becomes

where *P*_{Ar} is the permutation matrix. In Eq. (7), *C*_{A2} is expressed by

where “⊕” means the direct sum operation, and then *T*_{A11} and *T*_{A22} are 16 × 16 matrices, which are revealed in [32]. The matrix *P*_{Ar} in Eq. (8) is expressed as

where the permutation matrix *P*(*m*, *n*) is defined in [43], and the notation “⊗” means the Kronecker product. In Eq. (9), *A*_{A22} is presented as

First, the lower half of *C*_{N1} is divided into sixteen 8 × 1 column vectors *X _{i}*, where

*i*= 0, 1, 2, …, 15, and then

*T*

_{N1}becomes

Second, the coefficients in a single column vector can be shared. The vector coefficient computations are achieved by integrating several base coefficients [32]. After realizing the column vectors of *T*_{N1}, the lower half of *T*_{N1} is factorized as an integration of eight 1 × 16 row vectors depicted as *Y _{i}*, where

*i*= 8, 9, …, and 15, and

*T*

_{N1}becomes

Adder tree structures are utilized to calculate the aggregate results for the row vectors *Y*_{8}–*Y*_{15} [32]. By the duplicate operations for *T*_{N1}, *T*_{M1} is presented as

where *i* = 0, 1, 2, …, and 15. Then, *T*_{M1} becomes

where *Y _{i}* is a 16 × 1 row vector, where

*i*= 0, 1, …, and 7. The realization of

*T*

_{M1}equals that of

*T*

_{N1}. Finally, the operations of

*T*

_{M1}and

*T*

_{N1}are merged to

*T*

_{A22}. The computational operations

*T*

_{A22}require 630 additions and 326 shift operations [32]. The matrix

*T*

_{A11}in Eq. (9), which is also denoted as

*H*

_{i16}, is the 1D 16 × 16 inverse core transform in HEVC [30].

### 3.2. Hardware-sharing-based 16 × 16 integer core transform for HEVC

The 16 × 16 integer core transform in [30] changes into

where *C*_{B1} is revealed in [32]. By swapping each column of *C*_{B1}, it will be

where *P*_{Br} = *P*(8,2). By Eqs. (16) and (17), *H*_{i16} is expressed by

In Eq. (18), *C*_{B2} is presented as

and *T*_{B22} becomes

where

By the duplicate processed of *T*_{N1} in Section 3.1, *T*_{N2} turns into

where *U _{i}* is an 8 × 1 column vector, where

*i*= 0, 1, 2, …, and 7. Next,

*T*

_{N2}also is

where *V*_{i} is a 1 × 8 row vector, where *i* = 4, 5, 6, and 7. Adder tree schemes are applied to compute the summed outcomes of *V*_{4}–*V*_{7} [32]. By the same processes of *T*_{M1} in Section 3.1, *T*_{M2} becomes

where *i* = 0, 1, 2, …, and 7. Next, *T*_{M2} also is

where *V*_{i} is a 1 × 8 row vector, where *i* = 0, 1, 2, and 3. Then, adder trees are used to treat the row vectors *V*_{0}–*V*_{3} [32]. Finally, the calculations of *T*_{M2} and *T*_{N2} are merged to *T*_{B22}. The computational operations of *T*_{B22} are 164 additions and 106 shift operations [32]. Meantime, the *T*_{B11} in Eq. (19), which is also denoted as *H*_{i8}, is the 1D 8 × 8 inverse core transform in HEVC [30].

### 3.3. Hardware-sharing-based 8 × 8 integer core transform for HEVC

The 8 × 8 integer transform in [30] is described as

where *C*_{C1}, it changes into

where *H*_{i8} is presented by

In Eq. (27), *C*_{C2} becomes

where

In Eq. (28), *T*_{C22} is factorized as

where *S*_{1} is expressed by

where *S*_{2} is presented as

where *T*_{C22} becomes

In Eq. (32), the computations of *T*_{C22} require 36 additions and 28 shift operations [32]. The matrix *T*_{C11} in Eq. (28) is also the 1D 4 × 4 inverse core transform matrix in HEVC.

### 3.4. Hardware-sharing-based 4 × 4 integer core transform for HEVC

The 4 × 4 integer core transform matrix is indicated as

where *C*_{D1}, it changes into

where *H*_{i4} is described by

In Eq. (34), *C*_{D2} is rewritten as

In Eq. (36), *T*_{D11} becomes

where *T*_{D22} is indicated by *Z*_{5} and *Z*_{6} as

where *T*_{D22} are 10 additions and 10 shift operations [32]. Based on Eqs. (35)– (38), *H*_{i4} is changed into

By the abovementioned discussions, the hardware modules of 4 × 4, 8 × 8, and 16 × 16 inverse core transforms are shared to implement *H*_{i8}, *H*_{i16}, and *H*_{i32}, respectively [32]. By sharing the hardware of *H*_{i4} in Eq. (39), the cost-effective design of the 8 × 8, 16 × 16, and 32 × 32 inverse core transforms is obtained progressively. First, the hardware-sharing-based eight-point inverse transform is presented as

Next, the hardware-sharing-based 16-point inverse transform is described as

Finally, the hardware-sharing-based 32-point inverse transform is depicted as

In this section, the hardware-sharing transform architecture cuts down the hardware cost because the same submodules and coefficients of the transforms are extracted to be shared. **Figure 1** illustrates the architecture of the hardware-sharing-based inverse core transform design for 4 × 4/8 × 8/16 × 16/32 × 32 transforms [32].

### 3.5. Architecture comparison

The proposed 1D inverse core transform in [32] involves four inputs to sustain 4 × 4, 8 × 8, 16 × 16, and 32 × 32 transform modes. Several multiplexers are utilized to acquire the transform outputs of the 32 × 32 inverse core transform by the shared design of 4 × 4, 8 × 8, and 16 × 16 inverse core transforms [32]. **Table 2** lists the number of adders and shifters needed to calculate four modes of the 1D inverse core transform for HEVC. The developed architecture in [32] does not require any multiplier, and the fixed-coefficient multiplications are replaced with simple additions and shift operations. **Table 3** shows the comparison of three 16-point inverse transform designs. Compared with the previous works in [29] and [31], the applied architecture contains fewer adders. However, several more shifters are required. Compared with the cost of adders, the shifters need lower hardware expense. Thus, the used architecture decreases the hardware cost more efficiently than previous transform schemes do.

## 4. Case study [41]: multiple-standard multiple-mode transform design

### 4.1. Hardware-sharing design for 8 × 8 transforms mode

For H.264/AVC, the transform matrix is employed as a foundation matrix for the multistandard hardware-sharing scheme. Based on Eq. (3), the cost of the upper diagonal matrix in Eq. (43) is eight adders and two shifters.

where *U*_{4×4_AVS} in Eq. (44) costs 10 adders and four shifters.

where *U*_{4×4_VC1} for VC1 needs 14 adders and eight shifters.

where and

where

where *U*_{2} is implemented by (*C*_{5} · *C*_{5} ≪ 4) – (*C*_{1} ≪ 1), where “≪1” is left shifting one bit, and the cost in Eq. (47) requires 28 adders and 26 shifters.

By Eq. (3), on the other side, the down diagonal matrix *D*_{4×4_AVC} for H.264/AVC becomes Eq. (48), and it needs 17 adders and eight shifters.

where

For AVS, the *D*_{4×4_AVS} matrix becomes (49), and *D*_{4} and *D*_{5} are shared with the design in Eq. (48), and then *U*_{3} and *U*_{4} are also partially shared with the scheme in Eq. (48). In Eq. (49), it costs 24 adders and 12 shifters

where

For VC-1, the *D*_{4×4_VC1} matrix is factorized by Eq. (50), and the design requires 21 adders and 12 shifters

where *D*_{4×4_HEVC} matrix is expressed by Eq. (51), and it expends 44 adders and 20 shifters

where *D*_{4×4_AVS}, the *D*_{4×4_MPEG} matrix is presented by Eq. (52), and the design costs 48 adders and 32 shifters

### 4.2. Hardware-sharing design for 4 × 4 transforms mode

For AVS-M, the matrix *M*_{4×4_AVS} is presented by (53), and it spends 10 adders and six shifters

where *M*_{4×4_VC1} is expressed by Eq. (54), and the design requires 14 adders and 12 shifters

where

where *U*_{4×4_AVC}/8 equals the 4 × 4 inverse transform matrix in H.264/AVC. In addition, the matrix *U*_{4×4_HEVC} equals the 4 × 4 inverse transform matrix in HEVC. Thus, several multiplexers are used to share the hardware between the submatrices to decrease hardware cost.

### 4.3. Architecture comparison

The applied hardware-sharing-based 1D multistandard inverse integer transform scheme has two inputs, which sustain 4 × 4 and 8 × 8 transform modes. The hardware blocks of processing the 4 × 4 inverse transforms are shared with that of the upper diagonal matrix *U*_{8×8}. Thus, several multiplexers are utilized for *U*_{8×8} to compute the 4 × 4 inverse transforms without additional operations. For the multistandard applications, the hardware-sharing architecture of the fast 1D 4 × 4 and 8 × 8 inverse integer transforms is illustrated in [41]. The shifters are also realized by wiring. Compared with the individual designs without hardware shares, **Table 4** depicts that the used scheme in [41] decreases the number of shifters and adders by 50 and 75%, respectively.

Different 1D inverse integer transform modes | No. of adders | No. of shifters |
---|---|---|

Individual designs without hardware shares | 336 | 180 |

Hardware-sharing-based design in Section 4 | 82 | 90 |

Reduction of cost | 75% | 50% |

To implement the discussed architecture, a cell-based VLSI design flow is utilized to design, simulate, and verify the cost-effective hardware-sharing architecture. For fair comparisons among different transform structures, the normalized mode gain, which is required to normalize the gate counts, is described as follows: By matrix dimensions and without missing generality [40], the normalized mode gains defined for the 32 × 32, 16 × 16, 8 × 8, and 4 × 4 inverse integer transform matrices are 16, 4, 1, and 1/4, respectively.

The hardware-sharing-based design in Section 3 supports 4 × 4, 8 × 8, 16 × 16, and 32 × 32 inverse transform modes for HEVC. Thus, the normalized mode gain of the design is 21.25 (i.e., 16 + 4 + 1 + 0.25). Similarly, five 8 × 8 and five 4 × 4 inverse transform functions are provided by the hardware-shared design in Section 4. Therefore, the normalized mode gain is assigned by 6.25 (i.e., 5 + 1.25) [41]. Afterwards, the normalized gate counts are defined by [40, 41]

**Table 5** shows the hardware cost comparisons among different 1D multiple transform architectures, which includes single-standard multiple-mode [32] and multiple-standard multiple-mode [41] transform designs.

Architecture | Ahmed et al. [29] | Hardware-sharing based-design in Section 3 | Shen et. al. [26] | Martuza et. al. [28] | Qi et al. [36] | Wang et al. [38] | Hardware-sharing-based design in Section 4 |
---|---|---|---|---|---|---|---|

Gate counts | 144.8K | 115.7 K | 134.8 K | 39.4 K | 18 K | 23.06 K | 27.4 K |

Normalized mode gain | 21.25 | 21.25 | 25.75 | 5 | 3.5 | 4.5 | 6.25 |

Normalized gate counts | 6.81 K | 5.44 K | 5.23 K | 7.88 K | 5.14 K | 5.12 K | 4.38 K |

Supporting modes | Single-standard Multiple-mode | Single-standard Multiple-mode | Multiple- standard Multiple- mode | Multiple-standard Multiple-mode | Multiple-standard Multiple- mode | Multiple- standard Multiple- mode | Multiple-standard Multiple-mode |

Supporting standards/Transforms | HEVC:4 × 4, 8 × 8, 16 × 16, 32 × 32 modes | HEVC:4 × 4, 8 × 8, 16 × 16, 32 × 32 modes | H.264/AVC,VC-1:4 × 4,8 × 8 modes MPEG-1/2/4, AVS: 8 × 8 mode; HEVC: 4 × 4,8 × 8, 16 × 16, 32 × 32 modes | H.264/AVC, VC-1,AVS, HEVC:4 × 4, 8 × 8 modes | H.264/AVC,VC-1:4 × 4, 8 × 8 modes; MPEG-1/2/4:8 × 8 mode | H.264/AVC;, VC-1:4 × 4, 8 × 8 modes; MPEG-: 1/2/4, AVS 8 × 8 mode | H.264/AVC,VC-1, HEVC:4 × 4, 8 × 8 modes; MPEG-1/2/4, AVS: 8 × 8 mode;VP8, AVS-M: 4 × 4 mode |

## 5. Conclusion

For the single-standard multiple-mode transform design, this chapter discussed the 4 × 4, 8 × 8, 16 × 16, and 32 × 32 inverse core transforms in HEVC with a cost-effective and hardware-efficient design. By the symmetrical characteristics of the elements, the core transform matrices were factorized into several submatrices. Thus, the hardware of the (*N*/2) × (*N*/2) inverse core transform was shared with that of the *N* × *N* inverse core transform for *N* = 32, 16, and 8. Compared with the direct design without hardware shares, the applied transform scheme in Section 3 decreased the hardware cost of adders and shifters by 32 and 36%, respectively. Besides, for VLSI implementation, the design in Section 3 requires less normalized gate counts than the design does in [29].

For the multiple-standard multiple-mode transform design, this chapter also discussed the fast algorithm and hardware-sharing-based design of 4 × 4 and/or 8 × 8 inverse transforms among H.264/AVC, VC-1, HEVC, MPEG-1/2/4, AVS, and VP8 for multistandard video decoders. By only shifters and adders, the decomposition scheme of matrices was used to develop the hardware-shared scheme. The used structure in Section 4 decreased the number of shifters and adders by 50 and 75% more than the individual fast algorithm-based implementation did. Besides, for VLSI implementation, the design in Section 4 requires less normalized gate counts than the designs do in [26, 28, 36, 38].