A Stereo Acoustic Echo Canceller Using Cross-Channel Correlation

1. Introduction

Stereo acoustic echo canceller is becoming more and more important as an echo canceller is applied to consumer products like a conversational DTV. However it is well known that if there is strong cross-channel correlation between right and left sounds, it cannot converge well and results in echo path estimation misalignment. This is a serious problem in a conversational DTV because the speaker output sound is combination of a far-end conversational sound, which is essentially monaural, and TV program sound, which has wide variety of sound characteristics, monaural sound, stereo sound or mixture of them. To cope with this problem, many stereo echo cancellation algorithms have been proposed. The methods can be categorized into two approaches. The first one is to de-correlate the stereo sound by introducing independent noise or non-linear post-processing to right and left speaker outputs. This approach is very effective for single source stereo sound case, which covers most of conversational sounds, because the de-correlation prevents rank drop to solve a normal equation in a multi-channel adaptive filtering algorithm. Moreover, it is simple since many traditional adaptation algorithms can be used without any modification. Although the approach has many advantages and therefore widely accepted, it still has an essential problem caused by the de-correlation which results in sound quality change due to insertion of the artificial distortion. Even though the inserted distortion is minimized so as to prevent sound quality degradation, from an entertainment audio equipment view point, such as a conversational DTV, many users do not accept any distortion to the speaker output sound. The second approach is desirable for the entertainment types of equipments because no modification to the speaker outputs is required. In this approach, the algorithms utilize cross-channel correlation change in a stereo sound. This approach is also divided into two approaches, depending on how to utilize the cross-channel correlation change. One widely used approach is affine projection method. If there are small variations in the cross-channel correlation even in a single sound source stereo sound, small un-correlated component appears in each channel. The affine projection method can produce the best direction by excluding the auto-correlation bad effect in each channel and by utilizing the small un-correlated components. This approach has a great advantage since it does not require any modification to the stereo sound, however if the variation in the cross-channel correlation is very small, improvement of the adaptive filter convergence is very small. Since the rank drop problem of the stereo adaptive filter is essentially not solved, we may need slight inserted distortion which reduces merit of this method. Another headache is that the method requires P by P inverse matrix calculation in an each sample. The inverse matrix operation can be relaxed by choosing P as small number, however small P sometimes cannot attain sufficient convergence speed improvement. To attain better performance even by small P, the affine projection method sometimes realized together with sub-band method. Another method categorized in the second approach is “WARP” method. Unlike to affine projection method which utilizes small change in the cross-channel correlation, the method utilizes large change in the cross-channel correlation. This approach is based on the nature of usual conversations. Even though using stereo sound for conversations, most parts of conversations are single talk monaural sound. The cross-channel correlation is usually very high and it remains almost stable during a single talking. A large change happens when talker change or talker’s face movement happens. Therefore, the method applies a monaural adaptive filter to single sound source stereo sound and multi-channel (stereo) adaptive filter to non-single sound source stereo sound. Important feature of the method is two monaural adaptive filter estimation results and one stereo adaptive filter estimation result is transformed to each other by using projection matrixes, called WARP matrixes. Since a monaural adaptive filter is applied when a sound is single source stereo sound, we do not need to suffer from the rank problem.

In this chapter, stereo acoustic echo canceller methods, multi-channel least mean square, affine projection and WARP methods, all of them do not need any modification to the speaker output sounds, are surveyed targeting conversational DTV applications. Then WARP method is explained in detail.

2. Stereo acoustic echo canceller problem

2.1. Conversational DTV

Since conversational DTV should keep smooth speech communication even when it is receiving a regular TV program, it requires following functionalities together with traditional DTV systems as shown in Fig. 1.

Figure 1.

Audio System Example in a Conversational DTV

1. Mixing of broadcasting sound and communication speech: Two stereo sounds from the DTV audio receiver and local conversational speech decoder are mixed and sent to the stereo speaker system.

2. Sampling frequency conversion: Sampling frequency of DTV sound is usually wider than that of conversational service, such as $$f_{SH}=48kHz$$ for DTV sound and $$f_S=16kHz$$ for conversational service sound, we need sampling frequency conversion between DTV and conversational service audio parts

3. Stereo acoustic canceller: A stereo acoustic echo canceller is required to prevent howling and speech quality degradation due to acoustic coupling between stereo speaker and microphone.

Among the above special functionalities, the echo canceller for the conversational DTV is technically very challenging because the echo canceller should cancel wide variety of stereo echoes for TV programs as well as stereo speech communications.

2.2. Stereo sound generation model

A stereo acoustic echo canceller system is shown in Fig. 2 with typical stereo sound generation model, where all signals are assumed to be discrete time signals at the $$kth$$ sampling timing by $$f_S$$ sampling frequency and the sound generation model is assumed to be linier finite impulse response (FIR) systems which has a sound source signal $$x_{Si}(k)$$ as an input and stereo sound $$x_{Ri}(k)$$ and $$x_{Li}(k)$$ as outputs with additional uncorrelated noises $${x^{\prime }}_{URi}(k)$$ and $${x^{\prime }}_{ULi}(k)$$ . By using matrix and array notations of the signals as

$$\begin{array}{l}{X}_{Si}(k)=[x_{Si}(k),x_{Si}(k-1),\cdots x_{Si}(k-P+1)]\\ x_{Si}(k)={[}^{x_{Si}}\\ x_{Ri}(k)={[}^{x_{Ri}}\\ x_{Li}(k)={[}^{x_{Li}}\\ {x^{\prime }}_{URi}(k)={[}^{{x^{\prime }}_{URi}}\\ {x^{\prime }}_{ULi}(k)={[}^{{x^{\prime }}_{ULi}}\end{array}$$ E1

where $$P$$ and $$N$$ are impulse response length of the FIR system and tap length of the adaptive filter for each channel, respectively.

Then the FIR system output $$x_i^{}(k)$$ is 2N sample array and is expressed as

$$x_i^{}(k)=\left[\begin{array}{c}{x}_{Ri}^{}(k)\\ x_{Li}^{}(k)\end{array}\right]=\left[\begin{array}{c}{X}_{Si}^{}(k)g_{Ri}^{}(k)+{x^{\prime }}_{URi}^{}(k)\\ X_{Si}^{}(k)g_{Li}^{}(k)+{x^{\prime }}_{ULi}^{}(k)\end{array}\right]$$ E2

where $$g_{Ri}(k)$$ and $$g_{Li}(k)$$ are $$P$$ sample impulse responses of the FIR system defined as

$$\begin{array}{l}{g}_{Ri}(k)={[}^{g_{Ri,0}}\\ g_{Li}(k)={[}^{g_{Li,0}}\end{array}$$ E3

Figure 2.

Traditional Stereo Acoustic Echo Canceller System Configuration with Typical Stereo Sound Generation Model.

In(2), if $$g_{Ri}(k)$$ and $$g_{Li}(k)$$ are composed of constant array, $$g_{Ri}$$ and $$g_{Li}$$ during the $$ith$$ period, and small time variant arrays, $$\Delta g_{Ri}(k)$$ and $$\Delta g_{Li}(k)$$ which are defined as

$$\begin{array}{l}{g}_{Ri}={[}^{g_{Ri,0}}\\ g_{Li}={[}^{g_{Li,0}}\\ \Delta g_{Ri}(k)={[}^{\Delta }\\ \Delta g_{Li}(k)={[}^{\Delta }\end{array}$$ E4

(2) is re-written as

$$\left[\begin{array}{c}{x}_{Ri}^{}(k)\\ x_{Li}^{}(k)\end{array}\right]=\left[\begin{array}{c}{X}_{Si}^{}(k)g_{Ri}^{}+X_{Si}^{}(k)\Delta g_{Ri}^{}(k)+{x^{\prime }}_{URi}^{}(k)\\ X_{Si}^{}(k)g_{Li}^{}+X_{Si}^{}(k)\Delta g_{Li}^{}(k)+{x^{\prime }}_{ULi}^{}(k)\end{array}\right]$$ E5

This situation is usual in the case of far-end single talking because transfer functions between talker and right and left microphones vary slightly due to talker’s small movement. By assuming the components are also un-correlated noise, (5) can be regarded as a linear time invariant system with independent noise components, $$x_{URi}(k)$$ and $$x_{ULi}(k)$$ , as

$$\left[\begin{array}{c}{x}_{Ri}^{}(k)\\ x_{Li}^{}(k)\end{array}\right]=\left[\begin{array}{c}{X}_{Si}^{}(k)g_{Ri}^{}+x_{URi}^{}(k)\\ X_{Si}^{}(k)g_{Li}^{}+x_{ULi}^{}(k)\end{array}\right]$$ E6

where

$$\begin{array}{l}{x}_{URi}^{}(k)=X_{Si}^{}(k)\Delta g_{Ri}^{}(k)+{x^{\prime }}_{URi}^{}(k)\\ x_{ULi}^{}(k)=X_{Si}^{}(k)\Delta g_{Li}^{}(k)+{x^{\prime }}_{ULi}^{}(k)\end{array}$$ E7

In (6), if there are no un-correlated noises, we call the situation as strict single talking.

In this chapter, sound source signal( $$x_{Si}(k)$$ ), uncorrelated noises ( $${x^{\prime }}_{URi}(k)$$ and $${x^{\prime }}_{ULi}(k)$$ ) are assumed as independent white Gaussian noise with variance $${\sigma }_{xi}^{}$$ and $${\sigma }_{Ni}^{}$$ , respectively.

2.3. Stereo acoustic echo canceller problem

For simplification, only one stereo audio echo canceller for the right side microphone’s output signal $${\tilde{y}}_i(k)$$ , is explained. This is because the echo canceller for left microphone output is apparently treated as the same way as the right microphone case. As shown in Fig.2, the echo canceller cancels the acoustic echo $$y_i(k)$$ as

$$e_i(k)=y_i(k)-{\widehat{y}}_i(k)+n_i(k)$$ E8

where $$e_i(k)$$ is acoustic echo canceller’s residual error, $$n_i(k)$$ is a independent background noise, $${\widehat{y}}_i(k)$$ is an FIR adaptive filter output in the stereo echo canceller, which is given by

$${\widehat{y}}_i(k)={\widehat{h}}_{Ri}^T(k)x_{Ri}(k)+{\widehat{h}}_{Li}^T(k)x_{Li}(k)$$ E9

where $${\widehat{h}}_{Ri}(k)$$ and $${\widehat{h}}_{Li}(k)$$ are N tap FIR adaptive filter coefficient arrays.

Error power of the echo canceller for the right channel microphone output, $${\sigma }_{ei}^2(k)$$ , is given as:

$${\sigma }_{ei}^2(k)={(y_{Ri}(k)-{\widehat{h}}_{STi}^T(k)x_i^{}(k)+n_i(k))}_{}^2$$ E10

where $${\widehat{h}}_{STi}(k)$$ is a stereo echo path model defined as

$${\widehat{h}}_{STi}(k)={\left[\begin{array}{cc}{\widehat{h}}_{Ri}^T(k)& {\widehat{h}}_{Li}^T(k)\end{array}\right]}^T$$ E11

Optimum echo path estimation $${\widehat{h}}_{OPT}^{}$$ which minimizes the error power $${\sigma }_e^2(k)$$ is given by solving the linier programming problem as

$$Minimize\left[{\displaystyle \sum _{k=0}^{N_{LS}-1}}{\sigma }_{ei}^2(k)\right]$$ E12

where $N_{LS}$ is a number of samples used for optimization. Then the optimum echo path estimation for the $$ith$$ LTI period $${\widehat{h}}_{OPTi}^{}$$ is easily obtained by well known normal equation as

$${\widehat{h}}_{OPTi}^{}=({\displaystyle \sum _{k=0}^{N_{LS}-1}({\tilde{y}}_i(k)x_i^{}(k))})X_{NLSi}^{-1}$$ E13

where $$X_{NLSi}^{}$$ is an auto-correlation matrix of the adaptive filter input signal and is given by

$$X_{NLSi}^{}={\displaystyle \sum _{k=0}^{N_{LS}-1}(x_i^{}(k)x_i^T(k))}=\left[\begin{array}{cc}{A}_i& B_i\\ C_i& D_i\end{array}\right]=\left[\begin{array}{cc}{\displaystyle \sum _{k=0}^{N_{LS}-1}(x_{Ri}^{}(k)x_{Ri}^T(k))}& {\displaystyle \sum _{k=0}^{N_{LS}-1}(x_{Ri}^{}(k)x_{Li}^T(k))}\\ {\displaystyle \sum _{k=0}^{N_{LS}-1}(x_{Li}^{}(k)x_{Ri}^T(k))}& {\displaystyle \sum _{k=0}^{N_{LS}-1}(x_{Li}^{}(k)x_{Li}^T(k))}\end{array}\right]$$ E14

By (14), determinant of $$X_{NLSi}^{}$$ is given by

$$\left|X_{NLSi}^{}\right|=\left|A_i\right|\left|D_i-C_i{A}_i^{-1}{B}_i\right|$$ E15

In the case of the stereo generation model which is defined by(2), the sub-matrixes in (14) are given by

$$\begin{array}{l}{A}_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{RRi}{X}_{Si}^T(k)+2x_{URi}^{}(k){(X_{Si}^{}(k)g_{Ri}^{})}_{}^T+x_{URi}^{}(k)x_{URi}^T(k))}\\ B_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{RLi}{X}_{Si}^T(k)+x_{URi}^{}(k){(X_{Si}^{}(k)g_{Ri}^{})}_{}^T+x_{ULi}^{}(k){(X_{Si}^{}(k)g_{Ri}^{})}_{}^T+x_{URi}^{}(k)x_{ULi}^T(k))}\\ C_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{LRi}{X}_{Si}^T(k)+x_{ULi}^{}(k){(X_{Si}^{}(k)g_{Ri}^{})}_{}^T+x_{URi}^{}(k){(X_{Si}^{}(k)g_{Li}^{})}_{}^T+x_{ULi}^{}(k)x_{URi}^T(k))}\\ D_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{LLi}{X}_{Si}^T(k)+2x_{ULi}^{}(k){(X_{Si}^{}(k)g_{Li}^{})}_{}^T+x_{ULi}^{}(k)x_{ULi}^T(k))}\end{array}$$ E16

where

$$G_{RRi}=g_{Ri}^{}{g}_{Ri}^T,G_{RLi}=g_{Ri}^{}{g}_{Li}^T,G_{LRi}=g_{Li}^{}{g}_{Ri}^T,G_{LLi}=g_{Li}^{}{g}_{Li}^T$$ E17

In the cease of strict single talking where $$x_{URi}^{}(k)$$ and $$x_{ULi}^{}(k)$$ do not exist, (16) becomes very simple as

$$\begin{array}{l}{A}_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{RRi}{X}_{Si}^T(k))}\\ B_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{RLi}{X}_{Si}^T(k))}\\ C_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{LRi}{X}_{Si}^T(k))}\\ D_i={\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)G_{LLi}{X}_{Si}^T(k))}\end{array}$$ E18

To check the determinant $$\left|X_{NLSi}^{}\right|$$ , we calculate $$\left|X_{NLSi}^{}\right|\left|C_i\right|$$ considering $$B_i=C_i^T$$ as

$$\begin{array}{l}\left|X_{NLSi}^{}\right|\left|C_i\right|=\left|A_i^{}\right|\left|(D_i{C}_i-C_i{A}_i^{-1}{B}_i{C}_i\right|\\ =\left|A_i^{}\right|\left|(D_i{C}_i-C_i{B}_i{C}_i{A}_i^{-1}\right|\end{array}$$ E19

Then $$\left|D_i{C}_i-C_i{B}_i{C}_i{A}_i^{-1}\right|$$ becomes zero as

$$\begin{array}{l}\left|D_i{C}_i-C_i{A}_i^{-1}{B}_i{C}_i\right|\\ =\left|{\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)(G_{LLi}-G_{LRi}{G}^{-1}{}_{RRi}{G}_{RLi})X_{Si}^T(k))X_{Si}^{}(k)G_{LRi}{X}_{Si}^T(k))}\right|\\ =N{\sigma }_{xi}^2\left|{\displaystyle \sum _{k=0}^{N_{LS}-1}(X_{Si}^{}(k)(g_{Li}^T{g}_{Li}^{}(g_{Li}^{}{g}_{Ri}^T-g_{Li}^{}{g}_{Ri}^T{(g_{Ri}^{}{g}_{Ri}^T)}_{}^{-1}{g}_{Ri}^{}{g}_{Ri}^T)X_{Si}^T(k)})\right|\\ =0\end{array}$$ E20

Hence no unique solution can be found by solving the normal equation in the case of strict single talking where un-correlated components do not exist. This is a well known stereo adaptive filter cross-channel correlation problem.

3. Stereo acoustic echo canceller methods

To improve problems addressed above, many approaches have been proposed. One widely accepted approach is de-correlation of stereo sound. To avoid the rank drop of the normal equation(13), small distortion such as non-linear processing or modification of phase is added to stereo sound. This approach is simple and effective to endorse convergence of the multi-channel adaptive filter, however it may degrade the stereo sound by the distortion. In the case of entertainment applications, such as conversational DTV, the problem may be serious because customer’s requirement for sound quality is usually very high and therefore even small modification to the speaker output sound cannot be accepted. From this view point, approaches which do not need to add any modification or artifacts to the speaker output sound are desirable for the entertainment use. In this section, least square (LS), stereo affine projection (AP), stereo normalized least mean square (NLMS) and WARP methods are reviewed as methods which do not need to change stereo sound itself.

3.1. Gradient method

Gradient method is widely used for solving the quadratic problem iteratively. As a generalized gradient method, let denote $$M$$ sample orthogonalized error array $${\epsilon }_{Mi}^{}(k)$$ based on original error array $$e_{Mi}^{}(k)$$ as

$${\epsilon }_{Mi}^{}(k)=R_i^{}(k)e_{Mi}^{}(k)$$ E21

where $$e_{Mi}^{}(k)$$ is an $$M$$ sample error array which is defined as

$$e_{Mi}^{}(k)={[e_i^{}(k),e_i^{}(k-1),\cdots e_i^{}(k-M+1)]}_{}^T$$ E22

and $$R_i^{}(k)$$ is a $$M\times M$$ matrix which orthogonalizes the auto-correlation matrix $$e_{Mi}^{}(k)e_{Mi}^T(k)$$ . The orthogonalized error array is expressed using difference between adaptive filter coefficient array $${\widehat{h}}_{STi}^{}(k)$$ and target stereo echo path $$2N$$ sample response $$h_{ST}^{}$$ as

$${\epsilon }_{Mi}^{}(k)=R_i^{}(k)X_{M2Ni}^T(k)(h_{ST}-{\widehat{h}}_{STi}^{}(k))$$ E23

where $$X_{M2Ni}^{}(k)$$ is a Mx2N matrix which is composed of adaptive filter stereo input array as defined by

$$X_{M2Ni}^{}\text{(}k\text{)}=[x_i^{}(k),x_i^{}(k-1),\cdots x_i^{}(k-M+1)]$$ E24

By defining an echo path estimation error array $$d_{STi}^{}(k)$$ which is defined as

$$d_{STi}^{}(k)=h_{ST}-{\widehat{h}}_{STi}^{}(k)$$ E25

estimation error power $${\sigma }_{\epsilon i}^2(k)$$ is obtained by

$${\sigma }_{\epsilon i}^2(k)={\epsilon }_{Mi}^T(k){\epsilon }_{Mi}^{}(k)=d_{STi}^T(k)Q_{2N2Ni}^{}(k)d_{STi}^{}{(k)}_{}^{}$$ E26

where

$$Q_{2N2Ni}^{}(k)=X_{M2Ni}^{}(k)R_i^T(k)R_i^{}(k)X_{M2Ni}^T(k)$$ E27

Then, (26) is regarded as a quadratic function of $${\widehat{h}}_{STi}^{}(k)$$ as

$$f({\widehat{h}}_{STi}^{}(k))=\frac{1}{2}{\widehat{h}}_{STi}^T(k)Q_{2N2Ni}^{}(k){\widehat{h}}_{STi}^T(k)-{\widehat{h}}_{STi}^T(k)Q_{2N2Ni}^{}{h}_{ST}$$ E28

For the quadratic function, gradient $${\Delta }_i(k)$$ is given by

$${\Delta }_i(k)=-Q_{2N2Ni}^{}(k)d_{STi}^{}(k)$$ E29

Iteration of $${\widehat{h}}_{STi}^{}(k)$$ which minimizes $${\sigma }_{\epsilon i}^2(k)$$ is given by

$$\begin{array}{l}{\widehat{h}}_{STi}(k+1)={\widehat{h}}_{STi}(k)-\alpha {\Delta }_i(k)\\ ={\widehat{h}}_{STi}(k)+\alpha Q_{2N2Ni}^{}(k)d_{STi}^{}(k)\\ ={\widehat{h}}_{STi}(k)+\alpha X_{M2Ni}^{}(k)R_i^T(k)R_i^{}(k)e_{Mi}^{}(k)\end{array}$$ E30

where $$\alpha$$ is a constant to determine step size.

Above equation is very generic expression of the gradient method and following approaches are regarded as deviations of this iteration.

3.2. Least Square (LS) method (M=2N)

From(30), the estimation error power between estimated adaptive filter coefficients and stereo echo path response, $$d_i^T(k)d_i^{}(k)$$ is given by

$$d_i^T(k+1)d_i^{}(k+1)=d_i^T(k)(I_{2N}^{}-\alpha Q_{2N2Ni}^{}(k)){(I_{2N}^{}-\alpha Q_{2N2Ni}^{}(k))}_{}^T{d}_i^{}(k)$$ E31

where $$I_{2N}^{}$$ is a $$2N\times 2N$$ identity matrix. Then the fastest convergence is obtained by finding $$R_i^{}(k)$$ which orthogonalizes and minimizes eigenvalue variance in $$Q_{2N2Ni}^{}(k)$$ .

If M=2N, $$X_{2M2Ni}^{}(k)$$ is symmetric square matrix as

$$X_{M2Ni}^{}(k)=X_{M2Ni}^T(k)$$ E32

and if $$X_{M2Ni}^{}(k)\cdot X_{M2Ni}^T(k)(=X_{M2Ni}^T(k)\cdot X_{M2Ni}^{}(k))$$ is a regular matrix so that inverse matrix exists, $$R_i^T(k)R_i^{}(k)$$ which orthogonalizes $$Q_{2N2Ni}^{}(k)$$ is given by

$$R_i^T(k)R_i^{}(k)={(X_{2N2Ni}^T(k)X_{2N2Ni}^{}(k))}_{}^{-1}$$ E33

By substituting (33) for (30)

$${\widehat{h}}_{STi}(k+1)={\widehat{h}}_{STi}(k)+\alpha X_{2N2Ni}^{}(k){(X_{2N2Ni}^T(k)X_{2N2Ni}^{}(k))}_{}^{-1}{e}_{Ni}^{}(k)$$ E34

Assuming initial tap coefficient array as zero vector and $$\alpha =0$$ during 0 to 2N-1th samples and $$\alpha =1$$ at 2Nth sample, (34) can be re-written as

$${\widehat{h}}_{STi}(2N)=X_{2N2Ni}^{}(2N-1){(X_{2N2Ni}^T(2N-1)X_{2N2Ni}^{}(2N-1))}_{}^{-1}{y}_i^{}(2N-1)$$ E35

where $$y_i^{}(k)$$ is 2 $$N$$ sample echo path output array and is defined as

$$y_i^{}(k)={[y_i^{}(k),y_i^{}(k-1),\cdots y_i^{}(k-2N+1)]}_{}^T$$ E36

This iteration is done only once at $$2N-1th$$ sample. If $$N_{LS}^{}=2N$$ , inverse matrix term in (35) is written as

$$X_{2N2Ni}^T(k)X_{2N2Ni}^{}(k)={\displaystyle \sum _{k=0}^{N_{LS}-1}(x_i^{}(k)x_i^T(k))}=X_{NLSi}^{}$$ E37

Comparing (13) and (35) with　 (37), it is found that LS method is a special case of gradient method when M equals to 2N.

3.3. Stereo Affine Projection (AP) method (M=P $\le$ N)

Stereo affine projection method is assumed as a case when M is chosen as FIR response length P in the LTI system. This approach is very effective to reduce 2Nx2N inverse matrix operations in LS method to PxP operations when the stereo generation model is assumed to be LTI system outputs from single WGN signal source with right and left channel independent noises as shown in Fig.2. For the sake of explanation, we define stereo sound signal matrix $$X_{P2Ni}^{}(k)$$ which is composed of right and left signal matrix $$X_{Ri}^{}(k)$$ and $$X_{Li}^{}(k)$$ for P samples as

$$X_{P2Ni}^{}(k)={\left[\begin{array}{cc}{X}_{Ri}^T(k)& X_{Li}^T(k)\end{array}\right]}_{}^T=\left[\begin{array}{c}{X}_{2Si}^{}(k)G_{Ri}^T+X_{URi}^{}(k)\\ X_{2Si}^{}(k)G_{Li}^T+X_{ULi}^{}(k)\end{array}\right]$$ E38

where

$$X_{2Si}(k)=[x_{Si}(k),x_{Si}(k-1),\cdots x_{Si}(k-2P+2)]$$ E39

$$X_{URi}(k)$$ and $$X_{ULi}(k)$$ are un-correlated signal matrix defined as

$$\begin{array}{l}{X}_{URi}(k)=[x_{URi}(k),x_{URi}(k-1),\cdots x_{URi}(k-P+1)]\\ X_{ULi}(k)=[x_{ULi}(k),x_{ULi}(k-1),\cdots x_{ULi}(k-P+1)]\end{array}$$ E40

$$G_{Ri}^{}$$ and $$G_{Li}^{}$$ are source to microphones response (2P-1)xP matrixes and are defined as

$$G_{Ri}^{}=\left[\begin{array}{c}{g}_{2R,0,i}^T\\ g_{2R,1,i}^T\\ \cdots \\ g_{2R,P-1,i}^T\end{array}\right]=\left[\begin{array}{cccc}{g}_{Ri}^T& 0& \cdots & 0\\ 0& g_{Ri}^T& \ddots & 0\\ 0& 0& \cdots & 0\\ 0& 0& \cdots & g_{Ri}^T\end{array}\right],G_{Li}^{}=\left[\begin{array}{c}{g}_{2RL,0,i}^T\\ g_{2L,1,i}^T\\ \cdots \\ g_{2L,P-1,i}^T\end{array}\right]=\left[\begin{array}{cccc}{g}_{Li}^T& 0& \cdots & 0\\ 0& g_{Li}^T& \ddots & 0\\ 0& 0& \cdots & 0\\ 0& 0& \cdots & g_{Li}^T\end{array}\right]$$ E41

As explained by(31), $$Q_{2N2Ni}^{}(k)$$ determines convergence speed of the gradient method. In this section, we derive affine projection method by minimizing the max-min eigenvalue variance in $$Q_{2N2Ni}^{}(k)$$ . Firstly, the auto-correlation matrix is expressed by sub-matrixes for each stereo channel as

$$Q_{N2Ni}^{}(k)=\left[\begin{array}{cc}{Q}_{ANNi}^{}(k)& Q_{BNNi}^{}(k)\\ Q_{CNNi}^{}(k)& Q_{DNNi}^{}(k)\end{array}\right]$$ E42

where $$Q_{ANNi}^{}(k)$$ and $$Q_{DNNi}^{}(k)$$ are right and left channel auto-correlation matrixes, $$Q_{BNNi}^{}(k)$$ and $$Q_{CNNi}^{}(k)$$ are cross channel-correlation matrixes. These sub-matrixes are given by

$$\begin{array}{l}{Q}_{ANNi}^{}(k)=X_{2Si}^{}(k)G_{Ri}^T{R}_i^T(k)R_i^{}(k)G_{Ri}^{}{X}_{2Si}^T(k)+X_{URi}^{}(k)R_i^T(k)R_i^{}(k)X_{URi}^T(k)\\ +2X_{2Si}^{}(k)R_i^T(k)R_i^{}(k)X_{URi}^T(k)\\ Q_{BNNi}^{}(k)=X_{2Si}^{}(k)G_{Ri}^T{R}_i^T(k)R_i^{}(k)G_{Li}^{}{X}_{2Si}^T(k)+X_{URi}^{}(k)R_i^T(k)R_i^{}(k)X_{ULi}^T(k)\\ +2X_{URi}^{}(k)R_i^T(k)R_i^{}(k)X_{ULi}^T(k)\\ Q_{CNNi}^{}(k)=X_{2Si}^{}(k)G_{Li}^T{R}_i^T(k)R_i^{}(k)G_{Ri}^{}{X}_{2Si}^T(k)+X_{ULi}^{}(k)R_i^T(k)R_i^{}(k)X_{URi}^T(k)\\ +2X_{ULi}^{}(k)R_i^T(k)R_i^{}(k)X_{UTi}^T(k)\\ Q_{DNNi}^{}(k)=X_{2Si}^{}(k)G_{Li}^T{R}_i^T(k)R_i^{}(k)G_{Li}^{}{X}_{2Si}^T(k)+X_{ULi}^{}(k)R_i^T(k)R_i^{}(k)X_{ULi}^T(k)\\ +2X_{2Si}^{}(k)R_i^T(k)R_i^{}(k)X_{ULi}^T(k)\end{array}$$ E43

Since the iteration process in (30) is an averaging process, the auto-correlation matrix $$Q_{2N2Ni}^{}(k)$$ is approximated by using expectation value of it, $${\tilde{Q}}_{2N2Ni}^{}(k)=⟨Q_{2N2Ni}^{}(k)⟩$$ . Then expectation values for sub-matrixes in (42) are simplified applying statistical independency between sound source signal and noises and $$Tlz$$ function defined in Appendix as

$$\begin{array}{l}{\tilde{Q}}_{ANNi}^{}=Tlz(⟨{\tilde{X}}_{2Si}^{}(k)G_{Ri}^T{R}_i^T(k)R_i^{}(k)G_{Ri}^{}{\tilde{X}}_{2Si}^T(k)⟩)+Tlz(⟨{\tilde{X}}_{URi}^{}(k)R_i^T(k)R_i^{}(k){\tilde{X}}_{URi}^{}(k)⟩)\\ {\tilde{Q}}_{BNNi}^{}=Tlz(⟨{\tilde{X}}_{2Si}^{}(k)G_{Ri}^T{R}_i^T(k)R_i^{}(k)G_{Li}^{}{\tilde{X}}_{2Si}^T(k)⟩)\\ {\tilde{Q}}_{CNNi}^{}=Tlz(⟨{\tilde{X}}_{2Si}^{}(k)G_{Li}^T{R}_i^T(k)R_i^{}(k)G_{Ri}^{}{\tilde{X}}_{2Si}^T(k)⟩)\\ {\tilde{Q}}_{DNNi}^{}=Tlz(⟨{\tilde{X}}_{2Si}^{}(k)G_{Li}^T{R}_i^T(k)R_i^{}(k)G_{Li}^{}{\tilde{X}}_{2Si}^{}(k)⟩)+Tlz(⟨{\tilde{X}}_{ULi}^{}(k)R_i^T(k)R_i^{}(k){\tilde{X}}_{ULi}^{}(k)⟩)\end{array}$$ E44

where

$$\begin{array}{l}{\tilde{X}}_{2Si}^{}(k)={[}_{{\tilde{x}}_{2Si}}^{(}\\ {\tilde{X}}_{URi}^{}(k)=[{\tilde{x}}_{URi}^{}(k),{\tilde{x}}_{URi}^{}(k-1),\cdots {\tilde{x}}_{URi}^{}(k-P+1)]\\ {\tilde{X}}_{ULi}^{}(k)=[{\tilde{x}}_{ULi}^{}(k),{\tilde{x}}_{ULi}^{}(k-1),\cdots {\tilde{x}}_{ULi}^{}(k-P+1)]\end{array}$$ E45

with

$$\begin{array}{l}{\tilde{x}}_{2Si}(k)={[}^{x_{Si}}\\ {\tilde{x}}_{URi}(k)={[}^{x_{URi}}\\ {\tilde{x}}_{ULi}(k)={[}^{x_{ULi}}\end{array}$$ E46

Applying matrix operations to $$Q_{2N2Ni}^{}$$ , a new matrix $${Q^{\prime }}_{2N2Ni}^{}$$ which has same determinant as $$Q_{2N2Ni}^{}$$ is given by

$${Q^{\prime }}_{2N2Ni}^{}(k)=\left[\begin{array}{cc}{Q^{\prime }}_{ANNi}^{}(k)& 0\\ 0& {Q^{\prime }}_{DNNi}^{}(k)\end{array}\right]$$ E47

where

$${Q^{\prime }}_{ANNi}^{}=Tlz({{Q^{\prime }}^{\prime }}_{ANNi}^{}),{Q^{\prime }}_{DNNi}^{}=Tlz({{Q^{\prime }}^{\prime }}_{DNNi}^{})$$ E48

Since both $${\tilde{X}}_{2Si}^{}(k)G_{Ri}^T$$ and $${\tilde{X}}_{2Si}^{}(k)G_{Li}^T$$ are symmetric PxP square matrixes, $${{Q^{\prime }}^{\prime }}_{ANNi}^{}$$ and $${{Q^{\prime }}^{\prime }}_{BNNi}^{}$$ are re-written as

$$\begin{array}{l}{{Q^{\prime }}^{\prime }}_{ANNi}^{}=⟨{\tilde{X}}_{2Si}^{}(k)G_{Ri}^T{R}_i^T(k)R_i^{}(k)G_{Ri}^{}{\tilde{X}}_{2Si}^T(k)+{\tilde{X}}_{2Si}^{}(k)G_{Li}^T{R}_i^T(k)R_i^{}(k)G_{Li}^{}{\tilde{X}}_{2Si}^T(k)⟩\\ +⟨{\tilde{X}}_{URi}^{}(k)R_i^T(k)R_i^{}(k){\tilde{X}}_{URi}^T(k)⟩\\ =⟨{\tilde{X}}_{2Si}^{}(k)(G_{Ri}^T{G}_{Ri}^{}+G_{Li}^T{G}_{Li}^{}){\tilde{X}}_{2Si}^T(k)⟩R_i^T(k)R_i^{}(k)+⟨{\tilde{X}}_{URi}^{}(k){\tilde{X}}_{URi}^T(k)⟩R_i^T(k)R_i^{}(k)\\ =(N{\sigma }_{Xi}^2(G_{Ri}^{}{G}_{Ri}^T+G_{Li}^{}{G}_{Li}^T)+N{\sigma }_{Ni}^2{I}_P^{})R_i^T(k)R_i^{}(k)\\ {{Q^{\prime }}^{\prime }}_{DNNi}^{}=N{\sigma }_{Ni}^2{I}_P^{}{R}_i^T(k)R_i^{}(k)\end{array}$$ E49

As evident by(47), (48) and(49), $${Q^{\prime }}_{2N2Ni}^{}(k)$$ is composed of major matrix $${Q^{\prime }}_{ANNi}^{}(k)$$ and noise matrix $${Q^{\prime }}_{DNNi}^{}(k)$$ . In the case of single talking where sound source signal power $${\sigma }_X^2$$ is much larger than un-correlated signal power $${\sigma }_{Ni}^2$$ , $$R_i^T(k)R_i^{}(k)$$ which minimizes eigenvalue spread in $$Q_{2N2Ni}^{}(k)$$ so as to attain the fastest convergence is given by making $${{Q^{\prime }}^{\prime }}_{ANNi}^{}$$ as a identity matrix by setting $$R_i^T(k)R_i^{}(k)$$ as

$$R_i^T(k)R_i^{}(k)\simeq {(N{\sigma }_{Xi}^2(G_{Ri}^{}{G}_{Ri}^T+G_{Li}^{}{G}_{Li}^T))}_{}^{-1}$$ E50

In other cases such as double talking or no talk situations, where we assume $${\sigma }_X^2$$ is almost zero, $$R_i^T(k)R_i^{}(k)$$ which orthogonalizes $${{Q^{\prime }}^{\prime }}_{ANNi}^{}$$ is given by

$$R_i^T(k)R_i^{}(k)\simeq {(N{\sigma }_{Ni}^2{I}_P^{})}_{}^{-1}$$ E51

Summarizing the above discussions, the fastest convergence is attained by setting $$R_i^T(k)R_i^{}(k)$$ as

$$R_i^T(k)R_i^{}(k)={\left(X_{P2Ni}^T(k)X_{P2Ni}^{}(k)\right)}_{}^{-1}$$ E52

Since

$$\begin{array}{l}⟨X_{P2Ni}^T(k)X_{P2Ni}^{}(k)⟩=\\ ⟨\left[\begin{array}{cc}{G}_{Ri}^{}{X}_{2Si}^T(k)+X_{URi}^T(k)& G_{Li}^{}{X}_{2Si}^T(k)+X_{ULi}^T(k)\end{array}\right]\left[\begin{array}{c}{X}_{2Si}^{}(k)G_{Ri}^T+X_{URi}^{}(k)\\ X_{2Si}^{}(k)G_{Li}^T+X_{ULi}^{}(k)\end{array}\right]⟩\\ =⟨G_{Ri}^{}{X}_{2Si}^T(k)X_{2Si}^{}(k)G_{Ri}^T+G_{Li}^{}{X}_{2Si}^T(k)X_{2Si}^{}(k)G_{Li}^T+X_{URi}^T(k)X_{URi}^{}(k)+X_{ULi}^T(k)X_{ULi}^{}(k)⟩\\ \simeq N{\sigma }_{Xi}^2(G_{Ri}^{}{G}_{Ri}^T+G_{Li}^{}{G}_{Li}^T)+2N{\sigma }_{Ni}^{2}I{}_P\end{array}$$ E53

By substituting (52) for (30), we obtain following affine projection iteration :

$${\widehat{h}}_{STi}(k+1)={\widehat{h}}_{STi}(k)+\alpha X_i^{}(k){(X_{P2Ni}^T(k)X_{P2Ni}^{}(k))}^{-1}{e}_{Pi}^{}(k)$$ E54

In an actual implementation $$\alpha$$ is replaced by $$\mu$$ for forgetting factor and $$\delta I$$ is added to the inverse matrix to avoid zero division as shown bellow.

$${\widehat{h}}_{ST}(k+1)={\widehat{h}}_{ST}(k)\text{+}\alpha X_{P2Ni}^{}\text{(k)[}{X}_{P2Ni}^T\text{(k)}{X}_{P2Ni}^{}\text{(k)}+\delta I{]}^{-1}\mu e_{Pi}(k)$$ E55

where $$\delta (\ll 1)$$ is very small positive value and

$$\mu =diag[1,(1-\mu ),\cdots ,{(1-\mu )}^{p-1}]$$ E56

The method can be intuitively understood using geometrical explanation in Fig. 3. As seen here, from a estimated coefficients in a k-1th plane a new direction is created by finding the nearest point on the i th plane in the case of traditional NLMS approach. On the other hand, affine projection creates the best direction which targets a location included in the both i-1 and i th plane.

Figure 3.

Very Simple Example for Affine Method

3.4. Stereo Normalized Least Mean Square (NLMS) method (M=1)

Stereo NLMS method is a case when M=1 of the gradient method.

Equation (54) is re-written when M =1 as

$${\widehat{h}}_{STi}(k+1)={\widehat{h}}_{STi}(k)+\alpha x_i^{}(k){(x_{Ri}^T(k)x_{Ri}^{}(k)+x_{Li}^T(k)x_{Li}^{}(k))}^{-1}{e}_i^{}(k)$$ E57

It is well known that convergence speed of (57) depends on the smallest and largest eigen-value of the matrix $$Q_{2N2Ni}^{}$$ . In the case of the stereo generation model in Fig.2 for single talking with small right and left noises, we obtain following determinant of $$Q_{2N2Ni}^{}$$ for M=1 as

$$\begin{array}{l}\left|Q_{2N2Ni}^{}(k)\right|=\left|x_i^{}(k){(x_i^T(k)x_i^{}(k))}_{}^{-1}{x}_i^T(k)\right|\\ \simeq {(}_{g_{Ri}^T}^{g_{Ri}^{}}\left|(g_{Ri}^{}{g}_{Ri}^T+g_{Li}^{}{g}_{Li}^T)\right|\left|{\sigma }_N^2{I}_N^{}\right|\end{array}$$ E58

If eigenvalue of $$g_{Ri}^{}{g}_{Ri}^T+g_{Li}^{}{g}_{Li}^T$$ are given as

$$\left|(g_{Ri}^{}{g}_{Ri}^T+g_{Li}^{}{g}_{Li}^T)\right|={\lambda }_{\min i}^2\cdots {\lambda }_{\max i}^2$$ E59

where $${\lambda }_{\min i}^2$$ and $${\lambda }_{\max i}^2$$ are the smallest and largest eigenvalues, respectively.

$$\left|Q_{2N2Ni}^{}(k)\right|$$ is given by assuming un-correlated noise power $${\sigma }_{Ni}^2$$ is very small ( $${\sigma }_{Ni}^2\ll {\lambda }_{\min i}^2$$ ) as

$$\left|Q_{2N2Ni}^{}(k)\right|={(g_{Ri}^T{g}_{Ri}^{}+g_{Li}^T{g}_{Li}^{})}_{}^{-1}\cdot {\sigma }_{Ni}^2\cdots {\sigma }_{Ni}^2\cdot {\lambda }_{\min i}^2\cdots {\lambda }_{\max i}^2$$ E60

Hence, it is shown that stereo NLMS echo-canceller’s convergence speed is largely affected by the ratio between the largest eigenvalue of $$g_{Ri}^{}{g}_{Ri}^T+g_{Li}^{}{g}_{Li}^T$$ and non-correlated signal power $${\sigma }_{Ni}^2$$ . If the un-correlated sound power is very small in single talking, the stereo NLMS echo canceller’s convergence speed becomes very slow.

3.5. Double adaptive filters for Rapid Projection (WARP) method

Naming of the WARP is that this algorithm projects the optimum solution between monaural space and stereo space. Since this algorithm dynamically changes the types of adaptive filters between monaural and stereo observing sound source characteristics, we do not need to suffer from rank drop problem caused by strong cross-channel correlation in stereo sound. The algorithm was originally developed for the acoustic echo canceller in a pseudo-stereo system which creates artificial stereo effect by adding delay and/or loss to a monaural sound. The algorithm has been extended to real stereo sound by introducing residual signal after removing the cross-channel correlation.

In this section, it is shown that WARP method is derived as an extension of affine projection which has been shown in 3.3.

By introducing error matrix $E_i^{}(k)$ which is defined by

$$E_i^{}(k)=\left[\begin{array}{cccc}{e}_{Pi}^{}(k)& e_{Pi}^{}(k-1)& \cdots & e_{Pi}^{}(k-p+1)\end{array}\right]$$ E61

iteration of the stereo affine projection method in (54) is re-written as

$${\widehat{H}}_{STi}(k+1)={\widehat{H}}_{STi}(k)+\alpha X_{P2Ni}^{}(k){(X_{P2Ni}^T(k)X_{P2Ni}^{}(k))}^{-1}{E}_i^{}(k)$$ E62

where

$${\widehat{H}}_{STi}^{}(k)=\left[\begin{array}{cccc}{\widehat{h}}_{STi}^{}(k)& {\widehat{h}}_{STi}^{}(k-1)& \cdots & {\widehat{h}}_{STi}^{}(k-p+1)\end{array}\right]$$ E63

In the case of strict single talking, following assumption is possible in the $ith$ LTI period by (53)

$$⟨X_{PNi}^T(k)X_{PNi}^{}(k)⟩\cong G_{RRLLi}^{}$$ E64

where $$G_{RRLLi}^{}$$ is a PxP symmetric matrix as

$$G_{RRLLi}^{}=N{\sigma }_{Xi}^2(G_{Ri}^{}{G}_{Ri}^T+G_{Li}^{}{G}_{Li}^T)$$ E65

By assuming $$G_{RRLLi}^{}$$ as a regular matrix, (62) can be re-written as

$${\widehat{H}}_{STi}(k+1)G_{RRLLi}^{}={\widehat{H}}_{STi}(k)G_{RRLLi}^{}+\alpha X_{P2Ni}^{}(k)E_i^{}(k)$$ E66

Re-defining echo path estimation matrix $${\widehat{H}}_{STi}(k)$$ by a new matrix $${\widehat{H^{\prime }}}_{STi}(k)$$ which is defined by

$${{\widehat{H}}^{\prime }}_{STi}(k)={\widehat{H}}_{STi}(k)G_{RRLLi}^{}$$ E67

(66) is re-written as

$${{\widehat{H}}^{\prime }}_{STi}(k+1)={{\widehat{H}}^{\prime }}_{STi}(k)+\alpha X_{P2Ni}^{}(k)E_i^{}(k)$$ E68

Then the iteration is expressed using signal matrix $$X_{2Si}^{}(k)$$ as

$${{\widehat{H}}^{\prime }}_{STi}(k+1)={{\widehat{H}}^{\prime }}_{STi}(k)+\alpha \left[\begin{array}{c}{X}_{2Si}^{}(k)G_{Ri}^T+X_{URi}^{}(k)\\ X_{2Si}^{}(k)G_{Li}^T+X_{ULi}^{}(k)\end{array}\right]E_i^{}(k)$$ E69

In the case of strict single talking where no un-correlated signals exist, and if we can assume $$G_{Li}^{}$$ is assumed to be an output of a LTI system $$G_{RLi}^{}$$ which is PxP symmetric regular matrix with input $$G_{Ri}^{}$$ , then (69) is given by

$$\begin{array}{l}\left[\begin{array}{c}{{\widehat{H}}^{\prime }}_{STRi}(k+1)\\ {{\widehat{H}}^{\prime }}_{STLi}(k+1)\end{array}\right]=\left[\begin{array}{c}{{\widehat{H}}^{\prime }}_{STRi}(k)\\ {{\widehat{H}}^{\prime }}_{STLi}(k)\end{array}\right]+\alpha \left[\begin{array}{c}{X}_{2Si}^{}(k)G_{Ri}^{}{E}_i^{}(k)\\ X_{2Si}^{}(k)G_{Ri}^{}{G}_i^{}{E}_i^{}(k)\end{array}\right]\\ \left[\begin{array}{c}{{\widehat{H}}^{\prime }}_{STRi}(k+1)\\ {{\widehat{H}}^{\prime }}_{STLi}(k+1)G_{RLi}^{-1}\end{array}\right]=\left[\begin{array}{c}{{\widehat{H}}^{\prime }}_{STRi}(k)\\ {{\widehat{H}}^{\prime }}_{STLi}(k)G_{RLi}^{-1}\end{array}\right]+\alpha \left[\begin{array}{c}{X}_{2Si}^{}(k)G_{Ri}^{}{E}_i^{}(k)\\ X_{2Si}^{}(k)G_{Ri}^{}{E}_i^{}(k)\end{array}\right]\end{array}$$ E70

It is evident that rank of the equation in (70) is N not 2N, therefore the equation becomes monaural one by subtracting the first law after multiplying $${(G_{RLi}^{})}_{}^{-1}$$ from the second low as

$${\widehat{H}}_{MONRLi}(k+1)={\widehat{H}}_{MONRLi}(k)+2\alpha X_{Ri}^{}(k)E_i^{}(k)$$ E71

where

$${\widehat{H}}_{MONRLi}(k)={{\widehat{H}}^{\prime }}_{STRi}(k)+{{\widehat{H}}^{\prime }}_{STLi}(k)G_{RLi}^{-1}$$ E72

or assuming $$G_{Ri}^{}=G_{Li}^{}{G}_{LRi}^{}$$

$${\widehat{H}}_{MONLRi}(k+1)={\widehat{H}}_{MONLRi}(k)+2\alpha X_{Li}^{}(k)E_i^{}(k)$$ E73

where

$${\widehat{H}}_{MONRLi}(k)={{\widehat{H}}^{\prime }}_{STLi}(k)+{{\widehat{H}}^{\prime }}_{STRi}(k)G_{LRi}^{-1}$$ E74

Selection of the iteration depends on existence of the inverse matrix $$G_{RLi}^{-1}$$ or $$G_{LRi}^{-1}$$ and the detail is explained in the next section.

By substituting (67) to (72) and (74), we obtain following equations;

$${\widehat{H}}_{MONRLi}(k)={\widehat{H}}_{STRi}(k)G_{RRLLi}^{}+{\widehat{H}}_{STLi}(k)G_{RRLLi}^{}{G}_{RLi}^{-1}$$ E75

or

$${\widehat{H}}_{MONLRi}(k)={\widehat{H}}_{STRi}(k)G_{RRLLi}^{}{G}_{LRi}^{-1}+{\widehat{H}}_{STLi}(k)G_{RRLLi}^{}$$ E76

From the stereo echo path estimation view point, we can obtain $${\widehat{H}}_{MONRLi}(k)$$ or $${\widehat{H}}_{MONLRi}(k)$$ , however we can’t identify right and left echo path estimation from the monaural one. To cope with this problem, we use two LTI periods for separating the right and left estimation results as

$$\begin{array}{l}\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{MONLRi}\\ {\widehat{H}}_{}^T{}_{MONLRi-1}\end{array}\right]=\left[\begin{array}{cc}{G}_{RRLLi}^T& G_{RRLLi}^{}{G}_{RLi}^{-1}\\ G_{RRLLii-1}^T& G_{RRLLi-1}^{}{G}_{RLi-1}^{-1}\end{array}\right]\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{STRi}\\ {\widehat{H}}_{STLi}\end{array}\right]\cdots G_{RLi}^{}\text{ and }{G}_{RLi-1}^{}\\ \text{ are regular matrix}\\ \left[\begin{array}{c}{\widehat{H}}_{}^T{}_{MONLRi}\\ {\widehat{H}}_{}^T{}_{MONLRi-1}\end{array}\right]=\left[\begin{array}{cc}{G}_{RLRLi}^T{G}_{LRi}^{-1}& G_{RRLLi}^{}\\ G_{RRLLi-1}^T{G}_{LRi-1}^{-1}& G_{RRLLi-1}^{}\end{array}\right]\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{STRi}\\ {\widehat{H}}_{STLi}\end{array}\right]\cdots G_{LRi}^{}\text{ and }{G}_{LRi-1}^{}\\ \text{ are regular matrix}\\ \left[\begin{array}{c}{\widehat{H}}_{}^T{}_{MONLRi}\\ {\widehat{H}}_{}^T{}_{MONLRi-1}\end{array}\right]=\left[\begin{array}{cc}{G}_{RRLLi}^T& G_{RRLLi}^{}{G}_{RLi}^{-1}\\ G_{RRLLi-1}^T{G}_{LRi-1}^{-1}& G_{RRLLi-1}^{}\end{array}\right]\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{STRi}\\ {\widehat{H}}_{STLi}\end{array}\right]\cdots G_{RLi}^{}\text{ and }{G}_{LRi-1}^{}\\ \text{ are regular matrix}\\ \left[\begin{array}{c}{\widehat{H}}_{}^T{}_{MONLRi}\\ {\widehat{H}}_{}^T{}_{MONLRi-1}\end{array}\right]=\left[\begin{array}{cc}{G}_{RRLLi}^T{G}_{LRi}^{-1}& G_{RRLLi}^{}\\ G_{RRLLi-1}^T& G_{RRLLi-1}^{}{G}_{RLi-1}^{-1}\end{array}\right]\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{STRi}\\ {\widehat{H}}_{STLi}\end{array}\right]\cdots G_{LRi}^{}\text{ and }{G}_{RLi-1}^{}\\ \text{ are regular matrix}\end{array}$$ E77

where $${\widehat{H}}_{}^{}{}_{MONLRi}$$ and $${\widehat{H}}_{}^{}{}_{MONLRi-1}$$ are monaural echo canceller estimation results at the end of each LTI period, $${\widehat{H}}_{}^{}{}_{STRi}$$ and $${\widehat{H}}_{}^{}{}_{STLi}$$ are right and left estimated stereo echo paths based on the $$i-1th$$ and $$ith$$ LTI period’s estimation results.

Equation (77) is written simply as

$${\widehat{H}}_{}^{}{}_{MONi,i-1}=W_i^{-1}{\widehat{H}}_{}^{}{}_{STi}$$ E78

where $${\widehat{H}}_{}^T{}_{MONRLij}$$ is estimation result matrix for the $$i-1th$$ and $$ith$$ LTI period’s as

$${\widehat{H}}_{}^{}{}_{MONi,i-1}=\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{MONRLi}\\ {\widehat{H}}^T{}_{MONRLi-1}\end{array}\right]$$ E79

$${\widehat{H}}_{}^T{}_{STi}$$ is stereo echo path estimation result as

$${\widehat{H}}_{}^{}{}_{STi}=\left[\begin{array}{c}{\widehat{H}}_{}^T{}_{STRi}\\ {\widehat{H}}_{}^T{}_{STLi}\end{array}\right]$$ E80

$$W_i^{-1}$$ is a matrix which projects stereo estimation results to two monaural estimation results and is defined by

$$W_i^{-1}=\{\begin{array}{c}\left[\begin{array}{cc}{G}_{RRLLi}^T& G_{RRLLi}^{}{G}_{RLi}^{-1}\\ G_{RRLLi-1}^T& G_{RRLLi-1}^{}{G}_{RLi-1}^{-1}\end{array}\right]\cdots G_{RLi}^{}\text{ and }{G}_{RLi-1}^{}\text{ are regular matrix}\\ \left[\begin{array}{cc}{G}_{RRLLi}^T{G}_{LRi}^{-1}& G_{RRLLi}^{}\\ G_{RRLLi-1}^T{G}_{RLRi-1}^{-1}& G_{RRLLi-1}^{}\end{array}\right]\cdots G_{LRi}^{}\text{ and }{G}_{LRi-1}^{}\text{ are regular matrix}\\ \left[\begin{array}{cc}{G}_{RRLLi}^T& G_{RRLLi}^{}{G}_{RLi}^{-1}\\ G_{RLi-1}^T{G}_{LRi-1}^{-1}& G_{RRLLi-1}^{}\end{array}\right]\cdots G_{RLi}^{}\text{ and }{G}_{LRi-1}^{}\text{ are regular matrix}\\ \left[\begin{array}{cc}{G}_{RRLLi}^T{G}_{LRi}^{-1}& G_{RRLLi}^{}\\ G_{RRLLi-1}^T& G_{RRLLi-1}^{}{G}_{RLi-1}^{-1}\end{array}\right]\cdots G_{LRi}^{}\text{ and }{G}_{RLi-1}^{}\text{ are regular matrix}\end{array}$$ E81

By swapping right side hand and left side hand in(78), we obtain right and left stereo echo path estimation using two monaural echo path estimation results as

$${\widehat{H}}_{}^{}{}_{STi}=W_i^{}{\widehat{H}}_{}^{}{}_{MONi,i-1}$$ E82

Since $$W_i^{-1}$$ and $$W_i^{}$$ are used to project optimum solutions in two monaural spaces to corresponding optimum solution in a stereo space and vice-versa, we call the matrixes as WARP functions. Above procedure is depicted in Fig. 4. As shown here, the WARP system is regarded as an acoustic echo canceller which transforms stereo signal to correlated component and un-correlated component and monaural acoustic echo canceller is applied to the correlated signal. To re-construct stereo signal, cross-channel correlation recovery matrix is inserted to echo path side. Therefore, WARP operation is needed at a LTI system change.

Figure 4.

Basic Principle for WARP method

In an actual application such as speech communication, the auto-correlation characteristics $$G_{RRLLi}^{}$$ varies frequently corresponding speech characteristics change, on the other hand the cross-channel characteristics $$G_{RLi}^{}$$ or $$G_{LRi}^{}$$ changes mainly at a far-end talker change. So, in the following discussions, we apply NLMS method as the simplest affine projection (P=1).

The mechanism is also intuitively understood by using simple vector planes depicted in Fig. 5.

Figure 5.

Very Simple Example for WARP method

As shown here, using two optimum solutions in monaural spaces (in this case on the lines) the optimum solution located in the two dimensional (stereo) space is calculated directly.

4. Realization of WARP

4.1. Simplification by assuming direct-wave stereo sound

Both stereo affine projection and WARP methods require P x P inverse matrix operation which needs to consider its high computation load and stability problem. Even though the WARP operation is required only when the LTI system changes such as far-end talker change and it is much smaller computation than inverse matrix operations for affine projection which requires calculations in each sample, simplification of the WARP operation is still important. This is possible by assuming that target stereo sound is composed of only direct wave sound from a talker (single talker) as shown in Fig. 6.

Figure 6.

Stereo Sound Generation System for Single Talking

In figure 6, a single sound source signal at an angular frequency $\omega$ in the $ith$ LTI period, $$x_{Si}(\omega )$$ , becomes a stereo sound composed of right and left signals, $$x_{Ri}(\omega )$$ and $$x_{Li}(\omega )$$ , through out right and left LTI systems, $$g_{SRi}(\omega )$$ and $$g_{SLi}(\omega )$$ with additional un-correlated noise $$x_{URi}(\omega )$$ and $$x_{ULi}(\omega )$$ as

$$\begin{array}{l}{x}_{Ri}(\omega )=g_{SRi}(\omega )x_{Si}(\omega )+x_{URi}(\omega )\\ x_{Li}(\omega )=g_{SLi}(\omega )x_{Si}(\omega )+x_{ULi}(\omega )\end{array}$$ E83

In the case of simple direct-wave systems, (83) can be re-written as

$$\begin{array}{l}{x}_{Ri}(\omega )=l_{Ri}{e}^{-j\omega {\tau }_{Ri}}{x}_{Si}(\omega )+x_{URi}(\omega )\\ x_{Li}(\omega )=l_{Li}{e}^{-j\omega {\tau }_{Li}}{x}_{Si}(\omega )+x_{ULi}(\omega )\end{array}$$ E84

where $$l_{Ri}$$ and $$l_{Li}$$ are attenuation of the transfer functions and ${\tau }_{Ri}$ and ${\tau }_{Li}$ are analog delay values.

Since the right and left sounds are sampled by $f_S(={\omega }_S^{}/2\pi )$ Hz and treated as digital signals, we use z- domain notation instead of $\omega$ -domain as

$$z=\exp [2\pi \omega j/{\omega }_s]$$ E85

In z-domain, the system in Fig.4 is expressed as shown in Fig. 7.

Figure 7.

WARP Method using Z-Function

As shown in Fig.7, the stereo sound generation model for $$x_i^{}(z)$$ is expressed as

$$x_i^{}(z)=\left[\begin{array}{c}{x}_{Ri}^{}(z)\\ x_{Li}^{}(z)\end{array}\right]=\left[\begin{array}{c}{g}_{SRi}(z)x_{Si}^{}(z)+x_{URi}^{}(z)\\ g_{SLi}(z)x_{Si}^{}(z)+x_{ULi}^{}(z)\end{array}\right]$$ E86

where $$x_{Ri}^{}(z)$$ , $$x_{Li}^{}(z)$$ , $$g_{SRi}(z)$$ , $$g_{SLi}(z)$$ , $$x_{Si}^{}(z)$$ , $$x_{URi}^{}(z)$$ and $$x_{ULi}^{}(z)$$ are z-domain expression of the band-limited sampled signals corresponding to $$x_{Ri}^{}(\omega )$$ , $$x_{Li}^{}(\omega )$$ , $$g_{SRi}^{}(\omega )$$ , $$g_{SLi}^{}(\omega )$$ , $$x_{URi}^{}(\omega )$$ and $$x_{ULi}^{}(\omega )$$ , respectively. Adaptive filer output $${\widehat{y}}_i^{}(z)$$ and microphone output $$y_i^{}(z)$$ at the end of $$ith$$ LTI period is defined as

$$\begin{array}{l}{\widehat{y}}_i^{}(z)={\widehat{h}}_i^T(z)x_i^{}(z)\\ y_i^{}(z)=h_{}^T(z)x_i^{}(z)+n_i^{}(z)\end{array}$$ E87

where $$n_i^{}(z)$$ is a room noise, $${\widehat{h}}_i^{}(z)$$ and $${\widehat{h}}_i^{}(z)$$ are stereo adaptive filter and stereo echo path characteristics at the end of $$ith$$ LTI period respectively and which are defined as

$${\widehat{H}}_{STi}^{}(z)=\left[\begin{array}{c}{\widehat{h}}_{Ri}^{}(z)\\ {\widehat{h}}_{Li}^{}(z)\end{array}\right],H_{ST}^{}(z)=\left[\begin{array}{c}{h}_R^{}(z)\\ h_L^{}(z)\end{array}\right]$$ E88

Then cancellation error is given neglecting near end noise by

$$e_i^{}(z)=y_i(z)-{\widehat{H}}_{STi}^T(z)x_i^{}(z)$$ E89

In the case of single talking, we can assume both $$x_{URi}^{}(z)$$ and $$x_{ULi}^{}(z)$$ are almost zero, and (89) can be re-written as

$$e_i^{}(z)=y_i(z)-(g_{SRi}(z){\widehat{h}}_{Ri}^{}(z)+g_{SLi}(z){\widehat{h}}_{Li}^{}(z))x_{Si}^{}(z)$$ E90

Since the acoustic echo can also be assumed to be driven by single sound source $$x_{Si}^{}(z)$$ , we can assume a monaural echo path $$h_{Monoi}^{}(z)$$ as

$$h_{Monoi}^{}(z)=g_{SRi}(z)h_R^{}(z)+g_{SLi}(z)h_L^{}(z)$$ E91

Then (90) is re-written as

$$e_i^{}(z)=(h_{Monoi}^{}(z)-(g_{SRi}(z){\widehat{h}}_{Ri}^{}(z)+g_{SLi}(z){\widehat{h}}_{Li}^{}(z)))x_{Si}^{}(z)$$ E92

This equation implies we can adopt monaural adaptive filter by using a new monaural quasi-echo path $${\widehat{h}}_{Monoi}^{}(z)$$ as

$${\widehat{h}}_{Monoi}^{}(z)=g_{SRi}(z){\widehat{h}}_{Ri}^{}(z)+g_{SLi}(z){\widehat{h}}_{Li}^{}(z)$$ E93

However, it is also evident that if LTI system changes both echo and quasi-echo paths should be up-dated to meet new LTI system. This is the same reason for the stereo echo canceller in the case of pure single talk stereo sound input. If we can assume the acoustic echo paths is time invariant for two adjacent LTI periods, this problem is easily solved by satisfying require rank for solving the equation as

$$\left[\begin{array}{c}{\widehat{h}}_{Monoi}^{}(z)\\ {\widehat{h}}_{Monoi-1}^{}(z)\end{array}\right]=W_i^{-1}(z)\left[\begin{array}{c}{\widehat{h}}_{Ri}^{}(z)\\ {\widehat{h}}_{Li}^{}(z)\end{array}\right]$$ E94

where

$$W_i^{-1}(z)=\left[\begin{array}{cc}{g}_{SRi}(z)& g_{SLi}(z)\\ g_{SRi-1}(z)& g_{SLi-1}(z)\end{array}\right]$$ E95

In other words, using two echo path estimation results for corresponding two LTI periods, we can project monaural domain quasi-echo path to stereo domain quasi echo path or vice -versa using WARP operations as

$$\begin{array}{l}{\widehat{H}}_{STi}^{}(z)=W_i^{}(z){\widehat{H}}_{Monoi}^{}(z)\\ {\widehat{H}}_{Monoi}^{}(z)=W_i^{-1}(z){\widehat{H}}_{STi}^{}(z)\end{array}$$ E96

where

$${\widehat{H}}_{Monoi}^{}(z)=\left[\begin{array}{c}{\widehat{h}}_{Monoi}^{}(z)\\ {\widehat{h}}_{Monoi-1}^{}(z)\end{array}\right],{\widehat{H}}_{STi}^{}(z)=\left[\begin{array}{c}{\widehat{h}}_{Ri}^{}(z)\\ {\widehat{h}}_{Li}^{}(z)\end{array}\right]$$ E97

In actual implementation, it is impossible to obtain real $$W_i^{}(z)$$ , which is composed of unknown transfer functions between a sound source and right and left microphones, so use one of the stereo sounds as a single talk sound source instead of a sound source. Usually, higher level sound is chosen as a pseudo-sound source because higher level sound is usually closer to one of the microphones. Then, the approximated WARP function $${\tilde{W}}_i^{}(z)$$ is defined as

$${\tilde{W}}_i^{}(z)=\{\begin{array}{c}\left[\begin{array}{cc}1& g_{RLi}(z)\\ 1& g_{RLi-1}(z)\end{array}\right]\cdots RR-Transition\\ \left[\begin{array}{cc}1& g_{RLi}(z)\\ g_{LRi-1}(z)& 1\end{array}\right]\cdots RL-Transition\\ \left[\begin{array}{cc}{g}_{LRi}(z)& 1\\ 1& g_{RLi-1}(z)\end{array}\right]\cdots LR-Transition\\ \left[\begin{array}{cc}{g}_{LRi}(z)& 1\\ g_{LRi-1}(z)& 1\end{array}\right]\cdots LL-Transition\end{array}$$ E98

where $$g_{RLi}(z)$$ and $$g_{LRi}(z)$$ are cross-channel transfer functions between right and left stereo sounds and are defined as

$$g_{RLi}(z)=g_{SLi}(z)/g_{SRi}(z),g_{LRi}(z)=g_{SRi}(z)/g_{SLi}(z)$$ E99

The RR, RL, LR and LL transitions in (98) mean a single talker’s location changes. If a talker’ location change is within right microphone side (right microphone is the closest microphone) we call RR-transition and if it is within left-microphone side (left microphone is the closest microphone) we call LL-transition. If the location change is from right-microphone side to left microphone side, we call RL-transition and if the change is opposite we call LR-transition. Let’s assume ideal direct-wave single talk case. Then the $\omega$ domain transfer functions, $$g_{RLi}(\omega )$$ and $$g_{LRi}(\omega )$$ are expressed in z-domain as

$$g_{RLi}(z)=l_{RLi}\phi ({\delta }_{RLi,}z)z^{-d_{RLi}},g_{LRi}(z)=l_{LRi}\phi ({\delta }_{LRi,}z)z^{-d_{LRi}}$$ E100

where $${\delta }_{RLi,}$$ , and $${\delta }_{LRi,}$$ are fractional delays and $$d_{RLi}$$ and $$d_{LRi}$$ are integer delays for the direct-wave to realize analog delays ${\tau }_{RLi}$ and ${\tau }_{LRi}$ , these parameters are defined as

$$\begin{array}{l}{d}_{RLi}=INT[{\tau }_{RLi}{f}_S].d_{LRi}=INT[{\tau }_{LRi}{f}_S],\\ {\delta }_{RLi,}=Mod[{\tau }_{RLi}{f}_S],{\delta }_{LRi,}=Mod[{\tau }_{LRi}{f}_S]\end{array}$$ E101

$$\phi (\delta ,z)$$ is a “Sinc Interpolation” function to interpolate a value at a timing between adjacent two samples and is given by

$$\phi (\delta ,z)={\displaystyle \sum _{\nu =-\infty }^{\infty }\frac{\sin (\pi \nu -\delta )}{(\pi \nu -\delta )}}{z}^{-\nu }$$ E102

4.2. Digital filter realization of WARP functions

Since LL-transition and LR transition are symmetrical to RR-transition and RL-transition respectively, Only RR and RL transition cases are explained in the following discussions. By solving (96) applying WARP function in(98), we obtain right and left stereo echo path estimation functions as

$$\begin{array}{c}{\widehat{h}}_{Ri}(z)=\frac{{\widehat{h}}_{Monoi}(z)-{\widehat{h}}_{Monoi-1}(z)}{g^{}{}_{RLi-1}(z)-g^{}{}_{RLi}(z)}\\ {\widehat{h}}_{Li}(z)=\frac{g^{}{}_{RLi-1}(z){\widehat{h}}_{Monoi}(z)-g^{}{}_{RLi}(z){\widehat{h}}_{Monoi-1}(z)}{g^{}{}_{RLi-1}(z)-g^{}{}_{RLi}(z)}\end{array}\cdots RR-Transition$$ E103

or

$$\begin{array}{c}{\widehat{h}}_{Ri}(z)=\frac{{\widehat{h}}_{Monoi}(z)-g^{}{}_{RLi-1}(z){\widehat{h}}_{Monoi-1}(z)}{1-g^{}{}_{LRi}(z)g^{}{}_{RLi-1}(z)}\\ {\widehat{h}}_{Li}(z)=\frac{{\widehat{h}}_{Monoi-1}(z)-g^{}{}_{LRi}(z){\widehat{h}}_{Monoi}(z)}{1-g^{}{}_{LRi}(z)g^{}{}_{RLi-1}(z)}\end{array}\cdots RL-Transition$$ E104

By substituting (100) for (104), we obtain

$$\begin{array}{c}{\widehat{h}}_{Ri}(z)=\frac{{\widehat{h}}_{Monoi}(z)-{\widehat{h}}_{Monoi-1}(z)}{l_{RLi-1}\phi ({\delta }_{RLi-1,}z)z^{-d_{RLi-1}}-l_{RLi}\phi ({\delta }_{RLi,}z)z^{-d_{RLi}}}\\ {\widehat{h}}_{Li}(z)=\frac{l_{RLi-1}\phi ({\delta }_{RLi-1,}z)z^{-d_{RLi-1}}{\widehat{h}}_{Monoi}(z)-l_{RLi}\phi ({\delta }_{RLi,}z)z^{-d_{RLi}}{\widehat{h}}_{Monoi-1}(z)}{l_{RLi-1}\phi ({\delta }_{RLi-1,}z)z^{-d_{RLi-1}}-l_{RLi}\phi ({\delta }_{RLi,}z)z^{-d_{RLi}}}\end{array}\cdots RR-Transition$$ E105

and

$$\begin{array}{c}{\widehat{h}}_{Ri}(z)=\frac{{\widehat{h}}_{Monoi}(z)-l_{RLi-1}\phi ({\delta }_{RLi-1,}z)z^{-d_{RLi-1}}{\widehat{h}}_{Monoi-1}(z)}{1-l_{LRi}\phi ({\delta }_{LRi,}z)l_{RLi-1}\phi ({\delta }_{RLi-1,}z)z^{-(d_{RLi-1}+d_{LRi})}}\\ {\widehat{h}}_{Li}(z)=\frac{{\widehat{h}}_{Monoi-1}(z)-l_{LRi}\phi ({\delta }_{LRi,}z)z^{-d_{LRi}}{\widehat{h}}_{Monoi}(z)}{1-l_{LRi}\phi ({\delta }_{LRi,}z)l_{RLi-1}\phi ({\delta }_{RLi-1,}z)z^{-(d_{RLi-1}+d_{LRi})}}\end{array}\cdots RL-Transition$$ E106

Since $$\phi ({\delta }_{,}z)$$ is an interpolation function for a delay $$\delta$$ , the delay is compensated by $$\phi (-{\delta }_{,}z)$$ as

$$\phi (-{\delta }_{,}z)\cdot \phi ({\delta }_{,}z)=1.$$ E107

From(107), (105) is re-written as

$$\begin{array}{c}{\widehat{h}}_{Ri}(z)=\frac{({\widehat{h}}_{Monoi}(z)-{\widehat{h}}_{Monoi-1}(z))l_{RLi-1}^{-1}\phi (-{\delta }_{RLi-1,}z)z^{d_{RLi-1}}}{1-(l_{RLi}{l}_{RLi-1}^{-1})\phi (-{\delta }_{RLi-1,}z)\phi ({\delta }_{RLi,}z)z^{-(d_{RLi}-d_{RLi-1})}}\\ {\widehat{h}}_{Li}(z)=\frac{{\widehat{h}}_{Monoi}(z)-l_{RLi}{l}_{RLi-1}^{-1}\phi ({\delta }_{RLi,}z)\phi (-{\delta }_{RLi-1,}z)z^{-d_{RLi}+d_{RLi-1}}{\widehat{h}}_{Monoi-1}(z)}{1-(l_{RLi}{l}_{RLi-1}^{-1})\phi (-{\delta }_{RLi-1,}z)\phi ({\delta }_{RLi,}z)z^{-(d_{RLi}-d_{RLi-1})}}\end{array}\cdots RR-Transition$$ E108

These functions are assumed to be digital filters for the echo path estimation results as shown in Fig.8.

Figure 8.

Digital Filter Realization for WARP Functions

4.3. Causality and stability of WARP functions

Stability conditions are obtained by checking denominator of (108) and(106) $D_{RRi}(z)$ and $D_{RLi}(z)$ which are defined as

$$\begin{array}{l}\left|D_{RRi}(z)\right|1\cdots RR-Transition\\ \left|D_{RLi}(z)\right|1\cdots RL-Transition\end{array}$$ E109

where

$$\begin{array}{l}{D}_{RRi}(z)=l_{RLi}{l}_{RLi-1}^{-1}\phi (-{\delta }_{RLi-1,}z)\phi ({\delta }_{RLi,}z)z^{-(d_{RLi}-d_{RLi-1})}\cdots RR-Transition\\ D_{RLi}(z)=l_{LRi}{l}_{RLi-1}\phi ({\delta }_{LRi,}z)\phi ({\delta }_{RLi-1,}z)z^{-(d_{RLi-1}+d_{LRi})}\cdots RL-Transition\end{array}$$ E110

From(109),

$$\begin{array}{l}\left|\phi (-{\delta }_{RLi-1,}z)\phi ({\delta }_{RLi,}z)\right|\le \left|\phi (-{\delta }_{RLi-1,}z)\right|\left|\phi ({\delta }_{RLi,}z)\right|\cdots RR-Transition\\ \left|\phi ({\delta }_{LRi,}z)\phi ({\delta }_{RLi-1,}z)\right|\le \left|\phi ({\delta }_{LRi,}z)\right|\left|\phi ({\delta }_{RLi-1,}z)\right|\cdots RR-Transition\end{array}$$ E111

By using numerical calculations,

$$\left|\phi ({\delta }_{,}z)\right|1.2$$ E112

Substituting (112) for (109),

$$\begin{array}{l}{l}_{RLi}{l}_{RLi-1}^{-1}1/1.44\cdots RR-Transition\\ l_{LRi}{l}_{RLi-1}1/1.44\cdots RL-Transition\end{array}$$ E113

Secondly, conditions for causality are given by checking the delay of the feedback component of the denominators $D_{RRi}(z)$ and $D_{RLi}(z)$ . Since convolution of a “Sinc Interpolation” function is also a “Sinc Interpolation” function as

$$\phi ({\delta }_A{}_{,}z)\cdot \phi ({\delta }_B{}_{,}z)=\phi ({\delta }_A+{\delta }_B{}_{,}z).$$ E114

Equation (110) is re-written as

$$\begin{array}{l}{D}_{RRi}(z)=l_{RLi}{l}_{RLi-1}^{-1}\phi ({\delta }_{RLi,}-{\delta }_{RLi-1,}z)z^{-(d_{RLi}-d_{RLi-1})}\cdots RR-Transition\\ D_{RLi}(z)=l_{LRi}{l}_{RLi-1}\phi ({\delta }_{LRi}+{\delta }_{RLi-1,}z)z^{-(d_{RLi-1}+d_{LRi})}\cdots RL-Transition\end{array}$$ E115

The “Sinc Interpolation” function is an infinite sum toward both positive and negative delays. Therefore it is essentially impossible to endorse causality. However, by permitting some errors, we can find conditions to maintain causality with errors. To do so, we use a “Quasi-Sinc Interpolation” function which is defined as

$$\tilde{\phi }(\delta ,z)={\displaystyle \sum _{\nu =-N_F+1}^{N_F}\frac{\sin (\pi \nu -\delta )}{(\pi \nu -\delta )}}{z}^{-\nu }$$ E116

where $2N_F$ is a finite impulse response range of the “Quasi-Sinc Interpolation” $$\tilde{\phi }(\delta ,z)$$ . Then the error power by the approximation is given as

$${\displaystyle \oint \tilde{\phi }(\delta ,z)}{\tilde{\phi }}^{*}(\delta ,z)dz={\displaystyle \sum _{\nu =-\infty }^{-N_F}\frac{{\sin }^2(\pi \nu -\delta )}{{(\pi \nu -\delta )}^2}}{z}^{-\nu }+{\displaystyle \sum _{\nu =N_F+1}^{\infty }\frac{{\sin }^2(\pi \nu -\delta )}{{(\pi \nu -\delta )}^2}}{z}^{-\nu }$$ E117

Equation (116) is re-written as

$$\tilde{\phi }(\delta ,z)={\displaystyle \sum _{\nu =0}^{2N_F-1}\frac{\sin (\pi \nu -\delta )}{(\pi \nu -\delta )}}{z}^{-\nu -N_F+1}$$ E118

By substituting (118) for (115),

$$\begin{array}{l}{D}_{RRi}(z)\simeq l_{RLi}{l}_{RLi-1}^{-1}\tilde{\phi }({\delta }_{RLi,}-{\delta }_{RLi-1,}z)z^{-(d_{RLi}-d_{RLi-1}-N_F+1)}\cdots RR-Transition\\ D_{RLi}(z)\simeq l_{LRi}{l}_{RLi-1}\tilde{\phi }({\delta }_{LRi}+{\delta }_{RLi-1,}z)z^{-(d_{RLi-1}+d_{LRi}-N_F+1)}\cdots RL-Transition\end{array}$$ E119

Then conditions for causality are

$$\begin{array}{l}{d}_{RLi}-d_{RLi-1}\ge N_F-1\cdots RR-Transition\\ d_{RLi-1}+d_{LRi}\ge N_F-1\cdots RL-Transition\end{array}$$ E120

The physical meaning of the conditions are the delay difference due to talker’s location change should be equal or less than cover range of the “Quasi-Sinc Interpolation” $$\tilde{\phi }(\delta ,z)$$ in the case of staying in the same microphone zone and the delay sun due to talker’s location change should be equal or less than cover range of the “Quasi-Sinc Interpolation” $$\tilde{\phi }(\delta ,z)$$ in the case of changing the microphone zone.

4.4. Stereo echo canceller using WARP

Total system using WARP method is presented in Fig. 9, where the system composed of five components, far-end stereo sound generation model, cross-channel transfer function (CCTF) estimation block, stereo echo path model, monaural acoustic echo canceller (AEC-I) block, stereo acoustic echo canceller (AEC-II) block and WARP block.

Figure 9.

System Configuration for WARP based Stereo Acoustic Echo Canceller

As shown in Fig.9, actual echo cancellation is done by stereo acoustic echo canceller (AEC-II), however, a monaural acoustic echo canceller (AEC-I) is used for the far-end single talking. The WARP block is active only when the cross-channel transfer function changes and it projects monaural echo chancellor echo path estimation results for two LTI periods to one stereo echo path estimation or vice-versa.

5. Computer simulations

5.1. Stereo sound generation model

Computer simulations are carried out using the stereo generation model shown in Fig.10 for both white Gaussian noise (WGN) and an actual voice. The system is composed of cross-channel transfer function estimation blocks (CCTF), where all signals are assumed to be sampled at $f_S=8KHz$ after 3.4kHz cut-off low-pass filtering. Frame length is set to 100 samples. Since the stereo sound generation model is essentially a continuous time signal system, over-sampling (x6, $f_A=48KHz$ ) is applied to simulate it. In the stereo sound generation model, three far-end talker’s locations, A Loc(1)=(-0.8,1.0), B Loc(2)=(-0.8,0.5), C Loc(3)=(-0.8,0.0), D Loc(4)=(-0.8,-0.5) and D Loc(5)=(-0.8,-1.0) are used and R/L microphone locations are set to R-Mic=(0,0.5) and L-Mic=(0,-0.5), respectively. Delay is calculated assuming voice wave speed as 300m/sec. In this set-up, talker’s position change for WGN is assumed to be from location A to location B and finally to location D, in which each talker stable period is set to 80 frames. The position change for voice is from C->A and the period is set to 133 frames. Both room noise and reverberation components in the far-end terminals is assumed, the S/N is set to 20dB ~ 40dB.

Figure 10.

Stereo Sound Generation Model and Cross-Channel Transfer Function Detector

5.2. Cross-channel transfer function estimation

In WARP method, it is easily imagine that the estimation performance of the cross-channel transfer function largely affects the echo canceller cancellation performances. To clarify the transfer function estimation performance, simulations are carried out using the cross-channel transfer function estimators (CCTF). The estimators are prepared for right microphone side sound source case and left microphone side sound source case, respectively. Each estimator has two NLMS adaptive filters, longer (128) tap one and shorter (8) tap one. The longer tap adaptive filter (AF1) is used to find a main tap and shorter one (AF2) is used to estimate the transfer function precisely as an impulse response.

Figure 11 shows CCTF estimation results as the AF1 tap coefficients after convergence setting single male voice sound source to the locations C, B and A in Fig. 11. Detail responses obtained by AF2 are shown in Fig. 12.As shown the results, the CCTF estimation works correctly in the simulations.

Figure 11.

Impulse Response Estimation Results in CCTF Block

Figure 12.

Estimated Tap Coefficients by Short Tap Adaptive Filter in CCTF Estimation Block

Figure 13.

Cross-Channel Correlation Cancellation Performances

Cancellation performances of the cross-channel correlation under room noise (WGN) are obtained using the adaptive filter (AF2) and are shown is Fig. 13, where S/N is assumed to be 20dB, 30dB and 40dB. In the figure $$C{L}_{RL}(dB)$$ is power reduction in dB which is observed by the signal power before and after cancellation of the cross-channel correlation by AF2. As shown here, more than 17dB cross-channel correlation cancellation is attained.

5.3. Echo canceller performances

To evaluate echo cancellation performances of the WARP acoustic echo canceller which system is shown in Fig. 10, computer simulations are carried out assuming 1000tap NLMS adaptive filters for both stereo and monaural echo cancellers. The performances of the acoustic echo canceller are evaluated by two measurements. The first one is the echo return loss enhancement $$ERL{E}_{ij}(dB)$$ , which is applied to the WGN source case and is defined as

$${\mathrm{ERLE}}_{L\cdot (i-1)+j-1}=\{\begin{array}{c}10{\log }_{10}({\displaystyle \sum _{k=0}^{N_F-1}{y}^2{}_{i,j,k}}/{\displaystyle \sum _{k=0}^{N_F-1}{e}_{MON}^2{}_{i,j,k}})\cdots MonauralEchoCanceller\\ 10{\log }_{10}({\displaystyle \sum _{k=0}^{N_F-1}{y}^2{}_{i,j,k}}/{\displaystyle \sum _{k=0}^{N_F-1}{e}_{ST}^2{}_{i,j,k}})\cdots StereoEchoCanceller\end{array}$$ E121

where $$e_{MON}^{}{}_{i,j,k}$$ and $$e_{MON}^{}{}_{i,j,k}$$ are residual echo for the monaural echo canceller (AEC-I) and stereo echo canceller (AEC-II) for the $$kth$$ sample in the $$jth$$ frame in the $$ith$$ LTI period, respectively. The second measurement is normalized misalignment of the estimated echo paths and are defined as

$$\begin{array}{l}\\ {\mathrm{NORM}}_{L\cdot (i-1)+j-1}=10{\log }_{10}(\frac{{(h_R)}^T(h_R)+{(h_L)}^T(h_L)}{{(h_R-{\widehat{h}}_{Ri,j})}^T(h_L-{\widehat{h}}_{Li,j})})\end{array}$$ E122

where $${\widehat{h}}_{Ri,j}$$ and $${\widehat{h}}_{Li,j}$$ are stereo echo canceller estimated coefficient arrays at the end of $$(i,j)th$$ frame, respectively. $$h_R$$ and $$h_L$$ are target stereo echo path impulse response arrays, respectively.

5.3.1. WARP echo canceller basic performances for WGN

The simulation results for WARP echo canceller in the case of WGN sound source, no far-end double talking and no local noise, are shown in Fig. 14. In the simulations, talker is assumed to move from A to E every 80 frames (1sec). In Fig.14, the results (a) and (b) show ERLEs for monaural and stereo acoustic echo cancellers (AEC-I and AEC-II), respectively.

Figure 14.

WARP Echo Cancellation Performances for WGN Source

The WARP operations are applied at the boundaries of the three LTI periods for the talkers C, D and E. NORM for the stereo echo canceller (AEC-II). As shown here, after two LTI periods (A, B periods), NORM and ERLE improves quickly by WARP projection at WARP timings in the Fig. 16. As for ERLE, stereo acoustic echo canceller shows better performance than monaural echo canceller. This is because the monaural echo canceller estimates an echo path model which is combination of CCTF and real stereo echo path and therefore the performance is affected by the CCTF estimation error. On the other hand, the echo path model for the stereo echo canceller is purely the stereo echo path model which does not include CCTF.

Figure 15.

Echo Cancellation Performance Comparison for WGN Source

Secondary, the WARP acoustic echo canceller is compared with a stereo echo canceller based on an affine projection method. In this case, the right and left sounds at $$kth$$ sample in the $$(i,j)th$$ frame, $${x^{\prime }}_{Rijk}$$ and $${x^{\prime }}_{Lijk}$$ , are assumed to have independent level shift to the original right and left sounds, $$x_{Rijk}$$ and $$x_{Lijk}$$ , for simulating small movement of talker’s face as

$$\begin{array}{l}{x^{\prime }}_{Rijk}=(1+{\alpha }_{Level}\cdot \sin (2\pi k/(f_s\cdot T{}_X)))x_{Rijk}\\ {x^{\prime }}_{Lijk}=(1+{\alpha }_{Level}\cdot \cos (2\pi k/(f_s\cdot T{}_X)))x_{Lijk}\end{array}$$ E123

where $${\alpha }_{Level}$$ and $$T{}_X$$ are constants which determine the level shift ratio and cycle. Figure 15 shows the cancellation performances when $${\alpha }_{Level}$$ and $$T{}_X$$ are 10% and 500msec, respectively.

Figure 16.

WARP Echo Canceller Performances Affected by Far-End Back Ground Noise

In Fig. 15, the WARP method shows more than 10dB better stereo echo path estimation performance, NORM, than that of affine projection (P=3). ERLE by stereo echo canceller base on WARP method is also better than affine projection (P=3). ERLE by monaural acoustic echo canceller based on WARP method is somehow similar cancellation performance as affine method (P=3), however ERLE improvement after two LTI periods by the WARP based monaural echo canceller is better than affine based stereo echo canceller.

Figure 16 shows the echo canceller performances in the case of CCTF estimation is degraded by room noise in the far-end terminal. S/N in the far-end terminal is assumed to be 30dB or 50dB. Although the results clearly show that lower S/N degrade ERLR or NORM, more than 15dB ERLE or NORM is attained after two LTI periods.

Figure 17 shows the echo canceller performances in the case of echo path change happens. In this simulation, echo path change is inserted at 100frame. The echo path change is chosen 20dB, 30dB and 40dB. It is observed that echo path change affects the WARP calculation and therefore WARP effect degrades at 2^nd and third LTI period boundary.

Figure 17.

WARP Echo Canceller Cancellation Performance Drops Due to Echo Path Chance

Figure 18 summarizes NORM results for stereo NLMS method, affine projection method as WARP method. In this simulation, as a non-linear function for affine projection, independent absolute values of the right and left sounds are added by

$$\begin{array}{l}{x^{\prime }}_{Rijk}=x_{Rijk}+0.5\cdot {\alpha }_{ABS}\cdot (x_{Rijk}+\left|x_{Rijk}\right|)\\ {x^{\prime }}_{Lijk}=x_{Lijk}+0.5\cdot {\alpha }_{ABS}\cdot (x_{Lijk}-\left|x_{Lijk}\right|)\end{array}$$ E124

where $${\alpha }_{ABS}$$ is a constant to determine non-liner level of the stereo sound and is set to 10%. In this simulation, an experiment is carried out assuming far-end double talking, where WGN which power is same as far-end single talking is added between 100 and 130 frames.

As evident from the results in Fig. 18, WARP method shows better performances for the stereo echo path estimation regardless far-end double talking existence. Even in the case 10% far end signal level shit, WARP method attains more than 20% NORM comparing affine method (P=3) with 10% absolute non-linear result.

Figure 18.

Echo Path Estimation Performance Comparison for NLMS, Affine and WARP Methods

5.3.2. WARP echo canceller basic performances for voice

Figure 19 shows NORM and residual echo level (Lres) for actual male voice sound source. Since voice sound level changes frequently, we calculate residual echo level Lres (dB) instead of ERLE(dB) for white Gaussian noise case. Although slower NORM and Lres convergence than white Gaussian is observed, quick improvement for the both metrics is observed at the talker B and A border. In this simulation, we applied 500 tap NLMS adaptive filter. Affine projection may give better convergence speed by eliminating auto-correlation in the voice, however it is independent effect from WARP effect. WARP and affine projection can be used together and may contribute to convergence speed up independently.

Figure 19.

Residual Echo (Lres (dB) Level and Normalized Estimated Echo Misalignment.(NORM) for the voice Source at Far-end Terminal S/N=30dB. (Level shift 0, 500tap Step gain=1.0)

6. Conclusions

In this chapter stereo acoustic echo canceller methods are studied from cross-channel correlation view point aiming at conversational DTV use. Among many stereo acoustic echo cancellers, we focused on AP (including LS and stereo NLMS methods) and WARP methods, since these approaches do not cause any modification nor artifacts to speaker output stereo sound which is not desirable consumer audio-visual products such as DTV. In this study, stereo sound generation system is modeled by using right and left Pth order LTI systems with independent noises. Stereo LS method (M=2P) and stereo NLMS method (M=P=1) are two extreme cases of general AP method which requires MxM inverse matrix operation in each sample. Stereo AP method (M=P) can produce the best iteration direction fully adopting un-correlated component produced by small fluctuation in the stereo cross-channel correlation by calculating PxP inverse matrix operations in each sample. Major problem of the method is that it cannot cope with strict single talking where no un-correlated signals exist in right and left channels and therefore rank drop problem happens. Contrary to AP method, WARP method creates a stereo echo path estimation model applying a monaural adaptive filter for two LTI periods at a chance of far-end talker change. Since it creates stereo echo path estimation using two monaural echo path models for two LTI periods, we do not suffer from any rank drop problem even in a strict single talking. Moreover, using WARP method, computational complexity can be reduced drastically because WARP method requires PxP inverse matrix operations only at LTI characteristics change such as far-end talker change. However, contrary to AP method, it is clear that performance of WARP method may drop if fluctuation in cross-channel correlation becomes high. Considering above pros-cons in affine projection and WARP methods, it looks desirable to apply affine method and WARP method dynamically depending on the nature of stereo sound. In this chapter, an acoustic echo canceller based on WARP method which equips both monaural and stereo adaptive filters is discussed together with other gradient base stereo adaptive filter methods. The WARP method observes cross-channel correlation characteristics in stereo sound using short tap pre-adaptive filters. Pre-adaptive filter coefficients are used to calculate WARP functions which project monaural adaptive filter estimation results to stereo adaptive filter initial coefficients or vice-versa.

To clarify effectiveness WARP method, simple computer simulations are carried out using white Gaussian noise source and male voice, using 128tap NLMS cross-channel correlation estimator, 1000tap monaural NLMS adaptive filter for monaural echo canceller and 2x1000tap (2x500tap for voice) multi-channel NLMS adaptive filter for stereo echo canceller. Followings are summary of the results:

1. Considering sampling effect for analog delay, x6 over sampling system is assumed for stereo generation model. 5 far-end talker positions are assumed and direct wave sound from each talker is assumed to be picked up by far-end stereo microphone with far-end room background noise. The simulation results show we can attain good cross-channel transfer function estimation rapidly using 128tap adaptive filter if far-end noise S/N is reasonable (such as 20-40dB).

2. Using the far-end stereo generation model and cross-channel correlation estimation results, 1000tap NLMS monaural NLMS adaptive filter and 2-1000 tap stereo NLMS adaptive filters are used to clarify effectiveness of WARP method. In the simulation far-end talker changes are assumed to happen at every 80frames (1frame=100sample). Echo return loss Enhancement (ERLE) MORMalized estimation error power (NORM) are used as measurements. It is clarified that both ERLE and NORM are drastically improved at the far-end talker change by applying WARP operation.

3. Far-end S/N affects WARP performance, however, we can still attain around SN-5dB ERLE or NORM.

4. We find slight convergence improvement in the case of AP method (P=3) with non-linear operation. However, the improvement is much smaller than WARP at the far-end talker change. This is because sound source is white Gaussian noise in this simulation and therefore merit of AP method is not archived well.

5. Since WARP method assumes stereo echo path characteristics remain stable, stereo echo path characteristics change degrade WARP effectiveness. The simulation results show the degradation depends on how much stereo echo path moved and the degradation appears just after WARP projection.

6. WARP method works correctly actual voice sound too. Collaboration with AP method may improve total convergence speed further more because AP method improves convergence speed for voice independent from WARP effect.

As for further studies, more experiments in actual environments are necessary. The author would like to continue further researches to realize smooth and natural conversations in the future conversational DTV.

7. Appendix

If $N\times N$ matrix $Q$ is defined as

$$Q=X_{2S}^T(k)G_{}^T{G}_{}^{}{X}_{2S}^{}(k)$$ E125

where $X_{2S}^{}(k)$ is a $(2P-1)$ sample array composed of white Gaussian noise sample $$x_{}(k)$$ as

$$\begin{array}{l}{X}_{2S}^{}(k)=\left[x_{}(k),x_{}(k-1),\cdots x_{}(k-N+1)\right]\\ x_{}(k)={[}^{x_{}}\end{array}$$ E126

$$G$$ is defined as a $(2P-1)\times P$ matrix as

$$G_{}^{}=\left[\begin{array}{cccc}{g}_{}^T& 0& \cdots & 0\\ 0& g_{}^T& \ddots & 0\\ 0& 0& \cdots & 0\\ 0& 0& \cdots & g_{}^T\end{array}\right]$$ E127

where $$g$$ is P sample array defined as

$$g_{}={[g_0,g_1,\cdots ,g_{\nu }\cdots g_{P-1}]}^T$$ E128

Then $⟨Q⟩$ is a Toepliz matrix and is expressed using $P\times P$ ( $P\le N$ ) Toepliz matrix $⟨Q^{\prime }⟩$ as

$$⟨Q⟩=Tlz(⟨Q^{\prime }⟩)$$ E129

This is because $(u,v)th$ element of the matrix $⟨Q⟩$ , $a_{TlZ}^{}(u,v)$ is defined as

$$a_{TlZ}^{}(u,v)=⟨x_{}^T(k-u)G_{}^T{G}_{}^{}x(k-v)⟩$$ E130

Considering

$$⟨x_{}^T(k-u)G_{}^{T}G{x}_{}^{}(k-v)⟩=0\cdots \text{for all }\left|u-v\right|\ge P$$ E131

the element $a_{TlZ}^{}(u,v)$ is given as

$$a_{TlZ}^{}(u,v)=\{\begin{array}{c}a(u-v,0)\cdots P-1\ge u-v\ge 0\\ \begin{array}{c}a(0,v-u)\cdots P-1\ge v-u0\\ 0\cdots \left|u-v\right|\ge P\end{array}\end{array}$$ E132

By setting the $(u,v)$ th element of $P\times P$ ( $P\le N$ ) Toepliz matrix $⟨Q^{\prime }⟩$ as $a_{TlZ}^{}(u,v)$ ( $(0\le uP,0\le vP)$ ), we define a function $Tlz(⟨Q^{\prime }⟩)$ which determines $N\times N$ Toepliz matrix $Q$ .

It is noted that if $Q^{\prime }$ is a identity matrix $Q$ is also identity matrix.