## Abstract

The theoretical principles of intersymbol interference (ISI) and channel equalization in wireless communication systems are addressed. Several conventional and well-known equalization techniques are discussed and compared such as zero forcing (ZF) and maximum likelihood (ML). The main section in this chapter is devoted to an abstract concept of equalization approach, namely, dual channel equalization (DCE). The proposed approach is flexible and can be employed and integrated with other linear and nonlinear equalization approaches. Closed expressions for the achieved signal-to-noise ratio (SNR) and bit error rate (BER) in the case of ZF-DCE and ML-DCE are derived. According to the obtained outcomes, the DCE demonstrates promising improvements in the equalization performance (BER reduction) in comparison with the conventional techniques.

### Keywords

- channel equalization
- intersymbol interference (ISI)
- zero forcing (ZF) equalization
- maximum likelihood (ML) equalization
- bit error rate (BER)
- dual channel equalization (DCE)

## 1. Introduction

All types of microwave wireless communication systems under different digital modulation schemes and antennas configuration suffer from the channel effects and related problems such as attenuation, signal amplitude and phase distortions, time-varying fading (Doppler shift), multipath fading, and intersymbol interference (ISI). A common and well-known wireless channel modeling method is based on the representation of the channel as a band-limited digital filter, i.e., linear time-invariant (LTI) filter with specific transfer function (impulse response). Thus, to alleviate and reduce the channel effects, especially for multipath fading and ISI, it is possible to design a digital filter with transfer function that is inverse to the transfer function of the associated wireless channel. This digital filter is called the equalizer [1, Chapter 10]. Additionally, the employment of multiple antennas can also help to mitigate the multipath fading consequences (transmit/receive diversity) and increase the data rate (spatial multiplexing).

In practice, the wireless transmission system sends a sequence of messages (one-shot transmission) where these successive transmissions should not interfere even if they are closely spaced to increase the data rate. This interference between the successive transmissions is referred as intersymbol interference (ISI) that is able to complicate and reduce the detection performance [2, Chapter 4]. The simple symbol-by-symbol (SBS) detector (optimal in the case of additive white Gaussian noise AWGN channel) cannot be the maximum likelihood estimator for a sequence of message under ISI problem. A receiver for succession messages detection is shown in Figure 1 , where the matched filter outputs are processed by the receiver and SBS detector to generate the estimate

An equalizer or equalization method can be essentially embedded in the contents of the receiver block presented in Figure 1 . Different equalization techniques lead to different receiver structures that are not always optimal for detection, but rather are widely implemented as suboptimal cost-effective solutions that alleviate the ISI. Any equalization approach converts the band-limited channel with ISI into memory less appearing channel (AWGN-like) at the receiver output. The wireless channel equalization forms a major challenge in current and future communication networks.

In Section 2 of this chapter, the ISI between successive transmissions is modeled to prove that the distortion caused by this overlapping is unacceptable and some corrective actions should be applied. Some targeted and desired wireless channel responses that exhibit no ISI are discussed in Section 3 with the corresponding Nyquist criterion for the ISI-free channels. In fact, the signal-to-noise ratio (SNR) parameter used to quantify the receiver performance can be consistently considered for equalization techniques evaluation as well. Several conventional and well-known equalization approaches are presented in Section 4 such as zero-forcing equalizer (ZFE), minimum mean square error (MMSE) equalizer, and decision feedback equalizer (DFE). In the last section (Section 5), the proposed equalization approach and its performance are discussed and compared to other equalization techniques under the same initial conditions.

## 2. Successive message transmission and ISI

The frequency or wireless channel reuse is a common technique to transmit several succession messages separated by the symbol period (

Increasing the data rate *R* can be achieved by decreasing *T* which reduces or narrows the time between message transmissions and as a consequence increases ISI on the band-limited channel. The transmitted signal of *K* successive transmissions (conveying one of *MK *possible messages) is given by

The detection of *R* increases). The ISI can be analyzed by rewriting Eq. (2) using the following form:

where the original transmissions

produces orthogonal functions for all integer-multiple of

The band-limited noise-free channel output used for successive transmission of data symbol can be presented as

where

The Nyquist criterion [3, Chapter 1] can specify the conditions for ISI-free channel on which SBS detector is optimal. This criterion helps to construct band-limited functions to reduce the ISI negative effects. Another way to explain the ISI problem is to relate it with the channel frequency response. When consecutive symbols are transmitted using linear modulation over a wireless channel, the frequency response (impulse response) of the channel makes a transmitted symbol to be spread in the time domain. Thus, the ISI is generated because the previously transmitted and currently received symbols are overlapped. The Nyquist theorem could relate the time domain conditions to its equivalent frequency domain ones. Simply, considering the channel impulse response

The last condition can be represented in another form as follows (the Nyquist ISI criterion):

where

where

The eye diagram is a widely used convenient method to observe the effects of ISI and noise introduced by the channel where the quality of the received signal (the ability to correctly recover the symbols and timing) can be illustrated (oscilloscope presentation). The interpretation of the eye diagram gives important information such as:

sensitivity to timing error or jitter (smaller is better);

wasted power;

amount of distortion at sampling instants;

amount of noise tolerance (larger is better);

best time for sampling;

the matching degree between the transmitter and receiver filters;

the presence of ISI; and

measurement of eye opening is performed to estimate the achievable BER.

The eye diagram using raised cosine filtering is presented in Figure 4 for binary phase shift keying (BPSK)-modulated symbols at two roll-off factor values

## 3. The equalization main concept

As mentioned before, the equalization technique is the design of a digital filter with inverse or counter transfer function in accordance with the transfer function of the associated wireless channel. This concept is shown in Figure 5 where the frequency responses of the wireless channel and the equalizer are compared.

The equalizer design concept is presented in Figure 6 (a simple block diagram for the channel effect and the equalizer transfer function). The discrete time model that should be considered under the equalization technique designing process is presented using the following form:

where

where

where

is reduced using equalization.

It is useful at this point to demonstrate the effects of the equalization on the detection performance or quality at the receiver side by presenting the difference between the received symbols with and without applying equalization. This comparison is given in Figure 7 for the quadrature phase shift keying (QPSK)-modulated symbols. When the equalization is not applied, the uncertainty on the received signal constellation is very high (caused by the channel effect) and as a consequence the detection quality is low (detection performance degradation). Applying the equalization technique, the uncertainty level is lower and the detection performance is improved.

As a last word in this section, the ISI can produce a bias in the SNR value at the receiver. The error

For a given unbiased receiver [4] for a detection decision rule on a general signal constellation

where

## 4. Equalization techniques

In general, the channel equalization techniques are classified to linear and nonlinear algorithms or to blind (without training sequence) and nonblind based on the degree of knowledge. The earliest mentions of digital equalization techniques are made under different design criterions, for example, zero forcing (ZF) equalization [5], minimum mean square error (MMSE) equalization [6], maximum likelihood (ML) equalization [7], decision feedback equalization (DFE) [8], and maximum *a posteriori* (MAP) equalization [9]. In this chapter, a new nonlinear equalization approach is proposed based on the dual channel equalization (DCE) idea with the purpose to improve the equalization performance, namely, the BER reduction, in comparison with the other widely used equalizers under multiple input multiple output (MIMO) wireless channel. The simulation results demonstrate considerable and promising performance improvements applying the suggested equalization approach in comparison with the conventional techniques. In order to help the reader to acquire the basics of equalization concepts, a few well-known and conventional equalization techniques will be reviewed and discussed in the next subsections.

### 4.1. System model

The complex baseband MIMO wireless channel model is considered with number of transmit antennas equal to

The received signal at the input of the receiver (the I/O relation of the MIMO channel) can be presented in the following form:

where

where

The channel matrix

### 4.2. ZF equalization

ZF is a linear equalization that applies the inverse of the wireless channel frequency response to alleviate the channel effects on the received signal and restore the transmitted symbols. The name is assigned based on the reduction of ISI down to zero in the noise-free channel case (forcing the residual ISI to zero). The ZF equalizer can be designed using finite of infinite impulse response filters (FIR or IIR filters).

Employing ZF equalization technique with matrix

The general form of the ZF equalizer matrix

where

where

where *m*th diagonal element. The denominator of the achieved partial SNR given in Eq. (22) can be presented in terms of the *m*th column

The ZF equalizer tries to null and cancel out all the interfering terms that are sometimes accompanied with noise amplification, for this reason, ZF is not optimal under very noisy channels. In the case of FIR filter use (to deal with noncausal components a decision delay is applied), a complete elimination of ISI problem is not possible owing to the finite filter length. Alternative criterion called peak-distortion criterion can be applied to minimize the maximum possible signal distortion due to ISI at the equalizer output.

### 4.3. MMSE equalization

The main objective of MMSE equalizer is to minimize the variance of the error signal

where

The MMSE matrix

The BER for this equalizer considering the NP criterion and BPSK modulation can be approximated using the following form [11]:

where *Q*-function. The output SNRs of the

where

Again the MMSE equalizer can be implemented with FIR and IIR filters and in both cases the error signal

which is more general form of SNR in comparison with the form in Eq. (26).

### 4.4. ML equalization

The ML equalization technique tests all the possible data symbols and chooses the one that has the maximum probability of correctness at the output (optimal in the sense of minimizing the probability of error

As follows, the ML-based equalizer selects the data sequence

The achieved BER of ML equalizer is based on

In Figure 8 , a comparison between the ZF, MMSE, and ML equalizer performances in terms of BER as a function of energy per bit

### 4.5. Other equalization techniques

The main linear equalization drawback is that the equalizer filter enhances the noise at the output (increases the noise variance), and additionally, the noise is colored (especially for severely distorted channels). Employing noise prediction technique helps to avoid the described problem. The last short discussion leads to the basic idea about decision feedback equalization (DFE). The DFE structure consists of two filters: a feed forward-filter whose input is the channel output signal and a feedback-filter that feeds back the previous decisions for noise prediction process. This can be achieved by combining linear equalization technique like ZF or MMSE with noise variance prediction (ZF-DFE and MMSE-DFE). A simple block diagram for DFE is presented in Figure 9 .

The maximum *a posteriori* (MAP) equalizer [13] estimates the transmitted symbol *a posteriori* probability

The MAP equalizer can be used as SBS detector with maximum-likelihood sequence estimation (MLSE) for transmission over rapidly time-varying (TV) wireless channel as shown in Ref. [13].

When some signal properties are used for the determination of the instantaneous error which updates the adaptive filter coefficients or weights, the fractionally spaced equalization (FSE) is an effective approach under the absence of the training sequences (blind equalization) [14]. The FSE receives number of input samples equal to

In Ref. [15], a brief discussion about the recent equalization requirements and approaches is presented along with some important related references.

In optical communication field, a novel and efficient multiplier-less finite impulse response filter (FIR) based on chromatic dispersion equalization (CDE) is proposed for coherent receivers [16]. An iterative receiver is designed [17] for joint phase noise estimation, equalization, and decoding in a coded communication system with combined belief propagation, mean field, and expectation propagation (BP-MF-EP). In the frequency domain-based equalization, many important contributions are made recently, for example, in Ref. [18] and for single carrier frequency domain equalization (SC-FDE) in broadband wireless communication systems, a robust design under imperfect channel knowledge is considered based on a statistical channel estimation model where the equalization coefficients are defined under mean square error minimization criterion.

Another work deals with frequency domain equalization for faster-than-Nyquist (FTN) signaling is presented in Ref. [19] for doubly selective channels (DSCs) based on low complexity receivers with variational methods implemented in order to handle the interference of frequency domain symbols instead of using MMSE equalizer that involves high complexity in DSCs.

The frequency domain equalization is also proposed to be employed for broadband power line communications (PLC) as in Ref. [20]. PLC performance can benefit from frequency domain equalization techniques in the context of a cyclic-prefix single carrier modulation schemes. The study in Ref. [20] presents an equalization algorithm based on the properties of complementary sequences (CSs) to reduce the complexity by performing all the operations in the frequency domain without the necessity of noise variance estimator.

## 5. The new nonlinear equalization approach

### 5.1. Channel equalization with DCE

The proposed new nonlinear equalization approach applying dual channel equalization (DCE) is presented in this section. The DCE idea can be implemented and coupled with any standard channel equalizers such as ZF or ML equalizers. Figure 10 shows a simple flow chart for the DCE idea presented as a coupling process between two digital filters. These filters can have the same or different transfer functions and for simplicity of analysis, the similarity case is considered. The topology in Figure 10 is flexible and able to work with various equalization techniques. The squaring device block

### 5.2. ZF equalization with DCE

The system model presented in Section 4.1 is valid in the analysis of ZF-DCE. The signal matrix

where

Taking into consideration Eq. (37), the given form in Eq. (36) can be rewritten as:

where

where

where the squared singular values

where

The mathematical expectation of the noise power given by Eq. (42) is defined as follows:

The unitary matrix lemma states that the multiplication with a unitary matrix will not change the vector norm. Thus, owing to the fact that is

where

The general form used to determine the BER in the case of ZF equalization technique under BPSK modulation can be defined as [10]:

The BER form in Eq. (21) [10] is the solution of the last integral in Eq. (46) and together with Eq. (45) can present the final BER closed expression of ZF-DCE as follows:

As noticed in Eqs. (45) and (47), the derived forms are based on *i*th eigenvalue of the sample covariance matrix

where

### 5.3. ML equalization with DCE

For this case, the DCE flow chart presented in Figure 10 can be considered for ML-DCE design excluding the squaring device. The corresponding equalizer or channel matrix takes the following format:

Based on Eq. (31), the ML-DCE estimation is defined as:

The presented form in Eq. (51) can be simplified by the following operations:

It is obvious that to minimize the form in Eq. (52), the term

The related SNR at the ML-DCE output is determined using

Defining the achieved SNR for ML-DCE, it is possible to calculate the BER using the same form in Eq. (33). Taking into account the sample covariance matrix

The use of two equalization filters and the bandstop filter (the main DCE idea) helps us to construct new equalization matrices and convert linear equalizers such as ZF equalizer to nonlinear one. Additionally, the reference noise forming at the second equalizer output contributes in the definitions of achieved BER and SNR and can be employed to estimate the noise variance (power) at the equalizer input. In this section, two equalizers are introduced, namely, ZF-DCE and ML-DCE. The DCE concept can be implemented with other equalization techniques like MMSE and DFE as well. The proposed DCE performance is evaluated in the following subsection.

### 5.4. Simulation process and results

In this section, *N*= 2 and the number of receive antennas *M*= 2 under Rayleigh fading channel and for BPSK modulation. The overall simulation process for this system employing the conventional and DCE types of equalization techniques are presented in Figure 11 . The evolution and comparison criterion is based on the BER (probability of error) as a function of energy per bit

where

In Figure 12 , a comparison between the conventional ZF equalization technique and the modified ZF using dual channel equalization (DCE) is presented. The ZF-DCE approach demonstrates better performance (lower BER) comparing with the conventional ZF at the same

The ML-DCE and the conventional ML equalizer performances are compared in Figure 13 under the same initial conditions but for different SNR (

## 6. Conclusion remarks

The discussion of this chapter can be ended by making several remarks. The new equalization approach (DCE) shows promising outcomes in terms of improving the equalization performance by reducing the BER (sequence error probability).The obtained results can be simply generalized for other equalization techniques and for frequency domain equalization as well. Additionally, it is easy to prove that the DCE symbol error performance can be better than that of the conventional equalizers. The type of filters used to design the DCE equalizer is not mentioned or discussed but both FIR and IIR filters are possible and reasonable candidates.

Although the design and implementation complexity issues (the computational cost to design the equalizer and to equalize the channel, respectively) for the proposed above equalization structure are not discussed in this chapter, an ostensible comments can be made. The DCE design relies on other equalization approaches and does not exhibit significant overhead complexity. Thus, the complexity increases employing DCE but not with overwhelming degree. Finally, a complete analysis based on practical conditions and for time-invariant (TIV) and time-varying (TV) channel models under specific scenarios is required before addressing the feasibility of the presented DCE approach.

## Acknowledgments

This work is performed by Radio Physics Research Group (RPRG) at Polytechnic University of San Luis Potosi (UPSLP), San Luis Potosi, Mexico.