Open access peer-reviewed chapter

A Continuous-Time Recurrent Neural Network for Joint Equalization and Decoding – Analog Hardware Implementation Aspects

By Mohamad Mostafa, Giuseppe Oliveri, Werner G. Teich and Jürgen Lindner

Submitted: October 16th 2015Reviewed: March 30th 2016Published: October 19th 2016

DOI: 10.5772/63387

Downloaded: 1788

Abstract

Equalization and channel decoding are “traditionally” two cascade processes at the receiver side of a digital transmission. They aim to achieve a reliable and efficient transmission. For high data rates, the energy consumption of their corresponding algorithms is expected to become a limiting factor. For mobile devices with limited battery’s size, the energy consumption, mirrored in the lifetime of the battery, becomes even more crucial. Therefore, an energy-efficient implementation of equalization and decoding algorithms is desirable. The prevailing way is by increasing the energy efficiency of the underlying digital circuits. However, we address here promising alternatives offered by mixed (analog/digital) circuits. We are concerned with modeling joint equalization and decoding as a whole in a continuous-time framework. In doing so, continuous-time recurrent neural networks play an essential role because of their nonlinear characteristic and special suitability for analog very-large-scale integration (VLSI). Based on the proposed model, we show that the superiority of joint equalization and decoding (a well-known fact from the discrete-time case) preserves in analog. Additionally, analog circuit design related aspects such as adaptivity, connectivity and accuracy are discussed and linked to theoretical aspects of recurrent neural networks such as Lyapunov stability and simulated annealing.

Keywords

  • continuous-time recurrent neural networks
  • analog hardware neural networks
  • belief propagation
  • vector equalization
  • joint equalization and decoding

1. Introduction

Energy efficiency has been increasingly attracting more interest due to economical and environmental reasons. Mobile communications sector has currently a share of 0.2% in global carbon emissions. This share is expected to double between 2007 and 2020 due to the ever-increasing demand for wireless devices [1, 2]. The sustained interest in higher data rate transmission is strengthening this impact. While major resources are being invested in increasing the energy efficiency of digital circuits, there is, on the other hand, a growing interest pointing at alternatives to the digital realization [3], including a mixed (analog/digital) approach. In such an approach, specific energy consuming (sub)tasks are implemented in analog instead of a “conventional” digital realization. The analog implementation possesses a high potential to significantly improve the energy efficiency [4] because of the inherent parallel processing of signals that are continuous in both time and amplitude. This has been shown in the field of error correction coding with a focus on decoding of low-density parity-check (LDPC) codes. Our ongoing research on equalization reveals similar results. We do not intend “analog” for linear signal processing with all its disadvantages like component inaccuracies and susceptibility to noise and temperature dependency [5] but for nonlinear processinginstead. The work of Mead [6] and others on Neuromorphic analog very-large-scale integration (VLSI)has shown that “analog signal processing systems can be built that share the robustness of digital systems but outperform digital systems by several orders of magnitude in terms of speed and/or power consumption” [5].

The nonlinearity makes the analog implementation of an algorithm as robust as its digital counterpart [3, 5]. This profits from the match between the needed nonlinear operations for the algorithm and the physical properties of analog devices [7].

The capability of artificial neural networks (in the following neural networks) to successfully solve many scientific and engineering tasks has been shown oftentimes. Moreover, mapping algorithms to neural network structures can simplify the circuit design because of the regular (and repetitive) structure of neural networks and their limited number of well-definedarithmetic operations. Digital implementations can be considered precise (reproducibility of results under similar circumstances) but accurate (closeness of a result to the “true” value) onlyto the extent to which they have enough digits to represent [8]. This means, accuracy in digital implementations is achieved at the cost of efficiency (e.g., relatively larger chip area and more power consumption) [9]. An analog implementation is usually efficient in terms of chip area and processing speed [9], however, at the price of an inherent lack of the reproducibility of results [8] (because of a limited accuracy of the network components as an example [9]). However, by exploiting the distributed nature of neural structures the precision of the analog implementation can be improved despite inaccurate components and subsystems [8]

For a clear distinction between accuracyand precisionwhen used in hardware implementation context, we refer to [8].

. In other words, it is the distributed massively parallel nonlinear collective behavior of an analog implementation (of neural networks) which offers the possibility to make it as robust as its digital counterpart but more energy efficient

Energy efficiency is defined later as appropriate.

(additionally to smaller chip area). Particularly for recurrentneural networks (the class we focus on when considered as nonlinear dynamical systems), the robustness can be additionally achieved by exploiting “attracting” equilibrium points. In the light of this discussion, we map in this chapter a joint equalization and decoding algorithm into a novel continuous-timerecurrent neural network structure. This class of neural networks has been attracting a lot of interest because of their widespread applications. They can be either trained for system identification [10], or they can be considered as dynamical systems (dynamical solver). In the latter case, there is no needfor a computationally complex and time-consuming training phase. This relies on the ability of these networks (under specific conditions) to be Lyapunov stable.

Equalization and channel decoding (together, in the following detection) are processes at the receiver side of a digital transmission. They aim to provide a reliable and efficient transmission. Equalization is needed to cope with the interference caused by multipath propagation, multiusers, multisubchannels, multiantennas and combinations thereof [11]. Channel (de)coding is applied for further improving the power efficiency. Equalization and decoding are nonlinear discrete optimization problems. The optimum solutions, in general, are computationally very demanding. Therefore, suboptimum solutions are applied, often soft-valued iterative schemes because of their good complexity-performance trade-off.

For high data rates, the energy consumption of equalization and decoding algorithms is expected to become a limiting factor. The need for floating-point computation and the nonlinear and iterative nature of (some of) these algorithms revive the option of an analog electronic implementation [12, 13], embedded in an essentially digital receiver. This option has been strengthened since the emergence of the “soft-valued” computation in this context [4] since soft-values are a natural property of analog signals. In contrast to analog decoding, analog equalization did not attract that amount of attention.

Furthermore, jointequalization and decoding (a technique where equalizer and decoder exchange their localavailable knowledge) further improves the efficiency of the transmission as an example in terms of lower bit error rates, however, at the cost of more computational complexity [14]. Most of the work related to joint equalization and decoding is limited to the discrete-time realization. One of the very few contributions focusing on continuous-time joint equalization and decoding is given in [13]. The consideration in [13] is not “neural networks-based”. Stability and convergence are observed but not “deeply” considered.

We introduce in this chapter a novel continuous-time joint equalization and decoding structure. For this purpose, continuous-time single-layer recurrent neural networks play an essential role because of their nonlinear and recursive characteristic, special suitability for analog VLSI and since they serve as promising computational models for analog hardware implementation [15]. Both, equalizer and decoder are modeled as continuous-time recurrent neural networks. An additional proper feedback between equalizer and decoder is established for joint equalization and decoding. We also review individually, both continuous-time equalization and continuous-time decoding based on recurrent neural network structures. No training is needed since the recurrent neural network is serving as a dynamical solver or a computational model [15, 16]. This means, transmission properties are used to define the recurrent neural network (number of neurons, weight coefficients, activation functions, etc.) such that no training is needed. In addition, we highlight challenges emerging from the analog hardware implementation such as adaptivity, connectivity and accuracy. We also introduce our developed circuit for analog equalization based on continuous-time recurrent neural networks [3]. Characteristic properties of recurrent neural networks such as stability and convergence are addressed too. Based on the introduced model, we show by simulations that the superiority of joint equalization and decoding can be preserved in the analog “domain”.

The main motivation for performing joint equalization and decoding in analog instead of using conventional digital circuits is to improve the energy efficiency and to minimize the area consumption in the VLSI chips [17]. The proposed continuous-time recurrent neural network serves as a promising computational model for analog hardware implementation.

The remainder of this chapter is organized as follows: In Section 2, we describe the block transmission model. Sections 3 and 4 are dedicated to the equalization process, the application of continuous-time recurrent neural networks and the analog circuit design and its corresponding performance and energy efficiency. Sections 5 and 6 are devoted to the channel decoding and the application of continuous-time recurrent neural networks for belief propagation (a decoding algorithm for LDPC codes). For both equalization and decoding cases, analog hardware design aspects and challenges and the behavior of the continuous-time recurrent neural network as a dynamical system are discussed. The continuous-time joint equalization and decoding based on recurrent neural networks is presented in Sections 7 and 8. Simulation results are shown in Section 9. We finish this chapter with a conclusion in Section 10.

Throughout this chapter, bold small and bold capital letters designate vectors (or finite discrete sets) and matrices, respectively.

Except for L, Land Lchwhich are vectors.

All nonbold letters are scalars. diagm{B} returns the matrix Bwhere the nondiagonal elements are set to zeros. diagυ{b}returns a matrix where the vector bis put on the diagonal. 0Nis the all-zero vector of length N. 0, 1and Irepresent the all-zero, all-one and the identity matrix of suitable size, respectively. We consider column vectors. ()Hrepresents the conjugate transpose of a vector or a matrix, whereas ()Trepresents the transpose. zr=(z), zi=(z)returns the real and imaginary part of the complex-valued argument z=zr+ιzi, respectively. ι=1. tand lare designated to the continuous-time variable and the discrete-time index, respectively.

Advertisement

2. Block transmission model

The block transmission model for linear modulation schemes is shown in Figure 1. For details, see [18]:

Figure 1.

Block transmission model for linear modulation schemes.SRC(SNK) represents the digital source (sink). DET is the detector. COD performs the encoding process (adding redundancy).Mmaps encoded bits to complex-valued symbols.Ris the block transmit matrix.

  • SRC(SNK) represents the digital source (sink). SRCrepeatedly generates successive streams of kbits, i.e., q1, q2, ⋯ ,qM.

  • q(q^){0,1}kis the vector of source (detected) bits of length k.

  • qc{0,1}nis the vector of encoded source bits of length n>k. For an uncodedtransmission qc=q(and thus k=n).

  • COD performs a bijective map from qto qcwhere n>k(adding redundancy). We consider in this chapter binary LDPC codes. Only 2kcombinations of nbits out of overall 2ncombinations are used. The set of the 2kcombinations represent the code book C. rc=k/nis the code rate.

  • xψNis the transmit vector of length N.

  • Nis the block size. Successive transmit vectors are separated by a guard time to avoid interference between different blocks. Thus, Figure 1 describes the transmission for a single block and stays valid for the next block (possibly with a different R).

  • ψ={ψ1,ψ2,,ψ2m}, m/{0}is the symbol alphabet. There exist 2mNpossible transmit vectors. The set of all possible transmit vectors is χ. The mapping from qcto xis performed by M. Each symbol ψrepresents mbits. A special class of symbol alphabets are the so-called separablesymbol alphabet ψ(s)[19, 20].

  • x˜is the receive vector of length N. In general x˜N.

  • We distinguish:

    1. – For an uncoded transmission M×k=m×N.

    2. – For a coded transmission and N<n/m: One codeword lasts over many transmit blocks.

    3. – For a coded transmission and N=n/m: One codeword lasts exactly over a single transmit block.

    4. – For a coded transmission and N=M×n/m: Mcodewords are contained in a single transmit block.

  • R={rij:i,j{1,2,,N}}is the block transmit matrix of size N×N. Ris hermitian and positive semidefinite. The block transmit matrix Rcontains the whole knowledge about the transmission scheme (transmit and receive filters) and the physical propagation channel between transmitter(s) and receiver(s) [18].

  • n˜is a sample function of an additive Gaussian noise vector process of length Nwith zero mean and covariance matrix Φn˜n˜=N02Rwhere N02is the double-sided noise power spectral density.

  • DET is the detector including equalization and decoding.

The model in Figure 1 is a general model and fits to different transmission schemes like orthogonal frequency division multiplexing (OFDM), code division multiple access (CDMA), multicarrier CDMA (MC-CDMA) and multiple-input multiple-output (MIMO). The relation with the original continuous-time (physical) model can be found in [11, 18]. The model in Figure 1 can be described mathematically as follows [11]:

x˜=Rx+n˜.E1

By decomposing Rinto a diagonal part Rd= diagm{R} and a nondiagonal part R\d= RRd, Eq. (1) can be rewritten as:

x˜=Rdxsignal+R\dxinterference+n˜additive noise.E2

For the j-th element of the receive vector j{1,2,,N}Eq. (2) can be expressed as

x˜j=rjjxj+m=1mjNrjmxm+n˜j.E3

We notice from Eqs. (2), (3) that the nondiagonal elements of Rdescribe the interference between the elements of the transmit vector at the receiver side. For interference-free transmission R\d= 0. For an interference-free transmission over an additive white Gaussian noise (AWGN) channel R= I.

Figure 2 shows the channel matrix for a MIMO transmission scheme for different number of transmit/receive antennas. Figure 3 shows the channel matrix for OFDM with/without spreading. Figure 4 shows the channel matrix for MIMO-OFDM. In Figures 24, the darker the elements, the larger the absolute values of the entries of the corresponding matrix R, and hence larger the interference [21].

Figure 2.

Visualization of the channel matrix for a MIMO transmission scheme with eight transmit antennas and different receive antennas.

Figure 3.

Visualization of the channel matrix for OFDM with 16 subcarriers and spreading over four subcarriers with/without interleaving.

Figure 4.

Visualization of the channel matrix for a MIMO-OFDM transmission scheme with eight subcarriers and three transmit antennas.

Remark 1.For a clear distinction between channel matrixand block transmit matrix, we refer to [11, 18]. Generally speaking, the block transmit matrix Ris a block diagonal matrix of “many” channel matrices.

The detector DET in Figure 1 has to deliver a vector q^with a minimum bit error rate compared to q(conditional to the available computational power) given that COD, Mand Rare known at the receiver side. The optimum detection (maximum likelihood detection) for realistic cases is often infeasible. Therefore, suboptimum schemes are used, mainly based on separating the detection into an equalization EQ (to cope with interference caused by R\d) and a decoding DEC (to utilize the redundancy added by COD). In this case, we distinguish between separate and joint equalization and decoding, cf. Figure 5. The superiority of the latter one is widely accepted: The separate equalization and decoding as in Figure 5(a) in general leads to a performance loss since the equalizer does not utilize the knowledge available at the decoder [14]. Each of the components DET, EQ and DEC can be seen as a pattern classifier. By separating the detection into equalization and decoding, an optimum detection in general cannot be achieved anymore (even if optimum equalization and optimum decoding are individually applied). Nevertheless, this is a common practice.

Figure 5.

Detection: EQ is the equalizer, DEC is the decoder, DECI is a hard decision function. Notice the feedback from the decoder to the equalizer in (b), i.e., the turbo principle.

DECI in Figure 5 is a hard decision function. For a coded transmission, DECI is a unit step function. For an uncoded transmission, COD and DEC are removed from Figure 1 and Figure 5, respectively. DECI in this case is a stepwise function depending on the symbol alphabet ψwhich maps the (in general complex-valued) elements of the equalized vector xto the vector of detected symbols x^ψNcf. Figure 8. The map from x^to q^is then straightforward. In summary

  • For an uncoded transmission DECI: NψN.

  • For a coded transmission DECI: k{0,1}k.

3. Vector equalization

For an uncoded transmission, the detection DET reduces to a vector equalization EQ as shown in Figure 6.

Figure 6.

Uncoded block transmission model. Neither encoding at the transmitter nor decoding at the receiver. The detection reduces to a vector equalization EQ.

The optimum vector equalization rule (the maximum likelihood one) is based on the minimum Mahalanobis distance and is given as [21]

x^ML=argminξχ{12ξHRξ{ξHx˜}}.E4

For each receive vector x˜, the optimum vector equalizer calculates the Mahalanobis distance Eq. (4) to allpossible transmit vectors χof cardinality 2mNand decides in favor of that possible transmit vector x^MLwith the minimum Mahalanobis distance to the receive vector x˜, i.e., exhaustive search is required in general. This can be performed for small 2mNwhich is usually notthe case in practice. Therefore, suboptimum equalization schemes are applied, which trade-off performance against complexity.

4. Continuous-time single-layer recurrent neural networks for vector equalization

The dynamical behavior of continuous-time single-layer recurrent neural networks of dimension N′, abbreviated in the following by RNN

The abbreviation RNN in this chapter inherently includes the continuous-time and the single-layer properties.

, is given by the state-space equations [22]:

ϒedu(t)dt=u(t)+Wv(t)+W0e,v(t)=φ(u(t))=[φ1(u1(t)),φ2(u2(t)),,φN(uN(t))]T.E5

In Eq. (5), ϒeis a diagonal and positive definite matrix of size N′ × N′. v(t) is the output, u(t) is the inner state, eis the external input. v,u,eN.φj(·) : j∈ {1, 2, …, N′} is the j-th activation function. W={wjj:j,j{1,2,,N}}N×N, W0=diagv{[w10,w20,,wN0]T}N×Nare the weight matrices. The real-valued RNN (all variables and functions in Eq. (5) are real-valued) is shown in Figure 7, which is known as “additive model” or “resistance-capacitance model” [23]. In this case, wjj=RjRjjis the weight coefficient between the output of the j′-th neuron and the input of the j-th neuron, wj0=RjRj0is the weight coefficient of the j-th external input. We also notice that the feedback Wvin Eq. (5) and Figure 7 is a linear function of the output v. Moreover, ϒecan be given in this case as ϒe=diagv{[R1C1,R2C2,,RNCN]T}.

Figure 7.

Continuous-time single-layer real-valued recurrent neural network.v(t) is the output,u(t) is the inner state,eis the external input andφ(⋅) is the activation function. This model is known as “additive model” or “resistance-capacitance model” [23].

As a nonlinear dynamical system, the stability of the RNN is of primary interest [16]. This has been proven under specific conditions by Lyapunov’s stability theory in [24] for real-valued RNN and in [22, 25] for complex-valued ones, among others. The RNN in Eq. (5) represents a general purpose structure. Based on N′, φ, W, W0 a wide range of optimization problems can be solved. First and most well-investigated applications of the RNN include the content addressable memory [24, 26], analog-to-digital converter (ADC) [27] and the traveling salesman problem [28]. In all these cases, no training is needed since the RNN is acting as a dynamical solver. This feature is desirable in many engineering fields like signal processing, communications, automatic control, etc., and has first been exploited by Hopfield in his pioneering work [24, 29], where information has been stored in a dynamically stable RNN. We focus in the following on the vector equalization.

Remark 2.The dimension of a real-valued RNN is the same as the number of neurons.

Remark 3.Two real-valued RNNs each of N′ neurons are required to represent one complex-valued RNN (with dimension N′). This is possible by separating Eq. (5) into real and imaginary parts. However, this doubles in general the number of connections per neuron (and hence the number of multiplications) because of the required connections (represented by Wi) between the two real-valued RNNs as it can be seen from the following equation:

ϒeddt[ur(t)ui(t)]=[ur(t)ui(t)]+[WrWiWiWr][vr(t)vi(t)]+[W000W0][erei].E6

ϒein this case is a diagonal positive definite matrix of size 2 · N′ × 2 · N′ and

u(t)=ur(t)+ιui(t),e=er+ιeiv(t)=vr(t)+ιvi(t),W=Wr+ιWi.

A. Vector equalization based on RNN

The usage of the RNN for vector equalization became known for multiuser interference cancellation in CDMA environments [30, 31]. However, this was limited to the binary phase-shift keying (BPSK) symbol alphabet ψ= {−1, +1}. This has been generalized to complex-valued symbol alphabets in [21] by combining the results of references [20], [22], [32]

For discrete-timesingle-layer recurrent neural networks for vector equalization, we refer to references [19, 33].

. Based thereon, it has been proven that the RNN ends in a localminimum of Eq. (4) if the following relations are fulfilled [21], cf. Eqs. (1), (2), (5) and Figures 6 and 7.

e=x˜v=xN'=NW0=Rd1W=IRd1Rφ()=θ(opt)()E7
and therefore x^=DECI(v). Figure 8

shows an example of an eight quadrature amplitude modulation (8 QAM) symbol alphabet and its corresponding DECI function. The relations in Eq. (7) are obtained by the comparison between the maximum likelihood function of the vector equalization and the Lyapunov function of the RNN.

Figure 8.

An example of an 8 QAM symbol alphabet and its corresponding DECI function. Each element of the symbol alphabet (marked with ×) has its own “decisions region” visualized by different colors. The function DECI delivers that element of the symbol alphabet, where the input argument lies in its corresponding decision region.

The dynamical behavior of the vector equalization based on RNN can be given as, cf. Eqs. (1), (5), (7)

ϒedu(t)dt=u(t)+x(t)+Rd1R[xx(t)]+Rd1n˜,x(t)=θ(opt)(u(t))=[θ1(opt)(u1(t)),θ2(opt)(u2(t)),,θN(opt)(uN(t))]T.E8

The locally asymptotical stability of Eq. (8) based on Lyapunov functions has been proved in [21] (based on [22]) for separable symbol alphabets ψ(s). When Eq. (8) reaches an equilibrium point uep, i.e., du(t)dt=0Nu=uep, Eq. (8) can be rewritten as

uep=xep+Rd1R[xxep]+Rd1n˜.E9

If additionally, a correct equalization is achieved, i.e., xep=x, the inner state is

uep=x+Rd1n˜ne.E10

Thus, the RNN as vector equalizer, Eq. (8) acts as “analog dynamical solver” and there is no need for a training. The covariance matrix of neis Φnene=N02Rd1RRd1. We define

Σne=diagv{[σ12,σ22,,σN2]T}=diagm{Φnene}=N02Rd1.E11

In Eq. (7), θ(opt) (·) is the optimum activation function and depends on the symbol alphabet ψ. For BPSK (a real-valued case)

θ(opt)(u)=tanh(uσ2).E12
where σ2 is given in Eq. (11).

Remark 4.For separable symbol alphabets,
ψ=ψ(s)θ(opt)(u=ur+ιui)=θr(opt)(ur)+ιθi(opt)(ui)[19].

B. Analog hardware implementation aspects: equalization

The analog signal processing as a matter of topical importance for modern receiver architectures was recognized in [34], where an analog vector equalizer—designed in BiCMOS technology—was considered as a promising application for the analog processing of baseband signals. The equalizer accepts sampled vector symbols in analog form with an advantage that the equalizer does not require an ADC at the input interface. At very high data rates, the exclusion of an ADC softens the trade-off between chip area requirement and overall power consumption. We discuss in the following section the main features/challenges of the analog implementation of the vector equalizer based on RNN.

Structure: An RNN of dimension N′ (in general 2 ·N′ neurons) is capable to act as a vector equalizer as long as the block size at the transmitter side N(over all possible symbol alphabets, coding schemes and block sizes) is as maximum as N′, i.e., NN′.

Activation function: The definition of the optimum activation function θ(opt)(·) is not general, but depends on the symbol alphabet under consideration. Different symbol alphabets need different activation functions. However, we have proven in [20] that for square QAM symbol alphabets—the most relevant ones in practice—θ(opt) (·) can be approximated as a sum of a limited number of shifted and weighted hyperbolic tangent functions. Square QAM symbol alphabets are separable ones, cf. Remark 4. The analog implementation of the hyperbolic tangent well befits the large-signal transfer function of transconductance stages based on bipolar differential amplifiers [3, 34].

Adaptivity: A vector equalizer must be capable to adapt to different and time-variant interference levels. The adaptivity is regulated by the measurement of the block transmit matrix R, a task performed by a “channel estimation unit” (CEU). The weight matrices Wand W0 are then computed as in Eq. (7) and forwarded to the RNN (Figure 9). Thus, the weight matrices Wand W0 are notthe outcome of any training algorithm but related directly to R, cf. Eq. (7). This represents a typical example for the mixed-signal integrated circuit, where the weight coefficients are (obtained and) stored digitally, converted into analog values, later used as weight coefficients for the analog RNN [8].

Figure 9.

Uncoded block transmission model. The detection reduces to a vector equalization EQ. The channel estimation unit (CEU) estimates the block transmit matrixR.

For the j-th neuron in the additive model Figure 7, the ratio between two resistors Rjand Rjj′(Rjand Rj0) is used to configure each weight coefficient wjj′(wj0). According to the additive model, Rjj′and Rj0 can assume both positive and negative values, and the absolute value theoretically extends from Rjto infinite (for wjj′∈ [−1, +1]). This puts serious limitations to the direct implementation of the model. In [3], we showed how this difficulty can be overcome by using a Gilbert cell as a four-quadrant analog multiplier. A Gilbert cell [35] is composed of two pairs of differential amplifiers with cross-coupled collectors, and is controlled by a differential voltage input Gjiapplied at the base gate of the transistors. When biased with a differential tail current Iji=Iji+Iji, the differential output current Iji,w=Iji,w+Iji,wis a fraction wof the tail current Iji, as a function of the input voltage Gji:

Iji,w=Iji,w+Iji,w=fGc(Iji=Iji+Iji,Gji)=wIji[Iji,+Iji].E13

Accuracy: Locally asymptotical Lyapunov stability can be guaranteed for the RNN in Eqs. (5), (8) if, among others, the hermitian property is verified for the weight matrix W(the symmetric property in the real-valued case). Inaccuracies in the weights’ representation may jeopardize the Lyapunov stability and impact the performance of the vector equalizer. The first cause of weights’ inaccuracy may arise from the limited accuracy of the analog design in terms of components’ parasitics, devices’ mismatch, process variation, just to name a few. Those inaccuracies (if modest) are expected to slightly degrade the performance without causing a catastrophic failure, thanks to the high nonlinearity of the equalization algorithm. Moreover, it has been shown in [8], [36] that in some cases, they produce beneficial effects: These imperfections incorporate some kind of simulated annealing which enables escaping local minima by allowing occasionally “uphill steps” since the Lyapunov stable RNN is a gradient-like system. This feature is emulated in discrete-time by stochasticHopfield networks [23]. Non-precision of the weights may also arise from an insufficient resolution of the digital-to-analog converter (DAC) (Figure 9). On the other hand, an overzealous DAC design increases the chip area, the power consumption and adds complexity to the interface between the analog vector equalizer and the digital CEU. In this case, a conservative approach suggests to use a DAC with enough resolution to match the precision used by the CEU.

Interneuron connectivity and reconfigurability: Scaling the architecture of an analog VLSI design is not straightforward. A vector equalizer based on recurrent neural networks is composed by the repetition of equal sub-systems, i.e., the neurons. Using a bottom-up approach, the first step to scale the system involves the redesign of the single neuron in order to handle more feedback inputs. In a successive step, the neurons are connected together and a system-level simulation is performed to check the functionality of the system. However, several design choices must be made during the process and it is not guaranteed that the optimum architecture for a certain number of neurons is still the best choice when the number of neurons changes. For large N, the block transmit matrix R, defining the weight matrix W, is usually sparse. If a maximum number of nonzero elements over the rows of Ris assumed, the requirement for a full connectivity between the neurons in Figure 7 can be relaxed, and only a maximum number of connections per neuron will be necessary. In this case, however, in addition to the “adaptivity”, the RNN must be reconfigured according to the position of the nonzero elements in R. The hardware simplification given by the partial connectivity may be counterbalanced by the necessity of a further routing (e.g., multiplexing/demultiplexing) of the feedback. For special cases, where the block transmit matrix can be reordered around the diagonal, more independent RNNs can be simply used in parallel. In Figures 3(b) and 3(c), four independent RNNs, each of dimension four, can be used in parallel. Additionally, for specific transmission schemes such as MIMO-OFDM in Figure 4, the connectivity can be assumed limited (number of transmit antennas minus one) and fixed (crosstalk only between samesubcarriers, when used simultaneously on different transmit antennas).

Example 1.In Figure 4, eight RNNs (number of subcarriers) each of dimension of three (number of transmit antennas) can be used in parallel. Each neuron has two feedback inputs.

C. Circuit design

We review here the main features of the analog circuit design of an RNN as vector equalizer working with the BPSK symbol alphabet and composed of four neurons. Detailed explanation can be found in reference [3]. The RNN is realized in IHP 0.25 μm SiGe BiCMOS technology (SG25H3). A simplified schematic of a neuron is shown in Figure 10. Schematics of gray boxes are presented in Figure 11.

Figure 10.

A simplified schematic of a single neuron as a part of a (four neurons) RNN analog vector equalizer.ujis the inner state,ejis the external input andGjiis used for adapting the weight coefficientwjifrom the output of thei-th neuron to the input of thej-th neuron. The circuit is fully differential [3].

Figure 11.

Details of the circuit building blocks. Gilbert cell used as a four-quadrant analog multiplier, buffer stages, BJT differential pairs for the generation of the hyperbolic tangent function and a metal-oxide-semiconductor field-effect transistor (MOSFET) switch used as a sequencer [3].

The dynamical behavior of the circuit in Figures 10 and 11 is described as [3]

ϒdu(t)dt=u(t)+Wv(t)+W0e,RItN1tanh(u(t)2Vt)=v(t),τI=ϒ.E14

which is equivalent to Eq. (5). τ= R· Cis the time constant of the circuit. Ris shown in Figure 10 and Cis a fictitious capacitance between the nodes and uj+and uj.Vtis the thermal voltage and Itis the tail current in Figure 11. The circuit is fully differential and the differential currents and voltages are denoted as, cf. Figures 10 and 11:

Iji=Iji+IjiIj=Ij+IjIo=Io+Io,Iji,w=Iji,w+Iji,wuj=uj+ujej=ej+ejE15

(1) Performance:Simulation results based on the above described analog RNN are shown in Figure 12. The interference is described by the channel matrix Rtest.

Figure 12.

BER vs.Eb/N0for the analog RNN vector equalizer. Evolution time equals10τ. BPSK symbol alphabet and channel matrixRtest.

Rtest=[10.240.340.570.2410.320.290.340.3210.250.570.290.251]

The black dashed line shows the bit error rate (BER) for a BPSK symbol alphabet in an AWGN channel (an interference-free channel). Performance achieved by the maximum likelihood algorithm in Eq. (4) is included as a solid black line. The performance of the analog RNN vector equalizer

Analog RNN vector equalizer refers to the described analog hardware-implemented RNN for vector equalization.

is presented in a solid red line with square markers. Compared to the optimum algorithm, the signal-to-noise ratio (SNR) loss for the analog RNN vector equalizer can be quantified in approximately 1.7 dB at a BER of 10−4. This loss in SNR emphasizes the suboptimality of the RNN as vector equalizer and depends on the channel matrix. Figure 13 shows an example of a transient simulation for the analog RNN vector equalizer. The time constant is approximately τ= 40 ps. The SNR ratio is set to 2 dB and a series of three receive vectors are equalized in sequence. Because of the channel matrix and noise, the sampled vectors at the input of the equalizer x˜present different signs and values, compared to the sent vectors x(shown in square brackets). The equalization of each receive vector lasts 10τ. First half of this interval (evolution time) is used to reach a stable state, while the second half of the interval (reset time) is used to return to a predefined inner state (all-zero state) before the equalization of a new vector starts. At the end of the evolution time, a decision is made based on the sign of the output vector (the decision function DECI for BPSK is a sign function). In our example, a comparison between the sent and the recovered bits shows an error of one bit out of twelve, equivalent to a BER112, a result in line with the BER shown in Figure 12.

Figure 13.

An example of a transient simulation for the analog RNN vector equalizer.

(a) Inputsx˜(b) Outputs: soft decisionsxand hard decisionsx.

Remark 5.The evolution and reset times are the two limiting factors for the maximum throughput of the analog RNN vector equalizer. However, they cannot be unlimitedly minimized since the RNN needs a minimum evolution time to reach an equilibrium point representing a local minimum of the Lyapunov function, i.e., a local minimum of Eq. (4).

(2) Energy efficiency: The energy efficiency of a hardware “architecture” is the ratio between the power requirement (Watt) of the architecture and its achievement in a given time period. In our case, the throughput of the equalizer represents the achievement. Combining the value of τand the power consumption, the abovementioned analog vector equalizer is expected to win the competition versus common digital signal processing, thanks to three to four orders of magnitude better energy efficiency [3].

5. Channel coding

Channel coding (including encoding at the transmitter side COD and decoding at the receiver side DEC) aims to enable an error-free transmission over noisy channels with maximum possible transmit rate. This is done by adding redundancy (extra bits) at the transmitter side, i.e., the bijective map from qto qc(Figure 14,) such that the codewords qcare sufficiently distinguishable at the receiver side even if the noisy channel corrupts some bits during the transmission. Figure 14 shows a coded transmission over an AWGN channel.

For every received codeword, the optimum decoding (the maximum likelihood one) needs to calculate the distance between the received codeword and all possible codewords C, which makes it infeasible for realistic cases (except for convolutional codes which are not considered here). We focus on binary LDPC codes and their corresponding suboptimum decoding algorithm: the belief propagationwith BPSK symbol alphabet. LDPC codes [37] belong to the class of binary linear block codes and have been shown to achieve an error rate very close to the Shannon limit (a performance lower bound) for the AWGN channel and have been implemented in many practical systems such as the satellite digital video broadcast (DVB-S2) [38]. A binary linear block code is characterized by a binary parity check matrix Hof size (nk) × nfor n> k.

Figure 14.

Coded transmission over an BER channel.

Advertisement

6. Continuous-time single-layer high-order recurrent neural networks for belief propagation

One of the largest drawbacks of RNNs is their quadratic Lyapunov function [39]. Optimization problems associated with cost functions of higher degree cannot be solved “satisfactorily” by RNNs. Increasing the order of the Lyapunov function leads to a nonlinear feedback in the network. In doing so, we obtain the single-layer high-order recurrent neural network, named differently in literature, depending on the nonlinear feedback [3942].

Remark 6.High-order recurrent neural networks are in the literature exclusively real-valued.

Figure 15 shows the continuous-time single-layer high-order recurrent neural network, abbreviated in the following by HORNN

The abbreviation HORNN in this chapter inherently includes the continuous-time, single-layer and real-valued properties.

.

The dynamical behavior is given by

ϒdduˇ(t)dt=uˇ(t)+Wˇfˇ(vˇ(t))+Wˇ0eˇ,vˇ(t)=φˇ(uˇ(t))=[φˇ1(uˇ1(t)),φ2(uˇ2(t)),,φˇn(uˇnˇ(t))]T,ϒd=diagv{[Rˇ1Cˇ1,Rˇ2Cˇ2,,RˇnˇCˇnˇ]T}.E16

Figure 15.

Continuous-time single-layer real-valued high-order recurrent neural network.vˇ(t)is the output,uˇ(t)is the inner state,eˇis the external input andφˇ()is the activation function.fˇ(vˇ)is a real-valued continuously differentiable vector function withfˇ(0nˇ)=0nˇ[21].

The parameters in Eq. (16) can be linked to Figure 15 in the same way as Eq. (5) linked to Figure 7. fˇ(vˇ)is a real-valued continuously differentiable vector function. In addition, fˇ(0nˇ)=0nˇ. It is worth mentioning that the term “high-order” in this case refers to the interconnections between the neurons rather than the degree of the differential equation describing the dynamics. As for RNNs, this is still of first order, cf. Eq. (16).

Remark 7.In the special case fˇ(vˇ)=vˇ, the HORNN reduces to the (real-valued) RNN.

In order to apply HORNNs to solve optimization tasks, their stability has to be investigated. A property without which the behavior of dynamical systems is often suspected [39]. This was the topic of many publications [3942]. A common denominator of the locally asymptotical stability proof of the HORNN based on Lyapunov functions is

  • φˇ()is continuously differentiable and a strictly increasing function.

  • The right side of the first line of Eq. (16) can be rewritten as a gradient of a scalar function.

Figure 16.

Tanner graph of the systematic Hamming coden= 7 andk= 4.

A. Belief propagation based on HORNN

Originally proposed by Gallager [37], belief propagation is a suboptimum graph-based decoding algorithm for LDPC codes. The corresponding graph is bipartite (nparity nodes and nkcheck nodes) and known as Tanner graph [43]. This is shown in Figure 16 for the Hamming code with the parity check matrix HHammingEq. (17) where n= 7, k= 4. The belief propagation algorithm iteratively exchanges “messages” between parity and check nodes.

HHamming=[011110111101|100010001]E17

For every binary linear block code characterized by the binary parity check matrix Hof size (nk) × nfor n> k, three binarymatrices Pnh×nh, Snh×nhand Bnh×ncan be uniquely defined [44, 45] such that Eq. (16) and Figure 15 perform continuous-time belief propagation if the following relations are fulfilled:

uˇ=L,E18a
eˇ=BLch,E18b
φˇ()=tanh(2),E18c
vˇ=φˇ(L),E18d
Wˇ=P,E18e
Wˇ0=Inh×nh,E18f
fˇj=2atanh{jpos[S(j,:)=1]vˇj}forj,j{1,2,,nh},E18g
nˇ=nh.E18h

In Eq. (18)

L, Land Lchare vectors.

,
  • kis the length of the information word (qin Figures 1 and 14).

  • nis the length of the codeword (qcin Figures 1 and 14).

  • nh=1(1×(nk))TH1(n×1)[k+1,n(nk)]is the number of nonzero elements in H.

  • Lch,(n×1)is the vector of intrinsic log-likelihood ratio (LLR), which depends on the transition probability of the channel. For qc,j(the j-th element of qcfor j{1,2,,n}) it is given as

    Lch,j=ln px˙j=x˜j | qc,j=0)p(x˙j=x˜j | qc,j=1).E19

In the last relation, x˙jis the variable of the conditioned probability density function p(x˙j|qc,j). ln(⋅) is the natural logarithm. For an AWGN channel, N(0,σn2):Lch,j=x˜j2σn2.

  • L(nh×1)is the “message” sent from the variable nodes to the check nodes.

  • fˇ(nh×1)is the “message” sent from check nodes to variable nodes.

  • I(nh×nh)is an identity matrix of size nh×nh.

  • pos[S(j,:)=1]delivers the positions of the nonzero elements in the j-th row of the matrix S.

  • The dynamical behavior of belief propagation can be described based on Eqs. (16), (18) and Figures 14, 15, and 16 [45]

    ϒddL(t)dt=L(t)+Pfˇ(vˇ(t))+BLch,vˇ(t)=tanh(L(t)2),L(t)=BTfˇ(vˇ(t))+Lch.E20

    L(t)is the soft-output of the decoding algorithm, cf. Figures 5 and 14. The discrete-time description is given as [44]

    L[l+1]=Pfˇ(vˇ[l])+BLch,vˇ[l]=tanh(L[l]2),L[l]=BTfˇ(vˇ[l])+Lch.E21

    B. Dynamical behavior of belief propagation

    In a series of papers, Hemati et. al.[12, 17, 4649] also modeled the dynamics of analog belief propagation as a set of first-order nonlinear differential equations Eq. (20). This was motivated from a circuit design aspect, where ϒd(the same is valid for ϒe) can be seen as a bandwidth limitation of the analog circuit, realized taking advantage of the low-pass filter behavior of transmission lines Figure 17. We have shown in [45] that the model in Figure 17 also has important dynamical properties when compared with the discrete-time belief propagation Eq. (21) [44]. Particularly, the equilibrium points of the continuous-time belief propagation of Eq. (20) coincide with the fixed points of the discrete-time belief propagation of Eq. (21). This has been proved in [45]. In both cases

    Lep=Pfˇ(tanh(Lep2))+BLch.E22

    The absolute stability of belief propagation Eqs. (20), (21) was proven for repetition codes (one of the simplest binary linear block codes) in [44, 45]. In this case

    Lep=1(n×n)Lch.E23

    Figure 17.

    A simple model for analog decoding as presented in [46].

    Far away from repetition codes, it has been noticed that iterative decoding algorithms (belief propagation is one of them) exhibit depending on the SNR a wide range of phenomena associated with nonlinear dynamical systems such as existence of multiple fixed points, oscillatory behavior, bifurcation, chaos and transit chaos [50]. Equilibrium points are reached at “relatively” high SNR. The analysis in reference [50] is limited to the discrete-time case.

    Remark 8.The HORNN in Figure 15, Eqs. (18), (20) for belief propagation acts as a computational model.

    C. Analog hardware implementation aspects: decoding

    Many analog hardware implementation aspects have been already mentioned in Section 4-B. We mention here only additional aspects exclusively related to the analog belief propagation based on HORNN.

    Structure: In practice, different coding schemes (different parity check matrices H) with various (k, n) constellations are applied to modify the code rate rc=k/ndepending on the channel state. The HORNN in Figure 15 is capable to act as a continuous-time belief propagation (decoder) as long as the number of neurons nˇin Figure 15 equals (or is larger than) the maximum number of nonzero elements over all parity check matrices and all (k,n) constellations, i.e., nˇmaxHnh.

    Adaptivity: No training is needed. Wˇ0and Wˇare directly related to the parity check matrix H. In contrast to the analog RNN vector equalizer, the weight coefficients are binary, i.e., the weight matrices Wˇ0and Wˇdefine a feedback to be either existent or not. In such a case for Figure 15, Rˇjj, Rˇj0{Rˇj,}. Moreover, there is no need for high-resolution DAC for the weight coefficients.

    Interneuron connectivity: No full connection is needed since the matrix Pfor LDPC codes is sparse. The number of connections per neuron must equal the maximum number of nonzero elements in Prow-wise over all considered coding schemes and equals maxHP1(nh×1). If this is fulfilled and if interneuron connectivity control is available, the structure in Figure 15 becomes valid for all considered coding scheme.

    Vector function connectivity: For different coding schemes, the number of the arguments vˇjto evaluate the function fˇjchanges, cf. Eq. (18g). The maximum number of the arguments depends on the number of the nonzero elements in Srow-wise and equals maxHS1(nh×1). Thus, implementing the function fˇjaccording to this maximum number enables evaluating the function fˇjfor all considered coding schemes.

    Remark 9.For a specific coding scheme, the interneuron connectivity can be made fixed. The resulted HORNN structure in this case is valid also for all codeword lengths resulted after performing a puncturing of the original code.

    Remark 10.Both, the interneuron connectivity and the weight adaptation play a significant role, in the equalization as well as in the decoding. It can safely be said that they represent the major challenge of the circuit, since the analog circuit must be capable to perform equalization and decoding for a given number of possible combinations of block size, symbol alphabet, coding scheme, etc. Particularly for the decoding, the advantage of having a non-full connectivity is counterbalanced by a double (and very complex) (de)multiplexing of the signals (once for the vector function fˇand once for the interneuron connectivity).

    7. Joint equalization and decoding

    Figure 18.

    Two examples for joint equalization and decoding. Notice the feedback from the decoder to the equalizer, i.e., turbo principle.

    Figure 19.

    Joint equalization and decoding as described in [52].Lextrepresents the “knowledge” obtained by exploiting the redundancy of the code.

    Turbo equalization is a joint iterative equalization and decoding scheme. In this case, a symbol-by-symbol maximum aposteriori probability (s/s MAP) equalizer exchanges in an iterative way reliability values Lwith a (s/s MAP) decoder [51, 52]. This concept is inspired from the decoding concept of turbo codes, where two (s/s MAP) decoders exchange iteratively reliability values [53]. Despite its good performance, the main drawback of the turbo equalizer is the very high complexity of the s/s MAP-equalizer for multipath channels with long impulse response (compared with symbol duration) and/or symbol alphabets with large cardinality. Therefore, a suboptimum equalization (and a suboptimum decoding) usually replace the s/s-MAP ones (Figure 18).

    One discrete-time joint equalization and decoding approach has been introduced in [52] and is shown in Figure 19. x˜, Rdand Rare as in Eq. (2) and z−1 is a delay unit. We notice that there are two different (iteration) loops in Figure 19: the equalization loop (the blue one) on symbol basis (in the sense of ψ) and the decoding loop (the dashed one) on bit basis. a={1,2,3,},ρ, i.e., after each ρequalization loops, one decoding loop is performed. The conversion between symbol basis and bit basis (uto Lch) is performed by θS/L(), the way around (Lto x) by θL/S(). The expressions for θL/S()and θL/S()can be found in [52]. However, for BPSK, they are given as

    θS/L(u)=2σ2u,θL/S(L)=tanh(L2).E24

    σ2 is given in Eq. (11). If we consider onlythe equalization loop in Figure 19, we notice that it describes exactly the dynamical behavior of discrete-time recurrent neural networks [19, 25, 33, 5456]

    u[l+1]=[IRd1R]x[l]+Rd1x˜,x[l]=θL/S(θS/L(u[l])).E25

    Remark 11.If θL/S(θS/L(u))=θ(opt)(u), Eqs. (8), (25) share the same equilibrium/fixed points. For BPSK, it can be easily shown based on Eqs. (12), (24) that this is fulfilled.

    8. Continuous-time joint equalization and decoding

    Motivated by the expected improvement of the energy efficiency by analog implementation compared with the conventional digital one, we map in this section the joint equalization and decoding structure given in Figure 19 to a continuous-time framework. s/s MAP DEC in Figure 19 is replaced by a suboptimum decoding algorithm: the belief propagation. Moreover, equalization and decoding loops in Figure 19 are replaced by RNN and HORNN as discussed previously in Sections 4-A and 6-A, respectively. The introduced structure serves as a computational model for an analog hardware implementation and does not need any training.

    Figure 20 shows a novel continuous-time joint equalization and decoding based on recurrent neural network structures. The dynamical behavior of the whole system is described by the following differential equations:

    ϒedu(t)dt=u(t)+Wx(t)+W0x˜,E26a
    Lch(t)=θS/L(u(t)),E26b
    ϒddL(t)dt=L(t)+Pf(L(t))+BLch(t),E26c
    L(t)=BTf(L(t))Lext(t)+Lch(t),E26d
    x(t)=θL/S(L(t)),E26e
    fj=2atanh{jpos[S(j,:)=1]tanh(Lj2)}.E26f

    Figure 20.

    Continuous-timejointequalization and decoding based on a recurrent neural network structure for real-valued symbol alphabet (for complex-valued ones, cf. Remark 3).L(t),x(t)are the soft output of the decoder and the equalizer, respectively. (a) The equalization part. (b) The decoding part.

    Comparing Figure 20 with Figures 7 and 15, we notice that

    • The function φ

      (⋅) in Figure 7 (the optimum activation function θ(opt)()) has been split into two functions θS/L()and θL/S()in Figure 20(a). For BPSK symbol alphabet and based on Eqs. (12), (24), it can be easily shown that θ(opt)(u)=θL/S(θS/L(u)), cf. Remark 11.

    • The functions φˇ()and fˇ()in Figure 15, Eq. (18) have been merged to one function f()in Figure 20(b), Eq. (26f) f(L)=fˇ(φˇ(L)). L(t)x(t)are the soft output of the decoder and the equalizer, respectively.

    A. Special cases

    The novel structure in Figure 20 is general and stays valid for the following cases:

    Figure 21.

    Continuous-timeseparateequalization and decoding based on a recurrent neural network structure for real-valued symbol alphabet (for complex-valued ones, cf. Remark 3).x(t)is the soft output of the equalizer.

    • Separate equalization and decoding: In this case, Figure 20(a) is modified such that no feedback from decoder to equalizer is applied. This is shown in Figure 21(a). Onlyat the end of the separate equalization and decoding process, the output is given as L=Lext+Lch. We distinguish between two cases

      1. Equalization and decoding take place separately at the same time.

      2. Successive equalization and decoding: onlyafter the end of the equalization process, Lchare forwarded to the decoder and the decoder starts the evolution. We focus on this case.

    • Coded transmission over an AWGN channel: In this case, R=Iand hence based on Eq. (7) W=0, W0=I. Under these conditions, Eq. (26a) becomes linear and can easily be solved u(t)x˜and Lch(t)in Eq. (26b) becomes time independent Lch. In this case, Eq. (26c) reduces to Eqs. (16), (18).

    • Uncoded transmission over an “interference-causing” channel: In this case, P=0, B=0and Eq. (26c) becomes L=0nh. Under these conditions, Eq. (26a) reduces to Eqs. (5), (7), (8) (notice, however, Remark 11).

    B. Throughput, asynchronicity and scheduling

    The diagonal elements in ϒddefine the duration of the transient response the HORNN needs in order to converge eventually (in case of convergence). The larger they are, the longer is the transient response and consequently the less is the decoding throughput. The same is valid for ϒe. The diagonal elements of ϒebased on our analog RNN vector equalizer are in the range of a few tens of picoseconds.

    Unequal diagonal elements in ϒe(and ϒd) represent some kind of continuous-time asynchronicity [46]. Asynchronicity in discrete-time RNNs is desirable since it provides the ability to avoid limit cycles, which can probably occur in synchronous discrete-time RNNs [54, 57].

    Assuming ϒd=τdIand ϒe=τeI, we notice that the ratio τe/τdis comparable to the scheduling problem in the discrete-time joint equalization and decoding case. More precisely, how many iterations ρwithin the equalizer should be performed before a decoding process takes place, cf. Figure 19. This is optimized usually by simulations and is case dependent.

    From a dynamical point of view, the case τe/τd1(or τd/τe1) could be seen as a singular perturbation (in time). In this case, one part of Figure 20 can be seen as “frozen” compared with the other part.

    Remark 12. We notice that the parameters of the transmission model (block transmit matrix, symbol alphabet, block size, channel coding scheme) are utilized to define the parameters of the continuous-time recurrent neural network structure in Figure 20 such that no training is needed. This represents in practice a big advantage especially for analog hardware. However, to enable different coding schemes and symbol alphabets, either a full connectivity or a vector and interneuron connectivity controls are needed. Both structures are challenging from a hardware implementation point of view.

    Remark 13.For the ease of depiction, Figures 20 and 21 assume that one transmitted block contains exactly one codeword. This is not necessarily the case in practice. As an example, if one transmitted block contains two codewords, one RNN and two parallel HORNNs will be needed. On the other hand, if one codeword lasts over two transmitted blocks, two parallel RNNs and one HORNN is needed.

    9. Simulation results

    We simulate the dynamical system as given in Eq. (26) and Figure 20 based on the first Euler method [58]. We assume:

    Figure 22.

    BER vs.Eb/N0for evolution time equals20τ. Continuous-time (joint) equalization and decoding. BPSK symbol alphabet.

    • BPSK modulation scheme (symbol alphabet ψ={1,+1}).

    • Each transmitted block contains one codeword, cf. Remark 13.

    • ϒd=ϒe=τI.

    • Channel coding scheme: An LDPC code with k=204, code rate 0.5(n=408) and column weight 3taken from [59].

    • Multipath channels [60]:

      1. - Proakis-a abbreviated in the following by its channel impulse response haleading to a small interference.

      2. - Proakis-b abbreviated in the following by its channel impulse response hbleading to a moderate interference.

    The impulse response of haand hbare

    ha=[0.040.050.070.210.50.720.3600.210.030.07],hb=[0.4070.8150.407].

    The block transmission matrix Ris a banded Toeplitz matrix of the autocorrelation function of the channel impulse response [61]. The following cases are considered:

    • Uncoded transmission over AWGN channel. The bit error rate can be obtained analytically and is given as 12erfc{EbN0}[62]. erfc()is the complementary error function and Ebis the energy per bit.

    • Coded transmission over AWGN channel and continuous-time decoding at the receiver (HORNN-belief propagation).

    • Uncoded transmission over (the abovementioned) multipath channels and continuous-time equalization at the receiver (RNN-equalization).

    • Coded transmission over (the abovementioned) multipath channels. We distinguish between joint equalization and decoding (Figure 20) and separate equalization and decoding (Figure 21). In the latter case, equalization is performed firstly, and consequently the decoding.

    The evolution time for the whole system in all cases is 20 ⋅ τ, i.e., all simulated scenarios deliver the same throughput. For separate equalization and decoding, the evolution time of the equalization equals the evolution time of the decoding and equals 10 τ. The simulation results are shown in Figure 22. We notice the following:

    • Joint equalization and decoding outperforms the separate one, which is a fact we know from the discrete-time case. Our proposed model in Figure 20 is capable of “transforming” this advantage to the continuous-time case.

    • For the channel ha, the BER (for continuous-time joint equalization and decoding) is close to the coded BER curve.

    • For the channel hb, there exists a gap between the obtained results and the coded AWGN curve. This was expected, since hbrepresents a more severe multipath channel compared with ha.

    • If only equalization performance is considered, we compare between “Uncoded & EQ” curves and “Uncoded BER” curves. In Figure 22(a), the vector equalizer based on continuous-time recurrent neural networks is capable to remove already all interferences caused by the multipath channel ha, whereas in Figure 22(b), the “Uncoded & EQ” curve approaches an error floor.

    Remark 14.Interleaving and antigray mapping often encountered in the context of iterative equalization and decoding can be easily integrated in the proposed model in Figure 20. Antigray mapping will influence the functions θS/L()and θL/S(), whereas interleaving affects the matrix B.

    Advertisement

    10. Conclusion

    Joint equalization and decoding is a detection technique which possesses the potential for improving the bit error rates of the transmission at the cost of additional computational complexity at the receiver. Joint equalization and decoding is being considered only for the discrete-time case. However, for high data rates, the energy consumption of a digital implementation becomes a limiting factor and shortens the lifetime of the battery. Improving the energy efficiency revives the analog implementation option for joint equalization and decoding algorithms, particularly taking advantage of the nonlinearity of the corresponding algorithms.

    Continuous-time recurrent neural networks serve as promising computational models for analog hardware implementation and stand out due to their Lyapunov stability (the proved existence of attracting equilibrium points under specific conditions) and special suitability for analog VLSI. They have often been applied for solving optimization problems even without the need for a training. The drop of the training is particularly favorable for analog hardware implementation.

    In this chapter, we introduced a novel continuous-time recurrent neural network structure, which is capable to perform continuous-time joint equalization and decoding. This structure is based on continuous-time recurrent neural networks for equalization and continuous-time high-order recurrent neural networks for belief propagation, a well-known decoding algorithm for low-density parity-check codes. In both cases, the behavior of the underlying dynamical system has been addressed, Lyapunov stability and simulated annealing are a few examples. The parameters of the transmission system (channel matrix, symbol alphabet, block size, channel coding scheme) are used to define the parameters of the proposed recurrent neural network such that no trainingis needed.

    Simulation results showed that the superiority of joint equalization and decoding preserves, if this is done in analog according to our proposed model. Compared with the digital implementation, the analog one is expected to improve the energy efficiency and consume less chip area. We confirmed this for the analog hardware implementation of the equalization part. In this case, the analog vector equalization achieves an energy efficiency of a few picojoule per equalized bit, which is three to four orders of magnitude better than the digital counterparts. Additionally, analog hardware implementation aspects have been discussed. We showed as an example the importance of the interneuron connectivity, especially pointing out the challenges represented either by the hardware implementation of a massively distributed network, or by the routing of the signals using (de)multiplexers.

    Notes

    • For a clear distinction between accuracy and precision when used in hardware implementation context, we refer to [8].
    • Energy efficiency is defined later as appropriate.
    • Except for L, L⌣ and Lch which are vectors.
    • The abbreviation RNN in this chapter inherently includes the continuous-time and the single-layer properties.
    • For discrete-time single-layer recurrent neural networks for vector equalization, we refer to references [19, 33].
    • Analog RNN vector equalizer refers to the described analog hardware-implemented RNN for vector equalization.
    • The abbreviation HORNN in this chapter inherently includes the continuous-time, single-layer and real-valued properties.
    • L, L⌣ and Lch are vectors.

    © 2016 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

    How to cite and reference

    Link to this chapter Copy to clipboard

    Cite this chapter Copy to clipboard

    Mohamad Mostafa, Giuseppe Oliveri, Werner G. Teich and Jürgen Lindner (October 19th 2016). A Continuous-Time Recurrent Neural Network for Joint Equalization and Decoding – Analog Hardware Implementation Aspects, Artificial Neural Networks - Models and Applications, Joao Luis G. Rosa, IntechOpen, DOI: 10.5772/63387. Available from:

    chapter statistics

    1788total chapter downloads

    1Crossref citations

    More statistics for editors and authors

    Login to your personal dashboard for more detailed statistics on your publications.

    Access personal reporting

    Related Content

    This Book

    Next chapter

    Direct Signal Detection Without Data‐Aided: A MIMO Functional Network Approach

    By Xiukai Ruan, Yanhua Tan, Yuxing Dai, Guihua Cui, Xiaojing Shi, Qibo Cai, Chang Li, Han Li, Yaoju Zhang and Dameng Dai

    Related Book

    First chapter

    Introduction to Infrared Spectroscopy

    By Theophile Theophanides

    We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

    More About Us