Wireless Channel Model with Markov Chains Using MATLAB

1. Introduction

Channel modeling is the characterization of the wireless channel; it describes how the characteristics of the sent signal can be affected or what the conditions of the environment are, i.e. frequency, obstacles in the path, etc. The channel model helps to evaluate the performance of the system and to compare different techniques to mitigate the perturbations so the best fitted solution can be implemented according to the presented problem.

The simplest model is the free space loss which considers no obstacles between transmitter and receptor. (Parsons, 2000) However this is an ideal model that does not exist in real scenario because there are other losses in the wireless channel. Another option to describe a channel is by using statistical models which are based on probability density functions (pdf). Although they do not describe the behavior of the sent signal in a complete way, they give a good approximation of the conditions of the channel in a certain moment. (Nakagami, 1960; Abouraddy & Elnoubi, 2000) The main statistical channels and their corresponding pdf are:

• Additive White Gaussian Noise, AWGN. This channel only includes the sum of the white noise that follows a normal or Gaussian density.

$$\text{p}\left(\text{x}\right)\text{=}\frac{\text{1}}{\text{σ}\sqrt{\text{2π}}}\text{exp}\left(\text{-}\frac{{\left(\text{x-μ}\right)}^{\text{2}}}{{\text{2σ}}^{\text{2}}}\right)$$E1

Where:

p(x), Probability density function.

μ,Mean.

σ,Standard deviation.

σ²,Variance.

• Rice: represents a signal which is the result of the sum of the different multipath plus a dominant signal, known as the line of sight (LOS) between transmitter and receptor.

$$\text{p}\left(\text{x}\right)\text{=}\frac{\text{x}}{{\text{σ}}^{\text{2}}}\text{exp}\left(\text{-}\frac{\left({\text{x}}^{\text{2}}{\text{+A}}^{\text{2}}\right)}{{\text{2σ}}^{\text{2}}}\right){\text{I}}_{\text{0}}\left(\frac{\text{Ax}}{{\text{σ}}^{\text{2}}}\right)\text{, A}\ge \text{0}$$E2

Where:

p(x),Probability density function.

σ²,Variance of the signal or mean power of the signal before the detection of the envelope.

A,Amplitude of the dominant signal.

I₀(),Zero order modified Bessel function of the first kind.

The relation between the power of the dominant signal and the power of the multipath is represented in (3).

$$\text{K}\left(\text{dB}\right)=10\log \left(\frac{A^2}{2{\sigma }^2}\right)$$E3

Where:

K, Rice factor.

σ²,Variance of the signal or mean power of the signal before the detection of the envelope.

A,Amplitude of the dominant signal.

As A value decreases, the function is closer to characterize a Rayleigh fading environment. Otherwise, if the A value increases, the function is used to model an AWGN channel.

• Rayleigh: represents fast fading and describes the multipath phenomenon with no LOS between transmitter and receptor which can be caused by a non stationary user.

$$\text{p}\left(\text{x}\right)\text{=}\frac{\text{x}}{{\text{σ}}^{\text{2}}}\text{exp}\left(\text{-}\frac{{\text{x}}^{\text{2}}}{{\text{2σ}}^{\text{2}}}\right)\text{, x}\ge \text{0}$$E4

Where:

p(x), Probability density function.

σ²,Variance of the signal or mean power of the signal before the detection of the envelope.

Due to a wireless channel is a time variant channel, a better option to characterize a channel is Markov chains, which are a stochastic process with a limited number of states and whose transition between them is based on the probability of an event. States of Markov chains could be defined in agreement to channel conditions of an scenario, like it was done by E. Lutz who defined a two state channel model, one classified as good and the other one classified as bad depending on the conditions and therefore the attenuation level. Another important work is the one of B. Vucetic which characterizes the wireless channel of a certain area in Australia with a four state Markov chain by combining four different types of statistical channels.

Next sections explain in detail how to create a wireless channel model using Markov chains and its application in a communication system with a coder and modulator. To achieve this, it is also explained how to create a markovian chain using Matlab. The effects of this model are analyzed through BER curves. An analysis with a semimarkovian channel is explained in the last section and a comparative analysis is done between this model and that from Markov chains. Finally, conclusions about both subjects are given. All the analysis is also done with Matlab.

2. Markov chains

One type of stochastic process is Markov chains, named after Andrei Markov who studied the transitions between consonants and vowels in a poem at the beginning of the 20th century. In this process, a set is defined as {X(t), t = 0,1,2,...} whose number of elements is finite and denoted with real positive numbers. X(t) = i represents that the process is in state i at an instant of time t and take discrete values. It is said that there is a fixed probability p_ij that chain goes from a state i to a state j in the next time t. This process is represented in (5) and it is known as the Markov property.

$$\text{P}\left[\text{X}\left({\text{t}}_{\text{k+1}}\right){\text{=x}}_{\text{k+1}}{\text{|X(t}}_{\text{k}}{\text{=x}}_{\text{k}}\text{,…,X}\left({\text{t}}_{\text{1}}\right){\text{=x}}_{\text{1}}\right]\text{=P}\left[\text{X}\left({\text{t}}_{\text{k+1}}\right){\text{=x}}_{\text{k+1}}\text{|X}\left({\text{t}}_{\text{k}}\right){\text{=x}}_{\text{k}}\right]$$E5

Where,

X(t_k),Current sample.

X(t_k+1),Future sample.

X(t₁),...,X(t_k-1),Past samples.

x_k, State of the sample in the moment k.

Markov property indicates that given the present state, the next state is conditionally independent of the past. This statement is illustrated in Figure 1 which shows that the system can change from one condition to another or continues in the same state.

2.1. Elements of Markov chains

There are three important elements in Markov chains:

• Probability transition matrix P.

• Transition diagram.

• Steady-state vector π

The characteristics of each element will be given in next sections.

Figure 1.

Markov Property

2.1.1. Probability transition matrix

The switch between states is established in the probability transition matrix P. Each element of it represents the probability that switches or remains in the state. These switches are called transitions. P is a square matrix whose order is the same as the number of states. Equation (6) shows the structure of a probability transition matrix.

$$P\text{=}\left[\begin{array}{cccc}{\text{p}}_{\text{00}}& {\text{p}}_{\text{01}}& {\text{p}}_{\text{02}}& \text{…}\\ {\text{p}}_{\text{10}}& {\text{p}}_{\text{11}}& {\text{p}}_{\text{12}}& \text{…}\\ \text{.}& \text{.}& \text{.}& \\ {\text{p}}_{\text{i0}}& {\text{p}}_{\text{i1}}& \text{…}& \\ \text{.}& \text{.}& \text{…}& \end{array}\right]$$E6

Each element of P must satisfy the next condition.

$${\text{p}}_{\text{ij}}\ge \text{0, i,j=0,1,2,3…}$$E7

Where,

p_ij,Probability that current sample is in state i given the immediate precedent past time was in state j.

i,Transition state i.

j,Transition state j.

Condition of (7) is based on the argument that p_ij represents certain probability.

Another important characteristic of the probability transition matrix is that the sum of each row must be equal to one.

$${\displaystyle \sum _{\text{j}}{\text{p}}_{\text{ij}}}\text{=1, i=1,2,3,…}$$E8

Where

p_ij,Probability that current sample is in state i given the immediate precedent past time was in state j.

i,Transition state i.

j,Transition state j.

An advantage of markovian models is that they describe in an efficient way the characteristics of the system and therefore they can be solved by mathematic methods. In communication systems, Markov chains characterize in a better way the channel because it represents the combination of some phenomena that affects the signal during its wireless transmission, having a better approach to a real channel.

2.1.2. Transition diagram

A probability transition matrix can be also represented as a diagram called transition diagram. Each node ○ represents a state of the Markov chain indicated with a number inside; an arrow  connects state i with state j if a transition exists and the transition probability p_ij is written on that connecting arrow, even if the transition is to the same state. (Nerlson, 2002) The transition diagram is also used to describe the model definitions; it is easy to see in this diagram the possible switches between states according to the probability. Next figures show the probability transition diagram for a two, three and four states Markov chain with its transition matrix.

Figure 2.

Probability transition diagram for 2-state Markov chain

Figure 3.

Probability transition diagram for 3-state Markov chain

Figure 4.

Probability transition diagram for 4-state Markov chain

2.1.3. Steady-state vector

Another important element in Markov chains is the steady-state vector π, which represents the total appearing percentage of a state in a Markov chain. This vector can be computed by raising P to a large power. This is shown in (9).

$$P^{\text{n}}\to \text{1}\pi$$E9

Where,

P,Probability transition matrix.

π,Steady state probability vector

1,Column vector of ones: 1^T = (1,1,…)

One property of the π vector is that the sum of its elements must be equal to one as it is shown in (10).

$${\displaystyle \sum _{\text{i}}{\text{π}}_{\text{i}}}\text{=1}$$E10

Where,

π_i,Steady state probability for state i.

3. Creation of a Markov chains in Matlab

In this section you will find the steps and the code to create a Markov chain using Matlab for two and three states. This code will be useful to create a wireless channel model that, within a wireless communications system, will be simulated using Matlab in next section.

3.1. Two-state Markov chains

The simpler case in Markov chains is the one with two states. The first step for the creation of a chain is to establish a transition matrix that indicates the switch between states. The transition matrix and the steady state vector for a two state model are shown in (11) and (12).

$$P_{2states}\text{=}\left[\begin{array}{cc}{\text{p}}_{\text{11}}& {\text{p}}_{\text{12}}\\ {\text{p}}_{\text{21}}& {\text{p}}_{\text{22}}\end{array}\right]$$E11

$${\pi }_{2states}\text{=}\left[\begin{array}{cc}{\text{π}}_{\text{1}}& {\text{π}}_{\text{2}}\end{array}\right]$$E12

The input data for the Matlab code will be the probability transition matrix and the size of the output vector. Since this is a stochastic event, the first sample and the switch between stated will be obtained in a random way but always following the characteristics of P. It is also possible to have an initial distribution for the first sample. The precise steps to create the Matlab code are listed below.

1. Generate a random integer number between 1 and 2 (because of the two state markovian chain).

2. Generate a random natural number between 0 and 1.

3. If previous sample was 1 and the generated number in step b) is equal or less to p₁₂, next sample of the chain is going to be 2; otherwise will be 1.

4. If previous sample was 2 and the generated number in step b) is equal or less to p₂₁, next sample will switch to 1, otherwise will be 2.

5. Repeat steps b) to d) until complete the number of samples required. The generated series is the resultant Markov chain.

The Matlab code for these steps is shown next.

The output vector will determine the moment when the sent signal will be affected by certain probabilistic channel established previously.

3.2. Three-state Markov chains

Another common case in Markov chains is the three-state case. The proceeding to create this type of chain in Matlab is similar to the two-state model. The input is the 3 x 3 size probability transition matrix, like the one shown in (13) and the output vector will be also the vector containing the Markov chain with the characteristics set by P.

$$P_{3chan}\text{=}\left[\begin{array}{ccc}{\text{p}}_{\text{11}}& {\text{p}}_{\text{12}}& {\text{p}}_{\text{13}}\\ {\text{p}}_{\text{21}}& {\text{p}}_{\text{22}}& {\text{p}}_{\text{23}}\\ {\text{p}}_{\text{31}}& {\text{p}}_{\text{32}}& {\text{p}}_{\text{33}}\end{array}\right]$$E13

The π vector for a three-state Markov chain is shown in (14).

$${\pi }_{2chan}\text{=}\left[\begin{array}{ccc}{\text{π}}_{\text{1}}& {\text{π}}_{\text{2}}& {\text{π}}_{\text{3}}\end{array}\right]$$E14

The first step now will be to modify the probability transition matrix by computing the cumulative sum for each row because this will determine the transition between states. This step is show in (15).

$$P{\text{'}}_{3chan}\text{=}\left[\begin{array}{ccc}{\text{p}}_{\text{11}}& {\text{p}}_{\text{12}}{\text{+p}}_{\text{11}}& {\text{p}}_{\text{13}}{\text{+p}}_{\text{12}}{\text{+p}}_{\text{11}}\\ {\text{p}}_{\text{21}}& {\text{p}}_{\text{22}}{\text{+p}}_{\text{21}}& {\text{p}}_{\text{23}}{\text{+p}}_{\text{22}}{\text{+p}}_{\text{21}}\\ {\text{p}}_{\text{31}}& {\text{p}}_{\text{32}}{\text{+p}}_{\text{31}}& {\text{p}}_{\text{33}}{\text{+p}}_{\text{32}}{\text{+p}}_{\text{31}}\end{array}\right]\text{=}\left[\begin{array}{ccc}{\text{p}}_{\text{11}}& {\text{p}}_{\text{12}}\text{'}& \text{1}\\ {\text{p}}_{\text{21}}& {\text{p}}_{\text{22}}\text{'}& \text{1}\\ {\text{p}}_{\text{31}}& {\text{p}}_{\text{32}}\text{'}& \text{1}\end{array}\right]$$E15

Once P’_3chan was computed, the proceeding to create the resulting Markov chain is the same as the two-state one. The steps are listed next.

1. Generate a random integer number between 1 and 3 (because now it is a three-state model). This will be the first sample of the chain, although an initial distribution can be established for this case.

2. Generate a random natural number between 0 and 1.

3. If previous sample was 1 and the generated number in step b) is less than p₁₁, the next sample will be 1; if it is greater than p₁₂’, next sample will be 3, otherwise will be 2.

4. If preceding sample was 2 and the generated number in step b) is less than p₂₁, the next sample will be 1; if it is greater than p₂₂’, next sample will be 3; if not, it will be 2.

5. If previous sample was 3 and the generated number in step b) is less than p₃₁, next sample will be 1; if it is greater than p₃₂’, next sample will be 3, otherwise will be 2.

6. Repeat steps b) – e) until complete the required size of the Markov chain.

The Matlab code for the previous steps is presented next.

The methodology explained above can be used as a base to create an n-state Markov chain.

To proof the created Markov chain follows a specific probability transition matrix is by counting the number of each one of possible transitions and dividing them by the sum of the total transitions for the corresponding state. To verify that the chain follows also the steady-state vector it is necessary to count the number of times each state appeared in the chain and divide each count by the total number of samples (size of the output vector).

Two and three-state Markov chains Matlab programs described above represent the wireless transmission channel that will be implemented in a communication system. This will be done by simulating a message is being sent through it, all with Matlab.

4. Implementation of a Markov chains in Matlab

Another alternative to characterize a wireless channel is using Markov chains as it was said before. This model is a better option than statistic channels (AWGN, Rice, Rayleigh) because the former represent a statistical channel combination of the latter. To show the application of Markov chains in this area the next scenario is proposed.

“A wireless channel model must be created in such manner it characterizes the principal causes of attenuation, for instance: the obstacles between the transmitter and receiver, a mobile user or the influence of the line of sight (LOS).”

The statistic channels for the scenario area are defined as Rayleigh and Rice. The former characterize a mobile user and the latter the LOS between transmitter and receiver.

To simulate this scenario in Matlab, a communication system with a QPSK and rate 2/3 convolutional code is proposed. The block diagram of this system and the channel model are illustrated in Figure 3 and 4. Markov chains will be used as a base to create a wireless channel model and will represent the multiplicative fading, which is right after the transmitter, and will be switching between the possible channels according to the probability transition matrix, as it is shown in Figure 4. From now, this model will be called as a “markovian channel”.

To have a better understanding of this markovian model, several examples with different probability transition with its steady-state vector, calculated using (9), are proposed. The codes for the creation of a two and three state Markov chains are used to produce results. Interpretations are based on BER curves obtained by implementing the communication system and the markovian channel of figures below. Next sections explain with details the examples and its results.

4.1. Two-state Markov channel model

As it was said before, this is the simplest model in Markov chains. According to the proposed scenario, the two statistical channels used for this case are:

• Channel 1: Rice K = 14 dB. This channel represents the better conditions because of its strong LOS.

• Channel 2: Rayleigh. This channel corresponds to a mobile user and multipath generated from the movement.

Now, three examples about two-state markovian channel model are proposed next.

Figure 5.

Block diagram for a communication system with markovian channel

Figure 6.

Block diagram of the markovian channel

Example 1. The next two Markov channels are proposed.

$$P_{2chan1}\text{=}\left[\begin{array}{cc}\text{1}& \text{0}\\ \text{1}& \text{0}\end{array}\right]$$E16

$$P_{2chan2}\text{=}\left[\begin{array}{cc}\text{0}& \text{1}\\ \text{0}& \text{1}\end{array}\right]$$E17

And their steady-state vectors are shown in (18) and (19).

$${\pi }_{2chan1}\text{=}\left[\begin{array}{cc}\text{1}& \text{0}\end{array}\right]$$E18

$${\pi }_{2chan2}\text{=}\left[\begin{array}{cc}\text{0}& \text{1}\end{array}\right]$$E19

Channel P_2chan1 sets that channel 1 (Rice K = 14 dB) appears all the time, in other words, there is a strong LOS between transmitter and receiver, which can be referred as a rural environment where there are no tall buildings that blocks that line of sight. Conversely, channel P_2chan2 establish that channel 2 (Rayleigh) will appear all the time, this channels can be seen as an urban environment where there are a lot of obstacles between transmitter and receiver that will cause the multipath or also it can represents an user who is moving from one point to another one. These two channels were simulated with the communication system of Figure 3 in Matlab and the BER curves obtained are shown in Figure 5.

Figure 7.

Two state Markov cannel for Example 1

To verify the results of the markovian channel, BER curves of systems using exclusively each statistical channel were obtained too. It can be appreciated in Figure 5 that the resulting BER curve for P_2chan1 is exactly the same as the one resulting from the simulation of the Rice K = 14 dB statistical channel. Likewise, BER curve generated from P_2chan2 is exactly the same to the simulation where only Rayleigh statistical channel is implemented.

To proof the Markov channel follows the characteristics of (16) and (17), a counting of the transitions and the number of times both channels appeared is realized from the created Markov chain. The size of the simulated sent message was 100,000 bits, so the results of the count are given as percentages, which is equivalent to the probabilities of the transition matrixes. Those results are given in Table 1 and 2.

Table 1.

Counting of events for P_2chan1

Table 2.

Counting of events for P_2chan2

As it can be observed, obtained results from counting are equal to the probability transition matrix of (16). This means that 100% of chances there was a transition from channel 1 to 1 and from channel 2 to 1. Channel 1 appeared 99.99% which says that at the first sample of the chain was channel 2 but immediately after switched to channel 1, that is why the third column of Table 1 is not the same to the steady-state vector of (18) but it is very close to it.

For the case of P_2chan2, it is appreciated that channel 1 never appeared in the Markov chain, thus 100% of chances remained in channel 1, so data from the third column of Table 2 has the same characteristics of the steady state vector of (19).

Example 2. In contrast to Example 1, these two probability transition matrixes and its steady-state vectors are proposed.

$$P_{2chan3}\text{=}\left[\begin{array}{cc}\text{0}\text{.2}& \text{0}\text{.8}\\ \text{0}\text{.2}& \text{0}\text{.8}\end{array}\right]$$E20

$${\pi }_{2chan3}\text{=}\left[\begin{array}{cc}\text{0}\text{.2}& \text{0}\text{.8}\end{array}\right]$$E21

$$P_{2chan4}\text{=}\left[\begin{array}{cc}\text{0}\text{.8}& \text{0}\text{.2}\\ \text{0}\text{.8}& \text{0}\text{.2}\end{array}\right]$$E22

$${\pi }_{2chan4}\text{=}\left[\begin{array}{cc}\text{0}\text{.8}& \text{0}\text{.2}\end{array}\right]$$E23

The interpretation for the channel of P_2chan3 is that 80% of chances, Rayleigh channel will be present in the Markov chain meaning that the user is moving (walking, running) and LOS conditions are poor. For channel P_2chan4 probabilities are opposite than the other channel, in other words, there is a greater probability of having LOS than multipath, like in a rural area. Resulting BER curves for both channels are shown in Figure 8, where it shows that BER curves from markovian channels of (20) and (22) are very close to that curves resulting from simulating only the correspondent statistical channel but they are not the same, proving the combination of the channels according to the probability transition matrixes.

Figure 8.

Two State Markov Channel for Example 2

Example 3. A probability transition matrix shown in (24) with its steady-state vector in (25) are presented as a new wireless channel model. In this channel model the user has LOS the most part of the time but in some instants experiments multipath. This seems a semi urban environment because it can exists some building that cause multipath but they are not that tall to block the line of sight. BER curve for this model is shown in Figure 9.

$$P_{2chan5}\text{=}\left[\begin{array}{cc}\text{0}\text{.7}& \text{0}\text{.3}\\ \text{0}\text{.9}& \text{0}\text{.1}\end{array}\right]$$E24

$${\pi }_{2chan5}\text{=}\left[\begin{array}{cc}\text{0}\text{.75}& \text{0}\text{.25}\end{array}\right]$$E25

Figure 9.

Two State Markov Channel for Example 3

To verify the channel model was well implemented the counting of switches and how many times a state appeared is presented next.

Table 3.

Counting of events for P_2chan5

The count of transitions between states and the times each state (or probabilistic channel) appeared in the Markov chains indicate probability transition matrix shown in (25) was followed. This can be observed also in Figure 9 where the BER curve for the channel model presents an inclination to the curve obtained when only Rice channel is simulated but it is not identically to it.

4.2. Three-state Markov channel model

A two-state channel model is not enough to characterize a wireless channel. To create a better model it is necessary to add more statistical channel. According to the proposed scenario at the beginning of the section, the the appropriate three statistical channels are:

• Channel 1: Rice K = 14 dB, very strong LOS.

• Channel 2: Rayleigh, multipath and moving user.

• Channel 3: Rice K = 7 dB, not a strong LOS like channel 1.

As it was done for the two-state channel model, different probability transition matrixes with their steady state vectors are through BER curves to proof the correct implementation of the channel in a communication system of Figure 5.

Example 4. Three markovian channels are proposed.

$$P_{3chan1}\text{=}\left[\begin{array}{ccc}\text{1}& \text{0}& \text{0}\\ \text{1}& \text{0}& \text{0}\\ \text{1}& \text{0}& \text{0}\end{array}\right]$$E26

$$P_{3chan2}\text{=}\left[\begin{array}{ccc}\text{0}& \text{1}& \text{0}\\ \text{0}& \text{1}& \text{0}\\ \text{0}& \text{1}& \text{0}\end{array}\right]$$E27

$$P_{3chan3}\text{=}\left[\begin{array}{ccc}\text{0}& \text{0}& \text{1}\\ \text{0}& \text{0}& \text{1}\\ \text{0}& \text{0}& \text{1}\end{array}\right]$$E28

Its steady-state vectors are shown next.

$${\pi }_{3chan1}\text{=}\left[\begin{array}{ccc}\text{1}& \text{0}& \text{0}\end{array}\right]$$E29

$${\pi }_{3chan2}\text{=}\left[\begin{array}{ccc}\text{0}& \text{1}& \text{0}\end{array}\right]$$E30

$${\pi }_{3chan3}\text{=}\left[\begin{array}{ccc}\text{0}& \text{0}& \text{1}\end{array}\right]$$E31

The goal of these models is to verify the correct implementation of the markovian channels into a communication system. Each one of them establishes that they must stay always in the same channel. Results are compared with BER curves when only statistical channels are simulated. Figure 10 shows each BER curve obtained from the Markov chains models of (26) – (28) are almost the same to the ones obtained when only the corresponding statistical channel is simulated.

Markovian channel of (26) can be compared to a rural area where LOS is very probable because there are no tall buildings. Channel model of (27) can be interpreted as an urban zone because the multipath and users are constantly moving. Markov channel of (28) indicates LOS but not as strong as channel 1, so it can be seen as a semi-urban area because there are not a lot of tall buildings to block the transmission.

Example 5. Another three markovian channel models are proposed where it is indicated that each channel must be present in the Markov chain 80% of times. Probability transition matrixes and steady-state vectors are shown in (32)-(37).

Figure 10.

Three State Markov Channel for Example 4

$$P_{3chan4}\text{=}\left[\begin{array}{ccc}\text{0}\text{.8}& \text{0}\text{.1}& \text{0}\text{.1}\\ \text{0}\text{.8}& \text{0}\text{.1}& \text{0}\text{.1}\\ \text{0}\text{.8}& \text{0}\text{.1}& \text{0}\text{.1}\end{array}\right]$$E32

$${\pi }_{3chan1}\text{=}\left[\begin{array}{ccc}\text{0}\text{.8}& \text{0}\text{.1}& \text{0}\text{.1}\end{array}\right]$$E33

$$P_{3chan5}\text{=}\left[\begin{array}{ccc}\text{0}\text{.1}& \text{0}\text{.8}& \text{0}\text{.1}\\ \text{0}\text{.1}& \text{0}\text{.8}& \text{0}\text{.1}\\ \text{0}\text{.1}& \text{0}\text{.8}& \text{0}\text{.1}\end{array}\right]$$E34

$${\pi }_{3chan1}\text{=}\left[\begin{array}{ccc}\text{0}\text{.1}& \text{0}\text{.8}& \text{0}\text{.1}\end{array}\right]$$E35

$$P_{3chan6}\text{=}\left[\begin{array}{ccc}\text{0}\text{.1}& \text{0}\text{.1}& \text{0}\text{.8}\\ \text{0}\text{.1}& \text{0}\text{.1}& \text{0}\text{.8}\\ \text{0}\text{.1}& \text{0}\text{.1}& \text{0}\text{.8}\end{array}\right]$$E36

$${\pi }_{3chan1}\text{=}\left[\begin{array}{ccc}\text{0}\text{.1}& \text{0}\text{.1}& \text{0}\text{.8}\end{array}\right]$$E37

Results are shown in Figure 11 where each BER curve are very close to the curve resulted from the simulation of only statistical channels (according to each case), confirming what is indicated in each steady-state vector.

Figure 11.

Three state Markov cannel for Example 5

Example 6. Once the behavior of the markovian channel model created with Matlab, probability transition matrix of (38) is proposed for the scenario explained above. This model indicates that the probability of a type of LOS (Rician channel K = 14 dB or K = 7 dB) is less than the probability when the user is moving (Rayleigh channel). This model focus on a mobile user and when there is a LOS but it is not the best condition. The resulting curve presented in Figure 12 seems to follow a combination of the most probable statistical channel indicated in steady-state vector (but never equal to a particular channel), concluding that these model really represents a better characterization of the channel.

$$P_{3chan7}\text{=}\left[\begin{array}{ccc}\text{0}\text{.1166}& \text{0}\text{.5072}& \text{0}\text{.3762}\\ \text{0}\text{.0675}& \text{0}\text{.4682}& \text{0}\text{.4643}\\ \text{0}\text{.1507}& \text{0}\text{.5208}& \text{0}\text{.3285}\end{array}\right]$$E38

$${\pi }_{3chan7}\text{=}\left[\begin{array}{ccc}\text{0}\text{.1062}& \text{0}\text{.4940}& \text{0}\text{.4010}\end{array}\right]$$E39

The effect of the combination of statistical channels is proved with the counting of events (Table 4). These data is very similar to the probability transition matrix of (38).

Table 4.

Counting of events for P_3chan7

Figure 12.

Three state Markov cannel for Example 6

4.3. Four-state Markov channel model

The methodology applied to implement a markovian channel in a wireless communication system can be also used to create an n-state channel. Some authors have characterized channels up to four states. Let’s take the probability transition matrix proposed by B. Vucetic shown in (40) with these probabilistic channels:

• Channel A: Rice K = 14 dB, strong LOS.

• Channel B: Rayleigh, mobile user and multipath.

• Channel C: Rice K = 7 dB, regular LOS.

• Channel D: Rice K = 5 dB, a channel with more noise but not as bad conditions as Rayleigh, user without movement.

$$P_{4chan}\text{=}\left[\begin{array}{cccc}\text{0}\text{.679}& \text{0}\text{.179}& \text{0}\text{.129}& \text{0}\text{.013}\\ \text{0}\text{.052}& \text{0}\text{.925}& \text{0}\text{.023}& \text{0}\\ \text{0}\text{.104}& \text{0}\text{.007}& \text{0}\text{.75}& \text{0}\text{.139}\\ \text{0}& \text{0}& \text{0}\text{.778}& \text{0}\text{.222}\end{array}\right]$$E40

Computed steady-state vector of (41) was obtained by using property of (9).

$${\pi }_{4chan}\text{=}\left[\begin{array}{cccc}\text{0}\text{.1753}& \text{0}\text{.4480}& \text{0}\text{.3171}& \text{0}\text{.0596}\end{array}\right]$$E41

Through steady-state vector, this model establishes that the biggest probability is found between channel B (mobile user) and channel C (Rice K = 7 dB, regular LOS). This can be interpreted as an urban zone scenario where users are constantly moving and there is less possibility of line of sight. BER curve of Figure 13 shows this trend and the behavior of the markovian chain is proved by counting the transitions and states. These last results are confirmed in Table 5.

Table 5.

Counting of events for P_3chan7

Figure 13.

Four-state Markov channel

4.4. N-state Markov channel model

It has been proved until this point that the more statistical channels in a Markov chain wireless channel model, the better the channel will be characterized, however this must have a limit because maybe a large number of states in the chain could be exaggerated. Another analysis is suggested to demonstrate this hypothesis by proposing another markovian channels shown in Table 6. The common characteristic between all models is that Rayleigh channel (multipath, mobile user) has the biggest probability than a channel with some type of line of sight. The possible states are:

• Channel 1: Rice = 14 dB, strong LOS.

• Channel 2: Rayleigh, mobile user and multipath.

• Channel 3: Rice K = 7 dB, regular LOS.

• Channel 4: Rice K = 5 dB, the channel with more noise but “better” conditions than Rayleigh, user without movement.

• Channel 5: AWGN, without any type of multiplicative fading. This is the channel with best conditions because it represents practically no losses.

Using the same methodology of all examples above, BER curve was computed and it is presented in Figure 14. It can be appreciated from graphic that the curve for two, three and four states are different between them; nevertheless five-state channel is very similar to four-state model although they are not exactly the same. This is very peculiar because there is nothing similar to the other cases. So the conclusion is that until four-state markovian channel model is enough to characterize a wireless channel because results are going to be very similar with more states. (Sánchez-Salas & Cuevas-Ruíz, 2007)

Table 6.

Markovian channels and steady-state vector for 1, 2, 3, 4 and 5 states

Figure 14.

Comparison between n-states Markov channel model

5. Semi-markovian channel model

Now let’s make the supposition that a process can be in any possible states (1,2,…,N) and each time goes to state i remains there for a certain random amount of time with media μ_i and then makes a transition to state j with probability p_ij. This practice is called semi-markovian process and it is very similar to Markov chains because switches between states follow the Markov property. However, the remaining time in a state is a random variable with some probability distribution. (Ross, 2007)

The probability transition matrix for a semi-markovian process is shown in (42).

$$P\text{=}\left[\begin{array}{cccc}\text{0}& {\text{p}}_{\text{01}}& {\text{p}}_{\text{02}}& \text{…}\\ {\text{p}}_{\text{10}}& \text{0}& {\text{p}}_{\text{12}}& \text{…}\\ \text{.}& \text{.}& & \\ {\text{p}}_{\text{i0}}& {\text{p}}_{\text{i1}}& \text{0}& \\ \text{.}& \text{.}& \text{…}& \text{0}\end{array}\right]$$E42

The most relevant characteristic of the matrix above is that it has a zero diagonal matrix because it only considers transitions between states. The second important element in this kind of processes is the mean sojourn time w_i in each state and it is represented as in (43).

$$w\text{=}\left[\begin{array}{cccc}{\text{w}}_{\text{1}}& {\text{w}}_{\text{2}}& \text{…}& {\text{w}}_{\text{i}}\end{array}\right]$$E43

The evolution of a semi-markovian process is explained through probability transition matrix P and sojourn time vector w in this way: the process begins in state i and remains there for a mean amount of time w_i and then switches to state j with probability p_ij; after that remains in state j for another mean amount of time w_j and can switch to state k with probability p_jk or return to state i with probability p_ji and so on. The transition diagram for a two and three semi-markovian chain is shown in Figure 15 and 16.

Figure 15.

Transition diagram for a 3-state semimarkovian process

Figure 16.

Transition diagram for a 2-state semimarkovian process

If time follows an exponential probability density function, the process is called “continous-time Markov process”. When time in each state and transition is the same (time does not vary and generally is 1), then the process is called “discrete Markov chain”, consequently the analyzed models above are a particular case of semi-markovian process.

Semimarkovian process can be also applied to wireless channel modelling. The main difference between Markov chains and semimarkovian processes is that this last one includes the analysis of sojourn time in each state. In this way, the applications of the semi-markovian processes in a communication system centers in the analysis of the sojourn time in a particular condition. This can be useful to determine the performance of the designed communication system, the effectiveness of a fading mitigation technique in order to preserve optimal transmission conditions as long as possible, or to study some kind of fading that is only present in a certain hour or season, etc. An example is proposed to understand better the application.

Example 7. A three-state semimarkovian wireless channel model with probability transition matrix of (44) and mean sojourn time of (45) must be created. The remaining time follows an exponential probability density function. Analyze the model and give an interpretation of results for a wireless communication system like the one of Figure 5.

$$P_{3semiM}\text{=}\left[\begin{array}{ccc}\text{0}& \text{0}\text{.5072}& \text{0}\text{.4928}\\ \text{0}\text{.5357}& \text{0}& \text{0}\text{.4643}\\ \text{0}\text{.1507}& \text{0}\text{.8493}& \text{0}\end{array}\right]$$E44

$$w_{3semiM}\text{=}\left[\begin{array}{ccc}\text{2 }& \text{0}\text{.4}& \text{5}\end{array}\right]$$E45

The statistical channels that form this model are:

• Channel 1: Rice K = 14 dB, strong LOS.

• Channel 2: Rayleigh, multipath and moving user.

• Channel 3: Rice K = 7 dB, regular LOS.

Solution:

Markov chains Matlab code created in previous sections can be modified to create a semi-markovian process. The main change is that another random number is needed to determine the time in which the state will remain. The code is shown next.

Next figure shows obtained results of how long remains the process in each statistical channel by using the Matlab code above. It can be observed the time and exact moment when each channel is presented. The graphic can be useful in case a record of some phenomenon or fading exists, for example, the rain.

Figure 17.

Three state semimarkovian channel

However this image may not be very clear at first sight. Probability transition matrix and sojourn time vector gives a good description of the channel. Also the steady-state vector can be computed using equation (9), which is shown in (46).

$${\pi }_{3semiM}\text{=}\left[\begin{array}{ccc}\text{0}\text{.2680 }& \text{0}\text{.4097}& \text{0}\text{.3223}\end{array}\right]$$E46

From (45) and (46) it is observed that although channel 3 has not the largest probability, it does have the largest sojourn time of all of them. In other words, it is probable that channel with regular LOS is not experimented by the user so often but when it does, it remains there for much more time than any other channel. On the other hand, although the channel with multipath has the largest probability, it is not to be there for a large amount of time and it will switch to better conditions (strong or regular LOS). The behavior of these channels seems to characterize an environment in an urban zone where there are mobile users or there are many obstacles. In this type of situations it is necessary to compensate multipath by designing a link between transmitter and receiver with effective fading mitigation techniques.

To prove all what was said above, the counting of transitions and states for the simulated semi-Markov process was done.

Table 7.

Counting of events for P_3semiM

The percentage of each transition is practically the same as the original transition matrix of (44), especially because there were no transitions to the same state in the semi-markovian process created. The percentages observed in the second row of Table 7 are the data of the steady state vector of (46). Finally, the same mean sojourn times was obtained as the one indicated in (45) proving the simulated process is correct.

6. Conclusion

The characterization of a wireless channel has become fundamental in the design of a communication system because of the rapid growing of telecommunications. The basic models are based on probability density functions although they represent real perturbations when a signal is sent through air, their characteristics are not enough because a real channel is composed by the combination of many statistical models.

A better choice to wireless channel modeling is Markov chains which are a type of stochastic processes and are based on the probability of an event given another past event. This type of model offers the combination of statistical channels. For this reason it is necessary to establish certain number of states for the probability transition matrix, which characterizes the channel. The larger the number of states (statistical channels) in the markovian chain, the better the channel is modeled but there is also a limit. The elements that model a Markov chains channel model are the probability transition matrix and steady-state vector.

Markov chains are a special case of semi-markovian processes. The main difference between them is that in the latter there are no transitions to the same state and have another element: the sojourn vector. This vector represents the mean time of sojourn of a state. This type of model gives information about how long and in what exact moment stays in a particular state, therefore its application could be the analysis of time of a particular condition (LOS, multipath, shadows) or a cause of fading (rain, scintillation, snow). This model can be also useful to know the performance of the communication system or some fading mitigation technique.

Since the methodology exposed in this chapter is very flexible, explained models can be applied with other characteristics. Given the bases for a markovian and semi-markovian channel, the lector is invited to develop another application for wireless telecommunication channels.