Complexity Analysis in Channel Estimation Massive MIMO Compared with LMU and GRU

Abstract

MIMO: Multiple-input multiple-output technology uses multiple antennas to use reflected signals to provide channel robustness and throughput gains. It is advantageous in several applications like cellular systems, users are distributed over a wide coverage area in various applications such as mobile systems, improving channel state information (CSI) processing efficiency in massive MIMO systems. This chapter proposes two channel-based deep learning methods gated recurrent unit and a Legendre memory unit to enhance the performance in a massive MIMO system and compares the complexity analysis to the previous methods, The complexity analysis is based on the channel state information network combined with gated recurrent units and Legendre memory units compared to indicator parameters which show the difference between literature-based techniques.

1. Introduction

Complexity analysis in deep learning is a fascinating topic indeed when delving into the realm of deep learning, we often encounter intricate models with numerous layers and countless parameters [1]. It is essential for such models to truly comprehend their inner workings. Now, complexity analysis in deep learning involves assessing the computational requirements and efficiency of these models. One commonly used metric is time complexity, which examines how the computational costs increase with an increase in the input size [2]. It allows us to estimate the computational resources needed for training and inference tasks.

Another aspect to consider is the space complexity, which examines the memory requirements of deep learning models. As the number of layers and parameters grows, so does the memory needed to store them [3]. Understanding this helps us optimize memory usage and select suitable hardware for our neural networks. However, it’s worth mentioning that deep learning is an ever-evolving field, and complexity analysis is just a small part of the puzzle.

There are always new challenges, surprises, and complexities waiting to be unraveled. It’s a journey that continues to intrigue and captivate scientists and enthusiasts alike. The history of complexity analysis in deep learning. It is indeed an intriguing subject. The path to understanding the complexity of deep learning has been paved with both triumphs and challenges. Back in the early days, when I first ventured into the realm of theoretical physics, we were only scratching the surface of what would later become known as deep learning [4]. The concept of neural networks has been around for quite some time, but it was the advent of powerful, computational capabilities that truly allowed us to explore their potential. As we delved deeper, we began to realize that the complexity of deep learning models was no trivial matter.

The number of parameters, the intricate interconnections, and the sheer depth of these networks made it a fascinating puzzle to unravel. The primary challenge was to develop methods for analyzing and understanding the complexity of these models. We needed tools that could help us comprehend the behavior of deep learning systems, predict their performance, and discern their limitations. Over time, researchers developed various approaches to complexity analysis in deep learning [5]. They ranged from straightforward measures like counting the number of parameters or layers, to more sophisticated techniques such as analyzing the network’s computational and memory requirements. Additionally, advancements were made in characterizing the computational complexity of training deep learning models. The notion of time and resource complexity became crucial in comprehending the training process and predicting how long it might take, given a certain dataset and architecture [6].

2. Complexity analysis

2.1 Performance and complexity trade-off of LMU and GRU

The Legendre Memory Unit (LMU) is mathematically derived to orthogonalize its continuous-time history – doing so by solving d coupled ordinary differential equations (ODEs), whose phase space linearly maps onto sliding windows of time via the Legendre polynomials up to degree d−1.

The Gated Recurrent Unit (GRU) is a type of Recurrent Neural Network (RNN) that, in certain cases, has advantages over long short-term memory (LSTM) (Figure 1). GRU uses less memory and is faster than LSTM, however, LSTM is more accurate when using datasets with longer sequences (Figure 2).

Figure 1.

The block diagram of (GRU) [7].

Figure 2.

Time-unrolled LMU layer [8].

The RNN layers for CSI compression and reconstruction of massive MIMO systems might have a remarkable number of parameters. The recurrent neural network module, for example, can dd up to ${\mathbf{10}}^{\mathbf{8}}$ more parameters, raising storage and computation difficulties. To reduce the computational complexity and required memory, a fully connected layer-based autoencoder has been developed for channel state information feedback in time-varying channels [9].

While other research has investigated smaller recurrent neural networks RNNs, the least computationally expensive of these models still needs ${\mathbf{10}}^{\mathbf{7}}$ parameters per snapshot. When the compression ratio is small, the networks also suffer from a significant drop in feedback performance because they must keep the same compression ratio in succeeding time slots and cannot obtain accurate previous information in the first time slot [10]. Despite the reported performance of “stacked” GRUs, the minimum depth of recurrent layers required for CSI recovery accuracy has yet to be determined [11]. Despite the substantial number of RNN parameters, the performance gain should be significant enough to justify the memory overhead.

The practical constraint of how often such feedback can be transmitted, as well as how CSI of fading channels will vary due to the Doppler effect, should be considered in DNN-based CSI feedback and recovery [12]. To reduce computational complexity and model size, we seek to systematically exploit channel characteristics such as forward CSI temporal coherence. We directly leverage known channel coherence temporally by developing a simple but effective Legendre memory unit combined with gated recurrent unit channel state information feedback LMU to improve CSI recovery accuracy.

3. Model size and computational complexity

3.1 The structure

Table 1 shows the number of parameters and floating-point operations (FLOPs) to show the impact of the model size reduction. This information compares the storage size and computational complexity of using the FC-layer along with the CNN-layer in the CsiNet compression module and the related decompression module.

	Number of parameters		FLOPs
	FC-based	CNN-layer	FC-based	CNN-layer
CR = 1/4	2.1 M	14.4 K	4.2 M	0.9 M
CR = 1/16	1.1 M	7.2 K	2.1 M	0.5 M
CR = 1/32	0.5 M	3.7 K	1.0 M	0.2 M
CR = 1/64	0.2 M	2.5 K	0.5 M	0.1 M

Table 1.

Number of parameters and FLOPs comparison for FC-based and CNN-based dimension compression and decompression module.

M, million; K, thousand.

The CNN-layer-based dimension compression and decompression module reduces the number of parameters by more than 100 times and the number of FLOPs by at least four times. The comparison results show that our CNN architecture for CSI compression and decompression is a significant step forward in expanding the spectrum of practical applications for deep learning-based channel state information encoding, feedback, and reconstruction in massive MIMO wireless systems.

Despite the higher cost, CsiNet-GRU and LMU-GRUs can reduce the number of parameters by order of magnitude and save over (5/6) FLOPs over CsiNet. Table 2 shows the complexity analysis of the proposed LMU-GRUs in comparison to various state-of-the-art CSI feedback systems, where the number of parameters and MACCs1 stand for space and temporal complexity, respectively [14]. When compared to CsiNet, our frameworks add fewer parameters while considerably improving CSI recovery quality, as seen in Table 2.

	γ	CsiNet [12]	Conv-LSTM-CsiNet [13]	CsiNet-GRU [7]	LMU-GRUs
Params	1/4	2,103,904	28,326,904	2,107,684	2,108,992
	1/16	1,055,072	22,296,312	1,058,852	1,060,160
	1/32	530,656	19,477,624	534,436	535,744
	1/64	268,448	18,117,432	272,228	273,536
MACCs	1/4	4,366,336	121,708,544	12,424,448	12,497,120
	1/16	3,842,048	97,591,296	10,375,872	11,448,544
	1/32	3,579,904	86,319,104	9,851,548	10,924,256
	1/64	3,448,832	80,879,616	9,589,440	10,662,112

Table 2.

The number of parameters and MACCS.

Convolution layers are mainly responsible for the increase in MACCs. When CR is low, convolutional layers require more computation than dense layers. LMU-GRUs have much lower model parameters and MACCs than CsiNet and Conv-LSTM-CsiNet, which improve reconstruction accuracy at the expense of huge space and temporal complexity due to the dense layers in GRU cells [15].

Each network’s average number of floating-point operations (FLOPs) connected with a single timeslot is also shown in Table 3. When compared to the Conv-LSTM-CsiNet in each compression ratio, CsiNet-GRU, and LMU-GRUs can save more than (8/9) and 9/10) FLOPs, respectively. Even at low compression ratios, the amount of computation for Conv-LSTM-CsiNet does not decrease considerably [16].

	γ	CsiNet [12]	Conv-LSTM CsiNet [13]	CsiNet-GRU [7]	LMU-GRUs
Params	1/4	2.1 M	2.1 M	132.7 M	2.1 M
	1/16	1.0 M	542.9 K	118.5 M	1.0 M
	1/32	530.6 K	280.7 K	116.1 M	535.7 K
	1/64	268.4 K	149.6 K	115.0 M	273.5 K
FLOPs	1/4	412.9 M	44.5 M	41.2 M	39.2 M
	1/16	409.8 M	41.3 M	40.5 M	39.3 M
	1/32	409.2 M	40.8 M	40.4 M	40.1 M
	1/64	409.0 M	40.5 M	40.3 M	40.0 M

Table 3.

Model size and computational complexity of tested networks.

M, million; K, thousand.

Without loss of performance, we show that latent convolutional layers require much fewer parameters than FC layers. Table 3 compares the model size and computational complexity of CsiNet, Conv-LSTM-CsiNet, CsiNet-GRU, and LMU-GRUs of a single timeslot, respectively.

LMU-GRUs utilize 60 times fewer parameters than Conv-LSTM-CsiNet across all compression ratios. More crucially, LMU-GRUs can achieve greater CSI recovery accuracy while using 1/3000 of the number of parameters required by Conv-LSTM-CsiNet. A 16-fold reduction in compression ratio (from 1/4 to 1/64), for example, only saves 1% of FLOPs. CsiNet-GRU and CsiNet-LMU-GRUs, on the other hand, require significantly less computing complexity in proportion to lower compression ratios.

A 16-fold CR reduction (from 1/4 to 1/64), for example, reduces the number of FLOPs in CsiNet-GRU and LMU-GRUs networks by 9 and 2%, respectively.

When using CsiNet-GRU and LMU-GRUs as a cooperative learning method at UE, we found that 50 percent more parameters and FLOPs are required than during the training phase. This is because the trained decoder must be repeated on the UE side to generate decoded CSI for the encoder’s previous time slot.

Table 3 summarizes the number of parameters and floating-point operations (FLOPs) to show the impact of the proposed model size reduction. This data compares the storage space and computational difficulty of using the FC layer against the suggested CNN layer in the CSI compression module and the related decompression module.

The proposed LMU-GRUs dimension compression and decompression module, as shown in Table 3, reduces the number of parameters by over 100 times and the number of FLOPs by at least 4 times [17]. The comparison results show that our new LMU-GRUs architecture for CSI compression and decompression is a significant step forward in expanding the range of practical applications for deep learning-based CSI encoding, feedback, and reconstruction in massive MIMO wireless systems.

4. Conclusion

This chapter compares a channel state information (CSI) feedback network by extending the DL-based CsiNet technique to incorporate GRUs and LMU over GRU layers. The proposed LMU-GRUs technique achieved the number of parameters and MACCS compared to other CS-based and CSI-based methods. The work is motivated by the active use of the recurrent convolutional neural network (RCNN) model in model size and computational complexity of tested networks using a channel state information network. The basic concept is to compare the complexity analysis in deep learning techniques using the COST 2100 model to get results related to indicators parameters in time-varying MIMO channels and acquire training samples.

5. Recommendations for future work

Quantum-assisted Deep Learning (QDL) is receiving significant attention towards enhancing various performance metrics of communication networks. The classical DL faces various challenges; where a substantial challenge is to figure out the training method for complex topologies of neural networks (NNs) (which are of similar complexity to that of the natural structure of the human brain).

Acknowledgments

First of all, I thank ALLAH for giving me the will to achieve this work.

It is a great honor for me to take this opportunity to express my deep gratitude to Dr. Sherif El Dyasti, Assistant Professor, Electronics and Communication Department, College of Engineering and Technology, Arab Academy for Science, Technology and Maritime Transport (AAST), for his excellent cooperation, his expert help, continuous encouragement and valuable effort for completion of this work.

My special thanks and appreciation to Prof. Hazem Shatila, Virginia Polytechnic Institute and State University, Professor of Artificial Intelligence & Markovdata, CEO, thanks for spending his precious time and for his continuous encouragement that was behind the completion of this work.

Conflict of interest

“The authors declare no conflict of interest.”

Appendices and nomenclature

AI	artificial intelligence
AMP	approximate message-passing
AoA	analysis of alternatives
AE	autoencoder
BER	bit error rate
CE	channel estimation
CNN	convolutional neural network
CS	compressive sensing
CR	compression ratio
CSI	channel state information
CsiNet	channel state information network
CsiNet-GRU	channel state information network-gated recurrent unit
DL	deep learning
DNN	deep neural network
FDD	frequency division duplex
GRU	gated recurrent unit
LMU	Legendre memory unit
LASSO	least absolute shrinkage and selection operator
LSTM	long short-term memory
MIMO	multiple-input multiple-output
mmWave	millimeter wave
MSE	mean squared error
NMSE	normalized mean square error
RELU	rectified linear unit
RNN	recurrent neural network
SNR	signal-to-noise-ratio