Ground-Based HPA Pre-Distorter Using Machine Learning and Artificial Intelligent for Satellite Communication Applications

Abstract

This chapter describes an innovative design and implementation approach of a ground-based pre-distorter framework using machine learning and artificial intelligence (ML-AI) technology for high power amplifier (HPA) pre-distortion. The ML-AI technology enabler proposed is a combined multi-objective reinforce learning-and-adaptive neural network (MORL-ANN) and an operating environment predictor (OEP). The proposed framework addresses the signal distortions caused by a nonlinear HPA on the ground transmitter and a nonlinear HPA located at a satellite communication (SATCOM) transponder (TXDER). The TXDER’s HPA is assumed to operate under unknown conditions. The objective is twofold, namely, to demonstrate (i) an advanced decision science technique using ML-AI for future SATCOM applications and (ii) the feasibility of the proposed ground-based ML-AI framework using an end-to-end SATCOM emulator. A new OEP concept using a deterministic and Bayesian approach to improve the MORL-ANN pre-distorter (PD) performance will also be presented.

1. Introduction

The topic on HPA linearization and the use of machine learning and artificial intelligence (ML-AI) for the high-power amplifier (HPA) linearizer has been investigated in the recent past [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. These works were focused on the linearizer that are usually placed before the HPA and applicable to either a satellite or a ground system with the HPA operating at saturation. This chapter addresses the HPA linearization of an end-to-end satellite system with an uplink (U/L) signal to a satellite transponder (TXDER) and a downlink signal (D/L) from the TXDER. The proposed ground-based ML-AI HPA pre-distorter concept is intended to place on the ground tracking station and before the ground transmitter’s HPA. The novelty of our proposed ground-based ML-AI is to linearize the combined amplitude and phase distortions caused by both the ground and satellite TXDER HPAs.

In practice, a typical SATCOM system includes a U/L and a D/L signals. The U/L signal transmits from a transmit (TX) terminal to a satellite TXDER. The D/L signal transmits from the satellite TXDER to a received (RX) terminal. The TX and RX terminals can be on the ground or an airborne or a navy ship. For the sake of discussion, this chapter assumes it is a fixed ground terminal. For this scenario, the U/L signal is corrupted by the U/L propagation environment including weather, propagation path loss, and U/L radio frequency interferences (RFI). The signal passing through the satellite TXDER is corrupted by transponder processing noise and distortions caused HPA nonlinearity. The D/L signal is also corrupted by the D/L propagation environment including weather, propagation path loss, and D/L RFI. The combined distortion can cause serious Bit Error rate (BER) performance degradation to the received signal on the ground. The use of ML-AI technology to combine data science and decision science (a.k.a. data and decision sciences) can address these challenges. ML-AI can be used to observe the D/L signal amplitude and phase distortions behavior from the ground. It can also be used to predict the amount of distortions for signal compensation before the uplink transmission. In this context, observing the received signal and collecting the received data for predicting the signal distortion behavior is an application of data science. And deciding how much distortion for uplink signal compensation is an application of decision science. Therefore, this is a combined data and decision science technologies.

Through the Industrial Project for Graduate Program in Applied Mathematics (IPGPAM), a collaboration project between CSUF and Intelligent Fusion Technology (IFT) was initiated in 2019 to address this combined data and decision science technologies. During the 2019–2020 academic year, a CSUF team consisting of five graduate students and two faculty members at CSUF was formed to collaborate with the IFT team through this joint industry and university project. This project focused on the advanced mathematical modeling and simulation aspect of the ground-based ML-AI framework for future SATCOM applications. The IFT team provided the end-to-end SATCOM System Model (E2E-SSM) emulator as a platform for demonstrating the newly proposed ground-based ML-AI framework. The CSUF team was responsible for the development of a ground-based HPA pre-distorter (PD) to compensate for the amplitude-to-amplitude and amplitude-to-phase modulation (AM-AM/AM-PM) distortions caused by the HPA nonlinearity and imperfect satellite onboard processing. The problem became more complicated due to unknown operating conditions associated with the satellite system operations, along with the U/L and D/L RFI environment. The RFI can be friendly and unfriendly. Friendly RFI sources are from neighboring satellites using the same RF or the RF near the victim’s RF. Unfriendly sources are from adversary jammers. As discussed in [1], for unknown operating conditions, the existing ground-based ML-AI frameworks [2, 3, 4] using MORL-ANN require a very large amount of environmental data for all practical operating conditions for training purposes. Thus, with limited training data, trial and error learning-based processes such as MORL-ANN may not be practical in real satellite communication systems where actual operational conditions are varied and at times can be unpredictable. In addition, the use of MORL-ANN described in [2] can potentially run into “bottle-necks” without having proper training data under an unknown Operational Environment (OE). Also, based on our past simulation results, we have observed that MORL-ANN usually performs very well under a controlled operational environment, where OE conditions, such as system temperatures and propagation loss, are fluctuating predictably and well within the norms. However, when the OE conditions change extensively and rapidly, such as unpredictable RFI power and Total Electron Content (TEC) changing abruptly causing RX signal scintillation, the MORL-ANN might not perform well. As discussed in [1], to address the unknown and uncertain operational environment, our proposed ML-AI framework monitors the received Signal of Interest (SOI) in real time and uses OEP to estimate the operating conditions for reducing uncertainties associated with the observed data before applying MORL-ANN. This proposed technique helps to reduce the amount of data required for training the pre-distorter and avoid the above-mentioned bottleneck.

The CSUF-IFT team has successfully implemented and demonstrated the newly proposed ground-based ML-AI framework addressing satellite TXDER’s distortion under unknown HPA operating temperatures and operating Input Back-Off (IPBO). The implemented framework uses the IFT’s E2E-SSM emulator as an end-to-end platform [1]. The E2E-SSM emulator includes a sophisticated frequency hopping (FH) satellite modem (modulator-demodulator) and a satellite TXDER model that was verified and validated with existing global broadcasting satellite transponder and global wideband satellite transponder models. This chapter provides a summary of the work performed by the joint CSUF-IFT team. The chapter is organized as follows:

• Section 2 provides a description of the newly proposed ML-AI framework for the unknown operating environment and associated ground-based ML-AI and OEP components.

• Section 3 provides an overview of the E2E-SSM emulator provided by the IFT team.

• Section 4 presents an approach for implementing MORL-ANN for combating AM-AM and AM-PM distortions associated with HPA nonlinearity and unknown operational environment conditions.

• Section 5 discusses approaches for implementing OEP models to reduce uncertainties associated with operational environment conditions.

• Section 6 discusses E2E-SSM simulation results demonstrating the proposed ground-based HPA pre-distorter concept using combined MORL-ANN and OEP for improving performance under unknown TXDER’s operational temperature and HPA operating IPBO.

• Section 7 provides the conclusion and way-forward.

2. Proposed ground-based ML-AI framework for unknown operating environment

The newly proposed ground-based ML-AI framework for the unknown operating environment was developed by the CSUF team and discussed in [1]. Figure 1 illustrates a simplified version of the framework presented in [1]. The ground-based ML-AI framework consists of the following key components:

• FH Transmitted (FH-TX) Terminal: The IFT team developed and provided a MATLAB model capable of simulating square root raised cosine (SRRC) pulse shaping filter QPSK frequency hopping signal at S-band, X-band, and Ka-band frequencies. Section 3 provides a detailed description of this FH-TX model.

• Satellite TXDER Model: IFT also developed and provided a MATLAB wideband satellite TXDER model capable of processing S-band/X-band/Ka-band channelization under imperfect onboard processing conditions. A detailed description of the imperfect onboard processing conditions is also presented in Section 3.

• FH Receiver (FH-RX) Terminal: IFT also supplied the MATLAB FH-RX terminal model. The model is capable of demodulating the hopped frequency signal and recovering the transmitted bits. Section 3 also provides detailed description of this FH-TX model.

• MORL-ANN Module: The CSUF team is responsible for the design and MATLAB implementation of the newly proposed MORL-ANN module. A detailed description and implementation of this model will be discussed in Section 4.

• Operational Environment Predictor (OEP) Module: The CSUF team is also responsible for the design and implementation of the newly proposed OEP module in MATLAB. Section 5 describes the approach for this module.

Figure 1.

Proposed ML-AI framework for unknown operating environment.

3. E2E-SSM model using FH MODEM

Section 3 provides an overview of the MATLAB models for FH-TX terminal, satellite TXDER model, and FH-RX terminal of the E2E-SSM provided by the IFT team. These MATLAB models serve as the backbone of the proposed ML-AI framework providing an accurate E2E-SSM emulator for generating and collecting SATCOM data at the FH-RX terminal under various operating conditions of interest. The data collection part of this project is thought of as the data science aspect of this problem. For example, what type of data needs to be collected, what actual operating conditions we need to set the E2ESSM emulator, how to arrange the data for the decision-making process, etc.

3.1 IFT E2E-SSM using FH MODEM and satellite TXDER

This section presents an overview of the FH modulator–demodulator (MODEM) employed by the emulator and discusses the optimization of processing time allowing real-time simulation. In addition, the training data used for the demonstration of the ground-based ML-AI PD will also be addressed.

Figure 2(a)–(c) provides high-level block diagrams of the IFT MATLAB models for the ground FH-TX terminal, satellite TXDER model, and ground FH-RX terminal, respectively. The details of the QPSK modulator and demodulator can be found in [1]. The QPSK modulator can (i) generate a slow frequency hopping or high-frequency hopping rate by controlling the chip rate and (ii) produce a hopped signal with and without SRRC pulse shaping filter. In addition, the MATLAB ground FH-TX terminal also incorporates an HPA model that can accurately generate AM-AM/AM-PM distortions. The MATLAB satellite TXDER model can accurately generate signal distortions caused by imperfect satellite TXDER components. This includes (i) RF-to-IF (intermediate frequency) down-converter, (ii) analog-to-digital converter (ADC), (iii) digital channelizer, (iv) digital-to-analog converter (DAC), (v) IF-to-RF up-converter, and (vi) onboard HPA operation causing AM-AM/AM-PM distortions. The satellite TXDER model is capable of setting practical satellite operating temperatures, IPBO settings, and the amount of distortions caused by imperfect satellite TXDER components. Note that the IPBO setting controls the amount of HPA AM-AM/AM-PM distortions. The ground frequency FH-RX terminal is capable of de-hopping the signal and demodulating the QPSK signal to recover the transmitted data bits and calculate the bit error rate (BER).

Figure 2.

Block diagrams of IFT MATLAB models: (a) ground FH-TX terminal, (b) satellite transponder, and (c) ground FH-RX terminal.

In Figure 2, let ${\mathit{SNR}}_{\mathit{NU}}$ be the U/L (i.e., from ground FH-TX terminal to satellite TXDER) signal-to-noise power ratio (SNR), ${\mathit{SNR}}_{\mathit{DU}}$ be the D/L (i.e., satellite TXDER to FH-RX terminal) SNR, the intermodulation noise (a.k.a. IM noise) caused by the HPA nonlinearity at the TXDER is characterized by $\mathit{C}/\mathit{IM}$ (a.k.a. carrier-to-IM power ratio), and the overall SNR, ${\mathit{SNR}}_{\mathbf{0}}$, received at the FH-RX terminal, can be shown to have the following form:

$$\frac{1}{{\mathit{SNR}}_O}=\frac{1}{\left(\frac{1}{{\mathit{SNR}}_{\mathit{NU}}}\right)+\left(\frac{1}{{\mathit{SNR}}_{\mathit{ND}}}\right)+\left(\frac{1}{C/\mathit{IM}}\right)}$$E1

Let us assume that there in a U/L RFI with unknown signal-to-interference power, ${\mathit{SIR}}_{\mathit{U}}$, and a D/L RFI with unknown ${\mathit{SIR}}_{\mathit{D}}$, and the overall received ${\mathit{SNR}}_{\mathbf{0}}$ at the FH-RX terminal becomes:

$$\frac{1}{{\mathit{SNR}}_O}=\frac{1}{\left(\frac{1}{{\mathit{SNR}}_{\mathit{NU}}}\right)+\left(\frac{1}{{\mathit{SNR}}_{\mathit{ND}}}\right)+\left(\frac{1}{C/\mathit{IM}}\right)+\left(\frac{1}{{\mathit{SIR}}_U}\right)+\left(\frac{1}{{\mathit{SIR}}_D}\right)}$$E2

Using ML-AI, the FH-RX (see Figure 2(c)) observes the overall received ${\mathit{SNR}}_{\mathbf{0}}$ and predicts the amount of AM-AM and AM-PM distortions caused by the HPA located in the FH-TX terminal (see Figure 2(a)) and HPA located in the satellite TXDER (see Figure 2(b)). The ground-based ML-AI pre-distorter uses the predicted distortions and pre-distorts the transmitted signal to compensate for the combined AM-AM and AM-PM distortions. As shown in Eq. (2), the distortions depend on the IM at the TXDER. The IM level depends on the HPA operating point, TXDER HPA characteristics, and operating TXDER temperature. The HPA operating point is characterized by the IPBO. In addition, the unknown U/L RFI can change the HPA operating point (i.e., IPBO) causing unknown AM-AM/AM-PM distortions. This chapter investigates the performance of the proposed ground-based ML-AI pre-distorter framework shown in Figure 1 in the presence of unknown IPBO, HPA AM-AM/AM-PM characteristics, and TXDER operating temperature.

3.2 Reducing processing time of existing IFT E2E-SSM

As pointed out in Section 3.1, the signal distortion models caused by imperfect satellite TXDER components with the satellite TXDER include amplitude ripple caused by input/output RF filters, phase noise caused by RF up/down-converters, and quantization noise caused by ADC-DAC, AM-AM/AM-PM distortion effects. The current IFT E2E-SSM model requires excessive processing time, rendering it an inability to support real-time simulation or generate large training data for various operating temperatures and HPA IPBO’s. The CSUF graduate students worked on the optimization of the filtering and HPA functions of the IFT E2E-SSM model in MATLAB to (i) reduce processing time which allowed for real-time simulation and (ii) provide for varying HPA operating temperature and IPBO.

3.3 Training data for demonstrating proposed ML-AI framework

The CSUF graduate students fine-tuned the E2E-SSM emulator to allow for real-time simulation using the satellite system parameters as shown in Figure 3(a). Using the selected setup, the team generated E2E BER as captured in Figure 3(b). The E2E BER simulation results represent the observed BER performance of a practical FH-QPSK signal under unknown HPA’s operating conditions. The unknown HPA’s operating conditions considered in the simulation are characterized by the HPA operating temperatures and IPBO as parameters. As specified in Figure 3(a), the HPA operating temperatures and IPBO used in the simulation shown in Figure 3(b) are (i) 25°C, 27°C, and 30°C and (ii) IPBO = 0, 5, 7, 10, 13, and 15 dB, respectively.

Figure 3.

Generating training data: (a) satellite TXDER parameter setting, (b) E2E BER for fast FH as a function of bit signal-to-noise ratio (EbNo) with satellite system temperature and IPBO as parameters.

4. Approach for ground-based satellite system operational environment prediction (OEP)

The team proposes two approaches for predicting the operating environment, namely (i) OEP 1—deterministic environmental prediction and (ii) OEP 2—Bayesian environmental prediction. The goal here is to predict the operating conditions, including HPA’s temperature and IPBO, based on the measured EbNo and BER values observed by the receiver and pass the results to the MORL-ANN pre-distorter (PD) for training and predicting the amount of AM-AM/AM-PM distortions and compensation.

4.1 OEP 1: deterministic approach

Given a set of ${\left\{{\mathit{\text{EbNo}}}_i\right\}}_{i=1}^{N_{\mathit{\text{meas}}}}$, we measure the corresponding ${\left\{{\stackrel{\sim }{\mathit{BER}}}_i\right\}}_{i=1}^{N_{\mathit{\text{meas}}}}$ and compute the mean square error (MSE) based on the previously generated data set using:

$$\mathit{MSE}\left(T,\mathit{\text{IPBO}}\right)=\sum _{i=1}^{N_{\mathit{\text{meas}}}}{\left(\mathit{BER}\left({\mathit{\text{EbNo}}}_i,T,\mathit{\text{IPBO}}\right)-{\stackrel{\sim }{\mathit{BER}}}_i\right)}^2$$E3

The goal is to choose the best temperature $T^{\ast }$ and ${\mathit{\text{IPBO}}}^{\ast }$ that minimizes the least mean square error.

4.2 OEP 2: Bayesian approach

This approach uses the existing MATLAB function “fitcnb,” which fits a native Bayes classifier model. The native Bayes classifier model uses simple probabilistic classifiers based on Bayes’ theorem with strong but naïve independence assumptions between the features of the data.

5. Approach for ground-based MORL-ANN PD

The proposed ground-based MORL-ANN PD (or simply PD) can be implemented using a deep deterministic policy gradient (DDPG) technology enabler or a combined DDPG with deep-Q learning network (DQN). This section describes these two implementation approaches.

5.1 PD implementation using DDPG

The goal for the MORL-ANN PD is to pre-distort the TX signal such that the received (RX) signal is identical to the transmit (TX) signal [3, 4]. The ML-AI technology enabler that is available from MATLAB is the DDPG [5, 6, 7, 8]. The DDPG is suitable for the MORL-ANN training and prediction that involves tuning the parameters of a deep neural network (DNN). As depicted in Figure 4, DDPG is an actor-critic network that is the heart of the proposed ML-AI framework, where the actor observes the received data and decides on required actions, and the critic judges the actions and rewards or penalizes the actions using a pre-defined loss function. The word “deep” in DDPG represents the DNN with two or more hidden layers, “deterministic” means that there is only one-output, “policy” means that the PD has a policy for deciding an action, and “gradient” means that the PD uses gradient of the loss function to update previous values.

Figure 4.

(a) MORL-ANN implementation using DDPG and (b) DDPG actor-critic network block diagram.

DNN is a structure that consists of a sequence of functions (layers), which takes in our state (or a state-action pair) and returns to us an action (or our expected reward). Here, the MORL training occurs in episodes that consist of k steps. A step is a process whereby an action is generated by the agent. The action is processed by the environment, and the resulting reward is returned to the agent. For our MORL-ANN implementation, an episode consists of a single step. The MORL-ANN algorithm is expressed as follows:

$$\text{Form Policy Gradient ∇J}\left(\text{θ): where}\mathrm{J}\right(\text{θ)}=\mathrm{E}\left[{\text{Reward}}_{\text{Episode}}\right]$$E4

where our neural networks are determined by a parameter vector θ representing system operating conditions, such as operating temperature and IPBO

• Calculate ∇J(θ): using Monte Carlo Methods

• Perform a gradient ascent step:

$$\text{θ}\leftarrow \text{θ}+\text{𝛼∇J,}\text{where}0<\text{𝛼}<1\text{is the learning rate}.$$E5

The proposed DNN tuning requires fine-tuning the training parameters, involving:

• Layer size: layer size refers to the output sizes of the fully-connected layers in the networks. It is a vector of length 2.

• Mini batch size: when performing a gradient ascent step for DDPG, we approximate the policy gradient over k data points using Monte Carlo methods. Here, k is called the mini batch size.

• Gradient threshold: when the policy gradient’s Euclidean norm exceeds the gradient threshold, we rescale it so that its Euclidean norm is the gradient threshold. This controls the learning speed in the gradient ascent step.

The “Reward” is defined as the error of the signal amplitudes and error of the signal phase after the HPA is expressed in negative values. For tuning, we take the L2 norm between the post-HPA PD signal and the original transmitted signal from the ground FH-TX terminal. For final training, we will use the L2 norm between the “SlidingBucket” normalized signals for greater accuracy. The “SlidingBucket” is an algorithm that our team developed to emulate the automatic gain control (AGC) to maintain the IPBO level. The IPBO level is updated depending on the selected AGC loop time response (i.e., update rate). The CSUF graduate students¹ spent a tremendous amount of time fine-tuning the training parameters and found an optimum set of training parameters for the final simulation run. The simulation results are shown in Section 6.2.

5.2 PD implementation using combined DDPG and deep-Q learning

The CSUF graduate student team¹ proposed a hybrid concept to use the MATLAB’s DDPG for designing a good MORL-ANN PD and then deep-Q learning network (DQN) to make further corrections. Figure 5 illustrates the proposed concept. The DQN uses a single, smaller neural network. It uses much less memory and can feasibly take many small, discrete actions due to the agent’s smaller size, thereby more efficient for stabilization purposes. Thus, using the initial state prediction provided by OEP concerning the HPA operating temperature and IPBO, the DQN agent will take a much shorter time due to the small number of steps required to reach a desirable stable state. Combining DQN with DPPG can improve the training time and enhance the MORL-ANN PD performance.

Figure 5.

MORL-ANN implementation using combined DDPG-and-DQN.

6. Simulation results

This section provides the simulation results obtained from the IFT E2E-SSM emulator using the training data presented in Section 3.3 to demonstrate the ground-based ML-AI concept to compensate for the AM-AM/AM-PM distortions caused by the ground terminal’s HPA and satellite TXDER’s HPA in the presence of unknown operating conditions.

6.1 Ground-based OEP simulation results

This section presents the simulation results for proposed OEP using deterministic and Bayesian approaches.

6.1.1 OEP simulation results for deterministic approach: OEP 1

Figure 6 shows the results for the deterministic prediction simulation results for a measured BER at 0.001. The results show that the predicted operating conditions with the lowest error are a temperature of 27°C with an IPBO value of 5. Figure 6 presents the simulation results for the deterministic prediction when the measured BER is at 0.01 with a number of measured EbNo values of 2. These results show that only 35 out of 100 trials correctly predict both the operating temperature and IPBO.

Figure 6.

Deterministic prediction simulation results for BER = 0.001.

Figures 7 and 8 capture the simulation results for the deterministic prediction at a measured BER of 0.01 and 0.001, with a numerical measured value of 2 and 4, respectively. For the measured BER of 0.001, the results show that 96 out of 100 trials correctly predicted both operating temperature and IPBO. Thus, when the number of measured EbNo increases, the probability of correctly identifying the operating conditions improves.

Figure 7.

Deterministic prediction simulation results for BER = 0.01 with number of measured EbNo values of 2.

Figure 8.

Deterministic prediction simulation results for BER = 0.001 with number of measured EbNo values of 4.

6.1.2 OEP simulation results for Bayesian approach: OEP 2

Figure 9 provides the results on the probability of classification as a function of EbNo and BER obtained from the Naïve Bayesian classification model. Figure 9 shows that the probability of classification is at its highest, about 0.3, when BER = 0.3 and operating system temperature and IPBO are at 27°C and 15 dB, respectively.

Figure 9.

Probability of classification.

Figure 9 presents simulation results using the Naïve Bayesian prediction approach.

The results were obtained using four measured BER values for predicting HPA’s operating temperature and IPBO. Figure 10 shows that the highest predicted probability sum value is for 25°C with an IPBO value of 10, while the actual value is at 25°C with an IPBO value of 13.

Figure 10.

Probability of classification results for EbNo ranging from 0 to 8 dB.

Table 1 summarizes the results for comparison between the two OEP approaches. As shown in Table 1, the deterministic approach achieves the best performance when the number of measurements is 4 or more and the BER is at 10E-3 or less. However, for the Bayesian classification approach, the probability of classification is inconclusive when increasing the number of measured BERs. Our team expects that the use of a “Kernel” can improve the probability of classification of the operating environment.

	BER error = 10^−3			BER error = 10^−2			BER error = 10^−1
Deterministic
Number of meas	2	3	4	2	3	4	2	3	4
Both correct	87	92	96	35	40	54	8	8	7
Just temperature	2	0	0	15	12	9	31	37	36
Just IPBO	1	0	0	2	7	3	14	7	16
Both incorrect	10	8	3	48	41	34	47	48	41
Naïve Bayesian classification
Number of meas	2	3	4	2	3	4	2	3	4
Both correct	4	3	5	4	2	2	4	2	6
Just temperature	40	28	29	29	27	31	33	28	36
Just IPBO	6	6	3	1	7	8	9	5	5
Both incorrect	50	63	63	66	64	59	54	65	53

Table 1.

Deterministic vs. Bayesian classification.

6.2 Ground-based MORL-ANN Predistorter simulation results

Figure 11 shows the simulation results for MORL-ANN PD using MATLAB’s DDPG algorithm with the initial operating conditions prediction provided by OEP 1 approach. The results show that AM/AM (signal power curve) and AM/PM (signal phase curve) between the TX and RX signals are in good agreement, i.e., the PD provides an accurate prediction of AM/AM-AM/PM distortions and compensates for them. This means any inaccuracy associated with OEP eventually corrects itself through the MORL-ANN training and prediction processes. The results shown in Figure 11 also show that there is a slight disagreement between the actual and predicted AM/AM distortion causing a discrepancy between the TX and RX signals between the Vector Index 150 and 200.

Figure 11.

MORL-ANN PD simulation results using existing DDPG in MATLAB.

Our team has investigated the problem and learned that when using the combined deep Q-learning neural network (DQN) and DDPG during the learning and training process, the MORL-ANN PD performs better with the use of OEP. Figure 12 presents the simulation results using the combined DQN-DPPG approach for mitigating the AM/AM discrepancy between the TX and RX signals.

Figure 12.

MORL-ANN PD simulation results using the combined DDPG and DQN in MATLAB.

Based on the MATLAB implementation, our team has learned that the use of DDPG allowed to use a single, smaller neural network, so it uses much less memory. This implementation can feasibly take many small, discrete actions (due to the agent’s smaller size), so it is more convenient for stabilization. However, given a bad initial state, the agent will take a long time (many steps) or never reach a desirable state. Thus, for our problem, we recognize that the DDPG can train faster and typically produces better results than DQN alone.

7. Conclusion and way forward

This chapter provides a summary of the work performed by the CSUF-IFT team on an Industrial Collaboration Project during the 2019–2020 academic year. The project described in this chapter focuses on the MATLAB implementation and demonstration of the novel ground-based ML-AI framework presented in Figure 1. The proposed MATLAB implementation employs MORL-ANN combined with a deterministic OEP for predicting and compensating signal distortions caused by the ground terminal transmitter’ HPA and satellite TXDER’s HPA, and imperfect onboard signal processing. Preliminary results presented here have demonstrated the (i) feasibility of the proposed deterministic OEP for reducing operating environment uncertainties associated with unknown satellite TXDER’s HPA operating temperature and IPBO, and (ii) use of MORL-ANN using DPPG and MORL-ANN using combined DQN-DPPG for compensating of AM-AM/AM-PM distortions caused by combined ground station’s HPA and satellite TXDER’s HPA.

Considering the preliminary OEP simulation results, it has been shown that the current proposed environment predictor using Bayesian approach is inconclusive. The CSUF-IFT team continues to investigate the use of ML-AI technology to improve the Bayesian OEP.

Last but not least, the chapter has proposed an approach to combine the data science and decision science to solve a challenging problem in satellite communication in the presence of unknown operational conditions. Our team has developed an end-to-end SATCOM system model (E2E-SSM) emulator to generate a large amount of data for various practical operational conditions, including unknown IPBO, system operating temperature, HPA’s AM-AM/AM-PM characteristics, and SNR. Using the data obtained from the emulator, the team has also developed an innovative ML-AI framework to (i) learn the behavior of amplitude and phase distortions of the received downlink signal, (ii) predict the amount of amplitude and phase of the transmitted uplink signal, and (iii) and adjust them accordingly for negating the effects of the HPA nonlinearity on the end-to-end communication signals. Our simulation results presented in this chapter has demonstrated the feasibility of these proposed combined technologies. Our team is also investigating the use of the proposed ML-AI pre-distorter for future Global Navigation Satellite System (GNSS) applications. The results of these investigations will be reported in the near future.

Acknowledgments

IFT was awarded the SBIR Phase 1 in 2019, and contract number was FA9453-18-P-0233 from AFRL, entitled “Hybrid Low-Cost Predistortion Solutions for Next Generation Satellite Transponder,” as the prime contractor. The results presented in this chapter were done during Phase 1 (toward the end of Phase 1) through the Industrial Project for Graduate Program in Applied Mathematics (IPGPAM), which is a joint collaboration project between CSUF and IFT. Recently, IFT was awarded the SBIR Phase 2 contract, Contract No. FA9453-21-C-0539 from AFRL. Additionally, CSUF was awarded a subcontract from IFT, Subcontract Number: IFT083-02. The subcontract focused on maturing the proposed ML-AI technology enabler and OEP for future GNSS applications. The paper was cleared for public release, Case No. AFRL-2022-5077. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of AFRL.

Notes/thanks/other declarations

The authors want to thank Maria Rios of IFT for her careful review of the paper. The first author wishes to thank his wife, Thu-Hang Nguyen, for her enormous patience and boundless support during the preparation of this chapter.