Spatial-Temporal Data Analysis in Nonlinear System

Abstract

Spatial-temporal analysis is at the heart of data mining in Big Data Era. Most mathematical tools are incompetent to deal with spatial-temporal data. This phenomenon has greatly spurred the development of data science, especially in the field of BDA (big data analytics). This chapter proposes random matrix theory (RMT) to handle this problem, which begins by modeling spatial-temporal datasets as sequences, whose term is in the form of a random matrix each. Then, some fundamental RMT principles are briefly discussed, such as asymptotic spectrum laws, transforms, convergence rate, and free probability, in order to extract high-dimensional statistics from the random matrix as the indicators. The statistical properties of these indicators are discussed for a better understanding of the system. Finally, some potential application fields are given.

1. Introduction

Electric power system reliability and intelligent management are critical to our daily living. Engineers and academics have recently focused on the use of large-scale phase measuring units (PMUs) to improve wide-area monitoring, protection, and control [1, 2, 3, 4, 5].

Most existing algorithms in power grid are model-based, which are built upon mechanism assumptions/simplifications and linear system control theory, with a determined and typically analytic outcome. These models, however, are ineffective for today’s power system, which is of ever-increasing complexity and uncertainty [6, 7, 8, 9, 10]: 1) Interconnection of nearby utilities may frequently improve overall safety and efficiency, resulting in a huge interconnected system, such as the North American Power Grid, which serves almost 400 million customers throughout the continent [11]; 2) the continuous penetration of cell units (e.g., distributed generations) that are small-size, large-number, distributed-deployment, diverse-behaviors, smart-response, and uncertain-control [12]. 3) physical disciplines (mechanics, magnetism, electric, and electronics) of a system are closely intertwined, especially in a CHP (combined heat and power) system or even IES (integrated energy system) [13]; and 4) the construction of energy foundation for a smart city. Those above characteristics are advantageous to an open, flat, nonlinear, high-uncertainty, and distributed EIoT, as shown in Figure 1 [14]. For such an EIoT, a precise mechanistic model or even a proper descriptive representation can hardly be formulated, let alone model-based linear mode.

Figure 1.

Diagram of future energy internet of thing: its resource flow, data flow, and participants [14].

Furthermore, engineering data, such as sampling data in a power system, is not similar to image data. Various sensors, such as phasor measuring units (PMUs), are used to sample data from the grid. The huge dataset is in a high-dimensional vector space and in time series: the temporal variations (T sampling instants) and spatial fluctuations (N grid nodes) are recorded concurrently, and hence it is called spatial-temporal data.

Most mathematical tools are incompetent in this task [15]. Facing the above spatial-temporal data, we can hardly extract statistical information, particularly spatial-temporal correlations; the high-dimensional structure does not match the requirements of most traditional mathematical methods. Also, this task is incompatible with supervised training algorithms such as neural networks, due to the lack of or asymmetry of massive labeled data [16].

Fortunately, random matrix theory (RMT), by unifying time and space through their ratio c = T/N, can strictly and mathematically deal with such data. Moreover, linear eigenvalue statistics (LESs) built from data matrices follow Gaussian distributions for very general conditions, and other statistical variables are studied due to the latest breakthroughs in probability on the central limit theorems of those LESs.

2. Spatial−Temporal data analyzation mode, tools, theory, and applications in electric power system

2.1 Big data era, fourth paradigm, and data-driven model

The world’s science has altered, as seen in Figure 2 [17]. Initially, there was only experimental science, followed by theoretical science, which included Newton’s Laws, Maxwell’s equations, and so on. The theoretical models became too hard to solve analytically for many issues, and people had to start simulating. These simulations have carried us through much of the previous millennia. People nowadays are collecting data through intensive sensors or simulations. The data flood has an impact on experimental, theoretical, and computational science, and several state-of-the-art data technologies and data sciences have converged to provide tremendous promise for data-intensive scientific discovery, the so-called Fourth Paradigm.

Figure 2.

Science paradigms and fourth paradigm [17].

Data-driven becomes a natural and stressful topic in energy systems, as evidenced by IEEE TRANSACTIONS ON SMART GRID special issue “big data analytics for grid modernization” published in 2016 [18]. Data-driven approaches are also characterized as model-free; we no longer rely significantly on physical models, and hence can manage instances where physical parameters are incorrect or even totally unavailable. Data-driven mode enables a quick start to our task, especially for a modern energy system in which the behaviors and discipline of system cell units are strongly coupled.

2.2 Basic of spatial−Temporal data, and high-dimensional information

Spatial–temporal data analysis means that we simultaneously deal with a large number of variables (in N-dimensional spatial space), and each variable (i = 1, …, N) samples time series for a duration (in T-dimensional temporal space). A classical statistic theory treats fixed N only (typically N < 6 [3]), e.g., for ABC-dq0 transformation N = 3. This fixed small N is called the low-dimensional regime. In practice, we are interested in the case that N can vary arbitrarily in size compared with T (typically T > 60, N > 20, and c = N/T > 0 [15]). This fundamental difference is the primary motivation for studying BDA.

Spatial–temporal data mining is expected to contribute some (high-dimensional) information with domain-specific meaning attached as the supplement to DT-based situation awareness (SA). High-dimensional indicators (outputs of high-dimensional statistics) and deep features (outputs of deep learning) are two main types of representation of high-dimensional information.

2.3 Spatial-temporal data utilization architecture and tools

Most mathematical methods struggle to extract information from spatial-temporal data. This phenomenon has accelerated the development of data science, particularly in the field of AI and BDA. We describe a high-focus technique for each field: 1) DL (Deep learning), which is good at massive data modeling in AI [19] and 2) High-dimensional statistics, or more precisely, RMT, which does well in data analytics in BDA. Both tools use a set of (high-dimensional) methodologies for integrated spatial-temporal modeling and analysis, and they have already made profound impacts on many domains. Figure 3 depicts the architecture of large data mining based on DL and RMT.

Figure 3.

Architecture of spatial-temporal data utilization.

2.3.1 Deep learning and its advantages

DL is a cutting-edge data mining algorithm. As demonstrated in Figure 4, deep characteristics are learned at some level from comparatively hidden features in the hierarchy [20]. DL uses the enormous data in a non-handcrafted approach to create a deep (nonlinear) network model. A typical ANN (Artificial Neural Network) network is modeled as

Figure 4.

A typical ANN structure.

$$y=f\left(x\right)\triangleq f^L\left(W^L\dots f^2\left(W^2{f}^1\left(W^{1}x+b^1\right)+b^2\right)\dots +b^L\right)$$E1

The above DL network model can be built with little prior knowledge relevant to the physical mechanism or causal relationship. As a result, DL may be used in a variety of situations or even systems without major changes. For example, we use CNN (convolutional neural networks) for computer version system modeling [21], LSTM (long short-term memory) for prediction [22], and deep reinforcement learning for strategy optimization [23].

In a complicated system, DL has a competitive edge in terms of possible data use. Furthermore, the test error might be used to quantify the DL model’s performance on the generalization task, ensuring its usefulness in a real-world situation.

DL holds a competitive advantage for feasible data utilization in a complex system. In addition, the performance of the DL model on generalization task could be quantitatively evaluated by the test error, ensuring its usefulness in a real-world situation.

2.3.2 Big data analytics and RMT and the advantages

BDA uses spatial-temporal joint analysis to acquire high-dimensional statistics. Matrix-based variables, such as eigenvalue or the matrix variate itself, are likely to provide some insight to BDA [24]. These matrix-based variables are the variables of the N × T (large-dimensional) spatial-temporal data matrix that have an intrinsic statistical link, whether causal or not. These matrix-based variables are analytically intractable due to their high dimensionality rather than their big size. RMT is inextricably linked to this issue.

RMT understands the joint eigenvalue distribution as the statistic analytics in the asymptotic regime. In particular, by unifying time and space through their ratio c = T/N, BDA is acquired as the functionals of the eigenvalue distributions. For example, the matrix’s LES indicators [25] follow Gaussian distributions for very general conditions. Furthermore, many LES-derived variables, whose statistical properties are mostly derivable and provable, are studied due to the latest breakthroughs in high-dimensional probability [15]. In this sense, RMT is rigorous and fundamental in nature. Besides, RMT performs well with only moderate-size (unlabeled) data, which is often true for a domain-specific problem in EIoT.

2.4 Random matrix theory in a nutshell

2.4.1 RMT and its universality principle

Two ensembles, Gaussian unitary ensemble (GUE) and Laguerre unitary ensemble (LUE), are studied first in RMT [10]:

$$\mathbf{\Gamma }=\left\{\begin{array}{c}\frac{1}{2}\left(R+R^H\right),R\in {\mathrm{ℝ}}^{N\times N},\mathrm{GUE};\\ \frac{1}{T}{\mathit{RR}}^H,R\in {\mathrm{ℝ}}^{N\times T},\mathrm{LUE}.\end{array}\right.$$E2

where R is i.i.d. standard Gaussian random matrix.

We investigate the rate of convergence of the expected empirical spectral distribution (ESD) of Γ. Let hΓ (x) denotes the true eigenvalue density. Wigner’s Semicircle Law and Wishart’s M-P Law, respectively, for GUE and LUE, say that

$$h_{\mathrm{\Gamma }}\left(x\right)=\left\{\begin{array}{c}\frac{1}{2\pi }\sqrt{4-x^2},x\in \left[-2,2\right],\mathrm{GUE}\\ \begin{array}{cc}\frac{1}{2\mathit{\pi cx}}\sqrt{\left(x-a_1\right)\left(x-a_2\right)}& ,x\in \left[a_1,a_2\right],\mathrm{LUE}\end{array}\end{array}\right.$$E3

where $a_1={\left(1-\sqrt{c}\right)}^2,a_2={\left(1+\sqrt{c}\right)}^2$

Universality principle enables us to perform hypothesis tests under the assumption that the matrix entries are not Gaussian distributed but use the same test statistics as in Gaussian case. Numerous studies using field data [25, 26] demonstrate that M-P Law is universally valid with moderate matrix sizes, such as tens. This is the very reason why RMT is widely used in engineering.

2.4.2 Linear eigenvalue statistics and its properties

Consider a random matrix Γ∈R^N × T, and M is the covariance matrix M = 1/TΓΓ^H. The LES τ of Γ is defined in [27].

$${\tau }_{\phi }=\sum _{i=1}^N\mathrm{\phi }\left({\lambda }_i\right)=\mathrm{Tr}\phi \left(\mathbf{M}\right),$$E4

Law of Large Numbers tells us that N⁻¹τ_ϕ converges in probability to the limit

$$\underset{n\to \infty }{lim}\frac{1}{N}\sum _{i=1}^N\mathrm{\phi }\left({\lambda }_i\right)=\int \mathrm{\phi }\left(\lambda \right)\mathrm{\rho }\left(\lambda \right)\mathrm{d}\lambda$$E5

where ρ(λ) is the probability density function, which is given in Eq. (4). Therefore, we deduce that

$${\tau }_{\phi }=\sum _{i=1}^N\mathrm{\phi }\left({\lambda }_i\right)=\mathrm{Tr}\phi \left(\mathbf{M}\right)=N\int \mathrm{\phi }\left(\lambda \right)\mathrm{\rho }\left(\lambda \right)\mathrm{d}\lambda$$E6

The Central Limit Theorem (CLT) for LES is studied as the natural second step:

$$\begin{array}{l}{\sigma }^2\left({\tau }_{\phi }\right)=\frac{2}{c{\pi }^2}\underset{-\frac{\pi }{2}<{\theta }_1,{\theta }_2<\frac{\pi }{2}}{\iint }{\psi }^2\left({\theta }_1,{\theta }_2\right)\left(1-sin{\theta }_{1}sin{\theta }_2\right)d{\theta }_{1}d{\theta }_2\\ +\frac{{\kappa }_4}{{\pi }^2}{\left({\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}\phi \left(\zeta \left(\theta \right)\right)sin\theta \mathit{d\theta }\right)}^2\end{array}$$E7

See [25] for details.

2.4.3 LES-based hypothesis testing for random matrix

LES τ, as a positive scalar random variable defined in Eq. (5), is studied instead of the probability distribution of eigenvalues in Eq. (4). It can be viewed as a mathematically rigorous dimensionality reduction—the N × T random matrix is reduced to a positive scalar random variable.

As N → ∞, the asymptotic limit of LES τ expectation and variance, i.e., E(τ) and σ² (τ), is given, respectively, in Eqs. (7) and (8). These two equations are sufficient to study the scalar random variable τ. Universal principle, as well as engineering experiences, demonstrate that moderate values of N and T are accurate enough for our practical purposes. LES τ is robust against data flaw and insusceptible to noises [10]. All of these statistical properties make LES a good SA indicator.

2.5 High-dimensional situation awareness indicator system and its properties

According to Eq. (5), numerous LESs can be designed from a certain spatial-temporal data Γ. Similarly, other high-dimensional indicators, for instance, statistic indicators, deep features, and electrical features, are calculable as the outputs of data tools according to Figure 4. They are tied together to provide an insight into domain-specific SA criteria for detection, prediction, etc. The details about the high-dimensional SA indicator system and its successful application cases can be found in ref. [15].

With these indicators, the high-dimensional indicator system is built; it supplies a multiple view angle to gain insight into the system. Aiming to provide a domain-specific SA task, the test function ϕ plays a role as a flexible filter depending on our task. Table 1 lists the properties of LES indicators and makes a comparison with classical ones.

High-dimensional (LES) Indicators	Classical indicators
Data-driven	(Mechanism) model based
Supported by data science	Supported by physical laws or experience
Maybe unclearly defined in engineering	Clearly defined
Often probabilistic value	Often determined one
Often in high dimensions	Often in low dimensions
Able to harness the spatial-temporal data flexibly	Only a few data are available
Robust against bad data and insensitive to data selections	Susceptible to data selections (usually a single measurement at a time slice)
Pure statistical procedure	System errors are inevitable
Naturally coupling/decoupling for data block	Coupling/decoupling based on assumptions and simplifications
Random errors can be estimated with the model size (N, T)	Errors accumulation are inevitable and difficult to evaluate

Table 1.

High-dimensional indicator system for EIoT.

Table 1 tells that LES provides a better indicator system in the 4th Paradigm. The relation of the LES indicators to the classical ones, in some sense, is just like that of quantum physics to the classical one. Comparing experimental values with ideal theoretical values, LES conducts SA in a complex system statistically.

In short, RMT supplies us with a data-driven approach to indicator extraction for the informatization of a real system via sampling spatial-temporal data. A cluster of statistical indicators, via a mathematical procedure, is formed as a new epistemology for the system. Some advantages—such as data-driven and model-free mode, theoretic guided, fast in speed, reasonableness, sensitivity, flexibility, and robustness against bad data—have already been shown in our previous work [10, 17].

2.6 LES-based hypothesis testing for random matrix

To study the convergence as a function of N, we study LES instead of the probability distribution of eigenvalues in Eq. (4). For an arbitrary test function with enough smoothness, LES τ (see it as a random variable Y) is a positive scalar random variable defined in Eq. (5). As N → ∞, the asymptotic limit of its expectation, E(Y), is given in Eq. (6), and the asymptotic limit of its variance, σ₂ (Y), is given in Eq. (7). These two equations are enough to study the scalar random variable Y. This approach can be viewed as a dimensionality reduction—the random data matrix of size N × T is reduced to a positive scalar random variable Y! This dimension reduction is mathematically rigorous only when N, T → ∞, but N/T → c. Experiences demonstrate, however, that moderate values of N and T are accurate enough for our practical purposes. Moreover, our previous work shows that LES is robust against data errors (e.g., data loss, data out-of-synchronization) and insusceptible to (independent) random noises (not limited to white noises), which is not true to those low dimensional statistics, such as mean and variance of any single variable. All these statistical properties make LES a good matrix-based variable for a hypothesis testing design aiming to provide anomaly detection task.

We formulate the hypothesis testing in terms of the statistical properties of LES. Referring to the Gaussian property and standard scores, the detection is modeled as a binary hypothesis testing: the normal hypothesis H₀ (no anomaly present) and the abnormal one H₁, denoted by:

$$\begin{array}{l}{\mathrm{\mathscr{H}}}_0:\left|\frac{{\tau }_{\phi }-\mathbb{E}\left({\tau }_{\phi }\right)}{\sigma \left({\tau }_{\phi }\right)}\right|<ϵ,\\ {\mathrm{\mathscr{H}}}_1:\left|\frac{{\tau }_{\phi }-\mathbb{E}\left({\tau }_{\phi }\right)}{\sigma \left({\tau }_{\phi }\right)}\right|\ge ϵ,\end{array}$$E8

where $ϵ$ is a threshold value that needs to be preset—e.g., at a significance level of 0.05, the $ϵ$ should be set at 1.96.

3. Conclusion

This chapter, motivated for the future’s electrical grid, studies the nonlinear analysis based on RMT. Three ingredients are discussed in detail: 1) data modeling—modeling the spatial-temporal data as a sequence of random matrices, which are naturally connected to RMT. 2) data analytics—conducting high-dimensional analysis to obtain the statistical indicators. 3) interpretation—interpreting the indicator by studying its properties for a better understanding of the system.

The experimental indicators, which are fully derived from the sampling data, are applicable to various engineering functions. For example, by comparing the LESs with their theoretical prediction, anomaly detection can be implemented.

Future research directions include: (1) Model validation with different implementations of the grid, ranging from statistic, dynamic and real-world systems; (2) Data fusion with a number of random data matrices, using mathematical tools such as free probability; and (3) The use of Gaussian random matrices in replacement for general data matrices that are obtained from the electrical grid. The universality principle of RMT says that this replacement causes negligible errors.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (Grant No. 51907121).