Symplectic Geometry and Its Applications on Time Series Analysis

This chapter serves to introduce the symplectic geometry theory in time series analysis and its applications in various fields. The basic concepts and basic elements of mathematics relevant to the symplectic geometry are introduced in the second section. It includes the symplectic space, symplectic transformation, Hamiltonian matrix, symplectic principal component analysis (SPCA), symplectic geometry spectrum analysis (SGSA), symplectic geometry mode decomposition (SGMD), and symplectic entropy (SymEn), etc. In addition, it also briefly reviews the applications of symplectic geometry on time series analysis, such as the embedding dimension estimation, nonlinear testing, noise reduction, as well as fault diagnosis. Readers who are familiar with the mathematical preliminaries may omit the second section, i.e. the theory part, and go directly to the third section, i.e. the application part.


Introduction
From the viewpoint of mathematical systems, the time series observed in physics are usually regarded as coming from the Lagrangian systems, also called the conventional systems.The systems can be analyzed by the conventional Euclidean geometry [1].However, the systems in practice are usually nonlinear and complex.Thus, a lot of interesting time series in nature are complex due to nonlinear phenomena derived from nonlinear dynamical systems [2].The nonlinear dynamical systems have been described by Hamiltonian systems and dealt with by using symplectic geometry [3].Symplectic geometry is an even dimensional geometry living on even dimensional spaces.Different from the conventional Euclidean geometry that measures 1-dimensional lengths and angles, the symplectic geometry studies the metric properties (such as area) and can preserve the system structure in the phase space [4].Apart from applications on the classical dynamical systems to solve the equation problems, symplectic geometry has been also used on the studies of nonlinear time series [5][6][7][8].
According to Takens' embedding theorem, a time series can be reconstructed into an attractor in phase space [9].The reconstructed attractor is a geometrical object that can reflect the underlying dynamical system.In order to better understand the nature of the underlying system, the attractor and its properties are characterized in the phase space by various mathematical methods, such as dimension, fractal geometry, Lyapunov exponent, entropy and symplectic geometry [1,5,10,11].For dimension, fractal geometry, Lyapunov exponent, entropy, there are a more extensive discussion with mathematical details in some research literatures [12][13][14][15].Here, we only introduce how to apply symplectic geometry theory to extract the information from the reconstructed attractor and its application on physics, engineering and biomedical engineering.

Mathematical fundamental 2.1 Reconstruction of the system dynamics in phase space from a time series
The reconstruction from a time series of observation is the first and most crucial step in nonlinear time series analysis.It is also the basis of applications of symplectic geometry on time series analysis.Takens' embedding theorem allows us to reconstruct an equivalent attractor of the underlying dynamical system by embedding one time series.The theorem proves that the reconstructed attractor has the same dynamical characteristics as the attractor of the original system if the embedding dimension m is sufficiently large.Let a time series of observation x 1 , x 2 , …, x n .n is the number of samples.The reconstructed attractor can be given in N-dimensional space R N by the time-delay embedding [5]: where the number of dots in the attractor is m = n-N + 1, the embedding dimension is N. X is also called as the trajectory matrix of the original system in phase space.The corresponding program is given by matlab software as follows: - ------------------------------function matrixSignal = signalMatrix(x, N) % ----Construct data matrix----% % Synopsis: % matrixSignal = signalMatrix(x, N) % % Description: % It constructs a data matrix from a time series as a column vector, i.e., a % reconstruction attractor.% % Input: % x a time series with the length n.

Hamilton matrix from the reconstructed attractor
In the symplectic spaces, Hamiltonian system is the analysis fundamental for the real physical processes [4,5].A real system should be first described by a suitable Hamiltonian system, i.e. an even dimensional matrix.For a time series, its Hamiltonian matrix H can be defined by using its reconstructed attractor X. Definition 2.1 Let X be a d-dimensional matrix in a real number field R d .The matrix Xcan be given by removing the mean values of the columns of the X.We define the covariance matrix A of the matrix X: Here, A is a d Â d real number matrix.Definition 2.2 For a d Â d matrix A, the Hamiltonian matrix H can be defined: Here, H is a 2d Â 2d matrix.

Mathematical preliminaries in symplectic geometry
Symplectic geometry focuses on the study of area measure in symplectic space R 2n .Its basic concepts and basic properties are related but different from those of a Euclidean geometry (see Table 1).
In Euclidean space, the inner product is denoted as the measure of the length.The unit matrix is I, i.e. the main diagonal elements are 1, and the other elements are 0. Corresponding to the unit matrix I in Euclidean space, the unit matrix in symplectic space is defined as the unit symplectic matrix J, an even dimensional matrix: The properties of the matrix J have: Definition 2.3 For any two n-dimensional vectors x 2n Â 1 and y 2n Â 1 , the normal symplectic inner product is defined by using the inner product of Euclidean space: The normal symplectic inner product is also denoted briefly as the symplectic inner product in a real vector space R 2n .When n = 1, there is:

Space dimension 2n-dimension n-dimension
Unit matrix unit symplect matrix: unit matrix:

Analysis matrix
Hamiltonian matrix H H T = JHJ.

Eigenvalues of the matrix
The eigenvalues of H are AEμ.The eigenvalues μ of A are real.

Eigenvectors of the matrix
The eigenvectors of H are symplectic orthogonal.
The eigenvectors of A are orthogonal.

Table 1.
The comparison between symplectic geometry and Euclidean geometry.
x, y The symplectic inner product is a bilinear antisymmetric nonsingular cross product.In symplectic space, the length of any vectors is equal to 0. But there exists the concept of symplectic orthogonal cross-course.
Definition 2.4 Let x and y bea2n-dimensional real vector.If their symplectic inner product is equal to zero, i.e.: x, y then x and y are symplectic orthogonal.Otherwise, they are called as symplectic adjoint.
Definition 2.5 If a vector set {x 1, x 2 , …, x m , y 1 , y 2 , …, y m } in the real symplectic space R 2n (m ≤ n)is an adjoint symplectic orthonormal vector set, then the vectors where i, j =1,2,…, m.It is called as an adjoint symplectic orthonormal basis in the 2n-dimensional symplectic space.If a ii = 1, the vector set {x 1, x 2 , …, x m , y 1 , y 2 , …, y m } is a normal adjoint symplectic orthonormal vector set (a normal adjoint symplectic orthonormal basis in the space R 2n ).
The orthogonal of the Euclidean space is different from the symplectic orthogonal.If vectors x and y in the space R n are orthonormal, then they satisfy: where x 6 ¼ y.
If a vector set {x 1, x 2 , …, x m }∈R n is an orthonormal vector set, then any two vectors in the set satisfy: where i, j =1,2,…, m, i 6 ¼ j.Eq. ( 17) is similar to Eqs. ( 14) and ( 15).In the n-dimensional Euclidean space, the set {x i } is denoted as an orthonormal basis.If ||x i || = 1, the orthonormal basis is a normal orthonormal basis.
Theorem 2.1 Let {α i } be a normal adjoint symplectic orthonormal basis in a 2n-dimensional symplectic space Φ.Let the coordinates of any vectors β and γ in Φ be {x 1 , x 2 , … x n , x n +1 , …, x 2n } T and {y 1 , y 2 , … y n , y n +1 , …, y 2n } T , respectively.Referring to the basis {α i }, the coordinates can be described as: where i =1,2,…, n.Then the symplectic inner product of β and γ is as follows: Thus, the symplectic inner product operation is transformed to the matrix operation of ordinary vectors or matrices by applying a normal adjoint symplectic orthonormal basis.
Definition 2.6 Let S is a 2n Â 2n matrix, if S satisfies: then S is a symplectic matrix and the determinant |S|=1orÀ 1.Meanwhile, the inverse matrix and the transpose matrix of a symplectic matrix are a symplectic matrix, respectively.The symplectic matrix S is similar to an orthogonal matrix W in Euclidean space, like Eq. ( 20): Theorem 2. 2 The product of sympletcic matrixes is also a symplectic matrix.

Proof:
Let S i (i =1,2,…, n) be a symplectic matrix.The product matrix M: According to the above definition of symplectic matrix, there are: Thus, the product of symplectic matrixes is also a symplectic matrix.Definition 2.7 Ifa2n Â 2n matrix H is a Hamiltonian matrix, then the matrix H satisfies the following properties: x, Hy ¼ y, Hx , where x and y are 2n-dimensional vectors.In other words, if an evendimensional matrix H satisfies these properties above, the matrix H is a Hamiltonian matrix.In Euclidean space, a symmetric matrix A is similar to a Hamilitonian matrix H, like Eqs. ( 26) and ( 27): x, Ay ÀÁ ¼ y, Ax ÀÁ : ( Theorem 2.3 Let a matrix A be a n Â n real number matrix, if it can be built into a2n Â 2n matrix H in symplectic space in the following pattern: Then the matrix H is a Hamilton matrix.Proof: where J is the 2n Â 2n unit symplectic matrix.In terms of Definition 2.7, the matrix H is a 2n Â 2n Hamiltonian matrix.
Theorem 2.4 Let a 2n Â 2n matrix H be a Hamiltonian matrix.Then its properties keep unchanged at symplectic similar transform.That is, a Hamiltonian matrix H through a series of symplectic similar transforms is still a Hamiltonian matrix.
Proof: According to Definition 2.6, let the matrix S be a symplectic transform matrix.Then, the inverse matrix S À1 is also a symplectic matrix.For a Hamiltonian matrix H, let SHS À1 be the matrix M under the symplectic similar transformation of the matrices S and S À1 .Thus, Therefore, M is also a Hamiltonian matrix.Moreover, the matrix M is similar to the matrix H. Therefore, the Hamiltonian matrix H can keep unchanged at symplectic similar transform in symplectic space.
The eigenvalues of a Hamiltonian matrix have the specific characteristics of the Hamiltonian matrix.However, the eigenvalues may be complex or repeated eigenvalues.In order to obtain the real eigenvalues of a Hamiltonian matrix H, symplectic QR decomposition method is applied to deal with the Hamiltonian H: 2. Build a 2n Â 2n symplectic matrix Q and satisfy: Here B is an upper Hessenberg matrix.Besides, the matrix Q can be a 2n Â 2n Householder matrix.
3. Use the symplectic QR decomposition method to obtain eigenvalues: (36) 4. The eigenvalues of the Hamiltonian matrix H with multiplicity n are λ i ¼ ffiffiffiffi μ i p , i =1,2,…, n; λ nþi ¼Àλ i is also an eigenvalue with multiplicity n.
In symplectic space, the symplectic QR decomposition method allows the primary 2n-dimensional space transform into n dimensional space to resolve the eigenvalues of the Hamiltonian H, where the matrix Q is a symplectic unitary matrix.Thus, the consuming time of the calculation is only one fourth the number of floating-point operations.In general, one makes use of a Householder matrix instead of the matrix Q.
Theorem 2.5 If a 2n Â 2n matrix Q is a Householder matrix, then the matrix Q is a symplectic unitary matrix.
Proof: Let a Householder matrix Q where, '*' means the conjugate transposition.Then, there is Therefore, the Householder matrix Q is a symplectic unitary matrix.

Symplectic geometry spectrums of the reconstructed attractor from a time series
In symplectic space, the reconstructed attractor can keep its properties unchanged [5,6].Its symplectic geometry spectrums can be given by the symplectic geometry theory above.On the basis of Section 2.1 and 2.2, one can build a Hamiltonian matrix M from a time series of the observation.Due to the structure characteristics of the matrix M, its eigenvalues can be calculated by the 2n-dimensional symplectic space reducing into n-dimensional space.In terms of Theorem 1.5, a 2n Â 2n symplectic Householder matrix Q can be constructed.The matrix P in the matrix Q can be calculated by the matrix A in the matrix M. The specific steps are as follows: ⋮ a then, there is: where E (1) = (1, 0, …,0) T is a n Â 1 unit column vector.Then, the elementary reflective matrix P (1) can be calculated: So, there is Continue to deal with A (2) by repeating the above steps, let S (2) be Then, where E (2) = (0, 1, 0, …,0) T is a n Â 1 unit column vector.Then, the elementary reflective matrix P (2) can be calculated: Thus, we can get A (3) with all zeros elements except the first and second nonzero elements: Repeat the same steps until A (n) becomes an upper triangle matrix, one can construct a Householder matrix P as follows: Thus, a symplectic Householder matrix Q can be built to make the Hamiltonian matrix M transform as an upper Hessenberg matrix, namely: eigenvalues are the first k eigenvalues in the symplectic geometry spectrum.If k = n, p = P. Otherwise, p ⊂ P. Then the reestimated attractor matrix X ¼ pp T X ÀÁ , where p T X is defined the transformation coefficient matrix S.Ifp i is the ith eigenvector in P corresponding to the ith eigenvalue σ i in the symplectic geometry spectrum, S i will be the ith principal component coefficients, or called the projection of the p i th direction in the symplectic space: The corresponding p i th principal component matrix Xi is given as follows: Then, the reestimated attractor matrix is equal to the sum of Xi , i =1,…, n.
The reestimated time series x r is equal to the sum of each principal component, i.e. the sum of projections in different directions.If i = 1, the reestimated time series is a reduced noise data based on the first principal component.

Applications
Symplectic geometry theory has been applied to deal with a time series in fields of physics, engineering, biomedical engineering [6-8, 11, 16-24], since Lei et al. (2002) first proposed a symplectic geometry method to estimate the appropriate embedding dimension from a time series [5].Here, we will introduce four research cases in terms of the above theorem and properties of symplectic geometry for the time series analysis.
Lorenz chaotic system was accidentally discovered by Edward Norton Lorenz [25], an American meteorologist, in 1963 when he was studying weather forecast, and was known as the first chaotic attractor.Since then, people began to study chaos, a random-like phenomenon.Lorenz chaotic time series x comes from Lorenz chaotic system, which is a three-dimensional dynamical system as follows [5]: where σ = 10, b = 8/3, γ = 28.The state variable x is chosen as the analyzed data.The sampling interval is 0.005.The length n is 1000 points.
The attractor reconstructed from Lorenz chaotic time series x can reflect the Lorenz system.Here, the dimension of the reconstructed attractor is estimated by the above symplectic geometry method.Let the embedding dimension d be 3: 5: 23, where i =1:d.The matlab program is as follows: - ------------------------------- ---------------------------------Figure 1a shows the symplectic geometry spectrums SGS of x without noise according to the above equations based on symplectic geometry theory.We can see that the symplectic geometry spectrums turn abruptly into a flat area from i = 6, i.e. σ 1 > σ 2 > … > σ 5 >>σ 5 +1 ≥ … ≥ σ d .So, the embedding dimension of the time series x can be estimated at 6.But from the Figure 1b, we can see that it is difficult for the SVD method to determinate the embedding dimension from the time series x.The results indicate that the symplectic geometry method could better determinate the embedding dimension from a time series due to its preserving-measure properties.
In the practical engineering research, a lot of time series data due to their complexity are considered to be nonlinear, such as the surface EMG signal in biomedical engineering.As a kind of non-invasive measure for the contracting skeletal muscles, the surface EMG signal reflects some information about the muscle, limb movements and loading of the bones and joints.It has been widely applied to assess biomechanical and motor control deficits and other functional disorders, as well as to diagnose neuromuscular problems.However, due to noise interference, the study of surface EMG signal is still a great challenge in biomedical engineering.Many researches indicate that the surface EMG signal is complex and nonlinear.The embedding dimension estimation of the surface EMG signal is usually critical to analyze its nonlinear features.As an example, we use the above symplectic geometry method to estimate the embedding dimension of the surface EMG signal during forearm supination.The length of the surface EMG signal is 1000 points.The data sampling frequency is 1 kHz.Figure 2a shows the raw surface EMG signal.Figure 2b gives the symplectic geometry spectrums SGS of the data in Figure 2a.From Figure 2b, the symplectic geometry spectrums SGS change slowly at d = 6 and turn into noise floor with the increase of the index i.Then, the embedding dimension can be estimated at 6 for the surface EMG signal during forearm supination.
In the rotating machinery systems, it is extremely important for rolling bearings to detect faults from vibration signals.The Case Western Reserve University (CWRU) Bearing Data Center provides a website database for the vibration signals Symplectic geometry preserves the nature of a dynamic system under symplectic similar transformations.As an entropy measure in symplectic geometry, the SymEn value of a time series measures the lack of information in a dynamic system to reflect its properties.For the complexity of a rolling bearing, the SymEn estimate is applied to test its nonlinear characteristics from the vibration signals.
Figure 2 shows the SymEn values of the vibration signals at the drive end and their surrogate data sets based on the null hypothesis of a Gaussian linear stochastic process.Here, the length of each data is 6000 points.The embedding dimension d =7.
Meanwhile, the 39 sets of surrogate data are generated by the iterated amplitude adjusted Fourier transform (IAAFT) algorithm in the 95% confidence level [26].
From Figure 3, we can see that there are the significant differences between these SymEn values of the vibration signals of a rolling bearing and their surrogate data sets.The results indicate that the vibration data could contain nonlinear characteristics.The original vibration signals are not from a Gaussian linear stochastic process in the 95% confidence level but from a nonlinear dynamical system.It conforms that the rolling bearing system is a complex nonlinear dynamical system.
Due to the complexity of rolling bearings, it is often thought that the high dimensional features can better identify the faults of rolling bearings [27][28][29].However, the SymEn method can availably extract the low-dimensional features to identify the faults of rolling bearings from vibration signals quite precisely.In the practical engineering measurement, the vibration data of rolling bearings have often become contaminated with noise.The noise reduction is also beneficial to analyze the measured data.The SPCA method preserves the intrinsic nonlinear nature of the raw data.The symplectic principal components can better retrieve  The specific analysis procedures are as follows: 1. Build a Hamiltonian matrix from the measured data in terms of Eq. ( 1), Definition 2.1, 2.2 and Theorem 2.3; 2. Use the Eq. ( 44)-( 59) to compute a symplectic Householder transform matrix Q for the symplectic QR decomposition in the SPCA method; 3. Construct the first symplectic principal component eigenvector matrix p 1 ; 4. Calculate the first symplectic principal component coefficients S 1 , i.e.: 5. Get the first denoised data x 1 from the reestimated matrix in the following: 6. Let the first denoised data x 1 into the first step, and repeat the above steps, then obtain the second denoised data x 3 .
Figure 6 shows the effect of denoising for the vibration signals of rolling element fault (REF), No.225 data in the CWRU database [11].For the rolling element fault at the drive end, the fault state can be seen clearly by the second reducing noise (see Figure 6a).For the vibration signals at the fan end without faults, the periodical characteristics in the normal state can be shown after the two reducing noise (see Figure 6b).
Moreover, the noise reduction method based on the symplectic geometry has been used to denoise several time series data of Lorenz chaotic system, duffing chaotic system, Chua's chaotic system with noise, as well as the sunspot number [30].The details can be found in literatures [17,30].
Besides, the symplectic geometry method also further integrate other approaches to better investigate the fault extraction and identification for rotating systems, such as symplectic geometry mode decomposition [19] with power spectral entropy [7] as well as Lagrange multiplier [20], symplectic transformation based variational Bayesian learning [21].

Conclusions and future research
This chapter introduces the symplectic geometry theory in the research field of the time series analysis in view of the complexity of a time series.Corresponding to Euclidean geometry, the basic concepts and basic elements of mathematics of the symplectic geometry are given, such as the symplectic space, symplectic transformation, Hamiltonian matrix, symplectic entropy (SymEn), symplectic principal component analysis (SPCA), and so on.Based on the symplectic geometry theory, the symplectic geometry spectrum analysis (SGSA), the symplectic entropy (SymEn) method and the symplectic geometry mode decomposition (SGMD) method are demonstrated to investigate the principal characteristics of a time series in the symplectic space.Meanwhile, the corresponding matlab programs are given.At last, in order to facilitate readers to learn, use and develop the symplectic geometry method, some applications of symplectic geometry on time series analysis are presented, such as the embedding dimension estimation, nonlinear testing, fault diagnosis, as well as noise reduction.
The embedding dimension estimation is often the first step in nonlinear time series analysis.Case 1 and 2 show the embedding dimension estimation of Lorenz chaotic time series and the surface EMG signal based on symplectic geometry spectrum.Moreover, the symplectic entropy method is applied to detect the nonlinearity of vibration signals on rolling bearings and identify the faults of vibration signals on rolling bearings (see Case 3).Considering the noise pollution in the practical engineering measurement, to dispose of the noise problem is very necessary for the measured time series analysis.Case 4 uses the SPCA method based on symplectic geometry to investigate the denoise of the vibration signals for rolling element fault (REF) from the CWRU database.
Symplectic geometry provides a new research idea for data analysis in practice.Although the symplectic geometry theory has been developed and applied on the nonlinear time series analysis, the related research based on symplectic geometry still needs to be further developed.Many future challenges in the research of symplectic geometry theory and various applications on a number of diverse aspects need to be developed furtherly.This chapter is only to provide a snapshot of some current trends and future challenges in the research of symplectic geometry theory on the time series analysis.

Figure 1 .
Figure 1.The embedding dimension estimation of Lorenz chaos series with no noise based on: (a) the symplectic geometry method; (b) the SVD method.

Figure 4
shows the four working states of rolling bearings, i.e., NC, ORF, REF, and IRF, based on 2-dimensional features.The abscissa is the SymEn estimates of vibration signals at the drive end.The ordinate is those estimates of vibration signals at the fan end.We can see that the four states are obviously different.

Figure 2 .
Figure 2. The embedding dimension analysis of the surface EMG signal based on the symplectic geometry spectrums: (a) Typical surface EMG signal during forearm supination; (b) The symplectic geometry spectrums of the surface EMG data in (a), where abscissa is the analysis dimension d = 3, 8, 13, 18, 23, ordinate is SGS i = log σ i =tr σ i ðÞ ðÞ , where the index i =1:d.
There are 100% accuracies by RBF classifier for the four states of the rolling bearings.

Figure 5 4 :
plots the histogram of error values between output classes and target classes for the SymEn estimates as features of vibration signals.Case Noise reduction analysis of vibration signals based on SPCA[17,30].

Figure 3 .
Figure 3.The nonlinear analysis of vibration signals based on the SymEn method: (a) for the normal condition (NC); (b) for the outer race fault (ORF); (c) for the rolling element fault (REF); (d) for the inner race fault (IRF).The abscissa is the SymEn values of vibration signals and their surrogate data.

Figure 4 . 18 Structure
Figure 4.The states analysis of rolling for bearings with the SymEn estimates.

Figure 5 .
Figure 5.The analysis of error values identification accuracies of four states.

Figure 6 .
Figure 6.The two times denoising analysis for the vibration signals of rolling element fault (REF) in No.225 data from the CWRU database.(a) The abscissa is the number of data points; (b) the ordinate is the amplitude (v) of the data.