Zhang Neural Networks for Online Solution of Time-Varying Linear Inequalities

Abstract

In this chapter, a special type of recurrent neural networks termed “Zhang neural network” (ZNN) is presented and studied for online solution of time-varying linear (matrix-vector and matrix) inequalities. Specifically, focusing on solving the time-varying linear matrix-vector inequality (LMVI), we develop and investigate two different ZNN models based on two different Zhang functions (ZFs). Then, being an extension, by defining another two different ZFs, another two ZNN models are developed and investigated to solve the time-varying linear matrix inequality (LMI). For such ZNN models, theoretical results and analyses are presented as well to show their computational performances. Simulation results with two illustrative examples further substantiate the efficacy of the presented ZNN models for time-varying LMVI and LMI solving.

1. Introduction

In recent years, linear inequalities have played a more and more important role in numerous fields of science and engineering applications [1–9], such as obstacle avoidance of redundant robots [1, 2], robustness analysis of neural networks [3] and stability analysis of fuzzy control systems [5]. They, including linear matrix-vector inequality (LMVI) and linear matrix inequality (LMI), have now been considered as a powerful formulation and design technique for solving a variety of problems [7–11]. Due to their important roles, lots of numerical algorithms and neural networks have been presented and studied for online solution of linear inequalities [7–17]. For example, an iterative method was presented by Yang et al. for linear inequalities solving [12]. In [13], three continuous-time neural networks were developed by Cichocki and Bargiela to solve the system of linear inequalities. Besides, a gradient-based neural network and a simplified neural network were investigated respectively in [10] and [11] for solving a class of LMI problems (e.g., Lyapunov matrix inequalities and algebraic Riccati matrix inequalities).

It is worth pointing out that most of the reported approaches are designed intrinsically to solve the time-invariant (or say, static) linear inequalities. In view of the fact that many systems in science and engineering applications are time-varying, the resultant linear inequalities may be time-varying ones (i.e., the coefficients are time-varying). Generally speaking, to solve a time-varying problem, based on the assumption of the short-time invariance, such a time-varying problem can be treated as a time-invariant problem within a small time period [8]. The corresponding approaches (e.g., numerical algorithms and neural networks) are thus designed for solving the problem at each single time instant. Note that, as for this common way used to solve the time-varying problem, the time-derivative information (or say, the change trend) of the time-varying coefficients is not involved. Due to the lack of the consideration of such an important information, the aforementioned approaches may be less effective, when they are exploited directly to solve time-varying problems [7–9, 17–19].

Aiming at solving time-varying problems (e.g., time-varying matrix inversion and time-varying quadratic program), a special type of recurrent neural networks termed Zhang neural network (ZNN) has been formally proposed by Zhang et al. since March 2001 [7–9, 17–21]. According to Zhang et al.’s design method, the design of a ZNN is based on an indefinite Zhang function (ZF), with the word “indefinite” meaning that such a ZF can be positive, zero, negative or even lower-unbounded. By exploiting methodologically the time-derivative information of time-varying coefficients involved in the time-varying problems, the resultant ZNN models can thus solve the time-varying problems effectively and efficiently (in terms of avoiding the lagging errors generated by the conventional approaches) [18, 19]. For better understanding and to lay a basis for further investigation, the concepts of ZNN and ZF [18] are presented as follows.

Concept 1. Being a special type of recurrent neural networks, Zhang neural network (ZNN) has been developed and studied since 2001. It originates from the research of Hopfield-type neural networks and is a systematic approach for time-varying problems solving. Such a ZNN is different from the conventional gradient neural network(s) in terms of the problem to be solved, indefinite error function, exponent-type design formula, dynamic equation, and the utilization of time-derivative information.

Concept 2. Zhang function (ZF) is the design basis of ZNN. It differs from the common error/energy functions in the study of conventional approaches. Specifically, compared with the conventional norm-based scalar-valued positive or at least lower-bounded energy function, ZF can be bounded, unbounded or even lower-unbounded (in a word, indefinite). Besides, corresponding to a vector- or matrix-valued problem to be solved, ZF can be vector- or matrix-valued to monitor the solving process fully.

In this chapter, focusing on time-varying linear (matrix-vector and matrix) inequalities solving, we present four different ZNN models based on four different ZFs. Specifically, by defining the first two different ZFs, the corresponding two ZNN models are developed and investigated for solving the time-varying LMVI. Then, being an extension, by defining another two different ZFs, another two ZNN models are developed and investigated to solve the time-varying LMI. For such ZNN models, theoretical results and analyses are also presented to show their computational performances. Simulation results with two illustrative examples further substantiate the efficacy of the presented ZNN models for time-varying LMVI and LMI solving.

2. Preliminaries

As mentioned in Concept 2, the ZF is the design basis for deriving ZNN models to solve time-varying LMVI and LMI. Thus, for presentation convenience, in this chapter, the ZF is denoted by $E(t)$ with ${\displaystyle \dot{E}}(t)$ being the time derivative of $E(t)$. Based on the ZF, the design procedure of a ZNN model for time-varying LMVI/LMI solving is presented as follows [18, 19].

1. Firstly, an indefinite ZF is defined as the error-monitoring function to monitor the solving process of time-varying LMVI/LMI.

2. Secondly, to force the ZF (i.e., $E(t)$) converge to zero, we choose its time derivative (i.e., ${\displaystyle \dot{E}}(t)$) via the ZNN design formula (including its variant).

3. Finally, by expanding the ZNN design formula, the dynamic equation of a ZNN model is thus established for time-varying LMVI/LMI solving.

In order to derive different ZNN models to solve time-varying LMVI and LMI, the following two design formulas (being an important part in the above ZNN design procedure) are exploited in this chapter [7–9, 17–21]:

${\displaystyle \dot{E}}(t)=-\gamma ℱ(E(t)),$E1

${\displaystyle \dot{E}}(t)=-\gamma \text{SGN}(E_0)\odot ℱ(E(t)),$E2

where $\gamma >0\in R$, being the reciprocal of a capacitance parameter, is used to scale the convergence rate of the solution, and $ℱ(\cdot )$ denotes the activation-function array. Note that, in general, design parameter $\gamma$ should be set as large as the hardware system would permit, or selected appropriately for simulation purposes [22]. In addition, function $f(\cdot )$, being a processing element of $ℱ(\cdot )$, can be any monotonically increasing odd activation function, e.g., the linear, power-sigmoid and hyperbolic-sine activation functions [19, 23]. Furthermore, $E_0=E(t=0)$ denotes the initial error, and the unipolar signum function $\text{sgn}(\cdot )$, being an element of $\text{SGN}(\cdot )$, is defined as

$\text{sgn}(c)=(\begin{array}{c}\mathrm{1,}\text{if}c>\mathrm{0,}\\ \mathrm{0,}\text{if}c⩽0.\end{array}$E21

Besides, the multiplication operator $\odot$ is the Hadamard product [24] and is defined as follows:

$U\odot V=\left[\begin{array}{cccc}{u}_{11}{v}_{11}& u_{12}{v}_{12}& \cdots & u_{1n}{v}_{1n}\\ u_{21}{v}_{21}& u_{22}{v}_{22}& \cdots & u_{2n}{v}_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ u_{m1}{v}_{m1}& u_{m2}{w}_{m2}& \cdots & u_{mn}{v}_{mn}\end{array}\right]\in R^{m\times n}\mathrm{.}$E22

Note that, as for the presented design formulas (1) and (2), the former is the original ZNN design formula proposed by Zhang et al. to solve the time-varying Sylvester equation [20], while the latter is the variant of such a ZNN design formula constructed elaborately for time-varying linear inequalities solving [9]. Thus, for presentation convenience and better understanding, (1) is called the original design formula, while (2) is called the variant design formula for time-varying LMVI and LMI solving in this chapter.

Remark 1. For the variant design formula (2), when the initial error $E_0>0$, it reduces to ${\displaystyle \dot{E}}(t)=-\gamma ℱ(E(t))$, which is exactly the original design formula (1) for various time-varying problems solving [17–21]. In this case (i.e., ${\displaystyle \dot{E}}(t)=-\gamma ℱ(E(t))$), different convergence performances of $E(t)$ can be achieved by choosing different activation function arrays [17–21, 23]. For example, (2) reduces to ${\displaystyle \dot{E}}(t)=-\gamma E(t)$ with a linear activation function array used and with $E_0>0$. Evidently, its analytical solution is $E(t)=\text{exp}(-\gamma t)E_0$, which means that $E(t)$ is globally and exponentially convergent to zero with rate γ. By following the previous successful researches [17–21, 23], superior convergence property of $E(t)$ can be achieved by exploiting nonlinear activation functions, e.g., the power-sigmoid and hyperbolic-sine activation functions [23]. In addition, the convergence property can be further improved by increasing the γ value. Therefore, in the case of $E_0>0$, the global and exponential convergence property is guaranteed for $E(t)$. Note that, in the case of $E_0\le 0$, (2) reduces to ${\displaystyle \dot{E}}(t)=0$, meaning that $E(t)=E_0$ as time t evolves. In this situation, there is no need to investigate the convergence performance of $E(t)$ with the different activation function arrays and different γ values used.

According to the presented design formulas (1) and (2), by defining different ZFs (i.e., $E(t)$ with different formulations), different ZNN models are thus developed and investigated for time-varying LMVI and LMI solving.

3. Time-varying linear matrix-vector inequality

In this section, we introduce two different ZFs and develop the resultant ZNN models for time-varying linear matrix-vector inequality (LMVI) solving. Then, theoretical results and analyses are provided to show the computational performances of such two ZNN models.

Specifically, the following problem of time-varying LMVI [7, 9] is considered in this chapter:

$A(t)x(t)⩽b(t),$E3

in which $A(t)\in R^{n\times n}$ and $b(t)\in R^n$ are smoothly time-varying matrix and vector, respectively. In addition, $x(t)\in R^n$ is the unknown time-varying vector to be obtained. The objective is to find a feasible solution $x(t)$ such that (3) holds true for any time instant $t\ge 0$. Note that, for further discussion, $A(t)$ is assumed to be nonsingular at any time instant $t\in [\mathrm{0,}+\infty ]$ in this chapter.

3.1. ZFs and ZNN models

In this subsection, by defining two different ZFs, two different ZNN models are developed and investigated for time-varying LMVI solving.

3.1.1. The first ZF and ZNN model

To monitor and control the process of solving the time-varying LMVI (3), the first ZF is defined as follows [7]:

$E(t)=A(t)x(t)+{\text{Λ}}^2(t)-b(t)\in R^n\mathrm{,}$E4

where ${\text{Λ}}^2(t)=\text{Λ}(t)\odot \text{Λ}(t)$ with the time-varying vector $\text{Λ}(t)$ being $\text{Λ}(t)=[{\lambda }_1(t),{\lambda }_2(t),\cdots \mathrm{,}{\lambda }_n(t{)]}^{\text{T}}\in R^n$. In view of the fact that ${\text{Λ}}^2(t)\ge 0$, when $E(t)=0$, then we have

$A(t)x(t)-b(t)=-{\text{Λ}}^2(t)⩽0.$E23

That is to say, time-varying LMVI (3) solving can be equivalent to solving the time-varying equation $A(t)x(t)+{\text{Λ}}^2(t)-b(t)=0$. For further discussion, the following diagonal matrix $D(t)$ is defined:

$D(t)=\left[\begin{array}{cccc}{\lambda }_1(t)& 0& \cdots & 0\\ 0& {\lambda }_2(t)& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\lambda }_n(t)\end{array}\right]\in R^{n\times n}\mathrm{,}$E24

which yields

${\text{Λ}}^2(t)=D(t)\text{Λ}(t)\text{and}{{\displaystyle \dot{\text{Λ}}}}^2(t)=\frac{\text{d}{\text{Λ}}^2(t)}{\text{d}t}=2D(t){\displaystyle \dot{\text{Λ}}}(t).$E25

with ${\displaystyle \dot{\text{Λ}}}(t)$ being the time derivative of $\text{Λ}(t)$.

On the basis of ZF (4), by exploiting the original design formula (1), the dynamic equation of a ZNN model is established as follows:

$A(t){\displaystyle \dot{x}}(t)+2D(t){\displaystyle \dot{\text{Λ}}}(t)=-{\displaystyle \dot{A}}(t)x(t)+{\displaystyle \dot{b}}(t)-\gamma ℱ(A(t)x(t)+{\text{Λ}}^2(t)-b(t)),$E5

where ${\displaystyle \dot{x}}(t)$, ${\displaystyle \dot{A}}(t)$ and ${\displaystyle \dot{b}}(t)$ are the time derivatives of $x(t)$, $A(t)$ and $b(t)$, respectively. As for (5), it is reformulated as

$\left[\begin{array}{c}A(t)2D(t)\end{array}\right]\left[\begin{array}{c}{\displaystyle \dot{x}}(t)\\ {\displaystyle \dot{\text{Λ}}}(t)\end{array}\right]=\left[\begin{array}{c}-{\displaystyle \dot{A}}(t)0\end{array}\right]\left[\begin{array}{c}x(t)\\ \text{Λ}(t)\end{array}\right]+{\displaystyle \dot{b}}(t)-\gamma ℱ(\left[\begin{array}{c}A(t)D(t)\end{array}\right]\left[\begin{array}{c}x(t)\\ \text{Λ}(t)\end{array}\right]-b(t)).$E6

By defining the augmented vector $y(t)=[x^{\text{T}}(t),{\text{Λ}}^{\text{T}}(t{)]}^{\text{T}}\in R^{2n}$, (6) is further rewritten as follows:

$C(t){\displaystyle \dot{y}}(t)=P(t)y(t)+{\displaystyle \dot{b}}(t)-\gamma ℱ(Q(t)y(t)-b(t)),$E7

with ${\displaystyle \dot{y}}(t)$ being the time derivative of $y(t)$, and the augmented matrices are being defined as below:

$\begin{array}{l}C(t)={\left[\begin{array}{c}{A}^{\text{T}}(t)\\ 2D^{\text{T}}(t)\end{array}\right]}^{\text{T}}\in R^{n\times 2n},P(t)={\left[\begin{array}{c}-{{\displaystyle \dot{A}}}^{\text{T}}(t)\\ 0\end{array}\right]}^{\text{T}}\in R^{n\times 2n}\text{and}Q(t)={\left[\begin{array}{c}{A}^{\text{T}}(t)\\ D^{\text{T}}(t)\end{array}\right]}^{\text{T}}\in R^{n\times 2n}\mathrm{.}\hfill \end{array}$E26

In order to make (7) more computable, we can reformulate (7) to the following explicit form:

${\displaystyle \dot{y}}(t)=C^{†}(t)P(t)y(t)+C^{†}(t){\displaystyle \dot{b}}(t)-\gamma C^{†}(t)ℱ(Q(t)y(t)-b(t)),$E8

where $C^{†}(t)=C^{\text{T}}(t)(C(t)C^{\text{T}}(t{))}^{-1}\in R^{2n\times n}$ denotes the right pseudoinverse of $C(t)$ and the MATLAB routine “$\text{pinv}$” is used to obtain $C^{†}(t)$ at each time instant in the simulations. Therefore, based on ZF (4), ZNN model (8) is obtained for time-varying LMVI solving. Besides, for better understanding and potential hardware implementation, ZNN model (8) is expressed in the ith (with $i=\mathrm{1,2,}\cdots \mathrm{,2}n$) neuron form as

$y_i={\displaystyle \int }{\displaystyle {\displaystyle \sum _{k=1}^n}}{c}_{ik}({\displaystyle {\displaystyle \sum _{j=1}^m}}{p}_{kj}{y}_j+{{\displaystyle \dot{b}}}_k-\gamma f({\displaystyle {\displaystyle \sum _{j=1}^m}}{q}_{kj}{y}_j-b_k))\text{d}t\mathrm{,}$E27

where $y_i$ denotes the ith neuron of (8), $m=2n$ and $f(\cdot )$ is a processing element of $ℱ(\cdot )$. In addition, time-varying weights $c_{ik}$, $p_{kj}$ and $q_{kj}$ denote the $ik$th element of $C^{†}(t)$, the $kj$th element of $P(t)$ and $kj$th element of $Q(t)$, respectively. Moreover, time-varying thresholds ${{\displaystyle \dot{b}}}_k$ and $b_k$ denote, respectively, the kth elements of ${\displaystyle \dot{b}}(t)$ and $b(t)$. Thus, the neural-network structure of (8) is shown in Figure 1.

Figure 1.

Structure of the neurons in ZNN model (8) for time-varying LMVI (3) solving.

3.1.2. The second ZF and ZNN model

Being different from the first ZF (4), the second ZF is defined as follows [9]:

$E(t)=A(t)x(t)-b(t)\in R^n\mathrm{.}$E9

On the basis of such a ZF, by exploiting the variant design formula (2), another ZNN model is developed as follows:

$A(t){\displaystyle \dot{x}}(t)=-{\displaystyle \dot{A}}(t)x(t)+{\displaystyle \dot{b}}(t)-\gamma \text{SGN}(E_0)\odot ℱ(A(t)x(t)-b(t)),$E10

where the initial error $E_0=E(t=0)=A(0)x(0)-b(0)$. Therefore, based on ZF (9), ZNN model (10) is obtained for time-varying LMVI solving. Besides, for better understanding and potential hardware implementation, the block diagram of ZNN model (10) is shown in Figure 2, where $I\in R^{n\times n}$ denotes the identity matrix.

Figure 2.

Block diagram of ZNN model (10) for time-varying LMVI (3) solving.

3.2. Theoretical results and analyses

In this subsection, theoretical results and analyses of the presented ZNN models (8) and (10) for solving the time-varying LMVI (3) are provided via the following theorems.

Theorem 1. Given a smoothly time-varying nonsingular coefficient matrix $A(t)\in R^{n\times n}$ and a smoothly time-varying coefficient vector $b(t)\in R^n$ in (3), if a monotonically increasing odd activation function array $ℱ(\cdot )$ is used, then ZNN model (8) generates an exact time-varying solution of the time-varying LMVI (3).

Proof: To lay a basis for discussion, we define $x^{*}(t)$ as a theoretical time-varying solution of (3), i.e., $A(t)x^{*}(t)\le b(t)$. Then, a time-varying vector ${\text{Λ}}^{*}(t)$ would exist, which results in the time-varying matrix-vector equation as follows:

$A(t)x^{*}(t)+{\text{Λ}}^{*2}(t)=b(t).$E11

By differentiating (11) with respect to time t, we have

$A(t){{\displaystyle \dot{x}}}^{*}(t)+{\displaystyle \dot{A}}(t)x^{*}(t)+{{\displaystyle \dot{\text{Λ}}}}^{*2}(t)={\displaystyle \dot{b}}(t),$E12

with ${{\displaystyle \dot{x}}}^{*}(t)$ and ${{\displaystyle \dot{\text{Λ}}}}^{*2}(t)$ being respectively the time derivatives of $x^{*}(t)$ and ${\text{Λ}}^{*2}(t)$. Based on (7), (11) and (12), we further have

$A(t)({\displaystyle \dot{x}}(t)-{{\displaystyle \dot{x}}}^{*}(t))+{\displaystyle \dot{A}}(t)(x(t)-x^{*}(t))+{{\displaystyle \dot{\text{Λ}}}}^2(t)-{{\displaystyle \dot{\text{Λ}}}}^{*2}(t)=-\gamma ℱ(A(t)(x(t)-x^{*}(t))+{\text{Λ}}^2(t)-{\text{Λ}}^{*2}(t)),$E13

which is rewritten as

${\displaystyle \dot{{\displaystyle \tilde{E}}}}(t)=-\gamma ℱ({\displaystyle \tilde{E}}(t)),$E13

where ${\displaystyle \tilde{E}}(t)=A(t)(x(t)-x^{*}(t))+{\text{Λ}}^2(t)-{\text{Λ}}^{*2}(t)\in R^n$ with ${\displaystyle \dot{{\displaystyle \tilde{E}}}}(t)$ being the time derivative of ${\displaystyle \tilde{E}}(t)$.

As for (13), its compact form of a set of $n$ decoupled differential equations is written as follows:

${{\displaystyle \dot{{\displaystyle \tilde{e}}}}}_i(t)=-\gamma f({{\displaystyle \tilde{e}}}_i(t)),$E14

where $i=\mathrm{1,2,}\cdots \mathrm{,}n$. To analyze (14), we define a Lyapunov function candidate $v_i(t)={{\displaystyle \tilde{e}}}_i^2(t)/2\ge 0$ with its time derivative being

${{\displaystyle \dot{v}}}_i(t)=\frac{\text{d}{v}_i(t)}{\text{d}t}={{\displaystyle \tilde{e}}}_i(t){{\displaystyle \dot{{\displaystyle \tilde{e}}}}}_i(t)=-\gamma {{\displaystyle \tilde{e}}}_i(t)f({{\displaystyle \tilde{e}}}_i(t)).$E29

Since $f(\cdot )$ is a monotonically increasing odd activation function, i.e., $f(-{{\displaystyle \tilde{e}}}_i(t))=-f({{\displaystyle \tilde{e}}}_i(t))$, then we have

${{\displaystyle \tilde{e}}}_i(t)f({{\displaystyle \tilde{e}}}_i(t))(\begin{array}{c}>0,\text{if}{{\displaystyle \tilde{e}}}_i(t)\ne \mathrm{0,}\\ =0,\text{if}{{\displaystyle \tilde{e}}}_i(t)=\mathrm{0,}\end{array}$E30

which guarantees the negative definiteness of ${{\displaystyle \dot{v}}}_i(t)$. That is to say, ${{\displaystyle \dot{v}}}_i(t)<0$ for ${{\displaystyle \tilde{e}}}_i(t)\ne 0$_, while ${{\displaystyle \dot{v}}}_i(t)=0$ for ${{\displaystyle \tilde{e}}}_i(t)=0$ only. By Lyapunov theory, ${{\displaystyle \tilde{e}}}_i(t)$ converges to zero for any $i\in \{\mathrm{1,2,}\cdots \mathrm{,}n\}$, thereby showing that ${\displaystyle \tilde{E}}(t)$ is convergent to zero as well.

Besides, based on (11), we have $A(t)x^{*}(t)+{\text{Λ}}^{*2}(t)=b(t)$. Then, ${\displaystyle \tilde{E}}(t)$ is rewritten as ${\displaystyle \tilde{E}}(t)=A(t)x(t)+{\text{Λ}}^2(t)-b(t)$, which is equivalent to $A(t)x(t)-b(t)=-{\text{Λ}}^2(t)+{\displaystyle \tilde{E}}(t)$. Note that, as analyzed previously, ${\displaystyle \tilde{E}}(t)\to 0$ with time $t\to +\infty$. Thus, as time evolves,

$A(t)x(t)-b(t)=-{\text{Λ}}^2(t)+{\displaystyle \tilde{E}}(t)\to -{\text{Λ}}^2(t).$E31

Since $-{\text{Λ}}^2(t)\le 0$ (i.e., each element is less than or equal to zero), then we have $A(t)x(t)-b(t)\le 0$. This implies that $x(t)$ (being the first n elements of $y(t)$ of (8)) would converge to a time-varying vector which satisfies the time-varying LMVI (3); i.e., $x(t)\to x^{*}(t)$ to make (3) hold true. In summary, the presented ZNN model (8) generates an exact time-varying solution of the time-varying LMVI (3). The proof is thus completed. $\square$

Theorem 2. Given a smoothly time-varying nonsingular coefficient matrix $A(t)\in R^{n\times n}$ and a smoothly time-varying coefficient vector $b(t)\in R^n$ in (3), if a monotonically increasing odd activation function array $ℱ(\cdot )$ is used, then ZNN model (10) generates an exact time-varying solution of the time-varying LMVI (3).

Proof: Consider ZNN model (10), which is derived from the variant design formula (2). Thus, there are three cases as follows.

1. If the randomly generated initial state $x(0)\in R^n$ is outside the initial solution set $S(0)$ of (3), i.e., $E_0>0$ in (10), based on Remark 1 and the previous work [9], the global and exponential convergence of the error function $E(t)$ is achieved (or say, $E(t)=A(t)x(t)-b(t)\to 0$ globally and exponentially). This also means that the neural state $x(t)$ of ZNN model (10) is convergent to the theoretical time-varying solution of the matrix-vector equation $A(t)x(t)-b(t)=0$. Note that $A(t)x(t)-b(t)=0$ (i.e., $A(t)x(t)=b(t)$) is a special case of $A(t)x(t)\le b(t)$. Therefore, ZNN model (10) is effective on solving the time-varying LMVI (3), in terms of $x(t)$ being convergent to the time-varying solution set $S(t)$ of (3).

2. If $x(0)$ is inside $S(0)$ of (3), i.e., $E_0\le 0$ in (10), based on Remark 1, the error function $E(t)$ would remain $E_0$ with $t\to +\infty$. That is, $E(t)=E_0\le 0$, no matter how time t evolves. In this situation, ZNN model (10) is still effective on solving the time-varying LMVI (3), in terms of its neural state $x(t)$ always being inside $S(t)$ of (3).

3. If some elements of $x(0)$ are inside $S(0)$ of (3) while the others are outside $S(0)$, i.e., some elements of $E_0$ are greater than zero while the rest elements of $E_0$ are less than or equal to zero, then, (i) for the elements of $E(t)$ that have positive initial values (i.e., their initial values are greater than zero), they can be convergent to zero globally and exponentially; and (ii) for the rest elements of $E(t)$, they can be always equal to their initial values that are less than or equal to zero. In view of the fact that, as time evolves, each element of $E(t)=A(t)x(t)-b(t)$ is less than or equal to zero, ZNN model (10) is thus effective on solving the time-varying LMVI (3).

By summarizing the above analyses, the time-varying LMVI (3) is solved effectively via ZNN model (10), in the sense that such a model can generate an exact time-varying solution of (3). The proof is thus completed. $\square$

Remark 2. On the basis of two different ZFs (i.e., (4) and (9)), two different ZNN models (i.e., (8) and (10)) are obtained for online solution of the time-varying LMVI (3). Note that the former aims at solving (3) aided with equality conversion (i.e., from inequality to equation) and the original design formula (1), while the latter focuses on solving (3) directly with the aid of the variant design formula (2). The resultant ZNN model (8) is depicted in an explicit dynamics (i.e., ${\displaystyle \dot{y}}(t)=\cdots$), and ZNN model (10) is depicted in an implicit dynamics (i.e., $A(t){\displaystyle \dot{x}}(t)=\cdots$). As analyzed above and as demonstrated by the simulation results shown in Section 5, such two ZNN models are both effective on solving the time-varying LMVI (3). In summary, two different approaches for time-varying LMVI solving have been discovered and presented in this chapter; i.e., one is based on the variant of the original ZNN design formula, and the other is based on the conversion from inequality to equation. This can be viewed as an important breakthrough on (time-varying or static) inequalities solving [7–9].

4. Time-varying linear matrix inequality

In this section, being an extension, by defining another two different ZFs, another two ZNN models are developed and investigated for time-varying linear matrix inequality (LMI) solving.

Specifically, the following problem of time-varying LMI is considered [9]:

$A(t)X(t)⩽B(t),$E15

where $A(t)\in R^{m\times m}$ and $B(t)\in R^{m\times n}$ are smoothly time-varying matrices, and $X(t)\in R^{m\times n}$ is the unknown matrix to be obtained. Note that (15) is a representative time-varying LMI problem which is studied here. The design approaches presented in this chapter (more specifically, summarized in Remark 2) can be directly extended to solve other types of time-varying LMIs [8, 10, 11].

4.1. The first ZF and ZNN model

In order to solve the time-varying LMI (15), the first ZF is defined as follows:

$E(t)=A(t)X(t)+{\text{Λ}}^2(t)-B(t)\in R^{m\times n}\mathrm{,}$E16

where ${\text{Λ}}^2(t)=\text{Λ}(t)\odot \text{Λ}(t)$ with the time-varying vector $\text{Λ}(t)$ being

$\text{Λ}(t)=\left[\begin{array}{cccc}{\lambda }_{11}(t)& {\lambda }_{12}(t)& \cdots & {\lambda }_{1n}(t)\\ {\lambda }_{21}(t)& {\lambda }_{22}(t)& \cdots & {\lambda }_{2n}(t)\\ ⋮& ⋮& \ddots & ⋮\\ {\lambda }_{m1}(t)& {\lambda }_{m2}(t)& \cdots & {\lambda }_{mn}(t)\end{array}\right]\in R^{m\times n}\mathrm{.}$E32

In addition, for matrices $\text{Λ}(t)$ and ${\text{Λ}}^2(t)$, we have

$\text{vec}({\text{Λ}}^2(t))=D(t)\text{vec}(\text{Λ}(t)),$E33

where operator $\text{vec}(\cdot )\in R^{mn}$ generates a column vector obtained by stacking all column vectors of a matrix together [8, 18, 19]. In addition, the diagonal matrix $D(t)$ is defined as follows:

$D(t)=\left[\begin{array}{cccc}{{\displaystyle \tilde{\lambda }}}_1(t)& 0& \cdots & 0\\ 0& {{\displaystyle \tilde{\lambda }}}_2(t)& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {{\displaystyle \tilde{\lambda }}}_n(t)\end{array}\right]\in R^{mn\times mn}\mathrm{,}$E34

with the ith (with $i=\mathrm{1,}\cdots \mathrm{,}n$) block matrix being

${{\displaystyle \tilde{\lambda }}}_i(t)=\left[\begin{array}{cccc}{\lambda }_{1i}(t)& 0& \cdots & 0\\ 0& {\lambda }_{2i}(t)& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\lambda }_{mi}(t)\end{array}\right]\in R^{m\times m}\mathrm{.}$E31

By defining $u(t)=\text{vec}(X(t))\in R^{mn}$, $v(t)=\text{vec}(\text{Λ}(t))\in R^{mn}$ and $w(t)=\text{vec}(B(t))\in R^{mn}$, ZF (16) is reformulated as $E(t)=M(t)u(t)-w(t)+D(t)v(t)\in R^{mn}$, where $M(t)=I\otimes A(t)\in R^{mn\times mn}$ with $I\in R^{n\times n}$ being the identity matrix and $\otimes$ denoting the Kronecker product [18, 19]. Thus, on the basis of (16), by exploiting the original design formula (1), we have

$C(t){\displaystyle \dot{y}}(t)=P(t)y(t)+{\displaystyle \dot{w}}(t)-\gamma ℱ(Q(t)y(t)-w(t)),$E17

where the augmented vector $y(t)=[u^{\text{T}}(t),v^{\text{T}}(t{)]}^{\text{T}}\in R^{2mn}$, and ${\displaystyle \dot{y}}(t)\in R^{2mn}$ and ${\displaystyle \dot{w}}(t)\in R^{mn}$ are the time derivatives of $y(t)$ and $w(t)$, respectively. In addition, the augmented matrices are defined as

$C(t)={\left[\begin{array}{c}{M}^{\text{T}}(t)\\ 2D^{\text{T}}(t)\end{array}\right]}^{\text{T}}\in R^{mn\times 2mn},P(t)={\left[\begin{array}{c}-N(t)\\ 0\end{array}\right]}^{\text{T}}\in R^{mn\times 2mn}\text{and}Q(t)={\left[\begin{array}{c}{M}^{\text{T}}(t)\\ D^{\text{T}}(t)\end{array}\right]}^{\text{T}}\in R^{mn\times 2mn}\mathrm{,}$E35

where $N(t)={\displaystyle \dot{M}}(t)=I\otimes {\displaystyle \dot{A}}(t)\in R^{mn\times mn}$ with ${\displaystyle \dot{A}}(t)$ being the time derivative of $A(t)$.

Similarly, to make (17) more computable, we can reformulate (17) as the following explicit form:

${\displaystyle \dot{y}}(t)=C^{†}(t)P(t)y(t)+C^{†}(t){\displaystyle \dot{w}}(t)-\gamma C^{†}(t)ℱ(Q(t)y(t)-w(t)),$E18

where $C^{†}(t)=C^{\text{T}}(t)(C(t)C^{\text{T}}(t{))}^{-1}\in R^{2mn\times mn}$. Therefore, based on ZF (16), ZNN model (18) is obtained for time-varying LMI solving. Note that the neural-network structure of (18) is similar to the one shown in Figure 1, and is thus omitted here. Besides, as for ZNN model (18), we have the following theoretical result, with the related proof being generalized from the proof of Theorem 1 and being left to interested readers to complete as a topic of exercise.

Corollary 1. Given a smoothly time-varying nonsingular coefficient matrix $A(t)\in R^{m\times m}$ and a smoothly time-varying coefficient matrix $B(t)\in R^{m\times n}$ in (15), if a monotonically increasing odd activation function array $ℱ(\cdot )$ is used, then ZNN model (18) generates an exact time-varying solution of the time-varying LMI (15).

4.2. The second ZF and ZNN model

In this subsection, being different from the first ZF (16), the second ZF is defined as follows:

$E(t)=A(t)X(t)-B(t)\in R^{m\times n}\mathrm{.}$E19

On the basis of such a ZF, by exploiting the variant design formula (2), the following ZNN model for time-varying LMI solving is developed:

$A(t){\displaystyle \dot{X}}(t)=-{\displaystyle \dot{A}}(t)X(t)+{\displaystyle \dot{B}}(t)-\gamma \text{SGN}(E_0)\odot ℱ(A(t)X(t)-B(t)),$E20

where the initial error $E_0=E(t=0)=A(0)X(0)-B(0)$. Note that, due to similarity to the block diagram of (10), the block diagram of ZNN model (20) is omitted. Besides, as for ZNN model (20), we have the following theoretical result, of which the proof is generalized from the proof of Theorem 2 (and is also left to interested readers to complete as a topic of exercise).

Corollary 2. Given a smoothly time-varying nonsingular coefficient matrix $A(t)\in R^{m\times m}$ and a smoothly time-varying coefficient matrix $B(t)\in R^{m\times n}$ in (15), if a monotonically increasing odd activation function array $ℱ(\cdot )$ is used, then ZNN model (20) generates an exact time-varying solution of the time-varying LMI (15).

5. Simulative verifications

In this section, one illustrative example is first simulated for demonstrating the efficacy of the presented ZNN models (8) and (10) for solving the time-varying LMVI (3). Then, another illustrative example is provided for substantiating the efficacy of the presented ZNN models (18) and (20) for solving the time-varying LMI (15).

Example 1 In the first example, the following smoothly time-varying coefficient matrix $A(t)$ and coefficient vector $b(t)$ of (3) are designed to test ZNN models (8) and (10):

$A(t)=\left[\begin{array}{ccc}3+\sin (3t)& \cos (3t)/2& \cos (3t)\\ \cos (3t)/2& 3+\sin (3t)& \cos (3t)/2\\ \cos (3t)& \cos (3t)/2& 3+\sin (3t)\end{array}\right]\in R^{3\times 3}\text{and}b(t)=\left[\begin{array}{c}\sin (3t)+1\\ \cos (3t)+2\\ \sin (3t)+\cos (3t)+3\end{array}\right]\in R^3\mathrm{.}$E36

The corresponding simulation results are shown in Figures 3 through 9.

Specifically, Figures 3 and 4 illustrate the state trajectories synthesized by ZNN model (8) using $\gamma =1$ and the power-sigmoid activation function. As shown in Figures 3 and 4, starting from five randomly generated initial states, the $x(t)$ trajectories (being the first 3 elements of $y(t)$ in (8)) and the $\text{Λ}(t)$ trajectories (being the rest elements of $y(t)$) are time-varying. In addition, Figure 5 presents the characteristics of residual error $\parallel Q(t)y(t)-b(t){\parallel }_2=\parallel A(t)x(t)+{\text{Λ}}^2(t)-b(t){\parallel }_2$ (with symbol $\parallel \cdot {\parallel }_2$ denoting the two norm of a vector), from which we can observe that the residual errors of ZNN model (8) (corresponding to Figures 3 and 4) are all convergent to zero. This means that the $x(t)$ and $\text{Λ}(t)$ solutions shown in Figures 3 and 4 are the time-varying solutions of $A(t)x(t)+{\text{Λ}}^2(t)-b(t)=0$. In view of $-{\text{Λ}}^2(t)\le 0$, such a solution of $x(t)$ is an exact solution of the time-varying LMVI (3), i.e., $A(t)x(t)\le b(t)$. For better understanding, the profiles of the testing error function $\epsilon (t)=A(t)x(t)-b(t)$ (i.e., ZF (9)) are illustrated in Figure 6. As shown in the figure, all the elements of $\epsilon (t)$ are less than or equal to zero, thereby meaning that the $x(t)$ solution satisfies $A(t)x(t)\le b(t)$ (being an exact time-varying solution of (3)). These simulation results substantiate the efficacy of ZNN model (8) for time-varying LMVI solving. Besides, Figure 7 shows the simulation results synthesized by ZNN model (8) using different γ values (i.e., $\gamma =1$ and $\gamma =10$) and different activation functions (i.e., linear, hyperbolic-sine and power-sigmoid activation functions). As seen from Figure 7, the residual errors all converge to zero, which means that ZNN model (8) solves the time-varying LMVI (3) successfully. Note that, from Figure 7, we have a conclusion that superior computational performance of ZNN model (8) can be achieved by increasing the γ value and choosing a suitable activation function.

Figure 3.

State trajectories of $x(t)\in R^3$ synthesized by ZNN model (8) with $\gamma =1$ and the power-sigmoid activation function used for time-varying LMVI (3) solving.

Figure 4.

State trajectories of $\text{Λ}(t)\in R^3$ synthesized by ZNN model (8) with $\gamma =1$ and the power-sigmoid activation function used for time-varying LMVI (3) solving.

Figure 5.

Residual errors $\parallel Q(t)y(t)-b(t){\parallel }_2$ of ZNN model (8) for time-varying LMVI (3) solving.

Figure 6.

Profiles of $\epsilon (t)=A(t)x(t)-b(t)\in R^3$ synthesized by ZNN model (8) with $\gamma =1$ and the power-sigmoid activation function used for time-varying LMVI (3) solving.

Figure 7.

Residual errors $\parallel Q(t)y(t)-b(t){\parallel }_2$ of ZNN model (8) with γ fixed and different activation functions used for time-varying LMVI (3) solving.

Figure 8.

State trajectories of $x(t)\in R^3$ synthesized by ZNN model (10) with $\gamma =1$ and the power-sigmoid activation function used for time-varying LMVI (3) solving.

It is worth pointing out here that, in general, it may be difficult to know whether the initial state $x(0)$ used for simulation/application is outside the initial solution set $S(0)$ of the time-varying LMVI (3) or not. Thus, as for ZNN model (10), we focus on investigating its computational performance when some elements of $x(0)$ are outside $S(0)$ while the others are inside $S(0)$. In this case, some elements of the initial error $E_0=A(0)x(0)-b(0)$ are greater than zero, while the rest are less than or equal to zero. The corresponding simulation results synthesized by ZNN model (10) using $\gamma =1$ and the power-sigmoid activation function are illustrated in Figures 8 and 9. As shown in Figure 8, starting from five randomly generated initial states, the $x(t)$ trajectories of ZNN model (10) are time-varying. Besides, from Figure 9 which shows the profiles of the testing error function $\epsilon (t)=A(t)x(t)\le b(t)$, we can observe that the elements of $\epsilon (t)$ with positive initial values are convergent to zero, while the rest elements remain at their initial values. This result implies that the $x(t)$ solutions shown in Figure 6 are the time-varying solutions of (3), i.e., $A(t)x(t)\le b(t)$, thereby showing the efficacy of ZNN model (10) for time-varying LMVI solving. That is, ZNN model (10) generates an exact time-varying solution of the time-varying LMVI (3). Note that the computational performance of ZNN model (10) can be improved by increasing the value of γ and choosing a suitable activation function (which is similar to that of ZNN model (8)). Being a topic of exercise, the corresponding simulative verifications of ZNN model (10) are left for interested readers.

Figure 9.

Profiles of $\epsilon (t)=A(t)x(t)-b(t)\in R^3$ synthesized by ZNN model (10) with $\gamma =1$ and the power-sigmoid activation function used for time-varying LMVI (3) solving.

In summary, the above simulation results (i.e., Figures 3 through 9) have substantiated that the presented ZNN models (8) and (10) are both effective on time-varying LMVI solving.

Example 2 In the second example, the following smoothly time-varying coefficient matrices $A(t)$ and $B(t)$ of (15) are designed to test ZNN models (18) and (20):

$A(t)=\left[\begin{array}{cc}\sin (10t)& \cos (10t)\\ -\cos (10t)& \sin (10t)\end{array}\right]\in R^{2\times 2}\text{and}B(t)=\left[\begin{array}{cc}\cos (10t)+1& \sin (10t)+1.5\\ -\sin (10t)+1.5& -\cos (10t)+1\end{array}\right]\in R^{2\times 2}\mathrm{.}$E37

The corresponding simulation results are illustrated in Figures 10 through 13.

Figure 10.

Neural states synthesized by ZNN model (18) with $\gamma =1$ and the hyperbolic-sine activation function used for time-varying LMI (15) solving.

Figure 11.

Profiles of residual errors $\parallel Q(t)y(t)-w(t){\parallel }_2$ and $\epsilon (t)=A(t)X(t)-B(t)$ synthesized by ZNN model (18) with $\gamma =1$ and the hyperbolic-sine activation function used for time-varying LMI (15) solving.

Figure 12.

Residual errors $\parallel Q(t)y(t)-w(t){\parallel }_2$ of ZNN model (18) with γ fixed and different activation functions used for time-varying LMI (15) solving.

Figure 13.

Simulation results synthesized by ZNN model (20) with $\gamma =1$ and the hyperbolic-sine activation function used for time-varying LMI (15) solving.

On one hand, as synthesized by ZNN model (18) using $\gamma =1$ and the hyperbolic-sine activation function, Figure 10 shows the trajectories of $X(t)$ (being the first 4 elements of $y(t)$ in (18)) and $\text{Λ}(t)$ (being the rest elements of $y(t)$), which are time-varying. In addition, Figure 11(a) shows the characteristics of residual error $\parallel Q(t)y(t)-w(t){\parallel }_2=\parallel A(t)X(t)+{\text{Λ}}^2(t)-B(t){\parallel }_{\text{F}}$ (with symbol $\parallel \cdot {\parallel }_{\text{F}}$ denoting the Frobenius norm of a matrix), from which we can observe that the residual errors of ZNN model (18) all converge to zero. This means that the solutions of $X(t)$ and $\text{Λ}(t)$ shown in Figure 10 are the time-varying solutions of $A(t)X(t)+{\text{Λ}}^2(t)-B(t)=0$. That is, $X(t)$ satisfies $A(t)X(t)=B(t)-{\text{Λ}}^2(t)\le B(t)$, showing that such a solution is an exact time-varying solution of the time-varying LMI (15). For better understanding, Figure 11(b) shows the profiles of the testing error function $\epsilon (t)=A(t)X(t)-B(t)$, from which we can observe that all the elements of $\epsilon (t)$ are less than or equal to zero. These simulation results substantiate the efficacy of ZNN model (18) for time-varying LMI solving. Besides, Figure 12 shows the simulation results synthesized by ZNN model (18) using different γ values and different activation functions. As seen from Figure 12, the residual errors all converge to zero, which means that the time-varying LMI (15) is solved successfully via ZNN model (18). Note that, as for ZNN model (18), its computational performance can be improved by increasing the γ value and choosing a suitable activation function (as shown in Figure 12).

On the other hand, as synthesized by ZNN model (20) using $\gamma =1$ and the hyperbolic-sine activation function, Figure 13 shows the related simulation results, where some elements of the initial state $X(0)$ are outside the initial solution set $S(0)$ of the time-varying LMI (15) while the others are inside $S(0)$. From Figure 13(a), we can observe that the $X(t)$ trajectory of ZNN model (20) is time-varying. In addition, as shown in Figure 13(b), the errors ${\epsilon }_{11}(t)$ and ${\epsilon }_{21}(t)$ (being the elements of the testing error function $\epsilon (t)=A(t)X(t)-B(t)$) converge to zero, and the errors ${\epsilon }_{12}(t)$ and $e_{22}(t)$ are always equal to ${\epsilon }_{12}(0)<0$ and ${\epsilon }_{22}(0)<0$. This means that the $X(t)$ solution shown in Figure 13(a) is the time-varying solution of (15), i.e., $A(t)X(t)\le B(t)$, thereby showing the efficacy of ZNN model (20). That is, ZNN model (20) generates an exact time-varying solution of the time-varying LMI (15). Besides, the investigations on the computational performance of (10) using different γ values and different activation functions are left to interested readers to complete as a topic of exercise.

In summary, the above simulation results (i.e., Figures 10 through 13) have substantiated that the presented ZNN models (18) and (20) are both effective on time-varying LMI solving.

6. Summary

In this chapter, by exploiting two design formulas (1) and (2), based on different ZFs (i.e., (4), (9), (16) and (19)), four different ZNN models (i.e., (8), (10), (18) and (20)) have been developed and investigated to solve the time-varying LMVI (3) and time-varying LMI (15). For such ZNN models, theoretical results and analyses have also been presented to show their computational performances. Simulation results with two illustrative examples have further substantiated the efficacy of the presented ZNN models for time-varying LMVI and LMI solving.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (with number 61473323 and 61403149), and also by the Scientific Research Funds of Huaqiao University.