In this work, we derive necessary and sufficient conditions for the existence of an hierarchic equilibrium of a disturbed two player linear quadratic game with open loop information structure. A convexity condition guarantees the existence of a unique Stackelberg equilibria; this solution is first obtained in terms of a pair of symmetric Riccati equations and also in terms of a coupled of system of Riccati equations. In this latter case, the obtained equilibrium controls are of feedback type.
- differential games
- linear quadratic
- Riccati differential equations
- Stackelberg equilibrium
- worst-case disturbance
The study of linear quadratic (LQ) games has been addressed by many authors [1, 2, 3, 4]. This type of games is often used as a benchmark to assess the game equilibrium strategies and its respective outcomes. In a disturbed differential game, each player calculates its strategy taking into account a worst-case unknown disturbance. In non-cooperative game theory, the concept of hierarchical or Stackelberg games is very important, since different applications in economics and engineering exist [1, 5]. This is also the case of gas networks where a hierarchy may be assigned to its controllable elements—compressors, sources, reductors, etc… Also, for this application, the modelling as a disturbed game makes a lot of sense, since the unknown offtakes of the network can be modelled as unknown disturbances. Further research on Stackelberg games can be found for instance in AbouKandil and Bertrand ; Medanic ; Yong ; Tolwinski .
No assumptions/constraints are made of the disturbance. To be easier to understand the hierarchical concept, we consider only two players. Therefore, we study a LQ game of two players with Open Loop (OL) information structure where the players choose its strategy according to a modified Stackelberg equilibrium. Player-1 is the follower and chooses its strategy after the nomination of the strategy of the leader. Player-2, the leader, chooses its strategy assuming rationality of the follower. Both players find their strategies assuming a worst-case disturbance.
In this work, we consider a finite time horizon, where for applications this is chosen according to the periodicity of the operation of the problem being studied.
The disturbed case of the representation of optimal equilibria for noncooperative games has been studied [10, 11] considering a Nash equilibrium. It is the aim of this paper to generalise the work of Jank and Kun to Stackelberg games and extend the results presented in Freiling and Jank ; Freiling et al.  to the disturbed case. To calculate the controls, we use a value function approach, appropriately guessed. Thence, we obtain sufficient conditions of existence of these controls and its representation in terms of the solution of certain Riccati equations. Furthermore, a feedback form of the worst-case Stackelberg equilibrium is obtained.
In a future paper, we expect to present analogous conditions using an operator approach.
In Section 2, we define the disturbed LQ game and define Stackelberg worst-case equilibrium. In Section 3, we derive sufficient conditions for the existence of a worst-case Stackelberg equilibrium under OL information structure and investigate how are these solutions related to certain Riccati differential equations. Section 4 concludes the paper and outlines some directions for future work.
2. Fundamental notions
We start with the concept of best reply:
We say that is the best reply against if
holds for any strategy . We denote the set of all best replies by .
We study games of quadratic criteria, defined in a finite time horizon and subject to a linear dynamics, controlled players and also an unknown disturbance. Hereby also consider where is the information structure of Player-j. In this case, is of OL type.
the dynamics of the game are assumed to obey a linear differential equation
In this equation, where the initial and the final are finite and fixed, the state is an dimension vector of continuous functions defined in and with . The controls are square (Lebesgue) integrable and the dimension vector of continuous functions is also defined in . Also, the disturbance The different matrices are of adequate dimension and with elements continuous in .
the performance criteria are of the form
with symmetric matrices and symmetric, piecewise continuous and bounded matrix valued functions and
We observe that no cost functional is assigned to the disturbance term because no constraints can be applied on an “unpredictable” parameter. In what follows, we consider To extend the theory to since this is an hierarchical solution, we need to define the structure of the leaders and followers in the game. We can even have more than two hierachy levels. We assume that Player-2 is the leader and Player-1 is the follower.
The leader seeks a strategy in OL information structure and announces it before the game starts. This strategy is found knowing how the follower reacts to his choices. The follower calculates its strategy as a best reply to the strategy announced by the leader.
Consider the sets of functions such that (1) is solvable and exists, with in these conditions are said the sets of admissible controls and disturbance, respectively.
A function is called the worst-case disturbance, from the point of view of the th player belonging to the set of admissible controls, ifE5
holds for each . There exists exactly one worst-case disturbance from the point of view of the th player according to every set of controls.
We say that the controls form a worst-case Stackelberg equilibrium if
The leader chooses such that
The follower then chooses such that
To guarantee the uniqueness of OL Stackelberg solutions, matrices are assumed to satisfy and in Simaan and Cruz .
In what follows, we drop the dependence of the parameters in to reduce the length of the formulas.
3. Sufficient conditions for the existence of OL Stackelberg equilibria
In this section, we withdraw sufficient conditions for the existence of the worst-case Stackelberg equilibrium, using a value function approach.
A disturbed differential LQ game as defined in Definition 2.2 is said
with and exist on .
For any given admissible OL control of the leader, define by
Then, the following identity holds:
and a solution of (1).
where the maximum disturbance,
The corresponding minimal costs then are
The cost functional minimal value is obtained when we substitute in (9) the minimal control and themaximal disturbance.
Notice that is not depending on This, as a matter of fact, is only true if we consider OL information structure, since otherwise would depend on the trajectory and hence, via (1), also on In OL Stackelberg games, the leader tries next to find an optimal OL control that minimises while is defined by (10).
with and Also and exist in , where Also
For any given control of the leader, define functions and in by the following initial and terminal value problems:
Then, we obtain
and the following identity
and the dimensional zero matrix and and
Hence defining and We define to write these two equations as: (??) as:
Next, we consider the following value function
for some mappings , and where is symmetric for each
Now we associate certain terms
If then If then
Define and . Substitute this and in the calculations:
We end up with
Further, we assume the mappings to be chosen in such a way that the following terminal values hold
Then, we obtain and substituting:
Now, we substitute this into (23) and consider the infimal values over all possible control functions in
then we have:
equals if and As the leader chooses his strategy assuming rationality of the follower and worst-case disturbance, the follower should take also the worst-case disturbance into account.
To conclude, consider and hence
Then from (21):
Defining and we have:
Now, we substitute
The leader may choose its best answer either by accounting directly for its worst-case disturbance or by considering that the follower knows that there is a worst-case disturbance. In this work, the leader takes the worst-case disturbance directly into account.
Notice that in the term
do not depend on the choice of Since we shall study the situation for Player-2 when Player-1 applies his optimal response control defined in (10), we have to set From (7), we can see that depends on and hence also on
In order to derive from Theorems (3.1) and (3.3) sufficient conditions for the existence of a unique worst-case Stackelberg equilibrium, we must get rid of the -dependence on Therefore, we propose to restrict the set of admissible controls to functions representable in linear feedback form. This is what we do next.
admits a solution in .
Then, there exists a unique open loop disturbed Stackelberg equilibrium in feedback synthesis which is given by
considering worst-case disturbamces and where is a solution of the closed loop equation
The minimal cost for the follower, is as in (14), and for the leader is
From the convexity assumptions, it follows that and are all semidefinite. Therefore, as far as the convexity conditions hold, the standard Riccati matrix Eqs. (6) and (15) are globally solvable in .
It still remains the following questions to be answered (i) direct criteria for solvability of these equations if the convexity assumption is guaranteed as well as (ii) solvability of the coupled system of Eqs. (25)–(27).
Actually, this system of equations can also be written as a single, nonsymmetric Riccati matrix differential equation. Hence:
As it can be easily observed, all these Riccati equations are of nonsymmetric type:
where is a matrix of order whose coefficients are of adequate size. See AbouKandil et al.  for results on the existence of solution of Riccati equations.
4. Discussion and conclusions
High dimension problems appeal to the use of hierarchic and decentralised models as differential games. One example of these problems is large networks, as for instance the management and control of high pressure gas networks. Since this is a large dimension and geographically dispersed problem, a decentralised formulation captures the non-cooperative nature, and sometimes even antagonistic, of the different stake-holders in the network.
The network controllable elements can be seen as players that seek their best settings and then interact among themselves to check for network feasibility. The equilibrium sought by the players depends on the way the players are organised among themselves. It makes some sense to have some autonomous elements that run the network and others follow, as is the case of a main inlet point of a country, as it happens with the inlet of Sines in the portuguese network. The ultimate goal of the network is to meet customers’ demand at the lowest cost. As the main variation of the problem is due to the off-takes, these may be seen as perturbations to nominal consumption levels of a deterministic model.
Therefore, it makes some sense to view the gas transportation and distribution system as a disturbed Stackelberg game where the players play against a worst-case disturbance, that means a sudden change in weather conditions from one period of operation to the other. Neverthless, the theory is not ready, and also having in mind the development of algorithms, direct solution methods, and explicit solution representations need to be further investigated. In this work, we have obtained sufficient conditions for the existence of the solution of a 2-player game. However, direct criteria for solvability of this problem needs more work. Also, the solvability of the coupled system of Eqs. (25)–(27) has to be further investigated. Also, we would like to solve the same problem using an operator approach.
Similarly to what we have done in the past for Nash games, we would like to study this problem considering the underlying dynamics as a repetitive process, that seems to be adequate to capture the behaviour seemingly periodic of the network. Also, the boundary control of the network depends on the type of strategy sought by the players. The structure of these versions of the problems need to be examined.
The obtained results, in every stage of the work, should be applied to a single pipe and ideally using some operational data. Furthermore, we expect to apply the work to a simple network, which is not exactly a straightforward extension.
I would like to thank the reviewer for his valuable suggestions.
This work has been financed by National Funds through the Portuguese funding agency, FCT – Fundao para a Cincia e a Tecnologia under project: (i) UID/EEA/00048/2019 for the first author and (ii) UID/EEA/50014/2019 for the third author.