In this chapter, we discuss a leader-follower (also called Stackelberg) stochastic differential game with asymmetric information. Here the word “asymmetric” means that the available information of the follower is some sub- σ -algebra of that available to the leader, though they play as different roles in the classical literatures. Stackelberg equilibrium is represented by the stochastic versions of Pontryagin’s maximum principle and verification theorem with partial information. A linear-quadratic (LQ) leader-follower stochastic differential game with asymmetric information is studied as applications. If some system of Riccati equations is solvable, the Stackelberg equilibrium admits a state feedback representation.
- backward stochastic differential equation (BSDE)
- leader-follower stochastic differential game
- asymmetric information
- stochastic filtering
- linear-quadratic control
- Stackelberg equilibrium
Throughout this chapter, we denote by the Euclidean space of -dimensional vectors, by the space of matrices, by the space of symmetric matrices. and denote the scalar product and norm in the Euclidean space, respectively. appearing in the superscripts denotes the transpose of a matrix. denote the partial derivative and twice partial derivative with respect to for a differentiable function .
In practice, there are many problems which motivate us to study the leader-follower stochastic differential games with asymmetric information. Here we present two examples.
Example 1.1: (Continuous time principal-agent problem) The principal contracts with the agent to manage a production process, whose cumulative proceeds (or output) evolve on as follows:
where is the agent’s effort choice, represents the productivity of effort, and there are two additive shocks (due to the two independent Brownian motions ) to the output. The proceeds of the production add to the principal’s asset , which earns a risk free return , and out of which he pays the agent and withdraws his own consumption . Thus the principal’s asset evolves as
where is the initial asset. The agent has his own wealth , out of which he consumes , thus
Thus, the agent earns the same rate of return on his savings, gets income flows due to his payment , and draws down wealth to consume. In the above are all constants. At the terminal time , the principal makes a final payment and the agent chooses consumption based on this payment and his terminal wealth . In the above, we restrict to be nonnegative.
We consider an optimal implementable contract problem in the so-called “hidden savings” information structure (Williams , also in Williams ). In this problem, the principal can observe his asset and the agent’s initial wealth but cannot monitor the agent’s effort , consumption , and wealth for . The principal must provide incentives for the agent to put forth the desired amount of the effort. For any , the agent first chooses his effort and consumption , such that his exponential preference
is maximized. Here is the discount rate and denotes the risk aversion parameter. The above is called an implementable contract if it meets the recommended actions of the principal’s, which is based on the principal’s observable wealth . Then, the principal selects his payment and consumption to maximize his exponential preference
Let denote the -algebra generated by Brownian motions . Intuitively, contains all the information up to time . Let contains the information available to the agent, and contains the information available to the principal, up to time respectively. Moreover, . In the game problem, first the agent solves the following optimization problem:
where is a -adapted process pair. And then the principal solves the following optimization problem:
where is a -adapted process pair. This formulates a stochastic Stackelberg differential game with asymmetric information. In this setting, the agent is the follower and the principal is the leader. Any process quadruple satisfying the above two equalities is called a Stackelberg equilibrium. In Williams , a solvable continuous time principal-agent model is considered under three information structures (full information, hidden actions, and hidden savings) and the corresponding optimal contract problems are solved explicitly. But it can not cover our model.
Example 1.2: (Continuous time manufacturer-newsvendor problem) Let be the demand rate for a product in the market, which satisfies
where are constants. Suppose that the market is consisted with a manufacturer selling the product to end users through a retailer. At time , the retailer chooses an order rate for the product and decides its retail price , and is offered a wholesale price by the manufacturer. We assume that items can be salvaged at unit price , and that items cannot be stored, that is, they must be sold instantly or salvaged. The retailer will obtain an expected profit
When the manufacturer has a fixed production cost per unit , he will get an expected profit
In the above, we assume that .
Let denote the -algebra generated by , which contains all the information up to time . At time , let the information available to the retailer and the manufacturer, respectively, are both sub--algebras of . Moreover, . This can be explained from the practical application’s aspect. Specifically, the manufacturer chooses a wholesale price at time , which is a -adapted stochastic process. And the retailer chooses an order rate and a retail price at time , which are -adapted stochastic processes. For any , to select a -adapted process pair for the retailer such that
and then to select a -adapted process for the manufacturer such that
formulates a leader-follower stochastic differential game with asymmetric information. In this setting, the manufacturer is the leader and the retailer is the follower. Any process triple satisfying the above is called a Stackelberg equilibrium. In Øksendal et al. , a time-dependent newsvendor problem with time-delayed information is solved, based on stochastic differential game (with jump-diffusion) approach. But it cannot cover our model.
1.2. Problem formulation
Motivated by the examples earlier, in this chapter we study the leader-follower stochastic differential games with asymmetric information. Let be a complete probability space. is a standard -valued Brownian motion and be its natural augmented filtration and where is a finite time duration. Let the state satisfy the stochastic differential equation (SDE)
where and are control processes taken by the two players in the game, labeled 1 (the follower) and 2 (the leader), with values in nonempty convex sets , , respectively. , the solution to SDE Eq. (13) with values in , is the state process with initial state . Here are given -adapted processes, for each .
Let us now explain the asymmetric information character between the follower (player 1) and the leader (player 2) in this chapter. Player 1 is the follower, and the information available to him at time is based on some sub--algebra , where is the information available to the leader. We assume in this and next sections that . We define the admissible control sets of the follower and the leader, respectively, as follows.
The game initiates with the announcement of the leaders control . Knowing this, the follower would like to choose a -adapted control to minimize his cost functional
Here is an -adapted process, and is an -measurable random variable, for each . Now the follower encounters a stochastic optimal control problem with partial information.
SOCPF. For any chosen by the leader, choose a -adapted control , such that
In the following step, once knowing that the follower will take such an optimal control , the leader would like to choose a -adapted control to minimize his cost functional
Here are given -adapted processes, for each . Now the leader encounters a stochastic optimal control problem with partial information.
SOCPL. Find a -adapted control , such that
subject to Eqs. (13) and (17). Such a is called an optimal control, and the corresponding solution to Eq. (13) is called an optimal state. We will rewrite the problem for the leader in more detail in the next section. We refer to the problem mentioned above as a leader-follower stochastic differential game with asymmetric information. If there exists a control process pair satisfying Eqs. (16) and (18), we refer to it as a Stackelberg equilibrium.
In this chapter, we impose the following assumptions.
(A1.1) For each , the functions are twice continuously differentiable in . For each , functions and are continuously differentiable in and , respectively. Moreover, for each and any , there exists such that
for , and
1.3. Literature review and contributions of this chapter
Differential games are initiated by Issacs , which are powerful in modeling dynamic systems where more than one decision-makers are involved. Differential games have been researched by many scholars and have been applied in biology, economics, and finance. Stochastic differential games are differential games for stochastic systems involving noise terms. See Basar and Olsder  for more information about differential games. Recent developments for stochastic differential games can be seen in Hamadène , Wu , An and Øksendal , Wang and Yu [9, 10], and the references therein.
Leader-follower stochastic differential game is the stochastic and dynamic formulation of the Stackelberg game, which was introduced by Stackelberg  in 1934, when the concept of a hierarchical solution for markets where some firms have power of domination over others, is defined. This solution concept is now known as the Stackelberg equilibrium, which in the context of two-person nonzero-sum games, involves players with asymmetric roles, one leader and one follower. Pioneer study for stochastic Stackelberg differential games can be seen in Basar . Specifically, a leader-follower stochastic differential game begins with the follower aims at minimizing his cost functional in response to the leader’s decision on the whole duration of the game. Anticipating the follower’s optimal decision depending on his entire strategy, the leader selects an optimal strategy in advance to minimize his cost functional, based on the stochastic Hamiltonian system satisfied by the follower’s optimal decision. The pair of the leader’s optimal strategy and the follower’s optimal response is known as the Stackelberg equilibrium.
A linear-quadratic (LQ) leader-follower stochastic differential game was studied by Yong  in 2002. The coefficients of the the cost functionals and system are random, the diffusion term of the state equation contain the controls, and the weight matrices for the controls in the cost functionals are not necessarily positive definite. The related Riccati equations are derived to give a state feedback representation of the Stackelberg equilibrium in a nonanticipating way. Bensoussan et al.  obtained the global maximum principles for both open-loop and closed-loop stochastic Stackelberg differential games, whereas the diffusion term does not contain the controls.
In this chapter, we study a leader-follower stochastic differential game with asymmetric information. Our work distinguishes itself from these mentioned above in the following aspects. (1) In our framework, the information available to the follower is based on some sub--algebra of that available to the leader. Moreover, both information filtration available to the leader and the follower could be sub--algebras of the complete information filtration naturally generated by the random noise source. This gives a new explanation for the asymmetric information feature between the follower and the leader, and endows our problem formulation more practical meanings in realty. (2) Our work is established in the context of partial information, which is different from that of partial observation (see e.g., Wang et al. ) but related to An and Øksendal , Huang et al. , Wang and Yu . (3) An important class of LQ leader-follower stochastic differential game with asymmetric information is proposed and then completely solved, which is a natural generalization of that in Yong . It consists of a stochastic optimal control problem of SDE with partial information for the follower, and followed by a stochastic optimal control problem of forward-backward stochastic differential equation (FBSDE) with complete information for the leader. This problem is new in differential game theory and have considerable impacts in both theoretical analysis and practical meaning with future application prospect, although it has intrinsic mathematical difficulties. (4) The Stackelberg equilibrium of this LQ problem is characterized in terms of the forward-backward stochastic differential filtering equations (FBSDFEs) which arises naturally in our setup. These FBSDFEs are new and different from those in [10, 16]. (5) The Stackelberg equilibrium of this LQ problem is explicitly given, with the help of some new Riccati equations.
The rest of this chapter is organized as follows. In Section 2, we solve our problem to find the Stackelberg equilibrium. In Section 3, we apply our theoretical results to an LQ problem. Finally, Section 4 gives some concluding remarks.
2. Stackelberg equilibrium
2.1. The Follower’s problem
In this subsection, we first solve SOCPF. For any chosen , let be an optimal control for the follower and the corresponding optimal state be . Define the Hamiltonian function as
Let an -adapted process triple satisfies the adjoint BSDE
Proposition 2.1 Let (A1.1) hold. For any given , let be the optimal control for SOCPF, and be the corresponding optimal state. Let be the adjoint process triple. Then we have
holds, for any .
Proof Similar to the proof of Theorem 2.1 of , we can get the result.
Proposition 2.2 Let (A1.1) hold. For any given , let and be the corresponding state. Let be the adjoint process triple. For each , is concave, is convex, and
holds for , a.s. Then is an optimal control for SOCPF.
Proof Similar to the proof of Theorem 2.3 of , we can obtain the result.
2.2. The Leader’s problem
In this subsection, we first state the SOCPL. Then, we give the maximum principle and verification theorem. For any , by Eq. (23), we assume that a functional is uniquely defined, where
For the simplicity of notations, we denote and define on as , for , respectively. Then after substituting the above control process into Eq. (22), the leader encounters the controlled FBSDE system
Note that Eq. (26) is a controlled conditional mean-field FBSDE, which now is regarded as the “state” equation of the leader. That is to say, the state for the leader is the quadruple .
Remark 2.1 The equality does not hold in general. However, for LQ case, it is satisfied and we will make this point clear in the next section.
where . Note the cost functional of the leader is also conditional mean-field’s type. We propose the stochastic optimal control problem with partial information of the leader as follows.
SOCPL. Find a -adapted control , such that
Let be an optimal control for the leader, and the corresponding state is the solution to Eq. (26). Define the Hamiltonian function of the leader as
Let for and all their derivatives. Suppose that is the unique -adapted solution to the adjoint conditional mean-field FBSDE of the leader
Now, we have the following two results.
Proposition 2.3 Let (A1.1) hold. Let be an optimal control for SOCPL and be the optimal state. Let be the adjoint quadruple, then
Proof The maximum condition Eq. (31) can be derived by convex variation and adjoint technique, as Anderson and Djehiche . We omit the details for saving space. See also Li , Yong  and the references therein for mean-field stochastic optimal control problems.
Proposition 2.4 Let (A1.1) hold. Let and be the corresponding state, with . Let be the adjoint quadruple. For each , suppose that and are convex, and
Then is an optimal control for SOCPL.
Proof This follows similar to Shi . We omit the details for simplicity.
3. Applications to LQ case
In order to illustrate the theoretical results in Section 2, we study an LQ leader-follower stochastic differential game with asymmetric information. In this section, we let and . This game is a special case of the one in Section 2, but the resulting deduction is very technically demanding. We split this section into two subsections, to deal with the problems of the follower and the leader, respectively.
3.1. Problem of the follower
Suppose that the state satisfies a linear SDE
Here, is the follower’s control process and is the leader’s control process, which take values both in ; are constants. In the first step, for announced , the follower would like to choose a -adapted, square-integrable control to minimize the cost functional
In the second step, knowing that the follower would take , the leader wishes to choose an -adapted, square-integrable control to minimize
where are constants. This is an LQ leader-follower stochastic differential game with asymmetric information. We wish to find its Stackelberg equilibrium .
Define the Hamiltonian function of the follower as
For given control , suppose that there exists a -adapted optimal control of the follower, and the corresponding optimal state is . By Proposition 2.1, Eq. (36) yields that
where the -adapted process triple satisfies the BSDE
We wish to obtain the state feedback form of . Noting the terminal condition of Eq. (38) and the appearance of , we set
for some deterministic and differentiable -valued function , and -valued, -adapted process which admits the BSDE
In the above equation, are -adapted processes, which are to be determined later. Now, applying Itô’s formula to Eq. (39), we have
Note that Eq. (46) is not a classical FBSDFE, since the generator of the BSDE depends on an additional process . For given , it is important if Eq. (46) admits a unique -adapted solution . We will make it clear soon. For this target, first, by Eq. (37) and supposing that.
(A2.1) , ,
we immediately arrive at
admits a unique differentiable solution , then
which admits a unique -adapted solution . By Eqs. (55), (44), (45), and (47), we can uniquely obtain the solvability of Eq. (46). Moreover, we can check that the convexity/concavity conditions in Proposition 2.2 hold, and given by Eq. (47) is really optimal. We summarize the above procedure in the following theorem.
3.2. Problem of the leader
Since the leader knows that the follower will take by Eq. (47), the state equation of the leader writes
where and . Noting that Eq. (56) is a decoupled conditional mean-field FBSDE, its solvability for -adapted solution can be easily guaranteed.
The problem of the leader is to choose an -adapted optimal control such that the cost functional
is minimized. Define the Hamiltonian function of the leader as
Suppose that there exists an -adapted optimal control of the leader, and the corresponding optimal state is . Then by Propositions 2.3, 2.4, Eq. (58) yields that
where the -adapted process satisfies
In fact, the problem of the leader can also be solved by a direct calculation of the derivative of the cost functional. Without loss of generality, let , and set for sufficiently small, with . Then it is easy to see from the linearity of Eqs. (56) and (60), that the solution to Eq. (56) is . We first have
Let the -adapted process quadruple satisfy Eq. (60). Then we have
This implies Eq. (59).
In the following, we wish to obtain a “nonanticipating” representation for the optimal controls and . For this target, let us regard as the optimal state, put
and (suppressing some below)
Noting Eq. (59), we have
Applying Itô’s formula to (3.31), we get
Comparing and on both sides of Eq. (73), we have
Taking , we derive
Supposing that (denotes the unit matrix)
Note that the system of Riccati equations (83) is not standard, and its solvability is open. Due to some technical reason, we can not obtain the solvability of it now. However, in some special case, and are not coupled. Then we can first solve the first equation of , then that of by standard Riccati equation theory. We will not discuss for the space limit. And we will consider the general solvability of Eq. (83) in the future.
and the optimal “state” of the leader satisfies
where is governed by
We summarize the above analysis in the following theorem.
Theorem 3.2 Let (A2.1)(A2.3) hold, satisfy Eq. (83), be the -adapted solution to Eq. (86), and be the -adapted solution to Eq. (85). Define by Eqs. (71) and (81), respectively. Then Eq. (69) holds, and given by Eq. (84) is a feedback optimal control of the leader.
which is observable for the follower.
Remark 3.3 When we consider the complete information case, that is, disappears and , Theorems 3.1 and 3.2 coincide with Theorems 2.3 and 3.3 in Yong .
4. Concluding remarks
In this chapter, we have studied a leader-follower stochastic differential game with asymmetric information. This kind of game problem possesses several attractive features. First, the game problem has the Stackelberg feature, which means the two players play as different roles during the game. Thus the usual approach to deal with game problems, such as [6, 7, 8, 10], where the two players act as equivalent roles, does not apply. Second, the game problem has the asymmetric information between the two players, which was not considered in [3, 13, 14]. In detail, the information available to the follower is based on some sub--algebra of that available to the leader. Stochastic filtering technique is introduced to compute the optimal filtering estimates for the corresponding adjoint processes, which act as the solution to some FBSDFE. Third, the Stackelberg equilibrium is represented in its state feedback form for the LQ problem under some appropriate assumptions. Some new conditional mean-field FBSDEs and system of Riccati equations are introduced to deal with the leader’s LQ problem.
In principle, Theorems 3.1 and 3.2 provide a useful tool to seek Stackelberg equilibrium. As a first step in this direction, we apply our results to the LQ problem to obtain explicit solutions. We hope to return to the more general case in our future research. It is worthy to study the closed-loop Stackelberg equilibrium for our problem, as well as the solvability of the system of Riccati equations. These challenging topics will be considered in our future work.
Jingtao Shi would like to thank the book editor for his/her comments and suggestions. Jingtao Shi also would like to thank Professor Guangchen Wang from Shandong University and Professor Jie Xiong from Southern University of Science and Technology, for their effort and discussion during the writing of this chapter.
The main content of this chapter is from the following two published article papers: (1) Shi, J.T., Wang, G.C., & Xiong, J. (2016). Leader-follower stochastic differential games with asymmetric information and applications. Automatica, Vol. 63, 60–73. (2) Shi, J.T., Wang, G.C., & Xiong, J. (2017). Linear-quadratic stochastic Stackelberg differential game with asymmetric information. Science China Information Sciences, Vol. 60, 092202:1–15.