Stochastic Control for Jump Diffusions *

In this chapter, we will discuss the stochastic optimal control problem for jump diffusions. That is, the controlled stochastic system is driven by both Brownian motion and Poisson random measure and the controller wants to minimize/maximize some cost functional subject to the above stated state equation (stochastic control system) over the admissible control set. This kind of stochastic optimal control problems can be encountered naturally when some sudden and rare breaks take place, such as in the practical stock price market. An admissible control is called optimal if it achieves the infimum/supremum of the cost functional and the corresponding state variable and the cost functional are called the optimal trajectory and the value function, respectively.


Introduction
In this chapter, we will discuss the stochastic optimal control problem for jump diffusions.That is, the controlled stochastic system is driven by both Brownian motion and Poisson random measure and the controller wants to minimize/maximize some cost functional subject to the above stated state equation (stochastic control system) over the admissible control set.This kind of stochastic optimal control problems can be encountered naturally when some sudden and rare breaks take place, such as in the practical stock price market.An admissible control is called optimal if it achieves the infimum/supremum of the cost functional and the corresponding state variable and the cost functional are called the optimal trajectory and the value function, respectively.
It is well-known that Pontryagin's maximum principle (MP for short) and Bellman's dynamic programming principle (DPP for short) are the two principal and most commonly used approaches in solving stochastic optimal control problems.In the statement of maximum principle, the necessary condition of optimality is given.This condition is called the maximum condition which is always given by some Hamiltonian function.The Hamiltonian function is defined with respect to the system state variable and some adjoint variables.The equation that the adjoint variables satisfy is called adjoint equation, which is one or two backward stochastic differential equations (BSDEs for short) of [13]'s type.The system which consists of the adjoint equation, the original state equation, and the maximum condition is referred to as a generalized Hamiltonian system.On the other hand, the basic idea of dynamic programming principle is to consider a family of stochastic optimal control problems with different initial time and states and establish relationships among these problems via the so-called Hailton-Jacobi-Bellman (HJB for short) equation, which is a nonlinear second-order partial differential equation (PDE for short).If the HJB equation is solvable, we can obtain an optimal control by taking the maximizer/miminizer of the generalized Hamiltonian function involved in the HJB equation.To a great extent these two approaches have been developed separately and independently during the research in stochastic optimal control problems.
Hence, a natural question arises: Are there any relations between these two methods?In fact, the relationship between MP and DPP is essentially the relationship between the adjoint processes and the value function, or the Hamiltonian systems and the HJB equations or even more generally, the relationship between stochastic differential equations (SDEs for short) and PDEs.Such a topic was intuitively discussed by [5], [4] and [9].However, an important issue in studying the problem is that the derivatives of the value functions are unavoidably involved in these results.In fact, the value functions are usually not necessarily smooth.[19] first obtained the nonsmooth version of the relationship between MP and DPP using the viscosity solution and the second-order adjoint equation.See also the book by [18].
The aim of this chapter is to establish the relationship between MP and DPP within the framework of viscosity solutions in the jump diffusion setting.In this case, the state trajectory is described by a stochastic differential equation with Poisson jumps (SDEP for short).That is to say, the system noise (or the uncertainty of the problem) comes from a Brownian motion and a Poisson random measure.See [15] for theory and applications of this kind of equations.[16] proved the general MP where the control variable is allowed into both diffusion and jump coefficients.HJB equation for optimal control of jump diffusions can be seen in [12], which here is a second-order partial integral-differential equation (PIDE for short).[7] gave a sufficient MP by employing Arrow's generalization of the Mangasarian sufficient condition to the jump diffusion setting.Moreover, on the assumption that the value function is smooth, they showed the adjoint processes' connections to the value function.Let us state some results of [7] in detail with a slight modification to adapt to our setting.
However, it seems that the above HJB equation ( 2) and the relationship (3) lack generality, since they require the value function to be smooth, which is not true even in the simplest case; see Example 3.2 of this chapter.This is an important gap in the literature [7].The aim of this chapter is to bridge this gap by employing the notion of semijets evoked in defining the viscosity solutions.
The contribution of this chapter is as follows.Firstly, we give some basic properties of the value function and prove that the DPP still holds in our jump diffusion setting.Then we give the corresponding generalized HJB equation which now is a second-order PIDE.Secondly, we investigate the relationship between MP and DPP without assuming the continuous differentiablity of the value function.We obtain the relationship among the adjoint processes, the generalized Hamiltonian and the value function by employing the notions of the set-valued semijets evoked in defining the viscosity solutions, which is now interpreted as a set inclusion form among subjet, superjet of the value function, set contain adjoint processes and some "G-function" (see the definition in Section 2).It is worth to pointed out that the controlled jump diffusions bring much technique difficulty to obtain the above results.In fact, the solution of the control system is not continuous with jump diffusions.We overcome these difficulty and get the desired results in this chapter which have wide applicable background.
The rest of this chapter is organized as follows.In Section 2, for stochastic optimal control problem of jump diffusions, we give some basic properties of the value function and then set out the corresponding DPP and MP, respectively.In Section 3, the relationship between MP and DPP is proved using the notion of viscosity solutions of PIDEs.Some concluding remarks are given in Section 4.

Problem statement and preliminaries
Throughout this chapter, we denote by R n the space of n-dimensional Euclidean space, by R n×d the space of matrices with order n × d, by S n the space of symmetric matrices with order n × n.Let E ⊂ R l be a nonempty Borel set equipped with its Borel field B(E).Let π(•) be a bounded positive measure on (E, B(E)).We denote by Let T > 0 and let (Ω, F , P) be a complete probability space, equipped with a d-dimensional standard Brownian motion {W(t)} 0≤t≤T and a Poisson random measure N(•, •) independent of W(•) with the intensity measure N(dedt)=π(de)dt.We write Ñ(dedt) := N(dedt) − π(de)dt for the compensated Poisson martingale measure.For a given s ∈ [0, T), we suppose the filtration {F s t } s≤t≤T is generated as the following where N contains all P-null sets in F and σ 1 σ 2 denotes the σ-field generated by σ 1 ∪ σ 2 .I n particular, if s = 0 we write F t ≡F s t .
Let U be a nonempty Borel subset of R k .For any initial time s ∈ [0, T) and initial state y ∈ R n , we consider the following stochastic control system which is called a controlled jump diffusion process x s,y;u (s)=y. Here For a given s ∈ [0, T), we denote by U [s, T] the set of U-valued F s t -predictable processes.For given u(•) ∈U[s, T] and y ∈ R n ,aR n -valued process x s,y;u (•) is called a solution of (4) if it is an F s t -adapted RCLL (i.e., right-continuous with left-hand limits) process such that (4) holds.It is called the state trajectory corresponding to the control u(•) ∈U[s, T] with initial state y.We refer to such u(•) ∈U[s, T] as an admissible control and (x s,y;u (•), u(•)) as an admissible pair.For any s ∈ [0, T), we introduce the following notations.
We consider the following cost functional J(s, y; u(•)) = E T s f (t, x s,y;u (t), u(t))dt + h(x s,y;u (T)) , where f : [0, T] × R n × U → R, h : R n → R are given functions.For given (s, y) ∈ [0, T) × R n , the stochastic optimal control problem is to minimize (5) In this section, we first give some basic properties of the value function.Then we prove that the DPP still holds and introduce the generalized HJB equation and the generalized Hamiltonian function.The idea of proof is originated from [17], [18] while on some different assumptions.Then we introduce the Hamiltonian function, the H-function and adjoint processes, then give the MP which is a special case of [16].
We first discuss the DPP and make the following assumptions.

Remark 2.1
We point out that assumption (H2) on f , h allows them to have polynomial (in particular, quadratic) growth in x, provided that b, σ, c have linear growth.A typical example is the stochastic linear quadratic (LQ for short) problem.Note that (H2) is different from assumptions (2.5), pp. 4 of [14] and assumption (S2)', pp.248 of [18], where global Lipschitz condition is both imposed thus f , h are global linear growth in x.
We need the following lemma.
Lemma 2.1 Let (H1) hold.For k = 2, 4, there exists C > 0 such that for any 123 Stochastic Control for Jump Diff usions Estimates of the moments for SDEPs are proved in Lemma 3.1 of [14] for k ∈ [0, 2].I n fact, under assumption (H1) we can easily extend his result to the case k = 2, 4 by virtue of Buckholder-Davis-Gundy's inquality.We leave the detail of the proof to the interested reader.
We give some basic continuity properties of the value function V.The proof is similar to Proposition 2.2, Chapter 2 of [17].We omit the detail.Proposition 2.1 Let (H1), (H2) hold.Then there exist increasing, continuous functions ω1 : The following result is a version of Bellman's principle of optimality for jump diffusions.
The following result is to get the generalized HJB equation and its proof is similar to Proposition 3.4, Chapter 4 of [18].
Theorem 2.2 Suppose that (H1), (H2) hold and the value function Then V is a solution of the following generalized HJB equation which is a second-order PIDE: where, associated with a In the following, we discuss the stochastic MP and need the following hypothesis.
There exists a constant C > 0 and a modulus of continuity ω : For simplicity, we introduce the following notations.
and similar notations used for all their derivatives, for all t ∈ [0, T], u ∈ U.
Remark 2.2 Note that the integrand with respect to the compensated martingale measure Ñ in the second-order adjoint equation enters into the above maximum condition, while the counterpart in the diffusion case does not!This marks one essential difference of the maximum principle between an optimally controlled diffusion (continuous) process and an optimally controlled jump (discontinuous) process.

Relationship between Stochastic MP and DPP
In this section, we will establish the relationship between stochastic MP and DPP in the language of viscosity solutions.That is to say, we will consider the viscosity solutions of the generalized HJB equation (15).In our jump diffusion setting, we need use the viscosity solution theory for second-order PIDEs.For convenience, we refer to [1], [2], [14], [3], [11], [6] for a deep investigation of PIDEs in the framework of viscosity solutions.In Subsection 3.1, we first present some preliminary results concerning viscosity solutions and semijets.Then we give the relationship between stochastic MP and DPP in Subsection 3.2.Special cases on the assumption that the value function is smooth are given as corollaries.Some examples are also given to illustrate our results.

Preliminary results: viscosity solutions and semijets
To make the chapter self-contained, we present the definition of viscosity solutions and semijets, which is frequently seen in the literature and we state it adapting to the generalized HJB equation ( 15) in our jump diffusion setting.

and for any test function
) is both a viscosity subsolution and viscosity supersolution of ( 15), then it is called a viscosity solution of (15).
In order to give the existence and uniqueness result for viscosity solution of the generalized HJB equation (15), it is convenient to give an intrinsic characterization of viscosity solutions.Let us recall the right parabolic super-subjets of a continuous function on [0, T] × R n (see [18] or [3]).
is the set triple and the right parabolic subjet of v at ( t, x) is the set 127 Stochastic Control for Jump Diff usions From the above definitions, we see immediately that where S n + := {S ∈S n |S ≥ 0}, and A ± B := {a ± b|a ∈ A, b ∈ B} for any subsets A and B in a same Euclidean space.
The following result is useful and whose proof for diffusion case can be found, for instance, in Lemma 5.4, Chapter 4 of [18].
) is both a viscosity subsolution and viscosity supersolution of ( 15), then it is called a viscosity solution of (15).Proof The result is immediate in view of Proposition 3.1.In fact it is a special case of Proposition 1 of [3].
The following result is the existence and uniqueness of viscosity solution of the generalized HJB equation (15).
129 Stochastic Control for Jump Diff usions

Main results: Relationship between Stochastic MP and DPP
Proposition 2.1 tell us that the value function has nice continuity properties.But in general we cannot obtain the differentiablity of it.Therefore we should not suppose the generalized HJB equation (15) always admits an enough smooth (classic) solution.In fact it is not true even in the simplest case; see Example 3.2 in this subsection.This is an important gap in the literature (see Section 2, [7], for example).Fortunately, this gap can be bridged by the theory of viscosity solutions.This is one of the main contributions of this paper.
The following result shows that the adjoint process p, P and the value function V relate to each other within the framework of the superjet and the subjet in the state variable x along an optimal trajectory.Theorem 3.2 Suppose (H1)∼(H3) hold and let (s, y) ∈ [0, T) × R n be fixed.Let ( xs,y; ū (•), ū(•)) be an optimal pair of our stochastic optimal control problem.
We point out that in our jump diffusion setting the strict inequality V xx (t, xs,y; ū (t)) < −P(t) may happen in some cases, as shown in the following example.
Example 3.1 Consider the following linear stochastic control system with Poisson jumps x s,y;u (s)=y, for some δ = 0.Here N is a Poisson process with the intensity λdt and Ñ(dt) := N(dt) − λdt(λ > 0) is the compensated martingale measure.The quadratic cost functional is taken as x s,y;u (t) 2 dt.For any u(•) ∈U[s, T], applying Itô's formula to φ(t)x s,y;u (t) 2 , we have Integrating from s to T, taking expectation on both sides, we have This implies that ū(t)=− x s,y;u (t) λδ 2 , ∀t ∈ [s, T], is a state feedback optimal control and the value function is On the other hand, the second-order adjoint equation is which implies Then the decreasing property of function ρ(t) ≡ V xx (t, xs,y; ū(t)) + P(t) (noting ρ(T)=0) results in We proceed to study the super-subjets of the value function in the time variable t along an optimal trajectory.Different from its deterministic counterpart (see [18]) and similar to but more complicated than its diffusion (without jumps) counterpart (see [18] or [19]), we observe that it is not the generalized Hamiltonian G that is to be maximized in the stochastic MP unless V is sufficiently smooth.Instead, it is the following G-function which contains an additional term than the H-function in the stochastic MP (Theorem 2.3).Associated with an optimal pair ( xs,y; ū(•), ū(•)), its corresponding adjoint processes (p(•), q(•), γ(•, •)) and , e) ⊤ + Δc(t, e; u)Δc(t, e; u) ⊤ π(de) ⊤ π(de) = H(t, x, u, p(t), q(t), γ(t, •)) − 1 2 tr P(t) σ(t) σ(t) ⊤ + E c(t, e) c(t, e) ⊤ π(de) Note that, unlike the definition of generalized Hamiltonian function G, the G-function can be defined associated with only We first recall a few results on right Lesbesgue points for functions with values in abstract spaces (see also in pp.2013-2014 of [8]).

Definition 3.3
Let Z be a Banach space and let z : [a, b] → Z be a measurable function that is Bochner integrable.We say that t is a right Lesbesgue point of z if The second main result in this subsection is the following.
From this and (35), we have (64).The proof is complete.Theorem 3.4 is a generalization of the classical result on the relationship between stochastic MP and DPP (see Theorem 2.1 of [7]).On the other hand, we do have a simple example showing that both the set inclusions in (63) and (64) may be strict.
(65) Thus, it is clear that both the set inclusions in (63) and (64) of Theorem 3.4 are strict for t ∈ [0, T).
The following result is the special case when we assume the value function is enough smooth.
Corollary 3.1 Suppose that (H1)∼(H3) hold and let (s, y) ∈ [0, T) × R n be fixed.Let ( xs,y; ū(•), ū(•)) be an optimal pair of our stochastic optimal control problem and (p(•), q(•), γ(•, •)) be the first-order adjoint processes.Suppose the value function V ∈ C where G is defined by (16).Moreover, if V ∈ C 1,3 ([0, T] × R n ) and V tx is also continuous, then By martingale representation theorem (see Lemma 2.3, [16]) and Itô's formula (see [10]), the proof technique is quite similar to Theorem 4.1, Chapter 4 of [18].So we omit the detail.In fact, the relationship in (67) also can be seen in Theorem 2.1 of [7].See also (3) in Introduction of this chapter.Remark 3.5 (i) On the assumption that the value function V is smooth, the first equality in (66) show us the relationship between the derivative of V with respective to the time variable and the generalized Hamiltonian function G defined by ( 16) along an optimal state trajectory.
(ii) It is interesting to note that the second equality in (66) may be regard as a "maximum principle" in terms of the value function and its derivatives.It is different from the stochastic MP aforementioned (Theorem 2.3), where no value function or its derivatives is involved.
(iii) The three equalities in (67) show us the relationship between the derivative of V with respective to the state variable and the adjoint processes p, q, γ(•).More precisely, the three adjoint processes p, q, γ(•) can be expressed in terms of the derivatives of V with respective to the state variable along an optimal state trajectory.It is also interesting to note that from the third equality in (67), we observe that the jump amplitude of V x (t, xs,y; ū(t)) equal to −γ(t, •) which is just that V x (t, xs,y; ū (t)) = −p(t) tell us by the first-order adjoint equation (18).
Remark 3.6 By Remark 3.2 and Example 3.1, it can be seen that though the first classical relation in (67) of Corollary 3.1 is recovered from Theorem 3.2 when the value function V is smooth enough, the nonsmooth version of the second classical relation in (67), i.e., q(t)= P(t) σ(t), does not hold in general.We are also interested to the question that to what extent the third classical relation in (67) can be generalized when V is not smooth.However, it seems that Theorem 3.2 tells us nothing in this context while the following result gives the general relationship among p, q, γ(•), P, σ and c(•).

Lemma 3 . 1
Let z : [a, b] → Z be as in Definition 3.3.Then the set of right Lesbesgue points of z is of full measure in [a, b].
•, • and |•| denote the scalar product and norm in the Euclidean space, respectively.⊤ appearing in the superscripts denotes the transpose of a matrix.a ∨ b denotes max{a, b}.C always denotes some positive constant.