Spectral Properties of a Non-Self-Adjoint Differential Operator with Block-Triangular Operator Coefficients

Abstract

In this chapter, the Sturm-Liouville equation with block-triangular, increasing at infinity operator potential is considered. A fundamental system of solutions is constructed, one of which decreases at infinity, and the second increases. The asymptotic behavior at infinity was found out. The Green’s function and the resolvent for a non-self-adjoint differential operator are constructed. This allows to obtain sufficient conditions under which the spectrum of this non-self-adjoint differential operator is real and discrete. For a non-self-adjoint Sturm-Liouville operator with a triangular matrix potential growing at infinity, an example of operator having spectral singularities is constructed.

1. Introduction

The question of the generalization of the oscillatory Sturm theorem for scalar equations of higher orders and for equations with matrix coefficients for a long time remained open. Only in recent joint papers by F. Rofe-Beketov and A. Kholkin (see [1]) a connection was established between spectral and oscillatory properties for self-adjoint operators generated by equations of arbitrary even order with operator coefficients and boundary conditions of general form. Later, a Sturm-type oscillation theorem was proved [2] for a problem on finite and infinite intervals for a second-order equation with block-triangular matrix coefficients. In the case of non-self-adjoint differential operators, oscillation theorems have not been considered earlier.

Results turning out in self-adjoint and non-self-adjoint cases differentiate substantially. The theory of non-self-adjoint singular differential operators, generated by scalar differential expressions, has been well studied. An overview on the theory of non-self-adjoint singular ordinary differential operators is provided in V.E. Lyantse’s Appendix I to the monograph [3]. In the study of the connection between spectral and oscillation properties of non-self-adjoint differential operators with block-triangular operator coefficients [2, 4] the question arises of the structure of the spectrum of such operators. For scalar non- self-adjoint differential operators these questions were studied in the papers [5, 6, 7, 8]. The theory of singular non-self-adjoint differential operators with matrix and operator coefficients is relatively new. In the context of the inverse scattering problem, for an operator with a triangular matrix potential decreasing at infinity, the first moment of which is bounded, the structure of the spectrum was established in [9, 10]. The theory of equations with block - triangular operator coefficients the first results were published in 2012 in the works of the author [11, 12, 13].

In this works we construct the fundamental system of solutions of differential equation with block-triangular operator potential that increases at infinity, one of that is decreasing at infinity, and the second growing. The asymptotics of the fundamental system of solutions of this equation is established. The Green’s function is constructed for a non-self-adjoint system with a block-triangular potential, the diagonal blocks of which are self-adjoint operators. We obtained a resolvent for a non-self-adjoint differential operator, using which the structure of the operator spectrum is set. Sufficient conditions at which a spectrum of such non-self-adjoint differential operator is real and discrete are obtained. Here the rate of growth elements, not on the main diagonal, is subordinated to the rate of growth of the diagonal elements. In case of infringement of this condition, the operator can have spectral singularities [14].

2. The fundamental solutions for an non-self-adjoint differential operator with block – triangular operator coefficients.

Let us designate $H_k,k=\overline{1,r}$ as a finite-dimensional or infinite-dimensional separable Hilbert space with inner product $\left(\cdot ,\cdot \right)$ and norm $\left|\cdot \right|$ . Denote by $\mathbf{H}\mathbf{=}{H}_1\oplus H_2\oplus \dots \oplus H_r.$ Element $\overline{h}\in \mathbf{H}$ will be written in the form of $\overline{h}=\mathrm{col}\left({\overline{h}}_1,{\overline{h}}_2,\dots ,{\overline{h}}_r\right)$ , where ${\overline{h}}_k\in H_k$ , $k=1,2,\dots ,r$ , $I_k,I$ - are identity operators in $H_k$ and $\mathbf{H}$ accordingly.

We denote by $L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ the Hilbert space of vector-valued functions $y\left(x\right)$ with values in $\mathbf{H}$ with inner product $⟨y,z⟩={\int }_0^{\infty }\left(y\left(x\right),z\left(x\right)\right)\mathit{dx}$ and the norm $‖\cdot ‖$ .

Now let us consider the equation with block-triangular operator potential in $B\left(\mathbf{H}\right)$

$$l\left[\overline{y}\right]=-{\overline{y}}^{\prime \prime }+V\left(x\right)\overline{y}=\lambda \overline{y},0\le x<\infty ,$$ E1

where

$$V\left(x\right)=v\left(x\right)\cdot I+U\left(x\right),U\left(x\right)=\left(\begin{array}{cccc}{U}_{11}\left(x\right)& U_{12}\left(x\right)& \dots & U_{1r}\left(x\right)\\ 0& U_{22}\left(x\right)& \dots & U_{2r}\left(x\right)\\ \dots & \dots & \dots & \dots \\ 0& 0& \dots & U_{\mathit{rr}}\left(x\right)\end{array}\right),$$ E2

$v\left(x\right)$ is a real scalar function such that $0<v\left(x\right)\to \infty$ monotonically, as $x\to \infty$ , and it has monotone absolutely continuous derivative. Also, $U\left(x\right)$ is a relatively small perturbation, e. g. as $x\to \infty$ $\left|U\left(x\right)\right|\cdot v^{-1}\left(x\right)\to 0$ or $\left|U\right|v^{-1}\in L^{\infty }\left({\mathbb{R}}_{+}\right)$ . The diagonal blocks $U_{\mathit{kk}}\left(x\right), k=\overline{1,r}$ are assumed to be bounded self-adjoint operators in $H_k$ .

In case where

$$v\left(x\right)\ge {\mathit{Cx}}^{2\alpha },C>0,\alpha >1,$$ E3

we suppose that coefficients of the Eq. (1) satisfy relations:

$$\underset{0}{\overset{\infty }{\int }}\left|U\left(t\right)\right|\cdot v^{-\frac{1}{2}}\left(t\right)\mathit{dt}<\infty ,$$ E4

$$\underset{0}{\overset{\infty }{\int }}{v^{\prime }}^2\left(t\right)\cdot v^{-\frac{5}{2}}\left(t\right)\mathit{dt}<\infty ,\underset{0}{\overset{\infty }{\int }}{v}^{″}\left(t\right)\cdot v^{-\frac{3}{2}}\left(t\right)\mathit{dt}<\infty .$$ E5

In case of $v\left(x\right)=x^{2\alpha },0<\alpha \le 1$ , we suppose that the coefficients of the Eq. (1) satisfy the relation

$$\underset{a}{\overset{\infty }{\int }}\left|U\left(t\right)\right|\cdot t^{-\alpha }\mathit{dt}<\infty ,a>0.$$ E6

2.1 Construction of the fundamental system of solutions for an operator differential equation with a rapidly increasing at infinity potential

Consider first the case where $v\left(x\right)\ge {\mathit{Cx}}^{2\alpha },C>0,\alpha >1$ .

Condition (3) is performed, for example, quickly increasing functions $e^x,exp\left\{e^x\right\}$ etc.

Rewrite the Eq. (1) in the form

$$-{\overline{y}}^{\prime \prime }+\left(v\left(x\right)+q\left(x\right)\right)\overline{y}=\left(\left(\lambda +q\left(x\right)\right)I-U\left(x\right)\right)\overline{y},$$ E7

where $q\left(x\right)$ determined by a formula (cf. with the monograph [15])

$$q\left(x\right)=\frac{5}{16}{\left(\frac{v^{\prime }\left(x\right)}{v\left(x\right)}\right)}^2-\frac{1}{4}\frac{v^{″}\left(x\right)}{v\left(x\right)}.$$ E8

Now let us denote.

$${\gamma }_0\left(x,\lambda \right)=\frac{1}{\sqrt[4]{4v\left(x\right)}}\cdot exp\left(-\underset{0}{\overset{x}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right),{\gamma }_{\infty }\left(x,\lambda \right)=\frac{1}{\sqrt[4]{4v\left(x\right)}}\cdot exp\left(\underset{0}{\overset{x}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right).$$ E9

It is easy to see that ${\gamma }_0\left(x,\lambda \right)\to 0,{\gamma }_{\infty }\left(x,\lambda \right)\to \infty$ as $x\to \infty$ . These solutions constitute a fundamental system of solutions of the scalar differential equation

$$-z^{″}+\left(v\left(x\right)+q\left(x\right)\right)z=0,$$ E10

in such a way that for all $x\in \left[0,\infty \right)$ one has.

$$W\left({\gamma }_0,{\gamma }_{\infty }\right)≔{\gamma }_0\left(x,\lambda \right)\cdot {\gamma }_{\infty }^{\prime }\left(x,\lambda \right)-{\gamma }_0^{\prime }\left(x,\lambda \right)\cdot {\gamma }_{\infty }\left(x,\lambda \right)=1.$$ E11

Theorem 2.1 Under conditions (3) , (4) , (5) Eq. (1) has a unique decreasing at infinity operator solution $\mathrm{\Phi }\left(x,\lambda \right)\in B\left(\mathbf{H}\right)$ , satisfying the conditions

$$\underset{x\to \infty }{lim}\frac{\mathrm{\Phi }\left(x,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}=I\mathit{\text{and}}\underset{x\to \infty }{lim}\frac{{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right)}{{\gamma }_0^{\prime }\left(x,\lambda \right)}=I.$$ E12

Also, there exists increasing at infinity operator solution $\mathrm{\Psi }\left(x,\lambda \right)\in B\left(\mathbf{H}\right)$ satisfying the conditions

$$\underset{x\to \infty }{lim}\frac{\mathrm{\Psi }\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x,\lambda \right)}=I\mathit{\text{and}}\underset{x\to \infty }{lim}\frac{{\mathrm{\Psi }}^{\prime }\left(x,\lambda \right)}{{\gamma }_{\infty }^{\prime }\left(x,\lambda \right)}=I.$$ E13

Proof a. Eq. (7) equivalently to integral equation

$$\mathrm{\Phi }\left(x,\lambda \right)={\gamma }_0\left(x,\lambda \right)I+\underset{x}{\overset{\infty }{\int }}K\left(x,t,\lambda \right)\cdot \mathrm{\Phi }\left(t,\lambda \right)\mathit{dt},$$ E14

where

$$K\left(x,t,\lambda \right)=C\left(x,t,\lambda \right)\cdot \left[\left(\lambda +q\left(t\right)\right)I-U\left(t\right)\right],$$ E15

$$C\left(x,t,\lambda \right)={\gamma }_{\infty }\left(x,\lambda \right)\cdot {\gamma }_0\left(t,\lambda \right)-{\gamma }_{\infty }\left(t,\lambda \right)\cdot {\gamma }_0\left(x,\lambda \right),$$ E16

with $C\left(x,t,\lambda \right)$ being the Cauchy function that in each variable satisfies Eq. (10) and the initial conditions $C\left(x,t,\lambda \right)\left|{}_{x=t}\right.=0,C_x^{\prime }\left(x,t,\lambda \right)\left|{}_{x=t}=1,\right.C_t^{\prime }\left(x,t,\lambda \right)\left|{}_{x=t}=-1\right.$ . Set $\chi \left(x,\lambda \right)=\frac{\mathrm{\Phi }\left(x,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}$ to rewrite Eq. (14) in form

$$\chi \left(x,\lambda \right)=I+\underset{x}{\overset{\infty }{\int }}R\left(x,t,\lambda \right)\chi \left(t,\lambda \right)\mathit{dt},$$ E17

where $R\left(x,t,\lambda \right)=K\left(x,t,\lambda \right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}$ . Thus

$$\begin{array}{ll}\left|C\left(x,t\right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\right|& =\left|{\gamma }_0^2\left(t,\gamma \right)\cdot \frac{{\gamma }_{\infty }\left(x,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}-{\gamma }_0\left(t,\lambda \right)\cdot {\gamma }_{\infty }\left(t,\lambda \right)\right|=\\ & =\left|\frac{1}{2\sqrt{v\left(t\right)}}\cdot exp\left(-2\underset{0}{\overset{t}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right)\cdot exp\left(2\underset{0}{\overset{x}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right)-\frac{1}{2\sqrt{v\left(t\right)}}\right|=\\ & =\frac{1}{2\sqrt{v\left(t\right)}}\cdot \left|exp\left(-2\underset{x}{\overset{t}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right)-1\right|\end{array}$$ E18

and since with $x\le t$ one has $exp\left(-2\underset{x}{\overset{t}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right)\le 1$ , we deduce that

$$\left|C\left(x,t\right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\right|\le \frac{1}{\sqrt{v\left(t\right)}}.$$ E19

Hence.

$$\left|R\left(x,t,\lambda \right)\right|=\left|C\left(x,t\right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\gamma \right)}\cdot \left[\left(\lambda +q\left(t\right)\right)I-U\left(t\right)\right]\right|\le \frac{1}{\sqrt{v\left(t\right)}}\left(\left|\lambda \right|+\left|q\left(t\right)\right|+\left|U\left(t\right)\right|\right).$$ E20

By virtue of (3)–(5), (8),

$$\frac{1}{\sqrt{v\left(t\right)}}\left(\left|\lambda \right|+\left|q\left(t\right)\right|+\left|U\left(t\right)\right|\right)\in L\left(0,\infty \right),$$ E21

and therefore integral equation has a unique solution $\chi \left(x,\lambda \right)$ and $\left|\chi \left(x,\lambda \right)\right|\le \text{const}$ . By (17), one has that ${lim}_{x\to \infty }\chi \left(x,\lambda \right)=I$ , where the first part of formula (12) follows from.

Differentiable (14) to get $\frac{{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right)}{{\gamma }_0^{\prime }\left(x\right)}=I+{\int }_x^{\infty }S\left(x,t,\lambda \right)\chi \left(t,\lambda \right)\mathit{dt}$ , where $S\left(x,t,\lambda \right)=K_x^{\prime }\left(x,t,\lambda \right)\frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0^{\prime }\left(x,\lambda \right)}=C_x^{\prime }\left(x,t\right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0^{\prime }\left(x,\lambda \right)}\cdot \left[\left(\lambda +q\left(t\right)\right)I-U\left(t\right)\right]$ . We have similarly (18), that $\left|C_x^{\prime }\left(x,t\right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0^{\prime }\left(x,\lambda \right)}\right|\le \frac{1}{\sqrt{v\left(t\right)}}$ , and therefore $\left|S\left(x,t,\lambda \right)\right|\le \frac{1}{\sqrt{v\left(t\right)}}\cdot \left[\left|\lambda \right|+\left|q\left(t\right)\right|+\left|U\left(t\right)\right|\right]\in L\left(0,\infty \right)$ , where the second part of formula (12) follows from.

b. Denote by $\widehat{\mathrm{\Psi }}\left(x,\lambda \right)\in B\left(\mathbf{H}\right)$ block-triangular operator solution of Eq. (1) that increases at infinity, ${\mathrm{\Psi }}_{\mathit{kk}}\left(x,\lambda \right)\in B\left(H_k,H_k\right),k=\overline{1,r}$ -its diagonal blocks. Now Eq. (7) is equivalent to the integral equation

$$\widehat{\mathrm{\Psi }}\left(x,\lambda \right)={\gamma }_{\infty }\left(x,\lambda \right)\cdot I-\underset{0}{\overset{x}{\int }}K\left(x,t,\lambda \right)\cdot \widehat{\mathrm{\Psi }}\left(t,\lambda \right)\mathit{dt},$$ E22

where, just as in (14), the kernel $K\left(x,t,\lambda \right)$ is given by (15). Now set $\chi \left(x,\lambda \right)=\frac{\widehat{\mathrm{\Psi }}\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x,\lambda \right)}$ to rewrite Eq. (22) in form

$$\chi \left(x,\lambda \right)=I-\underset{0}{\overset{x}{\int }}R\left(x,t,\lambda \right)\cdot \chi \left(t,\lambda \right)\mathit{dt},$$ E23

where $R\left(x,t,\lambda \right)=C\left(x,t,\lambda \right)\cdot \frac{{\gamma }_{\infty }\left(t,\lambda \right)}{{\gamma }_{\infty }\left(x,\lambda \right)}\cdot \left[\left(q\left(t\right)+\lambda \right)\cdot I-U\left(t\right)\right]$ . Similarly we can prove that the integral Eq. (23) has a unique solution $\chi \left(x,\lambda \right)$ and $\left|\chi \left(x,\lambda \right)\right|\le \text{const}$ . Pass in (23) to a limit as $x\to \infty$ to get ${lim}_{x\to \infty }\chi \left(x,\lambda \right)=I+\tilde{C}\left(\lambda \right)$ where $\tilde{C}\left(\lambda \right)$ is block-triangular operator in $\mathbf{H}$ , that is

$$\underset{x\to \infty }{lim}\frac{\widehat{\mathrm{\Psi }}\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x,\lambda \right)}=I+\tilde{C}\left(\lambda \right).$$ E24

Now consider another block-triangular operator solution $\tilde{\mathrm{\Psi }}\left(x,\lambda \right)$ that increases at infinity diagonal blocks which are defined by.

$${\tilde{\mathrm{\Psi }}}_{\mathit{kk}}\left(x,\lambda \right)={\mathrm{\Phi }}_{\mathit{kk}}\left(x,\lambda \right)\underset{a}{\overset{x}{\int }}{\mathrm{\Phi }}_{\mathit{kk}}^{-1}\left(t,\lambda \right){\left({\mathrm{\Phi }}_{\mathit{kk}}^{\ast }\left(t,\lambda \right)\right)}^{-1}\mathit{dt},k=\overline{1,r},\left(a\ge 0\right),$$ E25

${\mathrm{\Phi }}_{\mathit{kk}}\left(x,\lambda \right)$ are the diagonal blocks of operator solution $\mathrm{\Phi }\left(x,\lambda \right)$ as in Section a). In view (16) and the definition of the functions ${\gamma }_0\left(x\right),{\gamma }_{\infty }\left(x\right)$ can be proved that

$$\underset{x\to \infty }{lim}\frac{{\tilde{\mathrm{\Psi }}}_{\mathit{kk}}\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x,\lambda \right)}=I_k,k=\overline{1,r}.$$ E26

Since $\widehat{\mathrm{\Psi }}\left(x,\lambda \right)$ and $\tilde{\mathrm{\Psi }}\left(x,\lambda \right)$ are the operator solutions of Eq. (1) that increase at infinity,

$$\widehat{\mathrm{\Psi }}\left(x,\lambda \right)=\tilde{\mathrm{\Psi }}\left(x,\lambda \right)+\mathrm{\Phi }\left(x,\lambda \right)\cdot C_0\left(\lambda \right),$$ E27

where $C_0\left(\lambda \right)$ is some block-triangular operator. Thus $\underset{x\to \infty }{lim}\frac{\widehat{\mathrm{\Psi }}\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x\right)}=\underset{x\to \infty }{lim}\frac{\tilde{\mathrm{\Psi }}\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x\right)}$ , hence, by virtue (26), $\underset{x\to \infty }{lim}\frac{{\mathrm{\Psi }}_{\mathit{kk}}\left(x,\lambda \right)}{{\gamma }_{\infty }\left(x\right)}=I_k,k=\overline{1,r}$ and in (24) has

$$\tilde{C}\left(\lambda \right)=\left(\begin{array}{cccc}0& C_{12}\left(\lambda \right)& \dots & C_{1r}\left(\lambda \right)\\ 0& 0& \dots & C_{2r}\left(\lambda \right)\\ \dots & \dots & \dots & \dots \\ 0& 0& \dots & 0\end{array}\right).$$ E28

The solution $\mathrm{\Psi }\left(x,\lambda \right)$ given by $\mathrm{\Psi }\left(x,\lambda \right)=\widehat{\mathrm{\Psi }}\left(x,\lambda \right){\left(I+\tilde{C}\left(\lambda \right)\right)}^{-1}$ is subject to first from condition (13). Use (12) to differentiate (27), then find the asymptotes of $\widetilde{{\mathrm{\Psi }}^{\prime }}\left(x,\lambda \right)$ as $x\to \infty$ similarly to (21) to obtain the second part of formula (13). Theorem is proved. $\square$

In this section, the fundamental system of solution is constructed for an operator differential equation with a rapidly increasing at infinity potential.

2.2 Asymptotic of the fundamental system solutions of equation with block-triangular potential

Now consider the case when $v\left(x\right)=x^{2\alpha },0<\alpha \le 1$ and coefficients of Eq. (1) satisfy the condition (6). Rewrite Eq. (1) in the form

$$-{\overline{y}}^{\prime \prime }+\left(x^{2\alpha }-\lambda +q\left(x,\lambda \right)\right)\overline{y}=\left(q\left(x,\lambda \right)\cdot I-U\left(x\right)\right)\overline{y},$$ E29

where $q\left(x,\lambda \right)$ determined by a formula

$$q\left(x,\lambda \right)=\frac{5{\alpha }^2}{4}{\left(\frac{x^{2\alpha -1}}{x^{2\alpha }-\lambda }\right)}^2-\frac{\alpha \left(2\alpha -1\right)x^{2\alpha -2}}{2\left(x^{2\alpha }-\lambda \right)}.$$ E30

Denote

$${\gamma }_0\left(x,\lambda \right)=\frac{1}{\sqrt[4]{4\left(x^{2\alpha }-\lambda \right)}}\cdot exp\left(-\underset{a}{\overset{x}{\int }}\sqrt{u^{2\alpha }-\lambda }\mathit{du}\right),$$ E31

$${\gamma }_{\infty }\left(x,\lambda \right)=\frac{1}{\sqrt[4]{4\left(x^{2\alpha }-\lambda \right)}}\cdot exp\left(\underset{a}{\overset{x}{\int }}\sqrt{u^{2\alpha }-\lambda }\mathit{du}\right).$$ E32

There solutions constitute a fundamental system of solutions of the scalar differential equation $-z^{″}+\left(x^{2\alpha }-\lambda +q\left(x,\lambda \right)\right)z=0$ , in such a way that for all $x\in \left[0,\infty \right)$ one has $W\left({\gamma }_0,{\gamma }_{\infty }\right)≔{\gamma }_0\left(x,\lambda \right)\cdot {\gamma }_{\infty }^{\prime }\left(x,\lambda \right)-{\gamma }_0^{\prime }\left(x,\lambda \right)\cdot {\gamma }_{\infty }\left(x,\lambda \right)=1$ .

We are about to establish the asymptotics¹ of ${\gamma }_0\left(x,\lambda \right)$ as $x\to \infty$ :

$${\gamma }_0\left(x,\lambda \right)={\left(2x^{\alpha }\right)}^{-\frac{1}{2}}\cdot {\left(1-\frac{\lambda }{x^{2\alpha }}\right)}^{-\frac{1}{4}}\cdot exp\left(-\underset{a}{\overset{x}{\int }}{u}^{\alpha }{\left(1-\frac{\lambda }{u^{2\alpha }}\right)}^{\frac{1}{2}}\mathit{du}\right).$$ E33

After expanding here the integral, we obtain the exponential as follows

$$exp\left(-\underset{a}{\overset{x}{\int }}{u}^{\alpha }\cdot \left(1-\frac{1}{2}\cdot \frac{\lambda }{u^{2\alpha }}-\sum _{k=2}^{\infty }\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!\cdot 2^k}\cdot {\left(\frac{\lambda }{u^{2\alpha }}\right)}^k\right)\mathit{du}\right).$$ E34

In case $\frac{\alpha +1}{2\alpha }=n\in N$ , i.e. $\alpha =\frac{1}{2n-1}$ , this expression after integration acquires the form:

$$c\cdot exp\left(-\frac{x^{1+\alpha }}{1+\alpha }+\frac{\lambda }{2}\cdot \frac{x^{1-\alpha }}{1-\alpha }+\sum _{k=2}^{n-1}\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!}\cdot {\left(\frac{\lambda }{2}\right)}^k\cdot \frac{x^{1-\left(2k-1\right)\alpha }}{1-\left(2k-1\right)\alpha }\right)$$

$$\cdot exp\left(\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n\cdot lnx+o\left(1\right)\right)=$$

$$=c\cdot exp\left(-\frac{x^{1+\alpha }}{1+\alpha }+\frac{\lambda }{2}\cdot \frac{x^{1-\alpha }}{1-\alpha }+\sum _{k=2}^{n-1}\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!}\cdot {\left(\frac{\lambda }{2}\right)}^k\cdot \frac{x^{1-\left(2k-1\right)\alpha }}{1-\left(2k-1\right)\alpha }\right)\cdot$$

$$\cdot x^{\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n}\cdot \left(1+o\left(1\right)\right).$$ E35

The asymptotics of ${\gamma }_0\left(x,\lambda \right)$ as $x\to \infty$ is as follows:

$$\begin{array}{c}{\gamma }_0\left(x,\lambda \right)=c\cdot exp\left(-\frac{x^{1+\alpha }}{1+\alpha }+\frac{\lambda }{2}\cdot \frac{x^{1-\alpha }}{1-\alpha }+\sum _{k=2}^{n-1}\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!}\cdot {\left(\frac{\lambda }{2}\right)}^k\cdot \frac{x^{1-\left(2k-1\right)\alpha }}{1-\left(2k-1\right)\alpha }\right)\\ \cdot x^{\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n-\frac{\alpha }{2}}\cdot \left(1+o\left(1\right)\right).\end{array}$$ E36

In particular, for $\alpha =1\left(n=1\right)$ , ${\gamma }_0\left(x,\lambda \right)$ has the following asymptotics at infinity:

$${\gamma }_0\left(x,\lambda \right)=c\cdot x^{\frac{\lambda -1}{2}}\cdot exp\left(-\frac{x^2}{2}\right)\left(1+o\left(1\right)\right).$$ E37

In case $\frac{\alpha +1}{2\alpha }\notin N$ we set $n=\left[\frac{\alpha +1}{2\alpha }\right]+1$ , with $\left[\beta \right]$ being the integral part of $\beta ,$ to obtain the following asymptotics for ${\gamma }_0\left(x,\lambda \right)$ at infinity:

$$\begin{array}{c}{\gamma }_0\left(x,\lambda \right)=c\cdot x^{-\frac{\alpha }{2}}exp\left(-\frac{x^{1+\alpha }}{1+\alpha }+\frac{\lambda }{2}\cdot \frac{x^{1-\alpha }}{1-\alpha }+\sum _{k=2}^{n-1}\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!}\cdot {\left(\frac{\lambda }{2}\right)}^k\cdot \frac{x^{1-\left(2k-1\right)\alpha }}{1-\left(2k-1\right)\alpha }\right)\cdot \\ \cdot exp\left(-\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n\cdot \frac{x^{-\alpha }}{\alpha }\right)\cdot \left(1+o\left(x^{-\alpha }\right)\right)\end{array}$$ E38

In particular, with $\alpha =\frac{1}{2}\left(n=2\right)$ one has

$${\gamma }_0\left(x,\lambda \right)={\mathit{cx}}^{-\frac{1}{4}}\cdot exp\left(-\frac{2}{3}{x}^{\frac{3}{2}}+\lambda x^{\frac{1}{2}}-{\left(\frac{\lambda }{2}\right)}^2{x}^{-\frac{1}{2}}\right)\cdot \left(1+o\left(x^{-\frac{1}{2}}\right)\right).$$ E39

A similar procedure allows to establish the asymptotics of ${\gamma }_{\infty }\left(x\right)$ as $x\to \infty .$ If $\frac{\alpha +1}{2\alpha }=n\in N$ , i.e. $\alpha =\frac{1}{2n-1}$ , then

$$\begin{array}{c}{\gamma }_{\infty }\left(x,\lambda \right)=c\cdot exp\left(\frac{x^{1+\alpha }}{1+\alpha }-\frac{\lambda }{2}\cdot \frac{x^{1-\alpha }}{1-\alpha }-\sum _{k=2}^{n-1}\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!}\cdot {\left(\frac{\lambda }{2}\right)}^k\cdot \frac{x^{1-\left(2k-1\right)\alpha }}{1-\left(2k-1\right)\alpha }\right)\cdot \\ \cdot x^{-\left(\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n+\frac{\alpha }{2}\right)}\cdot \left(1+o\left(1\right)\right).\end{array}$$ E40

With $\alpha =1\left(n=1\right)$ , this becomes

$${\gamma }_{\infty }\left(x,\lambda \right)=c\cdot x^{-\frac{\lambda +1}{2}}\cdot exp\left(\frac{x^2}{2}\right)\left(1+o\left(1\right)\right).$$ E41

In case $\frac{\alpha +1}{2\alpha }\notin N$ , we set $n=\left[\frac{\alpha +1}{2\alpha }\right]+1$ to get the asymptotics.

$$\begin{array}{c}{\gamma }_{\infty }\left(x,\lambda \right)=c\cdot x^{-\frac{\alpha }{2}}exp\left(-\frac{x^{1+\alpha }}{1+\alpha }+\frac{\lambda }{2}\cdot \frac{x^{1-\alpha }}{1-\alpha }+\sum _{k=2}^{n-1}\frac{1\cdot 3\cdot \dots \cdot \left(2k-3\right)}{k!}\cdot {\left(\frac{\lambda }{2}\right)}^k\cdot \frac{x^{1-\left(2k-1\right)\alpha }}{1-\left(2k-1\right)\alpha }\right)\cdot \\ \cdot exp\left(\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n\cdot \frac{x^{-\alpha }}{\alpha }\right)\cdot \left(1+o\left(x^{-\alpha }\right)\right).\end{array}$$ E42

In case $\alpha =\frac{1}{2}\left(n=2\right)$ , one has

$${\gamma }_{\infty }\left(x,\lambda \right)={\mathit{cx}}^{-\frac{1}{4}}\cdot exp\left(\frac{2}{3}{x}^{\frac{3}{2}}-\lambda x^{\frac{1}{2}}+{\left(\frac{\lambda }{2}\right)}^2{x}^{-\frac{1}{2}}\right)\cdot \left(1+o\left(x^{-\frac{1}{2}}\right)\right).$$ E43

Theorem 2.2 Under $0<\alpha \le 1$ and condition (6) , the statement of Theorem 2.1 is also valid for Eq. (1) .

Proof is similar to Theorem 2.1. Moreover, note that

$$\left|C\left(x,t,\lambda \right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\right|=\left|{\gamma }_0^2\left(t,\lambda \right)\cdot \frac{{\gamma }_{\infty }\left(x,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}-{\gamma }_0\left(t,\lambda \right)\cdot {\gamma }_{\infty }\left(t,\lambda \right)\right|=$$

$$\begin{array}{c}=\left|\frac{1}{2\sqrt{t^{2\alpha }-\lambda }}\cdot exp\left(-2\underset{a}{\overset{t}{\int }}\sqrt{u^{2\alpha }-\lambda }\mathit{du}\right)\cdot exp\left(2\underset{a}{\overset{x}{\int }}\sqrt{u^{2\alpha }-\lambda }\mathit{du}\right)-\frac{1}{2\sqrt{t^{2\alpha }-\lambda }}\right|\\ =\frac{1}{2\sqrt{t^{2\alpha }-\lambda }}\cdot \left|exp\left(-2\underset{x}{\overset{t}{\int }}\sqrt{u^{2\alpha }-\lambda }\mathit{du}-1\right)\right|.\end{array}$$ E44

As $x\le t$ , one has $exp\left(-2\underset{x}{\overset{t}{\int }}\sqrt{u^{2\alpha }-\lambda }\mathit{du}\right)\le 1$ , and that is why

$$\left|C\left(x,t,\lambda \right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\right|\le \frac{1}{\sqrt{t^{2\alpha }-\lambda }}.$$ E45

Hence

$$\left|R\left(x,t,\lambda \right)\right|=\left|C\left(x,t,\lambda \right)\cdot \frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\cdot \left[q\left(t,\lambda \right)\cdot I-U\left(t\right)\right]\right|\le \frac{1}{\sqrt{t^{2\alpha }-\lambda }}\left(\left|q\left(t,\lambda \right)\right|+\left|U\left(t\right)\right|\right).$$ E46

By virtue of (6) and (30), $\frac{1}{\sqrt{t^{2\alpha }-\lambda }}\left(\left|q\left(t,\lambda \right)\right|+\left|U\left(t\right)\right|\right)\in L\left(a,\infty \right)$ and therefore integral equation has a unique solution $\chi \left(x,\lambda \right)$ and $\left|\chi \left(x,\lambda \right)\right|\le \text{const}$ . By (17), one has that ${lim}_{x\to \infty }\chi \left(x,\lambda \right)=I$ , where the first part of formula (12) follows from.

The remaining statements of Theorem 2.1 are proved similarly. $\square$

From Theorem 2.2 and the asymptotic formulas (37), (39), (41), (43) follows.

Corollary 2.1 If $\alpha =1$ , i.e. $v\left(x\right)=x^2$ , then, under condition (6) , the solutions $\mathrm{\Phi }\left(x,\lambda \right)$ and $\mathrm{\Psi }\left(x,\lambda \right)$ have common (known) asymptotics, as in the quality ${\gamma }_0\left(x,\lambda \right)$ and ${\gamma }_{\infty }\left(x,\lambda \right)$ you can take the following functions.

$${\gamma }_0\left(x,\lambda \right)=x^{\frac{\lambda -1}{2}}\cdot exp\left(-\frac{x^2}{2}\right),..{\gamma }_{\infty }\left(x,\lambda \right)=x^{-\frac{\lambda +1}{2}}\cdot exp\left(\frac{x^2}{2}\right).$$ E47

If $\alpha =\frac{1}{2}$ , i.e. the coefficient $v\left(x\right)=x$ , and the condition (6) holds, then.

$${\gamma }_0\left(x,\lambda \right)=x^{-\frac{1}{4}}\cdot exp\left(-\frac{2}{3}{x}^{\frac{3}{2}}+\lambda x^{\frac{1}{2}}\right),{\gamma }_{\infty }\left(x,\lambda \right)=x^{-\frac{1}{4}}\cdot exp\left(\frac{2}{3}{x}^{\frac{3}{2}}-\lambda x^{\frac{1}{2}}\right).$$ E48

Remark 2.1 It is known that scalar equation

$$-{\phi }^{″}+x^2\cdot \phi =\mathit{\lambda \phi }$$ E49

for $\lambda =2n+1$ has the solution ${\phi }_n\left(x\right)=H_n\left(x\right)\cdot \mathit{exp}\left(-\frac{x^2}{2}\right)$ , where $H_n\left(x\right)$ is the Chebyshev – Hermitre polynomial, that at $x\to \infty$ has next asymptotics $H_n\left(x\right)={\left(2x\right)}^n\left(1+o\left(1\right)\right)$ . Hence the solution ${\phi }_n\left(x\right)$ of the Eq. (49) at $x\to \infty$ will have the following asymptotics at infinity: ${\phi }_n\left(x\right)={\left(2x\right)}^n\cdot \mathit{exp}\left(-\frac{x^2}{2}\right)\cdot \left(1+o\left(1\right)\right)$ .

In the case of $U\left(x\right)=0,v\left(x\right)=x^2$ in (2), the Eq. (1) is splitting into infinity system scalar equations of the form (49). The operator solution $\mathrm{\Phi }\left(x,\lambda \right)$ will be diagonal in this case. Denote by $\phi \left(x,\lambda \right)$ the diagonal elements of the operator $\mathrm{\Phi }\left(x,\lambda \right)$ . Then, by Corollary 2.1, the solution $\phi \left(x,\lambda \right)$ will have the following asymptotics at infinity: $\phi \left(x,\lambda \right)={\left(x\right)}^{\frac{\lambda -1}{2}}\cdot exp\left(-\frac{x^2}{2}\right)\left(1+o\left(1\right)\right)$ . In particular, for $\lambda =2n+1$ , this yields the solution proportional to ${\phi }_n\left(x\right)$ .

In this section, the asymptotics of the fundamental system of solutions for the Sturm-Liouville equation with block-triangular operator potential, increasing at infinity is established. One of the solutions is found decreasing at infinity, the other one increasing.

3. Green’s function for an operator differential equation with block – triangular coefficients

Let us suppose that at the $x=0$ given boundary conditions

$$cosA\cdot {\overline{y}}^{\prime }\left(0\right)-sinA\cdot \overline{y}\left(0\right)=0,$$ E50

where $A$ - the block-triangular operator of the same structure as the coefficients of the differential equation, $A_{\mathit{kk}}, k=\overline{1,r}$ - bounded self-adjoint operators in $H_k$ , which satisfy the conditions

$$-\frac{\pi }{2}{I}_k<<A_{\mathit{kk}}\le \frac{\pi }{2}{I}_k.$$ E51

Together with the problem (1), (50) we consider the separated system

$$l_k\left[y_k\right]=-{y_k}^{\prime \prime }+\left(v\left(x\right)I_k+U_{\mathit{kk}}\left(x\right)\right)y_k=\lambda y_k,k=\overline{1,r}$$ E52

with the boundary conditions

$$cos{A}_{\mathit{kk}}\cdot {y_k}^{\prime }\left(0\right)-sin{A}_{\mathit{kk}}\cdot y_k\left(0\right)=0,k=\overline{1,r}.$$ E53

Let $L^{\prime }$ denote the minimal differential operator generated by differential expression $l\left[\overline{y}\right]$ (1) and the boundary condition (50), and let ${L_k}^{\prime }$ , $k=\overline{1,r}$ denote the minimal differential operator on $L_2\left(H_k,\left(0,\infty \right)\right)$ generated by differential expression $l_k\left[y_k\right]$ and the boundary conditions (53). Taking into account the conditions on coefficients, as well as sufficient smallness of perturbations $U_{\mathit{kk}}\left(x\right)$ and conditions (51), we conclude that, for every symmetric operator ${L_k}^{\prime }$ , there is a case of limit point at infinity. Hence their self-adjoint extensions $L_k$ are the closures of operators ${L_k}^{\prime }$ respectively. The operators $L_k$ are semi-bounded below, and their spectra are discrete.

Let $L$ denote the operator extensions $L^{\prime }$ , by requiring that $L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ be the domain of operator $L$ .

The following theorem is proved in [4].

Theorem 3.1 Suppose that, for Eq. (1) conditions (3) - (5) are satisfied for $\alpha >1$ or condition (6) for $0<\alpha \le 1$ . Then the discrete spectrum of the operator $L$ is real and coincides with the union of spectra of the self-adjoint operators $L_k,k=\overline{1,r}$ , i.e., ${\sigma }_d\left(L\right)={\cup }_{k=1}^r\sigma \left(L_k\right)$ .

Comment 3.1 Note that this theorem contains a statement of the discrete spectrum of the non-self-adjoint operator $L$ only and no allegations of its continuous and residual spectrum.

Along with the Eq. (1) we consider the equation

$$l_1\left[\overline{y}\right]=-{\overline{y}}^{″}+V^{\ast }\left(x\right)\overline{y}=\lambda \overline{y}$$ E54

( $V^{\ast }\left(x\right)$ is adjoint to the operator $V\left(x\right)$ ). If the space $\mathbf{H}$ is finite-dimensional, then the Eq. (54) can be rewritten as

$$\tilde{l}\left[\tilde{y}\right]=-{\tilde{y}}^{\prime \prime }+\tilde{y}V\left(x\right)=\lambda \tilde{y},$$ E55

where $\tilde{y}=\left({\tilde{y}}_1{\tilde{y}}_2\dots {\tilde{y}}_r\right)$ and the equation is called the left.

For operator -functions $Y\left(x,\lambda \right),Z\left(x,\lambda \right)\in B\left(\mathbf{H}\right)$ let

$$W\left\{Z^{\ast },Y\right\}=Z^{\ast \prime }\left(x,\overline{\lambda }\right)Y\left(x,\lambda \right)-Z^{\ast }\left(x,\overline{\lambda }\right)Y^{\prime }\left(x,\lambda \right).$$ E56

If $Y\left(x,\lambda \right)$ - operator solution of the Eq. (1), and $Z\left(x,\lambda \right)$ - operator solution of Eq. (54), the Wronskian does not depend on $x$ .

Now we denote $Y\left(x,\lambda \right)$ and $Y_1\left(x,\lambda \right)$ the solutions of the Eqs. (1) and (54), respectively, satisfying the initial conditions

$$Y\left(0,\lambda \right)=cosA,Y^{\prime }\left(0,\lambda \right)=sinA,Y_1\left(0,\lambda \right)={\left(cosA\right)}^{\ast },{Y_1}^{\prime }\left(0,\lambda \right)={\left(sinA\right)}^{\ast },\lambda \in \mathbb{C}.$$ E57

Because the operator function $Y_1^{\ast }\left(x,\overline{\lambda }\right)$ satisfies equation

$$-{Y_1^{\ast }}^{\prime \prime }\left(x,\overline{\lambda }\right)+Y_1^{\ast }\left(x,\overline{\lambda }\right)\cdot V\left(x\right)=\lambda Y_1^{\ast }\left(x,\overline{\lambda }\right),$$ E58

the operator function $\tilde{Y}\left(x,\lambda \right)≕Y_1^{\ast }\left(x,\overline{\lambda }\right)$ is a solution to the left of the equation

$$-{\tilde{Y}}^{\prime \prime }\left(x,\lambda \right)+\tilde{Y}\left(x,\lambda \right)\cdot V\left(x\right)=\lambda \tilde{Y}\left(x,\lambda \right)$$ E59

and satisfies the initial conditions $\tilde{Y}\left(0,\lambda \right)=cosA, {\tilde{Y}}^{\prime }\left(0,\lambda \right)=sinA, \lambda \in \mathbb{C}$ .

Operator solutions of Eq. (54) decreasing and increasing at infinity will be denoted by ${\mathrm{\Phi }}_1\left(x,\lambda \right)$ , ${\mathrm{\Psi }}_1\left(x,\lambda \right)$ , and the corresponding solutions of the Eq. (59) denote by $\tilde{\mathrm{\Phi }}\left(x,\lambda \right)$ and $\tilde{\mathrm{\Psi }}\left(x,\lambda \right)$ . For the system operator solutions $Y\left(x,\lambda \right),\tilde{\mathrm{\Phi }}\left(x,\lambda \right)\in B\left(\mathbf{H}\right)$ of the Eqs. (1) and (59), respectively, will take the form of Wronskian $W\left\{\tilde{\mathrm{\Phi }},Y\right\}={\tilde{\mathrm{\Phi }}}^{\prime }\left(x,\lambda \right)Y\left(x,\lambda \right)-\tilde{\mathrm{\Phi }}\left(x,\lambda \right)Y^{\prime }\left(x,\lambda \right)$ and do not depend on $x$ .

Let us designate

$$G\left(x,t,\lambda \right)=\left\{\begin{array}{l}Y\left(x,\lambda \right){\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)0\le x\le t\\ -\mathrm{\Phi }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(t,\lambda \right)x\ge t\end{array}\right..$$ E60

In the following theorem it is proved that the operator function $G\left(x,t,\lambda \right)$ possesses all the classical properties of the Green’s function.

Theorem 3.2 The operator function $G\left(x,t,\lambda \right)$ is the Green’s function of the differential operator $L$ , i.e.:

1. The function $G\left(x,t,\lambda \right)$ is continuous for all values $x,t\in \left[0,\infty \right)$ ;

2. For any fixed $t$ , the function $G\left(x,t,\lambda \right)$ has a continuous derivative with respect to $x$ on each of the intervals $\left[0,t\right)$ and $\left(t,\infty \right)$ , and at $x=t$ it has the jump

$${G_x}^{\prime }\left(x+0,x,\lambda \right)-{G_x}^{\prime }\left(x-0,x,\lambda \right)=-I.$$ E61

1. For a fixed $t$ , the function $G\left(x,t,\lambda \right)$ of the variable $x$ is an operator solution of Eq. (1) on each of the intervals $\left[0,t\right)$ , $\left(t,\infty \right)$ , and it satisfies the boundary condition (50) , and at a fixed $x$ function $G\left(x,t,\lambda \right)$ of the variable $t$ is an operator solution of the Eq. (59) on each of the intervals $\left[0,x\right),\left(x,\infty \right)$ , and it satisfies the boundary condition ${\tilde{y}}^{\prime }\left(0\right)\cdot cosA-\tilde{y}\left(0\right)\cdot sinA=0.$

Proof The function $G\left(x,t,\lambda \right)$ is continuous with respect to $x$ at each of the intervals $\left[0,t\right)$ and $\left(t,\infty \right)$ . Similarly to the variable $t$ . To prove the continuity of the function $G\left(x,t,\lambda \right)$ for all $x,t\ge 0$ , it is sufficient that the identity shown as

$$Y\left(x,\lambda \right){\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\tilde{\mathrm{\Phi }}\left(x,\lambda \right)+\mathrm{\Phi }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(x,\lambda \right)\equiv 0.$$ E62

is satisfied for all $x\ge 0.$ This identity shown as

$$Y\left(x,\lambda \right){\left(\tilde{\mathrm{\Phi }}\left(x,\lambda \right)Y^{\prime }\left(x,\lambda \right)-{\tilde{\mathrm{\Phi }}}^{\prime }\left(x,\lambda \right)Y\left(x,\lambda \right)\right)}^{-1}\tilde{\mathrm{\Phi }}\left(x,\lambda \right)-$$

$$-\mathrm{\Phi }\left(x,\lambda \right){\left({\tilde{Y}}^{\prime }\left(x,\lambda \right)\mathrm{\Phi }\left(x,\lambda \right)-\tilde{Y}\left(x,\lambda \right){\mathrm{\Phi }}^{\prime }\left(x,\lambda \right)\right)}^{-1}\tilde{Y}\left(x,\lambda \right)\equiv 0$$ E63

or

$${\left(Y^{\prime }\left(x,\lambda \right)Y^{-1}\left(x,\lambda \right)-{\tilde{\mathrm{\Phi }}}^{-1}\left(x,\lambda \right){\tilde{\mathrm{\Phi }}}^{\prime }\left(x,\lambda \right)\right)}^{-1}\equiv {\left({\tilde{Y}}^{-1}\left(x,\lambda \right){\tilde{Y}}^{\prime }\left(x,\lambda \right)-{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right){\mathrm{\Phi }}^{-1}\left(x,\lambda \right)\right)}^{-1},$$

$$Y^{\prime }\left(x,\lambda \right)Y^{-1}\left(x,\lambda \right)-{\tilde{\mathrm{\Phi }}}^{-1}\left(x,\lambda \right){\tilde{\mathrm{\Phi }}}^{\prime }\left(x,\lambda \right)\equiv {\tilde{Y}}^{-1}\left(x,\lambda \right){\tilde{Y}}^{\prime }\left(x,\lambda \right)-{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right){\mathrm{\Phi }}^{-1}\left(x,\lambda \right),$$ E64

which is equivalent to

$$Y^{\prime }\left(x,\lambda \right)Y^{-1}\left(x,\lambda \right)-{\tilde{Y}}^{-1}\left(x,\lambda \right){\tilde{Y}}^{\prime }\left(x,\lambda \right)\equiv {\tilde{\mathrm{\Phi }}}^{-1}\left(x,\lambda \right){\tilde{\mathrm{\Phi }}}^{\prime }\left(x,\lambda \right)-{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right){\mathrm{\Phi }}^{-1}\left(x,\lambda \right)$$ E65

or to.

$$\begin{array}{c}{\tilde{Y}}^{-1}\left(x,\lambda \right)\left(\tilde{Y}\left(x,\lambda \right)Y^{\prime }\left(x,\lambda \right)-{\tilde{Y}}^{\prime }\left(x,\lambda \right)Y\left(x,\lambda \right)\right)Y^{-1}\left(x,\lambda \right)\\ \equiv \equiv -{\tilde{\mathrm{\Phi }}}^{-1}\left(x,\lambda \right)\left(\tilde{\mathrm{\Phi }}\left(x,\lambda \right){\mathrm{\Phi }}^{\prime }\left(x,\lambda \right)-{\tilde{\mathrm{\Phi }}}^{\prime }\left(x,\lambda \right){\mathrm{\Phi }}^{-1}\left(x,\lambda \right)\right){\mathrm{\Phi }}^{-1}\left(x,\lambda \right).\end{array}$$ E66

This follows from the fact that $W\left\{\tilde{Y},Y\right\}=W\left\{\tilde{\mathrm{\Phi }},\mathrm{\Phi }\right\}=0$ .

To make sure that the jump in the first derivative at $t=x$ is equal to $\left(-I\right)$ , i.e., that the equality (61) holds, it is sufficient to prove the identity

$$Y^{\prime }\left(x,\lambda \right){\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\tilde{\mathrm{\Phi }}\left(x,\lambda \right)+{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(x,\lambda \right)\equiv I.$$ E67

Now we consider the function

$$C\left(x,t,\lambda \right)=Y\left(x,\lambda \right){\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)+\mathrm{\Phi }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(t,\lambda \right),$$ E68

which is an analogue of the Cauchy function. This function is the solution of Eq. (1) of the variable $x,$ and it is the solution of Eq. (59) of the variable $t$ . By (62), we have $C\left(x,x,\lambda \right)\equiv 0$ . But in this case ${\left.{C_{\mathit{xx}}}^{\prime \prime }\right|}_{t=x}={\left.\left(V\left(x\right)-\mathit{\lambda I}\right)C\right|}_{t=x}\equiv 0$ , and, therefore, ${\left.{C_x}^{\prime }\left(x,t,\lambda \right)\right|}_{t=x}\equiv {\mathrm{\Omega }}_1\left(\lambda \right)$ , i.e.,

$$Y^{\prime }\left(x,\lambda \right){\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\tilde{\mathrm{\Phi }}\left(x,\lambda \right)+{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(x,\lambda \right)\equiv {\mathrm{\Omega }}_1\left(\lambda \right).$$ E69

It shows that ${\mathrm{\Omega }}_1\left(\lambda \right)=I,$ we obtain (61).

Since operator solutions $\mathrm{\Phi }\left(x,\lambda \right)$ and $\mathrm{\Psi }\left(x,\lambda \right)$ form a fundamental system of solutions of Eq. (1), the operator solution $Y\left(x,\lambda \right)$ of Eq. (1) satisfying the initial conditions (57), can be written as $Y\left(x,\lambda \right)=\mathrm{\Phi }\left(x,\lambda \right)A\left(\lambda \right)+\mathrm{\Psi }\left(x,\lambda \right)B\left(\lambda \right)$ , where $A\left(\lambda \right)=-W\left\{\tilde{\mathrm{\Psi }},Y\right\},B\left(\lambda \right)=W\left\{\tilde{\mathrm{\Phi }},Y\right\}$ ,

$$Y\left(x,\lambda \right)=\mathrm{\Psi }\left(x,\lambda \right)W\left\{\tilde{\mathrm{\Phi }},Y\right\}-\mathrm{\Phi }\left(x,\lambda \right)W\left\{\tilde{\mathrm{\Psi }},Y\right\}.$$ E70

Similarly, operator solution $\tilde{Y}\left(x,\lambda \right)$ of Eq. (59) can be represented in the form

$$\tilde{Y}\left(x,\lambda \right)=\tilde{W}\left\{\tilde{\mathrm{\Phi }},Y\right\}\tilde{\mathrm{\Psi }}\left(x,\lambda \right)-\tilde{W}\left\{\tilde{\mathrm{\Psi }},Y\right\}\tilde{\mathrm{\Phi }}\left(x,\lambda \right),$$ E71

where

$$\tilde{W}\left\{\tilde{\mathrm{\Phi }},Y\right\}=sinA\cdot \mathrm{\Phi }\left(0,\lambda \right)-cosA\cdot {\mathrm{\Phi }}^{\prime }\left(0,\lambda \right)=-\mathrm{\Omega }\left(0,\lambda \right)=-W\left\{\tilde{Y},\mathrm{\Phi }\right\}.$$ E72

Similarly we get $\tilde{W}\left\{\tilde{\mathrm{\Psi }},Y\right\}=-W\left\{\tilde{Y},\mathrm{\Psi }\right\}$ . Thus,

$$\tilde{Y}\left(x,\lambda \right)=W\left\{\tilde{Y},\mathrm{\Psi }\right\}\tilde{\mathrm{\Phi }}\left(x,\lambda \right)-W\left\{\tilde{Y},\mathrm{\Phi }\right\}\tilde{\mathrm{\Psi }}\left(x,\lambda \right).$$ E73

Substituting (70) and (73) into the formula (69), using the fact that the equality (69) is performed on $x$ identically, we obtain

$${\mathrm{\Omega }}_1\left(\lambda \right)=\underset{x\to \infty }{lim}\left[{\mathrm{\Psi }}^{\prime }\left(x,\lambda \right)\tilde{\mathrm{\Phi }}\left(x,\lambda \right)-{\mathrm{\Phi }}^{\prime }\left(x,\lambda \right)\tilde{\mathrm{\Psi }}\left(x,\lambda \right)\right].$$ E74

By Theorem 2.1, on the asymptotic behavior of functions $\mathrm{\Phi }\left(x,\lambda \right)$ and $\mathrm{\Psi }\left(x,\lambda \right)$ at infinity, we have

$${\mathrm{\Omega }}_1\left(\lambda \right)=\underset{x\to \infty }{lim}\left[{\gamma }_0\left(x,\lambda \right){{\gamma }_{\infty }}^{\prime }\left(x,\lambda \right)-{{\gamma }_0}^{\prime }\left(x,\lambda \right){\gamma }_{\infty }\left(x,\lambda \right)\right]\cdot I=W\left\{{\gamma }_0,{\gamma }_{\infty }\right\}\cdot I=I.$$ E75

This completes the proof of the formula (61), and with it the theorem 3.1. $\square$

Corollary . By the definition (60) , function $G\left(x,t,\lambda \right)$ is meromorphic of the parameter $\lambda$ with the poles coincide with the eigenvalues of the operator $L$ .

We constructed Green’s function for the non-self-adjoint differential operator.

4. Resolvent for an non-self-adjoint operator differential equation with block – triangular coefficients

We consider the operator $R_{\lambda }$ defined in $L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ by the relation

$$\left(R_{\lambda }\overline{f}\right)\left(x\right)=\underset{0}{\overset{\infty }{\int }}G\left(x,t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}=$$ E76

$$=-\underset{0}{\overset{x}{\int }}\mathrm{\Phi }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}+\underset{x}{\overset{\infty }{\int }}Y\left(x,\lambda \right){\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}.$$

Theorem 4.1 The operator $R_{\lambda }$ is the resolvent of the operator $L$ .

Proof One can directly verify that, for any function $\overline{f}\left(x\right)\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ , the vector-function $\overline{y}\left(x,\lambda \right)=\left(R_{\lambda }\overline{f}\right)\left(x\right)$ is a solution of the equation $l\left[\overline{y}\right]-\lambda \overline{y}=\overline{f}$ whenever $\lambda \notin \sigma \left(L\right)$ . We will prove that $\overline{y}\left(x,\lambda \right)\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ .

Since operator solutions $\mathrm{\Phi }\left(x,\lambda \right)$ and $\mathrm{\Psi }\left(x,\lambda \right)$ form a fundamental system of solutions of Eq. (1), the operator solution $Y\left(x,\lambda \right)$ of Eq. (1) satisfying the initial conditions (57), can be written as $Y\left(x,\lambda \right)=\mathrm{\Phi }\left(x,\lambda \right)A\left(\lambda \right)+\mathrm{\Psi }\left(x,\lambda \right)B\left(\lambda \right)$ , where $A\left(\lambda \right)=W\left\{\tilde{\mathrm{\Psi }},Y\right\},B\left(\lambda \right)=-W\left\{\tilde{\mathrm{\Phi }},Y\right\}$ ,

$$Y\left(x,\lambda \right)=\mathrm{\Phi }\left(x,\lambda \right)W\left\{\tilde{\mathrm{\Psi }},Y\right\}-\mathrm{\Psi }\left(x,\lambda \right)W\left\{\tilde{\mathrm{\Phi }},Y\right\}.$$ E77

Similarly, the operator solution $\tilde{Y}\left(x,\lambda \right)$ of Eq. (59) can be represented in the following form

$$\tilde{Y}\left(x,\lambda \right)=W\left\{\tilde{Y},\mathrm{\Phi }\right\}\tilde{\mathrm{\Psi }}\left(x,\lambda \right)-W\left\{\tilde{Y},\mathrm{\Psi }\right\}\tilde{\mathrm{\Phi }}\left(x,\lambda \right).$$ E78

By using formulas (77) and (78), we can rewrite the relation (76) as follows:

$$\begin{array}{c}\left(R_{\lambda }f\right)\left(x\right)=-\underset{0}{\overset{a}{\int }}\mathrm{\Phi }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}\tilde{Y}\left(t,\lambda \right)f\left(t\right)\mathit{dt}+{\overline{\chi }}_1\left(x,\lambda \right)-{\overline{\chi }}_2\left(x,\lambda \right)\\ +{\overline{\chi }}_3\left(x,\lambda \right)-{\overline{\chi }}_4\left(x,\lambda \right),\end{array}$$ E79

where $a>0$ and

$${\overline{\chi }}_1\left(x,\lambda \right)=\mathrm{\Phi }\left(x,\lambda \right){\left(W\left\{\tilde{Y},\mathrm{\Phi }\right\}\right)}^{-1}W\left\{\tilde{Y},\mathrm{\Psi }\right\}\underset{a}{\overset{x}{\int }}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt},$$ E80

$${\overline{\chi }}_2\left(x,\lambda \right)=\mathrm{\Phi }\left(x,\lambda \right)\underset{a}{\overset{x}{\int }}\tilde{\mathrm{\Psi }}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt},$$ E81

$${\overline{\chi }}_3\left(x,\lambda \right)=\mathrm{\Phi }\left(x,\lambda \right)W\left\{\tilde{\mathrm{\Psi }},Y\right\}{\left(W\left\{\tilde{\mathrm{\Phi }},Y\right\}\right)}^{-1}\underset{x}{\overset{\infty }{\int }}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt},$$ E82

$${\overline{\chi }}_4\left(x,\lambda \right)=\mathrm{\Psi }\left(x,\lambda \right)\underset{x}{\overset{\infty }{\int }}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}.$$ E83

Let us show that each of these vector-functions ${\overline{\chi }}_1\left(x,\lambda \right),{\overline{\chi }}_2\left(x,\lambda \right),{\overline{\chi }}_3\left(x,\lambda \right),{\overline{\chi }}_4\left(x,\lambda \right)$ belongs to $L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ . Since the operator solution $\mathrm{\Phi }\left(x,\lambda \right)$ decays fairly quickly as $x\to \infty$ , then $\left|\mathrm{\Phi }\left(x,\lambda \right)\right|\in L_2\left(0,\infty \right)$ . It follows that

$$\left|{\overline{\chi }}_1\left(x,\lambda \right)\right|\le c\left(\lambda \right)\cdot \left|\mathrm{\Phi }\left(x,\lambda \right)\right|\cdot \underset{a}{\overset{x}{\int }}\left|\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\right|\cdot \left|\overline{f}\left(t\right)\right|\mathit{dt}\le$$

$$\le c\left(\lambda \right)\cdot \left|\mathrm{\Phi }\left(x,\lambda \right)\right|\cdot {\left(\underset{a}{\overset{x}{\int }}\left|\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\right|\mathit{dt}\right)}^{\frac{1}{2}}\cdot {\left(\underset{a}{\overset{x}{\int }}\left|\overline{f}\left(t\right)\right|\mathit{dt}\right)}^{\frac{1}{2}}<$$

$$<c\left(\lambda \right)\cdot \left|\mathrm{\Phi }\left(x,\lambda \right)\right|\cdot {\left(\underset{a}{\overset{\infty }{\int }}\left|\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\right|\mathit{dt}\right)}^{\frac{1}{2}}\cdot {\left(\underset{a}{\overset{\infty }{\int }}\left|\overline{f}\left(t\right)\right|\mathit{dt}\right)}^{\frac{1}{2}}\le c_1\left(\lambda \right)\cdot \left|\mathrm{\Phi }\left(x,\lambda \right)\right|,$$ E84

and therefore ${\overline{\chi }}_1\left(x,\lambda \right)\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ . Similarly we get that ${\overline{\chi }}_3\left(x,\lambda \right)\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ . First we prove the assertion for the function ${\overline{\chi }}_2\left(x,\lambda \right)$ , when $\alpha >1$ and the coefficients of the Eq. (1) satisfy the conditions (3)-(5). In this case, we have $\left|{\overline{\chi }}_2\left(x,\lambda \right)\right|\le \left|\mathrm{\Phi }\left(x,\lambda \right)\right|{\int }_a^x\left|\tilde{\mathrm{\Psi }}\left(t,\lambda \right)\right|\left|\overline{f}\left(t\right)\right|\mathit{dt}$ .

By virtue of the asymptotic formulas for the operator solutions $\mathrm{\Phi }\left(x,\lambda \right)$ and $\mathrm{\Psi }\left(x,\lambda \right)$ we obtain that

$$\left|{\overline{\chi }}_2\left(x,\lambda \right)\right|\le c_1\left(\lambda \right){\gamma }_0\left(x,\lambda \right)\underset{a}{\overset{x}{\int }}{\gamma }_{\infty }\left(t,\lambda \right)\left|\overline{f}\left(t\right)\right|\mathit{dt}.$$ E85

Let us rewrite this relation in the following form

$$\left|{\overline{\chi }}_2\left(x,\lambda \right)\right|\le c_1\left(\lambda \right){\gamma }_0\left(x,\lambda \right){\gamma }_{\infty }\left(x,\lambda \right)\underset{a}{\overset{x}{\int }}\frac{{\gamma }_{\infty }\left(t,\lambda \right)}{{\gamma }_{\infty }\left(x,\lambda \right)}\left|\overline{f}\left(t\right)\right|\mathit{dt}.$$ E86

By using the definition of the functions ${\gamma }_0\left(x,\lambda \right)$ and ${\gamma }_{\infty }\left(x,\lambda \right)$ (see (9)) and by applying the Cauchy-Bunyakovskii inequality we obtain

$$\left|{\overline{\chi }}_2\left(x,\lambda \right)\right|\le \frac{1}{2}{c}_1\left(\lambda \right)\frac{1}{\sqrt{v\left(x\right)}}{\left(\underset{a}{\overset{x}{\int }}\sqrt{\frac{v\left(x\right)}{v\left(t\right)}}exp\left(-2\underset{t}{\overset{x}{\int }}\sqrt{v\left(u\right)}\mathit{du}\right)\mathit{dt}\right)}^{\frac{1}{2}}{\left(\underset{0}{\overset{\infty }{\int }}{\left|\overline{f}\left(t\right)\right|}^2\mathit{dt}\right)}^{\frac{1}{2}}.$$ E87

Since $t\le x$ , we get $exp\left(-2\underset{t}{\overset{x}{\int }}\sqrt{v\left(u\right)\mathit{du}}\right)\le 1$ , and then the latter estimate for ${\chi }_2\left(x,\lambda \right)$ can be rewritten as follows

$$\left|{\overline{\chi }}_2\left(x,\lambda \right)\right|\le c_2\left(\lambda \right)\frac{1}{\sqrt[4]{v\left(x\right)}}{\left(\underset{a}{\overset{x}{\int }}\frac{1}{\sqrt{v\left(t\right)}}\mathit{dt}\right)}^{\frac{1}{2}}\le c_2\left(\lambda \right)\frac{1}{\sqrt[4]{v\left(x\right)}}{\left(\underset{a}{\overset{\infty }{\int }}\frac{1}{\sqrt{v\left(t\right)}}\mathit{dt}\right)}^{\frac{1}{2}}.$$ E88

By formula (3), we get $\left|{\overline{\chi }}_2\left(x,\lambda \right)\right|\le \frac{c_3\left(\lambda \right)}{\sqrt[4]{v\left(x\right)}}$ , and hence, if $\alpha >1$ and the coefficients of the Eq. (1) satisfy the conditions (3)-(5), we have ${\overline{\chi }}_2\left(x,\lambda \right)\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ . In the case of $v\left(x\right)=x^{2\alpha },0<\alpha \le 1$ , the assertion can be proved similarly.

For the function ${\overline{\chi }}_4\left(x,\lambda \right)$ we will conduct the proof for the case when $v\left(x\right)=x^{2\alpha },0<\alpha \le 1$ and the coefficients of the Eq. (1) satisfy the condition (6). As in (85) we have $\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_1\left(\lambda \right){\gamma }_{\infty }\left(x,\lambda \right){\int }_x^{\infty }{\gamma }_0\left(t,\lambda \right)\left|\overline{f}\left(t\right)\right|\mathit{dt}$ , which can be rewritten as follows $\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_1\left(\lambda \right){\gamma }_0\left(x,\lambda \right){\gamma }_{\infty }\left(x,\lambda \right){\int }_x^{\infty }\frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\left|\overline{f}\left(t\right)\right|\mathit{dt}$ .

Let us use the asymptotics of the functions ${\gamma }_0\left(x,\lambda \right)$ and ${\gamma }_{\infty }\left(x,\lambda \right)$ , for example, in the case $\frac{\alpha +1}{2\alpha }=n\in N$ , i.e. $\alpha =\frac{1}{2n-1}$ (see (36) and (40)). Setting $a\left(\alpha ,\lambda \right)=\frac{1\cdot 3\cdot \dots \cdot \left(2n-3\right)}{n!}\cdot {\left(\frac{\lambda }{2}\right)}^n$ , we obtain

$$\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_2\left(\lambda \right)x^{-\alpha }\underset{x}{\overset{\infty }{\int }}\frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\left|\overline{f}\left(t\right)\right|\mathit{dt}\le c_2\left(\lambda \right)x^{-\alpha }{\left(\underset{a}{\overset{x}{\int }}{\left(\frac{{\gamma }_0\left(t,\lambda \right)}{{\gamma }_0\left(x,\lambda \right)}\right)}^2\mathit{dt}\right)}^{\frac{1}{2}}{\left(\underset{0}{\overset{\infty }{\int }}{\left|\overline{f}\left(t\right)\right|}^2\mathit{dt}\right)}^{\frac{1}{2}},$$ E89

$$\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_3\left(\lambda \right)x^{-\alpha }{\left(\underset{x}{\overset{\infty }{\int }}{\left(\frac{t}{x}\right)}^{2a\left(\alpha ,\lambda \right)-\alpha }exp\frac{-2x^{\alpha +1}\left({\left(\frac{t}{x}\right)}^{\alpha +1}-1\right)}{1+\alpha }\mathit{dt}\right)}^{\frac{1}{2}}.$$ E90

Replacing variables $t=\mathit{xu}$ , we get

$$\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_3\left(\lambda \right)x^{-\alpha +\frac{1}{2}}{\left(\underset{1}{\overset{\infty }{\int }}{u}^{2a\left(\alpha ,\lambda \right)-\alpha }exp\frac{-2x^{\alpha +1}\left(u^{\alpha +1}-1\right)}{1+\alpha }\mathit{du}\right)}^{\frac{1}{2}}.$$ E91

Since the inequality $exp\frac{-x^{\alpha +1}\left(u^{\alpha +1}-1\right)}{1+\alpha }\le x^{-2}$ holds for all $\alpha \in \left(0,1\right]$ and $u\in \left[1,\infty \right)$ with sufficiently large $u\in \left[1,\infty \right)$ , we have

$$\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_3\left(\lambda \right)x^{-\alpha -\frac{1}{2}}{\left(\underset{1}{\overset{\infty }{\int }}{u}^{2a\left(\alpha ,\lambda \right)-\alpha }exp\frac{-x^{\alpha +1}\left(u^{\alpha +1}-1\right)}{1+\alpha }\mathit{du}\right)}^{\frac{1}{2}}.$$ E92

Hence it follows that $\left|{\overline{\chi }}_4\left(x,\lambda \right)\right|\le c_4\left(\alpha ,\lambda \right)x^{-\alpha -\frac{1}{2}}$ , therefore ${\overline{\chi }}_4\left(x,\lambda \right)\in L_2\left(\mathbf{H},\left(0,\infty \right)\right).$ In case, where $0<\alpha \le 1$ and $\frac{\alpha +1}{2\alpha }\notin N$ and where $\alpha >1$ , the proof is similar.

Thus, $R_{\lambda }\overline{f}\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ for any function $\overline{f}\in L_2\left(\mathbf{H},\left(0,\infty \right)\right)$ . $\square$

Since the resolvent $R_{\lambda }$ is a meromorphic function of $\lambda$ , the poles of which coincide with the eigenvalues of the operator $L$ , the statement of Theorem 3.1 can be refined.

Theorem 4.2 If the conditions (3) - (5) ) where $\alpha >1$ or condition (6) where $0<\alpha \le 1$ are satisfied for the Eq. (1) , then the spectrum of the operator $L$ is real, discrete and coincides with the union of spectra of self-adjoint operators $L_k,k=\overline{1,m}$ , i.e. $\sigma \left(L\right)={\cup }_{k=1}^r\sigma \left(L_k\right)$ .

In this section, a resolvent for a non-self-adjoint differential operator with a block-triangular operator potential, increasing at infinity, is constructed. Sufficient conditions under which the spectrum is real and discrete are obtained.

5. Spectral singularities of differential operator with triangular matrix coefficients

Remark 5.1 If the perturbation $U\left(x\right)$ in Eq. (1) does not satisfy conditions (3) - (5) or condition (6) , then the statement of Theorem 4.2 ceases to be true, which is shown by the following example.

Example 5.1 Consider the equation:

$$l\left[\overline{y}\right]=-{\overline{y}}^{\prime \prime }+\left(\begin{array}{cc}{x}^2& q\left(x\right)\\ 0& {\pi }^2{x}^2\end{array}\right)\overline{y}=\lambda \overline{y},0\le x<\infty ,\overline{y}=\left(\begin{array}{c}{y}_1\\ y_2\end{array}\right)$$ E93

with the boundary condition

$$\overline{y}\left(0\right)=0.$$ E94

Together with the problem (93), (94), consider the separated system

$$l_1\left[y_1\right]=-{y_1}^{\prime \prime }+x^2{y}_1=\lambda y_1,$$ E95

$$l_2\left[y_2\right]=-y_2^{″}+{\pi }^2{x}^2{y}_2=\lambda y_2$$ E96

with the boundary conditions.

$$y_1\left(0\right)=0,y_2\left(0\right)=0.$$ E97

As above, denote by $L_0$ the differential operator generated by the differential expression $l\left[\overline{y}\right]$ (93) and the boundary condition (94), and by $L_1,L_2$ denote the minimal symmetric operators on $L_2\left(0;\infty \right)$ , generated by the differential expressions $l_1\left[y_1\right],l_2\left[y_2\right]$ and the boundary conditions (97). Their self-adjoint extensions $\widetilde{L_1},\widetilde{L_2}$ are the closures of the operators $L_1,L_2$ , respectively. The operators $\widetilde{L_1},\widetilde{L_2}$ are semi-bounded; let us denote their spectra by ${\sigma }_1=\sigma \left(\widetilde{L_1}\right),{\sigma }_2=\sigma \left(\widetilde{L_2}\right)$ .

The Eq. (95) (cf. (49)) has the solution $y_{1,n}\left(x\right)=H_n\left(x\right)\cdot \mathit{exp}\left(-\frac{x^2}{2}\right)$ for $\lambda =2n+1$ . Since $H_{2n+1}\left(0\right)=0$ , the eigenvalues of the operator $\widetilde{L_1}$ are ${\lambda }_n=4n+3$ . The sets ${\sigma }_1$ and ${\sigma }_2$ do not intersect.

Denote by $L$ the extension of the operator $L_0$ generated by the requirement on the functions from the domain of the operator $L$ to belong to $L_2\left(H_2,\left(0;\infty \right)\right)$ , and by $\sigma \left(L\right)$ its spectrum.

Denote by $Y\left(x,\lambda \right)=\left(\begin{array}{cc}{y}_{11}\left(x,\lambda \right)& y_{12}\left(x,\lambda \right)\\ 0& y_{22}\left(x,\lambda \right)\end{array}\right)$ the matrix solution of the Eq. (93), satisfying the initial conditions $Y\left(0,\lambda \right)=0,Y^{\prime }\left(0,\lambda \right)=I$ .

If some ${\lambda }_0\in \sigma \left({\tilde{L}}_1\right)$ , and $y\left(x,{\lambda }_0\right)$ - is the corresponding eigenfunction of the operator ${\tilde{L}}_1$ , then the vector function $\overline{y}\left(x,{\lambda }_0\right)=\left(\begin{array}{c}y\left(x,{\lambda }_0\right)\\ 0\end{array}\right)$ is the eigenfunction of the operator $L$ , corresponding to the eigenvalue ${\lambda }_0$ , i.e. ${\lambda }_0\in \sigma \left(L\right)$ . Moreover, ${\lambda }_0\in \sigma \left({\tilde{L}}_2\right)$ is the eigenvalue of the operator $L$ if and only if the solution $y_{12}\left(x,{\lambda }_0\right)$ of the equation

$$-{y_{12}}^{\prime \prime }+x^2{y}_{12}+q\left(x\right)y_{22}={\lambda }_0{y}_{12},$$ E98

satisfying the initial conditions $y_{12}\left(0,\lambda \right)={y_{12}}^{\prime }\left(0,\lambda \right)=0$ , belongs to $L_2\left(0;\infty \right)$ . Let $u\left(x,\lambda \right),v\left(x,\lambda \right)$ be the solutions of the Eq. (95), satisfying the initial conditions $u\left(0,\lambda \right)=0,u^{\prime }\left(0,\lambda \right)=1,v\left(0,\lambda \right)=-1,v^{\prime }\left(0,\lambda \right)=0$ , and let $C\left(x,t,\lambda \right)=u\left(x,\lambda \right)v\left(t,\lambda \right)-v\left(x,\lambda \right)u\left(t,\lambda \right)$ - be the Cauchy function of the Eq. (95). Then the solution $y_{12}\left(x,{\lambda }_0\right)$ is given by

$$y_{12}\left(x,{\lambda }_0\right)=\underset{0}{\overset{x}{\int }}q\left(t\right)\cdot C\left(x,t,{\lambda }_0\right)\cdot y_{22}\left(t,{\lambda }_0\right)\mathit{dt}.$$ E99

Choose the coefficient $q\left(x\right)=y_{22}\left(x,{\lambda }_0\right)e^{x^{\mu }}$ , where $\mu >2$ (for instance, $\mu =4$ ), and show that the integral $\underset{0}{\overset{\infty }{\int }}{y}_{12}^2\left(x,{\lambda }_0\right)\mathit{dx}$ diverges and, consequently, ${\lambda }_0\notin \sigma \left(L\right)$ . Indeed, since the solution $y_{22}\left(x,{\lambda }_0\right)$ has finitely many zeros, we conclude that, for any $x\ge N_1>0$ ,

$$y_{22}\left(x,{\lambda }_0\right)\ge c_1{e}^{-\alpha x^2},\alpha >0,$$ E100

and the Cauchy function decays no faster than $e^{-{\left(x-t\right)}^2}$ . Hence, if $\left|x-t\right|>N_2$ , we have

$$C\left(x,t,{\lambda }_0\right)\ge c_2{e}^{-{\left(x-t\right)}^2}.$$ E101

In the case of $\frac{x}{4}\le t\le \frac{x}{2}$ and $x\ge max\left(4N_1,2N_2\right)$ , the inequalities (100) and (101) are fulfilled simultaneously, therefore, $y_{12}\left(x,{\lambda }_0\right)>c_3{\int }_{\frac{x}{4}}^{\frac{x}{2}}{e}^{t^4}\cdot e^{-2\alpha t^2}\cdot e^{-{\left(x-t\right)}^2}\mathit{dt}$ . Since $e^{-{\left(x-t\right)}^2}\ge e^{-\frac{x^2}{4}}$ for $t\le \frac{x}{2}$ , we get $y_{12}\left(x,{\lambda }_0\right)>c_3{e}^{-\frac{x^2}{4}}{\int }_{\frac{x}{4}}^{\frac{x}{2}}{e}^{t^4}\cdot e^{-2\alpha t^2}\mathit{dt}$ . If $x$ is sufficiently large and $t\in \left[\frac{x}{4},\frac{x}{2}\right]$ , we have $e^{t^4-2\alpha t^2}>e^{\frac{1}{2}{t}^4}\ge e^{\frac{x^4}{32}}$ , hence for $x\to \infty y_{12}\left(x,{\lambda }_0\right)>c_3\frac{x}{4}{e}^{-\frac{x^2}{4}+\frac{x^4}{32}}\to \infty$ . It follows that $y_{12}\left(x,{\lambda }_0\right)\notin L_2\left(0;\infty \right)$ and ${\lambda }_0\notin \sigma \left(L\right)$ .

There arises the question on the nature of such values $\lambda$ .

Consider the equation with a triangular matrix potential:

$$l\left[\overline{y}\right]=-{\overline{y}}^{\prime \prime }+\left(\begin{array}{cc}p\left(x\right)& q\left(x\right)\\ 0& r\left(x\right)\end{array}\right)\overline{y}=\lambda \overline{y},0\le x<\infty ,\overline{y}=\left(\begin{array}{c}{y}_1\\ y_2\end{array}\right),$$ E102

where $p\left(x\right),q\left(x\right),r\left(x\right)$ are scalar functions, $p\left(x\right),r\left(x\right)$ are real functions and $p\left(x\right),r\left(x\right)\to \infty$ monotonically as $x\to \infty$ .

Let the boundary condition is given at $x=0$ :

$$cosA\cdot {\overline{y}}^{\prime }\left(0\right)-sinA\cdot \overline{y}\left(0\right)=0,$$ E103

where $A$ is a triangular matrix, $\mathit{\text{cosA}}=\left(\begin{array}{cc}cos{\alpha }_{11}& \mathit{cos}{\alpha }_{12}\\ 0& cos{\alpha }_{22}\end{array}\right)$ .

Consider the separated system

$$l_1\left[y_1\right]=-{y_1}^{\prime \prime }+p\left(x\right)y_1=\lambda y_1,$$ E104

$$l_2\left[y_2\right]=-y_2^{″}+r\left(x\right)y_2=\lambda y_2.$$ E105

with the boundary conditions

$$cos{\alpha }_{11}{y}_1^{\prime }\left(0\right)-sin{\alpha }_{11}{y}_1\left(0\right)=0,$$ E106

$$cos{\alpha }_{22}{y}_2^{\prime }\left(0\right)-sin{\alpha }_{22}{y}_2\left(0\right)=0.$$ E107

Let $L_0$ be the differential operator generated by the differential expression $l\left[\overline{y}\right]$ (103) and the boundary condition (104), and let $L_1,L_2$ be minimal symmetric operators on $L_2\left(0,\infty \right)$ generated by the differential expressions $l_1\left[y_1\right],l_2\left[y_2\right]$ and the boundary conditions (106), (108) respectively. Denote by $\widetilde{L_1},\widetilde{L_2}$ the self-adjoint extensions of the operators $L_1,L_2$ respectively. The operators $\widetilde{L_1},\widetilde{L_2}$ are semi-bounded; let us denote their spectra by ${\sigma }_1$ and ${\sigma }_2$ respectively. Denote by $L$ the extension of the operator $L_0$ and by $\sigma \left(L\right)$ its spectrum.

Let $u\left(x,\lambda \right),v\left(x,\lambda \right)$ be the solutions of the Eq. (104) with the boundary conditions $u\left(0,\lambda \right)=0,u^{\prime }\left(0,\lambda \right)=1,$ $v\left(0,\lambda \right)=-1,v^{\prime }\left(0,\lambda \right)=0.$ The general solution of the Eq. (104) has the form $\phi \left(x,\lambda \right)=u\left(x,\lambda \right)+\mathit{lv}\left(x,\lambda \right)$ up to a constant. Choose an $l$ such that the condition $\phi \left(b,\lambda \right)=0$ holds true. This equality is valid for $l=l\left(b,\lambda \right)=-\frac{u\left(b,\lambda \right)}{v\left(b,\lambda \right)}$ (the solution $v\left(x,\lambda \right)$ has finitely many zeros for a fixed $\lambda$ , hence $v\left(b,\lambda \right)\ne 0$ whenever $b$ is sufficiently large). Put ${\phi }_{11}^{\left(b\right)}\left(x,\lambda \right)=u\left(x,\lambda \right)+l\left(b,\lambda \right)v\left(x,\lambda \right)$ . Since for the operator $L_1$ there is the case of a limit point, then, as is known, $l\left(b,\lambda \right)$ has a unique limit $m\left(\lambda \right)$ as $b\to \infty$ , and the solution of the Eq. (104) satisfies ${\phi }_{11}\left(x,\lambda \right)=u\left(x,\lambda \right)+m\left(\lambda \right)v\left(x,\lambda \right)\in L_2\left(0,\infty \right)$ . Similarly we obtain that the solution of the Eq. (105) satisfies ${\phi }_{22}\left(x,\lambda \right)\in L_2\left(0,\infty \right)$ .

Denote by ${\mathrm{\Phi }}_b\left(x,\lambda \right)=\left(\begin{array}{cc}{\phi }_{11}^{\left(b\right)}\left(x,\lambda \right)& {\phi }_{12}^{\left(b\right)}\left(x,\lambda \right)\\ 0& {\phi }_{22}^{\left(b\right)}\left(x,\lambda \right)\end{array}\right)$ the matrix solution of the Eq. (103) satisfying the initial conditions ${\mathrm{\Phi }}_b\left(b,\lambda \right)=0,{{\mathrm{\Phi }}_b}^{\prime }\left(b,\lambda \right)=I$ . We have ${\phi }_{11}^{\left(b\right)}\left(x,\lambda \right)\to {\phi }_{11}\left(x,\lambda \right)\in L_2\left(0,\infty \right)$ ; ${\phi }_{22}^{\left(b\right)}\left(x,\lambda \right)\to {\phi }_{22}\left(x,\lambda \right)\in L_2\left(0,\infty \right)$ as $b\to \infty$ .

The solution ${\phi }_{12}^{\left(b\right)}\left(x,\lambda \right)$ is given by ${\phi }_{12}^{\left(b\right)}\left(x,\lambda \right)=\underset{0}{\overset{x}{\int }}q\left(t\right)\cdot C\left(x,t,\lambda \right)\cdot {\phi }_{22}^{\left(b\right)}\left(t,\lambda \right)\mathit{dt}$ , where $C\left(x,t,\lambda \right)=u\left(x,\lambda \right)v\left(t,\lambda \right)-v\left(x,\lambda \right)u\left(t,\lambda \right)$ is the Cauchy function of the Eq. (104).

Further, we have ${\phi }_{12}^{\left(b\right)}\left(x,\lambda \right)\to \underset{0}{\overset{x}{\int }}q\left(t\right)\cdot C\left(x,t,\lambda \right)\cdot {\phi }_{22}\left(t,\lambda \right)\mathit{dt}≔{\phi }_{12}\left(x,\lambda \right)$ as $b\to \infty$ . Put $\mathrm{\Phi }\left(x,\lambda \right)=\left(\begin{array}{cc}{\phi }_{11}\left(x,\lambda \right)& {\phi }_{12}\left(x,\lambda \right)\\ 0& {\phi }_{22}\left(x,\lambda \right)\end{array}\right)$ .

Together with the Eq. (102), we consider the left equation.

$$\tilde{l}\left[\tilde{y}\right]=-{\tilde{y}}^{\prime \prime }+\tilde{y}V\left(x\right)=\lambda \tilde{y},\tilde{y}=\left(y_1{y}_2\right).$$ E108

The matrix solutions of the Eq. (108) will be denoted by ${\tilde{\mathrm{\Phi }}}_b\left(x,\lambda \right)$ and $\tilde{\mathrm{\Phi }}\left(x,\lambda \right)$ .

Denote by $Y\left(x,\lambda \right)$ and $\tilde{Y}\left(x,\lambda \right)$ the solutions of the Eqs. (102) and (108) respectively satisfying the initial conditions

$$Y\left(0,\lambda \right)=cosA,Y^{\prime }\left(0,\lambda \right)=sinA,\tilde{Y}\left(0,\lambda \right)=cosA,{\tilde{Y}}^{\prime }\left(0,\lambda \right)=sinA,\lambda \in \mathbb{C}.$$ E109

Put

$$G_b\left(x,t,\lambda \right)=\left\{\begin{array}{l}Y\left(x,\lambda \right){\left(W\left({\tilde{\mathrm{\Phi }}}_b,Y\right)\right)}^{-1}{\tilde{\mathrm{\Phi }}}_b\left(t,\lambda \right)0\le x\le t\\ -{\mathrm{\Phi }}_b\left(x,\lambda \right){\left(W\left(\tilde{Y},{\mathrm{\Phi }}_b\right)\right)}^{-1}\tilde{Y}\left(t,\lambda \right)t\le x\le b\end{array}\right..$$ E110

The function $G_b\left(x,t,\lambda \right)$ is the Green function of the operator $L_b^0$ generated by the problem (102), (103), $y\left(b\right)=0$ , which spectrum coincides with the union of spectra of the operators $L_{b,1}^0,L_{b,2}^0$ generated by the problems (104), (106), $y_1\left(b\right)=0$ and (105), (107), $y_2\left(b\right)=0$ respectively. Eigenvalues of the operators $L_{b,1}^0$ and $L_{b,2}^0$ tend to ones of the operators ${\tilde{L}}_1$ and ${\tilde{L}}_2$ respectively as $b\to \infty$ , ${\mathrm{\Phi }}_b\left(x,\lambda \right)\to \mathrm{\Phi }\left(x,\lambda \right),$ ${\tilde{\mathrm{\Phi }}}_b\left(x,\lambda \right)\to \tilde{\mathrm{\Phi }}\left(x,\lambda \right)$ , and

$$\begin{array}{cc}W\left(\tilde{Y},{\mathrm{\Phi }}_b\right)& =cosA\cdot {{\mathrm{\Phi }}_b}^{\prime }\left(0,\lambda \right)-sinA\cdot {\mathrm{\Phi }}_b\left(0,\lambda \right)\to cosA\cdot {\mathrm{\Phi }}^{\prime }\left(0,\lambda \right)-sinA\cdot \mathrm{\Phi }\left(0,\lambda \right)=\\ & =W\left(\tilde{Y},\mathrm{\Phi }\right),W\left({\tilde{\mathrm{\Phi }}}_b,Y\right)\to W\left(\tilde{\mathrm{\Phi }},Y\right),\end{array}$$ E111

$$G_b\left(x,t,\lambda \right)\to G\left(x,t,\lambda \right)=\left\{\begin{array}{l}Y\left(x,\lambda \right){\left(W\left(\tilde{\mathrm{\Phi }},Y\right)\right)}^{-1}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)0\le x\le t\\ -\mathrm{\Phi }\left(x,\lambda \right){\left(W\left(\tilde{Y},\mathrm{\Phi }\right)\right)}^{-1}\tilde{Y}\left(t,\lambda \right)t\le x\end{array}\right..$$ E112

Poles of the Green function $G\left(x,t,\lambda \right)$ of the operator $L$ coincide with the zero set of the determinant $\mathrm{\Delta }\left(\lambda \right)≔det\mathrm{\Omega }\left(\lambda \right)$ , where.

$$\mathrm{\Omega }\left(\lambda \right)={\left.W\left(\tilde{Y},\mathrm{\Phi }\right)\right|}_{x=0}=cosA\cdot {\mathrm{\Phi }}^{\prime }\left(0,\lambda \right)-sinA\cdot \mathrm{\Phi }\left(0,\lambda \right).$$ E113

Since the matrices $cosA,sinA,\mathrm{\Phi }\left(0,\lambda \right),{\mathrm{\Phi }}^{\prime }\left(0,\lambda \right)$ are triangle, we have _/ $\mathrm{\Delta }\left(\lambda \right)={\mathrm{\Delta }}_1\left(\lambda \right)\cdot {\mathrm{\Delta }}_2\left(\lambda \right)$ , where ${\mathrm{\Delta }}_k\left(\lambda \right)=cos{\alpha }_{\mathit{kk}}\cdot {\phi }_{\mathit{kk}}^{\prime }\left(0,\lambda \right)-sin{\alpha }_{\mathit{kk}}\cdot {\phi }_{\mathit{kk}}\left(0,\lambda \right),k=1,2$ . On the other hand, zeros of the function ${\mathrm{\Delta }}_k\left(\lambda \right)$ are eigenvalues of the self-adjoint operator ${\tilde{L}}_k$ . Hence the poles of the Green function $G\left(x,t,\lambda \right)$ of the operator $L$ are situated on the real axis, and their set coincides with the union of spectra of the operators ${\tilde{L}}_1$ and ${\tilde{L}}_2$ .

Consider the operator $R_{\lambda ,b}$ defined on $L_2\left(H_2,\left(0;b\right)\right)$ by.

$$\left(R_{\lambda ,b}\overline{f}\right)\left(x\right)=\underset{0}{\overset{b}{\int }}{G}_b\left(x,t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}=-\underset{0}{\overset{x}{\int }}{\mathrm{\Phi }}_b\left(x,\lambda \right){\left(W\left(\tilde{Y},{\mathrm{\Phi }}_b\right)\right)}^{-1}\tilde{Y}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}+$$

$$+\underset{x}{\overset{b}{\int }}Y\left(x,\lambda \right){\left(W\left({\tilde{\mathrm{\Phi }}}_b,Y\right)\right)}^{-1}\tilde{\mathrm{\Phi }}\left(t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}.$$ E114

One can directly verify that the operator $R_{\lambda ,b}$ is the resolvent of the operator $L_b^0$ .

Let $\overline{f}\left(x\right)$ be an arbitrary vector function square integrable on $\left[0,\infty \right)$ . Choose a sequence of finite continuous vector functions $\left\{{\overline{f}}_n\left(x\right)\right\}\left(n=1,2,\dots \right)$ converging in mean square to $\overline{f}\left(x\right)$ . Substituting ${\overline{f}}_n$ for $\overline{f}$ in (114) and letting first $b\to \infty$ and then $n\to \infty$ , we obtain the following formula for the resolvent $R_{\lambda }$ of the operator $L$ : $\left(R_{\lambda }\overline{f}\right)\left(x\right)={\int }_0^{\infty }G\left(x,t,\lambda \right)\overline{f}\left(t\right)\mathit{dt}$ , where the Green function of the operator $L$ is defined by the formula (112).

Theorem 5.1 The operator $R_{\lambda }$ is the resolvent of the operator $L$ . The resolvent’s poles coincide with the union of the spectra of the self-adjoint operators ${\tilde{L}}_1$ and ${\tilde{L}}_2$ .

Remark 5.2 As in Example 5.1, if ${\lambda }_0\in \sigma \left({\tilde{L}}_2\right)$ and ${\phi }_{12}\left(x,{\lambda }_0\right)\notin L_2\left(0,\infty \right)$ , then ${\lambda }_0$ is the pole of the resolvent $R_{\lambda }$ of the operator $L$ but it is not the eigenvalue of this operator, i.e., ${\lambda }_0$ is the point of the spectral singularity of the operator $L$ .

Theorem 5.1 implies that, if the rate of the coefficient’s growth $q\left(x\right)$ of the Eq. (102) is subordinated to one of $p\left(x\right)$ and $r\left(x\right)$ , then the operator $L$ has no spectral singularities, and its spectrum is real and coincides with the union of the spectra of the operators ${\tilde{L}}_1$ and ${\tilde{L}}_2$ .

For a non-self-adjoint Sturm-Liouville operator with a triangular matrix potential growing at infinity, an example of operator having spectral singularities is constructed. A special role of these points was found first by M.A. Naimark in [16]. The notion “spectral singularity” was introduced later due to J. Schwartz [17] (see also Supplement I in the monograph [3]).

6. Conclusion

We consider the Sturm-Liouville equation with block-triangular, increasing at infinity operator potential. For him, built a fundamental system of solutions, one of which is decreasing at infinity, and the second is growing. The asymptotics of these solutions at infinity is defined. For non-self-adjoint operator generated by such differential expression obtained the Green’s function. A resolvent of such an operator is constructed. Sufficient conditions at which a spectrum of such non-self-adjoint differential operator is real and discrete are obtained.