Bilinear Applications and Tensors

Abstract

In this chapter, a theoretical approach to the vector space of tensor of order 3 and the vector space of bilinear applications will be presented in order to present an isomorphism between these spaces and several properties about tensor and bilinear applications. With this well-defined isomorphism, we will present how to calculate the product between tensor of second derivatives and a vector, where such a product is used in several numerical methods such as Chebyshev-Halley class and others mentioned in the introduction. In addition, concepts on differentiability are presented, allowing a better understanding for the reader about second-order derivatives seen as a tensor.

1. Introduction

Frequently, discretization of mathematical models demands solving a system of equations, which is generally nonlinear. Such mathematical problems might be written as

$$\text{find}{x}^{\ast }\in {\mathrm{I}\mathrm{R}}^n\text{such\hspace{0.17em}that}F\left(x^{\ast }\right)=0$$E1

where $F:—{\mathrm{I}\mathrm{R}}^n\to {\mathrm{I}\mathrm{R}}^n$.

There exist iterative methods for solving (1) that have cubic convergence rate, for instance, the methods belonging to the following class of methods named Chebyshev-Halley class, which was introduced by Hernández and Gutiérrez in [1]:

$$x^{k+1}=x^k-\left[I+\frac{1}{2}\mathcal{L}\left(x^k\right){\left(I-\alpha \mathcal{L}\left(x^k\right)\right)}^{-1}\right]J_F{\left(x^k\right)}^{-1}F\left(x^k\right),$$E2

for all $k\in \mathrm{I}\mathrm{N}$, where

$$\mathcal{L}\left(x\right)=J_F{\left(x\right)}^{-1}{\mathcal{T}}_F\left(x\right)\left(J_F{\left(x\right)}^{-1}F\left(x\right)\right),$$E3

and $J_F\left(x\right)$ and ${\mathcal{T}}_F\left(x\right)$ denote the first and second derivatives of $F$ evaluated at $x$, respectively. The parameter $\alpha$ is a real number and $I$ is the identity matrix in ${\mathrm{I}\mathrm{R}}^{n\times n}$.

Discretized versions of Chebyshev-Halley class have already been considered in [2] in such a way that the tensor of second derivatives of the function $F$ was approximated by bilinear operators. A tensor is a multi-way array or multidimensional matrix. A generalization of the Chebyshev-Halley class $\left(2\right)$ where no second-order derivative information is required but that also has cubic convergence rate, named inexact tensor-free Chebyshev-Halley class, was introduced by Eustaquio, Ribeiro, and Dumett [3]. Other families of iterative methods with cubic convergence rate were extensively described in Traub’s book [4].

Several alternatives exist for the product of the tensor of second derivatives of $F$ by vectors [5, 6, 7, 8], and this needs to be elucidated.

The aim of this chapter is to present concepts and relationships between tensors of order $3$ and bilinear applications, in order to relate them to the second derivative of a two-differentiable application. We will see later that given the vectors $u,v\in {\mathrm{I}\mathrm{R}}^n$, the i-th row of the matrix ${\mathcal{T}}_F\left(x\right)v$ is defined by $v^T{\nabla }^2{f}_i\left(x\right)$, where ${\nabla }^2{f}_i\left(x\right)$ is the Hessian of the i-th component of $F$ evaluated at $x$. The i-th component of the vector ${\mathcal{T}}_F\left(x\right)\mathit{vu}$ is defined by $v^T{\nabla }^2{f}_i\left(x\right)u$.

2. Tensors

Tensors naturally arise in some applications, such as chemometry [9], signal processing [10], and others. According to [8], for many applications involving high-order tensors, the known results of matrix algebra seemed to be insufficient in the twentieth century. There were some workshops and congresses on the study of tensors, such as:

• Workshop on Tensor Decomposition at the American Institute of Mathematics which took place at the Palo Alto, California, 2004, organized by Golub, Kolda, Nagy, and Van Loan. Details in [11];

• Workshop on Tensor Decompositions and Applications, 2005, organized by Comon and De Lathauwer. Details in [12]; and

• Minisymposium on Numerical Multilinear Algebra: A New Beginning, 2007, organized by Golub, Comon, De Lathauwer, and Lim and which took place at the Zurich.

Readers interested in multilinear singular value decomposition, eigenvalues, and eigenvectors may consult as references [5, 6, 7, 8, 13, 14]. In this text, we will focus our attention on tensors of order 3.

Let $I_1,I_2$, and $I_3$ be three positive integers. A tensor $\mathcal{T}$ of order $3$ is an three-way array where its elements $t_{i_1{i}_2}^{i_3}$ are indexed by $i_1=1,\dots ,I_1$, $i_2=1,\dots ,I_2$, and $i_3=1,\dots ,I_3$ and the n-th dimension of the tensor is denoted by $I_n$, for $n=\mathrm{1,2,3}$. For example, the first, second, and third dimensions of a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{2\times 4\times 3}$ are $\mathrm{2,4,3}$, respectively.

Obviously, tensors are generalizations of matrices. A matrix can be viewed as a tensor of order $2$, while a vector can be viewed as a tensor of order $1$.

From an algebraic point of view, a tensor $\mathcal{T}$ of order $3$ is an element of the vector space ${\mathrm{I}\mathrm{R}}^{I_1\times I_2\times I_3}$, whereas from the geometric point of view, a tensor $\mathcal{T}$ of order $3$ can be seen as a parallelepiped [15], with $I_1$ rows, $I_2$ columns, and $I_3$ tubes. Figure 1 illustrates a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{2\times 4\times 3}$.

Figure 1.

A tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{2\times 4\times 3}$.

In linear algebra, it is common to see a matrix through its columns. If $A\in {\mathrm{I}\mathrm{R}}^{m\times n}$, then $A$ can be viewed as $A=\left[a_1\dots a_n\right]$, where $a_j\in {\mathrm{I}\mathrm{R}}^m$ denotes the j-th column of the matrix $A$. In the case of tensor of order $3$, we can see them through fibers and slices. Hence follow the definitions.

Definition 1.1. A tensor fiber of a tensor of order $3$ is a one-dimensional fragment obtained by fixing only two indices.

Definition 1.2. A tensor slice of a tensor of order $3$ is a two-dimensional section (fragment), obtained by fixing only one index.

Generally in tensors of order $3$, a fiber is a vector and a slice is a matrix. We have three types of fibers:

• column fibers (or mode-$1$ fiber), where the indices $i_2$ and $i_3$ are fixed;

• row fibers (or mode-$2$ fiber), where the indices $i_1$ and $i_3$ are fixed; and

• tube fibers (or mode-$3$ fiber), where the indices $i_1$ and $i_2$ are fixed.

  We also have three types of slices:

• horizontal slice, where the index $i_1$ is fixed;

• lateral slice, where the index $i_2$ is fixed; and

• frontal slice, where the index $i_3$ is fixed.

For example, consider a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{2\times 4\times 3}$ with $i=1,2$, $j=\mathrm{1,2,3,4}$, and $k=\mathrm{1,2,3}$. The i-th horizontal slice, denoted by ${\mathcal{T}}^{i::}$, is the matrix

$${\mathcal{T}}^{i::}=\left(\begin{array}{ccc}{t}_{i1}^1& t_{i1}^2& t_{i1}^3\\ t_{i2}^1& t_{i2}^2& t_{i2}^3\\ t_{i3}^1& t_{i3}^2& t_{i3}^3\\ t_{i4}^1& t_{i4}^2& t_{i4}^3\end{array}\right),$$

the j-th lateral slice, denoted by ${\mathcal{T}}^{:j:}$, is the matrix

$${\mathcal{T}}^{:j:}=\left(\begin{array}{ccc}{t}_{1j}^1& t_{1j}^2& t_{1j}^3\\ t_{2j}^1& t_{2j}^2& t_{2j}^3\end{array}\right)$$

and the k-th frontal slice, denoted by ${\mathcal{T}}^{::k}$, is the matrix

$${\mathcal{T}}^{::k}=\left(\begin{array}{cccc}{t}_{11}^k& t_{12}^k& t_{13}^k& t_{14}^k\\ t_{21}^k& t_{22}^k& t_{23}^k& t_{24}^k\end{array}\right).$$E4

Figures 2 and 3 illustrate the three types of fibers and slices, respectively, of a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{2\times 4\times 3}$.

Figure 2.

Columns, rows, and tube fibers, respectively.

Figure 3.

Horizontal, lateral, and frontal slices, respectively.

2.1 Tensor operations

The first issue to consider in this subsection is how to calculate the product between tensors and matrices. It is well known from elementary algebra that given matrices $A\in {\mathrm{I}\mathrm{R}}^{m\times n}$ and $B\in {\mathrm{I}\mathrm{R}}^{R\times m}$, it is possible to calculate the product $\mathit{BA}$, because the first dimension (number of rows) of matrix $A$ agrees with the second dimension (number of columns) of matrix $B$, and each product element is the result of the inner product between rows of matrix $B$ and columns of matrix $A$.

The product between tensors of order $3$ and matrices or vectors is a bit more complicated. In order to obtain an element of the product between a tensor and a matrix, it is necessary to specify what dimension of the tensor will be chosen to agree with the number of columns of the matrix, and each resulting element will be a result of the inner product between the mode-$n$ fibers (column, row, or tube) and the columns of the matrix. We will use the solution adopted by [8], which defines the product mode-$n$ between tensors and matrices and the solution adopted by [5] that defines the contracted product mode-$n$ between tensors and vectors.

The mode-$n$ product is useful when one wants to decompose into singular values a high-order tensor in order to avoid the use of the generalized transpose concept. We refer to [5, 7, 8, 13] for details.

Definition 1.3. (mode-n tensor matrix product) The mode-$1$ product between a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a matrix $A\in {\mathrm{I}\mathrm{R}}^{R\times m}$ is a tensor

$$\mathcal{Y}=\mathcal{T}{\times }_{1}A\in {\mathrm{I}\mathrm{R}}^{R\times n\times p}$$

where its elements are defined by

$$y_{\mathit{rj}}^k=\sum _{i=1}^m{t}_{\mathit{ij}}^k{a}_{\mathit{ri}}\text{where}r=1,\dots ,R,j=1,\dots ,n,\text{and}k=1,\dots ,p.$$

The mode-$2$ product between a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a matrix $A\in {\mathrm{I}\mathrm{R}}^{R\times n}$ is a tensor

$$\mathcal{Y}=\mathcal{T}{\times }_{2}A\in {\mathrm{I}\mathrm{R}}^{m\times R\times p}$$

where its elements are defined by

$$y_{\mathit{ir}}^k=\sum _{j=1}^n{t}_{\mathit{ij}}^k{a}_{\mathit{rj}}\text{where}i=1,\dots ,m,r=1,\dots ,R\text{and}k=1,\dots ,p.$$

The mode-$3$ product between a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a matrix $A\in {\mathrm{I}\mathrm{R}}^{R\times p}$ is a tensor

$$\mathcal{Y}=\mathcal{T}{\times }_{3}A\in {\mathrm{I}\mathrm{R}}^{m\times n\times R}$$

where its elements are defined by

$$y_{\mathit{ij}}^r=\sum _{k=1}^p{t}_{\mathit{ij}}^k{a}_{\mathit{rk}}\text{where}i=1,\dots ,m,j=1,\dots ,n\text{and}r=1,\dots ,R.$$

To understand the mode-$n$ product in terms of matrix, consider matrices $A\in {\mathrm{I}\mathrm{R}}^{m\times n}$, $B\in {\mathrm{I}\mathrm{R}}^{k\times m}$, and $C\in {\mathrm{I}\mathrm{R}}^{q\times n}$. By Definition 1.3 we have

$$A{\times }_{1}B=\mathit{BA}\in {\mathrm{I}\mathrm{R}}^{k\times n}\text{and}A{\times }_{2}C={\mathit{AC}}^T\in {\mathrm{I}\mathrm{R}}^{m\times q}.$$

Thus, the singular value decomposition of matrix $A$ can be written as

$$U\mathrm{\Sigma }{V}^T=\left(\mathrm{\Sigma }{\times }_{1}U\right){\times }_{2}V=\left(\mathrm{\Sigma }{\times }_{2}V\right){\times }_{1}U.$$

The mode-$n$ product satisfies the following property [8]:

Property 1. Let $\mathcal{T}$ be a tensor of order $3$ and matrices $A$ and $B$ of convenient sizes. We have for all $r,s=\mathrm{1,2,3}$

$$\left(\mathcal{T}{\times }_{r}A\right){\times }_{s}B=\left(\mathcal{T}{\times }_{s}B\right){\times }_{r}A=\mathcal{T}{\times }_{r}A{\times }_{s}B\text{for}r\ne s\text{and}$$E5

$$\left(\mathcal{T}{\times }_{r}A\right){\times }_{r}B=\mathcal{T}{\times }_r\left(\mathit{BA}\right)$$E6

The idea of Bader and Kolda [5] to calculate the product between tensor and vector is to calculate the inner product of each mode-$n$ fiber (column, row, or tube) with the vector. It is not advantageous to treat an $m$-dimensional vector as a matrix $m\times 1$. For example, if we consider a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a vector $v\in {\mathrm{I}\mathrm{R}}^{m\times 1}$, with $m,n,p\ne 1$, by Definition 1.3, the product between $\mathcal{T}$ and $v$ is not well defined, but it is possible to calculate $\mathcal{T}{\times }_1{v}^T$.

Definition 1.4. (Contracted product mode-n between tensors and vectors) The contracted product mode-$1$ between a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a vector $v\in {\mathrm{I}\mathrm{R}}^m$ is the matrix

$$A=\mathcal{T}{\overline{\times }}_{1}v\in {\mathrm{I}\mathrm{R}}^{n\times p}$$

where its elements are defined by

$$a_{\mathit{jk}}=\sum _{i=1}^m{t}_{\mathit{ij}}^k{v}_i\text{where}j=1,\dots ,n\text{and}k=1,\dots ,p$$

where $v_i$ is the i-th component of the vector $v$.

The contracted product mode-$2$ between a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a vector $v\in {\mathrm{I}\mathrm{R}}^n$ is the matrix

$$A=\mathcal{T}{\overline{\times }}_{2}v\in {\mathrm{I}\mathrm{R}}^{m\times p}$$

where its elements are defined by

$$a_{\mathit{ik}}=\sum _{j=1}^n{t}_{\mathit{ij}}^k{v}_j\text{where}i=1,\dots ,m\text{and}k=1,\dots ,p$$

where $v_j$ is the j-th component of the vector $v$.

The contracted product mode-$3$ between a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times p}$ and a vector $v\in {\mathrm{I}\mathrm{R}}^p$ is the matrix

$$A=\mathcal{T}{\overline{\times }}_{3}v\in {\mathrm{I}\mathrm{R}}^{m\times n}$$

where its elements are defined by

$$a_{\mathit{ij}}=\sum _{k=1}^p{t}_{\mathit{ij}}^k{v}_k\text{where}i=1,\dots ,m\text{and}j=1,\dots ,n$$

where $v_k$ is the k-th component of the vector $v$.

A caution must be added when calculating the product between matrices and vectors by considering the definitions 1.3 and 1.4. For example, note that if $A\in {\mathrm{I}\mathrm{R}}^{m\times n}$, $u\in {\mathrm{I}\mathrm{R}}^n$, and $v\in {\mathrm{I}\mathrm{R}}^m$, then $A{\overline{\times }}_{2}u$ and $A{\times }_2{u}^T$ have the same elements, but

$$A{\overline{\times }}_{2}u\ne A{\times }_2{u}^T,$$

because $A{\overline{\times }}_{2}u\in {\mathrm{I}\mathrm{R}}^m$ (column vector) and $A{\times }_2{u}^T\in {\mathrm{I}\mathrm{R}}^{1\times m}$ (row vector). Note that, in relation to the matrix product of elementary algebra, we have

$$\mathit{Au}=A{\overline{\times }}_{2}u$$E7

$$v^{T}A=A{\times }_1{v}^T\ne A{\overline{\times }}_{1}v.$$E8

In particular, given a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{n\times m\times m}$ and a vector $v\in {\mathrm{I}\mathrm{R}}^m$, by Definition 1.4 together with (8), it follows that $\mathcal{T}{\overline{\times }}_{2}v\in {\mathrm{I}\mathrm{R}}^{n\times m}$ and

$$\left(\mathcal{T}{\overline{\times }}_{2}v\right){\overline{\times }}_{2}v=\left(\mathcal{T}{\overline{\times }}_{2}v\right)v\in {\mathrm{I}\mathrm{R}}^n.$$

The contracted product mode-$n$ satisfies the following property [5]:

Property 2. Given a tensor $\mathcal{T}$ of order $3$ and vectors $u$ and $v$ of convenient sizes, we have for all $r=\mathrm{1,2,3}$ and $s=2,3$ that

$$\left(\mathcal{T}{\overline{\times }}_{r}u\right){\overline{\times }}_{s-1}v=\left(\mathcal{T}{\overline{\times }}_{s}v\right){\overline{\times }}_{r}u\text{for}r<s.$$

For example, consider a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{2\times 4\times 3}$, and denote the k-th column and the q-th row of matrix $A$ by ${\mathrm{col}}_k\left(A\right)$ and ${\mathrm{row}}_q\left(A\right)$, respectively. Note that if:

1. 1. $x\in {\mathrm{I}\mathrm{R}}^2$, then $\mathcal{T}{\overline{\times }}_{1}x\in {\mathrm{I}\mathrm{R}}^{4\times 3}$ and

$$\begin{array}{c}{\mathrm{col}}_k\left(\mathcal{T}{\overline{\times }}_{1}x\right)=\left(\begin{array}{c}{a}_{1k}\\ a_{2k}\\ a_{3k}\\ a_{4k}\end{array}\right)=\left(\begin{array}{cc}{t}_{11}^k& t_{21}^k\\ t_{12}^k& t_{22}^k\\ t_{13}^k& t_{23}^k\\ t_{14}^k& t_{24}^k\end{array}\right)\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)={\left({\mathcal{T}}^{::k}\right)}^{T}x\text{and}\\ {\mathrm{row}}_j\left(\mathcal{T}{\overline{\times }}_{1}x\right)=\left(a_{j1}{a}_{j2}{a}_{j3}\right)=\left(\begin{array}{cc}{x}_1& x_2\end{array}\right)\left(\begin{array}{ccc}{t}_{1j}^1& t_{1j}^2& t_{11}^3\\ t_{2j}^1& t_{2j}^2& t_{21}^3\end{array}\right)=x^T{\mathcal{T}}^{:j:}\end{array}$$

1. 2. $x\in {\mathrm{I}\mathrm{R}}^4$, then $\mathcal{T}{\overline{\times }}_{2}x\in {\mathrm{I}\mathrm{R}}^{2\times 3}$ and

$$\begin{array}{c}{\mathrm{col}}_k\left(\mathcal{T}{\overline{\times }}_{2}x\right)=\left(\begin{array}{c}{a}_{1k}\\ a_{2k}\end{array}\right)=\left(\begin{array}{cccc}{t}_{11}^k& t_{12}^k& t_{13}^k& t_{14}^k\\ t_{21}^k& t_{22}^k& t_{23}^k& t_{24}^k\end{array}\right)\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\\ x_4\end{array}\right)=\left({\mathcal{T}}^{::k}\right)x\text{and}\\ {\mathrm{row}}_i\left(\mathcal{T}{\overline{\times }}_{2}x\right)=\left(\begin{array}{ccc}{a}_{i1}& a_{i2}& a_{i3}\end{array}\right)=\left(\begin{array}{cccc}{x}_1& x_2& x_3& x_4\end{array}\right)\left(\begin{array}{ccc}{t}_{i1}^1& t_{i1}^2& t_{i1}^3\\ t_{i2}^1& t_{i2}^2& t_{i2}^3\\ t_{i3}^1& t_{i3}^2& t_{i3}^3\\ t_{i4}^1& t_{i4}^2& t_{i4}^3\end{array}\right)=x^T{\mathcal{T}}^{i::}\end{array}$$

1. 3. $x\in {\mathrm{I}\mathrm{R}}^3$, then $\mathcal{T}{\overline{\times }}_{3}x\in {\mathrm{I}\mathrm{R}}^{2\times 4}$ and

$$\begin{array}{c}{\mathrm{col}}_j\left(\mathcal{T}{\overline{\times }}_{3}x\right)=\left(\begin{array}{c}{a}_{1j}\\ a_{2j}\end{array}\right)=\left(\begin{array}{ccc}{t}_{1j}^1& t_{1j}^2& t_{1j}^3\\ t_{2j}^1& t_{2j}^2& t_{2j}^3\end{array}\right)\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)=\left({\mathcal{T}}^{:j:}\right)x\text{and}\\ {\mathrm{row}}_i\left(\mathcal{T}{\overline{\times }}_{3}x\right)=\left(\begin{array}{ccc}{a}_{i1}& a_{i2}& a_{i3}\end{array}\right)=\left(\begin{array}{ccc}{x}_1& x_2& x_3\end{array}\right)\left(\begin{array}{cccc}{t}_{i1}^1& t_{i2}^1& t_{i3}^1& t_{i4}^1\\ t_{i1}^2& t_{i2}^2& t_{i3}^2& t_{i4}^2\\ t_{i1}^3& t_{i2}^3& t_{i3}^3& t_{i4}^3\end{array}\right)=x^T{\left({\mathcal{T}}^{i::}\right)}^T\end{array}$$

This example can be easily generalized to arbitrary dimensions. In particular, for a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{m\times n\times n}$ and a vector $x\in {\mathrm{I}\mathrm{R}}^n$, we have

$${\mathrm{row}}_i\left(\mathcal{T}{\overline{\times }}_{2}x\right)=x^T{\mathcal{T}}^{i::}$$E9

$${\mathrm{row}}_i\left(\mathcal{T}{\overline{\times }}_{3}x\right)=x^T{\left({\mathcal{T}}^{i::}\right)}^T$$E10

Lemma 1.5. Let $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{n\times n\times n}$ be a tensor. If ${\mathcal{T}}^{i::}$ is a symmetric matrix for all $i=1,\dots ,n$, then

$$\left(\mathcal{T}{\overline{\times }}_{2}u\right)v=\left(\mathcal{T}{\overline{\times }}_{2}v\right)u$$

for all $u,v\in {\mathrm{I}\mathrm{R}}^n$.

Proof. By Property 2, it follows that $\left(\mathcal{T}{\overline{\times }}_{2}u\right)v=\left(\mathcal{T}{\overline{\times }}_{3}v\right)u$. By (10), (11), and the symmetry of ${\mathcal{T}}^{i::}$, we have $\mathcal{T}{\overline{\times }}_{3}v=\mathcal{T}{\overline{\times }}_{2}v$.$\square$

3. Space of bilinear applications

In this section, we define bilinear applications on finite dimensional vector spaces, in order to relate them to the second derivative of a two-differentiable application, as well as a tensor of order 3.

Definition 1.6. Let $U,V,W$ be vector spaces. An application $f:U\times V\to W$ is a bilinear application if:

1. $f\left(\lambda u_1+u_2,v\right)=\mathit{\lambda f}\left(u_1,v\right)+f\left(u_2,v\right)$ for all $\lambda \in \mathrm{I}\mathrm{R}$, $u_1,u_2\in U$, and $v\in V$.

2. $f\left(u,\lambda v_1+v_2\right)=\mathit{\lambda f}\left(u,v_1\right)+f\left(u,v_2\right)$ for all $\lambda \in \mathrm{I}\mathrm{R}$, $u\in U$, and $v_1,v_2\in V$.

In other words, an application $f:U\times V\to W$ is a bilinear application if it is linear in each of the variables when the other variable is fixed. We denote by $\mathcal{B}\left(U\times V,W\right)$ the set of all bilinear applications of $U\times V$ in $W$. In particular, if $U=V$ and $W=\mathrm{I}\mathrm{R}$ in Definition 1.6, then $f:U\times U\to \mathrm{I}\mathrm{R}$ is a bilinear form in which we are used to quadratic forms, for example.

A simple example of bilinear application is the function $f:U\times V\to \mathrm{I}\mathrm{R}$ defined by

$$f\left(u,v\right)=h\left(u\right)g\left(v\right),$$E11

with $h\in U^{\ast }$ and $g\in V^{\ast }$, where $U^{\ast }$ denotes the dual space to $U$. In fact, we have for all $\lambda \in \mathrm{I}\mathrm{R},u_1,u_2\in U\text{and}v\in V$ such that

$$f\left(\lambda u_1+u_2,v\right)=h\left(\lambda u_1+u_2\right)g\left(v\right)=\left(\mathit{\lambda h}\left(u_1\right)+h\left(u_2\right)\right)g\left(v\right)=\mathit{\lambda f}\left(u_1,v\right)+f\left(u_2,v\right).$$

Similarly, it is easy to see that $f\left(u,\lambda v_1+v_2\right)=\mathit{\lambda f}\left(u,v_1\right)+f\left(u,v_2\right)$ for all $\lambda \in \mathrm{I}\mathrm{R},u\in U\text{and}{v}_1,v_2\in V$.

The next theorem ensures that a bilinear application $f:U\times V\to W$ is well defined when the image of $f$ applied in the bases elements of $U$ and $V$ is known.

Theorem 1.7. Let $U$, $V$, and $W$ be vector spaces; $\left\{u_1,\dots ,u_m\right\}$ and $\left\{v_1,\dots ,v_n\right\}$ bases of the $U$ and $V$, respectively; and $\left\{w_{\mathit{ij}}|i=1,\dots ,m\text{and}j=1,\dots ,n\right\}$ a subset of $W$. Then, there exists an only bilinear application $f:U\times V\to W$ such that $f\left(u_i,v_j\right)=w_{\mathit{ij}}$.

Proof. Let $u={\sum }_{i=1}^m{\alpha }_i{u}_i$ and $v={\sum }_{j=1}^n{\beta }_j{v}_j$ be arbitrary elements of $U$ and $V$, respectively. We defined an application $f:U\times V\to W$ by

$$f\left(u,v\right)=\sum _{i=1}^m\sum _{j=1}^n{\alpha }_i{\beta }_j{w}_{\mathit{ij}}.$$

It is easy to see that $f$ is a bilinear application and $f\left(u_i,v_j\right)=w_{\mathit{ij}}$. Such an application is unique because if $g$ is another bilinear application satisfying $g\left(u_i,v_j\right)=w_{\mathit{ij}}$, then

$$g\left(u,v\right)=g\left(\sum _{i=1}^m{\alpha }_i{u}_i,\sum _{j=1}^n{\beta }_j{v}_j\right)=\sum _{i=1}^m\sum _{j=1}^n{\alpha }_i{\beta }_{j}g\left(u_i,v_j\right)=$$E12

$$=\sum _{i=1}^m\sum _{j=1}^n{\alpha }_i{\beta }_j{w}_{\mathit{ij}}=f\left(u,v\right).$$E13

Therefore $g=f$.$\square$

The following theorem guarantees the isomorphism between space of bilinear applications and space of tensor of order $3$.

Theorem 1.8. Let $U$, $V$, and $W$ be vector spaces with dimensions $n$, $p$, and $m$, respectively. Then, the space $\mathcal{B}\left(U\times V,W\right)$ has dimension $\mathit{mnp}$.

Proof. The idea of the proof is to exhibit a basis for space $\mathcal{B}\left(U\times V,W\right)$. For this, let $\left\{w_1,\dots ,w_m\right\}$, $\left\{u_1,\dots ,u_n\right\}$, and $\left\{v_1,\dots ,v_p\right\}$ be bases of the $W$, $U$, and $V$, respectively. For each triple $\left(i,j,k\right)$, with $i=1,\dots ,m$, $j=1,\dots ,n$, and $k=1,\dots ,p$, we define a bilinear application $f_{\mathit{ij}}^k:U\times V\to W$ such that

$$f_{\mathit{ij}}^k\left(u_r,v_s\right)=\left\{\begin{array}{ll}{w}_i& \text{if}r=j\text{and}s=k\\ 0& \text{if}r\ne j\text{or}s\ne k.\end{array}\right.$$E14

Theorem 1.7 ensures the existence of the $f_{\mathit{ij}}^k$. We will then show that the set

$$A=\left\{f_{\mathit{ij}}^k|i=1,\dots ,m,j=1,\dots ,n\text{and}k=1,\dots ,p\right\}$$

is a basis of the space $\mathcal{B}\left(U\times V,W\right)$. Let $f\in \mathcal{B}\left(U\times V,W\right)$. We note in passing that

$$f\left(u_r,v_s\right)=\sum _{i=1}^m{a}_{\mathit{ir}}^s{w}_i$$E15

for all $r=1,\dots ,n$ and $s=1,\dots ,p$. Consider the bilinear application

$$g=\sum _{i=1}^m\sum _{j=1}^n\sum _{k=1}^p{a}_{\mathit{ij}}^k{f}_{\mathit{ij}}^k.$$

Our goal is to show that $g=f$. In particular, we have

$$g\left(u_r,v_s\right)=\sum _{i=1}^m\sum _{j=1}^n\sum _{k=1}^p{a}_{\mathit{ij}}^k{f}_{\mathit{ij}}^k\left(u_r,v_s\right)=\sum _{i=1}^m{a}_{\mathit{ir}}^s{w}_i=f\left(u_r,v_s\right)$$

for all $r=1,\dots ,n$ and $s=1,\dots ,p$. Therefore $g=f$. The set $A$ is linearly independent, because if

$$\sum _{i=1}^m\sum _{j=1}^n\sum _{k=1}^p{a}_{\mathit{ij}}^k{f}_{\mathit{ij}}^k=0,$$

then

$$0=\sum _{k=1}^p\sum _{i=1}^m\sum _{j=1}^n{a}_{\mathit{ij}}^k{f}_{\mathit{ij}}^k\left(u_r,v_s\right)=\sum _{i=1}^m{a}_{\mathit{ir}}^s{w}_i.$$

Since $\left\{w_1,\dots ,w_m\right\}$ is a basis of $W$, it follows that $a_{\mathit{ir}}^s=0$ for all $i=1,\dots ,m$, $r=1,\dots ,n$, and $k=1,\dots ,p$.$\square$

In particular, if the dimensions of the vector spaces $U$ and $V$ are $m$ and $n$, respectively, then the vector space $\mathcal{B}\left(U\times V,\mathrm{I}\mathrm{R}\right)$ has dimension $\mathit{mn}$. Now, as two vector spaces of the same finite dimension are isomorphic [16], there exists a matrix $m\times n$ associated with each $f\in \mathcal{B}\left(U\times V,\mathrm{I}\mathrm{R}\right)$. By considering $B=\left\{u_1,\dots ,u_m\right\}$ and $C=\left\{v_1,\dots ,v_n\right\}$ bases of $U$ and $V$, respectively, and if $u=\sum _{i=1}^m{\alpha }_i{u}_i$ and $v=\sum _{j=1}^n{\beta }_j{v}_j$, then by doing $f\left(u_i,v_j\right)=a_{\mathit{ij}}$ for all $i=1,\dots ,m$ and $j=1,\dots ,n$, we have

$$f\left(u,v\right)=\sum _{i=1}^m\sum _{j=1}^n{\alpha }_i{a}_{\mathit{ij}}{\beta }_j,$$

which in matrix form is $f\left(u,v\right)={\left[u\right]}_B^{T}A{\left[v\right]}_C$, where $A=\left(a_{\mathit{ij}}\right)$ and ${\left[v\right]}_C$ denote the vector components $v$ in the basis $C$. Hence follows the next definition:

Definition 1.9. Let $U$ and $V$ be vector spaces of finite dimension and ordered bases $B=\left\{u_1,\dots ,u_m\right\}\subset U$ and $C=\left\{v_1,\dots ,v_n\right\}\subset V$. We define, for each $f\in \mathcal{B}\left(U\times V,\mathrm{I}\mathrm{R}\right)$, the matrix $A=\left(a_{\mathit{ij}}\right)\in {\mathrm{I}\mathrm{R}}^{m\times n}$ of the $f$ relative to the ordered bases $B$ and $C$, whose elements are given by $a_{\mathit{ij}}=f\left(u_i,v_j\right)$ with $i=1,\dots ,m$ and $j=1,\dots ,n$.

Consider now the space $\mathcal{B}\left({\mathrm{I}\mathrm{R}}^m\times {\mathrm{I}\mathrm{R}}^n,{\mathrm{I}\mathrm{R}}^p\right)$ and the canonical bases $\left\{e_1,\dots ,e_m\right\}$, $\left\{{\overline{e}}_1,\dots ,{\overline{e}}_n\right\}$, $\left\{{\widehat{e}}_1,\dots ,{\widehat{e}}_p\right\}$ of the ${\mathrm{I}\mathrm{R}}^m$, ${\mathrm{I}\mathrm{R}}^n$, and ${\mathrm{I}\mathrm{R}}^p$, respectively. Consider $f\in \mathcal{B}\left({\mathrm{I}\mathrm{R}}^m\times {\mathrm{I}\mathrm{R}}^n,{\mathrm{I}\mathrm{R}}^p\right)$. For all $u\in {\mathrm{I}\mathrm{R}}^m$ and $v\in {\mathrm{I}\mathrm{R}}^n$, we have

$$f\left(u,v\right)=\sum _{j=1}^m\sum _{k=1}^n{u}_j{v}_{k}f\left(e_j,{\overline{e}}_k\right)$$

where $u_j$ and $v_k$ are the components of the $u$ and $v$ in the canonical bases of ${\mathrm{I}\mathrm{R}}^m$ and ${\mathrm{I}\mathrm{R}}^n$, respectively. Denote the i-th component of the $f$ by $f_i$. Note that $f_i\in \mathcal{B}\left({\mathrm{I}\mathrm{R}}^m\times {\mathrm{I}\mathrm{R}}^n,\mathrm{I}\mathrm{R}\right)$. So for each $i=1,\dots ,p$, we have

$$f_i\left(u,v\right)=\sum _{j=1}^m\sum _{k=1}^n{u}_j{v}_k{f}_i\left(e_j,{\overline{e}}_k\right).$$

By Definition 1.9, the matrix of the $f_i$ in relation to the canonical bases is the matrix

$$A_i=\left(t_{\mathit{ij}}^k\right)\in {\mathrm{I}\mathrm{R}}^{m\times n},$$

where $t_{\mathit{ij}}^k=f_i\left(e_j,{\overline{e}}_k\right)$. So, we can write

$$f_i\left(u,v\right)=u^T{A}_{i}v.$$

In general, we can define $p$ matrices $m\times n$ as a tensor $\mathcal{T}\in {\mathrm{I}\mathrm{R}}^{p\times m\times n}$; this means that the $p$ matrices can be seen as the horizontal slices of the tensor $\mathcal{T}$. We note in passing that we can write $f\left(u,v\right)$ as a product between tensor $\mathcal{T}$ and vectors $u$ and $v$, that is,

$$f\left(u,v\right)=\left(\begin{array}{c}{u}^T{A}_{1}v\\ u^T{A}_{2}v\\ ⋮\\ u^T{A}_{p}v\\ \end{array}\right)=\left(\mathcal{T}{\overline{\times }}_{2}u\right)v.$$E16

Thus, we can generalize Definition 1.9 as follows:

Definition 1.10. Let $U$ and $V$ be finite dimension vector spaces. For fixed bases $B=\left\{u_1,\dots ,u_m\right\}$ and $C=\left\{v_1,\dots ,v_n\right\}$ of the $U$ and $V$, respectively, we define, for each $f\in \mathcal{B}\left(U\times V,{\mathrm{I}\mathrm{R}}^p\right)$, the tensor $\mathcal{T}=\left(t_{\mathit{ij}}^k\right)\in {\mathrm{I}\mathrm{R}}^{p\times m\times n}$ of the $f$ relative to the ordered bases $B$ and $C$, whose elements are given by $t_{\mathit{ij}}^k=f_i\left(u_j,v_k\right)$ where $f_i$ is the i-th component of the $f$, that is, $f_i\in \mathcal{B}\left(U\times V,\mathrm{I}\mathrm{R}\right)$, with $i=1,\dots ,p$, $j=1,\dots ,m$, and $k=1,\dots ,n$.

4. Differentiability

Let $U$ be an open subset of ${\mathrm{I}\mathrm{R}}^m$ and $F:U\subset {\mathrm{I}\mathrm{R}}^m\to {\mathrm{I}\mathrm{R}}^n$ a differentiable application throughout $U$ and $a\in U$. Denote $\mathcal{L}$$\left({\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right)$ the set of all linear applications of ${\mathrm{I}\mathrm{R}}^m$ in ${\mathrm{I}\mathrm{R}}^n$. When $F^{\prime }:U\subset {\mathrm{I}\mathrm{R}}^m\to \mathcal{L}$$\left({\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right)$ is differentiable in $a\in U$, we say that the application $F$ is twice differentiable in $a\in U$ and then the linear transformation ${F^{\prime }}^{\prime }\left(a\right)\in \mathcal{L}({\mathrm{I}\mathrm{R}}^m,\mathcal{L}$$\left({\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right))$ is the second derivative of $F$ in $a\in U$.

The norm of ${F^{\prime }}^{\prime }\left(a\right)$ is naturally defined. For any $h\in {\mathrm{I}\mathrm{R}}^m$, it follows that

$$‖{F^{\prime }}^{\prime }\left(a\right)h‖=\underset{‖k‖=1}{sup}\left\{‖{F^{\prime }}^{\prime }\left(a\right)\mathit{hk}‖\mathrm{com}k\in {\mathrm{I}\mathrm{R}}^m\right\}$$

and then

$$‖{F^{\prime }}^{\prime }\left(a\right)‖=\underset{‖h‖=1}{sup}‖{F^{\prime }}^{\prime }\left(a\right)h‖=\underset{‖h‖=1}{sup}\underset{‖k‖=1}{sup}‖{F^{\prime }}^{\prime }\left(a\right)\mathit{hk}‖.$$

An important observation with respect to Theorem 1.8 is that the spaces $\mathcal{L}({\mathrm{I}\mathrm{R}}^m,\mathcal{L}$$\left({\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right))$ and $\mathcal{B}\left({\mathrm{I}\mathrm{R}}^m\times {\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right)$ are isomorphic. This means that ${F^{\prime }}^{\prime }\left(a\right)$ is a bilinear application belonging to space $\mathcal{B}\left({\mathrm{I}\mathrm{R}}^m\times {\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right)$. Such isomorphism can be found in classical analysis books [17, 18]. On the other hand, by the same theorem, the space of bilinear applications $\mathcal{B}\left({\mathrm{I}\mathrm{R}}^m\times {\mathrm{I}\mathrm{R}}^m,{\mathrm{I}\mathrm{R}}^n\right)$ and space of tensor ${\mathrm{I}\mathrm{R}}^{n\times m\times m}$ are also isomorphic.

In many practical applications, such as algorithm implementations, the second derivative ${F^{\prime }}^{\prime }\left(a\right)$ may be implemented as a tensor belonging to space ${\mathrm{I}\mathrm{R}}^{n\times m\times m}$. The question now is how the tensor elements are formed. For this, consider the application $A:\mathrm{I}\mathrm{R}\to {\mathrm{I}\mathrm{R}}^{n\times m}$ and $\alpha \in \mathrm{I}\mathrm{R}$. We have $A\left(\alpha \right)$ as a matrix with $n$ rows and $m$ columns. Its elements are denoted by $a_{\mathit{ij}}\left(\alpha \right)$ where $a_{\mathit{ij}}$ are components functions of $A$ with $i=1,\dots ,n$ and $j=1,\dots ,m$. Case $a_{\mathit{ij}}:\mathrm{I}\mathrm{R}\to \mathrm{I}\mathrm{R}$ is differentiable in $\alpha$ for all $i=1,\dots ,n$ and $j=1,\dots ,m$; the derivative of $A$ in $\alpha$ is the matrix

$$A^{\prime }\left(\alpha \right)=\left(a_{\mathit{ij}}^{\prime }\left(\alpha \right)\right)\in {\mathrm{I}\mathrm{R}}^{n\times m}.$$E17

The definition of the derivative of $A\left(\alpha \right)$(17) is a classical definition. We refer to [19] for details.

In the sense of generalizing (17), consider now $A:U\subset {\mathrm{I}\mathrm{R}}^p\to {\mathrm{I}\mathrm{R}}^{n\times m}$ a differentiable application in $u\in U$ with component function $a_{\mathit{ij}}:{\mathrm{I}\mathrm{R}}^p\to \mathrm{I}\mathrm{R}$ with $i=1,\dots ,n$ and $j=1,\dots ,m$. When $a_{\mathit{ij}}$ is differentiable in $u$ for all $i=1,\dots ,n$ and $j=1,\dots ,m$, we defined the derivative of $A$ in $u$ as the tensor

$$A^{\prime }\left(u\right)=\left(\nabla a_{\mathit{ij}}\left(u\right)\right)\in {\mathrm{I}\mathrm{R}}^{n\times m\times p}.$$E18

Note that in fact (18) is a generalization of (17). With fixed $i$ and $j$, $\nabla a_{\mathit{ij}}\left(u\right)$ is a tube fiber of the tensor $A^{\prime }\left(u\right)$, whose elements are

$$A^{\prime }{\left(u\right)}_{\mathit{ij}}^k=\frac{\partial a_{\mathit{ij}}}{\partial x_k}\left(u\right)$$E19

for all $k=1,\dots ,p$.

For example, consider an application $F:U\subset {\mathrm{I}\mathrm{R}}^2\to {\mathrm{I}\mathrm{R}}^3$ twice differentiable in $a\in U$ where $U$ is an open set. The Jacobian matrix of $F$ in $a$ is given by

$$J_F\left(a\right)=\left(\begin{array}{c}\nabla f_1{\left(a\right)}^T\\ \nabla f_2{\left(a\right)}^T\\ \nabla f_3{\left(a\right)}^T\end{array}\right)=\left(\begin{array}{cc}\frac{\partial f_1}{\partial x_1}\left(a\right)& \frac{\partial f_1}{\partial x_2}\left(a\right)\\ \frac{\partial f_2}{\partial x_1}\left(a\right)& \frac{\partial f_2}{\partial x_2}\left(a\right)\\ \frac{\partial f_3}{\partial x_1}\left(a\right)& \frac{\partial f_3}{\partial x_2}\left(a\right)\end{array}\right)$$

and its derivative is, by (18), the tensor

$$J_{F^{\prime }}\left(a\right)={\mathcal{T}}_F\left(a\right)=\left(\nabla \frac{\partial f_i}{\partial x_j}\left(a\right)\right)\in {\mathrm{I}\mathrm{R}}^{3\times 2\times 2}$$E20

where, by (19), its elements are described as

$$t_{\mathit{ij}}^k=\frac{{\partial }^2{f}_i}{\partial x_k\partial x_j}\left(a\right).$$

With fixed $i$, it is easy to see that the i-th horizontal slice of the ${\mathcal{T}}_F\left(a\right)$ is the Hessian matrix ${\nabla }^2{f}_i\left(a\right)$ defined by

$${\nabla }^2{f}_i\left(a\right)={\mathcal{T}}_F{\left(a\right)}^{i::}=\left(\begin{array}{cc}\frac{{\partial }^2{f}_i}{\partial x_1\partial x_1}\left(a\right)& \frac{{\partial }^2{f}_i}{\partial x_1\partial x_2}\left(a\right)\\ \frac{{\partial }^2{f}_i}{\partial x_2\partial x_1}\left(a\right)& \frac{{\partial }^2{f}_i}{\partial x_2\partial x_2}\left(a\right)\end{array}\right).$$E21

We note in passing that any column of the matrix ${\nabla }^2{f}_i\left(x\right)$ is a row fiber of the i-th horizontal slice.

As mentioned in the introduction, some numerical methods need to calculate the product between tensor ${\mathcal{T}}_F\left(a\right)$ and vectors in ${\mathrm{I}\mathrm{R}}^2$.

From Definition 1.4, it is possible to calculate the contracted product mode-$2$ and mode-$3$. As Hessian matrices are symmetrical, given $v\in {\mathrm{I}\mathrm{R}}^2$, by Lemma 1.5 together with (10) and (11), we have

$${\mathcal{T}}_F\left(a\right){\overline{\times }}_{3}v={\mathcal{T}}_F\left(a\right){\overline{\times }}_{2}v=\left(\begin{array}{c}{\mathrm{row}}_1\left({\mathcal{T}}_F\left(a\right){\overline{\times }}_{2}v\right)\\ {\mathrm{row}}_2\left({\mathcal{T}}_F\left(a\right){\overline{\times }}_{2}v\right)\\ {\mathrm{row}}_3\left({\mathcal{T}}_F\left(a\right){\overline{\times }}_{2}v\right)\end{array}\right)=\left(\begin{array}{c}{v}^T{\nabla }^2{f}_1\left(a\right)\\ v^T{\nabla }^2{f}_2\left(a\right)\\ v^T{\nabla }^2{f}_3\left(a\right)\end{array}\right)\in {\mathrm{I}\mathrm{R}}^{3\times 2}$$

and consequently it follows that

$$\left({\mathcal{T}}_F\left(a\right){\overline{\times }}_{2}v\right)u=\left(\begin{array}{c}{v}^T{\nabla }^2{f}_1\left(a\right)u\\ v^T{\nabla }^2{f}_2\left(a\right)u\\ v^T{\nabla }^2{f}_3\left(a\right)u\end{array}\right)\in {\mathrm{I}\mathrm{R}}^3$$E22

for all $u,v\in {\mathrm{I}\mathrm{R}}^2$.

This means that the tensor ${\mathcal{T}}_F\left(a\right)$ defined by (20) is the associate tensor to bilinear application ${F^{\prime }}^{\prime }\left(a\right)$, in relation to canonical basis of ${\mathrm{I}\mathrm{R}}^2$, by means of Definition 1.10. Without loss of generality, we have

$${\mathcal{T}}_F\left(a\right){\overline{\times }}_{3}v={\mathcal{T}}_F\left(a\right){\overline{\times }}_{2}v={\mathcal{T}}_F\left(a\right)v$$

and by means of Lemma 1.5, it follows that

$$\left({\mathcal{T}}_F\left(a\right)u\right)v=\left({\mathcal{T}}_F\left(a\right)v\right)u={\mathcal{T}}_F\left(a\right)\mathit{vu}.$$

To finish, we consider the following particular case. We know that the k-th column of Jacobian $J_F\left(x\right)$ is equal to product $J_F\left(x\right)e_k$, where $e_k$ is the k-th canonical vector of ${\mathrm{I}\mathrm{R}}^n$. It is worth noting what the slice of the matrix ${\mathcal{T}}_F\left(x\right)e_k$ is. By definition, we have

$${\mathcal{T}}_F\left(x\right)e_k=\left(\begin{array}{c}{e}_k^T{\nabla }^2{f}_1\left(x\right)\\ e_k^T{\nabla }^2{f}_2\left(x\right)\\ ⋮\\ e_k^T{\nabla }^2{f}_n\left(x\right)\end{array}\right)=\left(\begin{array}{c}{\mathrm{row}}_k{\nabla }^2{f}_1\left(x\right)\\ {\mathrm{row}}_k{\nabla }^2{f}_2\left(x\right)\\ ⋮\\ {\mathrm{row}}_k{\nabla }^2{f}_n\left(x\right)\end{array}\right)$$

Given that ${\mathrm{row}}_k{\nabla }^2{f}_i\left(x\right)$ is the k-th tube fiber of i-th horizontal slice, we have ${\mathcal{T}}_F\left(x\right)e_k$ as the k-th lateral slice, or, by symmetry of Hessians, it is the transpose of k-th frontal slice. In short, for the twice differentiable application $F:U\subset {\mathrm{I}\mathrm{R}}^n\to {\mathrm{I}\mathrm{R}}^m$, we have ${\mathcal{T}}_F\left(x\right)\in {\mathrm{I}\mathrm{R}}^{m\times n\times n}$ where the $m$ horizontal slices are the Hessians ${\nabla }^2{f}_i\left(x\right)$, with $i=1,\dots ,m$ and the $n$ lateral and frontal slices obtained by the following product ${\mathcal{T}}_F\left(x\right)e_k$, with $k=1,\dots ,n$.

5. Conclusions

In this text, we have shown some properties of tensors, in particular those of order $3$. In addition, we have approached bilinear applications, and we have shown the isomorphism between space of bilinear applications and of tensor of order $3$. As mentioned in the introduction, to solve a nonlinear system, some numerical methods use tensors, either in the iterative scheme or in the proof of theorems. For this reason, we have written a section on differentiability of applications by showing how to calculate the product between tensor of second derivatives and vectors.