Object Tracking Using Adapted Optical Flow

Abstract

The objective of this work is to present an object tracking algorithm developed from the combination of random tree techniques and optical flow adapted in terms of Gaussian curvature. This allows you to define a minimum surface limited by the contour of a two-dimensional image, which must or should not contain a minimum amount of optical flow vector associated with the movement of an object. The random tree will have the purpose of verifying the existence of superfluous vectors of optical flow by discarding them, defining a minimum number of vectors that characterizes the movement of the object. The results obtained were compared with those of the Lucas-Kanade algorithms with and without Gaussian filter, Horn and Schunk and Farneback. The items evaluated were precision and processing time, which made it possible to validate the results, despite the distinct nature between the algorithms. They were like those obtained in Lucas and Kanade with or without Gaussian filter, the Horn and Schunk, and better in relation to Farneback. This work allows analyzing the optical flow over small regions in an optimal way in relation to precision (and computational cost), enabling its application to area, such as cardiology, in the prediction of infarction.

1. Introduction

Object tracking is defined as a problem of estimating the object’s trajectory, done by means of a video image. There are several tools for tracking objects and are used in various fields of research, such as computer vision, digital video processing, and autonomous vehicle navigation [1]. With the emergence of high-performance computers, high-resolution cameras, and the growing use of so-called autonomous systems that, in addition to these items, require specialized tracking algorithms, increasingly accurate and robust for automatic video analysis, has currently been the target of numerous research on the development of new object tracking techniques [2, 3].

Object tracking techniques are applicable to motion-based reconnaissance cases [4], automatic surveillance systems [5], pedestrian flow monitoring in crosswalks [6], traffic control [7], and autonomous vehicular navigation [8]. Problems of this type are highly complex due to the characteristics of the object and the environment, generating many variables, which impairs performance and makes the application of tracking algorithms unfeasible to real-world situations. Some approaches seek to resolve this impasse by simplifying the problem, reducing the number of variables [9]. This process, in most cases, does not generate good results [10, 11], making it even more difficult to identify the main attributes to be selected to perform a task [12, 13].

Most of the object tracking problems occur in open environments, so-called uncontrolled [14]. The complexity of these problems has attracted the interest of the scientific community and generated numerous applied research in various fields of research. Current approaches, such as the ones that use convolutional neural networks—CNN, deal well with the high number of variables of these types of problems, providing space–temporal information of the tracked objects, through three-dimensional convolutions [15, 16, 17]. This ends up creating an enormous number of learnable parameters, which ends up generating an overfitting [11]. A solution to reduce this number of learnable parameters was combining space–time data, extracted using the optical flow algorithm, used in the Two-Stream technique [18, 19, 20]. However, this technique presents good results only for large datasets, showing itself to be inefficient for small datasets [15, 21].

In recent years, research using machine learning has been applied to tracking problems, gaining notoriety due to the excellent results obtained in complex environments and attribute extraction [21, 22, 23]. Deep learning stands out among these techniques for presenting excellent results to unsupervised learning problems, [24], object identification [25], semantic segmentation [26]. Random trees are also examples of machine learning techniques, and their excellent results, due to their precision and great capacity to handle a large volume of data and low overfitting tendency [27, 28], and widely used in research areas such as medicine, in the prediction of hereditary diseases [29], agriculture to increase the productivity of a given plantation crop and in astronomy, acting on the improvement of images captured by telescopes, in the spectrum electromagnetic radiation not visible to the human eye [30]. The possibilities of applications, and new trends and research related to machine learning techniques, with particular attention to random trees, allow the development of algorithms that can be combined with existing ones, in the case of optical flow algorithms, (belonging to computational field of view) taken advantage of in this way, the advantages of each [31, 32, 33].

Developing an algorithm whose objective is to track objects, using the particular advantages of these techniques in a combined way, justifies creating a tracking algorithm that combines the optical flow technique, adapted in this work in terms of the Gaussian curvature associated with a minimal surface, with a random trees waiting for it to capture on this surface a minimum number of optical flow vectors that characterize the moving object, accurately and with low computational cost, contributing not only in the fields of computational vision but in other branches of science, such as in medicine, it can help in the early identification of infarctions.

2. Related works

Due to the large number of studies related to the technique of object tracking, only a small number surrounding this theme will be addressed. The focus of this project is not to make a thorough study on the state of the art. With this in this item, the main works in the literature, associated with the tracking of objects, will be presented. Among the various approaches used for this context, we highlight those focused on the techniques of optical flow, and others belonging to machine learning, such as those that use identifications of patterns, which allow relating, framing, and justifying the development of this proposal and its importance, through its contribution, to the state of the art.

2.1 Object tracking

Object tracking is defined as a process that allows you to uniquely estimate and associate the movements of objects with consecutive image frames. The objects considered can be from one, the set of pixels belonging to a region of the image. The detection of pixels is done by a motion detector or objects, which allows to locate objects with similar characteristics that move, between consecutive frames.

These characteristics of the object to be tracked are compared with the characteristics of a reference object modeled by a classifier over a limited region of the so-called region of interest frame, where the probability of detection of the object is greater. Thus, according to [33], the detector of traced objects, locate several objects on the different parts of the region of interest and performs the comparison of these objects with the reference object. This process is performed for each frame and each object detected, candidate to be recognized as the greatest possible similarity, to the reference object can be represented, through a set of fixed-size characteristics, extracted from this region containing a set of pixels, which can be represented by a numerical array of data.

Thus, mathematically, the region containing a set of pixels belonging to the regions of the object of interest, where the characteristics that allow to test whether the region of the frame, in which the object to be traced is, is given by:

$${\mathrm{OC}}_{\mathrm{i}}\left(\mathrm{t}\right)=\mathrm{OC}\left(\mathrm{t}\right)\mid \parallel \mathrm{L}\left(\mathrm{OC}\left(\mathrm{t}\right)\right)-\mathrm{L}\left(\text{OR}\left(\mathrm{i}-1\right)\right)\parallel <\mathrm{\epsilon }$$E1

where, $L\left(\mathit{OC}\left(t\right)\right)$ is the position $\left(x,y\right)$ of the centroid of the candidate object $\mathit{OC}\left(t\right)$, $L\left(\mathit{\text{OR}}\left(i-1\right)\right)$, is the position of the object traced to the $\left(i–1\right)$—frame of the video and $0<\epsilon \in \mathbb{R}$, is an actual value associated with the size of the region of the object of interest.

According to the works of [34, 35], learning methods are used to adapt the changes of movement and other characteristics such as geometric aspect and appearance of the tracked object. These methods are usually used adaptive tracked object trackers and detectors. The following will be presented other types of object trackers, found in the literature.

According to [36], a classifier can be defined with a $f$ belonging to a family of functions $F$ parameterized by a set of classifier parameters. They form a detector of objects to be tracked which in turn is an integral part of a tracker. A classifier can also be training and thereby generate a set of classification parameters, producing the function $f$, that allows you to efficiently indicate the classes $v_i$ of the test data $x_i$ from a training set $C_t=\{\left(x_1,y_1\right),\dots ,\left(x_n,y_n\right)$}. The data is points in the space of the characteristics, which can be entropy, the gray level, among others.

The classifier aims to determine the best way to discriminate the data classes, on the space of characteristics. The test data form a set containing the characteristics of the candidate objects, which have not yet been classified. The position of the object to be tracked in the frame is defined as the position corresponding to the highest response of the detector of the object to be tracked on the $\mathit{ith}$-candidate objects. Therefore, the position of the object to be tracked is determined by the position of the $\mathit{ith}$-candidate object, which is most likely to belong to the class of the crawled object, given by the following equation:

$$\mathrm{L}\left(\text{OR}\left(\mathrm{t}\right)\right)=\mathrm{L}\left(\text{argmaxiP}\left(\mathrm{yi}=\mathrm{COR}|\mathrm{OCi}\left(\mathrm{t}\right),\mathrm{PR}\left(\mathrm{t}\right)\right)\right)$$E2

$$P\left(\mathit{yi}=\mathit{COR}|\mathit{OCi}\left(t\right),\mathit{PR}\left(t\right)\right)=\left\{\begin{array}{c}1/\mathrm{N},\mathrm{se}\parallel \mathrm{L}\left(\mathrm{OCi}\left(\mathrm{t}\right)\right)-\mathrm{L}\left(\text{OR}\left(\mathrm{t}|\mathrm{t}-1\right)\right)\parallel <\mathrm{\epsilon }\\ 0,\text{otherwise}\end{array}\right.$$E3

where the variable $\mathit{COR}$ in the equation (3), are the classes of the tracked objects and the candidate objects $\mathit{OC}$, all with equiprobability of occurrence. According to the types of classifiers used object detectors to be tracked, along with the initial detector, it is possible to use some learning technique to train them. One of the ways used is offline training [36] and adjusting the parameters of the classifier before running the tracker.

Offline-trained classifiers are generally employed in object detectors designed to detect all new objects of interest that enter the camera’s field of view [37]. The training set $C_t$, must contain characteristics $x_i$ extracted from the objects to be traced and diverse environmental characteristics. This allows new objects, with varied geometric characteristics and aspects, to be detected more efficiently. As for online training, the adjustment of the parameters of the classified is performed during the tracking process. For online trained classifiers, they are generally used in object detectors to be tracked. Thus, in each frame, the new extracted characteristics are used to adjust the classifiers.

2.1.1 Binary classification

In [38], trackers that use the detection tracking technique deal with object tracking, as a binary classification problem whose goal is to find the best function $f$ that separates the objects to be tracked R, of other objects in the environment. Object tracking seen as a binary classification problem, which is currently one of the subjects that receives the most attention in research in computing vision.

In [39], were developed trackers that used detectors of objects to be tracked, formed by classifiers in committee formed by binary classifiers said weak. For [40], a binary classifier is defined as a classifier, used in problems where the class $y_i$ of a ${\mathit{OC}}_i$ belongs to the set $Y=\left\{-1,1\right\}$. The negative class {−1} refers to the characteristics of the environment and other objects. A positive class {+1} refers to the class of the object to be tracked.

A classifier is said to be weak, when it has a probability of “hitting” a given data class, only slightly higher than a random classifier. The detector of the object to be tracked must separate the crawled object from the other objects and the environment. Its purpose and determine the position of the tracked object, according to the equations (1)–(3)$.$ According to [41, 42] each of the $\mathit{ith}$candidate object classes ${\mathit{OC}}_i$, it is defined according to Bayesian theory of decision, through minimal classification error. This means that, the decision given by observing, the sign of the difference between $P\left(y_i=\mathit{COR}|{\mathit{OC}}_i\left(t\right),\mathit{PR}\left(t\right)\right)$ and $P\left(y_i=\mathit{\text{CNOR}}|{\mathit{OC}}_i\left(t\right),\mathit{PR}\left(t\right)\right)$, so that the sum of these probabilities is unitary.

2.1.2 Monitoring systems

For [43], the term monitoring system, refers to the process of monitoring and autonomous control, without human intervention. This type of system has the function of detecting, classifying, tracking, analyzing, and interpreting the behavior of objects of interest. In [44, 45], this technique was used combined with statistical techniques for controlling people’s access to a specific location. It was also observed the use of intelligent monitoring systems, applied to building, port, or ship security [46, 47].

The functions comprised by a monitoring system are so-called low- and high-level tasks. Among some high-level tasks, we highlight the analysis, interpretation and description of behavior, the recognition of gestures, and the decision between the occurrence or not of a threat. Performing high-level tasks require that for each frame, the system needs to perform low-level tasks, which involve direct manipulation of the image pixels [48, 49, 50, 51, 52, 53, 54, 55, 56]. As an example, we highlight the processes of noise elimination, detection of connected components, and obtain information on the location and geometric aspect of the object of interest.

A monitoring system consists of five main components, which are presented in Figures 9. Some monitoring systems may not contain all components. The initial detector aims to detect the pixel regions of each frame that have a significant probability of containing an object to be tracked. This detector can be formed by a motion detector that detects all moving objects based on models of objects previously recorded in a database or based on characteristics extracted offline [40, 41]. The information obtained by the initial detector is processed by an image processor], which will have the function of eliminating noise, segmenting, and detecting the connected components.

The regions containing the most relevant pixels are analyzed and then classified as objects of interest by the classifier [50, 51, 52, 53, 54]. Objects of interest are modeled and are now called reference objects so that the tracker determines its position frame by frame [55, 56]. The information obtained by the initial detector is processed by an image processor], which will have the function of eliminating noise, segmenting, and detecting the connected components.

A tracker, an integral part of a detector, is defined as a function that allows estimating the position of objects at each consecutive frame, through and defines the region of the object of interest, for each $\mathrm{ith}$ object being tracked within a region of interest. This estimation of the movement is performed through the correct association of the captured and tracked objects, to consecutive video frames. The trace often and interpreted as a data binding problem. Figure 1 shows a schematic of the main components of a monitoring system.

Figure 1.

Main component of a monitoring.

2.2 State of the art in object tracking with optical flow

Several techniques that allow the calculation to have been developed in recent years to calculate the optical flow vector [57]. These methods are grouped according to their main characteristics and the approach used for the calculation of the optical flow. Thus, the differential methods performed in the studies in [56], the methods d and calculation of the optical flow through the frequency domain [46] the phase correlation methods [58], and the method of association between regions [59].

The method proposed in [56], allows the calculation of the optical flow for each point around a neighborhood of pixels. In [60], it is also considered a neighborhood of pixels, but in this case, the calculation of the optical flow is performed geometrically. In the work presented by [61] it is adding of the restrictions of regularization. In [62] turn active compare performance analyses were performed between the various algorithms and optical flow present in the literature.

This technique is considered robust for detaining and tracking moving objects from your images, both those captured by fixed or mobile cameras. This gives this technique, but high computational cost makes most practical applications unfeasible. Thus, to reduce this complexity, techniques of increasing resolutions were adopted in [63]. Also, for the same purpose, we used the techniques of subsampling on some of the pixels belonging to the object of interest to obtain optical flow [52].

Other authors also use a point of interest detector to select the best pixels for tracking and calculate the optical flow on these points [52, 64]. The reduction in the number of points to be tracked is associated with a decrease in computational complexity, so in [52] the points of interest were selected using the FAST algorithm [64].

The method developed by Lucas-Kanade [56], it is a differential method and widely used in the literature and having variations modifications. It allows you to estimate the optical flow for each point $\left(x,y,t\right)$ calculating the like transformation $(\mathit{TA}\left(x,t\right)$, applied to the pixels of a pixel grid, with center in $\left(x,y\right)$ by the following function $f\left(x,t\right),$ that is:

$$f\left(x,t\right)=min\left(\sum _{x\in \left(\mathit{\text{pixel grid}}\right)}\left[Q\left(x,t-1\right)-\left(Q\left(\mathit{TA}\left(x\right),t\right)\right]\ast g\left(x\right)\right]\right)$$E4

where $g\left(x\right)$ is a Gaussian smoothing filter centered on $x$.

New variations of the techniques were being proposed to make the calculation of the optical flow faster and faster. In [65] a tracker was proposed based on the algorithm of [56]. The translation of a point represented by a grid of rectangular sized pixels 25 × 25, was calculated and its validity is evaluated by calculating the SSD¹ in the grid pixels in $Q\left(t\right)$ and in $Q\left(t-1\right)$. If the SSD is high, the point is dropped and stops being traced.

In [51] objects were detected by subtracting the image from the environment and removed the movement of the camera with the calculation algorithm of the optical flow vector proposed by [56]. In the studies carried out in [66, 67], they showed that the reliability of the estimated optical flow reduced the case of some points of the object of interest whose optical flow cannot be represented by the same matrix given by the related transformation $\left(\mathit{TA}\left(x,t\right)\right)$ of the other points. Thus, to improve the robustness of the algorithm of [56, 67] proposed a calculation of the independent optical flow vector for each of the $N$ points belonging to the object of interest selected with the SURF (Speeded Up Robust Features) point detector in the initial frame.

In [67] they also modified Lucas - Kanade’s algorithm [56] by inserting the Hessian matrix in the calculation of the value of the variation of the related transformation $\left(\mathit{\Delta TA}\left(x,t\right)\right)$. The algorithm allows for more effective tracking when partial occlusions, deformations, and changes in lighting occur, as optical flow is not calculated considering all points of objects of interest.

Already in the proposal presented in [68] was the development of algorithm to detect people in infrared images that combines the information of the value of pixels with a method of motion detection. The algorithm forms a relevant pixel map by applying thresholding segmentation. While the camera is still, an image $M$ is built with the differentiation between frames. If the camera is in motion, $M$ is filled with the pixels obtained by the analysis of the moment of the optical flow calculated by the algorithm of [56]. The map of relevant pixels is replaced by the union between $M$ and the Pixel Map relevant to the first case and the second an interception between $M$ and the pixel map relevant case to compensate for the movement of the camera.

The method for tracking swimmers presented in [46], uses the information of the movement pattern by the optical flow and the appearance of the water that is modeled by a MoG.² This allows you to calculate an optical flow vector for each pixel of the video independently of the other, through $B$ which is an array composed of gradients in the directions $x$and $y$pixels in a grid of pixels.

In [69], a method was presented that incorporated physical restrictions to the calculation of optical flow. The tracker uses the constraints to extract the moving pixels with a lower failure rate. The calculation can be impaired when occlusions occur or when the environment has low light. The operator defines the physical constraints and selects the points of the $\mathit{\text{OR}}$ that are tracked by optical flow. Constraints can be geometric, kinematic, dynamic, of the property of the material that makes up the $\mathit{\text{OR}}$ or any other type of restriction.

In [70], the points that are tracked with the optical flow are defined by applying the Canny edge detector on the pixels of the reference pixel map. Pixels that produce a high response to the Canny detector are the selected points.

In [43], optical flow is used as a characteristic for tracking the contour of the object. The contour is shifted in small steps until the position in which the optical flow vectors are homogeneous is found.

In [64], they performed an estimate of the translation and orientation of the reference object by calculating the optical flow of the pixels belonging to its silhouette. The coordinates of the centroid position are defined by minimizing the Hausdorff distance between the mean of the optical flow vectors of the reference object and the candidate object to be chosen as the object of interest.

2.2.1 Optical flow as a function of Gaussian curvature

Optical flow is defined as a dense vector field associated with the movement and apparent velocity of an object, given by the translation of pixels from consecutive frames in an image region. It can be calculated from the brightness restriction, considered constant, from the corresponding pixels in consecutive frames.

Mathematically be a pixel $\left(x,y\right)$, associated with a luminous intensity $I\left(x,y\right)$, over an image surface or plane, and a time interval and a sequence of frames associated with an apparent offset of the pixel over that image surface or plane. Thus, the rate of variation of light intensity in relation to a time interval, associated with the apparent movement of the pixel, on a surface or plane of the image, being considered practically null can be given by:

$$\frac{\mathit{dI}\left(x,y\right)}{\mathit{dt}}=\frac{\partial I\left(x,y\right)}{\partial x}\frac{dx}{\mathit{dt}}+\frac{\partial I\left(x,y\right)}{\partial y}\frac{dy}{\mathit{dt}}+\frac{\partial I\left(u,v\right)}{\mathit{dt}}$$E5

$$=I_x{u}_x+I_y{v}_x+I_t$$E6

$$\frac{\mathit{dI}\left(x,y\right)}{\mathit{dt}}=0⟹I_x{u}_x+I_y{v}_x+I_t=0$$E7

So that equation (7) is called optical flow restriction and where the terms $I_x$, $I_y$, $I_t$ denote the derivatives relative to the brightness intensity relative to the coordinates $x,y$ and time $t$, and $u$ and $v$, $\left(u\left(x,y\right),v\left(x,y\right)\right)$ are the horizontal and vertical components of a vector representing the optical flow field, for the pixel $\left(x,y\right)$ in question.

The number of variables in equation (6) is greater than that of equations, which does not allow estimating components and vector, and determining a single solution for the optical flow restriction equation. With this, Lucas and Kanade proposed a solution to solve this problem. The solution method proposed by them considers the constant flow in a region formed by a set of pixels $N\times N$, so you can write the optical flow restriction equation for each pixel in this region, thus obtaining a systems of equations with 2 variables, that is:

$$\begin{array}{c}{\mathrm{I}}_{\mathrm{x}1}{\mathrm{v}}_{\mathrm{x}}+{\mathrm{I}}_{\mathrm{y}1}{\mathrm{v}}_{\mathrm{y}}+{\mathrm{I}}_{\mathrm{t}1}=0\\ {\mathrm{I}}_{\mathrm{x}2}{\mathrm{v}}_{\mathrm{x}}+{\mathrm{I}}_{\mathrm{y}2}{\mathrm{v}}_{\mathrm{y}}+{\mathrm{I}}_{\mathrm{t}2}=0\\ \begin{array}{c}\hfill ⋮\hfill \\ {\mathrm{I}}_{\mathrm{xp}}{\mathrm{v}}_{\mathrm{x}}+{\mathrm{I}}_{\mathrm{yp}}{\mathrm{v}}_{\mathrm{y}}+{\mathrm{I}}_{\mathrm{tp}}=0\end{array}\end{array}$$E8

Passing the set of equations given by equation (8) to the matrix form we have:

$$\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)\left(\begin{array}{c}{\mathrm{v}}_{\mathrm{x}}\\ {\mathrm{v}}_{\mathrm{y}}\end{array}\right)+\left(\begin{array}{c}{\mathrm{I}}_{\mathrm{t}1}\\ ⋮\\ {\mathrm{I}}_{\mathrm{tp}}\end{array}\right)=\left(\begin{array}{c}0\\ ⋮\\ 0\end{array}\right)$$E9

Using the least squares method, in the system of equations (9) in the form of matricial, the same can be solved. Therefore, the optical flow $v=\left(v_x,v_y\right)$ can be estimated for a particular region or window with $N\times N$ pixels, that is:

$$\begin{array}{c}\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right){\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)}^{\mathrm{t}}\left(\begin{array}{c}{\mathrm{v}}_{\mathrm{x}}\\ {\mathrm{v}}_{\mathrm{y}}\end{array}\right)=-\left(\begin{array}{c}{\mathrm{I}}_{\mathrm{t}1}\\ ⋮\\ {\mathrm{I}}_{\mathrm{tp}}\end{array}\right){\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)}^{\mathrm{t}}\\ \Rightarrow \left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right){\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)}^{\mathrm{t}}{\left(\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right){\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)}^{\mathrm{t}}\right)}^{-1}\left(\begin{array}{c}{\mathrm{v}}_{\mathrm{x}}\\ {\mathrm{v}}_{\mathrm{y}}\end{array}\right)\\ =-\left(\begin{array}{c}{I}_{t1}\\ ⋮\\ I_{\mathit{tp}}\end{array}\right){\left(\begin{array}{cc}{I}_{x1}& I_{y1}\\ ⋮& ⋮\\ I_{\mathit{xp}}& I_{\mathit{yp}}\end{array}\right)}^t{\left(\left(\begin{array}{cc}{I}_{x1}& I_{y1}\\ ⋮& ⋮\\ I_{\mathit{xp}}& I_{\mathit{yp}}\end{array}\right){\left(\begin{array}{cc}{I}_{x1}& I_{y1}\\ ⋮& ⋮\\ I_{\mathit{xp}}& I_{\mathit{yp}}\end{array}\right)}^t\right)}^{-1}\end{array}$$E10

Where:

$$\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)={\mathrm{A}}_{\mathrm{px}2}=\mathrm{A}$$

Therefore, one has that:

$$\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right){\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)}^{\mathrm{t}}{\left(\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right){\left(\begin{array}{cc}{\mathrm{I}}_{\mathrm{x}1}& {\mathrm{I}}_{\mathrm{y}1}\\ ⋮& ⋮\\ {\mathrm{I}}_{\mathrm{xp}}& {\mathrm{I}}_{\mathrm{yp}}\end{array}\right)}^{\mathrm{t}}\right)}^{-1}=$$

$$=\left(A·A^t\right){\left(A·A^t\right)}^{-1}=\left(\begin{array}{ccc}1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & 1\end{array}\right)={\mathit{Id}}_{\mathit{pxp}}=\mathit{Id}$$E11

Thus:

$$\mathit{Id}=-I_t·A^t{\left(A·A^t\right)}^{-1}$$E12

This method has a reduced computational cost to determine optical flow estimation when compared to other methods because it is simple, that is, it is since the region in which the variation of light intensity between pixels is minimal has a size $2\times 2,$ contained in a region $NxN$. In this way, the Optical Flow is determined in a region of $2\times 2$, between these two pixels, using only one matrix inversion operation (equation (12)).

To calculate the optical flow over the size region $NxN$, partial derivatives must be calculated in each pixel. However, considering almost null the variation of the intensity of light between pixels, over the region, the small differences in the accumulated intensities of brightness between pixels compromise the accuracy of the Optical Flow in relation to the determination of the actual motion object, that is, it gains in the processing speed and loses precision in the determination of the motion. When deriving equation (5) we have equation (13), that is:

$${\mathrm{\xi }}_{\alpha }^2=a_x{\alpha }_1+a_y{\alpha }_2+a_3{u}^2+a_4{v}^2+a_5\mathit{uv}+a_{6}u+a_{7}v+a_8$$E13

Where the terms ${\alpha }_x=\frac{\partial v_x}{\partial t}$, ${\alpha }_y=\frac{\partial v_y}{\partial t}$, are called the components of the acceleration vector, $\left(v_x,v_y\right)$ is the components of the velocity vector and the terms $a_1=I_x=\frac{\partial }{\partial x}I\left(x,y,t\right);a_2=I_y=\frac{\partial }{\partial y}I\left(x,y,t\right);a_3=I_{\mathit{xx}}=\frac{{\partial }^2}{{\partial x}^2}I\left(x,y,t\right);a_4=I_{\mathit{yy}}=\frac{{\partial }^2}{{\partial x}^2}\left(x,y,t\right);a_5=I_{\mathit{xy}}=\frac{\partial }{\partial \mathit{xy}}I\left(x,y,t\right);a_6=I_{\mathit{xt}}=\frac{\partial }{\partial \mathit{xt}}I\left(x,y,t\right)$; $a_7=I_{\mathit{yt}}=\frac{\partial }{\partial \mathit{yt}}I\left(x,y,t\right)$$a_8=I_{\mathit{tt}}=\frac{\partial }{{\partial t}^2}I\left(x,y,t\right)$, are the first and second partial derivatives of the $I\left(x,y,t\right)$.

In view of the small variations present and accumulated along the vector field associated with the optic flow, which cause an additional error in equation (13), a regularization adjustment was made, given by equation (14):

$${\mathrm{\xi }}_c^2={\left(\frac{\partial }{\partial x}{v}_x\left(x,y\right)\right)}^2+{\left(\frac{\partial }{\partial y}{v}_x\left(x,y\right)\right)}^2+{\left(\frac{\partial }{\partial y}{v}_y\left(x,y\right)\right)}^2+{\left(\frac{\partial }{\partial x}{v}_y\left(x,y\right)\right)}^2$$E14

Thus, combining equations (13) and (14), the error $\xi$ can be minimize by the equation (15):

$$\iint \left({\mathrm{\xi }}_{\mathrm{\alpha }}^2+{\mathrm{\alpha }}^2{\mathrm{\xi }}_{\mathrm{c}}^2\right)\text{dxdy}$$E15

where $\alpha$ is the value of the weights required for smoothing the variation of the associated optical flow. So, to get the $v_x=v_x\left(x,y\right)$ e $v_y=v_y\left(x,y\right)$, thus using the resources of the variational calculation one has:

$$\left\{\begin{array}{c}2{\mathrm{\alpha }}_3{v}_x+{\mathrm{\alpha }}_5{v}_y={\mathrm{\alpha }}^2{\nabla }^2{v}_x-{\mathrm{b}}_1\\ {\mathrm{\alpha }}_5{v}_x+2{\mathrm{\alpha }}_4{v}_y={\mathrm{\alpha }}^2{\nabla }^2{v}_y-{\mathrm{b}}_2\end{array}\right.$$E16

where ${\nabla }^2{v}_x$ is the Laplacian of $v_x$ e ${\nabla }^2{v}_y$ is the Laplacian of $v_y$ and the coefficients de $b_1$, $b_2$ can be given as:

$$\left\{\begin{array}{c}{\mathrm{b}}_1=\frac{\partial }{\mathrm{\partial t}}{\mathrm{\alpha }}_1+{\mathrm{\alpha }}_6\\ {\mathrm{b}}_2=\frac{\partial }{\mathrm{\partial t}}{\mathrm{\alpha }}_2+{\mathrm{\alpha }}_7\end{array}\right.$$E17

and replacing the coefficients $I_x$, $I_y$, $I_{\mathit{xx}}$, $I_{\mathit{yy}}$, $I_{\mathit{xy}}$, $I_{\mathit{xt}}$, $I_{\mathit{yt}}$, $I_{\mathit{tt}}$ in equation (16), one has:

$$I_{\mathit{xx}}{v}_x+I_{\mathit{xy}}{v}_y=\left(\frac{{\mathrm{\alpha }}^2}{3}\right)\nabla 2v_y-I_{\mathit{xt}}$$E18

$$I_{\mathit{xy}}{v}_x+2I_{\mathit{xy}}{v}_y=\left(\frac{{\mathrm{\alpha }}^2}{3}\right)\nabla 2v_x-I_{\mathit{yt}}$$E19

whereas ${\nabla }^2{v}_x=\overline{v_{x_1{j}_k}}-v_{x_1{j}_k}$, e ${\nabla }^2{\mathrm{v}}_{\mathrm{y}}=\overline{v_{y_1{j}_k}}-v_{j_1{j}_k}$, are the Laplacians of equations (18) and (19), given in their discretized digital forms together with equation (20),

$$\lambda =\left(\frac{{\alpha }^2}{3}\right)$$E20

It is possible to reduce the data system by (17), such as:

$$\left[{\lambda }^2\left(I_{\mathit{xx}}+I_{\mathit{yy}}+{\lambda }^2\right)+\kappa \right]v_x={\lambda }^2\left(I_{\mathit{yy}}+{\lambda }^2\right)\overline{v_x}-{\lambda }^2{I}_{\mathit{xy}}+\overline{v_x}+c_1$$E21

$$\left[{\lambda }^2\left(I_{\mathit{xx}}+I_{\mathit{yy}}+{\lambda }^2\right)+\kappa \right]v_x=-{\lambda }^2(I_{\mathit{yy}}\overline{v_x}+{\lambda }^2\left(I_{\mathit{yy}}+{\lambda }^2\right)\overline{v_y}+c_2$$E22

where the term $\kappa =I_x{I}_{\mathit{yy}}-{I_{\mathit{xy}}}^2$, it is called Gaussian curvature of the surface. And it is also that:

$$\left\{\begin{array}{c}{c}_1=I_{\mathit{xy}}{I}_{\mathit{yt}}-I_{\mathit{xx}}\left(I_{\mathit{yy}}+{\lambda }^2\right)\\ c_2=I_{\mathit{xx}}{I}_{\mathit{yy}}-I_{\mathit{xx}}\left(I_{\mathit{xx}}+{\lambda }^2\right)\end{array}\right.$$E23

Where $c_1,c_2$ they’re real constants.

Therefore, isolating terms $v_x$, $v_y$ and still replacing $c_1$ e $c_2$ in equations (21) and (22) respectively, resulting in equations (24) and (25):

$${\mathrm{v}}_{\mathrm{x}}=\overline{{\mathrm{v}}_{\mathrm{x}}}-\left[\frac{{\mathrm{\lambda }}^2\left({\mathrm{I}}_{\mathrm{yy}}+{\mathrm{\lambda }}^2\right)\overline{{\mathrm{v}}_{\mathrm{x}}}-{\mathrm{\lambda }}^2{\mathrm{I}}_{\mathrm{xy}}+\overline{{\mathrm{v}}_{\mathrm{x}}}+{\mathrm{c}}_1}{\left[{\mathrm{\lambda }}^2\left({\mathrm{I}}_{\mathrm{xx}}+{\mathrm{I}}_{\mathrm{yy}}+{\mathrm{\lambda }}^2\right)+\mathrm{\kappa }\right]{\mathrm{v}}_{\mathrm{x}}}\right]$$E24

$${\mathrm{v}}_{\mathrm{y}}=\overline{{\mathrm{v}}_{\mathrm{y}}}-\left[\frac{-{\mathrm{\lambda }}^2({\mathrm{I}}_{\mathrm{yy}}\overline{{\mathrm{v}}_{\mathrm{x}}}+{\mathrm{\lambda }}^2\left({\mathrm{I}}_{\mathrm{yy}}+{\mathrm{\lambda }}^2\right)\overline{{\mathrm{v}}_{\mathrm{y}}}+{\mathrm{c}}_2}{\left[{\mathrm{\lambda }}^2\left({\mathrm{I}}_{\mathrm{xx}}+{\mathrm{I}}_{\mathrm{yy}}+{\mathrm{\lambda }}^2\right)+\mathrm{\kappa }\right]{\mathrm{v}}_{\mathrm{x}}}\right]$$E25

The Algorithm 1 is a pseudocode to generate the proposed optical flow vector, through equations (24) and (25) and that allow estimating the speed and position of an object, through a sequence of video images.

Algorithm 1. Adapted optical flow (Gaussian curvature κ).

Begin

Input: Image sequence (video)

Output: Vector optic flow generator ($v_x$, $v_y)$

   For I = 1…N do

      Convert images to a gray tone

      Calculate the partial derivatives of 1°and 2°orders of $I\left(x,y,t\right)$

      Calculate constants $a_x$, $a_y$, $a_1,\dots ,a_8$, $b_1,b_2,\lambda ,c_1,$$c_2$

      Calculate the discretized Laplacians of ${\nabla }^2{v}_x$, ${\nabla }^2{v}_y$

      Calculating Gaussian curvature $\kappa$

      Calculate flow components (u, v)

   End For

End

2.2.2 Random forests

Developed by Breiman [63] in the mid-2000s, and later revised in [71] random trees are considered one of the best-supervised learning methods used in data prediction and classification. Due to its simplicity, low computational cost, great potential to deal with a large volume of data, and still present great accuracy of results, currently this method has become very popular being applied in various fields of science as data science [72]. Bioinformatics, Ecology, in real-life systems and recognition of 3D objects. In recent years, several studies have been conducted with the objective of making the technique more elaborate and seeking new practical applications [73, 74, 75].

Many studies were carried out with the aim of narrowing the existing gap between theory and practice can be seen in [58, 76, 77, 78]. Among the main components of random tree forests, one can highlight the bagging method [63], and the criterion of classification and regression called cart-split [79], which play critical roles.

Bagging (a bootstrap-aggregating contraction) is an aggregation scheme, which generates samples through the bootstrap method, from the original dataset. These methods are nonparametric and belong to the Monte Carlos method class [80], treating the sample as a finite population. Still, these methods are used when the distribution of the target population is not specified, and the sample is the only information available. How in this way a predictor of each sample is constructed, so that the decision is made through an average, and is more effective computational procedures to improve the indexable estimates, especially for large sets of high-dimensional data, where finding a good model in one step is impossible due to the complexity and scale of the problem. As for the cart-split criterion, it originates from the CART program [63], and is used in the construction of individual trees to choose the best cuts perpendicular to the Axes. However, while bagging and the CART division scheme are key elements in the random forest, both are difficult to mathematically analyze and are a very promising field for both theoretical and practical research.

In general, the set of trees is organized in the form of {$T_1$ (${\mathrm{Ɵ}}_1$), $T_2$ (${\mathrm{Ɵ}}_2$)… $T_i$ (${\mathrm{Ɵ}}_i$}, where $T_B$ is every tree and ${\mathrm{Ɵ}}_B$ are bootstrap samples with spare dimensions $q$x $\mathit{\text{mtry}}$, where $\mathit{\text{mtry}}$is equal to the number of variables that will be used on each node during the construction of each tree and $q$is approximately $0,67\times \left(n\right).$ Each of the trees produces a response $y_{1,i}$ for each of the samples $W$ {$T_1\left(W\right)$ = $y_{1,i}$, $T_2\left(W\right)$ = $y_{2,i}$, .., $T_i\left(W\right)$ = $y_{2,B}$} and the mean (regression) or majority vote (classification) of the tree responses will be the final response of the model for each of the samples.

3. Methodology

The methodology employed consisted of combining the optical flow algorithm in terms of Gaussian curvature, developed in this work together with the technique of random forest. The language used for the development of this algorithm was the MATLAB programming language, executed on a 64-bit 8th generation notebook, CORE i7. The input data is a video extension Avi, lasting 5 min of a vehicle and two cyclists, circulating in the vicinity of the beach of Costa Nova, in the locality of Ilhavo, in Aveiro, Portugal. The video was fragmented into a set of frames, analyzed two by two by the algorithm for the generation of the vector field of optical flow. After that, the resulting image associated with the flow and a minimal surface region, given by the Gaussian curvature. Next on this surface, the random trees analyzed which vectors presented important characteristics to characterize in an “optimal” way, the movement of the object (see Figure 2).

Figure 2.

Representative model of operation of a random forest.

After finishing the process of analysis of the movement of the objects, the execution times and accuracy of the results obtained by the proposed algorithm were compared in relation to the algorithms of Lucas Kanade, Horn and Shunck, Farneback and Lucas Kanade with or without Gaussian filter, allowing to validate the results obtained. After that, the implementation of the developed algorimo began.

4. Results

Figure 3 shows the vehicle and the two cyclists that were used to collect the image to which the results proposed in this work were obtained so that the choice was random on the right side. A graphical representation of the vector field of optical flow generated by the sequence of two consecutive frames, over 5 minutes of video is shown.

Figure 3.

(a) Left side: vehicle shift between moments $\left(t-1\right)$ and t. (b) Right side: representation of the corresponding optical flow.

On the right side of Figure 3, the optical flow associated with the movement of the vehicle between the time intervals from $\left(t-1\right)$ to $t$ is being represented. Note that the vector representation of this field associated with this flow was performed in such a way that the vectors generated by the field were superimposed in the horizontal direction of the central axis of the figure. Although there were other objects present at the site, that is, two cyclists and a car in the upper left corner, the object of interest considered was the vehicle close to the cyclists. This is shown on the right side of Figure 3, by the layout of this horizontal arrangement of vectors, which allows indicating whether the current movement and the predicted movement of the considered object is to the left or to the right.

The region with the highest horizontal vector density in Figure 3 is located on the left side, in blue. It is also observed that the number of vectors in this region, despite being spaced, starting from the center to the left, is greater in relation to the number of vectors on the right side. It is also possible, through it, to visually evaluate the movement behavior of the considered objects. This region, containing a higher vector density, corresponds to the current direction in which the object is heading and its predicted displacement. It is also possible to observe that this vector density increases towards the left side, passing through the central part, coming from the right, clearly indicating the direction of movement of the object, that is, the object moves to the left. In Figure 4, this process can be understood more clearly.

Figure 4.

Prediction and actual displacement of the object obtained through the optical flow.

In a similar way to the one mentioned in Figure 3, on the right side of Figure 5, the optical flow generated by the displacement of the moving vehicle is represented, between the instants $t$ to $\left(t+1\right)$. It is possible to observe that the vector representation of this field was performed in such a way that its vectors representing this field were also superimposed on the axis in the horizontal direction of the figure, creating a vector density created by this superposition. The object of interest considered remains the vehicle close to the cyclists. As can be seen on the right side of Figure 3, the arrangement of the horizontal vectors also allows to indicate the current movement and, if its movement prediction is to the left or to the right.

Figure 5.

Object remains on the right side, but with a medium offset to the right and displacement estimate still to the left.

It is possible to observe a small increase in the vector density to the left, but that has a great influence on the determination of the real and predicted position of the object in the considered time intervals. The Object continues with its actual movement to the left, as well as the predicted movement of the object to the left. However, he showed a slight movement to the left (direction where the cyclists are).

In Figure 6, a small variation of the optical flow is observed again in the associated movement between the instants $\left(t+1\right)$ to $\left(t+2\right)$. In this figure, the vehicle is next to the cyclists, both in the opposite direction to the vehicle. The movement of the vehicle continues without great variation in the direction, causing no period for cyclists or other vehicles in the opposite direction to it on the left side.

Figure 6.

Object remains on the right side, but with a slight shift to the right and offset estimate to the left.

In Figure 7, there was no optical flow variation in the associated movement between the time intervals $\left(t+2\right)$ to $\left(t+3\right)$. In this figure, the vehicle can be seen as it passes the two cyclists. This means that the non-significant variation in the optical flow vector field, keeping the number of vectors higher on the left side is associated with maintenance in the direction of movement of the object considered, that is, it continues to move on the left side.

Figure 7.

Object moving and keeping on the left for consecutive frames.

In Figure 8, the vehicle can be seen completely overtaking the two cyclists and approaching another vehicle in the opposite direction in the upper part of the image (left). The variation of the optical flow vector field remains the same. This indicates that the vehicle continues its trajectory, on the left side to the cyclists, however without posing a danger of collision for the other vehicle in the opposite direction.

Figure 8.

Object with unchanged offset pattern.

5. Analysis and discussing the results

This item will show how the performance evaluation of the proposed algorithm and accuracy was performed in relation to the Algorithms of Luca and Kanade, with or without Gaussian filter, Horn and Schunck, and Farneback.

The algorithm allowed to show on the display in real-time the displacement of the object on the right side and the set of vectors capable of representing the movement of the real-time or accumulated indicating the tendency, in this case, of the direction that the object should perform. This process was carried out in a similar way, using the other algorithms to make it possible to compare them. The behavior of the proposed algorithm and the other will be graphically shown.

The technique developed in this work allowed to generate an optical flow considering important geometric properties allowing to identify similar categories of moving objects and same characteristics. These geometric properties are intrinsically associated with the curvature of the object’s surface in three-dimensional space, called Gaussian curvature, in this case in a 2D image.

The modified optical flow, considering these properties, generated a dense optical flow, allowing the generation of a band, describing a track on the 2D plane. This allowed tracking the movement of the considered object. In the same Figure 8, it is possible to observe that at each time interval in which the object was monitored, the dispositions of the vectors for the left and right sides, as shown in Figures 3–7 were responsible for drawing the track associated with the displaced and that allowed tracking the object as it moves.

Figure 9 shows the vehicle that, when moving, generated the optical flow. In Figures 10 and 11, the variations of the optical flow between two-time intervals, ${\mathrm{\Delta }t}_i$ and ${\mathrm{\Delta }t}_n$, ($i<n$) are shown. In this way, the algorithm allowed tracking the progressive movement of the object (movement adopted as progressive, in this work) and, as this happens, it is possible to predict in which direction it is moving, that is, to the left or to the right (or keeping straight) line.

Figure 9.

Variation of the optical flow of the moving object.

Figure 10.

Vehicle movement.

Figure 11.

Object moving to the left side.

5.1 Optical flow algorithm in terms of the curvature

In the following items, the implementations of the Lucas and Kanade algorithms without or with a Gaussian filter, Horn and Schunck, and Farneback will be shown, using as input data the same sequence of video images used in the algorithm developed in this work. For each, the performance and accuracy obtained will be verified.

5.2 Lucas and Kanade algorithm without Gaussian filter

5.3 Lucas and Kanade Algorithm with Gaussian filter

5.4 Algoritmo de Horn and Schunck

5.5 Algoritmo de Farneback

For each of the 5 algorithms, 1 frame is shown containing 4 figures, with 2 upper and 2 lower. In each frame, the figure at the top left shows the variation of the vector field between two frames. The right frame, on the other hand, corresponds to the variation of the object’s movement in real-time. The lower ones, except for the proposed algorithm, correspond to the number of points on the right or the left, and with this, the movement will occur to the side that has the greatest number of points. In the case of the proposed algorithm, the process will take place through the analysis of vector density. So, to the side where there is greater vector density, this is the side to which the movement will be occurring (see Figures 12–20).

Figure 12.

Object moving to the right side.

Figure 13.

Variation of the optical flow of the moving object.

Figure 14.

Vehicle movement.

Figure 15.

Object moving to the left side.

Figure 16.

Object moving to the right side.

Figure 17.

Variation of the optical flow of the moving object.

Figure 18.

Vehicle movement.

Figure 19.

Object moving to the left side.

Figure 20.

Object moving to the right side.

Comparing the results presented by the algorithms, it is observed that in the developed model, it was possible to see a dense vector trail of the object, with a slight tendency of displacement to the left, as it continues its movement. In the other models, this was not possible, and it is necessary to resort to a score of points, in the lower table. This process is also possible in the proposed model, but not necessary, which means a reduction in computational cost (see Figures 21–28).

Figure 21.

Variation of the optical flow of the moving object.

Figure 22.

Vehicle movement.

Figure 23.

Object moving to the right side.

Figure 24.

Object moving to the left side.

Figure 25.

Variation of the optical flow of the moving object.

Figure 26.

Vehicle movement.

Figure 27.

Object moving to the left side.

Figure 28.

Object moving to the left side.

Comparing the results, it is observed that the Farneback algorithm also presents high vector density. But the proposed model, as previously said, presents a well-defined vector trail which suggests the non-use of the point count in the lower frame, which does not occur for the Farneback algorithm, indicating higher computational cost, which can affect the accuracy of this algorithm when compared to the proposed algorithm.

Comparing the Horn and Schunck algorithm, a low vector density is observed when compared to the proposed algorithm, which indicates lower accuracy when compared to the proposed algorithm.

Although the two techniques of Lucas and Kanade, are faster applications, indicating low computational cost when compared to the proposed algorithm, the factor of low vector density results in low precision in relation to the proposed method.

6. Final considerations

The proposed method presented good results, showing to be accurate and reasonable speed. This allows this application to be used in critical problems, i.e., to real-world problems. However, it presented limitations that could be verified when compared to the model with Lucas and Kanade, with a Gaussian filter, which is faster and presents good accuracy.

The proposed Method reached only approximately 50% execution speed in relation to the Lucas and Kanade Method, which motivates further improvements to the Method. The technique presented can be applied to other fields of research as in cardiology due to presenting great precision when submitted to small region, which is important because it can be applied with the objective of predicting infarctions and as a current contribution, for the state of the art is to characterize the optical flow in terms of Gaussian curvature, that makes it possible to highlight fields of research such as computational vision and differential geometry.

Acknowledgments

The authors of this work would like to thank the Institute of Electronics and Informatics Engineering of Aveiro, the Telecommunications Institute of Aveiro, and the University of Aveiro for the financial, technical-administrative, and structural support provided that allowed the accomplishment of this work.