A Novel Artificial Organic Controller with Hermite Optical Flow Feedback for Mobile Robot Navigation

Abstract

This chapter describes a novel nature-inspired and intelligent control system for mobile robot navigation using a fuzzy-molecular inference (FMI) system as the control strategy and a single vision-based sensor device, that is, image acquisition system, as feedback. In particular, FMI system is proposed as a hybrid fuzzy inference system with an artificial hydrocarbon network structure as defuzzifier that deals with uncertainty in motion feedback, improving robot navigation in dynamic environments. Additionally, the robotics system uses processed information from an image acquisition device using a real-time Hermite optical flow approach. This organic and nature-inspired control strategy was compared with a conventional controller and validated in an educational robot platform, providing excellent results when navigating in dynamic environments with a single-constrained perception device.

1. Introduction

Navigation of mobile robots is still a challenging problem due to the uncertain and dynamics of environments [1]. Typically, navigation consists of different tasks [1] such as: perception, exploration, mapping, localization, path planning, and path execution. In this work, navigation is limited to a strategy or methodology that guides a mobile robot to accomplish a task through an environment with obstacles—static or dynamic—without collisions, using sensors. Also, this work considers a robotic system with a unique vision-based sensor for perception.

Artificial intelligence (AI) has been widely used for navigation of mobile robots. Different AI methods have been proposed, such as: fuzzy logic [2], artificial neural networks [3], hybrid neuro-fuzzy controllers [4–6], evolutionary and meta-heuristics algorithms [7], and nature-inspired computing (e.g., particle swarm optimization and ant colony optimization) [8, 9]. Artificial intelligence has been used in different levels of cognition such as: reasoning, decision-making, and learning.

Furthermore, some applications using AI-based controllers and features based on optical flow to compute the control law are present in the literature. For instance, Williams et al. [10] propose a support vector machines (SVM)—kernel to carry out vehicle tracking using optical flow features. Meta-cognitive neuro-fuzzy inference systems (McFIS) for accurate detection of human actions from video sequences have been proposed by Subramanian et al. [11]. In fact, they employ optical flow-based features, and the functional relationship between these optical flow-based features and action classes is approximated using the McFIS classifier. Boroujeni et al. [12] propose a method for obstacle detection using optical flow and k-means clustering-based classifier to determine obstacles’ positions. Narayanan et al. [13] aim to develop a real-time system capable of rapidly detecting and tracking vehicles in a video stream using a monocular vision system. The detection is based on the Viola–Jones classifier and optical flow features [13]. In addition, Crnko et al. [14] propose a biomimetic measurement of optical flow and centroid for visual-servo control of hover flight.

In that way, this work proposes a novel nature-inspired and intelligent controller for navigation of mobile robots constrained to a single vision-based sensor, using a recent hybrid fuzzy inference system and artificial hydrocarbon networks (AHN) approach so-called fuzzy-molecular inference (FMI) system. This fuzzy-molecular inference system inherits characteristics from fuzzy inference systems (FIS) such as storage of knowledge from experts, dealing with uncertainty, and linguistic interpretation [15]; but also from artificial organic networks (AON), a learning method inspired on chemical organic compounds that allow packaging information in molecular units, organizing of information, and dealing with aggressive uncertainty and noise [16]. In advance, fuzzy-molecular inference systems can deal with uncertainty and under aggressive conditions of noise, with relative easiness of implementation [15]. Examples of FMI-based controllers can be found in literature [15, 17]. In addition, this proposed controller uses a real-time Hermite optical flow (RT-HOF) approach for detecting direction of moving objects in the visual range of the mobile robot. In fact, RT-HOF is a method that determines the displacement of each pixel at some time in an image sequence representing the motion of objects in the scene or the relative motion of the sensor, with the usage of the Hermite transform (HT) [18] as a biological image model in real-time. This RT-HOF approach has been proved to be easy to implement and faster than conventional optical flow methods [19], giving advantages to be used in online navigation.

The rest of the chapter is organized as follows: Section 2 briefly describes the fundamentals of the proposal: artificial hydrocarbon networks algorithm as part of artificial organic networks method, the fuzzy-molecular inference system, and the real-time Hermite optical flow method. Section 3 formalizes the proposed nature-inspired and intelligent control for mobile robots and its design. Later on, Section 4 describes the implementation of the proposed artificial organic controller in a mobile robot system. Section 5 presents experimental results of the proposed controller, and Section 6 concludes the work.

2. Nature-inspired and intelligent computing

This section describes the fundamentals of the proposed controller for robot navigation. First, artificial hydrocarbon networks learning algorithm is introduced. Then, the fuzzy-molecular inference system is described, and lastly, the real-time Hermite optical flow approach is presented.

2.1. Artificial hydrocarbon networks

Artificial hydrocarbon networks (AHN) is a nature-inspired learning algorithm based on the artificial organic networks framework. In fact, artificial organic networks (AON) technique is a meta-heuristic method and framework inspired on chemical organic structures and the energy involved in the interaction among them. Inspiration of AON method considers packaging information in independent structures named molecules which they can relate among them using chemical energy-based heuristics [16]. Therefore, AON method provides modularity, inheritance, organization, and structural stability in the output response [16].

Thus, artificial hydrocarbon network method is a supervised learning technique that inherits from AON framework, restricted in the shape and type of molecules occupied [16]. From the latter, AHN is inspired on chemical hydrocarbon compounds that constraint the method only using two atomic units, that is, hydrogen and carbon atoms, that can be related together with at most one and four degrees of freedom, respectively, as described below.

The basic unit with information of AHN is the so-called CH-molecule structure. It is made of two or more atoms related among them in order to define a functional behavior φ centred in the carbon atom and parameterized with the other hydrogen atoms linked to it, as shown in Eq. (1), where σ is a real value called the carbon value, H_i ∈ ℂ is the ith hydrogen atom linked to the carbon atom,k ≤ 4 represents the number of hydrogen atoms in the molecule, and x is the input signal representing a stimulus of the overall molecule.

$\phi \left(x\right)=\sigma {\displaystyle \prod _{i=1}^k}\left(x-H_i\right)$E1

Then, these molecules can form other types of structures so-called compounds, which include nonlinear relationships among all molecules [16]. In this work, forming compounds are restricted to be linear and saturated compounds like Eq. (2), where CH_k represents a molecule with k < 4 hydrogen atoms associated to it, n is the number of CH-molecules in the compound, and A − B represents a simple bond between two molecules A and B. At the end, this linear and saturated compound is made of two CH₃ molecules and (n − 2) CH₂ molecules [20].

$C{H}_3-\underset{n-2}{\underbrace{C{H}_2-\dots -C{H}_2}}-C{H}_3$E2

In terms of the behavior of the compound, it is expressed as a nonlinear function ψ that depends on the molecular behaviors and the type of bond between molecules. In this work, ψ is defined as a piecewise function of nCH-molecules like Eq. (3) [17], where L_i represents the ith bound at which a molecule can act over the input space, and φ_i is the ith molecular behavior in the forming compound. To this end, if the input domain is in the interval x ∈ [x_min, x_max], then L₀ = x_min and L_n = x_max.

$\psi \left(x\right)=\left\{\begin{array}{c}\hfill \begin{array}{cc}\hfill {\phi }_1\left(x\right)\hfill & \hfill L_0\le x<L_1\hfill \end{array}\hfill \\ \hfill \begin{array}{cc}\hfill \dots \hfill & \hfill \dots \hfill \end{array}\hfill \\ \hfill \begin{array}{cc}\hfill {\phi }_n\left(x\right)\hfill & \hfill L_{n-1}\le x\le L_n\hfill \end{array}\hfill \end{array}\right.$E3

Lastly, an artificial hydrocarbon network is a mixture AHN of a set of compounds interacting among them in definite ratios named stoichiometric coefficients α_m as shown in Eq. (4), where c is the number of compounds in the mixture, and ψ_m is the mth behavior of compound in the mixture [16]. In this work, the artificial hydrocarbon network structure is restricted to be one compound such that AHN(x) = ψ₁(x).

$AHN\left(x\right)={\displaystyle \sum _{m=1}^c}{\alpha }_m{\psi }_m\left(x\right)$E4

Figure 1.

AHN Algorithm to Create an Artificial Hydrocarbon Network.

Chemical heuristic rules for searching and building an optimum artificial hydrocarbon network with specific carbon and hydrogen values are computed using the so-called AHN-algorithm depicted in Figure 1 (for a detailed description, see for example [21]). Finally, examples of applications using this technique can be found in literature [20, 22–24].

2.2. Fuzzy-molecular inference system

Fuzzy-molecular inference (FMI) systems are ensembles of artificial hydrocarbon networks and fuzzy inference systems [15] that consist of three steps: fuzzification, fuzzy inference engine, and molecular defuzzification. The overall process maps crisp inputs to a fuzzy space, and resulting values are returned to as crisp outputs using a molecular approach based on artificial hydrocarbon networks. In addition, a knowledge base stores information about a specific problem [15].

The basis fuzzy-molecular rule is shown in Eq. (5), where x_i is a linguistic variable of the antecedents in a set of fuzzy partitions, F_i, y_f is the variable of the consequent in the fth fuzzy-molecular rule, M_j is the j-molecular unit of a linear and saturated hydrocarbon compound stimulated with the resulting fuzzy implication value of the T-norm Δ.

$R_f:\mathit{if}\Delta \left(x_1\in F_1,\dots ,x_i\in F_i,\dots ,x_k\in F_k\right),\mathit{then}{y}_f\in M_j$E5

First at the fuzzification step, the fuzzy-molecular inference system receives an input signal x also known as the linguistic variable, and it is mapped to a fuzzy value in the range of [0,1]. Then, let F be a fuzzy set and μ_F(x) the membership function of, F,∀ x ∈ X where X is the input domain space. Moreover, μ_F(x) ∈ [0, 1] is a real value representing the degree of belonging to x into the fuzzy set F. Additionally, this linguistic variable can be partitioned into k different fuzzy sets F_i. Then, its membership value can be computed over the set of membership functions ${\mu }_{F_i}\left(x\right)$ for all i = 1, …, k. To this end, the shape of the membership functions depends on the purpose of the problem domain [15].

The second step considers the fuzzy inference engine as a mechanism that computes the consequent value y_f of a set of fuzzy rules that accept membership values of linguistic variables, as already shown in Eq. (5). If assuming that μ_Δ(x₁, …, x_k) is the result of any T-norm function, then Eq. (5) can be rewritten as Eq. (6), using μ_Δ as the min function [15].

$R_f:\mathit{if}\Delta \left(x_1\in F_1,\dots ,x_i\in F_i,\dots ,x_k\in F_k\right),\mathit{then}{y}_f={\phi }_j\left(min\left\{{\mu }_{F_1}\left(x_1\right),\dots ,{\mu }_{F_k}\left(x_k\right)\right\}\right)$E6

Lastly, the molecular defuzzification step computes the crisp output value y as Eq. (7) using all fuzzy rules of the form as Eq. (6), where ${\mu }_{{\Delta }_f}$ is the fth fuzzy evaluation of the antecedents, and y_f is the fth consequent value for all fuzzy rules [15].

$y=\frac{{\displaystyle \sum }{{\mu }_{\Delta }}_f\left(x_1,\dots ,x_k\right)\cdot y_f}{{\displaystyle \sum }{{\mu }_{\Delta }}_f\left(x_1,\dots ,x_k\right)}$E7

The proper knowledge of the specific problem domain can be enclosed into a knowledge base, summarizing all fuzzy rules of Eq. (6), assigning any CH-molecule of the compound to each antecedent [15]. To this end, examples of applications using the FMI system can be found in literature [15–17, 22, 25].

2.3. Real-time Hermite optical flow

Optical flow (OF) is defined as a two dimensional distribution of “apparent velocities” of the objects in the scene that in most cases are associated with intensity variations of the objects into an image sequence [26]. It is represented by a vector field that represents the displacement of pixels. For example, Figure 2 shows the vector field obtained from frames 43 and 44 of the camera motion sequence [27].

Figure 2.

Optical Flow Representation in a Sequence of Images [19]. (a) Frame 43 of the Camera Motion Sequence. (b) Frame 44 of the Camera Motion Sequence. (c) Optical Flow Result.

Classical differential optical flow methods assume that the intensity value of the objects remain constant in two consecutive time instants of an image sequence L(X, t). This assumption was proposed by Horn and Schunck in 1981 [28] and is known as the constant intensity constraint like Eq. (8) where X = (x, y, 1)^T is the pixel location, W = (u, v, 1)^T represents the horizontal and vertical pixel displacements, respectively, between two images at time t and (t + 1).

$L\left(X+W,t+1\right)-L\left(X,t\right)=0$E8

If small displacements are assumed, a Taylor expansion is considered; and the optical flow constraint Eq. (9) is obtained where ∇₃L = (L_x, L_y, L_t)^T are the derivatives of the intensity image L(X) in x, y directions and time.

$W^{\mathrm{T}}\left({\nabla }_{3}L\right)=0$E9

If only the constant intensity constraint is used, it is not possible to determine the two components of displacement u and v; then, an ill-posed problem is obtained known as the aperture problem where only the normal component of the motion can be obtained [28]. Therefore, it needs additional constraints to fully calculate the optical flow. Recent differential optical flow methods have proposed additional constraints to overcome the aperture problem and to improve accuracy of the displacement obtained, that is, add local image constraints.

Horn and Schunck [28] proposed the smoothness constraint which assumes that the flow is smooth, that is, pixels within a neighborhood have similar magnitude and orientation. Thereby, the optical flow estimation method of Horn can be found minimizing, by an iterative approach, the energy functional of Eq. (10), where ∇₃L = (L_x, L_y, L_t)^T , Ω is the image domain, and α is a smoothness parameter. In most cases, the constant intensity constraint assumption of Eq. (8) is not fulfilled. Therefore, to overcome this problem, an additional term independent to intensity change is required. In [29, 30], a constant gradient constraint is proposed as shown in Eq. (11). A way to represent the most important visual characteristics is using bio-inspired models as the Hermite transform [31].

${\displaystyle \underset{\mathrm{\Omega }}{\int }}\left(W^{\mathrm{T}}\left({\nabla }_{3}L{\nabla }_3{L}^{\mathrm{T}}\right)W+\alpha {\left|\nabla W\right|}^2\right)dX=0$E10

$\nabla L\left(X+W\right)-\nabla L\left(X\right)=0$E11

The Hermite transform is an image model based in a polynomial decomposition from a perceptual standpoint. This image model emulates the local processing and the response of receptive fields, for example, the Gaussian derivative model of the human vision system [31, 32]. In fact, it is computed by convolution of the image L(x, y) with the analysis filters D_m,n − m like in Eq. (12), where L_m,n − m(x, y) are the cartesian Hermite coefficients, m and (n − m) are the analysis order in the orthogonal directions x and y, and (x₀, y₀) represents the position of the sampling lattice S.

$\begin{array}{cc}\hfill \hfill & \hfill L_{m,n-m}\left(x_0,y_0\right)=\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{-\infty }{\overset{\infty }{\int }}}{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}}L\left(x,y\right)D_{m,n-m}\left(x_0-x,y_0-y\right)\mathit{dxdy}\hfill \\ \hfill \hfill & \hfill n=0,1,\dots ,\infty \hfill \\ \hfill \hfill & \hfill m=0,1,\cdots ,n\hfill \end{array}$E12

The filter functions D_m,n − m(x, y) are defined by the analysis window v²(x, y)and the polynomials orthogonal to this window G_m,n − m(x, y) as expressed in Eq. (13). At last, the separable Hermite filters D_m,n − m(x, y) = D_m(x)D_n − m(y)are defined with a Gaussian $v\left(x,y\right)=\frac{1}{\sigma \sqrt{\pi }}exp\left(-\frac{\left(x^2+y^2\right)}{2{\sigma }^2}\right)$ with normalization factor for a unitary energy inv²(x, y), as in Eq. (14). Figure 3 shows an example of the Hermite filters D_n(x) for N = 5 (n = 0, 1, …, N).

$D_{m,n-m}\left(x,y\right)=G_{m,n-m}\left(-x,-y\right)v^2\left(-x,-y\right)$E13

$D_n\left(x\right)=\frac{{\left(-1\right)}^n}{\sqrt{2^{n}n!}}\frac{1}{\sigma \sqrt{\pi }}{H}_n\left(\frac{x}{\sigma }\right)exp\left(-\frac{x^2}{{\sigma }^2}\right)$E14

Figure 3.

Hermite Filters D_n(x) for N = 5 (n = 0, 1, …, N).

The steered Hermite transform (SHT) can be defined by rotating the cartesian Hermite coefficients L_m,n − m(x, y) by the angular functions g_m,n − m(θ) such that Eq. (15) holds, where l_{m,n − m,θ}(x, y) are steered Hermite coefficients for the local angle θ, and g_m,n − m(θ) expresses the directional selectivity of the filter as shown in Eq. (16) [33].

$l_{m,n-m,\theta }\left(x,y\right)=\left\{\begin{array}{c}\hfill {\displaystyle \sum _{k=0}^n}\left(L_{k,n-k}\left(x,y\right)\right)\left(g_{k,n-k}\left(\theta \right)\right),m=0\hfill \\ \hfill 0,m>0\hfill \end{array}\right.$E15

$g_{m,n-m}\left(\theta \right)=\sqrt{\left(\begin{array}{c}\hfill n\hfill \\ \hfill m\hfill \end{array}\right)}\left({cos}^m\theta \right)\left({sin}^{n-m}\theta \right)$E16

The steered Hermite coefficients l_{m,n − m,θ}(x, y) allow to adapt the analysis process to the local content of the image, for example, using directional Gaussian derivatives filters [34], and the angle θ can be estimated by a criterion of maximum energy direction (e.g., the gradient of the image). Figure 4 shows the steered Hermite coefficients of the House test image [35] for N = 3 (n = 0, 1, 2, 3).

Figure 4.

An Ensemble of the Steered Hermite Coefficients of the House Test Image for N = 3 (n = 0, 1, …, N). [36].

The steered Hermite transform can be used to increase the accuracy of the optical flow by using a biological image model and by incorporating the steered Hermite coefficients as constraint independent to intensity changes. In [19], a modified version of Horn and Schunck approach was proposed. This includes an expansion of the constant intensity constraint and incorporates the steered Hermite coefficient constraint shown in Eq. (17), where γ is a weight parameter.

$\left[L_0\left(X+W\right)-L_0\left(X\right)\left]+\gamma \right[{\displaystyle \sum _{n=1}^N}{l}_{n,\theta }\left(X+W\right)-{\displaystyle \sum _{n=1}^N}{l}_{n,\theta }\left(X\right)\right]=0\text{,}$E17

Applying a Taylor expansion of Eq. (17) and reducing terms using the relation $L_k=⟨L\left(x\right),H_k\left(\frac{x}{\sigma }\right)⟩$ [18], it can be obtained the optical flow Hermite constraint equation as denoted in Eq. (18):

$\begin{array}{cc}\hfill \left[u{L}_{0,1}\left(X\right)+v{L}_{1,0}\left(X\right)+\frac{\partial L_0\left(X\right)}{\partial t}\right]+\gamma {\displaystyle \sum _{n=1}^N}\left[u{l}_{n,{\theta }_{\left(m\right)+1}}\left(X\right)+v{l}_{n,{\theta }_{\left(n\right)+1}}\left(X\right)+\frac{\partial l_{n,\theta }\left(X\right)}{\partial t}\right]=0\hfill & \hfill \hfill \end{array}$E18

Where

$\begin{array}{ccc}\hfill \hfill & \hfill \hfill & \hfill l_{n,{\theta }_{\left(m\right)+1}}\left(X\right)={\displaystyle \sum _{n=1}^N}{L}_{\left(m\right)+1,n-m}\left(X\right)\cdot g_{m,n-m}\left(\theta \right),\hfill \\ \hfill \hfill & \hfill \hfill & \hfill l_{n,{\theta }_{\left(n\right)+1}}\left(X\right)={\displaystyle \sum _{n=1}^N}{L}_{m,\left(n+1\right)-m}\left(X\right)\cdot g_{m,n-m}\left(\theta \right)\text{.}\hfill \end{array}$E19

Figure 5.

Pseudocode for the Real-Time Hermite Optical Flow [19].

Using variational calculus, the corresponding Euler–Lagrange equations computed are like in Eqs. (19) and (20) where $L_{0t}\left(X\right)=\frac{\partial L_0\left(X\right)}{\partial t},l_{n,{\theta }_t}\left(X\right)=\frac{\partial l_{n,\theta }\left(X\right)}{\partial t}$, and ∇²u represent the Laplacian of u. To this end, Figure 5 shows the pseudocode for the real-time Hermite optical flow (RT-HOF) iterative solution by Gauss–Seidel.

$\left[L_{0,1}^2\left(X\right)+\gamma L_{0,1}\left(X\right){\displaystyle \sum _{n=1}^N}\left(l_{n,{\theta }_{\left(n\right)+1}}\left(X\right)\right)\right]u+\left[L_{0,1}\left(X\right)L_{1,0}\left(X\right)+\gamma L_{0,1}\left(X\right){\displaystyle \sum _{n=1}^N}\left(l_{n,{\theta }_{\left(m\right)+1}}\left(X\right)\right)\right]v=$E19

${\alpha }^2{\nabla }^{2}u-\left[L_{0t}\left(X\right)+\gamma {\displaystyle \sum _{n=1}^N}\left(l_{n,{\theta }_t}\left(X\right)\right)\right]$E19

$\left[L_{0,1}\left(X\right)L_{1,0}\left(X\right)+\gamma L_{1,0}\left(X\right){\displaystyle \sum _{n=1}^N}\left(l_{n,{\theta }_{\left(n\right)+1}}\left(X\right)\right)\right]u+\left[L_{1,0}^2\left(X\right)+\gamma L_{1,0}\left(X\right){\displaystyle \sum _{n=1}^N}\left(l_{n,{\theta }_{\left(m\right)+1}}\left(X\right)\right)\right]v=$E22

${\alpha }^2{\nabla }^{2}v-\left[L_{1,0}\left(X\right)L_{0t}\left(X\right)+\gamma {\displaystyle \sum _{n=1}^N}\left(l_{n,{\theta }_t}\left(X\right)\right)\right]$E20

3. Nature-inspired and intelligent controller for robot navigation

As stated above, this work proposes a nature-inspired and intelligent controller for navigation of mobile robots constrained to a single vision-based sensor, like Figure 6. In fact, this artificial organic controller consists of a fuzzy-molecular inference system as the control strategy (i.e., using a hybrid fuzzy inference system with artificial hydrocarbon networks), and a motion feedback from a single vision-based sensor that processes image sequences using the real-time Hermite optical flow approach. Following, main components of the artificial organic controller are described.

Figure 6.

Block Diagram of the Proposed Artificial Organic Control.

3.1. Control law based on a Fuzzy-molecular inference system

Consider a mobile robot with a single vision-based sensor, for example, a video camera, which it can move around a flat environment. If the environment consists of static and dynamic obstacles, the aim of the motion controller is to avoid obstacles as well as to move the robot in a proper way.

In that sense, the robot perceives the environment through the camera, obtaining relative displacements of the objects in front of the robot (see Section 3.2) in two axes: horizontal (û) and vertical $\left(\widehat{v}\right)$. Then, these relative displacements are used as inputs to the robot controller, as shown in Figure 6. In particular, the control law is proposed to be implemented using the fuzzy-molecular inference system as follows:

3.1.1. Fuzzification step

The design of this step considers selecting the set of inputs to the controller. As described above, two inputs are used: relative horizontal displacement û and relative vertical displacement $\widehat{v}$. Fuzzification also requires partitioning the domain space into k fuzzy sets {F₁, …, F_k} with their corresponding membership functions $\left\{{\mu }_{F_1}\left(\widehat{u}\right),\dots ,{\mu }_{F_k}\left(\widehat{u}\right)\right\}$ for input û and $\left\{{\mu }_{F_1}\left(\widehat{v}\right),\dots ,{\mu }_{F_k}\left(\widehat{v}\right)\right\}$ for input $\widehat{v}$. For example, fuzzy partitions for û can be: “negative,” “zero,” and “positive”. Notice that the number of fuzzy sets k might be different for each input. It is remarkable to say that the membership functions should be selected accordingly.

3.1.2. Fuzzy inference engine step

This step requires the definition of fuzzy rules with respect to the goal of the controller. Notice that the consequent values y_f in Eq. (6) are computed using the molecular behavior φ_j of the molecule M_j assigned to that specific rule R_f. Thus, a set of CH-molecules has to be defined (see below).

3.1.3. Molecular defuzzification step

The last step of the proposed controller considers the computation of the control signal y that has to be selected in order to activate the associated actuator in the robot, for example, the direct current motor coupled to wheels. For instance, if the mobile robot has a differential mechanism with two wheels, then two DC motors are coupled, for example, left motor (y_L) and right motor (y_R). Thus, defuzzification step uses Eq. (7) to compute y_L and y_R, and a set of CH-molecules defined for each output signal (y_L and y_R).

As restricted in previous section, the set of CH-molecules are arranged in a linear and saturated hydrocarbon compound like Eq. (2). Thus, it is required to set the number of molecules n in the compound, the set of hydrogen atoms H_i for each molecule, the carbon value σ for each molecule, and the set of bounds L_i at which the molecules act. Refer to Eqs. (1) to (4) if needed. As an example, Figure 7 shows a typical representation of the artificial hydrocarbon compound [15, 16] so-called structural formula, where H values represent the hydrogen values, exponent values at C’s represent carbon values, and L values are the bounds of molecules.

Figure 7.

Example of the Structural Formula Representation of an Artificial Hydrocarbon Compound.

3.2. Motion feedback using the RT-HOF approach

A vision-based sensor is proposed to use in mobile robots in order to retrieve motion feedback. In particular, a video camera located in front of the robot can be used. For instance, two adjacent images are obtained. Then, the real-time Hermite optical flow algorithm is used to compute relative displacements of objects between the two images. This procedure outputs a map of displacements. Since the relative displacements in the map are two dimensional vectors, decomposition in horizontal (u) and vertical (v) axes is computed for each displacement vector. To this end, the mean value per axis of all decompositions is calculated, obtaining the relative mean horizontal displacement and the relative mean vertical displacement values, û and $\widehat{v}$, respectively.

If one object at the time is in front of the video camera, then û represents the mean displacement of the object relative to the horizontal perspective of the image, that is, the displacement of the object from left-to-right or right-to-left in front of the robot, and $\widehat{v}$ represents the mean displacement of the object relative to the vertical perspective of the image, that is, the displacement of the object from near-to-far or far-to-near the robot. Thus, both û and $\widehat{v}$ values can be employed as motion feedback of the robot and inputs of the proposed controller.

4. Implementation on navigation mobile robots

This section introduces the case study employed for implementing and validating the proposed artificial organic controller based on the FMI system and the RT-HOF approach for motion feedback. First, a description of the robotics system is presented. Then, the design of the particular artificial organic controller is shown. Lastly, it describes the process of calibration of the motion feedback system.

4.1. Description of the robotics system

The robotics system used in this case study is based on a previous work reported in [19]. The mobile robot was implemented with the LEGO Mindstorms EV3 as both the mechanical and the electrical platform because it is easy and practical to use. The robot was built as tank configuration using two direct current (DC) motors and two rubber bands as actuators, and the robot was planned to be in a differential steering configuration. Also, a webcam was mounted on the robot to be used as the single vision-based sensor. For experimental purposes, the intelligent brick of the LEGO Mindstorms EV3 set was used as the interface between the mobile robot and a computer with MATLAB. The latter was employed to compute the control of the whole system. Both the webcam and the intelligent brick communicated with the computer via USB. Figure 8 shows a diagram of the robotic system [19]. The technical specifications of the robotic system are summarized in Table 1.

Figure 8.

Schematic of the Robotic System Used in the Case Study Based on [19].

Component	Description
Webcam sensor	1.3 M pixels, 1280 × 1024 max resolution
DC motor actuators	9V-input, 0.55A-nominal, 2.03 W mechanical power,; 4.95 W electrical power, 160 rpm max nominal
USB communication	9600 baud rate
Windows-based workstation	Intel Xeon Six-Core Processor 2.6 GHz, 32 GB RAM

Table 1.

Technical Specifications of the Robotic System [19]

4.2. Design of the artificial organic controller

The fuzzy-molecular inference system-based controller was developed using the methodology described in Section 3. Two inputs were selected as follows: relative mean horizontal displacement û and relative mean vertical displacement $\widehat{v}$. Also, the input domain space was partitioned in three fuzzy sets for each input like: “negative” (N), “zero” (Z), and “positive” (P). The proposed membership functions for both inputs û and $\widehat{v}$ are depicted in Figures 9 and 10, respectively.

Figure 9.

Input Membership Functions of u.

Figure 10.

Input Membership Functions of v.

In addition, Table 2 shows the set of fuzzy rules to this artificial organic controller, and Figure 11 shows the artificial hydrocarbon compound developed for this case study. Notice that there are three CH-molecules representing “move-backward” (M_B), “stop-motion” (M_S), and “move-forward” (M_F), and two identical compounds are used, one for each wheel in the robot.

$\widehat{u}$	$\widehat{v}$	yL	yR
N	N	MF	MS
N	Z	MF	MF
N	P	MS	MF
Z	N	MF	MS
Z	Z	MF	MF
Z	P	MS	MF
P	N	MF	MB
P	Z	MF	MF
P	P	MB	MF

Table 2.

Set of Fuzzy Rules Used in the Artificial Organic Controller

Figure 11.

Structural Formula of the Proposed Artificial Hydrocarbon Compound.

4.3. Calibration of the motion feedback

Previous work [19] proved that the real-time Hermite optical flow algorithm (Figure 5) can be used in soft and hard real-time frameworks, arising the opportunity to employ this technique in mobile robots.

In fact, the motion feedback system implemented in this work acquires images at 100ms of frame rate. Then, a pair of two adjacent images are analysed using the real-time Hermite optical flow algorithm of Figure 5. A map of the displacements is then obtained showing where objects in images are moving. In particular, an automated segmentation is done in order to get only the displacements that reach an empirical threshold, assuming that dynamic obstacles perform displacements greater than this threshold. For this case study, a threshold value of T = 1.0 was selected, using trial-and-error. Additionally, this case study supposes that one obstacle at most is in front of the robot and restricts the visual region of the robot at the centre of the frames (i.e., frames are divided in three vertical regions, selecting the central one). Later on, the automated segmentation finds the dynamic obstacle and computes its local displacements. To this end, the robotics system computes the mean values ûand $\widehat{v}$, as described in Section 3.2.

5. Experimental results

In order to validate the performance of the proposed controller, three tests were conducted about the actions taken from the mobile robot when dealing with dynamic obstacles. In addition, a comparison between the proposed controller and a rule-based controller previously developed in [19] was run. In the following section, these experimental results are summarized.

5.1. Tests on avoiding dynamic obstacles

This experiment consisted on steering the robot when dynamic objects are across the vision range of it. In particular, the robot avoids the obstacle in the opposite direction of its movement. In that sense, the proposed artificial organic controller is able to compute the relative displacement of the obstacle with respect to the robot, and then, the nature-inspired control strategy could calculate the power supply offering to the two motors, independently.

In advance, three tests were conducted. The first test considered the dynamic object (i.e., another robot) to move from left-to-right perpendicular to the direction of the robot, expecting that the robot steers to the left. The second test consisted in the same dynamic object, but moving from right-to-left perpendicular to the direction of the robot, expecting that the latter moves to the right. Finally, the third test considered a dynamic obstacle moving in a perpendicular direction with respect to the direction of the robot. Tables 3 to 5 summarize the mean relative horizontal û and vertical $\widehat{v}$ displacements computed with the RT-HOF algorithm, the output power supplies for both motors, left y_L and right y_R, in the robot, and the interpretation of the steering direction of the robot. In addition, Figures 12 to 14 show the trajectories obtained from the artificial organic controller in the robot during the three tests.

$\widehat{u}$	$\widehat{v}$	yL	yR	direction
0.00	0.00	15.1	15.1	straight
1.73	0.88	-15.1	15.1	left (fast)
1.22	2.09	-15.1	15.1	left (fast)
1.01	-0.03	0.0	8.7	left (slow)
0.00	0.00	15.1	15.1	straight

Table 3.

(Left-to-Right Test) Inputs and Outputs Values of the Artificial Organic Controller

$\widehat{u}$	$\widehat{v}$	yL	yR	direction
−2.34	-0.64	15.1	0.0	right (slow)
−0.86	-0.53	8.0	−0.2	right (slow)
−0.26	-1.18	4.2	4.5	straight
0.11	0.61	9.5	9.2	straight
−1.01	-0.36	15.1	0.0	right (slow)
-0.56	-0.26	1.3	0.2	stop

Table 4.

(Right-to-Left Test) Inputs and Outputs Values of the Artificial Organic Controller

$\widehat{u}$	$\widehat{v}$	yL	yR	direction
−1.39	0.28	15.1	−15.1	Right (fast)
−0.24	0.23	4.7	5.2	Straight
0.28	0.16	2.8	2.3	Straight
−0.48	0.22	1.2	0.3	Stop
−1.29	0.90	15.1	−15.1	Right (fast)

Table 5.

(Diagonal Test) Inputs and Outputs Values of the Artificial Organic Controller

Figure 12.

Test 1 (Left-to-Right): Trajectory Response of the Artificial Organic Controller.

Figure 13.

Test 2 (Right-to-Left): Trajectory Response Of The Artificial Organic Controller.

Figure 14.

Test 3 (Diagonal): Trajectory Response of the Artificial Organic Controller.

Observing at Figures 12 and 13, the correction of the direction is done fast in the robot while looking the dynamic obstacle in front. Reactiveness in the third test (Figure 14) is quite lower than in the first two tests. Currently, the robot can avoid obstacles in all situations. As an example, Figure 15 shows the visual range of the robot and the relative displacements obtained from the RT-HOF algorithm.

Figure 15.

Example of the Visual Range of the Robot and the Relative Displacements Obtained from the RT-HOT Algorithm.

5.2. Comparison of mobile robot navigation controllers

An inspection comparison was also done in order to highlight advantages and limitations on the performance of the proposed artificial organic controller. In fact, the controller in the previous work [19] reported a rule-based controller. Figures 16 to 18 show the response of the avoidance obstacles in the same configuration of tests: Dynamic object moves from left-to-right, from right-to-left and in diagonal, respectively.

Figure 16.

Test 1 (Left-to-Right): Trajectory Response of the Rule-Based Controller [19].

Figure 17.

Test 2 (Right-to-Left): Trajectory Response of the Rule-Based Controller [19].

Figure 18.

Test 3 (Diagonal): Trajectory Response Of The Rule-Based Controller [19].

First, the new artificial organic controller has better performance in terms of time reaction. Using the rule-based controller, the robot reacts in large time with respect to the distance of the dynamic object. It also considers a slow action that can be seen in the curvature of the depicted trajectory, and the robot does not return to the main direction (once the direction is corrected, the robot does not go back to the previous direction). The latter might be a disadvantage in the overall time of navigation, and the orientation of the robot could be lost.

On the other hand using the proposed artificial organic controller, the robot reacts in short time with respect to the distance of the dynamic object. In addition, once the steering is performed, the power supply to the wheels is recovered to the initial power. Moreover, the action is done fast (in contrast with the other controller) and the direction of the robot comes back close to the initial orientation. As an advantage, this controller might be used to surround dynamic obstacles that can minimize the overall time of the performance.

It is remarkable to say that the rule-based controller is a reactive approach in contrast to the deliberative approach performed with the artificial organic controller. From the perspective of agents systems, deliberative approaches can deal with uncertainty, imprecise, and noise in a suitable way [37]. In addition, the artificial organic controller can be more scalable, reliable, and easy to understand. In advance, using the hybrid fuzzy-molecular inference system gives an easy way to tune and interpret the performance of the controller, in comparison with the rule-based controller. Lastly, learnability is allowed in the proposed controller.

6. Conclusions

This chapter presents a novel artificial organic controller for mobile robot navigation, based on a hybrid fuzzy-molecular inference system as the nature-inspired control strategy and a single vision-based motion feedback using the real-time Hermite optical flow approach.

In fact, this artificial organic controller proposed the fuzzy-molecular inference system because it inherits characteristics from fuzzy inference systems like storage of knowledge from experts, dealing with uncertainty, and linguistic interpretation; but also from artificial hydrocarbon networks method that allows packaging information in molecular units, organizing of information, and dealing with aggressive uncertainty and noise. In addition, this proposed controller uses a real-time Hermite optical flow approach.

Experimental results showed that the proposed artificial organic controller can avoid dynamic obstacles. In addition, the comparison with a rule-based controller highlights that the proposed controller is easy to implement and to understand. Also, it is scalable and reliable in terms of uncertainty, imprecise, and noise.

For future work, this research is looking for implementing a learning mechanism in the artificial organic controller in order to automate the adaptation to the environment, and membership functions could be tuned automatically. To this end, complex environments should be tested to improve the real-time Hermite optical flow algorithm.