Multi-Object Recognition Using a Feature Descriptor and Neural Classifier

Abstract

In the field of object recognition, feature descriptors have proven to be able to provide accurate representations of objects facilitating the recognition task. In this sense, Histograms of Oriented Gradients (HOG), a descriptor that uses this approach, together with Support Vector Machines (SVM) have proven to be successful human detection methods. In this paper, we propose a scheme consisting of improved HOG and a classifier with a neural approach to producing a robust system for object recognition. The main contributions of this work are: First, we propose an improved gradient calculation that allows for better discrimination for the classifier system, which consists of performing a threshold over both the magnitude and direction of the gradients. This improvement reduces the rate of false positives. Second, although HOG is particularly suited for human detection, we demonstrate that it can be used to represent different objects accurately, and even perform well in multi-class applications. Third, we show that a classifier that uses a neuronal approach is an excellent complement to a HOG-based feature extractor. Finally, experimental results on the well-known Caltech 101 dataset illustrate the benefits of the proposed scheme.

1. Introduction

Computer vision is a discipline through which a machine is enabled in order to recognize the world around it using visual perception, allowing it to deduce the structure and properties of a three-dimensional world from one or more two-dimensional images. In this respect, Forsyth-Ponce and Ballard-Brown argue that computer vision refers to the construction of explicit and significant descriptions of physical objects from images [1, 2]. That is, computer vision enables a machine to extract and analyze spectral, spatial, and temporal information of the different objects contained within an image. While spectral information includes frequency (color) and intensity (grayscale), spatial information refers to aspects such as shape and position (one, two, and three dimensions) and temporal information comprising stationary aspects (presence and/or absence) and time-dependent (events, movements, and processes).

Due to this ability, tasks such as failure detection [3, 4]; verification [5, 6]; identification [7, 8]; tracking analysis [9, 10]; and recognition [11, 12] can be performed by a computer vision system.

The object recognition task is of particular interest to this research as it plays a significant role in a computer vision system and is necessary even in order to complete some of the tasks listed above. It is evident that there are an increasing number of areas and/or applications requiring object recognition, e.g., fruit sorting [13], face detection [14], people detection [15], face recognition [16], object tracking [17], automatic traffic sign recognition [18], and vehicle license plate recognition [7], among many others. Gonzales and Woods define object recognition as the task of organizing input data into previously defined classes, using significant features extracted from objects that are immersed in an image containing irrelevant details [19]. Considering this definition, it is evident that both feature extraction and classification are extremely important for an object recognition task to achieve its aim. The feature extraction process applies operations to an image in order to obtain information describing the objects it contains. Moreover, this information should be able to discriminate between different object classes. The goal of this process is to improve the effectiveness and efficiency of the classification process [20]. The classification process uses the information generated by the feature extractor to perform both phases comprising it, learning, (thereby creating a bank of models), and recognition (responsible for determining which objects belonging to the bank of models is present in analyzed image) [21].

The development of the work presented in this paper is motivated by the increasing demand and necessity of techniques and algorithms that efficiently perform tasks related to the recognition of objects, as well as their implementation in real applications, both industrial and research, where a visual perception system is required. Considering the above, this work proposes the design and implementation of an object recognition system using methods based on feature descriptor and neural classifier. We will focus on a type of descriptor called the feature descriptor. In their original form, these descriptors characterize shape in 2D images as histograms of edge pixels, HOG (Histograms of Oriented Gradients) algorithm belongs to this family of descriptors [22]. Two practical objectives have been defined to meet the proposal. First, study and development of an improved HOG algorithm that can accurately represent different objects. Second, to demonstrate that the performance of a neuronal approach in the task of classification and labeling of the data generated by HOG is competitive with the techniques currently used.

The HOG method for object representation, proposed by Dalal and Triggs [23], describes the appearance of local regions (objects) within an image by means of the distribution of intensity gradients or edge directions. For this purpose, the HOG method applies a similar principle to be used by different methods, such as edge orientation histograms [24], shape contexts [25], and scale-invariant feature transform (SIFT) [26], which counts the number of occurrences of gradient orientation in specific portions of an image but differs in that it is computed on a dense grid of uniformly spaced cells and uses overlapping local contrast normalization for improved accuracy.

HOG has demonstrated that it is capable of generating representations that provide discriminative information from the objects in an image using normalized representations of objects. Since it operates on local cells, it is invariant to geometric and photometric transformations. For improved accuracy, the local histograms can be contrast-normalized by calculating a measure of the intensity across a larger region of the image, called a block, and then using this value to normalize all cells within the block. This normalization results in better invariance to changes in illumination and shadowing. Furthermore, HOG is invariant to changes in the data background as well as object position.

Although, the HOG method is particularly suited for human detection in images [11, 12, 23, 27, 28, 29], in this paper we show that it can be used to represent different objects accurately, and even perform well in multi-class applications.

On the other hand, artificial neural networks (ANN) have been successfully used in a variety of classification tasks in real industrial, scientific, and business applications [13, 30, 31, 32, 33]. Several kinds of neural networks can be used for this purpose, but we decided to use feedforward multi-layer networks or multi-layer perceptron (MLP), which are the most widely studied and used neural network classifiers.

MLP offers features that make it a competitive alternative to various conventional classification methods [34]. First, adaptability, MLP is capable of developing its own feature representation. That is to say, MLP can organize data into the vital aspects or features that enable one pattern to be distinguished from another. Second, generalization, MLP has the ability to respond appropriately to input patterns different from those involved in the learning process. Third, MLP is a universal function approximator, thus it can approximate any function with arbitrary accuracy. Fourth, since MLP uses a nonlinear activation function, it is a nonlinear model, which makes it flexible in modeling real-world complex relationships.

1.1 Related work

Similar works with which our proposal has been compared, because they use descriptors with similar characteristics to ours, are described below. In [35], an object model is generative and probabilistic, so appearance, scale, shape, and occlusion are all modeled by probability density functions, which here are Gaussians. Learning is carried out using the expectation–maximization algorithm, presented in [36], which iteratively converges to learn an object category using the detecting regions and their scales, and then estimating the parameters of the above densities from these regions, such that the model gives a maximum-likelihood description of the training data. Recognition proceeds by first detecting features, and then evaluating these features through a process Bayesian, using the model parameters estimated in the learning.

Zhang et al. proposed a scheme that represents the local features of the object by means of PCA-SIFT method and the global features using shape context method [37]. Both sets of features are presented to two-layer AdaBoost training network. Boosting refers to the general method of producing a very accurate prediction rule by combining relatively inaccurate rules-of-thumb. It has been used widely in computer vision, particularly for object recognition. Layer 1 chooses as “good” features, those that have the best ability to discriminate the target object class from the nontarget object class. Then, layer 2 locates the final “good” features based on the distances between the most discriminant local features selected by layer 1. This two-layered boosting method produces two strong classifiers, which can then be used in a cascaded for recognition tasks.

On the other hand, in [38] Zhang et al. presented a scheme that represents images as distributions (signatures or histograms) of features extracted with different key point detectors and descriptors. The proposed scheme represents an object from the union of a detector with a descriptor. Two complementary local region detector types are used: The Harris-Laplace detector, which responds to corner-like regions, and the Laplacian detector, which extracts blob-like regions. At the most basic level, these two detectors are invariant to scale transformations only. To achieve invariance to other transformations, such as rotation or illumination, this scheme may include the descriptors SIFT, SPIN, and RIFT. For classification process, the authors use the well-known Support Vector Machines.

In [39], Leibe et al. propose a novel method for detecting and localizing objects of a visual category in cluttered real-world scenes. For the objects representation, this proposal introduces a basic object representation model known as Implicit Shape Model (ISM). ISM consists of a class-specific alphabet of local appearances that are prototypical for the object category, and of a spatial probability distribution that specifies where each codebook entry may be found on the object. Then, the object detection is implemented as a probabilistic Hough voting procedure from which hypotheses are found by a scale-adaptive Mean-Shift search.

More recently, Leptev showed how histogram-based image descriptors can be combined with a boosting classifier to provide a robust object detector [40]. Each feature is represented by a histogram of local image measurements within a region. For this purpose, HOG features are adopted and considered histograms of alternative image measurements such as color and second-order image derivatives. The HOG features are formed from orientation of local image gradient at each point using Gaussian derivatives of image computed for scale parameter defined. The histograms are normalized to the l1 unit norm. At the training, the features for normalized training images are computed, and apply AdaBoost to select a set of features and the corresponding weak classifiers optimizing classification performance.

The overall organization of the paper is as follows, after the introduction, we examine theoretical issues of HOG and MLP in Section 2. Section 3 presents the object recognition system based on HOG and MLP proposed in this paper. Experimental results are discussed in Section 4. Finally, Section 5 concludes the paper.

Table 1 shows a summary of the main features that the mentioned schemes and the one proposed in this paper possess.

Method	Feature extraction	Classifier
[35]	Detector of Kadir and Brady	Bayesian approach
[37]	PCA-SIFT/shape context	multi-layer Adaboost
[38]	SIFT/SPIN/RIFT	SVM
[39]	Hessian-Laplace/Harris-Laplace context	k-means/Agglomerative/RNN
[40]	Histogram-based descriptor	AdaBoost
Our proposal	HOG	MLP

Table 1.

Main features summary of the mentioned schemes and our proposal.

2. Theoretical background of HOG and MLP models

As mentioned earlier, we will focus on a type of descriptor called the feature descriptor. In general terms, this approach uses 4 steps to calculate a descriptor from an image, 1. An edge detector is applied to the image. 2. a basis point is chosen, which is a coordinate in the edge map, then a template is defined, mainly circular, centered at that point and it is divided into sections of the same size. These sections divide the image into regions, each of which corresponds to one dimension of the feature vector. 3. The value of a dimension is calculated as the number of edge pixels that fall into the region (a histogram that summarizes the spatial distribution of edges in the image relative to the chosen basis point). It is common to use the term “bin” to refer to the region in the image as well as the dimension in the feature vector. 4. Feature vector normalization.

With this approach, if the bins were small enough to each contain one pixel, then the histogram would be an exact description of the shape in the support region.

The HOG descriptor is a member of this family, it was proposed by Dalal and Triggs in [23] and extensively documented by Dalal in his PhD thesis, which was supervised by Triggs [41]. HOG offers a successful and popular object representation, particularly with human representation [22, 42]. HOG is inspired by SIFT, thus it can be regarded as a dense version of SIFT. Both algorithms are based on histograms of gradient orientations weighted by gradient magnitudes. However, there are important differences between these algorithms. First, these two algorithms differ slightly with regard to the type of spatial bins that they use, as HOG has a more sophisticated way of binning. Second, HOG computes the descriptors by means of blocks in dense grids at some single scale without orientation alignment, while in SIFT, descriptors are computed at sparse, scale-invariant key image points, and rotated to align orientation. Third, SIFT only computes the gradient histogram for patches around specific interest points obtained by taking the difference of Gaussians in the scale space (this is a local descriptor and SIFT features are usually compared by computing the Euclidean distance between them). HOG, on the other hand is computed for an entire image by dividing it into smaller cells and summing up the gradients over every pixel within each cell in an image (HOG is used to classify patches using classifiers such as SVM). Finally, SIFT is used for the identification of specific objects since the Gaussian weighting involved enables it to describe the importance of a particular point, while HOG does not have such a bias. HOG, therefore, is better suited to the classification task than SIFT.

The main idea behind the HOG descriptors is that the appearance and shape of a local object within an image can be described by the distribution of intensity gradients or edge directions. That is to say, the HOG features concentrate on the contrast of silhouette contours against the background. HOG is a window-based descriptor, whereby the window is typically computed by dense sampling over all image points.

In general, the HOG algorithm can be divided into 4 phases: gradient computation, orientation binning, descriptor blocks, and normalization blocks. In the first phase, the gradients are computed using Gaussian smoothing followed by discrete derivative masks. The experiments by Dalal and Triggs demonstrated that a simple 1-$D\left[-1,0,1\right]$ mask at none scale smoothing $\left(\sigma =0\right)$ works best. The second phase is responsible for dividing the gradient image into small-connected regions called cells (typical cell size is $6\times 6$ or $8\times 8$ pixels), and within each cell, a frequency histogram is computed representing the distribution of edge orientations within the cell. For this purpose, each pixel calculates a weighted function of the gradient magnitude (called vote, typically the magnitude itself is used) based on the orientation of the gradient element centered on it. Then, the edge orientations are quantized into q bin uniformly spaced over 0–180° when an unsigned gradient is used, or 0–360° when a signed gradient is used. In a third phase, groups of adjacent cells are considered as spatial regions called blocks, (typical block size is $2\times 2$ or $3\times 3$ cells), and using an overlap between blocks significantly improves the algorithm performance. The grouping of cells into a block is the basis for the grouping and normalization of histograms. Two classes of block geometries commonly used are square or rectangular (R-HOG), with associations of spatial cells in squares or rectangles, and circular blocks (C-HOG) partitioned into cells in log-polar fashion. Finally, the blocks defined in the previous phase are normalized. For this purpose, let $v$ be the unnormalized block, ${‖v‖}_k$ be its k-norm for $k=1,2$, and $ϵ$ be a small normalization constant to avoid division by zero [41], then the following schemes can be used:

1. $L2-\mathit{norm}$$\left(v\to v/\sqrt{{‖v‖}_2^2+{ϵ}^2}\right)$;

2. $L2-\mathit{Hys}$ ($L2-\mathit{norm}$ followed by clipping, limiting the maximum values of $v$ to 0.2, and renormalizing);

3. $L1-\mathit{norm}$$\left(v\to v/{‖v‖}_1+ϵ\right)$;

4. $L1-\mathit{\text{sqrt}}\left(v\to \sqrt{v/{‖v‖}_1+ϵ}\right)$.

The final descriptor is then the vector of all components of the normalized cell responses from all of the blocks in the detection window.

On the other hand, it is common to find a support vector machine (SVM) classifier to complement a feature extractor HOG within a scheme of object recognition. One purpose of this paper is to determine the performance of an object recognition system that uses HOG and neural network-based classifier, particularly a multi-layer perceptron (MLP).

MLP was derived from the neuronal model known as perceptron, which was presented by Rossenblatt in 1958 [43]. The perceptron is based on the model of McCulloch and Pitts [44] and presents a learning rule based on error correction. MLP is an ANN composed of an input layer, n hidden layers, and an output layer, all consisting of m type-perceptron neurons. MLP is the most studied neural network model, which can approximate any continuous nonlinear function arbitrarily well on a compact interval. Due to this property, MLP became popular in order to parametrize nonlinear models and with classification purposes. Furthermore, the characteristics of the MLP are well known to solve the problem that occurs when the data available in training are generally not sufficient to cover the variability of the object’s appearance. This problem is present in most recognition systems, including our own.

MLP belongs to the category of supervised classifiers. Then, with the training set, composed of p pairs of input–output vectors that define the behavior of the system that the ANN will adapt, is defined as

$$\left\{\left({\mathbf{x}}^1,{\mathbf{t}}^1\right),\left({\mathbf{x}}^2,{\mathbf{t}}^2\right),\dots ,\left({\mathbf{x}}^p,{\mathbf{t}}^p\right)\right\}=\left\{\left({\mathbf{x}}^{\mu },{\mathbf{t}}^{\mu }\right)|\mu =1,2,\dots ,p\right\}$$E1

where ${\mathbf{x}}^{\mu }={\left[x_i^{\mu }\right]}_n$ and ${\mathbf{t}}^{\mu }={\left[t_k^{\mu }\right]}_q$ are the input and target vectors, respectively.

MPL structure is defined as follows: It has an input layer of n units, one or more successive hidden layers of intermediate units, and a layer of q output units. Considering an MLP with r hidden layers, where ${\mathbf{x}}^{\mu }={\left[x_i^{\mu }\right]}_n$is the $\mu$-th input vector that belongs to the training set (defined by Eq. (1)). Then, ${\left({\mathbf{y}}^l\right)}^{\mu }={\left[{\left(y_{j^l}^l\right)}^{\mu }\right]}_{m^l}$, ${\mathbf{z}}^{\mu }={\left[z_k^{\mu }\right]}_q$ and ${\mathbf{t}}^{\mu }={\left[t_k^{\mu }\right]}_q$ represent the output of the l-th hidden layer, the output generated by MLP and the output target that MLP must generate, respectively, when ${\mathbf{x}}^{\mu }$ is presented to the network; where $l=\left(1,2,\dots ,r\right)$, $m^l$ and q indicate the number of units comprising the l-th hidden layer and the output layer, respectively.

Each unit’s output of layer l will be connected to the input of each unit in the layer l + 1, and a synaptic weight will be associated with each of these connections. Thus, the synaptic weights and thresholds of the first hidden layer are represented by ${\mathbf{W}}^1={\left[w_{j^{1}i}^1\right]}_{m^1\times n}$ and ${\mathbf{\theta }}^1={\left[{\theta }_{j^1}^1\right]}_{m^1}$, respectively. For remaining hidden layers, $l=\left(2,\dots ,r\right)$, the synaptic weights and thresholds are defined as ${\mathbf{W}}^l={\left[w_{j^l{j}^{l-1}}^l\right]}_{m^l\times m^{l-1}}$ and ${\mathbf{\theta }}^l={\left[{\theta }_{j^l}^l\right]}_{m^l}$. Finally, the synaptic weights and thresholds of the output layer are defined as ${\mathbf{W}}^{r+1}={\left[w_{{\mathit{kj}}^r}^{r+1}\right]}_{q\times m^r}$ and ${\mathbf{\theta }}^{r+1}={\left[{\theta }_k^{r+1}\right]}_q$.

MLP is a feed-forward neural network, and its operation is defined as follows. The output of first hidden layer is expressed mathematically as follows:

$$y_{j^1}^1=f\left(\sum _i{w}_{j^{1}i}^1\bullet x_i-{\theta }_{j^1}^1\right)$$E2

Whereas the output of the l-th hidden layer, with $l=\left(2,\dots ,r\right)$, is computed by

$$y_{j^l}^l=f\left(\sum _{j^{l-1}}{w}_{j^l{j}^{l-1}}^l\bullet y_{j^{l-1}}^{l-1}-{\theta }_{j^l}^l\right)$$E3

Finally, the MLP operation is defined as:

$$z_k=g\left(\sum _{j^r}{w}_{k{j}^r}^{r+1}\bullet y_{j^r}^r-{\theta }_k^{r+1}\right)$$E4

Typically, activation functions for units of hidden layers, $f\left(\bullet \right)$, are nonlinear; e.g., unipolar sigmoid function $1/\left(1+e^{-x}\right)$ and bipolar sigmoid function $\left(1-e^{-x}\right)/\left(1+e^{-x}\right)$. Activation functions of this type introduce the nonlinearity into the network and enable the MLP to approximate any nonlinear function with arbitrary accuracy. The activation functions of the units in the output layer, $g\left(\bullet \right)$ may be linear or nonlinear, depending on the application.

The MLP evolution based its success on the design of training algorithms that could minimize the error committed by the network by adequately and automatically modifying the values of the synaptic weights. In this sense, the training algorithm called backpropagation is the most popular used to adapt the MLP to a specific application because it is conceptually simple and computationally efficient. In 1974 Paul Werbos developed the basic principles of backpropagation, while developing his PhD thesis, by implementing a system that estimated a dynamic model for predicting social communications and nationalism [45]. In 1986 Rumelhart et al. formalized the backpropagation algorithm as a method that allows a type-MLP ANN to learn the association that exists between a set of input patterns and the corresponding classes [46]. Over time, backpropagation has become one of the most widely used neural learning methods, proving to be an efficient tool in applications of pattern recognition, dynamic modeling, sensitivity analysis, and the control of systems over time, among others.

The backpropagation algorithm looks for the minimum error function in weight space using the method of gradient descent. This method is applied for training the units of hidden layers of an MLP; that is to say, the basic idea of this algorithm states that updating the synaptic weights of the units of a layer depends on the error generated by the layer itself and errors generated by the following layers. The aforementioned is established by the mathematical structure of the backpropagation algorithm, which can be expressed as follows:

$${\mathit{\Delta w}}_{k{j}^r}^{r+1}=\epsilon \bullet \delta z_k\bullet y_{j^r}^r,$$E5a

where $\delta z_k=\left(t_k-z_k\right)g^{\prime }\left(u_k^{r+1}\right)$ and $u_k^{r+1}=\sum _{j^r}{w}_{k{j}^r}^{r+1}\bullet y_{j^r}^r-{\theta }_k^{r+1}$

$${\mathit{\Delta w}}_{j^l{j}^{l-1}}^l=\epsilon \bullet \delta y_{j^l}^l\bullet y_{j^{l-1}}^{l-1},$$E5b

where $\delta y_{j^l}^l=f{\prime }_{j^l}^l\left(u_{j^l}^l\right)\sum _k\delta z_k\bullet w_{k{j}^l}^{l+1}$ and $u_{j^l}^l=\sum _{j^{l-1}}{w}_{j^l{j}^{l-1}}^l\bullet y_{j^{l-1}}^{l-1}-{\theta }_{j^l}^l$; for $l=\left(2,\dots ,r\right)$

$${\mathit{\Delta w}}_{j^{1}i}^1=\epsilon \bullet \delta y_{j^1}^1\bullet x_i$$E5c

where $\delta y_{j^1}^1=f{\prime }_{j^1}^1\left(u_{j^1}^1\right)\sum _{j^2}\delta y_{j^2}^2\bullet w_{j^2{j}^1}^2$ and $u_{j^1}^1=\sum _i{w}_{j^{1}i}^1\bullet x_i-{\theta }_{j^1}^{1.}$$\epsilon$ is a small-valued constant that defines the learning rate of the network.

3. Object recognition based on histogram of oriented gradients and multi-layer perceptron

The Object recognition system based on the histogram of oriented gradients and multi-layer perceptron (ORS HOG-MLP) proposed in this paper presents the following contributions: (1) offers good performance in multiclass applications. (2) Determine the performance of an object recognition system that uses HOG and neural network-based classifier, particularly an MLP. (3) In order to improve the characterization process of the image, a modification that improves the properties representation of the gradient calculation algorithm is proposed.

The ORS HOG-MLP is an algorithm that is geared to automatic object recognition. Figure 1 shows the elements that are part of this system and the relationship that exists between them.

Figure 1.

Block diagram of ORS HOG-MLP.

Initially, a window detector process takes samples over the entire image, each sample has a fixed size and it is called a detection window. Better performance of the algorithm occurs when there is an overlap between detection windows. The image will be sampled several times and on each occasion, the detection window size will be different. This feature is intended to avoid image segmentation and get an algorithm robust to the object’s size.

Then the ORS HOG-MLP algorithm is applied to the extracted detection window. First, the gradient of an image at each pixel is computed. Let the discrete version of a detection window be represented as the matrix $\mathbf{A}={\left[a_{\mathit{ij}}\right]}_{\mathit{wi}\times \mathit{hi}}$; $\mathit{wi}$ and $\mathit{hi}$ are the width and height of the detection window, respectively, and $a$ represents the $\mathit{ij}$-th pixel value, $0\le a\le 2^n-1$, and $n$ is the number of bits necessary to represent the value of a pixel. Now, the aim of this process is to compute the magnitude and direction of the gradients for each pixel. For this purpose, we use a 1-$D\left[-1,0,1\right]$ mask at none scale smoothing $\left(\sigma =0\right)$ which is applied over all image pixels. The magnitudes and directions obtained by this process are grouped into two matrices ($\mathbf{M}$ and $\mathbf{Th}$); these matrices are defined as

$$\begin{array}{l}\mathbf{M}={\left[m_{\mathit{ij}}\right]}_{\mathit{wi}\times \mathit{hi}};m_{\mathit{ij}}=\sqrt{{\left[a_{i+1,j}-a_{i-1,j}\right]}^2+{\left[a_{i,j+1}-a_{i,j-1}\right]}^2}\\ \mathbf{Th}={\left[{\mathit{th}}_{\mathit{ij}}\right]}_{\mathit{wi}\times \mathit{hi}};{\mathit{th}}_{\mathit{ij}}={\mathit{tan}}^{-1}\frac{a_{i,j+1}-a_{i,j-1}}{a_{i+1,j}-a_{i-1,j}}\end{array}$$E6

Figure 2 shows the results of gradient computation process. A detection window includes information that does not belong to the interest object (information about other objects and background information), during the gradient computation process, this information corrupts the interest object.

Figure 2.

Results of gradient computation process. (a) Original image. (b) Gradient magnitude. (c) Pixel intensity proportional to the gradient direction (gradient direction). (d) Gradient magnitude improved. (e) Pixel intensity proportional to the gradient direction improved.

In order to improve the representation of the interest object properties by reducing this information, we propose to apply a threshold operation on the matrices $\mathbf{M}$ and $\mathbf{Th}$,

$$\begin{array}{c}{m}_{\mathit{ij}}=\left\{\begin{array}{cc}0,& \text{if}{m}_{\mathit{ij}}<\mathit{\text{umbral}}\_\mathit{grad}\\ n_{\mathit{ij}},& \text{otherwise}\end{array}\right.\\ {\mathit{th}}_{\mathit{ij}}=\left\{\begin{array}{cc}0,& \text{if}{m}_{\mathit{ij}}<\mathit{\text{umbral}}\_\mathit{grad}\\ {\mathit{th}}_{\mathit{ij}},& \text{otherwise}\end{array}\right.\end{array}$$E7

Figure 2 also shows the result obtained by applying this process on the matrices.

Now, both matrices $\mathbf{M}$ and $\mathbf{Th}$ are divided into $\mathit{cx}\times \mathit{cy}$ small-connected regions of $\mathit{px}\times \mathit{py}$ pixels, called magnitude cells and direction cells, and defined as ${\mathbf{cm}}_{\mathit{lk}}={\left[c\_m_{\mathit{ij}}^{\mathit{lk}}\right]}_{\mathit{px}\times \mathit{py}}^{\mathit{cx}\times \mathit{cy}}$ and ${\mathbf{cth}}_{\mathit{lk}}={\left[c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}\right]}_{\mathit{px}\times \mathit{py}}^{\mathit{cx}\times \mathit{cy}}$; $\mathit{cx}=\mathit{wi}/\mathit{px}$ and $\mathit{cy}=\mathit{hi}/\mathit{py}$ (see Figure 3). Then, each pixel calculates a weighted function of its gradient magnitude based on its gradient orientation to contribute to the building of the histogram of the cell to which it belongs. For this purpose, a vector of $p=9$ bins uniformly spaced over 0–180° is defined for the quantification process of gradient orientations: $\left\{{\mathit{bin}}_0,{\mathit{bin}}_1,\dots ,{\mathit{bin}}_8\right\}=\left\{{10}^{°},{30}^{°},\dots ,{170}^{°}\right\}$. The distance between bins is denoted by $d_{\mathit{\text{bins}}}={20}^{°}$.

Figure 3.

Details of orientation binning and descriptor block phases.

Let $\mathbf{MC}=\left[{\mathbf{Mc}}_{\mathit{lk}}\right]$ be a matrix of $\mathit{cx}\times \mathit{cy}$ elements, called bins matrix, where the ${\mathbf{Mc}}_{\mathit{lk}}=\left[{\mathit{mc}}_o^{\mathit{lk}}\right]$ element is a $p$-dimensional vector containing the corresponding bins $\left({\mathit{mc}}_o^{\mathit{lk}}\right)$ of the histogram of the lk-th cell, $o=0,1,\dots ,p-1$ (see Figure 3). Furthermore, considering two adjacent bins, ${\mathit{bin}}_o$ and ${\mathit{bin}}_{o+1}$, where ${\mathit{bin}}_o\le c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}\le {\mathit{bin}}_{o+1}$, the distances between $c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}$ and each of the bins are defined as $d_o=\left(c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}-{\mathit{bin}}_o\right)$ and $d_{o+1}=\left({\mathit{bin}}_{o+1}-c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}\right)$.

Thus, the result of this process is defined as

$$\begin{array}{c}{\mathit{mc}}_o^{\mathit{lk}}=\left\{\begin{array}{cc}{\mathit{mc}}_o^{\mathit{lk}}+\left(1-\frac{d_o}{d_{\mathit{\text{bins}}}}\right)\bullet c\_m_{\mathit{ij}}^{\mathit{lk}},& \text{if}{\mathit{bin}}_o<c\_{\mathit{th}}_{i\mathrm{j}}^{\mathit{lk}}\\ {\mathit{mc}}_o^{\mathit{lk}}+c\_m_{i\mathrm{j}}^{\mathit{lk}},& \text{if}{\mathit{bin}}_o=c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}\end{array}\right.\\ {\mathit{mc}}_{o+1}^{\mathit{lk}}=\left\{\begin{array}{cc}{\mathit{mc}}_{o+1}^{\mathit{lk}}+\left(1-\frac{d_{o+1}}{d_{\mathit{\text{bins}}}}\right)\bullet c\_m_{\mathit{ij}}^{\mathit{lk}},& \text{if}c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}<{\mathit{bin}}_{o+1}\\ {\mathit{mc}}_{o+1}^{\mathit{lk}}+c\_m_{\mathit{ij}}^{\mathit{lk}},& \text{if}{\mathit{bin}}_{o+1}=c\_{\mathit{th}}_{\mathit{ij}}^{\mathit{lk}}\end{array}\right.\end{array}$$E8

Figure 4 shows the result of applying orientation binning process.

Figure 4.

Result of orientation binning process. (a) Original image. (b) Gradient magnitude. (c) Gradient direction. (d) Histograms of the cells. (e) Descriptor blocks phase.

In the next process, sets of $2\times 2$ adjacent cells are grouped into spatial regions called descriptor blocks ($\mathrm{n}\_\mathbf{Mc}$ - number of $\mathbf{Mc}$ per block). In order to get better performance, the blocks are formed using an overlap of one cell on both x-axis and y-axis (see Figures 3 and 4e). The result of this process is the matrix $\mathbf{BL}=\left[\mathbf{bl}\right]$, composed of $\mathit{bx}\times \mathit{by}$ descriptor blocks; where $\mathit{bx}=1,2,\dots ,\mathit{cx}-1$, $\mathit{by}=1,2,\dots ,\mathit{cy}-1$ and a $\mathbf{bl}$ block is defined as

$${\mathbf{bl}}_{\mathit{bx},\mathit{by}}=\left\{{\mathbf{Mc}}_{\mathit{bx},\mathit{by}},{\mathbf{Mc}}_{\mathit{bx},\mathit{by}+1},{\mathbf{Mc}}_{\mathit{bx}+1,\mathit{by}},{\mathbf{Mc}}_{\mathit{bx}+1,\mathit{by}+1}\right\}$$E9

Then, each descriptor block must be normalized. For this purpose, we decided to use the $L2-\mathit{Hys}$ block normalization scheme, which first applied the $L2-\mathit{norm}$ (scheme)

$$\begin{array}{l}{\mathbf{bl}}_{\mathit{bx},\mathit{by}}=\frac{{\mathbf{bl}}_{\mathit{bx},\mathit{by}}}{\sqrt{{‖{\mathbf{bl}}_{\mathit{bx},\mathit{by}}‖}_2^2+{ϵ}^2}}\\ =\frac{{\mathbf{bl}}_{\mathit{bx},\mathit{by}}}{\sqrt{\sum _o{\left|{\mathit{mc}}_o^{\mathit{bx},\mathit{by}}\right|}^2+\sum _o{\left|{\mathit{mc}}_o^{\mathit{bx},\mathit{by}+1}\right|}^2+\sum _o{\left|{\mathit{mc}}_o^{\mathit{bx}+1,\mathit{by}}\right|}^2+\sum _o{\left|{\mathit{mc}}_o^{\mathit{bx}+1,\mathit{by}+1}\right|}^2+{ϵ}^2}}\end{array}$$E10

Thus, each $\mathit{mc}.$ (bin) of the ${\mathit{bl}}_{\mathit{bx},\mathit{by}}$ block is limited to a maximum value of 0.2: $\mathit{mc}=0.2,\left|\text{if}\mathit{mc}>0.2\right.$, and then it is renormalized again with $L2-\mathit{norm}.$

Eventually, the final object descriptor, $\mathbf{Od}$, is a $r$-dimensional vector of all components (bins) of the normalized cell responses from all of the blocks in the detection window

$$\begin{array}{l}\mathbf{Od}={\left[\mathit{od}\right]}_r=\left\{{\mathbf{bl}}_{1,1},{\mathbf{bl}}_{1,2},\dots ,{\mathbf{bl}}_{\mathit{bx},\mathit{by}}\right\}\\ =\left\{\left\{{\mathbf{Mc}}_{1,1},{\mathbf{Mc}}_{1,2},{\mathbf{Mc}}_{2,1},{\mathbf{Mc}}_{2,2}\right\},\left\{{\mathbf{Mc}}_{1,2},{\mathbf{Mc}}_{1,3},{\mathbf{Mc}}_{2,2},{\mathbf{Mc}}_{2,3}\right\},\dots ,\right.\\ \left.\left\{{\mathbf{Mc}}_{\mathit{bx},\mathit{by}},{\mathbf{Mc}}_{\mathit{bx},\mathit{by}+1},{\mathbf{Mc}}_{\mathit{bx}+1,\mathit{by}},{\mathbf{Mc}}_{\mathit{bx}+1,\mathit{by}+1}\right\}\right\}\\ =\left\{\left\{\left\{{\mathit{mc}}_0^{11},\dots ,{\mathit{mc}}_8^{11}\right\},\left\{{\mathit{mc}}_0^{12},\dots ,{\mathit{mc}}_8^{12}\right\},\left\{{\mathit{mc}}_0^{21},\dots ,{\mathit{mc}}_8^{21}\right\},\left\{{\mathit{mc}}_0^{22},\dots ,{\mathit{mc}}_8^{22}\right\}\right\},\right.\\ \left\{\left\{{\mathit{mc}}_0^{12},\dots ,{\mathit{mc}}_8^{12}\right\},\left\{{\mathit{mc}}_0^{13},\dots ,{\mathit{mc}}_8^{13}\right\},\left\{{\mathit{mc}}_0^{22},\dots ,{\mathit{mc}}_8^{22}\right\},\left\{{\mathit{mc}}_0^{23},\dots ,{\mathit{mc}}_8^{23}\right\}\right\},\dots ,\\ \left\{\left\{{\mathit{mc}}_0^{\mathit{bx},\mathit{by}},\dots ,{\mathit{mc}}_8^{\mathit{bx},\mathit{by}}\right\},\right.\left\{{\mathit{mc}}_0^{\mathit{bx},\mathit{by}+1},\dots ,{\mathit{mc}}_8^{\mathit{bx},\mathit{by}+1}\right\},\left\{{\mathit{mc}}_0^{\mathit{bx}+1,\mathit{by}},\dots ,{\mathit{mc}}_8^{\mathit{bx}+1,\mathit{by}}\right\},\\ \left.\left.\left\{{\mathit{mc}}_0^{\mathit{bx}+1,\mathit{by}+1},\dots ,{\mathit{mc}}_8^{\mathit{bx}+1,\mathit{by}+1}\right\}\right\}\right\}\end{array}$$E11

where $r=\mathit{bx}\times \mathit{by}\times \mathrm{n}\_\mathbf{Mc}\times p$ and ${\mathit{od}}_0={\mathit{mc}}_0^{11},{\mathit{od}}_1={\mathit{mc}}_1^{11},\dots ,{\mathit{od}}_8={\mathit{mc}}_8^{11},{\mathit{od}}_9={\mathit{mc}}_0^{12},\dots ,{\mathit{od}}_{17}={\mathit{mc}}_8^{12},\dots ,{\mathit{od}}_{r-9}={\mathit{mc}}_0^{\mathit{bx}+1,\mathit{by}+1},{\mathit{od}}_{r-1}={\mathit{mc}}_8^{\mathit{bx}+1,\mathit{by}+1}$

Figure 5 shows the sequence of processes that generate the HOG descriptor of the object.

Figure 5.

Sequence of processes for generating the HOG descriptor of the object.

Now, the MLP belonging to ORS HOG-MLP must be adapted to be able to recognize the interest objects. For this purpose, the processes described above are applied to $q$ interest objects and each descriptor obtained is associated with its corresponding class, $c={\left[c\right]}_k$; thus, the training set of MLP is defined as

$$\left\{\left({\mathbf{Od}}^1,{\mathbf{c}}^1\right),\left({\mathbf{Od}}^2,{\mathbf{c}}^2\right),\dots ,\left({\mathbf{Od}}^q,{\mathbf{c}}^q\right)\right\}=\left\{\left({\mathbf{Od}}^{\mu },{\mathbf{c}}^{\mu }\right)|\mu =1,2,\dots ,q\right\}$$E12

The MPL structure is established as follows: It has one hidden layer of $h$ units, the input layer has $r$ units and the output layer has $v$ units. Considering the training set, the backpropagation learning algorithm is used to generate the bank of models, which includes the vital information of the objects that integrate the training set. Essentially, the bank of models is present in the synaptic weights, ${\mathbf{W}}^1={\left[w_{\mathit{jo}}^1\right]}_{h\times r}$and ${\mathbf{W}}^2={\left[w_{\mathit{kj}}^2\right]}_{v\times h}$, that define the connections between neurons that integrate the MLP.

Finally, the operation process of ORS HOG-MLP, when the ${\mathbf{Od}}^{\mu }$ object is presented, is defined as

$$c_k^{\mu }=g\left(\sum _{j=0}^{h-1}{w}_{\mathit{kj}}^2\bullet f\left(\sum _{o=0}^{r-1}{w}_{\mathit{jo}}^1\bullet {\mathit{od}}_o^{\mu }-{\theta }_j^1\right)-{\theta }_k^2\right)$$E13

where $k=0,2,\dots ,v$.

4. Experimental results and discussion

This section is intended to measure the ORS HOG-MLP performance through a set of experiments. In the first experiment, system performance is analyzed when it is adapted to recognize only one object class and was conducted on several object classes. In the second experiment, the system behavior is analyzed when it is configured to recognize multiple classes. Finally, our scheme’s performance is with other results reported in the literature.

Before we present and discuss the data obtained from experiments, let us introduce concepts and methods used to measure ORS HOG-MLP performance. Let $O$ be an object presented to ORS HOG-MLP and the system generates the class $c^0$ to indicate that the object does not belong to its bank of models, then

A True Positive (TP) occurs when the class generated by the system is ${\mathbf{c}}^{\mu }$ and $O$ belongs to the class ${\mathbf{c}}^{\mu }$, for $\mu =2,\dots ,q$; this indicates a successful classification.

A true negative (TN) occurs when the class generated by the system is ${\mathbf{c}}^0$ and $O$ does not belong to the bank of models; this indicates a successful rejection.

A False Positive (FP) occurs when the class generated by the system is ${\mathbf{c}}^{\mu }$ and $O$ does not belong to this class, for $\mu =2,\dots ,q$; this indicates an incorrect classification.

A False Negative (FN) occurs when the class generated by the system is ${\mathbf{c}}^0$ and $O$ belongs to the bank of models; this indicates an incorrect rejection.

From these concepts, objective methods, which give information concerning system performance are obtained.

True positive rate (TPR). TPR determines the sensitivity of the system, i.e. it measures the proportion of successful classification obtained by system; TPR is defined as

$$\mathrm{TPR}=\frac{\text{number of}\mathrm{TP}}{\text{number of}\mathrm{TP}+\text{number of}\mathrm{FN}}$$E14

False Positive Rate (FPR). FPR indicates the proportion of wrongly classified objects; FPR is defined as

$$\mathrm{E}\left(u\right)=\left(\text{FPPW}\left(u\right),\mathrm{FNR}\left(u\right)\right)$$E15

Accuracy (ACC). ACC is used to evaluate the tendency of the system to ascertain correct TPs, defined as

$$\mathrm{ACC}=\frac{\text{number of}\mathrm{TP}+\text{number of}\mathrm{TN}}{\text{number of}\mathrm{TP}+\text{number of}\mathrm{TN}+\text{number of}\mathrm{FP}+\text{number of}\mathrm{FN}}$$E16

False negative rate (FNR). FNR indicates the miss rate of the system, defined as

$$\mathrm{FNR}=\frac{\text{number of}\mathrm{FN}}{\text{number of}\mathrm{TP}+\text{number of}\mathrm{FN}}=1-\mathrm{TPR}$$E17

False positives per window (FPPW). FPPW indicates the number of errors by detection window and it is defined as

$$\text{FPPW}=\frac{\text{number of}\mathrm{FP}}{\mathrm{N}}$$E18

where N is the total number of windows processed. Finally, the proposed MLP classifier returns a real value for each detection window, this value is thresholding with a fixed value $u$ in order to determine whether or not it is an object belonging to a class of bank of models. Thus, FNR and FPPW are dependent functions of $u$, allowing the plotting of evaluation curves ROC (Receiver Operating Characteristic) [47], Eq. (19), that show the tradeoff between the miss rate and the FPPW for each $u$.

$$\mathrm{E}\left(u\right)=\left(\text{FPPW}\left(u\right),\mathrm{FNR}\left(u\right)\right)$$E19

The Caltech 101 dataset, collected by Fei-Fei et al. [48], was used for benchmarking our proposal. For both training and operation phases of the system, positive images (those that contain only interest objects) and negative images (those that do not contain objects of interest or background) were generated.

The Caltech 101 dataset consists of a set of 1179 negative images for the system training phase $\left(I_{-}^{\mathit{\text{train}}}\right)$ and a set of 1179 negative images for the system operation phase $\left(I_{-}^{\mathit{\text{test}}}\right)$. The number of positive images for training and operation phases is $\left(W_{+}^{\mathit{\text{train}}\_\gamma }\right)$ and $\left(W_{+}^{\mathit{\text{test}}\_\gamma }\right)$, respectively, varies for each class (see Table 2).

Data set	Positive images; V-Scan
	Training $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{\gamma }}\right)$	Testin $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{\gamma }}\right)$
Airplane	200 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cA}}\right)$	200 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cA}}\right)$
Butterfly	40 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cB}}\right)$	40 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cB}}\right)$
CarSide	86 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cC}}\right)$	86 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cC}}\right)$
Chair	30 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cD}}\right)$	30 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cD}}\right)$
Electric guitar	30 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cE}}\right)$	30 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cE}}\right)$
Faces	100 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cF}}\right)$	100 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cF}}\right)$
Helicopter	40 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cG}}\right)$	40 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cG}}\right)$
Horses	85 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cH}}\right)$	85 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cH}}\right)$
Ketch	50 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cI}}\right)$	50 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cI}}\right)$
Laptop	40 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cJ}}\right)$	40 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cJ}}\right)$
Motorbikes	200 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cK}}\right)$	200 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cK}}\right)$
Piano	45 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cL}}\right)$	45 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cL}}\right)$
Revolver	40 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cM}}\right)$	40 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cM}}\right)$
SoccerBall	30 $\left(W_{\mathbf{+}}^{\mathit{\text{train}}\_\mathit{cN}}\right)$	30 $\left(W_{\mathbf{+}}^{\mathit{\text{test}}\_\mathit{cN}}\right)$

Table 2.

Positive images per class.

In order to demonstrate the performance of our proposal when only one object needs to be identified, in first experiment, ORS HOG-MLP was adapted to individually identify each class of objects in Table 2. With the intention of adding robustness to the system, from $I_{-}^{\mathit{\text{train}}}$ and $I_{-}^{\mathit{\text{test}}}$, additional negative windows for training and testing of the system, $W_{-}^{\mathit{\text{train}}}=1743$ and $W_{-}^{\mathit{\text{test}}}=1909$, respectively, were generated. Thus, the training and testing of the final sets of negative images were defined as: $I{\prime }_{-}^{\mathit{\text{train}}}=I_{-}^{\mathit{\text{train}}}+W_{-}^{\mathit{\text{train}}}=2922$, and $I{\prime }_{-}^{\mathit{\text{test}}}=I_{-}^{\mathit{\text{test}}}+W_{-}^{\mathit{\text{test}}}=3288$.

Then, negative images are associated with class 0, and positive images of the object under study, defined by $\gamma$, to class 1. The training set is defined as $\left\{\left(\left\{I{\prime }_{-}^{\mathit{\text{train}}}\right\},{\mathit{c}}^0\right),\left(\left\{W_{+}^{\mathit{\text{train}}\_\gamma }\right\},{\mathit{c}}^1\right)\right\}$, this set is presented to ORS HOG-MLP in order to adapt it to recognize the object $\gamma$. On the other hand, the ORS HOG-MLP performance is evaluated by applying the recognition phase to sets $I{\prime }_{-}^{\mathit{\text{test}}}$ and $W_{+}^{\mathit{\text{test}}\_\gamma }$. This process was repeated for all objects belonging to Table 2.

The HOG algorithm parameters were adjusted as follows: The number of cells by detection window varies depending on the object shape, $9$ bins uniformly spaced over 0–180° are defined, the size of blocks is of $2\times 2$ adjacent cells and overlap of one cell in both $x$-axis and $y$-axis is used. The MLP is defined with the following features: The hidden layer has 5 neurons, the activation functions of neurons in the hidden layer and output layer are sigmoid, and random initial weights in the range of $\left[-\mathrm{0.25,0.25}\right]$, neurons bias is −1.0, learning rate $\epsilon =0.01$, and for each object, 20,000 iterations were carried out to train the network. Figure 6 shows some examples of detections, and Figure 7 and Table 3 summarize the results of the first experiment.

Figure 6.

Examples of detection results on the Caltech 101 dataset. Detected objects are enclosed in rectangles.

Figure 7.

Performance of our proposal based on evaluation curves ROC: (a) Airplane class, (b) Butterfly class, (c) Motorbikes class.

Data set	Performance					Number; of cells
	TPR	FPR	ACC	FP	FN
Airplane	0.955	0.00097	0.9963	3	9	16x8
Airplane	0.945	0.00060	0.9960	2	11	8x8
Butterfly	0.45	0.00130	0.9916	4	22	12x8
Butterfly	0.45	0.00190	0.9910	6	22	8x8
Carside	0.767	0.00097	0.9927	3	20	12x8
Carside	0.732	0.00097	0.9918	3	23	8x8
Chair	0.133	0.0	0.9916	0	26	12x8
Chair	0.233	0.00032	0.9923	1	23	8x8
Electric guitar	0.400	0.00032	0.9939	1	18	12x8
Electric guitar	0.400	0.00032	0.9939	1	18	8x8
Faces	0.960	0.00010	0.9987	0	4	16x8
Faces	0.760	0.00010	0.9924	0	24	8x8
Helicopter	0.375	0.00356	0.9884	11	25	16x8
Helicopter	0.500	0.00453	0.9891	14	20	8x8
Horses	0.741	0.0	0.9930	0	22	12x8
Horses	0.823	0.00388	0.9914	12	15	8x8
Ketch	0.680	0.00226	0.9926	7	16	10x10
Ketch	0.740	0.00064	0.9952	2	13	8x8
Laptop	0.650	0.00032	0.9952	1	14	16x16
Laptop	0.625	0.00064	0.9945	2	15	8x8
Motorbikes	0.960	0.00032	0.9972	1	8	16x8
Motorbikes	0.980	0.00001	0.9987	0	4	8x8
Piano	0.800	0.00064	0.9964	2	9	12x8
Piano	0.777	0.0	0.9968	0	10	8x8
Revolver	0.700	0.00194	0.9942	6	12	16x8
Revolver	0.675	0.00129	0.9945	4	13	8x8
Soccerball	0.241	0.00032	0.9926	1	22	12x12
Soccerball	0.241	0.00064	0.9923	1	22	8x8
Watch	0.660	0.00226	0.9871	7	34	12x8
Watch	0.700	0.00259	0.9880	8	30	8x8

Table 3.

Results of first experiment.

Considering the results shown in Table 3, the average value of TPR and FPR parameters, 0.6387 and 0.001228, respectively, show that the probability of correctly classifying an object is approximately 64%, and the probability of misclassifying an object is less than 1%. Meanwhile, the ACC parameter indicates that the system accuracy is over 98% (e.g., for motorbikes, corresponding to 198 out of 200 correct detections with 2 false positives). These results also show that using a detection window size of $8\times 8$ or $16\times 8$ cells does not significantly affect system performance.

In the definition of MLP structure, tests were performed using 5, 10, 15, and 20 neurons in the hidden layer. The results show variations of 0.5% of ${10}^{-3}$ FPPW, so we decided to work with the smaller number of neurons to reduce the computational cost of the system.

In second experiment, ORS HOG-MLP is configured as a multi-class recognition system. Therefore, the system is trained to recognize several groups of objects belonging to Table 2, where the number of objects by group can be 2, 3, 4, or 6. Thus, the training set is defined as $\left\{\left(\left\{I{\prime }_{-}^{\mathit{\text{train}}}\right\},{\mathit{c}}^0\right),\left(\left\{W_{+}^{\mathit{\text{train}}\_\gamma }\right\},{\mathit{c}}^1\right),\dots ,\left(\left\{W_{+}^{\mathit{\text{train}}\_\gamma }\right\},{\mathit{c}}^{\mu }\right)\right\}$, $\mu =2,3,4,6$, and $\gamma$ identifies the different objects belonging to a group; e.g. $\left\{\left(\left\{I{\prime }_{-}^{\mathit{\text{train}}}\right\},{\mathit{c}}^0\right),\left(\left\{W_{+}^{\mathit{\text{train}}\_\mathit{cA}}\right\},{\mathit{c}}^1\right),\left(\left\{W_{+}^{\mathit{\text{train}}\_\mathit{cC}}\right\},{\mathit{c}}^2\right),\left(\left\{W_{+}^{\mathit{\text{train}}\_\mathit{cK}}\right\},{\mathit{c}}^3\right)\right\}$ is a group that includes the airplane, carside, and motorbike objects, and the negative images are associated with the class 0. For this group, the recognition phase uses $I{\prime }_{-}^{\mathit{\text{test}}}$, $W_{+}^{\mathit{\text{test}}\_\mathit{cA}}$, $W_{+}^{\mathit{\text{test}}\_\mathit{cC}}$, and $W_{+}^{\mathit{\text{test}}\_\mathit{cK}}$ to evaluate system performance.

The configuration parameters of the HOG algorithm and MLP are the same as those used in Experiment 1. Table 4 shows the results of this experiment. The results of second experiment show that system performance is not affected when operating in multiclass mode. This is deduced from the average values of TPR and ACC, which indicate that the probability of correctly classifying an object is approximately 68% and that the system accuracy fluctuates between 97% and 99%. e.g., for group2 = {airplane¹, carside², electricguitar³, motorbikes⁴, revolver⁵, watch⁶}, using a block of 8×8 cells, ORS HOG-MLP presents an ACC = 98% for airplane¹ (corresponding to 196 out of 200 correct detections with 4 false positives), and it presents an ACC = 99% for motorbikes⁴ (corresponding to 198 out of 200 correct detections with 2 false positives). Those results make the robustness of the proposed system evident, when it is used in multi-object recognition applications.

Group	Objects; number	Performance
		TPR^1	ACC^1	TPR^2	ACC^2	TPR^3	ACC^3	TPR^4	ACC^4	TPR^5	ACC^5	TPR^6	ACC^6	#Cells
1	6	0.025	0.976	0.03	0.979	1.000	0.989	0.660	0.983	0.75	0.985	0.103	0.980	8x8
2	6	0.930	0.988	0.670	0.983	0.100	0.983	0.970	0.991	0.600	0.987	0.630	0.981	8x8
2	6	0.960	0.993	0.770	0.989	0.030	0.986	0.995	0.996	0.680	0.992	0.630	0.984	12x8
3	6	0.960	0.993	0.180	0.985	0.130	0.987	0.970	0.99	0.450	0.988	0.975	0.994	8x8
4	4	0.935	0.994	0.710	0.990	0.350	0.990	0.955	0.995	—	—	—	—	8x8
4	4	0.945	0.993	0.670	0.988	0.28	0.987	0.975	0.995	—	—	—	—	12x8
5	3	0.920	0.993	0.700	0.989	0.950	0.995	—	—	—	—	—	—	8x8
5	3	0.930	0.994	0.690	0.990	0.960	0.996	—	—	—	—	—	—	16x8
6	2	0.950	0.995	0.980	0.997	—	—	—	—	—	—	—	—	8x8
6	2	0.975	0.997	0.980	0.997	—	—	—	—	—	—	—	—	12x8
7	2	0.930	0.997	0.600	0.994	—	—	—	—	—	—	—	—	8x8

Table 4.

Results of second experiment.

The groups are integrated as follows: group1 = {butterfly¹, hair², faces³, ketch⁴, laptop⁵, soccerball⁶}, group2 = {airplane¹, carside², electricguitar³, motorbikes⁴, revolver⁵, watch⁶}, group3 = {airplane¹, butterfly², chair³, faces⁴, laptop⁵, motorbikes⁶}, group4 = {airplane¹, carside², helicopter³, motorbikes⁴}, group5 = {airplane¹, carside², motorbikes³}, group6 = {airplane¹, motorbikes²}, group7 = {faces ¹, revolver²}.

Finally, the performance of the proposed scheme is compared with several object recognition schemes that have been cited frequently in related studies in this area. Table 5 shows the results of this comparison.

Method		Data set
		Motorbikes	Cars
Object recognition scheme proposed in [35]		92.5%	88.5%
Object recognition scheme proposed in [37]		99.0%	—
Object recognition scheme proposed in [38]		98.5%	95.0%
Object recognition scheme proposed in [39]		97.4%	96.7%
Object recognition scheme proposed in [40]		89.6%	66.3%
HOG + MLP (our proposal, one object)		99.87%	99.27%
HOG + MLP (our proposal, multi-objects)	3 objects	99.6%	99.0%
	4 objects	99.5%	99.0%
	6 objects	99.1%	98.9%

Table 5.

Performance comparison on accuracy for object recognition for two of the Caltech categories with other methods from the literature.

Table 5 shows a comparison of our scheme’s performance with other results reported in the literature. With an ACC performance of 99%, our method presents a significant improvement over the previous result. The table also shows that the performance of the proposed scheme is not significantly affected when used in multi-object recognition applications. Zhang et al. reported a similar performance with 99% using a scheme composed of PCA-SIFT method and shape context method for object representation and two-layer AdaBoost network as classification technique [37]. However, due to the methods and techniques used by Zhang et al., this scheme presents a computational cost relatively greater than that presented by our proposal.

5. Conclusions

Although object recognition is a very active area of research, it is still considered an overly complex task due to the following difficulties: 1. Objects of the same class with high variability in appearance. A class of objects can integrate elements with variations in shape, color, and texture. In addition, multiple factors such as position, lighting, and occlusions, among others, can increase these differences. 2. The lack of reference images for the training phase of the classifier. The available data are generally not enough to cover the variability in appearance of objects. Furthermore, there may be significant differences in the conditions of training and system operation.

This research has placed special emphasis on the study of HOG algorithms for the feature extraction stage. This is because HOG has demonstrated that using normalized representations of objects can generate representations that provide discriminative information from the objects in an image. Furthermore, since HOG operates on local cells, it is invariant to geometric and photometric transformations as well as to changes in background and object position. On the other hand, it seeks to exploit the well-known features of the MLP to solve the problem that occurs when the limited data available during training are generally not enough to cover the variability in the appearance of objects.

It is important to emphasize that the proposed improvement in the step of calculating the HOG algorithm gradient reduces the rate of false positives. It was also demonstrated that HOG can accurately represent different objects and offers good performance in multiclass applications. Finally, we show that a classifier that uses a neuronal approach is an excellent complement to a HOG-based feature extractor.

It is the intention of this working group to use the proposed system in autonomous systems applications through its modeling on reconfigurable logic.