Errors obtained for different window size (10/79 images).
During the last years many agrochemicals (pesticides, herbicides, insecticides, so on) have been used for pest control in the world, and in Mexico too. This causes great damage to the health of people who has contact with pesticides and to the environment.
In recent years, large amounts of pesticides are used to achieve record harvests (for example, DDT has been used in Mexico for more than 50 years) , ). DDT is forbidden in the U.S., Canada and Europe because causes cancer. To reduce the pesticide amount it is necessary to locate precisely the distribution of insects and caterpillars. This task is very important not only for crops but also for monitoring forests. The forest health demands the efforts to fight threats of different kinds of insects and caterpillars , for example with Emerald ash borer, Agrilus planipennis Fairmaire .
For reducing the amounts of pesticides it is needed more precisely locate the distribution of pests in crops, in order to exterminate them effectively and to achieve the least damage. For this reason we need to develop an image recognition system intended to achieve the least possible exposure and contact of the persons who works in agriculture with any type of pesticide, as well as decreased crop areas where apply.
The task is not easy because many insects use the technique of mimesis (imitation) to hide them among the foliage. In this work the characteristics of the textures are used to find the insects. For this reason we are dedicated to development of computer vision system based on neural networks to locate the insects. This will permit us to avoid using large amounts of pesticides that are harmful to both farmers and crops.
Man's efforts in creation of computers have allowed the construction of machines able to solve automatically and quickly certain operations that are tedious. With the construction of the first computers in XX century, advances in science and technology have had a boom, allowing to easily implementing algorithms to solve many problems. Current developments are directed to study human capabilities as a source of new ideas for the design of new machines. Thus, the artificial intelligence is an attempt to discover and describe aspects of human intelligence that can be simulated by machines. This discipline was developed in recent years and has applications in some fields such as computer vision, theorem proving, information processing, neural networks as part of artificial intelligence among others methodologies.
After thorough investigation of related literature dedicated to neural networks and their applications, making a focus on pattern recognition, we selected the structure of LIRA neural classifier as the basis for our computer vision system.
It is interesting for us to investigate the recognition of the larvae. There is wide variety of larvae and caterpillars, and some of them are very dangerous. The most common of the insects are larvae in foliage of forests, gardens and trees.
The aim of this chapter is to propose an alternative for pest control in crops (in this case caterpillars and/or larvae) and avoid using large amounts of pesticides. It is also necessary to monitor the areas of localization of these pests to know how they are distributed. With this knowledge it is possible to dose the amounts of pesticides applied to the field.
The monitoring of forests and fields with different types of agricultural plants is very important. Mobile robot (1), for example, in Fig.1 may inspect the plants with a camera (2) on the board. It is possible to do the monitoring with airplanes and video observations of some regions of interest.
Artificial neural networks (ANN) are currently an active multidisciplinary field, involving researchers from different areas such as electronics, physics, mathematics, engineering, biology or psychology because they present a way to emulate certain characteristics of humans such as the ability to memorize and associate events to carry out certain tasks.
2. Artificial neural networks
Neural networks are a simplified model of human brain, which is an example of a system that is able to acquire knowledge through experience. An ANN is a system for the treatment of information whose basic unit is inspired by the fundamental cell of the human nervous system: the neuron.
Historically two groups of researchers have worked with artificial neural networks. One group was motivated by the aim of using ANNs to study and to model the different biological learning processes. A second group was motivated by the aim of obtaining highly effective machine learning algorithms, regardless of whether they reflect the biological processes.
The importance of ANNs is because they are an alternative solution to complex problems related to the recognition of shapes or patterns, prediction, coding, control and optimization.
The evolution of computer systems inspired by the human brain, and therefore endowed with certain "intelligence", is the combination of simple elements of process (neurons) interconnected and operated in parallel. These new methods are applied to resolve the different problems.
The first studies on ANNs started in the earliest 40s of XX century and from there have been strongly increasing till today, thanks to the works of many scientists and advances in hardware. Many scientists have developed the neural networks, due to this fact the neural networks have taken an important place in science.
Warren McCulloch and Walter Pitts made the first mathematical model of neuron . Pitts and McCulloch model is based on the idea that neurons operate using binary pulses. This model introduces the idea of a threshold function later used by many models such as, for example, the Hopfield neural networks (discrete bidirectional associative memory) , . This model generated great interest to provide sophisticated behavioral measures through simple calculations. The model is a key factor in learning ability.
Donald Hebb developed a mathematical model of learning procedure . Hebb studied on neurons and the classic conditions for learning. He developed a learning paradigm that now has his name: Hebbian learning.
Inspired by the work of McCulloch and Pitts, Minsky and Edmonds designed a machine with 40 neurons whose connections were adjusted according to a series of events that occurred when performing certain tasks. The machine was built with tubes, motors and relays, and was able to successfully model the behavior of a rat looking for food in the maze. Marvin Minsky and Papert proposed the results of artificial neural networks analysis , they argued that there were a number of fundamental problems with the network research program.
Albert Uttley began to develop new paradigms of artificial neural networks, creating a theoretical machine that consists of processing elements. The processing element was a linear separator that fit input parameters using the Shannon entropy measure. These machines have been used to simulate atmospheric phenomena, as well as adaptive recognition of patterns.
In 1957, Frank Rosenblatt generalized the of McCulloch-Pitts model of cells by adding learning, calling this model the PERCEPTRON . A two-level model was developed, which adjusted the weights of the connections between input and output levels in proportion to the error between the desired output and the output obtained. Rosenblatt attempted to extend their learning process to a three-level PERCEPTRON, but he did not found a solid mathematical method to train the hidden layer connections.
Bernard Widrow proposed an artificial neural network design very similar to PERCEPTRON, called Adaptive Linear Element or ADALINE . The ADALINE had two levels, much like the PERCEPTRON, adjusted the weights between the input and output levels according to the error between the expected value and the obtained output. The difference between these two models is very small, but the applications that are addressed to are very different. In 1960, Widrow and his colleagues mathematically proved that the error between the desired output and that obtained in certain circumstances can be minimized to the extent we want. Both the PERCEPTRON as the ADALINE maintained the problem of linear separability. The ADALINE has been used for adaptive signal processing, control systems and adaptive antenna systems.
Steinbuch was among the first researchers who developed the methods of information encoding in neural networks. Steinbuch networks were applied to handwriting recognition, to machinery fault diagnosis and control of multiple production processes.
Stephen Grossberg is the most influential and formal of all researchers in artificial neural networks . Grossberg made important studies on psychological processes and phenomena (mind) and biological (brain) of human information processing and tried to bring the two (mind and brain) in a unified theory. Grossberg's work included strict mathematical analysis that allowed the realization of new paradigms of neural networks. These made it possible to have direct access to information while operating in real time.
Shun-Ichi Amari combined the activity of biological neural networks with rigorous mathematical models of neural networks. One of these is a solution to the famous problem of credit allocation. His studies include the treatment of dynamic neural networks and randomly connected, competitive learning studies and mathematical analysis of associative memories.
James Anderson worked with a memory model based on the association of activation of the neuron synapse, performed a linear associative memory model, following the Hebb approach. He used a new method of error correction, and linear threshold function substituted by another ramp function, creating a new model called Brain-state-in-a-box.
Kunihiko Fukushima began work on artificial neural networks in the latest 60's, studying spatial and space-time vision systems, and brain . His most notable work was the creation of an artificial neural network paradigm for multi-view, which has been improved over time. Fukushima called his first project COGNITRON and made an improved version called NEOCOGNITRON .
A. Harry Klopf studied the relationship between the psychology of the mind and brain biology since 1969. He theorized that the neuron is a component that moves the brain searches for targets. It is an adaptive system that increases the effectiveness of excitatory synapses when depolarized, and increases the efficiency of inhibitory synapses when hyperpolarized.
Teuvo Kohonen began his research on artificial neural networks with random connections paradigms in 1971 . Kohonen's work focused on associative memories and correlation matrices, similar to the work of Anderson, Steinbuch and Piske. Kohonen and Ruohonen later extended the model of linear associative memory, which linearly independent vectors required for good performance. He later did research on teaching methods and developed the LVQ (Learning Vector Quantization), a competitive learning system.
Robert Hecht-Nielsen was the principal designer of one of the first neural computer dedicated to processing the neural network paradigms . The neuro-computer, the TRW MARK III, is supported by a DIGITAL VAX computer, and was commercially available from 1986.
John Hopfield described the method of state analysis of auto-associative networks , . He introduced an energy function in his studies of systems. Hopfield shows that you can build an energy equation that describes the activity of a single layer neural network in discrete time, and that this energy can be dissipated and the system converge to a local minimum. This analysis raised the interest to apply artificial neural network paradigms for difficult problems that conventional computers cannot solve. Hopfield extended his model to consider continuous time.
The artificial neural networks (ANNs) are computer systems, which mimics the computational abilities of biological systems by using a number of interconnected artificial neurons. Artificial neurons are simple emulations of biological neurons; they take the information from sensors or other artificial neurons, perform simple operations on the data and pass the result to other artificial neurons. ANNs operate by artificial neurons and have processing their data in this way. They use both the parallel logic (all neurons are functioning in the same layer) combined with the serial operations (information from one layer is transferred to neurons in another layer). The three main characteristics, which describe a neural network, and thereby contribute to their functional abilities, are: structure, dynamics and learning.
ANNs offer specific advantages of information processing, which makes it the technology of decision making in many application areas. These advantages include:
Fault tolerance via redundant information coding,
Easy integration into existing technology.
During the last few years we have investigated the neural classifiers that have demonstrated good results in recognition of different objects, for example, handwritten digits (on MNIST database), texture recognition in micromechanics, recognition of pin-hole relative position, etc.  - .
LIRA neural classifier permits us to create a new system to recognize larvae. This work is very important to reduce pesticide application in fields, forests, and so on  - . If we can localize the distribution of larvae we can reduce the pesticide quantity. The pesticides are very dangerous for people´s health. In the literature, there are different methods of larvae recognition , . This work is done to keep the trees healthy and to obtain good harvests. We propose using the LIRA neural classifier for larvae recognition during field inspection.
3. Image database
We used the image database from  (www.forestryimages.org) as color images (in this article we will present several photos from this database). We selected 79 images for the first image database and 55 images for the second image database to test our LIRA neural classifier. The first image database contains images with different number of larvae on the image (Fig.2). The number of larvae may vary from one to dozens in one image.
The second image database contains only one larva. The task of larvae recognition is very difficult because the caterpillars, larvae have different forms. The larvae have very different textures (Fig.3). Some of them contain bristles, others are smooth.
To work with this database of images we have to form two sets of images: one for system training and another set to test the system.
We used 79 images and divided them in two parts: for the training process and for recognition (for example, 10 images for training and 69 for recognition, or 20 for training and 59 for recognition). All images were presented in BMP format with resolution of pixels.
To investigate our neural classifier with these images we have marked the images. In Fig.6 we present an original image and the marked larvae. In this case we can train our system with “teacher” (supervised training), all larvae are marked with white (Fig. 6).
We will describe the results of recognition after LIRA neural classifier description and will analyze the possibility of improving of the obtained results.
4. LIRA neural classifier
There are two types of image recognition systems. The first type contains a feature extractor and a classifier (Fig.7). The feature extractor transforms the image into a parameter vector. Each component of this vector corresponds to a specific feature used to solve different problems of recognition.
The second type of image recognition system contains only the classifier. Systems for image recognition such as Rosenblatt's Perceptron , Fukushima´s Neocognitron  among others, belong to this type of system. These systems are characterized by the feature extractor that is incorporated in their internal structure.
Two major differences of this classifier are based on how an image is presented at the input of the classifier and the encoder. If the image is presented in binary form, i.e. the input values can be only black and white (binary), we call it the Binary LIRA. If the input image is presented with gray scale values, it is called the Grayscale LIRA. The LIRA neural classifier has been tested in handwritten digit recognition tasks , microscrews image classification , micro devices assembly tasks  and has demonstrated good results.
It is necessary to say that science and engineering of image and pattern recognition have developed rapidly in recent years. The digital image processing is a set of techniques applied to the digital representation of an image that makes up a scene, in order to recognize or classify some elements of interest to facilitate further analysis by a computer vision system . The image processing techniques are applied when it is necessary:
Improve or modify an image,
Highlight some aspect of the information contained therein,
Measure, compare or classify an item in the image,
Recognize image content.
The LIRA neural classifier is described in detail in  - . Here we give only the main characteristics of the LIRA neural classifier. It consists of 3 layers as perceptron of Rosenblatt. We included our modifications to the structure and algorithms of the perceptron . So, we can speak about 4 layers (Fig.8): sensor layer S, intermediate layer I, associative layer A, and response layer R. The sensor layer S in our modifications is scanned with window of size () that we define for a task. Inside of the window () we generate many small windows (). The position of the window we select randomly (position is the position of the left upper corner of the window). The range was [0, W] and [0, H]. Between S and I layer we randomly generate connections once and do not change them during the experiments with the neural classifier. The neurons in layer I can be ON and OFF neurons. In Fig.8, layer I, we present two ON-neurons and three OFF-neurons. ON and OFF-neurons work in a following manner:
where is the output value of the neuron; is input value of the neuron i; is the threshold of the neuron i. These neurons permit us to extract the features of the image. The neurons of associative layer A present the binary code of the image. Every neuron ai has the output only when all ONN and OFF neurons respond. If any neuron of ONN or OFF neurons has no answer (), the neuron ai has output equal to “0”. So in associative layer A we have a small number of active neurons (with output equal to”1”).
In many experiments which we made to resolve different recognition tasks it was demonstrated that the number of associative neurons may vary from various tens thousands to various hundreds thousands. Large number of neurons in associative layer permits us to improve the recognition rate. The acceptable time of calculations we obtain due to rare coding principle when the number of active neurons in associative layer A is much less than the total number of neurons. The calculations are made only for active neurons.
Connections between neurons of A layer and R layer (the rule of connecting is “all neurons of A layer are connected with all neurons of R layer”) have weights () that are changed during the training process. The rule of weight changing is the following. If the system recognizes the class correctly we do not change the weights. If the system recognizes the class incorrectly we change the weights to the incorrect class (decreasing them) and change the weights to the correct class (increasing them). The recognized class is defined with the neuron of layer R that has maximal excitation. In our task the layer R contains only two neurons that correspond to two classes: class of larvae and class of background.
Then with these rules we developed and programmed the computer vision system based on neural classifier LIRA for image recognition.
The work of the system is divided into two phases: the first phase is the training process of the artificial neural network using a larvae image base that contains larvae of different shapes and colors, with different amounts and with different positions. The second phase is a verification of the system with the other part of the image base of caterpillars (larvae).
The main objective of this investigation is design and programming of the computer vision system based on the LIRA neural classifier. This classifier is useful for the task of caterpillar recognition on crops, considering the different characteristics of larvae and diversity in size and shape that they may have.
The program of the neural model was written in C++ Borland 6 and the experiments were realized with a computer that contains a processor Intel Pentium Dual Core @2.20GHz with 1Gb RAM memory, Windows XP.
We can summarize the working algorithm of the system as follows. An image is presented as input to the LIRA neural classifier. Then the algorithm calculates the image code and it is processed with the LIRA neural classifier to obtain the output of the neural classifier, i.e., the class to be recognized by the classifier. It means the presence or absence of larvae/caterpillar(s) in the image. The image code is stored in memory for using in successive cycles of training process instead of the input image. This gives us possibility to save significantly the computing resources and time.
The methodology for the construction of the system is as follows:
Initialize the weights of connections between layers A and R.
Input the image to the input layer S of the system.
Encode the image by means of layer I to the feature vector.
Calculate the vector of characteristics of the layer A (obtain the properties of the image).
Get the class type recognized in output layer R.
Modify the weights Wij according to the obtained output.
Repeat step # 2 to # 6 until you reach the fixed number of cycles.
The training process is supervised training (for this purpose we used marked images). During this training process the system makes adjustments of the connection weights between A and R layers of internal structure of the LIRA neural network. In the training process a training set of images is presented. Every window (h x w) of these images must have a label associated with the class it represents. With this the classifier can be trained.
The training process consists of several cycles, in each cycle every image from the training set is presented to a classifier together with its label. After calculating one neuron with maximum excitation of the output layer R of the classifier is selected as response of the system. The correct class corresponding to the input image is read as the label associated with this. If the response of the system coincides with label, nothing to be done. If we have different answers, the system need to be trained more. The training process is repeated until a convergence criterion will be satisfied:
1 st After completing a certain number of cycles.
2 nd The error value is zero.
Once the algorithm satisfies any convergence criterion the training process ends.
To test the system or make the system verification we need to apply another set of images that were not participated in the training process. The objective of this stage is to verify the efficiency of the LIRA neural classifier. The purpose of this stage is obtaining of the system recognition rate. The response of the system is a neuron with maximum excitation values in the R layer (Fig.8). If is an excitation of i neuron in the R layer, than the recognized class has the maximum value (equation (2)).
Using the formula (2), R-layer neuron with the highest output value is detected, and called the winner neuron. This neuron represents the recognized class for the input image. For images from the recognition set we calculate number of errors (incorrect responses). According to the obtained results, the efficiency of the system for the proposed task may be validated.
5. Experimental results
In this article we present the results obtained with LIRA neural classifier in the recognition of larvae. The characteristics of LIRA neural classifier are the following: 32,000 associative neurons with two “ON” neurons and three “OFF” neurons. We did 150 cycles of the training process. The error number we calculated for all images. The number of errors is number of samples that were recognized incorrectly.
The tests were carried out in two phases:
Varying the number of images and allowing a fixed number of training cycles.
Varying the number of training cycles and fixing the number of images.
The first phase consisted of varying the number of images for training that is, taking a different number of images from the base 79. In this case it took 10/79 images (Number of Images for Training / Total Number of Images), 20/79 images for each test. In all these tests a number of training cycles was 150.
In the second phase the number of training cycles was varied, i.e., from 50 cycles to 300 cycles, leaving a fixed number of 10 images for training in both cases.
Finally, we calculated the percentage of error for each training process according to the following formula: the number of error divided on the total number of windows per image generated in each process.
|20 x 20||1084||10.53|
|40 x 40||488||11.1|
|60 x 60||274||10.59|
|80 x 80||187||12.27|
|100 x 100||128||11.78|
|120 x 120||93||18.74|
We decided to reduce the image database to preserve images, for example, with one larva on the image. So we prepared a new database with 55 images from the previous database. The results are presented in Table 3. We can see that in this case we have improvement in recognition. But, in the future it is necessary to improve the recognition quality of the LIRA neural classifier for the database of 79 images.
With these results for the first phase, we can do the experiments with changing the number of training cycles. You can see also that the window size for the image coding is very important too. For the window of (20 x 20) pixels the error number is less than for a window of (120 x 120) pixels. This is because for a relatively small window the image characteristics are presented in better way than for a much larger window.
|20 x 20||1922||10.53|
|40 x 40||826||11.13|
|60 x 60||428||10.73|
|80 x 80||389||11.82|
|100 x 100||205||12.63|
|120 x 120||168||13.21|
|20 x 20||1235||8.29|
|40 x 40||502||8.71|
|60 x 60||291||8.36|
|80 x 80||205||10.18|
|100 x 100||149||12.7|
|120 x 120||108||10.31|
We measured the response time for the system with the following features: windows with size (40 x 40) pixels, 150 cycles of training, 10/79 images for training, and 69/79 images for recognition. The time for the recognition of 69/79 images is only 9.26 s. Coding of images with smaller windows requires more time and uses more computer resources. So the response time achieved by the system for training is 0.292 s and for image recognition is 0.042 s for each sample.
|20 x 20||1112|
|40 x 40||474|
|60 x 60||269|
|80 x 80||187|
|100 x 100||128|
|120 x 120||108|
|140 x 140||77|
|160 x 160||66|
|180 x 180||40|
|200 x 200||34|
|20 x 20||1196|
|40 x 40||491|
|60 x 60||261|
|80 x 80||179|
|100 x 100||117|
|120 x 120||99|
|140 x 140||78|
|160 x 160||58|
|180 x 180||46|
|200 x 200||34|
So we can do the following conclusion about the LIRA neural classifier quality.
For the first experiments we selected different window size of pixels (Fig.8) that scanned the image from the database. For this purpose we selected 10 images from the database for training of the LIRA neural classifier and the rest of the images we used for recognition and calculation of errors. The best result of 10.53 %, i.e. the minimal number of errors, we obtained for a window of pixels. The recognition rate in this case is of 89.47%.
We have to explain that the absolute number of errors is maximal for the minimal window (20 x 20) pixels but in relation to the number of samples for every image the error number is minimal number. For example, for window (20 x 20) pixels we have 3750 samples for image. For window (200 x 200) pixels we have 24 samples. Every window we move with a step equaled to the half of the window size. So if the window has size of pixels, the movement step is 10 pixels. If the window size is pixels, the movement step is 100 pixels. The window size is very important in pattern recognition because from this area we collect the features (ON- and OFF-neurons). So if we have a smaller size of window the more precise description of the image we obtain.
We investigated the influence of the number of images in the training set on the recognition rate. The second step we made with 20 images in the training set and 59 images in the recognition set. We change the window size in the same way as in the previous experiment.
In this case the number of errors is increased for the sizes of windows (pixels and pixels) and in other cases the errors have almost the same value.
We can explain this by the fact that in the training set we had different images that instead of improving the results made them worse. This is connected with very different images of the database.
The recognition rate was improved to 91.71%. It is the best result that our algorithm demonstrated in this investigation.
A system for larva recognition is presented in this article. This system is based on LIRA neural classifier. The best result that we obtained in this investigation is about 10.53 % of errors for the database of 79 images (Table 1 and Table 2). In this case we have 89.47% of recognition rate. The worth result was 18.74 % of errors that means 81.26% of recognition rate. The best result for the database of 55 images is 8.29% (91.71% of recognition rate). For a task of this complexity level the result is not bad. But, the LIRA neural classifier can be improved and can demonstrate better results in the future.
7. Impact of the investigation
The prerequisite in the field before using LIRA technique is the necessity to train the LIRA neural classifier with real images of larvae, insects or/and caterpillars. With this training the users can adapt the LIRA neural classifier to new environment, new field conditions.
To reduce the errors in the proposed task it is needed to continue this investigation. Firstly, it is important to obtain more representative image database. The neural networks, especially the neural classifiers, can be trained better with increasing number of images.
Secondly, the further improvement of the computer vision system is connected with investigation and selection of the system parameters, for example, the number of ONN and OFF neurons in I layer of LIRA. The number of neurons in A layer can vary and has a great influence on the training process.
New ideas in classification and recognition, new methods of feature extraction can be combined with LIRA neural classifier in future.
This work was supported in part by the Projects PAPIIT IN110510-3, PAPIIT IN119610-3. Our acknowledgment to the authors of the image database www.forestryimages.org.