Physical principles and most remarkable characteristics of the main systems used for the geometrical characterization of tree crops and their main advantages and disadvantages.
Precision variable rate spray is one of the research hotspots in the field of modern agriculture spraying applications. Variable rate spraying of the canopy allows growers to apply adjusted volume rate of pesticides to the target, based on canopy size, and to apply plant protection products in an economical and environmentally sound manner. In the field of pesticide application, knowledge of the geometrical characteristics of plantations will guarantee a better adjustment of the dosage of the agrochemicals applied. This technology is integrated with intelligent real-time sensors, which have a high potential for agricultural precision spray applications. This book chapter presents the foundations and applications in agriculture of the primary systems used for real-time spray target detection of the geometrical characterization of tree plantations. Systems based on infrared, ultrasonic, light detection and ranging (LIDAR), and stereo vision sensors were discussed, respectively, on their performances to detect spray targets. Among them, laser scanners and stereo vision systems are probably the most promising and complementary techniques for achieving three-dimensional (3D) pictures and maps of plants and canopies. The advantages of data fusion applied in real-time target detection and its accuracy in density estimation of the plants were stressed.
- variable-rate sprayer
- infrared sensor
- ultrasonic sensor
- stereo vision sensor
- LIDAR sensor
- data fusion
In agriculture, chemicals are often essential for crop protection. Pesticide spray applications have facilitated high-quality and abundant products for ornamental nurseries and orchards. However, despite these achievements, conventional sprayers are grossly inefficient because the same amounts of chemicals are discharged continuously in the field regardless of the plants present, canopy structure, or leaf foliage density. Canopies are spatially variable, and a uniform dose may not be adequate for the entire orchard. Since plants are often either over or under sprayed, resulting in environmental pollution issues and inadequate pest control. Besides, growing pressure from farmers, environmental organizations, and public opinion are encouraging lawmakers to try to reduce pesticide losses to the environment. Spraying at an adequate volume application rate on a site-specific basis would help reduce the amount of agrochemicals used in the framework of precision horticulture and precision fruticulture.
Canopies are spatially variable, and knowing the structural characteristics of the canopy is a crucial consideration for improving the efficiency of the spray application process for tree crops. The introduction of electronic systems in the development of new equipment helps to reduce both operating and environmental costs by optimizing the efficiency of the pesticide treatments. For instance, machines that spray only in the presence of plants, not in the gaps between them, have already been developed for cabbage vegetable crops , peach, and apple tree cultures . An essential goal for orchard and vineyard spraying systems is a real-time adjustment of the operating parameters according to the target density, with the aim of keeping the droplets in the canopy, thus improving spray deposition and reducing spray drift.
Therefore, to reduce pollution during spray operations, interest in variable-rate spray technology is growing. A promising solution is the new intelligent variable-rate spray technology that automatically controls spray outputs to match plant presence, canopy characteristics, and travel speeds. This currently available technology can reduce pesticide use and off-target losses, and thus its use will benefit farmers, consumers, and the environment. Advances in sensing and detection technologies may facilitate precision autonomous operations that could improve crop yield and quality while saving energy, reducing workforce, and being environmentally friendly. Real-time sensor and control systems on sprayers are necessary to achieve a uniform spray deposit on the crop canopies and to reduce spray losses. These sensor systems are based on different kinds of physical principles, which may allow efficient monitoring of the canopies. The premise of precision spraying is the detection of the characteristic information of the target plant, which is the foundation and basis for the spraying. However, obtaining accurate data in an easy, practical, and efficient way is a significant problem to be solved. This book chapter will review the real-time sensor based on the precision variable spray method.
2. Infrared sensor-based detection technology
All objects with a temperature above absolute zero emit heat energy in the form of radiation. Infrared sensor is an electronic sensor that measures infrared light radiating from objects in its field of view. This technique works entirely by detecting infrared radiation emitted by or reflected from objects. An infrared detector utilization is in the automatic target detection system. Infrared sensor-detecting techniques have been adopted in automatic target-detecting orchard sprayers to discern targets and control the spraying system automatically. These sprayers can be commercialized easily due to the low price of infrared sensor detectors. Developed countries such as the USA, EU, and Russia are developing automatic target-detecting sprayers that utilize infrared imaging techniques [3–5]. Due to the problems related to infrared image processing, these sprayers remain in the experimental stage.
He et al.  designed a precision orchard sprayer based on automatic infrared target-detecting and electrostatic spraying techniques (Figure 1). The sensors are aimed at the top, middle, and bottom segments of the tree canopy to detect different shapes of fruit trees and provide signals to the control system. Experimental results show that the new automatic target-detecting orchard sprayer with an infrared sensor can save more than 50–75% of pesticides, improve the utilization rate (over 55%), control efficiency, and significantly reduce environmental pollution caused by the pesticide application.
Bargen et al.  designed a red/near-infrared reflectance sensor system for detecting plants. These reflectance characteristics have been determined using spectra-radiometry technology. Detection of plants is possible based upon the distinct reflectance characteristics of plants, soil, and residues. Optical filters were used to select the spectral bandwidth sensitivities for the red and near-infrared ray photodetectors. The reflectance values were digitized for incorporation into a normalized difference index in order to provide a stronger indication that a live plant is present within the field of view of the sensor. This sensor system was combined with a microcontroller for activating a solenoid-controlled spray nozzle on a single-unit prototype spot agricultural sprayer. Jiao et al.  designed infrared photoelectric switch and applied it to spraying on aspen. The experiment proved that infrared photoelectric switch attained the request of the design and reduced the cost of spraying. The interval of target identification was less than 0.3 m, and range of target identification was between 0.2 and 15 m. Adjustable work minimum pass spacing was less than 3.0 m. Jianjun et al.  developed an infrared detecting system consisting of integrated circuit for orchard automatic target sprayer. This system satisfied in detail the design requirements of stability, sensibility, compact volume, and anti-interference from environmental ray, and the detectable distance between the detector and the targets was variable from 0 to 6.15 m, and the space between two spraying targets was no more than 0.3 m.
Infrared detection technology in plant targeting is more applicable for dense and large target reflectors under high light intensity. It will get the best detector sensitivity near midpoint of detection distance and give better detection results for plants with high leaf reflectivity. However, when utilizing infrared target detection for plant pesticide spraying, the operation of an infrared detecting system for automatic target orchard sprayer was hard to work well in rough environment interference resulting from designed defects, including short detectable distance, complicated circuit, and the high cost of the automatic target detector. Although temperature and humidity have little impact on the detection results, plant appearance, light intensity, walking speed, and plant space have evident influences on detecting effect, especially the plants’ appearance and light intensity. Plant density and light intensity are proportional both to detection distance and width. The speed of detector has a linear correlation with the minimum distance of individual plant efficiently distinguished. Plant space is monotonously correlative to detecting sensitivity . Besides, due to the limitation of this sensor, the detection method based on infrared technology cannot detect the characteristic information such as the specific size and size of the target, that is, the qualitative calculation and analysis cannot be realized. Also, the detection process is easily exposed to external light influence [11, 12], and with the continuous growth of modern agricultural spray operation requirements, the technology has been gradually unable to meet the development needs.
3. Ultrasonic sensor-based detection technology
Another type of system is based on the use of ultrasonic sensors to measure distances quickly and automatically. These sensors have three essential elements: an emitter of ultrasonic waves, a chronometer, and a wave receiver. Their operation is based on determining the flight time of an ultrasonic wave from the point of emission to the point of detection after bouncing off an object. The potential application of ultrasonic sensor includes orchard management based on rapid quantification of tree volume. The information could be used in variable-rate application of agrochemicals within a grove. There was without spraying when there was no vegetation, half spraying when there was little vegetation in front of the sensors and full spraying when sensors detected the width of the canopy above a given threshold. This achievement led the way to a continuous variation of flow rate according to the variability of the canopy along citrus groves, vineyard, and fruit orchard rows [11, 13–15].
Different researches have been conducted for automatic measurement of canopy dimensions in groves. For decades, ultrasonic sensors have been employed in agriculture for different purposes [16, 17]. One of these applications is detection and ranging to obtain structural data from trees. The first advances in this field were related to the application of plant protection materials such as pesticides in different orchards. When dose adjustment according to canopy structure was proposed , some researchers began to design electronic systems for measuring canopy structural parameters. The first proposed systems to determine canopy volume used many ultrasonic sensors on a vertical mast  or mounted on the sprayer . Because of the state-of-the-art of the application technologies, using this information in real time was not possible. The use of ultrasonic sensors has been reported only for the detection of canopy presence by [2, 21]. In this method, spraying was done exclusively when the canopy was in front of the sprayer. Another application was citrus trees spraying from constant given distance . The nozzles were located on a movable arm, which follows the boundary of the tree according to data collected from sensors. Ultrasonic sensors were placed 50 and 75 cm apart. The same authors improved another sprayer that was able to spray with three different dosages according to width estimation of the canopy made by ultrasonic sensors .
In the USA, the performance of a sprayer prototype using ultrasonic sensors was tested by Giles et al. . The system adjusted the flow rate of the sprayer to the canopy size variations measured by the sensors. The spray boom was divided into three sections each side, and these sections were independently turned on and off according to the readings of ultrasonic sensors, placed at different heights. Spray savings were reported, but there was also less spray deposition on some foliage areas when the control system was used. In the late 1980s, sprayer models appeared on the market, which were able to turn off the spray when there was a gap between trees . It is beneficial for saving spray in young orchards or when there are wide gaps between trees, reducing the spray drift and the chemical cost. However, these systems do not account for variations in canopy shape, which are found in most of the orchards. More recently, another approach was made by Balsari and Tamagnone  with an ultrasonic control system mounted on a ducted air-assisted sprayer. In this case, the number of working nozzles could be adjusted to tree height, according to the readings of sensors placed at different heights. Tumbo et al.  proposed the use of ultrasound sensors to estimate the volume of citrus trees using the principle of time of flight to determine the distance to the target. Adopting the same system, Zaman and Salyani  proved that forward speed is not as important as tree density on volume estimation. Planas et al.  reported interferences between adjacent sensors spaced less than 60 cm apart. This method assumes the constant distance from the sensor to the tree center, and a small variation on this distance results in a large error on the final volume estimation. Balsari et al.  went one step further analyzing the crop identification system and concluded that there is a relationship between canopy density and its ultrasonic echo signal. Palleja and Landers  reported a low-cost system using four ultrasonic sensors and a microcontroller board to estimate the canopy density as a function of the ultrasonic echoes. It was tested as the growing season progressed and the data obtained highly correlated with the season, but they were not compared to actual canopy density.
Moltó et al.  developed a prototype to turn off the spray in the gap between two tree canopies and with the possibility of making up for the variation of canopy volume at the beginning and end of each tree (Figure 2), using the action of two electro valves at each boom section. An automatic sprayer has been developed that, using an electronic control system, adapts the dose of the product to the actual amount of leaf mass. This system is based on a cheap, 8-bit, conventional microcontroller that receives information about the tree shape from two ultrasound sensors and actuates through several electro-hydraulic valves mounted on a specially designed hydraulic circuit. The system allows spraying higher doses in the central part of the tree, where there is more vegetation in globular shaped canopies. Under the conditions of field test experiments, the system achieved savings of up to 37% of the product while maintaining the quality of the treatment. These savings depend on the size, shape, and distance between trees in each particular orchard.
Gil et al.  pointed out that target detection with ultrasonic sensors can be used to adapt the applied dose following the principles of the variable-rate technology. A multinozzle air-blast sprayer (Figure 3) was fitted with three ultrasonic sensors and three electro-valves, to modify the flow rate from the nozzles in real time, in relation to the variability of crop width. A constant application rate of 300 l/ha−1 was compared with a variable-rate application using the tree row volume principle at a 0.095 l/m−3 canopy. The total flow rate sprayed by the nozzles was modified according to the variations of crop width measured by the ultrasonic sensors. On average, 58% less liquid was applied compared to the constant rate application, with similar deposition on leaves with both treatments.
Solanelles et al.  designed an electronic control system for pesticide application proportional to the canopy width of tree crops (Figure 4). A prototype of an electronic control system based on ultrasonic sensors and proportional solenoid valves for a proportional application to the canopy width of tree crops was mounted on an air-assisted sprayer. The sprayer flow rate adjustment was based on the relationship between the actual tree width measured by the ultrasonic sensors and the maximum tree width of the orchard. The prototype was tested in olive, pear, and apple orchards to assess the system’s performance in different crop geometries. Metal tracers were used so that spray deposits for each treatment could be measured on the same samples, reducing sampling variability. Liquid savings of 70, 28, and 39% in comparison to a conventional application were recorded in the olive, pear, and apple orchards, respectively, which resulted in lower spray deposits on the canopy but a higher ratio between the total spray deposit and the liquid sprayer output. A reduction of the maximum tree width parameter in the control algorithm in the apple orchard reduced spray savings but increased spray deposition, with spray savings mainly in the middle level of the outside canopy, compared to conventional air-assisted applications. As a result of this work, the prototype was assembled with ultrasonic sensors with a working range of 0.4–3.0 m.
Gil et al.  designed, implemented, and validated a variable-rate sprayer vineyard prototype (Figure 5). This prototype can modify the sprayed volume application rate according to the target geometry by using an algorithm based on the canopy volume inspired by the tree row volume model. Variations in canopy width along the row crop are electronically measured using several ultrasonic sensors placed on the sprayer and used to modify the emitted flow rate from the nozzles in real time; the objective during this process is to maintain the sprayed volume per unit canopy volume. Field trials carried out at different crop stages for Merlot and Cabernet Sauvignon vines (Vitis vinifera) indicated a good relationship between the applied volume and canopy characteristics. The potential pesticide savings were estimated to be 21.9% relative to the costs of a conventional application. This conclusion is in accordance with the results of similar research on automated spraying systems.
Zaman and Salyani  evaluated the repeatability of ultrasonic measurements of tree volume, determined the effects of ground speed and foliage density on the ultrasonic measurements, and quantified the difference between volumes of the North and South canopy halves of citrus trees. An experiment was conducted to examine the effects of the canopy foliage density and ground speed on the performance of the Durand-Wayland ultrasonic system in tree volume measurement (Figure 6). The difference between ultrasonic and manual volumes ranged from −17.3 to 28.71% at the 95% confidence level. About 95% of the ultrasonic measurements were repeatable within −12.7 to 30.9% of the manual volume. Canopy foliage density had significant effect on ultrasonic measurements of canopy volume. The volume difference was higher in light than dense trees. There was no significant effect of ground speed (1.6–4.7 km/h) on ultrasonic volume measurements. Variability of the measurements in partially defoliated canopies increased as ground speed increased. There was a significant difference between the volumes of two sides of the trees.
Schumann and Zaman  developed a software for real-time ultrasonic mapping of tree canopy size. A schematic layout of ultrasonic transducer system and manually measured tree dimensions were used for calculation of tree canopy sizes in a citrus grove. Vehicle and trailer with vertical array of 10 ultrasonic transducers and differential global positioning system (DGPS) were used to measure tree heights and volumes. Transducers are mounted from 0.6 to 6.0 m above the ground (Figure 7). The data collected with this automated system were compared with manually measured size data of 30 trees to estimate accuracy, and a grove of 376 citrus trees was surveyed twice with the system to estimate repeatability. Results showed no significant differences between ultrasonically and manually measured tree sizes ranging in height from 2.1 to 4.3 m and in volume from 6.3 to 54.0 m3/tree−1. The system located tree positions for GIS mapping purposes within 1.37 m, 95% of the time.
Palleja and Landers  proposed a real-time method, based on an array of ultrasonic sensors, to estimate canopy density in apple orchards and vineyards (Figure 8). This estimation could be used as a reference to adjust the canopy spraying machine parameters with the aim of improving deposition and avoiding drift. Two sets of experiments were carried out: the first one using a single ultrasound sensor in a greenhouse to determine the signal behavior and adjust the algorithms. The second set of experiments was conducted in the orchard and vineyard, under real working conditions. Results show that the signal obtained is highly correlated with the growing season, and it has similar values on both sides of the row, with an error of 14.1% in vineyards and 3.8% in apple trees and it is sensitive enough to detect hailstorm effects on the canopy. The ultrasound echoes and the canopy density are proportional. The greater the density, the more the echoes produced. The sprayers incorporate a set of four ultrasound sensors and a Louvre system, which allows air volume to be adjusted from 0 to 100%. Four ultrasound sensors were attached on the front of the sprayer, at 2.2 m from the nozzles, and distributed at different heights.
Palleja and Landers  developed a nonexpensive system to estimate the crop density using ultrasound sensors (Figure 9). It is important to note that canopy spraying is rarely, if ever, conducted after harvest and it is often done before blossom, in the dormant period. The real-time capabilities of the ultrasonic system allow the sprayer to be adjusted in order to improve spray deposition and reduce spray drift. As well as density, dead plants or row ends are easily detectable, and the sprayer can automatically switch the nozzles on/off.
Maghsoudi et al.  designed an electronic control system for the detection and estimation of tree canopy dimensions for application rate adjustment. Three ultrasonic ranging sensors were utilized to estimate the distance to the target at three different heights (Figure 10). A multilayer perceptron (MLP) neural network with gradient-descent back-propagation algorithm, tangent-sigmoid transfer function, and 3-7-6 topology was used for volume estimation of tree sections. Training and validation errors as well as R2 values indicated the reliability of the network for volume prediction. Results of t-test for comparing the number of spray droplet impacts, coverage of (artificial) target, spray quality parameter, and relative span factor between variable rate and conventional spraying were not significant, which indicates the consistency of spray distribution in selective application. Experiments showed a reduction in pesticide usage of about 34.5% by means of variable-rate technology (41.3, 25.6, and 36.5, respectively for the top, middle, and bottom sections of tree canopy). Precise application of agrochemicals reduces both costs and environmental pollution by supporting a decrease in the amount of delivered spray.
Jeon et al.  evaluated ultrasonic sensor for variable-rate spray applications. Ultrasonic sensors were subjected to simulated environmental (Figure 11) and operating conditions to determine their durability and accuracy. Conditions tested included exposure to extended cold, outdoor temperatures, crosswinds, temperature change, dust clouds, travel speeds, and spray cloud effects. After exposure to outdoor cold conditions for 4 months, the root mean square (RMS) error in distance measured by the ultrasonic sensor increased from 3.31 to 3.55 cm, which was not statistically significant. Neither the presence of dust cloud nor the changes in crosswind speeds over a range from 1.5 to 7.5 m/s had significant effects on the mean RMS errors. Varying sensor travel speed from 0.8 to 3.0 m/s had no significant influence on sensor detection distances. Increasing ambient temperature from 16.7 to 41.6°C reduced the detection distance by 5.0 cm. The physical location of the spray nozzle concerning the ultrasonic sensor had a significant effect on mean RMS errors. The mean RMS errors of sensor distance measurements ranged from 2.3 to 83.0 cm. The RMS errors could be reduced to acceptable values by proper controlling of the sensor/spray nozzle spacing on a sprayer. Also, multiple-synchronized sensors were tested for their measurement stability and accuracy (due to possible cross-talk errors) when mounted on a prototype sprayer. It was found that isolating the pathway of the ultrasonic wave of each sensor reduced detecting interference between sensors during multiple sensor operations.
Of the various types of sensors used in current precision spray systems, ultrasonic sensors that are affordable, relatively robust during outdoor conditions, and capable of estimating the canopy volume of trees satisfactorily have been used by several researchers. It was proved that the ultrasonic system is capable of sensing density. However, it has strong and weak points. The main advantages of ultrasonic sensors are their robustness and low price. Ultrasonic sensors have relatively low costs and can be easily implemented. The system works in real time, and it works through netted canopies, has a small error during most of the season, and can be used as a reference for canopy density. However, the main drawback is the large angle of divergence of ultrasonic waves, and it has to be calibrated and very uneven fields generate inconsistent data. The error remained low up to and including harvest date at the end of September, but significant errors must be expected at the late season. This limits the resolution and accuracy of the measurements taken and also requires the use of many units to cover a typical agricultural scene . The reflection of the sound waves emitted by an ultrasonic sensor is significantly affected by the directional angle and material of the measured plane. Different leaves of a fruit tree have different angles, which will also change when the wind blows. As a result, the angle of tree leaves can easily affect the measurement of the leaf wall area and cause errors in the determination of the distance from the fruit tree and the leaf wall area .
4. LIDAR sensor-based detection technology
Another detection principle, which is being used rapidly, is based on the light detection and ranging (LIDAR) sensor technology. This technology is a nondestructive remote sensing technique for the measurement of distances. It is ideal for detecting and measuring nonmetallic or biological objects , which provides a relatively novel tool for generating a unique and comprehensive mathematical description of the tree structure. LIDAR is a remote laser range sensor based on the measurement of the elapsed time between the transmission of a pulsed laser beam and the reception of its echo from a reflecting object; this time-of-flight (TOF) is used to estimate the distance between the laser and the object. The advantage of the laser light relative to the ultrasonic waves is that the measurement beam is thinner and less divergent and can be combined with a scanning mechanism to obtain a bidimensional scan pattern to report information about a large area . Terrestrial LIDAR is now used in characterizing canopy structure for different applications like forestry or agriculture.
The use of terrestrial LIDARs in agriculture enables the measurement of structural parameters of the orchards such as the volume of the trees. The ability to very quickly (thousands of points per second) measure the distance between the sensor and the objects around it allows us to obtain 3D cloud points that, by applying appropriate algorithms, makes it possible to digitally reconstruct and describe the structure of trees with high precision . For these reasons, despite their limitation for dusty environments, LIDAR systems have turned out to be one of the most used sensors for the geometric characterization of tree crops.
The capacity of LIDAR to quantify spatial variations, which is an essential aspect of vegetation structure, is a significant advance over some previous methods. LIDAR systems can be used to quantify changes in canopy structure at various time scales, which can provide detailed assessments of canopy growth and allocation responses to field experiments. Laser technology offers unique options regarding the viewing angle and distance information needed to model canopy structure; hence, there is an emergency to thoroughly investigate LIDAR structural applications . The LIDAR system developed made it possible to obtain 3D digitalized images of crops from which a significant amount of plant information, such as height, width, volume, leaf area index, and leaf area density, could be obtained.
In agricultural applications, it is, however, possible to use two-dimensional (2D) terrestrial LIDAR sensors, which are much cheaper to use . 2D LIDAR sensors obtain a point cloud corresponding to a plane or section of the object of interest. The fact that these sensors only scan in one plane does not necessarily limit their scope to 2D perception . Hence, this sensor gives as output a point cloud that, postprocessed, can be exploited for the construction of a 3D image. Rosell Polo et al.  proposed the use of a 2D LIDAR scanner in agriculture to obtain 3D structural characteristics of plants. The results obtained for fruit orchards, citrus orchards, and vineyards showed that this technique could provide fast, reliable, and nondestructive estimates of 3D crop structure. It can be concluded that LIDAR systems were able to measure the geometric characteristics of plants with sufficient precision for most agriculture applications.
Early works were concentrated on comparing of manual volume estimation with LIDAR and ultrasonic sensor measurements . Results indicated good correlation between the estimation made by LIDAR and ultrasonic sensors, while correlation with manual measurements was lower. Observation showed larger differences between manual and sensor estimations in less dense trees. This canopy information was used to adjust agrochemical dose rate  and estimate fruit yield in citrus groves . LIDAR sensor in relation to vertical sampling resolution can gather much more information from canopy parameters for a more accurate estimation in comparison with array of ultrasonic sensors [26–28, 42]. The results of these tests were satisfactory, but extrapolation of these results to trees with different structures is not easy. Although several groups have developed prototypes to adjust the application flow rate to the variations in canopy structural parameters using ultrasonic sensors, a review of various targeted spraying methods  showed that solutions for variable-rate spraying in orchards are still in prototype phase; however, there are already commercially available sprayers for weed control and plant fertilization in arable land.
Rosell et al.  proposed a method of 2D LIDAR scanner in agriculture to obtain three-dimensional (3D) structural characteristics of plants (Figure 12). There was a great degree of concordance between the physical dimensions, shape, and global appearance of the 3D digital plant structure and the real plants, revealing the coherence of the 3D tree model obtained from the developed system with respect to the real structure. For some selected trees, the correlation coefficient obtained between manually measured volumes and those obtained from the 3D LIDAR models was as high as 0.976.
Escolà et al.  designed, implemented, and validated a prototype (Figure 13) running a variable-rate algorithm to adapt the volume application rate to the canopy volume in orchards on a real-time and continuous basis. The orchard prototype was divided into three parts: the canopy characterization system (using a LIDAR sensor), the controller executing a variable-rate algorithm, and the actuators. The controller determines the intended flow rate by using an application coefficient (required liquid volume per unit canopy volume) to convert canopy volume into a flow rate. The sprayed flow rates are adjusted via electromagnetic variable-rate valves. The goal of the prototype was to keep the actual application coefficients as close as possible to the objective. Strong relationships were observed between the intended and the sprayed flow rates (R2 = 0.935) and between the canopy cross-sectional areas and the sprayed flow rates (R2 = 0.926). In addition, when spraying in variable-rate mode, the prototype achieved significantly closer application coefficient values to the objective than those obtained in conventional spraying application mode.
Palleja and Landers  analyzed the sensitivity of the tree volume estimates in the spatial trajectory of a LIDAR (Figure 14) relative to different error sources. The sequence of two-dimensional scans performed with a LIDAR attached to a tractor can be interpreted as the three-dimensional outline of the trees of the grove and used to estimate their volume. The sensitivity of the tree volume estimates relative to different error sources in the estimated spatial trajectory of the LIDAR is analyzed. Tests with pear trees have demonstrated that the estimation of the volume is very sensitive to errors in the determination of the distance from the LIDAR to the center of the trees (with errors up to 30% for an error of 50 mm) and in the determination of the angle of orientation of the LIDAR (with errors up to 30% for misalignments of 2°). Therefore, any experimental procedure for tree volume estimate based on a motorized terrestrial LIDAR scanner must include additional devices or procedures to control or estimate and correct these error sources.
The main advantages of LIDAR sensors are their high speed and accuracy of measurement, and they provide a 3D point cloud of the object being measured. LIDAR sensors facilitate the description of the geometric structure of trees. However, the scale of these remote sensing techniques is relatively large, and consequently, the sensing resolution may be insufficient for a real-time variable-rate application in a liner production field. In addition, remote sensing techniques typically have a chronological gap between detection and application, resulting in application errors. To reduce this problem, a LIDAR system or a laser scanner has been used to measure canopy volume. Promising results were reported for using this system in which measured canopy volume was close to manually measured volume [39, 45, 46]. Unfortunately, the narrow row spacing in a liner field may restrict LIDAR from being used on variable-rate liner sprayers. It is also a relatively expensive sensor ($2000–6000), and the high cost of these instruments limits their use. Furthermore, a typical tree liner sprayer treats multiple rows at a time. Each liner row would require an individual LIDAR system to measure its tree canopy variation. Thus, controlling a variable-rate application sprayer would require several LIDAR systems. This would increase the application cost to an impractical level.
5. Computer stereo vision-based detection technology
A video camera can capture video images of fruit trees and segregate parameters such as the leaf wall area, height, and density based on the color information through video processing techniques. However, due to a lack of measured distance information, distance can only be estimated based on the precalibrated distance from the video camera, which may easily generate relatively large errors. Computer stereo vision implies the extraction of 3D information from digital images, as obtained by a CCD or CMOS image sensor-based digital camera, which can provide a 3D field image by combining two monocular field images taken simultaneously using a binocular camera . The main advantage of stereoscopic vision over conventional monocular vision is its ability to detect ranges: distances between scene objects and the camera. Monocular cameras create planar images in which each pixel is the result of a two-dimensional (2D) projection of the 3D world. Stereovision adds a third coordinate, or range, which completes the full localization of any point within a 3D Cartesian frame. The natural outcome of a stereovision sensor is a 3D point cloud that renders the captured scene with a degree of detail proportional to the resolution of the acquired images. Every single point in the 3D cloud comes from a stereo-matched pixel and will be endowed with three coordinates that identify its exact spatial position .
Berenstein et al.  proposed grape clusters and foliage detection algorithms for autonomous selective vineyard sprayer (Figure 15). Novel machine vision algorithms were developed to detect gaps between grapevines in order to reduce pesticide use during foliage spraying and to detect the exact location of grape clusters to target spraying toward them. A spraying robot equipped with these detection capabilities and a pan/tilt head with a spray nozzle would be able to spray selectively and precisely, reducing significant amounts of spraying material and human labor. Results show 90% accuracy of grape cluster detection leading to 30% reduction in the use of pesticides.
Microsoft’s Kinect system can capture the color and depth information of a scene in real time. This system consists of a red, green, and blue (RGB) video camera, a monochrome complementary metal-oxide semiconductor (CMOS) video camera, and an IR transmitter. The color CMOS camera generates color images, and the IR transmitter and the IR CMOS camera generate depth images. The Kinect system outputs a 640 × 480 RGB image and an IR depth image. Because conventional depth sensors (e.g., laser ranging radars) are deficient concerning sensitive information readability, depth cameras have become an essential means for measuring the depth-of-field information of scenes. Under ideal conditions, the resolution of depth information acquired by a depth camera can reach 3 mm. Xiao et al.  designed an intelligent precision orchard pesticide spray technique based on the depth-of-field extraction algorithm (Figure 16). To obtain desirable spray effect, the advantages of color and depth information using Microsoft’s Kinect system were integrated. To adjust and control the spray intensity of sprayers and the dose of sprayed pesticides, an equation for calculating the leaf wall area average distance of fruit trees was proposed. A comparison with the measured distances showed that the distances calculated based on the data acquired by the Kinect system were accurate. The results of the experiment on peach trees, apricot trees, and grapevines demonstrated that the intelligent orchard pesticide precision spray model established based on the average distance and the leaf wall area density can improve the efficiency in spraying pesticides, reduce waste and environmental pollution, and achieve automated and precision orchard production.
6. Advanced data fusion application technique and future directions
The integration of data and knowledge from several sources is known as data fusion. To overcome the inherent drawbacks and combine the advantages of different kinds of sensors, multimodal sensor fusion has been widely used [49–51]. Briefly, data fusion can be defined as a combination of multiple sources to obtain improved information; in this context, improved information means less expensive, higher quality, or more relevant information. Data fusion is the process of integrating multiple data sources to produce more consistent, accurate, and useful information than that provided by any individual data source. Fusion of the data from two sources (dimensions 1 and 2) can yield a classifier superior to any classifiers based on dimension 1 or dimension 2 alone . In general, all tasks that demand any parameter estimation from multiple sources can benefit from the use of data/information fusion methods. Data fusion techniques have been extensively employed on multisensory environments with the aim of fusing and aggregating data from different sensors. The goal of using data fusion in multisensory environments is to obtain a lower detection error probability and a higher reliability by using data from multiple distributed sources .
The use of spatial sensors with the agricultural application has increased rapidly in recent years as their costs decline. Because of their ability to provide instantaneous information that can be used for feature extraction and object detection, vision systems and laser scanners are becoming more common in outdoor agricultural applications such as tree detection, map construction, mobile robot localization, and navigation. Vision systems are low-cost solutions for extracting different features (e.g., color, edge, and texture), while laser scanners are popular sensors in outdoor applications as they provide precise range and angle measurements in large angular fields. Fusing images from cameras with range data from laser scanners enable mobile robots and vehicles to more confidently perform a variety of tasks in outdoor environments . There are differences between the data acquired from the laser scanner and the camera images. The 2D laser scanner generates a single horizontal scan of the environment, whereas the camera provides an instantaneous image of the local environment with precise depth information. A laser scanner provides range and bearing data, while the camera primarily provides intensity and color information. There are some standard features in both types of data. For example, many corners and edges correspond to a sudden change in the range of the laser scan data and a sudden variation in image intensity .
Shalal et al. [55, 56] presented a novel tree trunk detection algorithm using camera and laser scanner data fusion (Figure 17). The innovation and contribution of this study developed a new tree trunk detection algorithm using low-cost camera and laser scanner data fusion as a component of fully automated operation to enhance the detection capability and to discriminate between trees and nontree objects. The laser scanner is used to detect the edge points and determine the width of the tree trunks and nontree objects, while the camera images are used to verify the color and the parallel edges of the tree trunks and nontree objects. The algorithm automatically adjusts the color detection parameters after each test, which shows to increase the detection accuracy. The algorithm was able to detect the tree trunks and discriminate between trees and nontree objects with a detection accuracy of 96.64% showing that the fusion of both vision and laser scanner technologies produced robust tree trunk detection. Fusion of data from these sensors was found to improve tree detection because the laser scanner can provide reliable ranges, angles, and width of the tree trunks and nontree objects, while the vision system can distinguish between tree trunks and other nontree objects from different features.
Data fusion as a new method was demonstrated for detecting trees and nontree objects using a camera and laser scanner data fusion. The utilization of both camera and laser scanner data enhanced the tree trunk detection. Projecting from the laser scanner to the image plane and selecting the region of interest with the required features were useful since it reduces the processing time and minimizes the effect of the noise in the other parts of the image. The developed algorithm relies only on the onboard sensors without adding any artificial landmarks such as tags or reflective tapes on the trees in the orchard. The algorithm automatically adjusts the color detection parameters after each test, which was observed to improve the detection accuracy. Above all, the fusion of data from the vision and laser sensors improves plant canopy detection because the laser scanner can provide accurate ranges, angles, and widths of the tree and objects, while the vision system can distinguish between a tree and other objects.
7. Discussion and conclusion
Development of new, environmentally friendly alternative variable-rate sprayer application techniques only began in the last four decades. Its objective has been to use variable-rate sprayer dosage rates that are as low as possible and to apply variable-rate sprayer only to places where this was necessary, with minimum losses transferred to the environment. Various procedures and methods for tree canopy detection have been suggested and developed by both computer and agricultural scientists . The detailed review indicates that the establishment of an appropriate variable-rate sprayer is still one of the critical issues in plant protection. Improvement of electronic tree canopy sensing should facilitate electronic measurements of the tree canopy characteristics and enable more precise control of variable-rate sprayer dosage, which can then ensure a faster response of the entire system at higher driving speeds in the orchard. Some researchers suggested that electronic characterization of the tree canopy could be carried out more efficiently by using some detection approaches, including ultrasonic, imaging, and optical detection systems.
The analysis of sensing systems for electronic canopy characterization indicates that the infrared and ultrasonic sensors as the oldest and simplest approaches are still an appropriate tool for determining average canopy characteristics such as the ends of rows and significant gaps between well-separated trees. Furthermore, when equipped with appropriate software, the infrared and ultrasonic sensor transceivers can be used for measuring the tree density. For this reason, these types of sensors will remain on sprayers in the near future, because it can simplify the operator’s repetitive work in the orchards and might serve as an input parameter for adjusting variable-rate sprayer dosage from a particular nozzle.
The analysis of the different existing detection systems to characterize the structure of tree plantations shows the existence of several aspects that limit the use of most of the systems under field conditions, some sensors remaining suitable for this purpose. Laser scanners and stereo vision are direct competitors and are probably the most promising and complementary techniques for achieving 3D maps of plants and canopies, although infrared and ultrasonic sensors remain an attractive option for specific applications. In fact, the possibilities of combining sensors for this purpose are innumerable. In the near future, it is highly likely that we will see a notable advance in this field of research with the increased use of the new generation of flash LIDAR sensors, capable of measuring 3D structures of plants in real time and at a moderate cost compared to alternative detection systems.
The usefulness of using camera sensor to facilitate the quantification of the density of the plantations has also been mentioned. However, it has become clear that there is still a long way to go and both the geometric characterization of plants and variable application techniques must be improved. More highlighted advanced stereo vision measurement sensing systems for electronic canopy characterization sound very attractive for detection of the tree canopy, because this technique captures a massive image of an orchard in a short time. However, the computer-generated digital 3D terrain model of the orchard still cannot assure characterization of canopy diameter, height, and number of leaves with sufficient precision for estimating the leaf area index needed for appropriate adjustment of the variable-rate sprayer dosage.
In this chapter, variable-rate sprayer applications based on real-time sensor technologies have been reviewed. Based on the results from reports and literatures, Table 1 summarizes the operating principles and the main pros and cons of the exposed sensors and methods for the measurement of the geometrical properties of plants and crops.
|Infrared sensors||All objects with a temperature above absolute zero emit heat energy in the form of radiation.|
Infrared sensors measure infrared light radiating from objects in their field of view.
Work entirely by detecting infrared radiation emitted by or reflected from objects.
|Temperature and humidity have little impact on the detection results.|
Measurement relatively independent of atmospheric conditions.
|Accurate measurement of the 3D characteristics of the canopy remains unfeasible for the moment.|
Plant appearance, light intensity, walking speed, and plant space have evident influences on detecting effect.
Deficient spatial resolution for applications in agriculture.
Short detectable distance, complicated circuit.
|Ultrasonic sensors||Measure the distance to an object by using sound waves.|
Based on determining the flight time of an ultrasonic wave from the point of emission to the point of detection after bouncing off an object.
|Robustness and low price make ultrasonic sensors suitable for agricultural applications.|
Relatively easy to implement.
|The large angle of divergence of ultrasonic wave beams limits the resolution and accuracy of the measurements taken.|
The use of many units to cover a common agricultural scene is required.
|LIDAR sensors||Based on the measurement of the distance from a laser emitter to an object or surface using a pulsed laser beam.|
Time-of-flight LIDAR measures the time that a laser pulse takes to travel between the sensor and the target.
|High speed of measurement allows obtaining cloud points quickly.|
Applying appropriate algorithms makes it possible to digitally reconstruct and describe the structure of trees with high precision.
Plant information, such as height, width, volume, leaf area index, and leaf area density can be obtained with sufficient precision.
|The estimation of the volume is very sensitive to errors in the determination of the distance from the LIDAR to the center of the trees and in the determination of the angle of orientation of the LIDAR.|
Motorized terrestrial LIDAR scanners must include additional devices or procedures to control or estimate and correct these error sources.
|Stereo vision||Provides a 3D field image by combining two monocular field images taken simultaneously using a binocular digital camera.|
Computer algorithms are necessary to convert the original camera coordinate arrays of the objects into their real-world coordinates.
|Provides realistic 3D image of plants and tree crops.|
Measures directly the 3D vegetation structure including those plant physical parameters that are important for production management, such as crop size and volume.
|Offer less accuracy than laser-based systems and need appropriate calibration and recording procedures.|
When several images are processed together, the magnitude of the data files grows considerably, complicating the handling and storage of 3D information and requiring long processing times. The problem becomes more critical when real-time processing is required.
With regard to agricultural applications, innovative techniques represent an essential contribution to the improvement of variable-rate sprayer application. The different sensing system can detect tree canopy characteristics precisely, and when combined with sophisticated decision-making models, they enable accurate variable-rate sprayer dosage control. The coordinated use of multiple sensors, the development of new real-time data processing algorithms, and the simplification of crop adaptable application systems are objectives for the future of this research line. Obtaining a precise geometrical characterization of a crop at any point during its production cycle by means of a new generation of affordable and easy-to-use detection systems, such as LIDAR and stereo vision systems, will help to establish precise estimations and provide valuable information on which to base more sustainable pesticide dosages. Without any doubt, optical sensing systems for electronic canopy characterization including a LIDAR sensor provide the most accurate and detailed information about the tree canopy. When supported with the proper software, a LIDAR-based signal can represent a perfect tool for creating a 3D space at low installation costs, which is essential for guiding a robotic arm equipped with nozzles and small vents along the tree row in real time. For all these reasons, LIDAR will represent the crucial sensor in the further development of both trailer-mounted and self-propelled sprayer prototypes, which should find the widespread commercial application.
In the near future, the evolution and development of new sensors devoted to the geometric characterization of tree crops will enable significant and much needed advances in optimizing the use of variable-rate sprayer in agriculture, as well as an increase in production and quality by improving training systems. It is worth noting that the benefits of variable spray affect millions of cultivated hectares and therefore impact directly on the society and the environment in which we live. It is therefore of vital importance to continue devoting major efforts to the development of increasingly accurate, robust, and affordable systems capable of measuring the geometric characteristics of plantations, which support the development of the different areas of a sustainable and precision agriculture. However, it is still necessary to resolve several technological and commercial questions. The former include improving detection systems, especially with regard to developing software for the postprocessing steps and improving the speed of calculation and decision making. Among the latter, it is essential to produce low-cost sensors and control systems to facilitate large-scale deployment.
This research project was financially supported by the National Natural Science Foundation of China for Young Scholars (Grant No. 51605210), by Yunnan Applied Basic Research Youth Project (Grant No. 2015FD011), by Introduced Talents Scientific Research Staring Foundation of KMUST (Grant No. 14118940), by Scientific Research Fund Project of Yunnan Provincial Education Department (Grant No. 2015Y079), and by Opening Project of the Key Laboratory of Bionic Engineering (Ministry of Education) (K201621), Jilin University.