Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

Abstract

Unmanned aerial vehicles carrying multimodal sensors for precision agriculture (PA) applications face adaptation challenges to satisfy reliability, accuracy, and timeliness. Unlike ground platforms, UAV/drones are subjected to additional considerations such as payload, flight time, stabilization, autonomous missions, and external disturbances. For instance, in oil palm plantations (OPP), accruing high resolution images to generate multidimensional maps necessitates lower altitude mission flights with greater stability. This chapter addresses various UAV-based smart farming and PA solutions for OPP including health assessment and disease detection, pest monitoring, yield estimation, creation of virtual plantations, and dynamic Web-mapping. Stabilization of UAVs was discussed as one of the key factors for acquiring high quality aerial images. For this purpose, a case study was presented on stabilizing a fixed-wing Osprey drone crop surveillance that can be adapted as a remote sensing research platform. The objective was to design three controllers (including PID, LQR with full state feedback, and LQR plus observer) to improve the automatic flight mission. Dynamic equations were decoupled into lateral and longitudinal directions, where the longitudinal dynamics were modeled as a fourth order two-inputs-two-outputs system. State variables were defined as velocity, angle of attack, pitch rate, and pitch angle, all assumed to be available to the controller. A special case was considered in which only velocity and pitch rate were measurable. The control objective was to stabilize the system for a velocity step input of 10m/s. The performance of noise effects, model error, and complementary sensitivity was analyzed.

1. Introduction

Malaysia is the world’s second largest exporter of palm oil (Figure 1) with approximately 5.08 million ha of land under cultivation [1]. Major percentage of these plantations is owned by small-scale private farmers that have huge demands to affordable low-cost autonomous platforms for applications, such as scouting, palm census, yield monitoring, spraying, and most importantly health assessment and disease detection. The ability to collect high spatial resolution aerial images using drones is changing the way the oil palm growers are approaching the business [2]. Conventional methods of practicing precision agriculture (PA) in oil palm plantations such as remote sensing and spraying are being replaced by integrated fixed-wing or multirotor unmanned aerial vehicles (UAV) [3], allowing collection of information to be instantly accessible for immediate decisions. Precision farming for increasing oil palm yield requires optimization of returns on inputs while preserving resources based on sensing, measuring, and health assessment of the plantations [4]. Relying on satellites images of palms, there is a substantial lag in terms of accessing the data quickly enough. Professionals have been using satellite and piloted airplane remote sensing platforms [5] for plantation scouting applications, such as vegetation cover assessment [6], vegetation mapping [7], crop monitoring [8], and forest fire applications [9]; however, the difference that drone technology [10] and agricultural robotics [11, 12, 13] have made is around the speed and accuracy of delivering that information. Digital agriculture [4] offers great opportunities for mechanization and automation of farming tasks in oil palm plantations through automation of data collection by means of ground or aerial surveillance and data processing software to predict or estimate palms yields.

Figure 1.

Comparison between world exports of palm oil, with Malaysia as the second largest exporter. (data: [1]).

Conventional scouting of oil palms on a regular basis (Figure 2), as well as palm census and quantification of the amount of fresh fruit bunches (FFB) for yield monitoring, is a labor-intensive task that is either ignored or carried out manually by the use of hand counters. Traditional scouting of palms is an ineffective practice that requires expert knowledge and postprocessing lab equipment. It involves spending hours and hours of human observation inside the unpleasant hot and humid plantation and does not provide accurate and comprehensive information because several parameters are ignored due to measurements difficulties (i.e., tasks that involve climbing trees, measuring canopy diameter, etc.). Other than the inaccuracy and biases statistics, manual scouting involves additional costs for each extra observation, hazards, and safety issues (i.e., falling from trees, bugs, snake bites, etc.). Satellite imaging services are extremely costly, and they can take images only once a day and have to be ordered in advance. The resolution of these images is low and can be influenced greatly with certain sky cloud conditions. Ground sensing platforms are also time consuming and are limited to small fields of view. Yield reduction due to high-density palm areas that cause etiolation is an issue in plantation management. Palm densities are an important and limiting factor for growth, nutritional status, fruiting, and hence for the plantation yield. Optimal palm densities depend on different factors, such as cultivars, climate, soil characteristics, and land preparation. Refilling of palm gaps and correction of nonoptimal densities are of high priority for a good plantation management. Conventional methods that are solely based on visual observation are inaccurate, particularly when coverage is large and dominant topography is hillocky.

Figure 2.

Tedious field work with conventional scouting of oil palm plantation.

Precision agriculture of oil palm is one of the largest markets in Malaysia that will be hit by UAV and robotics. These devices are the future of PA and are sometimes referred to as the next step in data-driven agriculture. UAV/drones carrying multi-spectral and multimodal data acquisition devices face adaptation challenges to satisfy information, accuracy, and timeliness as the bases of a successful precision agriculture (PA) operation. These platforms have contributed to significant reductions of in-field walking costs and observational experiments. UAVs are defined as “an aircraft that is equipped with necessary data processing units, sensors, automatic control, and communications systems and is capable of performing autonomous flight missions without the interference of a human pilot” [14]. The global market for agricultural UAV drones is estimated to reach 3.7 billion US dollars by the year 2022 (Source: Radiant Insight Research firm). Aerial photography from UAS has bridged the gap (see the schematic diagram shown in Figure 3) between ground-based observations and remotely sensed imagery of conventional aircraft and satellite platforms and has made possible great improvements in crop scouting, yield mapping, field boundary mapping, soil sampling and soil property mapping, weeds and pest control and mapping, vehicle’s guidance, navigation control, and spraying. These devices are easy to use and are typically flexible, low cost, light-weight, and low airspeed aircraft. They have revolutionized smart farming and precision agriculture, from planting to harvesting, from seeding to sensing, and from scouting to spaying. UAS drones are widely available on demand, and their functionalities can be customized for different farming applications and can provide a cost-effective monitoring platform without requiring an expert operator. With this technology, several problems associated with the data resolution from piloted aircraft and satellite imaging have been solved. They are capable of providing live data from a wide range of sensors, such as those shown in Figure 3 (multispectral, NIR, LiDAR, etc.) at precision resolutions measured as centimeters per pixel. Such information contributes to the in-depth analysis for the crop health assessment or the inventory management databases. With the UAV technology, the following can be achieved: information about accurate planted area for replanting or thinning, palm census for creating inventory database, calculating the total land area in use, finding distances between each palm to specific spots, calculating canopy diameter, palm height, and palm density, creating 2D, 3D, GIS, NDVI maps for plantation, identifying palm status based on Orthomosaics and digital elevation models, detecting healthy and unhealthy palms (stress assessment), monitoring exposed soil for variable rate technology application, quantification of fresh fruit bunches and mature fruits for yield calculation, monitoring chlorophyll content and nutrient estimation, and measuring leaf area index, drought assessment, biomass indication, weed detection, and inventory management. Data and information such as these are useful for developing decision support systems and yield prediction models.

Figure 3.

Typical components of a UAV-based remote sensing platform for precision agriculture of oil palm.

2. Adaptation of UAV for oil palm remote sensing

UAV drones can be well adapted for oil palm plantations, where field work is tedious. They allow observation of individual palm trees and can operate unnoticed and below cloud cover that prevents larger high-altitude aircraft and satellites from performing the same mission. Moreover, they can be deployed quickly and repeatedly, and they are less costly and safer than piloted aircraft, are flexible in terms of flying height and timing of missions, and can obtain very high-resolution imagery. As an aerial remote sensing platform, a UAV drone must be adapted to satisfy the basic requirements of image data collection from oil palm plantation. Other than the selection of proper sensors, the stability and accuracy are vital to provide geo-referenced images for extraction of useful information. Adaptation of UAV technology for oil palm plantations involves integration of vision sensors, machine vision algorithms, and control system for (i) yield monitoring and yield mapping, (ii) automated airborne pest monitoring using thermal cameras, (iii) identification and counting of specific insects from very high-resolution optical images, (iv) development of decision support system (DSS) using geo-referenced images as a basis for a GIS-based system giving oil palm growers the possibility to incorporate data directly to their precision farming platforms, (v) identification and mapping of Ganoderma disease using hyperspectral camera, (vi) automated retrieving of oil palm canopy chlorophyll and nutrient content from multispectral and hyperspectral UAV acquired images, and (vii) dynamic Web mapping and inventory management of oil palm productivity using in situ sensors. This paper is the first of series reporting on design and development of an affordable fixed-wing UAV to be used as a flexible scouting test bed for oil palm plantations. Schematic diagram illustrating the early stages of technological development for introducing a UAV platform to local farmers and the general steps and procedure involved with setting up a UAV remote sensing platform for agricultural applications are shown in Figures 4 and 5, respectively.

Figure 4.

Schematic diagram illustrating the early stages of technological development for introducing a UAV platform to local farmers (source: Adaptive AgroTech Consultancy International).

Figure 5.

General steps and procedure involved with setting up a UAV remote sensing platform for agricultural applications.

2.1. Recommendations for purchasing UAV for agricultural application

A comprehensive document including recommendation for choosing the best UAV drone for precision agricultural and smart farming applications is available in [15]. Specifications of sample multirotor and fixed-wing UAV recommended for precision agriculture of oil palm are also provided in the Appendix. Compared with piloted airplanes and satellite imaging, the ability of UAVs in collecting higher resolution aerial images at a significantly lower cost can provide oil palm growers with more accurate information on palm height, crown size, and normalized difference vegetation index (NDVI), enabling practicing of data-driven techniques for early and accurate yield estimation and health assessment. While a typical UAV may cost as little as USD1000, it can be integrated with custom instrumentations, controllers, sensors, and software to operate as a flexible remote sensing or variable rate technology platform to contribute to plantation management, growth, and soil condition assessment mapping application (i.e., 2D, 3D, GIS, NDVI), risk/hazard/safety management, spraying application, and academic and research application. In specific, UAV remote sensing in oil palm precision agriculture can contribute to automatic palm detection and counting, automatic measurements of palm height and crown diameter measurements, calculation of planted and unplanted areas for replanting or thinning, analyzing palm status based on Orthomosaics and digital elevation models, inventory management and health assessment based on physical appearances and vegetation indices, model-based yield prediction, yield monitoring and mapping, rapid estimation of nutrient contents, and disease detection. It should be noted that agricultural UAV activity is considered commercial operation with a high-tech platform for data acquisition or spraying applications that should be carried out by licensed professionals or certified pilots. Price range for a complete package is between USD1500 to over USD25000 depending on the application. Multicopter drones can fly for 3–45 minutes on a one battery charge and are more suitable for regular use in small-scale plantation without the requirement to special takeoff and landing areas. Fixed-wing UAVs need to be planned for mission flights and reliable landing for use in larger plantations. It is better to purchase drones that can be controlled via mobile or tablets or are fully autonomous from takeoff to landing (i.e., the entire mission can be performed by a single start button). For a multicopter, it is also important to check for the live standstill view feed. This feature allows plantation managers to find specific spots and issues for closer inspection. One of the key considerations in purchasing scouting UAV is the NDVI and NIR camera options. For the sake of cost saving, an affordable regular 3D camera with two lenses can be purchased for less than USD300 and modified slightly with a blue plastic filter to produce NIR images. However, a more expensive UAV that can collect data faster will compensate the extra costs in a long run.

2.2. Oil palm health assessment and disease detection

Health assessment in oil palm plantations is crucial for spotting fungal infection and bacterial disease on the palms. By aerial scanning the plantation using visible RGB camera, NIR, hyperspectral, and multispectral sensors, it is possible to identify temporal and spatial reflectance variations before they can be detected by naked eyes and associate these changes with palms heaths for an early response. For instance, NDVI cameras can calculate the vegetation index describing the relative density and health of the palms, and thermal camera can show the heat signature of different spots in the plantations. A conceptual demonstration of a UAV remote sensing platform equipped with NDVI sensor for oil palm health assessment is shown in Figure 6.

Figure 6.

Conceptual demonstration of a UAV-remote sensing platform for oil palm health assessment with NDVI camera.

The platform shown in Figure 7 can be customized and integrated with hyperspectral camera as shown in Figure 8, for the detection of Ganoderma boninense, which is a serious threat to oil palm plantations in Malaysia and has caused great losses to healthy palms. This disease causes both basal stem rot and upper stem rot and remains South East Asia’s most devastating oil palm diseases, with direct loss of the stand, reduced yield of diseased palms, and the resultant requirement for earlier replanting. Using naked eye, the Ganoderma disease can only be recognized at a very late stage with serious symptoms of foliar chlorosis and breakage at older fronds, the presence of decayed tissues at palm base, and production of fruiting bodies. When symptoms of the disease appear on young palms, it is too late and younger palms die within 6 to 24 months, whereas mature palms may survive for 3 years. Reports also indicate that the basal stem rot can kill up to 80% of the total standing palms. Despite the several efforts in controlling this disease, the available methods are slow, and current strategies are still immature. To our knowledge, no effective method or a robust sensing instrumentation has been commercialized for early detection of this disease at an early stage. Research reports have highlighted that oil palm yields are highly correlated with most of the nutrients. There are extensive publications on the hyperspectral analysis of images with application in agriculture that shows promising methods to be adapted for early detection of Ganoderma disease in oil palm. In order to adapt a UAV remote sensing platform for this purpose, several questions should be addressed as follow: (i) at what stages of infection can the hyperspectral imaging detect the Ganoderma disease symptoms? (ii) what are the unique spectral characteristics of Ganoderma spectral reflectance data? (iii) what statistical or mathematical methods are the best for analyzing the Ganoderma spectral data? and (iv) how well can a low-cost multiband radiometer assist a scouting crew to detect the suspicious HLB-infected trees? We can begin with a hypothesis that wavelet analysis of reflectance data can improve detection of nutrient concentration in oil palm. This hypothesis can be studied by the use of the Matlab Wavelet CIR images Toolbox. Preliminary studies have demonstrated the potential of wavelet analysis for retrieving foliar nitrogen content and photosynthetic pigment concentrations from leaf and canopy reflectance spectra, but further research is needed to develop the approach. Our research will contribute to saving of more palm trees and consequently a higher yield which has a significant impact on large scale plantations and the economy of Malaysia. A project can be proposed with the long-term goal of developing a fast UAV-based screening technique that can assist oil palm growers in detecting suspicious Ganoderma-infected palms. Such a project may involve the following systematic steps and methodology: (i) study the spectral characteristics of GB in lab conditions, (ii) developing a classification method to identify the disease and separate it from other palm stresses and other diseases with similar symptoms, (iii) evaluating the possibility of using a low-cost spectral radiometer for fast screening of Ganoderma-infected palms, (iv) developing an instrumented platform for collecting and geo-referencing hyperspectral images in the plantations, and (v) conducting a field trial to evaluate the effectiveness of hyperspectral imagery for detecting the disease in the plantations. Reflectance spectra of vegetation, measured in the visible and infrared region, contain information on plant pigment concentration, leaf cellular structure, and leaf moisture content. In this research, we propose to study the capability of hyperspectral imaging and spectroscopy in the range of 300-2500 nm for early detection of anomalies in oil palm trees as a result of Ganoderma infection. Preliminarily hyperspectral imaging data indicated that Ganoderma-infected leaves have different spectral characteristics compared to healthy leaves. A quick and efficient method of detecting and mapping Ganoderma at the field level will assist growers to better manage and control this disease and can financially benefit growers. In the first year of the study, we will study the spectral characteristics of Ganoderma-infected oil palm leaves in laboratory conditions and compare them with other nutrient deficiency symptoms. Accordingly, we will develop a classification method to identify the symptoms of Ganoderma and separate it from plant stresses and other diseases with similar symptoms. Also, in the first year, we will study how well a low-cost spectral radiometer can detect Ganoderma symptoms. Based on the results from the first year of the study, we will develop an instrumented platform for collecting and geo-referencing hyperspectral images and evaluate the effectiveness of hyperspectral imagery for detecting suspicious Ganoderma-infected palm trees in the grove.

Figure 7.

Feasibility of using autonomous UAV-based hyperspectral imaging for detection of Ganoderma boninense disease in oil palms.

Figure 8.

Thermal camera and night vision (top row figures) and high-resolution RGB images approach (bottom row figures) for UAV based pest monitoring in oil palm plantations.

2.3. Pest monitoring

Oil Palm growers lose some portion of their yields to insects and pests infestation. Traditional methods of locating pests in thousands of hectare plantations are not effective. For example, early detection of an invasive pest like rats in palm plantations with labor requires a great amount of time and luck. Obviously, conventional methods are not accurate, and plantation managers have to make an educated guess before sending the crew to a large field to check for infested spots. For the purpose of pest monitoring, a solution is to have a UAV imagery platform equipped with a thermal camera and high-resolution RGB vision sensors for accurate identification of the spots in the oil palm plantations fields that are diagnosed with specific insects and pests. This approach may also involve development of a decision support system (DSS) using georeferenced insect count as a basis for a GIS-based system, giving plantation managers the possibility to incorporate data directly to their precision farming platforms. Specific steps involve (i) platform setup, that is integration of the UAV, vision sensor, and control system, (ii) perception which refers to the development of a real-time machine vision algorithm for pest monitoring (to refine the aerial images captured by the UAV in order to provide plantation managers with the most usable data), and (iii) action stage, which is the development of the DSS for creation of the prescription map. When pests are spotted, spraying UAV can be used for dropping a targeted load of pesticide. The spraying UAV can be equipped with distance-measuring and light detection sensors such as lasers, ultrasonic echoing, or LiDAR methods to scan the ground and adjust the flight altitude with the varying topography of the plantation and therefore apply the correct amount of spraying liquids for even coverage and avoid collisions. This practice will result in an increased efficiency while reducing the amount of penetrating spray chemical in the soil and groundwater. It is estimated that UAV spraying is five times faster than conventional tractor and machinery equipment.

The FLIR Vue Pro thermal camera shown in Figure 8 is designed for small UAVs and can be used for agricultural applications. It has different lens options for different type of view and specific applications. The thermal sensor resolution of this camera is 640 by 512 pixels and records 30 frames per second for smooth video. The light weight and small size of this camera will not affect the UAV center of gravity during the flight or sacrifice the flight time. It comes with the mounting accessories that can be used with most UAV platforms. It can also be used with transmitters for live feeds. The FLIR Vue Pro thermal camera does not have a separate battery and can be charged through a 6 V power from the UAV. Image data are stored on a standard micro SD card. An application connects the camera with the computer via Bluetooth. The thermal imager Optris PI 640 shown in the figure is the smallest measuring VGA infrared camera available. With an optical resolution of 640 × 480 pixels, the PI 640 delivers pin-sharp radiometric pictures and videos in real time. With a body sized 45 × 56 × 90 mm and weighing only 320 grams (lens included), the optris PI 640 counts among the most compact thermal imaging cameras on the market. Temperature range is between −20 and 900°C (optional up to 1500°C), spectral range is between 7.5 and 13 μm, and frame rate up is to 125 Hz. For the purpose of validation, images taken at varying heights and resolutions will be compared with the ground truth pictures taken on the ground with a mobile device. The research findings may lead to new pest management strategies that use UAV and other imaging technologies for detecting invasive pests in other farm fields, e.g., oil palm plantations. The thermal camera can also be used for spotting the areas that are drier and require attention.

2.4. Yield monitoring

Quantification of FFB from UAV stream images for yield map creation is the first step toward practicing PA in oil palm plantations. With the available high-tech imaging sensors and using real-time image processing and remote sensing techniques (i.e., pixel-based or object-based [16], template matching [17, 18, 19] image analysis, learning algorithms methods for classification [20, 21] and for extracting useful information from an image), it is possible to measure oil palm yield on much smaller scales. One of the benefits of using autonomous UAV is their affordable price and lower cost per each mission flight that make them suitable for academic research in yield monitoring applications. The idea is to evaluate the feasibility of having UAV agent robots that can fly over and inside oil palm plantations and collect high-resolution detailed photos from different angles for automated creation of yield maps. These maps can tell growers where and when to apply the optimal amount of inputs (i.e., fertilizer, pesticide, water) for creating further sustainability. Of course, mobile robots with camera and sensors mounted on top of them can also be used for such application; however as mentioned earlier, we are proposing a research idea that involves a swarm or fleet of small-scale UAVs similar to what is shown in the figure that simultaneously fly inside the plantation for image data collection. By using different sensor-based measurement and imaging techniques on each UAV, a real-time machine-vision system can be developed for accurate identification of the amount of FFB on the palms. Such technology is highly demanded by oil palm growers as a fast, accurate, and reliable tool for estimating palm numbers and FFB in large-scale plantations. In determining instantaneous oil palm yield, two parameters must be known, weight and coordination of FFB on each palm. The weight of the FFB can be estimated using a machine vision algorithm that quantifies the number of fruits on each palm (Figure 5). These estimated weights are then georeferenced with coordinates of the corresponding palm using computer programs for the creation of database and yield map. Collected data will be processed by custom-built GIS software for creation of yield map and inventory database. A conceptual illustration of integrated fixed-wing UAV-based inventory management and health assessment system with mobile application and cloud computing is shown in Figure 9.

Figure 9.

Feasibility of UAV imaging system for yield monitoring of oil palm (top) and a proposed methodology for UAV-based yield monitoring of apple and orange fruits using deep learning algorithms [22].

2.5. Virtual plantations and dynamic Web mapping

One of the limitations of doing research on oil palm plantation is the lack of accurate data and input variables for modeling and simulation purposes. UAV technology can be integrated with image acquisition techniques for three-dimensional reconstruction of the environment and creation of virtual plantations. Examples of 3D reconstructed plantation are shown in Figure 10. The information extracted from these 3D models can lead to the development of dynamic Web inventory management and mapping system. A 3D reconstruction model of oil palm plantation can be created by using range data methods or depth map using laser range finder sensors and 3D scanner instrumentations. This approach is however costly and not affordable by local oil palm farmers. Alternatively, passive methods, also called image-based reconstruction methods (i.e., photogrammetry technique), have been introduced using a normal camera and image sensors, which do not interfere with the reconstructed object. In this method, a UAV equipped with a normal RGB camera will collect images of the oil palm plantations from different views and angles. Computer software will then process these images to create a 3D model, and filter specific wavelength to generate images that corresponds to vegetation index and palm health. For example, a red edge image can describe nitrogen content and water stress. The potential of UAV image data to simulate the physical process of palm photosynthesis as a result of different crown sizes and densities intercepting different amounts of light radiation can be evaluated using virtual plantations. A virtual plantation can be used to estimate palm height, crown size, and inventory database (Figure 11) for generating dynamic Web maps and yield prediction models. These maps can identify how different palm height, crown sizes, plantation densities, and row orientations in different locations can affect the water and fertilizer demand. Moreover, mathematical models can be established based on the validated information from virtual plantations for estimating nitrogen demand and fertilizer application. These maps also provide precision rich data for academic and educational purposes. Researchers can access to detailed measurements of palm trunk and crown size and the spacing between different palms, leaf area index, and crown density as a preliminary study for the possibility of autonomous variable rate applications and robotic harvesting.

Figure 10.

Example of virtual plantation generated by UAV imaging [23].

Figure 11.

Conceptual illustration of a fixed-wing UAV Web mapping system integrated with mobile application and cloud computing for yield prediction and inventory management in oil palm plantation.

For the purpose of a sensor Web-based approach for dynamic Web mapping, observations from a UAV can be combined with in situ sensor data to derive typical information offered by a dynamic Web mapping service (WMS). This will provide daily maps of vegetation productivity for oil palm plantation with a spatial resolution of 250 m. Results will present the vegetation productivity model, the sensor data sources, and the implementation of the automated processing facility. An evaluation will be made of the opportunities and limitations of sensor Web-based approaches for the development of Web services, which combine both UAV and in situ sensor sources. A conceptual illustration is provided in Figure 11. A yield estimation model can be developed by establishing performing regression analysis between palm height (x₁), crown size (x₂), palm age (x₃), vegetation index (x₄), nutrient content (x₅), and soil parameters (x₆): Yield = func(x₁, x₂,…, x₆). This model will be based on comprehensive information of each palm location, size, and health, will provide managers with an estimation of yield, and make decisions for sustainable practices methods for production increase without necessary needs for expanding the plantation into natural forests.

3. Stabilizing a fixed-wing Osprey UAV

The fixed-wing Osprey drone shown in Figure 11 is a commercially available, low-cost experimental flight test bed manufactured by Unmanned Aerial Research (Florida, USA) that is suitable for investigating novel control approaches [24] and is a flexible platform for remote sensing research applications in precision agriculture of oil palm. An example application can be found in the work of [25], where the fixed-wing J-HAWK UAV was used for palm tree counting at Melaka Pindah oil palm plantation in Malaysia. This drone can carry large payloads while maintaining excellent performance with virtually no degradation in handling qualities. It is a well-constructed, durable aircraft with mission versatility and a cavernous payload volume that is easily accessible, featuring two long aluminum tracks on the floor for mounting payloads in limitless configurations. Some of the specifications according to the manufacturing website are as follows: payload capacity: 31.75 kg, empty weight: 15.87 kg, payload volume 0.0566 m³ (0.203H × 0.304 W × 0.889 L), max cruise: 90 kts, landing speed (no flaps): 25 kts, power (DA-100): 10 hp by a reliable custom desert aircraft 100 cc motor with 3-blade carbon fiber propeller, wingspan 3.352 mm, and length 2.362 m. We begin with dynamic analysis and controller design for this drone in the presence of actuator limits and sensor noise for autonomous flight missions with greater accuracy and stability. The communication architecture, modules, and designed control system is shown in Figure 12.

Figure 12.

Architecture, modules, and control system for a the proposed UAV in precision agriculture of oil palm.

For the purpose of this paper, we have concentrated our analysis on controller design for two outputs, velocity and pitch rate, by adjusting two control inputs, the elevator and the thrust. In specific, our control objective was to design a single controller, i.e., proportional-integral-derivative (PID), Linear-quadratic regulator (LQR) full state feedback, ($C=I_{4\times 4}$), and LQR plus observer (with $C$ defined by the dynamic model), that (i) stabilize the drone with a velocity step input of 10 m/s and (ii) minimize rise time, percentage overshoot, and steady state error over the widest possible initial conditions. Based on the field experiments data in the reviewed literature [24], the maximum ${\delta }_{\mathit{\text{thrust}}}$ and ${\delta }_{\mathit{\text{elev}}}$ and their rate of change were considered $\pm 200N$ and $\pm {30}^{°}$ and $\pm 200N/\mathit{sec}$ and $\pm {300}^{°}/s$, respectively. In addition, the noise for the velocity sensor and the pitch rate sensor were assumed to be $\pm 0.4m/s$ and $\pm {1.7}^{°}/s$, respectively. The dynamics associated with this drone under standard aircraft assumptions were de-coupled into both lateral and longitudinal directions. For the sake of this control design, we only analyze the longitudinal dynamics. The longitudinal dynamics of the Osprey have been mathematically modeled as a fourth-order multiple-input multiple-output (MIMO) system with two inputs and two outputs [24]. The dynamics have been linearized for the Osprey aircraft flying at 25 m/s at an altitude of 60 m. In standard state-space form, they are given as:

$A=\left[\begin{array}{cccc}-0.1470& 11.0767& 0.0841& -9.8065\\ -0.0316& -7.1712& 0.8281& 0\\ 0& -37.3527& -9.9628& 0\\ 0& 0& 1& 0\end{array}\right]$, $B=\left[\begin{array}{cc}{3.10}^{-3}& 0.06\\ {10}^{-5}& {10}^{-4}\\ 0.98& 0\\ 0& 0\end{array}\right]$, $C=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 1& 0.\end{array}\right]$

where $x=\left[V\alpha q\theta \right]$, with the state variables defined as $V$: velocity, $\alpha$: angle of attack, $q$: pitch rate, and $\theta$: pitch angle. The control inputs are $u={\left[\begin{array}{cc}{\delta }_{\mathit{\text{elev}}}& {\delta }_{\mathit{\text{thrust}}}\end{array}\right]}^T$. Our controller design process begins with analyzing the mathematical model of the given dynamic system. The state-space model was first converted to a convenient transfer functions (TF) given in (1) and (2). Converting the SS model into TF form using MATLAB “tf(sys)” yields transfer function from input “${\delta }_{\mathit{\text{elev}}}$” to outputs given in (1), and the sets of transfer function from input “${\delta }_{\mathit{\text{thrust}}}$” to outputs give in (2).

$${\mathrm{H}}_{11}\left(\mathrm{s}\right)=\mathrm{V}=\frac{0.03357{\mathrm{s}}^3+0.6577{\mathrm{s}}^2+3.407\mathrm{s}-68.91}{{\mathrm{s}}^4+17.28{\mathrm{s}}^3+105.2{\mathrm{s}}^2+18.44\mathrm{s}+11.58}$$

$$H_{21}\left(s\right)=q=\frac{0.98s^3+7.171s^2+1.416s}{s^4+17.28s^3+105.2s^2+18.44s+11.58}$$E1

$${\mathrm{H}}_{12}\left(\mathrm{s}\right)=\mathrm{V}=\frac{0.06{\mathrm{s}}^3+1.029{\mathrm{s}}^2+6.153\mathrm{s}+0.03663}{{\mathrm{s}}^4+17.28{\mathrm{s}}^3+105.2{\mathrm{s}}^2+18.44\mathrm{s}+11.58}$$

$$H_{22}\left(s\right)=q=\frac{-0.003735s^2+0.07027s}{s^4+17.28s^3+105.2s^2+18.44s+11.58}$$E2

We first perform open-loop analysis to determine possible control strategies. The open-loop responses (Figure 13) from each of the four TFs were then analyzed individually. According to the TF in (1) and (2), the terms with the highest coupling can be obtained by considering the simple steady state case. Substituting jω = 0, in all the terms, it can be observed that the static gain relationship is high for ${\delta }_{\mathit{\text{elev}}}$ versus velocity output. This also makes physical sense as a change in the pitch would slowdown the Osprey. It is also noted that the pitch rate has zeros at origin. This suggests that the system has inherent derivative property and hence has a tendency to amplify noises. Based on the open-loop response shown in Figure 13, it can also be seen that the effect of ${\delta }_{\mathit{\text{elev}}}$ on the velocity is more than other inputs. From Figure 13, the following key points are helpful in controller design for the system, (i) ${\delta }_{\mathit{\text{elev}}}$ has more effect on velocity than any other input, (ii) the velocity falls sharply with input ${\delta }_{\mathit{\text{elev}}}$, and (iii) ${\delta }_{\mathit{\text{thrust}}}$ has limited effect in both velocity and pitch rate.

Figure 13.

Open-loop step response analysis of the Osprey drone velocity and pitch rate for the elevator and thrust inputs.

For the PID controller design shown in Figure 14, the system was set at initial conditions [${\delta }_{\mathit{\text{elev}}}=4$ and V = 25 m/s]. A step input of 10 m/s was given at time t = 60 s. The following gains were used for the PID velocity controller: $K_p$ = 200, $K_i$ = 80, and $K_d$ = 20. For the PID pitch controller, $K_p$, $K_i$, and $K_d$ were respective selected as 6, 0.2, and 10. After introducing the noise, the new selected gains for the PID velocity controller were $K_p$ = 50, $K_i$ = 11 and $K_d$ = 11. For the PID pitch controller, the new $K_p$, $K_i,$ and $K_d$ were chosen 100, 4, and 1, respectively. The decrease in $K_p$ compared with the previous case for the velocity controller should be noticed. Hence, in this case, the gain of the pitch controller was driven high and the other low. Since pitch rate has very high impact on the other system variables, noise in the pitch rate influences the system heavily. Therefore to improve tuning the controller, a simple first-order TF = 2/(S + 15) (low pass filter) was inserted in the loop (shown in the second Simulink block of Figure 14).

Figure 14.

Simulink blocks for the PID controller in the absence and presence of noise and actuator limits.

The LQR controller is the solution of the optimization problem that optimizes the cost of errors and the cost of actuation effort, with appropriately weighted states. The optimization function is defined as $J=\int \left(x^T\mathit{Qx}+u^T\mathit{Ru}\right)\mathit{d\sigma }$. In the state space form, the obtained LQR controller is expressed as $u=-\mathit{Kx}$. For this solution, an LQR controller was first derived using the MATLAB “lqr” command. The cost weighting matrices Q and R were selected as unit matrices, and the LQR was realized. Simulink blocks for the designed LQR controller with full state feedback are shown in Figure 15. The weighting matrices used in this case were as follow:

Figure 15.

Simulink blocks for the designed LQR controller with full state feedback.

$Q=\left[\begin{array}{cccc}5& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 0.5& 0\\ 0& 0& 0& 4000\end{array}\right]$, $R=\left[\begin{array}{cc}1& 0\\ 0& 0.05\end{array}\right]$

It is noted that the control effort for pitch is the most optimized parameter in Q. This value was selected on the basis that pitch is the most influential state variable and controlling pitch translates control of all the other parameters. In addition, the weight for pitch rate is low because the effort to control pitch rate is harder and introduced more oscillations in the system.

For the LQR controller with observer (Figure 16), the observer design allows controller to use full-state feedback techniques in situations where only a subset of states is available to the controller. The observer matrix L adds gain to the feedback loop, in order to ensure stability and quicker response of the state observer system. While this helps stability, the L gain adversely amplifies the sensor noise. Therefore, a trade-off has to be made on the noise resilience versus the system’s robustness. The matrix L was determined through these steps: (i) the system output states were checked for controllability and observability using Matlab code “obsv” and “ctrb,” (ii) the poles of the system were found and the system was found to be stable, (iii) for the observer system to be more stable and faster, the poles were multiplied by a factor of 3, (iv) these scaled poles were then used in the Ackermann’s formula for pole-placement design to find L and design a mimic of the original system, the observer, and (v) the state output from the observer can now be fed to the LQR controller. The weighting matrices used in this case are:

Figure 16.

LQR controller with observer block.

$Q=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 5& 0\\ 0& 0& 0& 1000\end{array}\right]$, $R=\left[\begin{array}{cc}1& 0\\ 0& 0.001\end{array}\right]$

It can be seen that the value of pitch gains in the Q is four times smaller than the previous case. The gains were reduced to take control over noise in the system. In the other words, these reductions help eliminate the noise in the system. From step three of the observer design, we know that the observer matrix L adds gain to feedback loop. This gain helps amplifying the noise and then feeding them into the control loop back again. Noise introduces similar problems faced with the PID controller. With high gains, the noise amplifies and combined with actuator nonlinearities drives the system into instability. With lesser gains and actuator effort, noise is damped and absorbed by the system.

4. Simulation results and discussion

Results of the simulation for the designed controllers are shown in Figure 17 through Figure 20. It can be seen from Figure 17 that the step change applied at time 60 s has an effect on the pitch, and the PID controller is managed to minimize this effect. When noise is introduced to the system (Figure 18), because the coupling gain between pitch and velocity are very high, the pitch rate sensor noise distorts the response considerably. Moreover, since the tuned gains exploited the infinite actuator capabilities, the response of the system was quick and the steady state error was almost zero; however, due to the nonlinearities, the system had to be tuned again. Since elevation was directly related to the pitch rather than the pitch rate and to avoid the dynamics of the “rate” signal, pitch was compared against the elevator angle to generate the error signal. To accomplish this, the pitch rate was simply integrated using an ideal integrator (1/s).

Figure 17.

PID performance without noise and actuator limits.

Figure 18.

PID performance with noise and actuator nonlinearities.

It can be seen from the results that the state variables pitch and velocity are closely coupled variables. The coupling terms connecting these two quantities exhibit every high gains, hence the control design was challenging in regulating these variables independent of the other. This coupling needed special attention during control design. It should be noted that on the basis of tuning complexity, only two PID controllers were used in the control problem, as if the system was a weakly coupled system. Since PID control is ideally suited for single-input single-output systems (SISO) and only for weakly coupled MIMO systems, a perfect performance was not expected to achieve with the two PID controllers. Nevertheless, a reasonable performance was still achieved when the system was considered ideal, i.e., free from nonlinearities and noise. When noise was introduced to the system, the velocity suffered because of high noise content in the pitch signal. The noise also introduced dangerous oscillations in the system, limiting controller gains significantly and hence slowing down the overall system. Several instabilities caused due to the rate limit and saturation were evident. The integral gain of the PID acted on error build-up caused by saturation and hence pushing the system into instability. After reducing the gains in the loop, the controller was then tuned by trial and error procedures. The relative performance of PID with respect to other controllers is summarized in Table 1.

Model	Price ($)	Weight (Kg)	Size (mm)	Camera resolution	Coverage	Flight time (min)	Max altitude (m)	Flight speed (km/h)
Parrot Disco Pro AG Drone	6875	UAV: 0.78; Take-Off: 0.94	Wing span: 1150 × 580 × 120	—	—	—	—	—
RF70 UAV	3000	Payload: 3	—	1080 P	600 acres/hour	45–60	—	18
AgDrone UAS	10,000	—	—	1080 P	—	60	—
DT-26 Crop mapper	120,000	—	—	1080 P	—	60	—	110
Quad Indigo	25,000	—	—	1080 P	—	45	—
E384 Mapping Drone	2400	UAV: 2.5; Payload: 1	Wingspan: 1900 Length: 1300	—	1000 acres in 100 minutes at 5 cm resolution	90	—	47
PrecisionHawk Lancaster 5	—	Payload: 1	—	1 cm/pixel	300 acres/flight	45	—	—
Xena observer	—	Take-Off: 5	—	—	—	27	5000	—
Xena thermo	—	Take-Off: 4.6	—	—	—	32	5000	—
AEE AP10 Drone	299	—	—	1080 P Full HD Video at 60 FPS	—	25	500	71
UAV drone crop sprayer		UAV: 9; Payload: 10; Take-Off: 13	800 × 800 × 70 (L.W.H)	—	—	16	1000	—
DJI drone sprayer	15,000	—	—	—	7–10 acres/hour	—	—	29
Yamaha’s helicopters spray & survey	130,000	UAV: 71; Payload: 30	—	—	10 acres	—	—	—
JMR-V1000 6-rotor 5 L	665–3799	UAV: 6.5; Take-Off: 18	875 × 1100 × 480 (L.W.H)	—	—	14–18	—	11–22
AG-UAV Sprayers1	—	UAV: 8; Payload: 6	Height: 650	—	—	8–15	—	—
AG-UAV Sprayers2	—	UAV: 14.2; Payload: 20	Height: 650	—	—	15–30	—	—
AG-UAV Sprayers3	—	UAV: 9.5; Payload: 10	Height: 650	—	—	10–20	—	—
DJI AGRAS MG-1 Sprayer	7999	Payload: 10	—	—	7–10 Acres Per Hour	—	—	—
Hercules Heavy Lift UAV (HL6)	—	UAV: 8; Payload: 6	Height: 660	—	—	30	—	37
Hercules Heavy Lift UAV (HL10)	—	UAV: 9.5; Payload: 10	Height: 660	—	—	30	—	37
Hercules Heavy Lift UAV (HL20)	—	UAV: 14; Payload: 20	Height: 660	—	—	60	—	37
Multirotor UAVs	—	—	—	—	—	10–40	—	—
AgStar GoPro FPV Camera Payload	1950	—	—		—	—	—	—
DJI Phantom 3	469	—	—	2.7 K HD videos, 12 MP photo	—	25	—	—
Fixed Wing UG-II	—	UAV: 11; Take-Off: 15	2240 × 1600 × 650 (L.W.H)	—	—	180	—	65–110
Professional Electric Six Rotor Drone UA-8 Series	—	Payload: 3	860 × 860 × 540	—	—	28	5000	36
Yuneec H520 Hexacopter	2500–4500	—	—	4 K/2 K/HD video or 20 MP images	—	—	—	—
Ag-drone AK-61	6999	Take-Off: 22; Payload: 10	—	—	—	10–15	0.5–5 m	18–36
YM-6160	5000	Take-Off: 21.9; Payload: 10	—	—	—	10–15	0.5–5 m	18–36
Skytech TK110HW	32–52	—	—	0.3 MP	—	6–7	—	—
JJRC H8D 5.8G FPV RTF RC	169–175	UAV: 0.023	330 × 330 × 115	—		8	—	—
X810 Long Range Uav Sprayer	4000–6500	Payload: 10	2490 × 1645 × 845 (L.W.H)	—	—	25–40	—	—
Syma X8C	68.99	—	508 × 508 × 165 (L.W.H)	2 MP HD Camera	—	5–8	—	—

Controller	Noise	Actuator limits	Rise time (s)	Settling time (s)	Overshoot (%)
PID	—	—	0.61	0.95	4.2
PID	Y	Y	1.28	Inf	10.1
LQR	—	Y	1.01	1.9	2.1
LQR	Y	Y	3.3	3.95	0
Observer	Y	Y	1.9	Inf	34.5

Table 1.

A comparison between the proposed controllers.

LQR controllers however work in the state-space and are suited for MIMO control. It assumes full state feedback; that is, all the system’s states are available for the controller to take decisions, even though this might not be a case in reality. Therefore, we designed the observer to deal with this issue. The outputs of the LQR controlled system response with actuator dynamics are shown in Figures 19 and 20. Unlike the PID controller, the LQR handles actuator dynamics inconsequentially. Appropriate waiting matrices were assigned, and the LQR controller matrix was obtained by using the MATLAB “lqr” command. The LQR trivially performed well with actuator nonlinearities. By weighting the gains in the Q and R matrices, it was possible to avoid high actuation effort and thus saturation. But rate limit did affect the rise time. The LQR also suffered from oscillations, when noise was introduced.

Figure 19.

LQR full state feedback response without noise.

Figure 20.

LQR with observer response with noise.

From the plots of LQR with observer (Figure 20), it can be seen that the system is in the verge of instability and the noise content of the pitch signal disturbs the velocity severely. The relative stability of the given system can be discussed in terms of the gain margin and phase margin. Based on the Bode plots analysis of the open-loop system (plots not provided for the sake of paper page limits), the differential term in the elevator input to output relationships reduces the phase margin of system considerably. Model errors and disturbance in the pitch rate could easily drive the system to instability. This agrees with the findings in the controller design exercised.

5. Conclusion

Health assessment and conventional scouting of oil palms on a regular basis, as well as palm census and quantification of the amount of fresh fruit bunches (FFB) for yield monitoring, are labor-intensive tasks that are either ignored in large scale plantations or are carried out manually by the use of labor force. Traditional scouting is not only an ineffective practice but also requires expert knowledge and post-processing lab equipment to provide useful information. Advances in aerospace engineering, control system, and computing have contributed significantly to the improvement of UAV-based remote sensing platforms. This paper discussed some of the potential applications of UAVs for precision agriculture of oil palm plantations. We also highlighted some of the adaptation challenges faced by UAV drones, including platform stability due to the flight dynamics parameters and winds, climate factors and light reflection degrading quality of the acquired images, and regulations and restrictions law by the Federal Aviation Administration. As a response to the needs of small-scale plantation owner for an affordable UAV platform, a fixed-wing Osprey drone was proposed and used in designing an auto-flight control. The aircraft can be externally actuated by controlling the thrust (${\delta }_{\mathit{\text{thrust}}}$) and the elevator (${\delta }_{\mathit{\text{elev}}}$). Initially, all states of the dynamic model were assumed to be available to the controller. A case was then considered when only velocity and pitch rate could be measured. We conclude that the MIMO control problem of the Osprey drone falls in the class of systems that exhibit high level of coupling between the inputs. We also conclude that the LQR design procedure was simple compared with the PID and performed better than PID in the presence of noise. Unlike PID, the LQR was more vulnerable to have steady state error. With changes in the ${\delta }_{\mathit{\text{elev}}}$ command, the velocity was affected considerably and would never recover unlike integral action of PID. The introduction of an observer in an already noisy system added more uncertainty in the system, thus pushing the system toward instability. The observer added to the gain of the feedback loop and hence amplifying noise. Even with various combinations of weighting matrices, the steady state oscillations were as high as 20%. In conclusion, it is observed that the LQR is a robust and effective controller for MIMO control. The LQR was found to be robust against noise and disturbance in the system too.

A. Appendix

Names and specifications of sample multi-rotor and fixed-wing UAV recommended for precision agriculture of oil palm.