From Pillars to AI Technology-Based Forest Fire Protection Systems

Abstract

The importance of forest environment in the perspective of the biodiversity as well as from the economic resources which forests enclose, is more than evident. Any threat posed to this critical component of the environment should be identified and attacked through the use of the most efficient available technological means. Early warning and immediate response to a fire event are critical in avoiding great environmental damages. Fire risk assessment, reliable detection and localization of fire as well as motion planning, constitute the most vital ingredients of a fire protection system. In this chapter, we review the evolution of the forest fire protection systems and emphasize on open issues and the improvements that can be achieved using artificial intelligence technology. We start our tour from the pillars which were for a long time period, the only possible method to oversee the forest fires. Then, we will proceed to the exploration of early AI systems and will end-up with nowadays systems that might receive multimodal data from satellites, optical and thermal sensors, smart phones and UAVs and use techniques that cover the spectrum from early signal processing algorithms to latest deep learning-based ones to achieving the ultimate goal.

1. Introduction

Fire is an important ecological factor that has affected both the structure and distribution of numerous plant communities across the globe. Fire probably first appeared, as a natural disturbance factor, as soon as there was any existing terrestrial vegetation [1]. Prior to human Influence, the main ignition sources were lightning, volcanic and earthquake activity [2, 3, 4]. Fire was therefore a natural process occurring periodically in the natural vegetation succession cycle, contributing to the continued rejuvenation and promoting the productivity of many plant communities and ecosystems [5]. The periodicity of ecosystem burning was determined by the availability of a fuel load able to sustain a fire after a natural ignition event. Fuel load and consequently the flammability of an ecosystem increases with age ensuring that the ecosystem will be burned at a relatively mature stage. The effect of fire on landscape and ecosystems has been so determinative that the distribution, composition and structure of many biomes across the globe could not be explained by climate and soil alone and fire also needs to be taken into account [6, 7].

Later, fire became a very important human tool, widely used or misused for the improvement of living conditions. Archeological evidence from the Petralona caves of northern Greece and elsewhere indicate that fire has been used by man for at least half a million years [8]. The Paleolithic hunter and food gatherer used fire not only as a source of energy but also as a vegetation and landscape management tool. Both fire frequency and intensity increased dramatically with considerable impacts on the natural ecosystems and the Mediterranean flora. This change in fire characteristics has shifted the equilibrium between fire and ecosystem function, transforming fire from a natural ecological factor that initiates succession into a human-induced land degradation factor [9]. Despite the major human intervention in the relation between fire and ecosystem function, the Mediterranean ecosystems and vegetation types retain the ability to recover from fire quite rapidly [5], assuming fire frequency does not greatly exceed the natural return interval [10]. Wildfires, however, are not just a vegetation and ecosystem degradation factor but also a factor which can have significant social and economic consequences, especially when they occur in the rural urban interface.

The occurrence of wildfires and especially large ones is the result of the combined action of two driving forces, namely fuel availability and continuity and weather patterns. Over the last years, significant efforts have been made to study the historic and current trends in fire regime and identify the relevant significance of the above driving forces in determining the past and current fire regime [11, 12, 13, 14]. The main conclusion of these studies is that there is an observed change in the fire regime of the Mediterranean Europe after the 1970s with a significant increase both in total number of fires (NF) and area burned (AB). There is a clear indication that wildfires have changed from fuel driven, before the 1970s, to weather driven after that date.

The decade of 1970s coincides with significant socioeconomic changes in the Mediterranean Europe, which may explain the sudden change in fire regime, better than the climatic changes and the observed increase in drought, which are more gradual and have also occurred in other climatically similar regions without associated changes in fire regime [12]. The abandonment of agricultural land in the mountainous and semi-mountainous areas, the decrease of livestock, the increased urbanization, a significant decrease to the use of wood as an energy source and the recovery of vegetation in the abandoned field, have all caused significant changes in the landscape structure and composition [15].

Another significant consequence of the changed fire regime in Mediterranean Europe is the expansion of fire to regions and ecosystems that are not considered fire prone and the vegetation components do not possess fire adaptive traits. An analysis of the fire characteristics in Greece for the year 2007 [16] revealed that 9.3% of the burned area falls in an altitudinal zone exceeding 1000 m, confirming previously reported similar trends of an increased occurrence of fires at higher altitudes. Furthermore, an analysis of landscape dynamics in the nature reserve of Dadia Forest National Park [17], revealed that during 2001–2011 large wildfires became a significant destruction factor, threatening the long-term sustainability of the reserve, although low intensity fires could be a mechanism for maintaining landscape heterogeneity.

The urbanization of population apart from land abandonment also led to an increased and urgent need for residential areas in the receptor cities, bringing the city borders close to semi-natural ecosystems and subsequently increasing the wildland-urban interface (WUI). Furthermore, the improvement of the economic status, led to the creation of settlements in forested areas, often maintaining extensive parts of semi-natural vegetation formations creating a rather deadly WUI. The most dreadful example of such a situation is the 2018 fire in eastern Attica which led to the death of 100 people and significant loss of properties and infrastructures. Finally, the increase of tourism in recent decades led to an increase of the residential zones in coastal areas increasing again the WUI and frequency of ignitions [18].

Despite the above changes in fire regime and the catastrophic incidences experienced in southern Europe, and Greece in particular, the wildfire management continues to rely almost exclusively on fire suppression and traditional means of fire detection. An analysis of the fire characteristics in Greece during the period 2007–2011 revealed some interesting findings regarding the efficiency of the currently applied wildfire management [16]. In 2007, the great majority of fire events burnt relatively small areas lower than 1000 ha. However, 11% of the events evolved to megafires resulting in the worst year ever recorded in terms of area burnt (more than 200,000 ha), although the number of fires was almost equal to the one in 2011, and according to the Forest Fires in Europe 2007 report [19], it was high but not the highest since 1980. The entire year of 2007, the summer period in particular, was characterized by extreme weather conditions with the highest temperatures recorded for almost a century, three consecutive heat waves from June to August and wind patterns that favored the spread and intensity of fires [20]. As stated in [20, 21], 2007 represents an example of how the weather conditions will be like by the end of the century as a result of climate change.

The almost 90% efficiency of the currently employed fire management in keeping most wildfires at relatively small sizes, even under the most favorable for fire spread conditions, reveals that this approach, which relies exclusively on fire suppression and traditional methods of early fire detection has reached its efficiency limits. This is because it is a small number of fire events which turn into megafires and destroy ecosystems, properties, infrastructures and most importantly have a high cost in human lives, and such events unfortunately are not avoided. Thus, a new approach is needed in wildfire management which will not rely exclusively on fire suppression, but it will utilize the technological advancements, the wide availability of remote sensing data and the large amount of research related to risk assessment and early fire detection.

In this chapter, after the brief fire’s history, in Section 2, the basic forest fire monitoring systems technologies are presented. The vital requirements of an autonomous early forest fire detection system, its main modules and the necessary methods for wildfire management and risk assessment, smoke and fire detection based on images and video as well as navigational autonomy issues for UAVs are presented in Section 3. Finally, in Section 4 contains our conclusions.

2. Forest fire monitoring systems technologies

Forest fire monitoring system can be broadly classified into the following categories, each one strongly related with a corresponding technology [22]:

1. Human-based observation systems

2. Satellite-based systems

3. Wireless sensor networks based systems

4. Optical and thermal cameras based systems

2.1 Satellite-based systems

Regarding the satellite-based systems, there have been some initiatives for forest fire detection purposes. Specifically:

• the advanced very high-resolution radiometer (AVHRR) [23], launched in 1998

• the moderate resolution imaging spectroradiometer (MODIS) [24], launched in 1999, and the Visible Infrared Imaging Radiometer Suite (VIIRS) [25] that was launched in 2011 gave the capability for the use of a new generation of operational moderate resolution-imaging following the legacy of the AVHRR on NOAA and MODIS on Terra and Aqua satellites.

However, all these are not sufficient for early forest fire detection due to the fact that satellites follow orbits which deprive the demand for image acquisition of forests around the clock. In addition, beyond the ineffective forest fire scanning, the quality of images is heavily dependent on weather condition.

2.2 Wireless sensor networks based systems

There are numerous contributions in the literature, which are based on wireless sensor networks (WSN) technology. In particular:

• Sudha et al. [26] implemented a system based on IoT devices, which enables the continuous monitoring of a forest area, covering issues relative to fire detection and animal tracking.

• Toledo-Castro et al. [27] in order to minimize the false alarms rate of their own WSN system, introduced a fuzzy logic-based model were.

• Xu et al. [28] proposed a formula for sensor distribution in a forest area, solving problems of optimal coverage by taking into account the topology of the covered area.

• In terms of energy efficiency, Wang et al. [29] proposed an algorithm that minimizes the energy consumption of their WSN-based forest monitoring system.

• Finally, Kadri et al. [30] implemented a WSN-based system whose reliability, robustness and durability were examined, setting the foundations for future works.

Despite of their robustness and efficiency, WSN-based systems have their own limitations mainly the limited energy capacity, the relatively low communication speed and demanding installation and maintenance.

2.3 Optical and thermal sensors based systems

On the other hand, optical and thermal sensors based systems seem to be more popular than the satellite ones, because of the existence of a number of well-established practices introduced in systems EYEfi, FireWatch (Germany) and ForestWatch (Canada), FireHawk (South Africa), FireVu (England) and UraFire (France) [22, 31]. In all these systems, different kinds of fire detection sensors could be used:

• video cameras sensitive in visible spectra used for smoke recognition during the day and fire flame recognition during the night

• infrared (IR) thermal imaging cameras used for the detection of heat flux from the fire

• IR spectrometers which identify the spectral characteristics of smoke gases and

• light detection and ranging (LIDAR) systems which measure laser light backscattered by the smoke particles

In all those systems, automatic forest fire detection is based on smoke recognition during the day and flame recognition during the night. The main disadvantage of those optical-based systems is the high rate of false alarms, due to atmospheric conditions (clouds, shadows and dust particles), light reflections and human activities. Thus:

• their performance in terms of detection’s speed was slower and less reliable than a trained human tower observer

• the detection performance depends on the fire’s size and its distance from the optical/thermal sensor

• since the cameras are unable to take into account the topography of the land in localization calculations, their estimations might have large localization errors. Thus, even though there is a lot of image-based fire detection techniques, it is apparent that their performance is heavily dependent on the coverage of sensors as well as the topology and the forest’s specific form

For all those reasons, it is common for these systems to be supervised by a human operator. After the fire alarm is generated and the suspicious region of the image is detected, the human operator confirms or discards the alarm. In cases where the human operator is not sure regarding a fire alarm, he can switch the system to manual operation and make additional inspections. Using the system in such a way, human operator efficiency is highly improved.

2.4 Unmanned aerial vehicles (UAV) based systems

By means of the modern sensory technologies, a forest fire can be quickly and accurately detected. For this, unmanned aerial vehicles (UAVs) can provide a full range of multi-sourced data for fire monitoring, which can provide an input to an algorithm which will determine the validity of a fire ignition event. UAVs can be classified, based on their configuration, as fixed wing, flapping wing and rotary wing models [23].

Fixed-wing UAVs are suitable for long-distance missions, but they need long runway to take-off and land. Flapping-wing UAVs are relatively new in research and do fly like birds and insects. On the plus side, they can perform vertical take-off and landing. Regarding the rotary wings UAVs, they are suitable for flight missions, which include hovering, maneuverability and easy control. However, rotary UAVs have a more limited flight time autonomy than the fixed-wing UAVs (about 30 min with no load and good weather conditions).

Generally, UAVs are used across the world for various civil applications in search and rescue, surveillance, journalism, and agriculture among others. According to Sherstjuk et al. [24], UAVs should fulfill some requirements for their autonomous operation in forest areas for early detection of wildfires:

• all-weather suitability

• self-localization

• navigational autonomy

• cooperation

• payload

• availability

Ideally, all UAVs should perform their missions around the clock even in the most difficult weather conditions. At this time, most UAVs are ranked according to max wind resistance, operating temperature, waterproof and magnetic interference resistance. This requirement is quite important and should be taken into account during the determination of an autonomous system based on UAVs.

3. Requirements of an early forest fire detection system

An early forest fire detection system should be able to notify firefighters as soon as possible in order to minimize the fire caused damage. Moulianitis et al. [32] defined the major requirements of an autonomous early forest fire detection system, as follows:

• robust continuous monitoring of the forest area (CMO)

• fast detection of fire (FDF)

• determination of the exact location of fire (ELF)

• early notification (ENO)

• minimization of faulty alarms (FA)

as well as all the following abilities: configurability, interaction ability, dependability, motion ability, perception ability and decisional autonomy, which should be incorporated in these kinds of forest monitoring systems. More specifically, configurability allows a forest system to be configured for every forest environment. Interaction ability enables the secure, non-faulty communication between components of system in order for fire fighters to monitor and specify the location of fire sources on time. Dependability specifies the level of trusting upon the system. Motion ability determines the capability of system to move towards fire, ensuring minimum flight time and consumption. Perception ability determines the level of autonomy on fire detection and its exact location. Decisional autonomy corresponds to the capability of system to verify if an incident of a potential fire ignition is real and not a false estimation so as to notify local fire brigade. Finally, the above capabilities are classified to the requirements as follows, in Table 1 .

Table 1.

Abilities of a potential early forest fire detection system [32].

3.1 Wildfire management and risk assessment

The increased amount of resources allocated in fire suppression, and yet the inability to prevent the catastrophic megafires, imposes the need to rethink the problem and pay serious attention to fire prevention and quick-fire detection. An important step towards a holistic approach in wildfire management is the identification of the most vulnerable areas and ecosystems, which can then be managed in a way that will prevent the evolvement of an ignition incident into a megafire. Advanced applications and methods which involve geographic information systems, remote sensing data and methods, geospatial statistics, existing knowledge on fire behavior in various fuel types, reliable records of the pyric history and weather data offer a great tool to land managers to plan for fire management under any scenario.

Wildland urban interfaces form particularly vulnerable areas today for the reasons described above. Although in these areas fires rarely turn into megafires, they inflict serious damage to human properties and often cost human lives. Various methods have been proposed to assess the relevant vulnerability of WUI to wildfires. Some of them focus on landscape structure, studying the vegetation and build environment spatial patterns, using typical methods of landscape ecology and landscape analysis [33]. Such methods, especially when they involve the existence of escape routes or fire barriers, can provide significant service in planning more fire resistant settlements in the WUI. Molina et al. in [34] proposed an ignition index for application in the WUI which integrates fuel components, such us fine fuel moisture content, physiographic parameters, weather data and flammability of vegetation components which results in a reliable estimation of the potential risk of fire and, therefore, it can help in prioritizing areas for strict protection and budget allocation.

The Fire Weather Index (FWI), a component of the Canadian Forest Fire Danger Rating System (CFFDRS) is another widely used method for estimating fire risk. It has been tested in several fire prone areas across the globe and, with some adjustments related to the peculiarities of each region, it provides relatively accurate estimations on the possibility of a wild fire [35, 36]. The FWI integrates some important aspects of wildfires regarding fuel moisture, weather conditions and fire behavior, resulting in an index which can then be classified into danger classes from low to extreme. This knowledge can be used in order to increase or decrease the degree of preparedness and alert.

Remote sensing data and methods are also often employed in an effort to manage wildfires in an effective manner, by providing mapping products with high spatial accuracy and predictive value in relation to potential fire behavior and fire risk [37, 38, 39]. Today, there is wide availability of remote sensing data and methods and many of them are offered free of charge. Sentinel 2 data at a spatial resolution of up to 10 m can be employed for land mapping at a high spatial scale, while its temporal resolution of 5 days ensures updatability of the resulting products. Landsat data on the other Hand, apart from landcover mapping, they can also be employed for the calculation of surface temperatures and moisture, both important determinants of fire behavior, during the vulnerable period.

Under the situation of increased fuel load and increased potential for intensive hot fires, the development and use of accurate tools for early assessment of fire risk and the potential behavior of fire is of particular importance. It could lead to the adoption of appropriate measures for managing the most vulnerable areas, towards decreasing the fuel load or developing appropriate strategies for the effective suppression of fire. Therefore, accurate fire propagation models can be used in the operational support of forest fires suppression, in the development of fire propagation scenarios, in the training of volunteer fire fighters, in the planning of actions to be taken by Civil Protection Agencies and in the decision support of local competent authorities. The fire behavior simulation models FARSITE [40] and Flammap [41, 42] are employed in many studies both for assessing potential fire behavior and fire danger with promising results. FARSITE is a two dimensional model which simulates fire behavior in both space and time under varying site and weather conditions. It is based on [43] fire spread model while it incorporates various other models from the international literature that deal with other aspects of fire behavior such as spotting, fire spread of ground and crown fires, etc. The great advantage of this model is that it allows the real time simulation of fire behavior, while at the same time it allows the simulation of the fire-fighting tactics and forces. The results of fire behavior simulation with FARSITE are spatial and non-spatial data regarding fire intensity and spread, flame height and others.

Unlike FARSITE, Flammap is a one-dimensional model which simulates fire behavior only in space independently of the prevailing weather and other conditions during the event. While FARSITE allows the simulation of fire behavior under real conditions, with the advantages mentioned above, Flammap allows the identification of potential hotspots with extreme fire behavior, in terms of intensity and rate of spread. Such hotspots might not be detected properly using FARSITE simply because at the particular time that fire passes from them, the weather conditions, among others, may not favor a fire with extreme behavior. Such spots, however, might be the ones that need careful management for fire prevention and future protection of an area.

It is now widely accepted that the effective management of wildfires needs the integration of several disciplines, including forest and landscape ecology, fire ecology, pyrology, environmental modeling, remote sensing and others in a supplementary manner. Furthermore, particular attention needs to be paid in the pyric history of a region, local knowledge, historical land uses and current trends in order to unravel the mysteries of wildfire and increase the effectiveness of fire prevention and fire suppression. The aim of an effective wildfire management strategy should aim not in the complete elimination of wild fires, which is practically impossible, but in the restriction of their ecological, economic and social cost.

3.2 Image/video-based fire/smoke detection algorithms

In most existing monitoring systems, the following problems should be addressed:

• detection of fire/smoke using images captured from a stationary camera

• identification of fire/smoke using images captured from the camera of a UAV

In both problems, images should be analyzed for deciding whether there is a fire or smoke in the scene in order appropriate actions be activated. The choice of the cameras type is of vital importance for the system’s operation.

The most used types of cameras is optical or thermal. There are many well-known techniques in the literature that use one of these type of cameras [44, 45, 46, 47] for solving the above-mentioned problems. However, in order to obtain a more accurate and robust system, that can detect and identify fire and smoke, under general photometric conditions, in both daylight and night, the joint use of the optical and thermal cameras is proposed. The aforementioned problems can be decomposed into the following strongly related sub-problems:

1. Pixel-based segmentation using temperature and color

2. Pixel-based segmentation by exploiting the temporal information (video)

3. Compactification of segmented regions using spatial information

4. Compactification of segmented regions by exploiting the temporal information and finally

5. The classification using image/video learning-based techniques

that we are going to analyze in the following subsections.

3.2.1 Pixel-based image segmentation using temperature and color

In this section, we are going to describe an image segmentation method, based on temperature and color images, respectively. This method can be applied independently:

• in each pixel of thermal image for the fire detection problem and

• in each pixel of the optical image for both smoke and fire detection

3.2.1.1 Thermal camera-based detection

Thermal camera is used mainly for fire detection and identification, by exploiting the high temperature of fire and its variability with respect to other objects existing in the forest scene. If the flame is “visible” from the thermal camera, it produces high intensity values in the infrared spectra. Thus, by applying a simple hard thresholding rule on each pixel of the image, an intensity-based segmentation into its “hot” and “cold” regions can be easily achieved.

3.2.1.2 Optical camera-based detection

In most real fire and smoke detection systems, optical cameras are used. Most well-known fire and smoke detection techniques are color-based. The basic idea behind these approaches is to adopt a color model for extracting pixels with high probability of being fire or/and smoke. Many color models are well known and have been proposed in the literature. However, the most used, that we are also going to adopt here, are RGB, HSI and YCbCr.

• For flame color identification the use of RGB and HIS color space is proposed in [45, 46]. However, since the most distinctive flame’s feature is its chromaticity and not its luminance, the use of YCbCr color space is also proposed in [48], where the separation between luminance and chrominance is high.

• A simple well-known rule which is used for the solution of the smoke detection problem is the following one:

  • High luminance and low chrominance in an image indicate probable smoke’s presence [47].

Based on that rule in [49] the use of the RGB color model rules along with a simple fuzzy classification scheme for smoke color segmentation, is proposed.

The above-mentioned techniques can, safely, be used for pixel-based classification in both UAV’s and Stationary camera, are easily programmable and perform on a real time basis. The joint use of optical and thermal camera results in a system with high accuracy in the flame detection. However, the unsatisfactory results in smoke classification, which is of vital importance for the system because of smoke’s early appearance in camera’s view, makes the use of temporal information essential for the system.

The main limitation of optical camera is its behavior in illumination variation during different times of the day. Smoke for example, which is an essential factor for early fire detection, because of its faster appearance in camera’s view, is not visible during night, due to the low luminance of the scene. However, this fact is compensated by the improvement of the sensitivity, due to low luminance and temperature, in both modalities in the fire case.

3.2.2 Pixel-based segmentation using temporal information

The shortcomings of IR imaging in long distances and the fact that color information cannot be used by itself to detect fire, because of the variability in color, density, lighting and background, make video fire detection techniques an important component of the system.

The first step in many object detection and tracking algorithms, which is also useful in fire and smoke detection case, is the background subtraction. This step is vital for the efficient operation of the next steps and the whole algorithm. Background subtraction is used both in fire and smoke detection, the output of the algorithm which is the moving parts of the image are further analyzed by color and temperature detection for the presence of fire or smoke.

There are many techniques used for background subtraction, the most known of them are the following:

• Running Gaussian average

• Temporal median filter

• Mixture of Gaussians

• Kernel density estimation

• Sequential Kernel density approximation

• Eigen-backgrounds

Each one of these techniques has different requirements in memory and complexity. Most of them are based on a model in which every pixel is considered as statistically independent from the others. The basic steps in all background subtraction methods are the following:

1. background estimation and

2. pixel’s classification in background/foreground.

Because in a forest the background changes dynamically with time, many background objects such as trees and leaves are moving. Thus, the most common choice in such cases is the Mixture of Gaussian model, proposed in [50] which can adapt in multiple backgrounds with high performance. According to this technique, every pixel’s intensity is modeled by a mixture of Gaussian distribution, composed by ${\rm K}$ different gaussian distributions. Each one of the $K$ distributions describes a different object of the scene that can be classified as background or foreground. An important issue in background subtraction techniques is the selection of the parameters to be updated. In the smoke detection case, it is important for the system to operate robustly at day and night, independent of weather changes and at different seasons, so it is necessary to update the parameters based on the scene changes, in such a way that the smoke occurrence is not learned as background.

3.2.3 Compactification of segmented regions using spatial information

3.2.3.1 Pixel-based case

Ideally, the result of the image segmentation algorithms (independently if they come from color or temperature image processing) we would like to be compact binary images that classify the fire/non-fire and smoke/non-smoke like pixels. Although a certain number of non-fire or/and non-smoke pixels can be removed using simple nonlinear filtering, it is still difficult to solve all possible problems that can be inhered using just color and background information (e.g., such a difficult problem is to distinguish a moving object that has a similar color to a real fire). Therefore, these binary masks are usually analyzed by exploiting somehow the spatial information. To this end, for the identification of the connected components of these pixels, morphological operators and graph-based techniques are used [51]. The simplest way to reduce the false alarms in both fire and smoke detectors, is to eliminate isolated pixels that are falsely classified as fire and smoke, respectively. This idea sounds good in most cases, except special scenarios, regarding fires that can be captured in just a few pixels of image because of their very long distance from the optical or thermal sensor. In such a case, the elimination of isolated pixels could be achieved by using a median filter in the segmented mask obtained from the above-mentioned methods. However, because of the median filtering:

• the boundaries of the fire are smoothed, and

• denoising emerges some non-fire regions to be falsely detected as fire ones.

In order to overcome both problems, the inhered connectivity of the fire is imposed, that is only the pixels corresponding to a region with area larger than a specified threshold are kept. However, in many cases fire is not fully connected and in practice, the areas of different objects in the scene are ordered and further analyzed using different descriptors [52, 53]. Concerning the classification of flames, since they have varying colors even within a small area, spatial color difference analysis [51] that focuses on this characteristic, can be used for the separation of fire-colored objects from true fires. Specifically:

• range filters [54],

• variance/histogram analysis [55] and

• spatial wavelet analysis [53],

are some of the commonly used tools for analyzing the spatial color variations in fire’s pixel values.

On the other hand, for the smoke classification problem these techniques are not always efficient for the following reasons:

• smoke regions often do not show high spatial color variation as flames do,

• textured smoke-colored moving objects are difficult to distinguish from smoke thus increasing the false alarms.

In general, smoke in a fire is gray and this reduces the color variation in the background. Therefore, in YUV color space the reduction of the dynamic range of chrominance color components U and V after the appearance of smoke in the field of view of camera, is expected.

3.2.3.2 Block-based case

The spatial information can be also exploited in a block-based way. To this end images are cut in blocks of size MxM and properly annotated, thus forming a dataset. This dataset can be used to further test the color-based algorithms, in a block-based classification scenario. To this end, the following M ² hypotheses:

H _i:{ $i$ of M² fire/smoke pixels are enough for classifying the candidate block as a fire/smoke one, i = 1,2,…, M ².

are formed and the classification problem can be easily solved as a classical hypothesis testing problem.

3.2.4 Compactification of segmented regions using temporal information

A dynamic texture or pattern in video, such as smoke, flames, water and leaves in the wind can be simply defined as a texture with motion [47]. Although dynamic textures are easily detected by human eyes, they are difficult to discern using computer vision methods as the spatial location and extent of dynamic textures can vary with time and they can be partially transparent. Dynamic texture techniques are also applied to the flame and smoke detection problem [47]. Ordinary moving objects in video, such as walking people, have a pretty stable and almost periodic motion over time, thus they can be easily separated from fire or smoke. On the other hand, flame and smoke regions exhibit chaotic boundary contours. Therefore, disorder analysis of boundary contours of a moving object is useful for fire and smoke detection. Some well-known metrics that are used for this reason are:

• randomness of area size [54, 55]

• boundary roughness [52] and

• boundary area disorder [56].

The above-mentioned metrics, can be combined with any well-known tracking method to follow the motion of the supposed fire (or smoke), in order to classify it. More specifically, in each frame of the given sequence of images a clustering-based scenario can be applied, thus forming several possible classes and:

• a linear Kalman filter, or

• an extended Kalman, or even

• a particle filter

based tracking scenario can be used for solving the detection problem.

In Figure 1(a) , we can see the change in area of smoke as time evolves in a smoke region in a video. It is evident that the smoke area drastically increases in successive video frames (b–f) and this fact can be used for its separation from other moving objects. The linear Kalman filter can be used for that purpose and its performance is expected to be excellent. Other factors that can be exploited for the smoke’s separation problem are the wind’s velocity and its direction. These quantities can be safely used in a tracking system for predicting the smoke’s motion.

Figure 1.

Change in area of smoke’s region as time evolves (a) in successive video frames (b–f).

3.2.5 Image/video learning-based techniques for classification

In this section, we are going to concentrate ourselves on linear and nonlinear learning-based techniques [57, 58, 59, 60, 61, 62, 63] in order to solve the above defined detection problems. In particular:

• A linear discriminator, tailored to the problem at hand, is a PCA based one [59, 62] with its performance in both fire and smoke detection problems to be very promising. In particular, the performance of the technique in the fire detection problem is very good (sensitivity: 95%—false alarms: 29%), while its performance in the smoke detection problem can be considered as promising (sensitivity: 85%—false alarms: 30%).

• Other powerful classification techniques such as KNN and SVM [63, 64] can be used for solving the detection problems with SVM-based classifier performing better. More precisely, the results we obtained for the fire detection problem were excellent (sensitivity: 99.5%—false alarms: 1%) while the corresponding for the smoke detection problem were quite satisfactory (sensitivity: 85%—false alarms: 30%).

• Finally, recently deep convolutional neural networks have been used [65, 66] for solving successfully the forest fire detection problem and the obtained results are promising.

Some results on images for the fire detection obtained from the application of the block-based PCA and SVM classifiers are shown in Figure 2 .

Figure 2.

Fire detection using the block-based PCA approach (top row) and block-based SVM approach (bottom row).

3.2.6 A video learning-based technique

The above-mentioned methods can also be applied for the classification problem on a sequence of images (video). They can easily be generalized by using spatio-temporal 3-D blocks, though we could also extract the necessary information from the video using region covariance descriptors of the blocks [47] for its efficient solution. A popular approach for the classification of the multi-dimensional feature vectors obtained from each candidate flame or smoke blocks is again based on SVM classifiers, typically using Radial Basis Function (RBF) kernels. Many frames of fire and non-fire video sequences must be used for training these SVM classifiers, otherwise the number of false alarms (false positives or true negatives) might be significantly large [67].

3.3 Navigational autonomy

Autonomous navigation of UAV depends mainly on drone’s ability to localize itself. In outdoor applications, UAVs can execute used-defined waypoint mission, using geo-localization of their spatial positions. This GPS approach is ideal for missions with limited presence of obstacles and tolerable small error in positioning accuracy. Nevertheless, this GPS localization method is not functional in indoor environments or outdoor environments in which navigation is heavily based on local events. To deal with this problem, simultaneous localization and mapping (SLAM) techniques are employed. SLAM is the computational problem of constructing or updating a map of unknown environment while simultaneously keeping track of a robot’s location within it. Several works have been published in that field [25], but it remains an open research area. Yang et al. [68] proposed an algorithm for the detection of landing sites, using monocular camera. An alternative SLAM approach was proposed by [69], in which SLAM algorithm estimation comes from fusion of data such as GPS, orientation sensor and monocular camera. Regarding the applications of SLAM in forest environments with UAVs, there is a scarcity of research footprint, which either highlights a new research area or confirms that UAV forest monitoring is quite functional with GPS localization accuracy.

Despite self-localization attribute of UAVs, autonomous navigation is based on motion planning. Given that UAVs are able to follow user defined waypoints, an outdoor autonomously navigated UAV should be able to generate routes (a list of waypoints) between current position and target position, ensuring a collision free route automatically. This can be achieved through motion planning, a process of breaking down a desired movement task into discrete motions that satisfy movement constraints and possibly optimize some aspect of the movement. In literature, there are many UAV 3D path planning algorithms, which can be divided in five categories: (a) Exact and approximate cell Decomposition, (b) control-based methods, (c) potential fields, (d) bioinspired algorithms and (e) randomized planning.

Exact and approximate cell decomposition technique splits the workspace into discrete cells corresponding to the obstacle free portion of the environment, resulting a graph roadmap, where the vertices represent the individual cells and edges indicate adjacency among the cells. Control-based methods are based on motion equations of drone, in order to navigate a drone along a specified trajectory. Potential fields represent obstacles with vector repulsive forces and goal position with a vector attractive force. So, the drone navigates in workspace using gradient descent to follow potentials to the goal. Bio-inspired algorithms originate from mimicking biological behavior to deal with problems. This path planning method leaves out the process of constructing complex environment models, and proposes a strong searching method to converge to the goal stably. Finally, randomized planning is achieved through random generation of nodes in a graph, such as rapidly exploring random tree [70] and probabilistic roadmap [71]. In literature, there are many variants of randomized planning algorithms which incorporate optimal search algorithms, such as Dijsktra’s algorithm [72], A* [73] and D* [74] respectively.

Despite the great contribution of motion planning in navigational autonomy, there is a trade-off between effectiveness and energy consumption. There are two ways to execute a motion planner, offline and online. The first one requires a computer, in which motion planner is executed, and a telemetry link between computer and UAV. This method is ideal for small outdoor areas due to limited range of antenna radio waves. The latter one includes an installed mini-computer on drone, which will be able to generate routes without any transmission link. Online motion planning is quite effective for long-distance routes, but requires more energy for the operation of installed computer.

Another set of important parameters for an efficient outdoor path planning algorithm, is time and energy indexes. Sometimes, minimum time paths come from minimum Euclidean distance between start and goal position, while some others do not. This happens due to wind forces which impede the waypoint mission execution. To handle that problem, [75, 76, 77, 78] proposed path planning approaches which extends path planning capabilities to a wind efficient navigation model, which is both time and energy optimal, while extend the capabilities of outdoor motion planning based on GPS positioning as Prasad et al. [79]. Thanellas et al. [77] show among others, that wind information can offer energy efficient paths, which are time optimal. This technique can be proved beneficial to outdoor environments in which drones execute waypoint missions without any knowledge of environmental factors which drain battery level.

These works prove that wind aware path planning methods are beneficial to outdoor environments. However, there is a problem of real-time wind prediction. More specifically, there are wind predictions which are not updated on time to predict wind variances for optimal wind path planning. Oettershagen et al. [15] support that onboard processing of a wind map can minimize the delay of wind data acquisition. There is still a lot of work to be done, but wind path planning is for sure an important parameter for long paths.

Sometimes the effective covering of a forest area depends from the multiple monitoring or sequential deployment of UAVs. In these cases, UAVs should be able to coordinate their behavior and to cooperate with each other in order to solver their tasks optimally. Additional information that aims the coordination of the UAVs can be provided by static cameras or other static sensors. The information of the sensors can be used to update the flight plans of the UAVs towards better covering and early detection of wildfire incidents.

The UAVs should be able to carry all required sensors and systems for fire perception purposes. Optical and thermal cameras are the most common sensors used to detect smoke and fire. On board computers are used for signal processing and feature recognition. Light weight weather stations are mounted on the drone in order to detect locally the direction and the velocity of the wind in order to adapt the flight plan of the UAV for energy and time efficiency.

All UAVs should be equipped with onboard communication devices that guarantee receiving commands from a ground command center, sending information back to it, as well as exchanging information with the other UAVs.

4. Conclusions

In this chapter, a brief history of fire and forest fire protection systems was presented. The basic forest fire monitoring systems technologies were reviewed and satellite, WSN and optical/thermal cameras based systems were emphasized. In addition, the vital requirements of an autonomous early forest fire detection system, its main modules and methods for wildfire management and risk assessment, smoke and fire detection based on images and video as well as navigational autonomy issues for UAVs were also presented.

Acknowledgments

This research was supported by the EU and Greek State on behalf of the pro- gram “Forest Monitoring System for Early Fire Detection and Assessment in the Balkan-Med Area (SFEDA)”, MIS: 5013503.