Use of Drones for Digitization and Monitoring the Built Cultural Heritage: Indoor and Outdoor

Abstract

Digitizing is the way for a revolutionary approach in knowing, analyzing, continuous monitoring, and preserving the tangible immovable cultural heritage. The built cultural heritage requires the most performant means and techniques to acquire information indoor and outdoor. Drones are the best platforms for this purpose in terms of operating costs, data accuracy, and mission planning flexibility. In this chapter, we present a survey on the main applications of drones in the field of built cultural heritage analyzing the usability of this technology. Essential technical issues that are important for the operation and understanding of the use of drones in specific missions for the study of built heritage are also discussed.

1. Introduction

Digitization has become an ongoing goal on the agenda of economic development and social transformation. It offers a broad perspective on the very near future of humanity embodied in current paradigms such as the web-driven economy, e-government, e-society, or e-communities that are based on digital democracy and promote it at the same time. Digitization is a priority all over the world and is seen as a strategy for the profound development of all sectors of human activity. One of the most relevant examples is the program promoted by the European Commission under the slogan “A Europe fit for the digital age”, which guides how digital technology is changing people’s lives by empowering people with a new generation of technologies [1]. Several concepts are now circulating such as big data and cloud computing and a number of technologies such as data mining, data analytics, data fusions, and deep learning are currently used and are constantly improving to keep up with the huge production of data in all fields.

Cultural heritage is a part of data production and has been contributing to the informational treasure of humanity for millennia. The digitization of cultural heritage is only a step in collecting and manipulating data with two major purposes: storage for information preservation, respectively data analysis for the study, and advanced research. A recent European Commission report on shaping Europe’s digital future focuses on 3D digitization of cultural heritage [2]. This is a roadmap for the digitization of tangible cultural heritage that highlights that the integration of data obtained through different scanning techniques is the right approach for the future. Knowledge of technologies for transforming tangible heritage objectives into data by scanning across different spectral bands and dimensional measurements, including software components for data analysis and presentation, is very important. The digitization of tangible cultural heritage is not only a fashionable technology but a tool that tends to become a standard for the collection, preservation, and dissemination efforts of arts and cultural heritage worldwide [3]. All in all, digitization is a necessity for a better knowledge and interpretation of things, so research becomes much more efficient using data instead of physical artifacts, especially in the case of tangible real estate. Sometimes access to the physical object is impossible, or very expensive, and then, a set of data captured with the right sensors is very useful. On the other hand, data become more democratic and thus can reach the general public through the media or virtual products in the service of knowledge of cultural heritage. In fact, through digitization, tangible cultural heritage becomes digital heritage, which is a subcategory of intangible cultural heritage.

In the last decade, drones have been used in many industries such as construction and infrastructure, agriculture, environmental monitoring, mining, GIS, and so on. For all these areas, drones provide imaging data of various types: single aerial pictures, thermal and multispectral images, stereoscopic images, video content, data from laser scanning, and remote sensing. A significant number of bibliographic sources report on drone technology and airborne sensors and their specific applications and services. Most case studies are presented even by professional drone manufacturers, and a wide collection of information can be found on their websites, for example, [4, 5, 6]. A recent report on the leading manufacturers of drone technology, including their applications, can be found in the reference [7].

At present, drones have begun to be part of the arsenal of means of investigating cultural heritage, offering the possibility to fly over and supervise heritage objectives from the air, with low operating costs. In principle, they offer photogrammetry services but the applications are open to possible remote exploration and sensing tasks in archeological sites, instead of humans. An extensive and recent synthesis of the use of drones in the service of cultural heritage, including examples of applications and case studies conducted around the world, can be found in [8].

2. Drones and digitizing

A drone is an unmanned aerial vehicle (UAV) that can be remotely controlled by a human operator in a specific area of ​​action. This type of aircraft is an excellent platform for various scanning equipment, and sensors capable of transmitting acquired data in real time, as well as its current position. Drones can provide a wide range of services, but most applications include airborne surveillance and monitoring tasks. There are drones for military purposes and drones for civilian use, but we will discuss here drones with civilian applications.

The basic mission of drones in the service of cultural heritage is to scan various objects, artifacts, sets of objects, places built of cultural interest, and using different techniques for obtaining digital images.

2.1 Photogrammetry

Traditionally, aerial photogrammetry is the science and technology of obtaining reliable information about physical objects and the terrestrial environment through the process of recording, measuring, and interpreting photographic images captured from height. Currently, digitization has extended the field of photogrammetry to the analysis and processing of images based on mathematical and geometric models with software-implemented algorithms. Automatic image processing works with huge amounts of data that drones are able to provide by mobile scanning over areas of interest.

Aerial images can be processed and interpreted in different ways. One of the most used methods is a 3D reconstruction based on 2D images. This task defines particular uses of the drone in controlled overflight scenarios, which differ from one objective to another. Another method is orthophotography through which the objectives are mapped 2D, resulting in the digital map of the objective and the area flown over with the planimetry information. These methods include geometric models and algorithms for analytical geometry. Another category of methods aims at chromatics and image illumination, which involve extracting components and color ranges, estimating specular reflection, determining ambient lighting and its interaction with materials in order to render physical objects. This is where digital image analysis algorithms take place in the visible or multispectral domain. The combination of methods, for example, orthophotography with chromatic methods produces orthomosaic maps, and by the combination with multispectral data, various indexed maps are obtained based on normalized difference vegetation index (NDVI), optimized soil-adjusted vegetation index (OSAVI), chlorophile map, or processing CIR Composite (color infrared), or digital surface model (DSM).

2.2 Laser scanning

This is a technique for directly obtaining 3D images using laser radiation using LiDAR (laser imaging, detection, and ranging) devices. Unlike photogrammetry, which is a passive method of capturing images, LiDAR is an active method that involves laser emission in the NIR or UV spectrum. Mobile laser scanning is also beginning to be accessible to drones through aerial LiDAR equipment that has evolved to meet the requirements of weight, size, and performance. Laser scanning involves technical conditions and additional requirements to photogrammetry. Knowing the position of the drone as accurately as possible at all times is crucial for the quality of LiDAR data and therefore, these systems have integrated inertial navigation sensors with very high accuracy. Laser scanning has several definite advantages versus classical photogrammetry, but it cannot surpass resolution performance, image realism, data accuracy, and ultimately the cost of photogrammetry equipment. In the LiDAR technique for each scan radius (direction) only two parameters are obtained: flight time—which is directly proportional to the distance and intensity of the reflected radiation. With this information about each scanned point, a synthetic image of the objects is built respecting their geometry with a certain precision, while all the chromatic characteristics are conventionally chosen. However, some advantages prevail for laser scanning technology: It can operate at night, in an atmosphere with clouds and smoke, and can reconstruct more precisely the surfaces covered by vegetation. Also, the time required for post-processing LiDAR data is much shorter than when processing photo images. In various applications, LiDAR technology is used in addition to the classic photo-video technique.

The drone, as a system, is capable of providing raw imaging data for the above-mentioned processing, while a suite of application software programs effectively performs the appropriate processing to extract the desired information. In fact, these are stand-alone software tools that perform advanced data processing including artificial intelligence techniques.

2.3 Technical issues

The configuration of drones for civilian use is of a VTOL (vertical take-off and landing)-type aircraft with fixed wings or the most popular with rotary wings. Here are the main component systems (subsystems) of a professional drone for civilian use:

• The structure and the propulsion engines: It constitutes a unitary assembly made of resistant and light materials in a compact and aerodynamic configuration with rotor-type propellers. The structure usually has foldable elements so that it can be stored and transported more easily.

• The sensor system: It provides data for drone self-monitoring and navigation data. On the main directions of movement, there are video sensors for detecting obstacles and measuring the distance to them and also IR sensors for detecting and telemetry of obstacles up and down. For this purpose, the drones can also be equipped with additional (redundant) ultrasonic or LiDAR sensors. Navigation sensors include the compass, the global navigation system receiver (for GPS coordinates), and the inertial measurement system (IMU) consisting of a gyroscope and accelerometers.

• The airborne surveillance system: It generally consists of a video camera with controllable orientation, but may also include a thermal imaging camera or multispectral cameras depending on the mission of the drone.

• The communication system: It contains the airborne transceiver with separate frequency channels for the remote control of the drone flight and the airborne systems, respectively for image downlink, as well as the paired transceiver in the portable remote control unit. The communication subsystem also contains a number of interfaces for data communication such as the USB port, the micro-SD card slot, and the port for connecting additional accessories to the drone (beacon, speaker, lighting projector).

• The power system: It includes the drone battery that supplies all the subsystems in the drone composition, respectively the battery of the remote control equipment.

• The electronic command and control system: It represents the brain of the drone and it ensures all the functions of the onboard subsystems such as control of the propulsion system, control of sensors, control of telecommunications, and control of surveillance equipment. The control of the major subsystems of the drone includes various parallel command and real-time control tasks such as independent speed control of each engine, stabilization of surveillance cameras, battery control, and radio power control. The brain structure of the drone is based on a multiprocessor architecture with a powerful master processor and several slave processors with distinct responsibilities.

• The remote control equipment: It is the user’s portable unit—an HMI (human-machine interface) that provides the graphical control interface and the effective means of command of the drone (buttons and sticks). Usually, this role can be provided with a tablet or smartphone, but professional drones come with their own dedicated remote control unit that includes the display.

Last but not the least, a special and vital component of drones is the software system that is distributed on both components: built-in drone, respectively on the portable remote control unit. The software component actually defines the drone’s brain and its so-called intelligence, effectively ensuring all the processes for its proper functioning.

Figure 1 shows an overview of the Mavic 2 Enterprise model, where the main subsystems can be identified. Full details can be found by accessing the official manufacturer’s website available from: https://www.dji.com/mavic-2-enterprise/downloads

Figure 1.

Professional drone Mavic 2 Enterprise with native surveillance camera mounted in front of the gimbal joint.

2.4 Features and functional parameters

Here, we will review the basic characteristics of drones and detail the functional parameters that are relevant to the tasks of digitizing the objectives of tangible cultural heritage.

We mainly distinguish between technical characteristics and operational characteristics, the latter depending largely on the former, and together, they determine the use class of the drone, its performance, and finally the purchase price on the market. First of all, we need to understand that drone performance is the result of a technical compromise that is reflected in their operational capabilities. Current technology manages to optimize this compromise by balancing power and speed requirements versus flight distance and height, weight and gauge versus air range (maximum flight time), data processing, and transmission capability versus sensor resolution.

In general, the mission of a drone is to acquire images with very good resolution from precisely defined and very well-controlled positions. In other words, drones must provide quality digital material for photogrammetry and image processing techniques. Thus, in addition to the general performance of maximum speed, maximum service ceiling above sea level, and maximum flight time, the following features are very important: hovering accuracy range, parameters of the camera, and gimbal of camera. In Table 1 has given selectively these characteristics for a reference model—the Mavic 2 Enterprise drone.

Technical/operational feature	Value/limits	Notes
Max takeoff weight	1100 g
Max speed	72 kph	Near sea level, no wind
Max ascent speed	5 m/s
Max descent speed	3 m/s
Max service ceiling	6000 m	Above sea level
Max flight time (no wind)	31 min	At a consistent speed of 25 kph
Max hovering time (no wind)	29 min
Hovering accuracy range	Vertical:
	±0.1 m, ±0.5 m	With vision positioning; With GPS positioning
	Horizontal:
	±0.3 m, ±1.5 m	With vision positioning; With GPS positioning
Parameters of camera	Effective pixels: 12 megapixels	Sensor: 1/2.3″ CMOS
	Auto focus at: 0.5 - ∞
	Max image size: 4056 × 3040 (4:3) 4056 × 2280 (16:9)	Photo format JPEG, DNG (RAW)
	Video resolution: 4 K, 2.7 K and FHD ISO Range; Photo: 100–1600 (auto), 100–12,800 (manual) Video: 100–3200	Video Format MP4/MOV (MPEG-4 AVC/H.264)
Gimbal	Mechanical range: Tilt: −135 to +45°; Pan: −100 to +100°; Controllable range: Tilt: −90 to +30°; Pan: −75 to +75°; Stabilization: 3-axis (tilt, roll, pan); Max control speed (tilt): 120°/s; Angular vibration range: ±0.005°

Table 1.

Selected features of Mavic 2 Enterprise drone.

Considerations related to the accuracy of data collected by drones are discussed in [9]. The quality of the images provided by a drone is described by three essential characteristics [10].

1. The pixel resolution of an image is the number of pixels that make up the image. It is expressed by the number of columns and rows, such as 4056 × 3040, or directly by the total number of pixels, such as 12.3 Mpixels (4056 × 3040 = 12,330,240). This parameter is important for data sharing and storage, image display, and digital zoom.

2. Ground sampling distance (GSD), in mm/pixel, is the distance between the centers of two adjacent pixels, measured on the object observed in the image. This parameter depends on the size of the camera sensor and its actual number of pixels, but also on the distance to the photographed object. For example, a GSD of 1 mm/pixel means that one pixel per image is 1 mm in the real world. A smaller GSD means that the object will appear larger and that smaller details will be visible in the image. For example, a photo image can reach one million pixels/m², while a LiDAR image can only reach a few hundred pixels/m². Ground sampling distance is an important measure to consider for photogrammetry and measurements in images. However, GDS does not fully describe the ability to detect and characterize an object or detail in an image.

3. Spatial resolution or angular resolution describes the smallest details visible in the image. Unlike theoretical GSD, spatial resolution can be expressed in a different unit, which takes into account blur, image noise, contrast, and in general the effects of image processing: compression, denoising, edge clarity, etc. Spatial resolution is therefore a correct metric to quantify the ability to detect and characterize an object in the image. Spatial resolution is often expressed in “pairs of lines per millimeter.” This unit is used to describe the spatial frequency of alternating black and white line patterns.

Finally, another photometric parameter that influences image quality is the ISO exposure value at the image sensor. Under normal lighting conditions (daylight), the exposure value is set to the lower limit of the range values and vice versa, and at lower lighting levels, the exposure value is set above. However, a high ISO value of exposure produces image noise, and a long exposure time produces motion blur when the camera moves. This reduces the image quality, and eventually the ability to distinguish small details in the image.

3. Drones in the service of heritage

3.1 Applications and methods

A survey of the latest applications of the use of mandrels for cultural heritage purposes reveals two aspects. First, there are various subdomains or particular purposes with concrete tasks where drones, as providers of digital content, prove their usefulness. Specific applications can be classified as follows:

1. Reproduction of virtual models, especially for architectural heritage, is the widest class of applications. HBIM (historical building information modeling) technology as part of BIM (building information modeling) technology is one of the most used digitization activities in the service of the tangible cultural heritage in which drones prove their effectiveness. Here, based on panoramic images captured by drones, 3D reconstruction is the most frequently addressed technique. A suite of cultural heritage virtualization projects can be viewed on the following websites: [https://www.3deling.com/heritage/], [https://iconem.com/en/].

2. Non-destructive analysis of heritage sites and objects is an area of activity that can fully exploit the drone service in data acquisition. We mention here exterior and interior photogrammetry missions on frescoes, mosaics, upholstered surfaces, decorative stucco, and bas-reliefs.

3. The conservation of the material patrimony requires as accurate and complete information as possible in the effective restoration activity. The reference digital models help both the restoration work and the sustainable preservation and management of the heritage.

4. The actual restoration action can be automated and effectively driven by data by robotic interventions and reconstruction by additive techniques, such as 3D printing technology.

5. Artifact authentication is another activity that can fully benefit from the digital support provided by drones in special situations when the place cannot be explored on land or the object is in dangerous or contaminated places and when there is a risk of destroying the artifact by other types of examination.

Second, the review of recent literature reporting various applications of drones in the service of cultural heritage reveals the complementarity of several digitization technologies with that of drones, as well as strengths, weaknesses, and limitations of these technologies [11, 12, 13, 14]. As the main technique for capturing images, traditional aerial photogrammetry has now become accessible through drones at a very good performance-cost ratio. Photogrammetry and laser scanning are the basic techniques applicable by various methods with distinct equipment, but for the production of digital content of cultural heritage objectives, several scanning techniques are available. The main concepts frequently used in the digitization of the material cultural heritage are based on the following methods:

• Close-range photogrammetry (CRP) is considered when the subject is observed from less than 400 m either from the ground or from the air. This is a cheap and sufficiently accurate method for 3D photogrammetry based on stereoscopically associated overlapping 2D images. For aerial applications with drones, CRP is the ideal solution because the cameras have a lower weight and size, compared to laser scanners, for example.

• Structure from Motion (SfM) is a technique based on automated photogrammetry that facilitates the collection of moving images. This is the standard method for 3D reconstruction in the field of cultural heritage. In principle, it is applied within the CRP with the determination of the best overflight height and the establishment of the optimal spatial resolution, and the orthophotography acquisitions with an overlap of at least 60% are scheduled. The image collection is then processed with SfM software based on 3D reconstruction algorithms.

• Airborne LiDAR scanning (ALS) is a complementary or alternative photogrammetry technique to create a digital terrain model (DTM) or digital elevation model (DEM). 3D reconstruction of cultural heritage objectives by laser scanning with drones is becoming an increasingly accessible technology.

• Terrestrial laser scanning (TLS) is used as a basic technique or to complete the acquisition of 3D images of cultural heritage objectives with fixed ground equipment, which gives a very good data accuracy.

• Mobile laser scanning (MLS) contributes to massive point-capture technology along with photogrammetry, using LiDAR equipment mounted on land vehicles, ALS, or with handheld scanning devices. The use of this equipment involves special SLAM (simultaneous localization and mapping) technology for capturing images and point clouds in motion and real time.

In specific applications for the material cultural heritage, there are some peculiarities that influence the scanning techniques used, as follows:

1. Objects are motionless, so there are virtually no relative dynamics and images can be considered static.

2. Some artifacts require photography from a short distance outdoor but also indoor.

3. Indoor, natural lighting is usually poor.

4. In inaccessible places, the real size of objects is generally not precisely known, so the exclusive photogrammetric interpretation is relative.

Interesting studies addressing the combined use of air and ground scanning technologies for cultural heritage objectives are reported in [13, 15].

3.2 Outdoor and indoor missions

We have seen that drones can be used successfully for both outdoor and indoor photogrammetry and laser scanning operations. Most applications are outdoor missions for HBIM tasks but some indoor missions are suitable for drones, in concrete situations these being the only means that can make data acquisition at reasonable cost-effectiveness. In [16], it is presented a comparative study of digitization of land surfaces, photogrammetry versus laser scanning, conducted for four types of drones. These results are interesting and useful for professionals in the field of cultural heritage. A project reported in [17] focused on HBIM for Byzantine churches in Cyprus using exclusively low-altitude outdoor photogrammetry, provides methodological details, and results obtained with a drone equipped with a 20 MP camera. In Romania, there are some important cultural heritage objectives that are being investigated by photogrammetry with the help of a drone. One of them is the large architectural monument—the medieval castle named Corvin Castle, also known as Hunyadi Castle, in Hunedoara (Figure 2). The other is the Adamclisi Fortress in Dobrogea, which is an ancient Roman architectural complex, today in ruins (Figure 3).

Figure 2.

The Corvin castle in Hunedoara, Romania.

Figure 3.

The Adamclisi Fortess ruins in Dobrogea, Romania.

These applications require the planning of particular flight missions with predefined itineraries for photogrammetric capture with different viewing angles on ground objectives. Usually, two gimbal angles are used for the camera: −90°, that is, vertical downward direction, called nadiral view, and oblique direction at −45°. Practically, a methodology and planning of photography are established for each objective. The goal is to best capture the elevation of objects.

The indoor missions in the field of cultural heritage are to complete the HBIM from inside when the TLS and other MLS methods are not applicable. Recent case studies with the use of drones for visual inspection in enclosed spaces such as mine galleries, cisterns, or sewers are reported in [18]. In the case of indoor scanning missions, the drone does not benefit from GNNS services, that is, GPS signal for positioning; however, piloting the drone is done in P (positioning) mode when the vision systems to locate and stabilize itself and obstacle sensing function is enabled. Other indoor scanning purposes require drones hovering over the artifacts in order to capture the best image possible. In these conditions, hovering accuracy is the feature that counts, and the best results are obtained by piloting the drone in T-mode (Tripod), which makes the aircraft more stable during the shooting. An example for this use case is the inspection of the roman mosaic arts in Constanta during the expertise for restoration. This is the subject of nondestructive analysis by evaluation of the morphological and chromatic characteristics that represent suitable metrics for making decisions based on image processing [19]. Figure 4 presents this artifact in the present condition of conservation. For a reliable analysis, quality imaging data obtained by correct photogrammetry techniques are required. Thus, for correct analysis, the images of the mosaic, as a primary source of data, must meet certain conditions from the acquisition phase, as follows: (i) to be taken orthographic shots, (ii) to be captured under uniform lighting conditions, without shadows, reflections, etc., (iii) to be taken from the same height (constant distance) for the entire interest surface, and (iv) the resolution must be as high as possible. In general, the photogrammetric method is sufficient for the inspection of artifacts such as flat decorative surfaces, so that 2D orthogonal images obtained by single shots provide all the planimetry and color information necessary for morphological and chromatic analysis. Using CRP with SfM techniques, it is possible to obtain details for DTM by estimating the deformations of the mosaic surface, the degree of degradation by erosion, and the lack of mosaic elements or the degree of intervention by adding material. ALS is not an option for scanning the decorative mosaic because an acceptable value of the GSD parameter cannot be achieved. Also, due to the restriction of access on the surface of the mosaic, scanning by terrestrial means is not possible in this case. In Figure 5, it can be seen two shots taken manually at the arbitrary angle but also the effect of non-uniform environmental lighting.

Figure 4.

Ancient mosaic art in Constanta. (a) Indoor floor view. (b) Details of the artifact.

Figure 5.

Different images of the artifact.

4. Conclusions

Professional drones are actually considered UAS (Unmanned Aerial Systems), which means more than an unmanned aerial vehicle. They are equipped with specific scanning systems that define their role and operational functions. The drone is a sufficiently stable platform for close-range photogrammetry (CRP) missions and is an excellent indoor scanning device due to its small size, good maneuverability, and flight qualities. We see great potential for the use of drones for interior photogrammetry on decorative artifacts where the information of interest concerns their planimetry and chromatics. The ease of use of airborne cameras in terms of gimbal stabilizer-controlled mobility, controlled focusing, and exposure function combined with the drone’s ability to hover at a short distance from the artifact gives drones high versatility for digital image acquisition. By using the auto exposure bracketing (AEB) function, for example, the camera can take several successive photos (usually three) with slightly different settings. Then, the images can be combined automatically, for example, in a single image with a high dynamic level or can be stored separately, so that the images with the most suitable appearance can be later taken from the batch.

Regarding the digitization of cultural heritage objectives, the main data are obtained through photogrammetric techniques, which in most cases cannot be exceeded in terms of accuracy and amount of data provided by LiDAR techniques. Moreover, the chromatic analysis of images can be performed exclusively by photographic techniques. Laser scanning techniques have several strengths that make them rather useful as complementary methods in digitizing cultural heritage objectives. Thus, the ALS technique generally helps in the case of infrastructures covered with vegetation and in the case of noisy photographic images when the estimation of the 3D model would be deficient.

Professional drones are becoming increasingly affordable handy tools for use in the field of material cultural heritage.

Acknowledgments

This study was supported by the grant PN-III-P1-1.2-PCCDI-2017-0476, no.51PCCDI/2018, from UEFISCDI-MEN.