Open Access is an initiative that aims to make scientific research freely available to all. To date our community has made over 100 million downloads. It’s based on principles of collaboration, unobstructed discovery, and, most importantly, scientific progression. As PhD students, we found it difficult to access the research we needed, so we decided to create a new Open Access publisher that levels the playing field for scientists across the world. How? By making research easy to access, and puts the academic needs of the researchers before the business interests of publishers.
We are a community of more than 103,000 authors and editors from 3,291 institutions spanning 160 countries, including Nobel Prize winners and some of the world’s most-cited researchers. Publishing on IntechOpen allows authors to earn citations and find new collaborators, meaning more people see your work not only from your own field of study, but from other related fields too.
To purchase hard copies of this book, please contact the representative in India:
CBS Publishers & Distributors Pvt. Ltd.
www.cbspd.com
|
customercare@cbspd.com
Although underwater drones are no novel technology, their widespread use in civil and industrial applications has not been widely accepted so far. Apart from that, the decrease in size and costs along with an increase in robustness of underwater drones and the ease of handling provide a strong basis for underwater drone technology to grow in various markets. This chapter introduces the application of underwater drone technology in maritime operations, focusing on the micro ROV class. An introductory framework evaluation based on a structured literature analysis of the current state of research is conducted in order to provide a structured outlook on areas of drone operations in the maritime domain. Furthermore, the combination of micro ROV and artificial intelligence in the form of a neural network based on deep learning is introduced. This contribution provides an introductory analysis regarding both operational sides of science and the industry in order to shed light on the existing literature gap as ground for future research.
Stein Maritime Consulting, University of Antwerp Faculty of Business and Economics, Antwerp, Belgium
*Address all correspondence to: stein@stein-maritim.de
1. Introduction
As 71% of the world’s surface is covered by water [1], this element is and will be crucial for our survival. The oceans reflect the largest habitat of wildlife on this planet, whilst also providing very large amounts of crucial resources and facilitating over 90% of the global trade via shipping operations. Knowledge about what happens underwater is important to maintain and explore the ocean’s potentials in balance with its fragile ecologic system. However, the oceans even today remain the least explored area of this planet for reasons of unavailability to the human eye. Whilst only about 5% of the oceans were stated to be explored in 2016 [1], the number increased to approx. 20% in 2020 [2]. Advances in underwater robotics have promoted comprehensive studies of oceans and exploration of areas previously out of reach of humans [3]. Even though deep sea and offshore operations of unmanned vehicles have existed for decades through operating working-class systems, their cost and size were a limiting factor accessible to specific industries only. With advancements in the so-called inspection class systems [1], the availability of underwater drones to new markets and research facilities has risen. In the past, researchers were often excluded from deep sea operations due to cost-intense training, resulting in scientists being dependent on a third party data provider or industry partner [3]. The cross-sector partnership of research and industry remains crucial for sustainable human activities in the ocean [4]. However, the widespread availability of reliable, cheap and easy-to-operate systems enhances the independent data-capturing opportunities, allowing both scientific and entrepreneurial developments.
Drones reflect a human-controlled robot that is designed to carry out tasks in remote areas. There are many classes and definitions of these robots generally originating from the aviation industry. The term “drone” dates back to World War 2 air force target practice operations and is still used for unmanned robotics nowadays. In the 1960s/1970s, underwater drones were predominantly developed by the Navy with systems like CURV I–III before the technology was adopted by the oil and gas industry in the 1980s [1].
This chapter follows the terminology for ROV (remotely operated vehicles) that in other contributions might be referred to as UUV (underwater unmanned vehicles). As this chapter focuses on human-controlled hardware, the area of AUV (autonomous underwater vehicles) is acknowledged but disregarded. The systems described in this chapter are operated on the surface through a cable for data connection. From aviation robotics, it is known that two to five times the weight of the pilot in specialised and redundant equipment is needed to ensure both the pilot’s and the vehicle’s safety [5]. These scales can to some extent be transferred to ROV operations, resulting in a reduction in size and costs of the system by excluding the human from robotic costs. This fact, in combination with current technological advances, allows a growing number of ROV systems to access areas formerly impossible or at least challenging for human intervention. It is worth mentioning that ROV systems and their corresponding operations are seen as individual entities from any infrastructure in the context of this chapter. There are, however, first projects which view ROV technology as a logical extension of infrastructure, such as ARES (Autonomous Robotics for the Extended Ship) [6] that might change this point of view in the future.
1.1 ROV system classification
This chapter follows the general classification of ROV systems provided by [1] as shown in Figure 1. Manned and autonomous vehicles are disregarded due to the limitations of this chapter’s evaluation. The intervention class ROV systems are named for the purpose of a holistic point of view but are also not part of the evaluation. These systems will remain available for a limited number of industries or sectors only, given their high investment costs and their weight of up to 5.000 kg [1] requiring specialised crew and LARS (Launch and Recovery System). The focus of this chapter lies on the inspection class and its micro sub-category that allows independent, mobile operations of one person or a very small team of pilots.
Figure 1.
Underwater vehicle classification matrix Source: Author based on [1].
Micro ROV systems often weigh less than 10 kg and are mobile enough to be transported in a small box or even in a backpack format. This high degree of mobility allows for a flexible operation out of a helicopter (e.g. for disaster response) or onboard a ship even in an open-sea anchorage, where the ROV operator enters the ship over a pilot ladder with the ROV. The micro ROV systems are predominantly reduced to video and manoeuvring systems only allowing for a fast and cost effective inspection. Some systems allow basic grabbing attachments for small retrieving operations or automated positioning systems. On a more advanced side, the medium-sized ROV provides open frame space for additional sensors to be added. These additional sensors then come with a trade-off regarding costs and size, often requiring a winch for launch and recovery. Opposite to micro ROVs, medium-class ROVs are operated via a communication station requiring a power supply and more complex human interaction compared to smartphones, tablets and/or handheld controller-operated micro systems. Whilst the early micro ROV systems were limited in operational scope due to limited bandwidth in copper fibre cables, recent generations use transition technologies allowing 500–1000 Mb/s over distances up to 500 m [1]. The technological advances of the micro class provide a huge potential for these small and relatively low-cost systems to enter the different maritime markets for the great benefit of different stakeholders.
This chapter introduces the inspection class ROVs with a focus on micro or handheld ROVs to evaluate their future potential for scientific and economic operations. Whilst the offshore industry operated ROV systems for decades, micro ROVs have entered the markets only quite recently, resulting in a gap of knowledge and data. In order to cope with the scarcity of existing literature of newly emerging technologies, a mixed method approach of quantitative and qualitative methodologies as well as grounded theory concepts [7, 8, 9, 10] are applied. Grounded theory approaches have a widespread acceptance in innovation science and are, therefore, chosen to be fitting for this chapter’s methodology.
On the qualitative side of this chapter, an introductory framework evaluation based on existing literature is applied. This method is fitting to examine relationships of key factors within a research setup and combine various data seats to one summarising narrative. The importance of structured frameworks as a basis for future research for academic areas with limited existing literature is pointed out by prior contributions [11, 12, 13].
The quantitative contribution of this chapter provides a structured approach describing the global ROV market and provides a 2030 forecast based on a literature review of market reports. Both the inspection- and the intervention-classes are regarded separately in order to gain an understanding of the potential of the relatively new micro ROV class. This approach reflects only a first attempt at providing quantitative insights for reasons of limited available market data on ROV figures.
There is sufficient evidence that the underwater drone market will further increase in importance and acceptance, resulting in new market segments using this thriving technology. This chapter identifies 4 major market segments, where micro ROVs are likely to gain market share in the future compared to intervention class ROVs and/or conventional divers. These segments are in line with some of the most recent sustainable development goals (SDG) [14] displayed in Figure 2.
Figure 2.
Major micro ROV growth trends based on SDG Source: Author based on [14].
As part of the global energy transition, the shift from fossil to renewable energy sources is globally agreed on. Water will play a central role in this transition in the form of hydrogen production, cooling systems, offshore and nearshore wind farms or tidal power plants, to name the current most prominent approaches. Whilst conventional operations like seafloor mapping of new areas, cable operations and deep inspections remain intervention class driven, inspection class ROVs might get a growing market share in medium deep inspections of less than 300 m depth. Apart from the new construction of modern wind farms, existing farms have operated for decades and will be required to be decommissioned over the coming decades [15]. Data provided by ROVs is already seen as essential for timely investigation of the role of offshore infrastructure from an ecological point of view and to predict the environmental effects of their removal [3].
The aspect of global food production will not only affect land-based farming but also the gathering of marine-based food resources. In 2022, the world population has reached 8 billion people with an estimated growth to 9 billion by 2037 [16]. In order to feed the global population, new and more efficient ways of food production are required. This will, amongst others, affect the aquaculture sector, which operates in shallow waters of less than 100 m depth. A growing number of micro ROV systems are already implemented in checking the nets and fish already dating back to 1987 [17] and with a growing number of recent contributions [18, 19, 20]. Apart from inspection operations, micro ROV systems can deliver video data of the fish swarm that can be used to train neural networks for various applications of efficiency enhancement as briefly introduced in Chapter 4.
As of January 2023, emission reduction of global shipping has come into force in the form of MARPOL Annex VI at MEPC 76, regulations 23, 25 and 28, resulting in the Energy Efficiency eXisting Ship Index (EEXI) and the Carbon Intensity Indicator (CII) for existing ships. These regulations require ship operators to calculate their fuel consumption and rank ships based on their emission efficiency. If a ship exceeds a certain efficiency level, it will be regarded as unfit for global trade, resulting in decommissioning. This affects a current global fleet of 102.899 vessels [21] with an average age of 21.9 years [21]. In order to stay operational under these regulations, ships must find innovative ways of enhancing their energy efficiency. Micro ROVs will play an important role in terms of hull inspection for marine fouling assessment, which is briefly introduced in Chapter 4.
The area of marine science has greatly benefitted from the rise of low-cost and easily deployable micro ROV systems over the past decades. Whilst the absence of scientists in offshore operations for reasons already explained results in a certain dependency on companies or data providers, researchers can conduct their own data collection in medium-depth operations of less than 300 m. The amount of contributions from operating micro and medium-sized ROV systems has risen and is likely to continue to rise as technology evolves. It is likely for micro ROVs to become even cheaper, more robust, easier to operate and equitable with more external sensor technology in the upcoming years.
As introduced in this chapter’s methodology, market data on ROV systems is scarce. Although the offshore ROV market has a forerun of several decades, market data is limited and not fully publicly available. In a first quantitative attempt to evaluate the offshore and the micro ROV markets, a structured analysis of market reports of leading data providers has been conducted. Based on a web search in April 2023, global offshore ROV market data and forecast values have been collected and combined. As a result, a number of 15 market reports were clustered, of which 10 reports contain data of the intervention class and 4 reports contain inspection class data. Approaches of collecting and clustering ROV market data have not been conducted amongst the existing literature and this attempt reflects only a brief analysis based on available market data. The reports evaluated the current global ROV market values between 2021 and 2023 as well as a market forecast based on the compound annual growth rate (CAGR). Fifteen offshore ROV market report data were gathered using a web search of publicly available data. The CAGR values were calculated annually for the period 2022–2030 using the values provided. Due to the fact that the values had a high deviation between the different market predictions, the third quartile of each value was applied. This resulted in 11 reports being included in the calculation and four reports being disregarded. One report only provided CAGR predictions without the annual value rate of the market, so it was included in the CAGR evaluation but not in the annual value calculation. After the third quartile application, the mean value of each year was calculated from the offshore ROV market data. Only four reports were found on the micro ROV market with limited value deviation so that mean values were chosen without quartile exclusion.
The market evaluation in Table 1 indicates similar CAGR growth rates till 2030 of 8.21% for the intervention class and 8.09% for the inspection class. The total market values, however, are 5–6 times higher in the intervention class given the very large investment in hardware. The market value for micro ROVs in 2022 is stated as 420 Mill. $ (Figure 3). It is noted that CAGR evaluation assumes linear growth that is limited in its explanatory power albeit it underlines a prediction of a certain trend. The trend implies, that the global ROV market of both major classes will almost double by 2030, as shown in Figures 3 and 4, once more underlining the argumentation of the innovation potential of this hardware.
Year
2022
2023
2024
2025
2026
2027
2028
2029
2030
CAGR
Intervention class
N
10
10
10
10
10
10
10
10
10
11
Average (Mil $)
2.800
3.047
3.323
3.626
3.960
4.328
4.733
5.179
5.635
8,21
Delta (Mil $)
752
870
1.008
1.173
1.368
1.597
1.867
2.185
2.582
2,48
Inspection class ROV
N
4
4
4
4
4
4
4
4
4
4
Average (Mil $)
420
451
485
522
562
605
653
706
763
8,09
Delta (Mil $)
168
170
174
177
183
190
200
212
228
3,34
Table 1.
Global ROV market Forecast 2022–2030.
Source: Author.
Figure 3.
Global ROV market Forecast 2022–2030. Source: Author.
Figure 4.
Global ROV market Forecast 2022–2030. Source: Author.
3.1 Micro ROV system overview
It was reported in 2015, that 700 ROVs were in operation globally whereas over 550 were intervention class systems [21]. This number appears to be excluding the micro ROV class, leaving the existing 150 ROV systems most likely to be related to specialised equipment rather than off the shelf industrial products. A first market analysis using web search on micro ROV systems in Figure 5 reveals a number of 22 micro ROV systems having entered the market in the past decade since 2012. These ROV systems were launched by 12 global manufacturers mostly founded within the past two decades between 2001 and 2016. In order to differentiate the evaluated systems from other ROV classes, a maximal investment cost of 25.000$ was defined as the upper threshold. Systems exceeding this limit were not regarded in this evaluation. Figure 5 reveals micro ROV systems by their market entry as well as their costs (based on 2023 values) and their maximum operational depth rating. The SeaOtter-1 marks the turning point in 1994, being the first micro ROV of less than 25.000$. It is possible that this system was more expensive at market entry given the past monetary value, but in 2023, its market price remained around 21.000$. The SeaOtter-1 was upgraded to a version 2 in 2007, almost one decade before the majority of other systems entered the market. The average age of market launch is 2017/2018, whereas the largest number of micro ROV systems entered the market in 2020.
Figure 5.
Market overview of micro ROVs based on price, depth and release. Source: Author.
From an investment perspective, the average cost of a micro ROV system is 10.350$, however, only the 2011 launched DTG-2, later in 2019, being upgraded to the DTG-3 system ranges at this price level based on 2023 market data. The data reveals, that two clusters within the micro ROV segment have formed, being systems above the average value of approx. 10.000$ and those below. The average price of the more expensive cluster reaches 17.900$ whereas the lower cost cluster ranges at 4.100$ on average. Although these results only provide a first, superficial character where more in-depth research is required for more valid statements, it reveals that two clusters in the micro ROV market have formed since 2011.
From an operational depth point of view, micro ROV systems on average reach a maximum diving depth of 100 m or 305 m (1.000 ft). Some individual systems offer rates in between, but no micro ROV system exceeds the 305 m (1.000 ft) level officially. It shall be mentioned, however, that the standard configuration of the ROV systems evaluated comes with less cable length than their maximum depth rating. Some ROVs such as the BlueROV2 require a different frame material replacing plastic with aluminium for larger depths. In order to reach the operative maximum of the system, the required investments exceed the standard off the shelf values displayed. Of the 12 global micro ROV manufacturers, China is the dominating country of origin with five companies as shown in Table 2. It was mentioned in the beginning, that micro ROV systems offer a high degree of flexibility in terms of operations. The system weight data of the ROV and all its necessary control stations, however, require a differentiated evaluation of each system according to the individual operational needs. Some systems weigh up to 37 kg, which is tough to launch in the water without winches and impossible to carry, for example, on a pilot ladder on board a ship at an anchorage. Furthermore, some systems operate on batteries whilst others require a source of electricity.
After providing a first market overview and system survey above, it is the intention of this chapter to also highlight the different operational options of micro ROV systems that will most likely increase in application in the near future. A first systematic overview has been provided in 2017 [1], whereas Table 2 introduces an updated version including additional operation areas and a wider spectrum of current and future ROV operations.
Although not purposefully collected for marine science reasons, offshore companies over the past decades have collected a considerable treasure of data through their ROV operation video recording. Both videos and images of industrial ROVs represent one of the most substantial visual datasets available from the oceans [22, 23, 24]. Studies have already taken into consideration historical ROV data to characterise fauna communities and reefs within oil and gas infrastructure [3, 25, 26]. This data helps understand the impact of oil and gas operations on marine wildlife and to, therefore, assess its environmental, social and economic benefits. Other contributions apply ROV systems in seafloor mapping [27], where micro ROV systems could potentially find an assisting role in the future as well. Given the depth limitations of current micro ROV systems of 305 m (1.000 ft), Deep sea ROV research operations below [24] will further require intervention class solutions.
With regard to disaster response, ROV systems have already been applied and delivered scientific insights. In 2011, following the Tokoku earthquake and tsunami disaster in Japan, micro ROV systems have been operated to inspect critical infrastructure and assist with victim identification [28, 29]. Different systems have been applied along the Sanriku coast and the Fukushima Daiichi nuclear power plant. In 2015, a micro ROV was operated on the Costa Concordia Wreck [30] for inspection and documentation purposes. ROV systems provide great value for inaccessible or dangerous environments and will likely find more application in future disaster response. Underwater archaeology already benefits from micro ROV systems for various purposes, such as diving buddy [1] documentation and surface recognition [31] and planning and supporting of underwater sites [32]. Especially for archaeological sites of a limited size, micro ROV systems offer a great benefit for a fast and mobile operation for documentation and 3D mapping. In 2023, a micro ROV type Blueye Pro was operated north of Berlin in Germany to construct 3D documentation of sunken inland shipwrecks of up to 35 m depth. Reaching these wrecks requires advanced diving skills and diving accidents have happened in the past trying to reach these wrecks. The ROV operation launched from an inflatable boat on the surface was again a very successful demonstration of micro ROV capabilities in the context of 3D mapping and underwater archaeology as displayed in Figure 6.
Figure 6.
Micro ROV Operation on archaeological site. Source: Author and Kaffenkahn e.V. /Kai Dietterle und Uwe Klimek.
The 2022 Russia-Ukraine war has shown that military operations do not take place in remote areas but happen in the vicinity of western borders. It has been reported that sea mines were used in front of Odessa port in the Black Sea with the intention of disrupting maritime trade and preventing grain exports [33]. Retrieving these mines will be a military operation where ROV systems will most likely play an assisting role. Studies have already pointed out the possibilities of ROV-based detection of unexploded ordnance (UXO) in the past. A 2012 contribution [34] described the ROV detection capabilities of UXO in seaport operations. Recent contributions connect specialised ROV systems for electromagnetic detection [35] with UXO detection in the context of offshore site surveys [36]. Although not named specifically in the contributions, micro ROVs can provide a value-adding service of confirming suspicious detections and/or providing visual confirmation on potential UXO findings. Due to high manoeuvrability and low financial risk, small ROV systems are a valuable choice for close proximity evaluation of mines. A 2022 test in German waters confirmed the applicability of micro ROVs in very close proximity to UXOs as shown in Figure 7.
Figure 7.
Visual UXO detection examples of a micro ROV. Source: Author.
From a security point of view, micro ROV systems will furthermore deliver value-adding services to maritime operations. Maritime security has been defined by the International Ship and Port Facility Security (ISPS) Code since 2002 as part of the Safety of Live at Seas (SOLAS) convention. Drones in general have been evaluated towards their potential for ISPS assistance from an operational point of view [37]. It is shown that ROV operations can assist all 3 major ISPS operation categories of monitoring, inspecting and management, either assisting human operations or replacing them with a full remote operation [37].
Industrial inspection of infrastructure reflects the largest operational area for micro ROV systems as highlighted in Table 3. A growing number of inspections and tests has led to an increased number of publications and is likely to further contribute to future studies. Amongst the most important areas of inspection in line with the sustainable development goals (SDG) in Figure 2 lies the inspection of ship hulls for emission reduction. Marine fouling is defined as algae, pocks, mussels and barnacles that attach to the ship’s hull when the vessel is not moving. This fouling increases the drag of the vessel, causing fuel consumption to rise in order to maintain a certain speed. First contributions quantify this excess fuel consumption to range from 6.5 to 17.6% [38]. The aspect of ROV-based ship hull inspection was first introduced by [39] in 1983 using a magnetic vehicle attached to the hull. These systems, however, only operate on relatively clean surfaces, whereas under heavy marine fouling conditions the magnetic wheels cannot attach to the hull. Diving ROV systems have later been tested and found more suitable for these operations, with the first study dating back to 1999 [40]. Diving micro ROV systems allow for a quick inspection of the ship’s hull given their good manoeuvrability and, as shown in Figure 5, both investment costs and operational depth are uniformly favourable for ship inspections of less than 20 m depth.
Marine Science
Disaster Response
Underwater Archaeology
Military Operations
Security
Industrial Inspection
Offshore
Nearshore
Oceanography
Oil spill detection
Retrieving of artefacts
UXO identification
Contraband detection
Wind farm inspection
Port inspection
Reef research
Maritime accidents
Documentation
Surveillance
Pipeline inspection
Ship inspection
Water quality assessment
Search & rescue
Dive buddy
oil Platform inspection
Bridge inspection
Wildlife research
Ghost net detection
Tank inspection
Aquaculture inspection
Hazardous environment inspection
Table 3.
Micro ROV Operation Framework Source: Author based on [1].
Source: Author.
As already mentioned, micro ROV systems allow for a new way of gathering important data for various maritime operations displayed in Table 3. Making good use of this data follows four main steps revealed in Figure 8; being the acquisition of the data, the storage, as well as visualisation and data transformation. Data storage is commonly done by databases either provided by a stakeholder (e.g. a port, a ship operator, a research facility) or via cloud storage on the web, depending on the individual requirements for data accessibility and security. To further enhance data security accompanied with ROV footage, first studies discuss the use of blockchain technology [41] for secure communication within ROV networks. This aspect will likely gain future importance with regard to AUV swarm operations and requires further research towards its applicability in the ROV domain. The visualisation of data is a current topic of interest in the industry and its stakeholder. A classical method of drone data visualisation lies in 3D models based on photogrammetry where points in different photos of the same infrastructure are connected to a point cloud and later a photo realistic 3D model. The use of unmanned aerial vehicles for 3D modelling in port operations has already been introduced [42] whilst 3D modelling of underwater infrastructure is currently growing in importance. 3D models of infrastructure do allow for a three-dimensional spectating of the infrastructure but the model itself remains static with no data integration possible. Data can be aligned to a 3D model using a dashboard structure, where both the 3D model and the data are displayed separately. Another more modern approach with growing industry and academic interest is the combination of a 3D model and data streams into a digital twin as a combination of a 3D infrastructure directly connected to its data sources. First contributions already state that “The combination of computer simulation with techniques to ‘sense the environment’ and comprehend large amounts of data via Big Data Analytics and Machine Learning are fuelling new Cyber-Physical Systems (CPS) and smart applications in society and industry” [43]. The final step is data transformation, where the collected and visualised data is used to gain new knowledge. This can be achieved via forecasting or more advanced statistical analysis that, for reasons of simplicity, is combined under the more broad term “data science”. Furthermore, the transformation step allows for the training of artificial intelligence, as will be introduced in Chapter 4.
Figure 8.
ROV-based data management steps and its applications. Source: Author.
Artificial intelligence (AI) as part of general algorithms is a topic of modern computer science with a huge interest. Since the acquisition of ChatGPT in 2023 [44] AI has been widely discussed in the media, ranging from value-adding solutions to dystopic science fiction. The example of ChatGPT shows how natural language processing (NLP) can reach human-like logic, also referred to as intelligence. NLP, however, is only one aspect of AI, whereas another aspect of neural networks or deep learning offers far more applicable scenarios for underwater drones, hence becoming “a big research topic” [45] with the emergence of underwater vehicles solving the problem of how to collect underwater images [46]. It is mainly due to the improvements in computing power that neural networks can be integrated into operative tasks. Powerful modern computer languages such as Python and Tensorflow with their fast number of libraries and powerful deep models [47] further boost developments in this area. Getting high quality data for network training was also a challenging task for a long time [48] with already introduced dependencies of researchers on companies or data providers [3] that also change with the rise of micro ROV systems. A remaining challenge lies in the labelling of the data, which to this point requires detailed, manual and time-consuming labour. Labelling steps often need to be duplicated hundreds or thousands of times to build a robust deep network [48]. The absence of existing deep networks for underwater applications defines a current gap in research that needs to be overcome by studies [47]. These projects are further confronted with the fact that underwater visual content is entirely different because of the domain specific object categories, background patterns and optical distortion artefacts [47] hence, making applications of well-known terrestrial data-based neural network approaches models inapplicable [47]. It is furthermore stated that, based on the absence of underwater NN literature, existing networks are often limited to only performing simple tasks but not being suitable for multi-object semantic segmentation [47].
The underlying objective of semantic segmentation is to classify each pixel of an image with regard to its classified category to, finally, predicting a result map containing “semantic” information [46]. The segmentation basically separates the source into individual and non-overlapping portions for computer-based image analysis and understanding [49]. The challenge underwater, however, lies in the changing light conditions, the existence of blur and the absence of clear foreground and background characteristics, causing underwater image segmentation to lag behind land-based methods [49]. Whilst initial contributions focus on fish or other distinguishable underwater patterns, more advanced currently ongoing studies experiment with marine fouling recognition with fuzzy structures as shown in Figure 9.
Figure 9.
Underwater AI examples of fouling detection. Top left original; top right output; bottom left and right operation examples. Source: Author.
By operating an encoder-decoder model with custom architecture, marine fouling on ship hulls can be identified under the above-mentioned challenging underwater conditions in various examples of complexity. Applying a custom deep neural network to underwater imagery can be an effective way to identify different classes of biofouling on ship hulls. By training the neural network on a dataset of labelled images of biofouling, the system can learn to recognise and classify various types of marine organisms and other debris that accumulate on the hull. This can help ship owners and operators to better understand the extent and type of biofouling on their vessels, which in turn can inform decisions about hull cleaning and maintenance schedules. Additionally, using automated image analysis can be more efficient and accurate than manual inspection methods, making it a valuable tool for marine industry professionals to only name one example of AI and ROV combinations.
The rise of micro ROV systems in the past decade has opened a potential game-changing scenario for marine science and entrepreneurship. Since the 1970s ROV systems have been huge, complex and expensive systems only accessible to a few companies, mainly in the oil and gas industry. Comparable to the advent of personal computers in every household, the micro ROV class opens the area of underwater inspection and data collection to the open markets in a fast, easy to operate and relatively cheap manner. Whilst both the use of this new technology in academia and the industry has risen, there is still a considerable research gap existing today.
This chapter introduces the micro ROV class both from an academic and a business perspective by introducing initial market data, a comprehensive literature analysis and a structured framework analysis as basis for future research. The author introduces four main areas of rising ROV growth trends based on the sustainable development goals of the maritime industry, which are the global energy transition, global seafood production and emission reduction in shipping and marine science in general. Based on existing literature, the ROV operations framework of micro ROV systems is differentiated into 6 main categories: marine science, disaster response, underwater archaeology, military operations, security operations and infrastructure inspection. As rising drone hardware and its operation will inevitably generate data, 4 ROV data management steps are introduced. The most advanced step of data transfer, in the eyes of the author, lies in the application of artificial intelligence in combination with ROV hardware. This means in detail the use of ROV-generated data for deep learning approaches of training neural networks for automated image recognition. This relatively new concept for underwater data is briefly introduced.
This contribution adds value to the existing literature in shedding light on the literature gap of micro ROV hardware research in general and especially in combination with artificial intelligence operations. Further applied research including the use of micro ROV systems is required to better draw conclusions on the introduced potential of this emerging technology.
References
1.Capocci R, Dooly G, Omerdić E, Coleman J, Newe T, Toal D. Inspection-class remotely operated vehicles—A review. Journal of Marine Science and Engineering. 2017;5(1):13
2.The Nippon Foundation—GEBCO Seabed 2030 Project. Nearly a Fifth of World’s Ocean Floor Now Mapped. Available from: https://www.gebco.net/documents/seabed2030_brochure.pdf [accessed on 04 April 2023]
3.McLean DL, Macreadie P, White DJ, Thomson PG, Fowler A, Gates AR, et al. Understanding the global scientific value of industry ROV data, to quantify marine ecology and guide offshore decommissioning strategies. In: Proceedings of the Offshore Technology Conference Asia, Kuala Lumpur, Malaysia, 20-23 March. Houston, TX: Offshore Technology Conference; 2018. DOI: 10.4043/28312-MS
4.Visbeck M. Ocean science research is key for a sustainable future. Nature Communications. 2018;9:690. DOI: 10.1038/s41467-018-03158-3
5.Sprague KL. Civilian applications and policy implications of commercial unmanned aerial vehicles [Doctoral dissertation] Massachusetts Institute of Technology. 2004
6.Longo F, Padovano A, Caputi L, Gatti G, Fragiacomo P, D’Augusta V, et al. Distributed Simulation for Digital Twins: an Application to Support the Autonomous Robotics for the Extended Ship. In: 2022 IEEE/ACM 26th International Symposium on Distributed Simulation and Real Time Applications (DS-RT). IEEE; 2022, September. pp. 179-186
7.Glaser B, Strauss A. Grounded theory: The discovery of grounded theory. Sociology: The Journal of the British Sociological Association. 1967;12:27-49
8.Strauss A. Notes on the nature and development of general theories. Qualitative Inquiry. 1995;1(1):7-18
9.Strauss A, Corbin JM. Grounded Theory in Practice. Sage: Thousand Oaks, CA; 1997
10.Strauss A, Corbin JM. Grounded theory methodology. Handbook of Qualitative Research. 1994;17:273-285
11.Miles MB, Huberman AM. Qualitative Data Analysis: A Sourcebook of New Methods. Beverly Hills, CA: Sage; 1994
12.Shields P, Rangarajan N. A Playbook for Research Methods: Integrating Conceptual Frameworks and Project Management. Stillwater: New Forums Press; 2013
13.Maxwell JA. Qualitative Research Design, An Interactive Approach. 3rd ed. Thousand Oaks, CA: Sage; 2013
14.International Maritime Organozation. IMO and sustainable development goals. 2015. Available from: https://www.imo.org/en/MediaCentre/HotTopics/Pages/SustainableDevelopmentGoals.aspx [accessed on 04 April 2023]
15.Fowler AM, Jørgensen A-M, Svendsen JC, Macreadie PI, Jones DO, Boon AR, et al. Environmental benefits of leaving offshore infrastructure in the ocean. Frontiers in Ecology and the Environment. 2018;16:571-578. DOI: 10.1002/fee. 1827
16.United Nations. Day of Eight Billion. 2022. Available from: https://www.un.org/en/dayof8billion [accessed on 04 April 2023]
17.Klepaker RA, Vestgård K, Hallset JO, Balchen JG. The application of a free-swimming ROV in aquaculture. IFAC Proceedings. 1987;20(7):181-185
18.Amundsen HB, Caharija W, Pettersen KY. Autonomous ROV inspections of aquaculture net pens using DVL. IEEE Journal of Oceanic Engineering. 2021;47(1):1-19
19.Karlsen HØ, Amundsen HB, Caharija W, Ludvigsen M. Autonomous Aquaculture: Implementation of an autonomous mission control system for unmanned underwater vehicle operations. In: OCEANS 2021. San Diego–Porto: IEEE; 2021, September. pp. 1-10
20.Akram W, Casavola A, Kapetanović N, Miškovic N. A visual servoing scheme for autonomous aquaculture net pens inspection using ROV. Sensors. 2022;22(9):3525
21.IMCA. IMCA World-Wide ROV Personnel and Vehicle Statistics for 2014. Richmond, VA: International Marine Contractors Association; 2015
22.Jones DOB. Using existing industrial remotely operated vehicles for deep-sea science. Zoologica Scripta. 2009;38:41-47. DOI: 10.1111/j.1463-6409.2007.00 315.x
23.Gates ART, Horton A, Serpell-Stevens C, Chandler LJ, Grange K, Robert A, et al. Ecological role of an offshore industry artificial structure. Frontiers in Marine Science. 2019;6:675. DOI: 10.3389/fmars.2019.00675
24.Macreadie PI, McLean DL, Thomson PG, Partridge JC, Jones DOB, Gates A, et al. Eyes in the sea: Unlocking the mysteries of the ocean using industrial, remotely operated vehicles (ROVs). Science of the Total Environment. 2018;634:1077-1091. DOI: 10.1016/J.SCITOTENV.2018.04.049
25.Pradella N, Fowler AM, Booth DJ, Macreadie PI. Fish assemblages associated with oil industry structures on the continental shelf of north-western Australia. Journal of Fish Biology. 2014;84:247-255. DOI: 10.1111/jfb.12274
26.Thomson PG, Fowler AM, Davis AR, Pattiaratchi CB, Booth DJ. Some Old movies become classics – a case study determining the scientific value of ROV inspection footage on a platform on Australia’s North West Shelf. Frontiers in Marine Science. 2018;5:471. DOI: 10.3389/fmars.2018.00471
27.Lambertini A, Menghini M, Cimini J, Odetti, et al. Underwater drone architecture for marine digital twin: Lessons learned from SUSHI DROP project. Sensors. 2022;22(3):744
28.Murphy RR, Dreger KL, Newsome S, Rodocker J, Steimle E, Kimura T, et al. Use of remotely operated marine vehicles at Minamisanriku and Rikuzentakata Japan for disaster recovery. In: 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics. IEEE; 2011, November. pp. 19-25
29.Osumi H. Application of Robot Technologies to the Disaster Sites. In: Lessons learned from the Great East Japan Earthquake Disaster: Report of the JSME Research Committee on the Great East Japan Earthquake Disaster. Tokio: Japan Society of Mechanical Engineers; 2014. pp. 58-73
30.Allotta B et al. Development of Nemo ROV for the inspection of the Costa Concordia Wreck. Proceedings of the Institution of Mechanical Engineers, Part M, Journal of Engineering for the Maritime Environment. 2015;231(1):3-18
31.Forney C, Forrester J, Bagley B, McVicker W, White J, Smith T, et al. Surface reconstruction of Maltese cisterns using ROV sonar data for archeological study. In: Advances in Visual Computing: 7th International Symposium, ISVC 2011, Las Vegas, NV, USA, September 26-28, 2011. Proceedings, Part I 7. Berlin Heidelberg: Springer; 2011. pp. 461-471
32.Bruno F, Muzzupappa M, Lagudi A, Gallo A, Spadafora F, Ritacco G, et al. A ROV for supporting the planned maintenance in underwater archaeological sites. In: Oceans 2015-Genova. IEEE; 2015, May. pp. 1-7
33.CNBC. Russia and Ukraine are battling over underwater mines as the global food crisis worsens. 2022. Available at: https://www.cnbc.com/2022/06/10/russia-and-ukraine-battle-overunderwater-mines-in-the-black-sea.html [accessed on 04 April 2023]
34.Keranen J, Schultz G, Bassani C, Segal S, Kinnaman B. Remotely-Operated Vehicle applications in port and harbor site characterization: Payloads, platforms, sensors, and operations. In: 2012 Oceans. IEEE; 2012, October. pp. 1-9
35.Schultz G, Miller J, Keranen J, Shubitidze F. ROV-based 3D controlled source electromagnetics for UXO detection and classification. In: Symposium on the Application of Geophysics to Engineering and Environmental Problems. Society of Exploration Geophysicists and Environment and Engineering Geophysical Society; 2021, June, 2021. pp. 346-346
36.Shmatkov AA, Antonov NA. Experience in application of a specialized ROV for the detection of UXOs when performing offshore site surveys. Engineering and Mining Geophysics 2021. 2021;2021(1):1-6
37.Stein M. Integrating unmanned vehicles in port security operations: An introductory analysis and first applicable frameworks. Oceans Yearbook. 2018;32:556-583
38.Adland R, Cariou P, Jia H, Wolff FC. The energy efficiency effects of periodic ship hull cleaning. Journal of Cleaner Production. 2018;178:1-13
39.Nicinski S. Development of a remotely operated ship hull inspection vehicle. In: Proceedings OCEANS’83. IEEE; 1983, August. pp. 583-587
40.Lynn DC, Bohlander GS. Performing ship hull inspections using a remotely operated vehicle. In: Oceans’ 99. MTS/IEEE. Riding the Crest into the 21st Century. Conference and Exhibition. Conference Proceedings (IEEE CatNo. 99CH37008). Vol. 2. IEEE; 1999, September. pp. 555-562
41.Ahuja NJ, Kumar A, Thapliyal M, Dutt S, Kumar T, Pacheco DADJ, et al. Blockchain for unmanned underwater drones: Research issues, challenges, trends and future directions. 2022. arXiv preprint arXiv:2210.06540
42.Stein M. Aero-triangulation and Photogrammetry in Sensible Infrastructures–How the Industry benefits from low-cost 3D Modelling conducted by Unmanned Aerial Vehicles (UAV). 2019
43.Chen G, Wang P, Feng B, Li Y, Liu D. The framework design of smart factory in discrete manufacturing industry based on cyberphysical system. International Journal of Computer Integrated Manufacturing. 2020;33(1):79-101
44.Forbes. Microsoft confirms its $10 billion investment into ChatGPT, changing how Microsoft competes with google, apple and other tech giants. 2023. Available from: www.forbes.com/sites/qai/2023/01/27/microsoft-confirms-its-10-billion-investment-into-chatgpt-changing-how-microsoft-competes-with-google-apple-and-other-tech-giants/?sh=2db5e8333624. [Accessed on April 20th 2023]
45.Meng L, Hirayama T, Oyanagi S. Underwater-drone with panoramic camera for automatic fish recognition based on deep learning. IEEE Access. 2018;6:17880-17886
46.Liu F, Fang M. Semantic segmentation of underwater images based on improved Deeplab. Journal of Marine Science and Engineering. 2020;8(3):188
47.Islam MJ, Edge C, Xiao Y, Luo P, Mehtaz M, Morse C, et al. Semantic segmentation of underwater imagery: Dataset and benchmark. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2020, October. pp. 1769-1776
48.O’Byrne M, Pakrashi V, Schoefs F, Ghosh B. Semantic segmentation of underwater imagery using deep networks trained on synthetic imagery. Journal of Marine Science and Engineering. 2018;6(3):93
49.Jian M, Liu X, Luo H, Lu X, Yu H, Dong J. Underwater image processing and analysis: A review. Signal Processing: Image Communication. 2021;91:116088
Written By
Michael Stein
Submitted: 11 April 2023Reviewed: 24 April 2023Published: 29 July 2023
Open access peer-reviewed chapter
