Perspective Chapter: Training Autonomous Ships for Safe Navigation

Abstract

The capabilities of autonomous (surface) sea vessels have been improving in recent years, as a result of advances in communication, sensing and navigation systems. An autonomous vessel must be capable of accomplishing its voyage in a safe manner, i.e., without endangering other nearby vessels or disrupting their navigation. This chapter discusses topics related to safe navigation of autonomous vessels, particularly regarding their ability to plan safe sailing routes under dynamic sea traffic conditions. The chapter proposes an autonomous vessel training approach where the learning vessel’s navigation system plans routes in a high fidelity training environment that utilises AIS data. The resulting route is then assessed for safety risks, and a feedback score is used to improve the planning capability. The approach is demonstrated with the scenario of autonomously crossing the English Channel.

1. Introduction

1.1 Autonomous ships

An autonomous (sea) surface ship, also known as Maritime Autonomous Surface Ship (MASS), can be defined as a ship which, to a varying degree, can operate independently of human interaction. The idea of autonomous unmanned operation of sea vessels is not new [1]. Currently, unmanned but guided sea vessels are used for specialist tasks such as cleaning operations, surveillance etc. Autonomous water vessels have been used experimentally for passenger and cargo transport in restricted (e.g., canals) and fixed routes (e.g., ferries). However, no large unmanned commercial ship is currently operational, due to the inherent safety risks involved.

1.2 Scope of autonomous ships

There are variants to the degree of autonomy of ships. This includes remotely controlled ships, semi-autonomous ships where some of the vessel activities are controlled by a remote station and fully autonomous ships. Autonomy Level 4 is defined in [2] as the level where all operational tasks are always performed by an automated system, and Autonomy Level 3 as the level where control and decision making are autonomous, but human monitoring is also involved. This Chapter focuses on Autonomy Levels 3 and 4, although it does not exclude lower levels of autonomy (Levels 2 and 1) where some remote control of the vessel is involved. It must be noted that such vessels do not currently exist in operation, with [3] predicting that they will become operational towards the end of this decade, subject to regulatory approval.

1.3 Benefits of autonomous ships

Autonomous vehicles are deployed in many fields such as space exploration, logistics, self-driving cars etc., to reduce human errors and improve precision [4]. In transport and logistics applications, autonomous vehicles can help to reduce traffic congestion and therefore pollution. In addition, automation of many ship functions may reduce the risks of human error, however it may introduce new risks [5, 6], as discussed later on in this Chapter.

1.4 Autonomous ship projects

Innovative autonomous ship demonstrators have emerged in the past years such as the one by Rolls-Royce and ferry operator Finferries, who in 2018 demonstrated the world’s first fully autonomous ferry in the archipelago south of the city of Turku, Finland. A five-year project from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Senseable City Lab, produced a fleet of autonomous boats for the City of Amsterdam [7]. The latest version of this boat, Roboat II, which is capable of carrying passengers, uses Lidar, GPS and other sensors to navigate its surroundings.

1.5 Safety of autonomous ships

Safety is paramount for the success of autonomous ships. Their level of safety must match and in certain areas exceed that of manned ships. Autonomous ships may reduce the risk of human error due to absence of crew, but they may create new types of risks. Towards ensuring safety of future autonomous ships, there is ongoing legislation activity led by international organisations such as IMO. The AAWA (Advanced Autonomous Waterborne Applications) initiative [8], a joint industry and academic research project on autonomous ships that ran between 2015 and 2017, sought to analyse different scientific challenges related to autonomous ship operations, technology needs, risks, incentives and regulations/liabilities. According to the AAWA initiative, amongst the different autonomy issues, major topics are marine situational awareness and autonomous navigation [3].

1.6 Regulatory framework for autonomous shipping

It can be argued that the technical barriers to autonomous shipping are less formidable than the regulatory ones. Therefore, international organisations such as the Maritime Safety Committee of IMO, have commenced work on addressing a regulatory framework in order to assess how Maritime Autonomous Surface Ships could be regulated [9]. The Committee also focused on safety, secure and environmentally sound Maritime Autonomous Surface (MASS) operations.

1.7 Chapter organisation

The Chapter is organised as follows. The next section reviews the functional architecture of an unmanned ship and identifies the ship’s key functions, focusing on technologies and systems for voyage planning. Then, the section considers the IT architecture onboard an autonomous ship that supports autonomous navigation.

Section 3 is concerned with concepts and regulations of safe navigation of autonomous ships. It covers training issues of autonomous ships to carry out safe operations. It reviews training methods and techniques for safe navigation, such as simulation based training, as well as automated training and learning, from the fields of Machine Learning and Artificial Intelligence.

Section 4 illustrates the concept of training autonomous ship navigation systems in realistic environments. It also discusses safe navigation concepts. It concludes with a scenario of safe route planning through the English Channel.

Section 5 summarises the idea of training autonomous ship safe navigation, its potential and shortcomings.

Finally, Section 6 contains a discussion about the future of autonomous ships in general and how advances in information technologies can shape it.

2. Functional and safety properties of unmanned ships

2.1 Functions of autonomous vessels

Table 1 from the MUNIN project [10], lists the function categories of ships in general, and also those functional areas that do not apply to unmanned ships. From Table 1, the focus of this Chapter can be defined, namely Group 1 (‘Voyage’) that involves the activities of high level voyage planning execution and monitoring. However, functions from other functional categories such as Group 3 (‘observations’) and Group 4 (‘Safety, emergencies’) are also incorporated. The next section mainly discusses safety enhancing systems onboard autonomous ships.

Group	Description
1. Voyage	High level voyage planning, execution and monitoring
2. Sailing	Manoeuvring, avoidance, communication
3. Observations	Environment, objects, ships
4. Safety, emergencies	Other ships, own ship, environment
5. Security	Antipiracy, ISPS, access control and lock-down
6. Crew, passenger	Not applicable to unmanned ship
7. Cargo, stability, strength	Ship stability, hull integrity, cargo monitoring
8. Technical	Power generation and distribution, emissions to air/water
9. Special functions	Not applicable to bulk ships (tugs, offshore, …)
10. Administration	Log keeping, operational communication, reporting

Table 1.

Function groups of unmanned vessels [10].

2.2 Unmanned ship onboard systems

Autonomous ships utilise a variety of systems for situation awareness and communication with the outside world. This includes GPS, vision, environment (wind, wave) sensors, radar, cameras, sonar and so on for sensing, as well as radio, AIS, satellite transceivers etc., for communications purposes. The signals and data received via such systems are transmitted to the autonomous ships control system.

According to [11], as per Figure 1, the autonomous ship consists of a Path Planning subsystem that communicates information about the planned path to a Collision Avoidance subsystem. The Path Planning subsystem is informed about the position and trajectory of nearby objects by a State Awareness subsystem. In turn, the State Awareness subsystem employs sensing devices (camera, radar, sonar, etc.) and communication systems such as AIS. Autonomous navigation systems (ANS), which can make the navigational decisions and command the ship’s propulsion systems, are a central part of autonomous or remotely controlled vessels [12].

Figure 1.

Autonomous navigation system.

2.3 AIS

Amongst the communications systems onboard an autonomous ship, the Automatic Identification System (AIS) plays a prominent role as a context awareness and navigation safety device. AIS is a worldwide automatic positioning system based on vessel transponders that transmit a signal in the VHF band. This alerts other vessels and shore stations with AIS receivers to the presence of that vessel. The signals and accompanying information can then be received by any vessel, land station or satellite, fitted with an AIS receiver, and is typically displayed on a screen of chart-plotting software.

AIS Provides three types of information [13].

Fixed, or static information including data such as: Maritime Mobile Service Identity, Call Sign and name of vessel, IMO Number, length and beam, type of ship and location of position-fixing antenna.

Dynamic information, which, apart from navigational status information, is automatically updated from the ship sensors connected to AIS. This includes the ship’s position with accuracy indication and integrity status, position time stamp, course over ground, speed over ground, heading, navigational status and rate of turn.

Voyage-related information, which might need to be manually entered and updated, such as: ship’s draught, hazardous cargo, e.g., dangerous goods, harmful substances, destination, ETA and route plan (waypoints), at the discretion of the ship’s master.

3. Training autonomous ships for safe navigation

3.1 Autonomous Sea navigation

The ability of autonomous navigation is essential in autonomous systems [14]. Autonomous navigation of ship relies on various sensors (vision, radar, sonar) to detect along the navigation path and environment and take into account vessel properties to achieve safe travel [4]. Autonomous navigation is achieved by training or pre-programming the ship with data about the vessel behaviour in various sailing scenarios. The autonomous behaviour then relies on advanced machine vision and other pattern recognition techniques in order for example to detect and avoid obstacles along the navigation path.

3.2 Safe navigation regulations

Safety is an essential prerequisite for the successful adoption of autonomous ships. There is a danger that regulation falls behind technological innovation [15], therefore proactive regulatory forming activities are required. IMO [9] for instance, has recently completed a regulatory scoping exercise on Maritime Autonomous Surface Ships that was designed to assess existing IMO instruments to see how they might apply to ships with varying degrees of automation.

COLREGs is a set of safety regulations by IMO that describe potential collision scenarios such as crossing, head-on and overtaking, and suggests possible manoeuvres to avoid a collision. Although the rules provide a set of guidelines for safe manoeuvring at sea, they are aimed at human control navigators [16]. This subjective nature of COLREGs is one of the major causes of ship collisions. Indeed, it is estimated that human error contributes to between 89% and 96% of marine collisions (Rothblum, 2000 in [16]).

Safe navigation means that the autonomous ship does not endanger itself or other nearby ships through its course. The ship should avoid interrupting the course of other ships, should not force them to take evasive action, i.e., to stop, reduce/increase speed or alter their course.

3.3 Safe sailing concepts and techniques

Most of the ship collision accidents are due to human errors, which is a large threat in open sea [4]. In autonomous vessels, the actual risks of collision are currently unknown, as such ships have yet to be deployed, it is however important that collision avoidance technologies are developed and integrated in the overall ship control system.

To understand the risks of collision at sea and what constitutes safe sailing, some concepts must be introduced. Collision risk assessment is vital before the vessel makes any course changing decisions. Collision risk assessment must determine the closest point of approach (CPA), the corresponding time to the closest point of approach (TCPA) and a projected area of danger (PAD), by extrapolating the other vessel’s position over time [16]. A typical method based on the distance factor in ship collision evaluation is the ship domain [17]. The ship domain is created as a safe area around the ship, based on the safe distance. The ship domain shape and radius depend on the ship’s characteristics such as its geometry. Once a vessel is inside the autonomous ship domain, the autonomous ship should anticipate and then execute the correct manoeuvres, based on the predicted course of the other vessel. Figure 2 illustrates the ship domain concept. In addition, as shown in Figure 2, the risk of collision depends on the angle of approach of the two ships. As per COLREG regulations, a head on approach is more risky than one where a ship crosses the path of another ship (or overtakes it) from a stern direction.

Figure 2.

Risk zones around a ship.

3.4 AI and machine learning approaches for training autonomous ships for safe navigation

As the safe sailing and collision avoidance of autonomous ships must equal or better those of human ship operators, it is important that such ships are equipped with artificial intelligence capabilities. Already, machine learning techniques such as Deep Learning, are being applied in various fields of the maritime industry such as detecting anomalies, ship classification, collision avoidance, risk detection of cyberattacks, navigation in ports and so on. Of particular interest to this Chapter is the machine learning technique of reinforcement-based learning [18].

3.5 Reinforcement based learning

Reinforcement learning is an approach to machine learning in which the agents are trained to make a sequence of decisions. It is defined as the learning process in which an agent learns action sequences that maximise some notion of reward [19]. The trained agent interacts with the environment and make decisions or choices. The agent is provided with contextual information about the environment and choices. After it makes a choice, the agent is then provided with the feedback or rewards based on how well the action taken, or the decision made by it, resulted in achieving the desired goal. In many practical applications, the goal is not merely to reach a particular destination, but to do so while maximising some desired utility measure, which could include obeying the rules of the road, maintaining safety, etc. [20].

3.6 Simulators and reinforcement-based learning

Various simulators are used in maritime field to train personnel to navigation, ship handling and the ship bridge equipment. Simulators typically consist of real equipment, real consoles, and instrumentation while the ship and its environment are virtual [21].

As in most cases it is not practical or safe to train an autonomous ship in a real life environment, a high fidelity simulation environment can provide a substitute. In such an environment, the actions of the trained autonomous ship system interact with the simulation actions, and the system receives feedback (‘reward’) from the simulation. Effectively, therefore, the simulation environment becomes the reinforcement based learning agent’s training environment. This is the basis of the approach explored in the next section.

4. Case study: autonomously crossing the English Chanel

4.1 Background

According to Wikipedia, the English Channel shown in Figure 3, is about 560 km long and varies in width from 240 km at its widest to 34 km at its narrowest in the Strait of Dover. The English Channel contains some of the busiest shipping lanes in the world. To ease navigation and improve safety, southbound and northbound traffic lanes were introduced in the 1980s. There is a lateral separation of approximately one nautical mile. This lateral separation removed the need for the ships to alter course for each other as risk of collision does not exist.

Figure 3.

The English Channel and the simulated area (shown as a red circle). Source: OpenStreetMap.

4.2 Simulation environment for autonomous navigation training

Using publicly available AIS data from Marine Traffic (www.marinetraffic.com), we recorded ship positions in the area of interest (an area of approximately 40 km² off the English coast as shown in Figure 4a–d, on regular intervals, over a period of 7 days. The recording interval was set to 10 min. Recorded data included ship’s type, heading and speed. Some of these recordings are visualised in Figure 4a–d. Effectively, the recorded data allowed us to replay any ship’s trajectory over a time period. The collected AIS data were used to create a high fidelity simulation/training environment for training an autonomous ship’s navigation system to plan a safe route across the westbound traffic lane of the English Channel. The training approach consists of several training sessions where the steps of a training session can be summarised as follows:

1. Set the initial position of the autonomous ship, in this training scenario just north of the westbound traffic lane as shown in Figure 4a–d.

2. Set the autonomous ship domain radius. In this scenario we set the ship’s domain to be a circle with a radius of 1 nautical mile (1.852 km). This safety range is common according to the literature; however, it applies to manned vessels. Minimum separation guidelines for situations involving unmanned vessels have not yet been defined, to the best of our knowledge.

3. Set the heading and the speed of the autonomous ship. In this training scenario heading values range from 45 to 135 degrees relative to the direction of the southbound traffic. Speed is approximately 18 kph (~10 knots). Both heading and speed remain constant for the duration of the session.

4. At 10 min intervals, update the ship’s position based on its heading and speed. For the new ship position and from the AIS data, find all ships that are within the autonomous ship’s domain (i.e. safety radius).

5. Calculate the safety risk for each nearby ship, taking into account the angle of the ship’s direction relative to the autonomous ship’s direction, using as reference the diagram of Figure 2. Calculate a total risk index for each step and a total risk index for the entire session.

6. Record the heading, initial traffic state in the area and total risk index.

7. The training objective is that, over a period of several training sessions, the autonomous controller will become increasingly more capable of plotting a route that minimises the total risk index, for any initial traffic state.

Figure 4.

(a) Crossing traffic at right angles (b) paths using different crossing angles (c) a different traffic lane crossing angle (d) another traffic lane crossing angle.

The main difficulty of this training approach is calculating a meaningful and realistic risk index as the current regulations on safe navigation are (a) oriented towards humans and (b) are vague in several aspects.

Figure 4a–d serve to visualise the collision risk index calculation approach. In these figures, the ship domain corresponds approximately to one grid cell and therefore the ship represents potentially a collision risk to other ships occupying the same cell, subject also to their relative headings as explained in Section 4.2.

In Figure 4a for instance, at 7.37 am and 7.47 am the autonomous ship (shown as a black shape) is being approached at approximately right angles (medium risk) by two other ships. A similar situation is shown in Figure 4b, although the collision risk appears visually to be lesser. Interestingly, COLREG recommends that traffic is being approached at right angles, i.e., as approximately shown in Figure 4a. The collision risk seems to be lower in Figure 4c and least in Figure 4d, where there are fewer states (grid cells) where other ships are entering the autonomous ship’s domain at unsafe angles.

It may be difficult to completely automate the calculation of the rewards function (i.e. the risk index) in an unsupervised fashion, mainly due to lack of precise regulations and consensus regarding collision risk scenarios. However, a supervised (and therefore, more time consuming) approach might prove more feasible. In supervised training, human safety experts manually evaluate different ship encounter scenarios and label them with risk indices that reflect the ships’ relative angles of approach, but also speed and ship types. A challenge in supervised training would be to include sufficient numbers of scenarios and provide adequate coverage of all possible situations the autonomous ship is likely to encounter in a particular sea region.

5. Conclusions

While most of the underpinning autonomous ship technologies are available, the overall framework (technical, human, legal) required for their adoption is not. In this Chapter, we illustrated some of the safety issues facing unmanned ships in real life navigation conditions. While autonomous decision making capabilities can indeed by developed, the behaviour of unmanned vessels must also be predictable and understood by other seafarers As argued in [22], navigation is about coordination. This means that there is a need for the autonomous navigation system to decide independently what to do, in a situation where other ships must also make independent decisions. Therefore, collision avoidance, for instance becomes a game of co-ordination, since both ships have to choose independently mutually compatible strategies (Cannell, 1981 in [22]), and must trust the other to comply with their obligations. Along the same line of thinking, the autonomous ship also needs some means of understanding the plans and intentions of the human operated ships. Yanchin and Petrov [23] proposes that the crew of the manned ship uses dedicated equipment to communicate with the autonomous ship to explain their plans, so that the autonomous ship is aware of the human’s decisions and thus can infer its own.

Although the approaches advocated in this chapter are based on concepts of machine learning (in particular reinforcement based learning), they can be embedded in other types of applications such as high fidelity simulation environments for training ship crews. It can become, for example, part of software to train crew to identify and analyse marine traffic situations [24]. In fact, due to the wealth of ship data (Big Data) accumulated mainly from AIS, but also by other monitoring stations, the feasibility of intelligent tutors, underpinned by Big Data and used for both machine and human training is a possibility. An intelligent navigation tutor can provide realistic navigation scenarios (using real life data from AIS databases), and also critique the navigation decisions of the autonomous or human navigator, including the remote operators of unmanned ships, and also analyse and explain the consequences of a navigation decision. It must be noted however, that the current simulation approach, although based on real data, can never become a complete substitute for training onboard real ships, in real sea environments. In the current simulator, for instance, other ships behave as if they are not aware of the existence of the autonomous vehicle, thus their possible reactions to it cannot not be simulated.

However, more advanced future simulations based on intelligent agent concepts, could introduce a new type of simulation environment where virtual ships react to their encounters with the autonomous ship by for instance, altering their speed, course, etc. Thus, we argue that the future of autonomous ship training might require more advanced simulation environments that employs intelligent (software) agent approaches. This of course ties with the future vision of autonomous ships as agent-based systems with intelligence distributed amongst them [25].

A survey of the autonomous ship literature, carried out by [26], reports that nearly no paper discussed typical encounter situations in high-density traffic areas, such as traffic separation schemes and narrow channels. Thus, this Chapter has hopefully created awareness of the need for more research into safe navigation/collision avoidance in high traffic sea environments, where the risks of collision is higher.

6. Future directions

Fully autonomous ships seem currently to be a medium to long term goal. For example, according to Rolls-Royce, the first voyage of a fully automatic ship will happen by 2035 [8]. As Section 1 of this Chapter has reported, although autonomous and remote-controlled ships are being trialled in some sea areas, these are limited to specific ship types such as ferries [27] and waterways [28]. For example, cities with canals can utilise autonomous boats that ferry goods and people, helping to reduce road congestion. IMO also predicts that autonomous or semi-autonomous operation would be limited to short voyages, for example from one specific port to another, across a short distance. Many national regulators, as for example in Norway and in Finland, have encouraged the trialling autonomous or remotely controlled ship operations within national waters. This has led however, to many countries developing their own regulations during the initial trial stages. Therefore, in the long run, cooperation is needed between different countries to ensure consistency of regulations.

The AAWA initiative [8] concluded that hybrid variations between remote and autonomy solutions are more likely to occur first. As previously mentioned, although the technology to make ships autonomous current exists, much still needs to be done to ensure it is reliable. For example, as this Chapter argued earlier on, autonomous ships may reduce the risk of human error given they have no crew, but new types of risks will be created, and this means that vessels will need to be as safe as existing ships, and possibly even more so. A major topic that was addressed in the AAWA initiative was situational awareness and autonomous navigation. At present, human operators supplement any sensor technology that has been installed onboard autonomous ships. In the future, in totally autonomous ships, an array of complimentary (and redundant) sensing technologies has to be installed and fused, to provide sensory data to their collision avoidance systems. In conclusion, this chapter has identified novel perspectives and issues for the safe navigation of autonomous ships that stem from the complex rules and situations that apply to navigation in real life sea environments. Existing technological solutions for autonomous ships will have to be evaluated in greater depth to understand new risks, legal challenges and the stakeholders involved in autonomous operations, and liability issues need to be addressed. As IMO recommends, new autonomous ship terminology and definitions, must be created that clarifies the meaning of conventional marine terms such as ‘master’, ‘responsible person’, etc., in an autonomy context.

Finally, although machine learning and, in particular, deep learning based systems are proving to be an effective mechanism for safe navigation and collision avoidance, [25, 26, 29], they need to coexist and safely interact with more conventional ship technologies, and with the human crew too.