Embedded Knowledge and Autonomous Planning: The Path Towards Permanent Presence of Underwater Networks

plan ψ is a tuple containing a set of partially instantiated actions and a set of constraints over these partially grounded actions. Constraints can of the form of ordering constraints, interval preservation constraints, point truth constraints and binding constraints. Ordering constraints indicate the ordering in which the actions should be executed. Interval preservation constraints link preconditions and effects over actions already ordered. truth constraints assure the existence of precondition facts at certain points of the plan. Autonomous Underwater Vehicles (AUVs) are remarkable machines that revolutionized the process of gathering ocean data. Their major breakthroughs resulted from successful developments of complementary technologies to overcome the challenges associated with autonomous operation in harsh environments. Most of these advances aimed at reaching new application scenarios and decreasing the cost of ocean data collection, by reducing ship time and automating the process of data gathering with accurate geo location. With the present capabilities, some novel paradigms are already being employed to further exploit the on board intelligence, by making decisions on line based on real time interpretation of sensor data. This book collects a set of self contained chapters covering different aspects of AUV technology and applications in more detail than is commonly found in journal and conference papers. They are divided into three main sections, addressing innovative vehicle design, navigation and control techniques, and mission preparation and analysis. The progress conveyed in these chapters is inspiring, providing glimpses into what might be the future for vehicle technology and applications.


Introduction
Oceanographic observatories, year-round energy industry subsea field inspections and continuous homeland security coast patrolling now all require the routine and permanent presence of underwater sensing tools. These applications require underwater networks of fixed sensors that collaborate with fleets of unmanned underwater vehicles (UUVs). Technological challenges related to the underwater domain, such as power source limitations, communication and perception noise, navigation uncertainties and lack of user delegation, are limiting their current development and establishment. In order to overcome these problems, more evolved embedded tools are needed that can raise the platform's autonomy levels while maintaining the trust of the operator. Embedded decision making agents that contain reasoning and planning algorithms can optimize the long term management of heterogeneous assets and provide fast dynamic response to events by autonomously coupling global mission requirements and resource capabilities in real time. The problem, however, is that, at present, applications are mono-domain: Mission targets are simply mono-platform, and missions are generally static procedural list of commands described a-priori by the operator. All this, leaves the platforms in isolation and limits the potential of multiple coordinated actions between adaptive collaborative agents. In a standard mission flow, operators describe the mission to each specific platform, data is collected during mission and then post-processed off-line. Consequently, the main use for underwater platforms is to gather information from sensor data on missions that are static and incapable to cope with the long term environmental challenges or resource changes. In order for embedded service agents to make decisions and interoperate, it is necessary that they have the capability of dealing with and understanding the highly dynamic and complex environments where these networks are going to operate. These decision making tools are constrained to the quality and scope of the available information. Shared knowledge representation between embedded service-oriented agents is therefore necessary to provide them with the required common situation awareness. Two sources can provide this type of information: the domain knowledge extracted from the expert (orientation) and the inferred knowledge from the processed sensor data (observation). In both cases, it will be necessary for the information to be stored, accessed and shared efficiently by the deliberative agents while performing a mission. These agents, providing different capabilities and working in collaboration, might even be distributed among the different platforms or sharing some limited resources.

Contribution
In this chapter, we first provide a review to the different approaches solving the decision making process for UUV missions. Then, we propose a semantic framework that provides a solution for hierarchical distributed representation of knowledge for multidisciplinary agent interaction. This framework uses a pool of hierarchical ontologies for representation of the knowledge extracted from the expert and the processed sensor data. It provides a common machine understanding between embedded agents that is generic and extendable. It also includes a reasoning interface for inferring new knowledge from the observed data and guarantee knowledge stability by checking for inconsistencies. This framework improves local (machine level) and global (system level) situation awareness at all levels of service capabilities, from adaptive mission planning and autonomous target recognition to deliberative collision avoidance and escape. It acts as as an enabler for on-board decision making.
Based on their capabilities, service-oriented agents can then gain access to the different levels of information and contribute to the enrichment of the knowledge. If the required information is unavailable, the framework provides the facility to request other agents with the necessary capabilities to generate the required information, i.e. an target classification algorithm could query the correspondent agent to provide the required object detection analysis before proceeding with its classification task. Secondly, we present an algorithm for autonomous mission adaptation. Using the knowledge made available by the semantic framework, our approach releases the operator from decision making tasks. We show how adaptation plays an important role in providing long term autonomy as it allows the platforms to react to events from the environment while at the same time requires less communication with the operator. The aim is to be effective and efficient as a plan costs time to prepare. Once the initial time has been invested preparing the initial plan, when changes occur, it might be more efficient to try to reuse previous efforts by repairing it. Also, commitments might have been made to the current plan: trajectory reported to other intelligent agents, assignment of mission plan sections to executors or assignment of resources, etc. Adapting an existing plan ensures that as few commitments as possible are invalidated. Using plan proximity metrics, we prove how similar plans are more likely to be accepted and trusted by the operator than one that is potentially completely different. Finally, we show during a series of in-water trials how these two elements combined, a decision making algorithm and shared knowledge representation, provide the required interoperability between embedded service-oriented agents to achieve high-level mission goals, detach the operator from the routinary mission decision making and, ultimately, enable the permanent presence of dynamic sensing networks underwater.

Unmanned decision making loop
In this section, we describe the decision making process currently used by UUV systems and we introduce the unmanned decision loop, where observations, orientations, decisions and actions (OODA) occur in a loop enabling adaptive mission planning. In order to describe the unmanned decision loop, we need to start by modelling the mission environment. A mission environment is defined by the tuple Π =(Σ, Ω), where: • Σ is the mission domain model containing information about domain, i.e. the platform and the environment of execution, and • Ω is the mission problem model containing information about the problem, i.e. mission status, requirements, and objectives.
The set of all possible mission environments for a given domain is defined as the domain space (e.g., the domain space of the underwater domain). It is denoted by Θ. A mission environment Π is an element of one and only one Θ. From this model, a mission plan π that tries to accomplish the mission objectives can be produced. However, this mission environment evolves over time t as new observations of the domain model Σ t and the problem model Ω t continuously modify it: The decision making process to calculate a mission plan π t for a given mission environment Π t occurs in a cycle of observe-orient-decide-act. This process was termed by Boyd (1992) as the OODA-loop, and it was modelled on human behaviour. Inside this loop, the Orientation phase contains the previously acquired knowledge and initial understanding of the situation of the mission environment (Π t−1 ). The Observation phase corresponds to new perceptions of the mission domain model (Σ t ) and the mission problem model (Ω t ) that modify the mission environment. The Decision component represents the level of comprehension and projection, the central mechanism enabling adaptation before closing the loop with the Action stage. Note that it is possible to make decisions by looking only at orientation inputs without making any use of observations. In this case, Eq. 1 becomes Π t ← Π t−1 . In the same way, it is also possible to make decisions by looking only at the observation inputs without making use of available prior knowledge. In this case, Eq. 1 becomes Π t ← Σ t ∪ Ω t . In current UUVs implementations, the human operator constitutes the decision phase. See Figure 1 for a schematic representation of the control loop. When high bandwidth communication links exist, the operator remains in the OODA-loop during the mission execution taking the decisions. For each update of the mission environment Π t received, the operator decides on the correspondent mission plan π t to be performed. From the list of actions in this mission plan, the mission executive issues the correspondent commands to the platform. Examples of the implementation of this architecture are existing Remotely Operated Vehicles (ROVs). However, when communication is unreliable or unavailable, the operator must attempt to include all possible if-then-else cases to cope with execution alternatives before the mission starts. This is the case of current UUVs implementations that follow an orientation-only model. Figure 2 shows this model, where the OODA-loop is broken because observations are not reported to the human operator.
We will now discuss a few recent UUV implementations which show where the state-of-the-art is currently positioned. Most implementations rely on pre-scripted mission    plan managers that are procedural and static and might not even consider conditional executions (Hagen, 2001). At this level, the mission executive follows a sequence of basic command primitives and issues them to the functional control layer of the platform. Description about how these approaches maintain control of underwater vehicles can be found in Fossen (1994), Ridao et al. (1999) and Yuh (2000). In this situation, decisions taken by the operator are made using only orientation inputs related to some previous experience and a-priori knowledge. This has unpredictable consequences, in which unexpected situations can cause the mission to abort and might even cause the loss of the vehicle (Griffiths, 2005;von Alt, 2010). More modern approaches are able to mitigate this lack of adaptability by introducing sets of behaviours that are activated based on observations (Arkin, 1998). Behaviours divide the control system into a parallel set of competence-levels. They can be seen as manually scripted plans generated a-priori to encapsulate the decision loop for an individual task. Under this approach, the key factor is to find the right method for coordinating these competing behaviours.
The subsumption model, attributed to Brooks (1986), arbitrates behaviour priorities through the use of inhibition (one signal inhibits another) and suppression (one signal replaces other) networks. Most recent UUV control systems are a variant of the subsumption architecture. This model was first applied to the control of UUVs by Turner (1995) during the development of the ORCA system. This system used a set of schemas in a case-based framework. However, its scalability remains unclear as trials for its validation were not conducted. Later, Oliveira et al. (1998) developed and deployed the CORAL system based on Petri nets. The system was in charge of activating the vehicle primitives needed to carry out the mission. These primitives were chained by preconditions and effects. The scaling problem was addressed by Bennet & Leonard (2000) using a layered control architecture. Layered control is a variant of the subsumption model that restricts of interaction between layers in order to keep it simple (Bellingham et al., 1990). The system was deployed for the application of adaptive feature mapping. Another approach for coordinating behaviours is vector summation that averages the action between multiple behaviours. Following this principle, the DAMN system developed by Rosenblatt et al. (2002) used a voting-based coordination mechanism for arbitration implementing utility fusion with fuzzy logic. The MOOS architecture developed by Newman (2002) was also able to guide UUVs by using a mission control system called Helm. Helm's mission plan was described by a set of prioritised primitive tasks. The most suitable action was selected using a set of prioritised mission goals. It used a state-machine for execution, a simplified version of a Petri net.
The O 2 CA 2 system (Carreras et al., 2007) also used a Petri net representation of the mission plan (Palomeras et al., 2009). The system maintains the low level control (dynamics) from the guidance control (kinematics) uncoupled (Caccia & Veruggio, 2000). Although it contained a declarative mission representation, missions were programmed manually. A detailed survey of other behaviour-based approaches applied to mission control systems for UUVs can be found in Carreras et al. (2006). More recently, Benjamin et al. (2009) has applied multiple objective decision theory to provide a suitable framework for formulating behaviour-based controllers that generate Pareto-optimal and satisfying behaviours. This approach was motivated by the infeasibility of optimal behaviour selection for real-world applications. This approach has been implemented and deployed as part of the IvP Helm extension to MOOS. This method seems to be a more suitable for behaviour selection, although more computationally expensive. Also, the approach is limited to the control of only the direction and velocity parameters of the host platform.
After reviewing this related work, two problems affecting the effectiveness of the decision loop become evident. Firstly, orientation and observation should be linked together because it is desirable to place the new observations in context. Secondly, decision and action should be iterating continuously. These two problems have not been addressed together by previous approaches. These are the two of the goals that we address in this chapter. In order to achieve them autonomously, two additional components are required: a status monitor and a mission plan adapter. The status monitor reports any changes detected in the mission environment during the execution of a mission. When the mission executive is unable to handle the changes detected by the status monitor, the mission planner is called to generate a new modified mission plan that agrees with the updated mission environment. Figure 3 shows the OODA-loop for autonomous decision making. Comparing it to the previous Figure 2, the addition of status monitor and mission planner removes the need for human decisions in the loop. Note that the original mission plan π 0 could also be autonomously generated as long as the high-level goals are provided by the human operator in Π 0 .  Adaptive mission planning enables a true unmanned OODA-loop. This autonomous decision making loop copes with condition changes in the mission environment during the mission execution. As a consequence, it releases the operator from decision making tasks in stressful environments containing high levels of uncertainty and dynamism. The potential benefits of adaptive mission planning capabilities for autonomous decision making in UUVs were promoted by Turner (2005), Bellingham et al. (2006) and . Possibly the most advanced autonomous decision making framework for UUVs has been developed at the Monterey Bay Aquarium Research Institute. This architecture, known as T-REX, has been deployed successfully inside the Dorado AUV . This is now providing adaptive planning capabilities to oceanographers for maximising the science return of their UUV missions Rajan et al., 2007). Using deliberative reactors for the concurrent integration of execution and planning (McGann, Py, , live sensor data can be analysed during mission to adapt the control of the platform in order to measure dynamic and episodic phenomenon, such as chemical plumes (McGann, Py, Rajan, Henthorn & McEwen, 2008;McGann et al., 2009). Alternative approaches to adaptive plume tracing can also be found in the works of Farrell et al. (2005) and Jakuba (2007). Their research goals of all these approaches have been motivated by scientific applications and do not consider the needs of the human operators or the maritime industry. However, autonomy cannot be achieved without humans, as it is necessary for this autonomy to be ultimately accepted by an operator. Our research is geared towards improving human access to UUVs in order to solve the maritime industry's primary requirement of improving platform operability (Patrón et al., 2007). We propose a goal-based approach to solving adaptive mission planning. The advantage of this approach is that it provides high levels of mission abstraction. This makes the human interface simple, powerful and platform independent, which greatly eases the operator's task of designing and deploying missions (Patrón, 2009). Ultimately, operators will not need any specialist training for an UUV specific platform, and instead missions will be described purely in terms of their goals. Apart from ease of use, we have also demonstrated using a novel metric (Patrón & Birch, 2009) that adaptive mission planners can produce solutions which are close to what a human planner would produce (Patrón et al., 2009a). This means that our solutions can be trusted by an operator. Another advantage of our research over other state-of-the-art UUV implementations, is that we are industry focussed. Our service-oriented approach provides goal-based mission planning with discoverable capabilities, which meets industry's need for platform independence (Patrón et al., 2009b). Finally, our plan repair approach optimises the resources required for adaptability and maximises consistency with the original plan, which improves human acceptance of autonomy. Resource optimisation and consistency are very important properties for real world implementations, as we demonstrate in our sea trials (Patrón, Miguelanez, Petillot & Lane, 2008). Section 3 describes how do we link together orientation and observation. Section 4 presents an approach to the continuous iteration of decision and action.

Semantic knowledge-based situation awareness
Unmanned vehicle situation awareness SA V consists in enabling the vehicle to autonomously understand the 'big picture' (Adams, 2007). This picture is composed of the experience gained from previous missions (orientation) and the information obtained from the sensors while on mission (observation). Ontologies allow the representation of knowledge of these two components.
Ontologies are models of entities and interactions, either generically or in some particular practice of knowledge (Gruber, 1995). The main components of an ontology are concepts and axioms. A concept represents a set or class of entities within a domain (e.g., a fault is a concept within the domain of diagnostics). Axioms are used to constrain the range and domain of the concepts (e.g., a driver is a software that has a hardware). The finite set of concept and axiom definitions is called the Terminology Box TBox of the ontology. In the past, authors such as Matheus et al. (2003) and Kokar et al. (2009) have used ontologies for situation awareness in order to assist humans during information fusion and situation analysis processes. Our work extends these previous works by using ontologies for providing unmanned situation awareness in order to assist autonomous decision making algorithms in underwater vehicles. One of the main advantages of using a knowledge base over a classical data base schema to represent SA V is the extended querying that it provides, even across heterogeneous data systems. The meta-knowledge within an ontology can assist an intelligent agent (e.g., status monitor, mission planner, etc.) with processing a query. Part of this intelligent processing is due to the capability of reasoning. This enables the publication of machine understandable meta-data, opening opportunities for automated information processing and analysis. For instance, a status monitor agent using meta-data about sensor location could automatically infer the location of an event based on observations from nearby sensors . Inferences over the ontology are made by reasoners. A reasoner enables the domain's logic to be specified with respect to the context model and applied to the corresponding knowledge i.e., the instances of the model (see Fig. 4). A detailed description of how a reasoner works is outside of the scope of this article. For the implementation of our approach, we use the open source reasoner called Pellet (Sirin et al., 2007).

Semantic knowledge-based framework
A library of knowledge bases comprise the overall knowledge framework used in our approach for building SA V (Miguelanez et al., 2010;Patrón, Miguelanez, Cartwright & Petillot, 2008). Reasoning capabilities allow concept consistency providing reassurance that SA V remains stable through the evolution of the mission. Also, inference of concepts and relationships allows new knowledge to be extracted or derived from the observed data. In order to provide with a design that supported maximum reusability (Gruber, 1995;van Heijst et al., 1996), we adopt a three-level segmentation structure that includes the (1) Foundation,   Foundational Ontologies (FOs) represents the very basic principles and includes Upper and Utility Ontologies. Upper ontologies describe generic concepts (e.g., the Suggested Upper Merged Ontology or SUMO (Niles & Pease, 2001)) while Utility ontologies describe support concepts or properties (e.g. OGC_GML for describing geospatial information (Portele, 2007)). FOs meet the requirement that a model should have as much generality as possible, to ensure reusability across different domains.
The Core Ontology provides a global and extensible model into which data originating from distinct sources can be mapped and integrated. This layer provides a single knowledge base for cross-domain agents and services (e.g., vehicle resource / capabilities discovery, vehicle physical breakdown, and vehicle status). A single model avoids the inevitable combinatorial explosion and application complexities that results from pair-wise mappings between individual metadata formats and ontologies.
In the bottom layer, an Application Ontology provides an underlying formal model for agents that integrate source data and perform a variety of extended functions. As such, higher levels of complexity are tolerable and the design is motivated more by completeness and logical correctness than human comprehension. Target areas of these Application Ontologies are found in the status monitoring of the vehicle and its environment and the planning of the mission. Figure 6 represents the relationship between the Foundation Ontologies (Upper and Utility), the Core Ontology and the Application Ontology for each service-oriented agent. Raw data gets parsed from sensors into assertions during the mission using a series of adapter modules for each of the sensing capabilities. It also shows that the knowledge handling by the agent during its decision making process is helped by the reasoner and the rule engine process.

Foundation and core ontology
To lay the foundation for the knowledge representation of unmanned vehicles, consideration was placed on the Joint Architecture for Unmanned Systems (JAUS) (SAE, 2008a). This standard was originally developed for the Unmanned Ground Vehicles (UGVs) environment only but has recently been extended to all other environments, such as air and water, trying to provide a common set of architecture elements and concepts. The JAUS model classifies four different sets of Knowledge Stores: Status, World map, Library and Log. Our experience has shown that overlap exists between these different sets of knowledge stores. The approach proposed in this paper provides more flexibility in the way the information can be accessed and stored, while still providing JAUS 'Message Interoperability' (SAE, 2008b) between agents.
Within the proposed framework, JAUS concepts are considered as the Foundation Ontology for the knowledge representation. The Core Ontology developed in this work extends these concepts while remaining focused in the domain of unmanned systems. Some of the knowledge concepts identified related with this domain are: • Platform: Static or mobile (ground, air, underwater vehicles), • Payload: Hardware with particular properties, sensors or modules, • Agent: Software with specific capabilities, • Sensor: A device that receives and responds to a signal or stimulus, • Driver: Module for interaction with a specific sensor / actuator, Additionally, the Standard Ontology for Ubiquitous and Pervasive Applications (SOUPA) (Chen et al., 2004) is used as an Utility Ontology. By providing generic context-aware concepts, it enables the spatio-temporal representation of concepts in the Core Ontology.

Application Ontology
Each service-oriented agent has its own Application Ontology. It represents the agent's awareness of the situation by including concepts that are specific to the expertise of the agent.
In the case study presented in this chapter, these agents are the status monitor and the mission planner. Together, they provide the status monitor and mission adapter components described in Fig. 3 required for closing the OODA-loop and provide on-board decision making adaptation.

Status Monitoring Application Ontology
The Status Monitoring Application Ontology is used to express the SA V of the status monitor agent. To model the behaviour of all components and subsystems considering from sensor data to possible model outputs, the Status Monitoring Application Ontology is designed and built based on ontology design patterns (Blomqvist & Sandkuhl, 2005). Ontology patterns facilitate the construction of the ontology and promote re-use and consistency if it is applied to different environments. In this work, the representation of the monitoring concepts are based on a system observation design pattern. Some of the most important concepts identified for status monitoring are: • Data: all internal and external variables (gain levels, water current speed), • Observation: patterns of data (sequences, outliers, residuals,...), • Symptom: individuals related to interesting patterns of observations (e.g., low gain levels, high average speed), • Status: links the latest and most updated event information to the systems being monitored (e.g. sidescan transducer), Please note how some of these concepts are related to concepts of the Core Ontology (e.g. an observation comes from a sensor). These Core Ontology elements are the enablers for the knowledge exchange between service-oriented agents. This will be shown in the demonstration scenario of Section 5.3.

Planning Application Ontology
The Plan Application Ontology is used to express the SA V of the mission planner agent. It uses concepts originally defined by the Planning Domain Definition Language (PDDL). The PDDL language was originally created by Ghallab et al. (1998) to standardise plan representation. Concepts are extracted from the language vocabulary and the language grammar is used for describing the relationships and constraints between these concepts. For the adaptation mission planning process, the Planning Application Ontology also requires concepts capable of representing the diagnosis of incidents or problems occurring in some parts of the mission plan (van der Krogt, 2005). Some of the most important concepts identified for mission plan adaptability are: • Resource: state of an object (physical or abstract) in the environment (vehicle, position, sensor, etc.), • Action: Modification of the state of resources (calibrate, classify, explore, etc.), • Gap: A non-executable action, • Execution: When an action is executed successfully, • Failure: An unsuccessful execution of an action, Please note how some of these concepts are also related to concepts of the Core Ontology (e.g. a list of capability concepts is required to perform a mission action).

Adaptive mission planning
The adaptive mission planning process involves the detection of events, the effects that these events have on the mission plan and the response phase. The detection of events is performed by the status monitoring agent. The mission plan diagnosis and repair is undertaken by the adaptive mission planner agent.

Status Monitor Agent
The Status Monitor Agent considers all symptoms and observations from environmental and internal data in order to identify and classify events according to their priority and their nature (critical or incipient). Based on internal events and context information, this agent is able to infer new knowledge about the current condition of the vehicle with regard to the availability for operation of its components (i.e. status). In a similar way, environmental data is also considered for detecting and classifying external events in order to keep the situation awareness of the vehicle updated.  Fig. 7. Schematic representation of the autonomous mission generation, replan and repair processes using partial plan representation of the mission plans.

Mission plan adaptation agent
This section describes the mission environment model used by the mission plan repair techniques presented in this chapter. We continue using the mission environment model previously described. We generalise this model to any step q in the mission execution timeline. An instance of an UUV mission environment at a given step q can be simply defined as Π q = Σ q , Ω q . The mission domain model Σ q contains the set of propositions defining the available resources in the system P V and the set of actions or capabilities A V . The mission problem model Ω q contains the current platform state x q and the mission requirements Q O . Based on this model, we analyse how to calculate mission plans on the plan space (Sacerdoti, 1975). A plan space is an implicit directed graph whose vertices are partially specified plans and whose edges correspond to refinement operations. In a real environment where optimality can be sacrificed by operability, partial plans are seen as a suitable representation because they are a flexible constrained-based structure capable of being adapted. A partial plan ψ is a tuple containing a set of partially instantiated actions and a set of constraints over these partially grounded actions. Constraints can be of the form of ordering constraints, interval preservation constraints, point truth constraints and binding constraints.
Ordering constraints indicate the ordering in which the actions should be executed. Interval preservation constraints link preconditions and effects over actions already ordered. Point truth constraints assure the existence of precondition facts at certain points of the plan. Binding constraints on the variables of actions are used to ground the actions to variables of the domain. Figure 8 shows a partial plan representation of an UUV mission. Partial plans are flexible to modification. They provide an open approach for handling extensions such as temporal and resource constraints. Due to nature of the constraints, it is easy to explain a partial plan to a user. Additionally, it is easily extensible to distributed multi-agent mission planning. Figure 7 explains the processes of mission plan repair and mission plan replan for mission plan adaptation for UUVs using a partial plan representation of the mission plans. At the initial step, a partial ordered plan ψ 0 is generated satisfying the original mission environment Π 0 . The ψ 0 is then grounded into the minimal mission plan π 0 including all constraints in ψ 0 . At step q, the semantic knowledge-based framework is updated by the diagnosis informationΠ q providing a modified awareness of the mission environment Π q . From here, two mission adaptation processes are possible: Mission replan generates a new partial plan ψ q , as done at the first stage, based only on the knowledge of Π q . On the other hand, mission plan repair re-validates the original plan by ensuring minimal perturbation of it. Given the partial plan at the previous step ψ q−1 and the diagnosis informationΠ q , the mission repair problem produces a solution partial plan ψ q that satisfies the updated mission problem Π q ,by modifying ψ q−1 . The final step for both approaches is to ground ψ q to its minimal mission plan π q . It can be seen that mission repair better exploits the orientation capabilities for decision making: instead of taking the new mission environment as a given, it uses the diagnosis information about the changes occurred to guide the adaptation process.
We have now identified the benefits of mission plan repair over mission replan. Mission plan repair modifies the partial plan ψ q , so that it uses a different composition, though it still maintains some of the actions and the constraints between actions from the previous partial plan. However, mission plan adaptation can also be achieved by mission execution repair by looking directly at the mission plan instantiation π q . Execution repair modifies the instantiation of the mission plan π q such that a ground action g a h q that was previously instantiated by some execution e q is newly bound by another action execution instance e ′ q . Executive repair is less expensive and it is expected to be handled directly by the mission executive agent. Plan repair, however, is computationally more expensive and requires action of the mission planner agent. The objective is to maximise the number of execution repairs over plan repairs and, at the plan repair level, maximise the number of decisions reused from the previous mission instantiation. The information provided by the semantic-base knowledge base during the plan diagnosis phase is critical. Executive repair fixes plan failures identified in the mission plan during the diagnosis stage. Our approach uses ontology reasoning in combination with an action execution template to adapt the mission plan at the executive level. Once a mission plan π q is calculated by the mission planner, its list of ground actions is transferred to the executive layer. In this layer, each ground action g a h q of π q gets instantiated into an action execution instance e t q using the action template for the action a h available in the Core Ontology of the knowledge base. At the end of this phase, each e t q contains the script of commands required to perform its correspondent ground action. Flexibility in the execution of an action instance is critical in real environments. This is provided by a timer, an execution counter, a time-out register and a register of the maximum number of executions in the action execution instance. Additionally, three different outputs control the success, failure or time-out of its execution. These elements handle the uncertainty during the execution phase and enable the executive repair process. This minimise the number of calls to the adaptive mission planner agent and therefore the response time for adaptation.
Plan repair uses a strategy to repair with new partial plans the plan gaps identified during the plan diagnosis stage. Our approach uses an iteration of unrefinement and refinement strategies on a partial-ordered planning framework to adapt the mission plan. Planning in the plan space is slower than in the state space because the nodes are more complex. Refinement operations are intended to achieve an open goal from the list of mission requirements or to remove a possible inconsistency in the current partial plan. These techniques are based on the least commitment principle, and they avoid adding to the partial plan any constraint that is not strictly needed. A refinement operation consists of one or more of the following steps: adding an action, an ordering constraint, a variable binding constraint or a causal link. A partial plan is a solution to the planning problem if has no flaw and if the sets of constraints are consistent. Flaws are either subgoals or threats. Subgoals are open preconditions of actions that have not been linked to the effects of previous actions. Threats are actions that could introduce inconsistencies with other actions or constraints. We implemented a recursive non-deterministic approach based on the Partial ordered Planning (PoP) framework (Penberthy & Weld, 1992). This framework is sound, complete, and systematic. Unlike other Plan space planners that handle both types of flaws (goals and threats) similarly, each PoP recursive step first refines a subgoal and then the associated threats (Ghallab et al., 2004). In our implementation, we introduce a previous step capable of performing an unrefinement of the partial plan when necessary. During the unrefinement strategy we remove refinements from the partial plan that are reported by the plan diagnosis phase to be affecting the consistency of the mission plan with the mission environment, i.e. to remove constraints and finally the actions if necessary. In simple terms, when changes on the ABox Planning Application Ontology are sensed (Π q ) that affect the consistency of the current partial plan ψ q−1 , the plan repair process is initiated. The plan repair stage starts an unrefinement process that relaxes the constraints in the partial plan ψ q−1 that are causing the mission plan to fail. The remaining temporal mission partial plan ψ ′ q−1 is now relaxed to be able to cope with the new mission environment. However, this relaxation could open some subgoals and introduce threats in the partial plan that need to be addressed. The plan repair stage then executes a refinement process searching for a new mission plan ψ q that is consistent with the new mission environment Π q and removing these possible flaws. By doing this, it can be seen that the new mission plan ψ q is not generated again from Π q (re-planned) but recycled from ψ q−1 (repaired). This allows re-use of the parts of the plan ψ q−1 that were still consistent with Π q .

Architecture
The combination of the status monitor agent, the adaptive mission planner, the mission executive and the semantic knowledge-based framework is termed as the Semantic-based Adaptive Mission Planning system (SAMP). The SAMP system implements the four stages of the OODA-loop. Figure 9 represents the customised version of Figure 3 for SAMP. The status monitor agent reports to the knowledge base the different changes occurring in the environment and the modifications of the internal status of the platform. The knowledge base stores the ontology-based knowledge containing the expert orientation provided a priori and the observations reported by the status monitor. A mission planner agent generates and  Fig. 9. Architecture of the SAMP system. The embedded agents are the planner, executive, monitor, and knowledge base. These agents interconnect via set of messages. The system integrates to the functional layer of a generic host platform by an abstract layer interface (ALI).
adapts mission plans based on the situation awareness stored in the knowledge base. The mission executive agent executes mission commands in the functional layer based on the sequence of ground actions received from the mission planner. An Abstract Layer Interface (ALI) based on JAUS-like messages (SAE, 2008a) over UDP/IP packages implemented using the OceanSHELL protocol (Oce, 2005) provides independence from the platform's functional layer making the system generic and platform independent.

Simulation results
A set of synthetic simulated scenarios have been implemented to test the performance of the SAMP system. The tests are based on the mine counter measure (MCM) operation, where UUVs support and provide solutions for mine-hunting and neutralisation. A set of 15 selected MCM scenarios were simulated covering the variability of missions described by the concepts of operations for unmanned underwater vehicles presented in the UUV (2004) and the JRP (2005). For each scenario, the detection of a failure in one of the components of the system was simulated. The mission plan was adapted to the new constraints using replanning methods and the mission plan repair approach based on partial plans introduced in Section 4. The performance of the two approaches was compared by looking at the computation time and the Plan Proximity (Patrón & Birch, 2009) of the adaptive mission plan provided to the original reference mission plan. Figure 10 left shows the computation time in milliseconds required for adapting the mission to the new constraints for replan (dark grey bars) and repair approaches (light grey bars). Note that a logarithmic scale is used for the time values. Figure 10 right displays the Plan Proximity to the original plan of the replan strategy result versus the repair strategy result. It can be seen that plans adapted using the mission repair strategy tend to be closer to the original plan than using the mission replan strategy. In these results, 14 out of 15 scenarios were computed faster by using mission plan repair. This computation was on average 9.1x times faster. Also, 14 out of 15 scenarios showed that mission plan repair had greater or equal Plan Proximity values as compared to mission replan. In general, our mission repair approach improves performance and time response while at the same time finds a solution that is closer to the original mission plan available before adaptation.

Experimental results
This section shows the performance of the SAMP system inside the real MCM application using a REMUS 100 UUV platform (see Fig. 11.left) in a set of integrated in-water field trial demonstration days at Loch Earn, Scotland (56 o 23.1N,4 o 12.0W). The REMUS UUV had a resident guest PC/104 1.4GHz payload computer where the SAMP system was installed. SAMP was capable of communicating with the vehicle's control and status modules and taking control of it by using an interface module that translated the ALI protocol into the manufacturer's Remote Control protocol (REM, 2008).
area 1 area 2 Fig. 11. Left: REMUS UUV deployment before starting one of its missions. Right: Procedural mission uploaded to the vehicle control module and a priori seabed classification information stored in the knowledge base. The two dark grey areas correspond to the classified seabed regions. Figure 11.right shows the procedural waypoint-based mission as it was described to the vehicle's control module. This was known as the baseline mission. It was only used to start the vehicle's control module with a mission in the area of operation before taking control of it using the SAMP system. The baseline mission plan consisted on a start waypoint and two waypoints describing a North to South mission leg at an approximate constant Longitude (4 o 16.2W). This leg was approximately 250 meters long and it was followed by a loiter pattern at the recovery location. The track obtained after executing this baseline mission using the vehicle control module is shown in Fig. 11 with a dark line. A small adjustment of the vehicle's location can be observed on the top trajectory after the aided navigation module corrects its solution to the fixes received from the Long Baseline (LBL) transponders previously deployed in the area of operations.  On the payload side, the SAMP system was oriented (in the OODA-loop sense) using a priori information about the environment and the platform and a declarative description of the goals of the mission. The a priori knowledge and the platform configuration capabilities was represented using Core Ontology concepts (see Fig. 12). Knowledge about the environment was provided based on automatic computer-aided seabed classification information generated from previous existent data ( Reed et al., 2006). The two classified seabed areas are shown in Fig. 11. The declarative description of the mission requirements was represented using concepts from the Planning Application Ontology. They could be resumed as 'survey all known areas maximizing efficiency'.

Pre-mission reasoning
Please note that the previously described separation between Core knowledge and Planning knowledge gracefully aligns with the separation between platform engineers and mission scientists on current UUV operations. If the platform capabilities were described in Core Ontology terms by the engineers that manufactured the platform, it can be seen how, by using the SAMP approach, a scientific operator that only cares about the data should be able to describe the mission to the platform without knowing anything about the custom properties of the platform. It is, therefore, important to assist the operator in knowing if the platform capabilities can match the mission requirements before starting the mission: • Is this platform configuration suitable to successfully perform this mission?
In order to answer this question, new knowledge could be inferred from the initial Core Ontology orientation. The Core Ontology rule engine was executed providing with additional knowledge. A set of predefined rules helped orienting the knowledge base into inferring new relationships between instances. An example of a rule dealing with the transfer of payload capabilities to the platform is represented in Eq. 2.
core : isCapabilityO f (?Capability,?Payload)∧ core : isPayloadO f (?Payload,?Plat f orm) → core : isCapabilityO f (?Capability,?Plat f orm) (2) Once all the possible knowledge was extracted, it was possible to query the knowledge base in order to extract the list of capabilities of the platform (see Eq. 3) and the list of requirements of the mission (see Eq. 4).

SELECT
This way, it was possible to autonomously extract that the requirements of the mission of the experiment were 1 : • core:WaypointManeuver_Capability ∈ jaus:Maneuver_Capability • core:ComputerAidedClassification_Capability ∈ jaus:Autonomous_RSTA-I_Capability • core:ComputerAidedDetection_Capability ∈ jaus:Autonomous_RSTA-I_Capability • core:SidescanSensor_Capability ∈ jaus:Environmental_Sensing_Capability which were a subset of the platform capabilities. Therefore, for this particular case, the platform configuration suited the mission requirements.

In mission adaptation
For these experiments, SAMP was given a static location in which to take control of the host vehicle. At this point, the mission planner agent generated a mission plan based on the knowledge available and the mission requirements. The instantiation of this mission plan is described in Fig. 13 using Planning Application Ontology concepts. The mission was then passed to the executive agent that took control of the vehicle for its execution. While the mission was executed the status monitor agent maintained the knowledge base updated (in the OODA-loop sense) by reporting changes in the status of hardware components, such as batteries and sensors, and external parameters, such as water currents. When observations indicated that some of these changes were affecting the mission under execution, the mission planner was activated in order to adapt the mission to the changes. This indication was detected by the planner agent by querying the knowledge base with the following question: • Are the observations coming from the environment affecting the mission currently under execution?
In order to explain the reasoning process involved during the event detection, diagnosis and response phases of the mission adaptation process, a component fault as an internal event was temporarily simulated in the host vehicle. The fault simulated the gains of the starboard transducer of the sidescan sonar dropping to their minimum levels half way through the lawn mower survey of the first area. For the detection phase, the low gain signals from the transducer triggered a symptom instance, which had an associated event level. This event level, represented in the Status Monitoring Application Ontology using a value partition pattern, plays a key role in the classification of the instances in the Event concept between being critical or incipient. This classification is represented axiomatically in the Eqs. 5 and 6.
After the Event individuals were re-classified, the Status property of the related component in the Core Ontology was updated. During the diagnosis event phase, a critical status of a component is only considered to be caused by a critical event. Therefore, due to the fact that the sidescan sonar component is composed of two transducers, port and starboard, one malfunctioned transducer was only diagnosed as an incipient Status of the overall sidescan sonar component. During the response phase, the Status property of the Core Ontology components were used by the mission planner to perform the plan diagnosis of the mission under execution. The query to the knowledge base shown in Eq. 7 reported that the two survey actions in the mission plan were affected by the incipient status of the sidescan sonar.
An incipient Status of the action parameters indicates that the action can still be performed by adapting the way it is being executed, an execution repair. If both transducers were down, a critical status of the sidescan sensor is diagnosed and a plan repair adaptation of the mission plan would have been required instead. In that case, the adaptive mission planner would have looked for redundant components or similar capabilities to perform the action or to drop the action from the plan. The same procedure was used after the transducer recovery was reported to adapt the survey action to the normal pattern during the second lawn mower survey. In a similar process, SAMP adapted the lawnmower pattern survey of the areas to the detected water current Status at the moment of initialising the survey of the areas. The timeline of the mission executed using the SAMP approach is described in the following figures: Figure 14 represents the final trajectory of the vehicle in 2D and 3D using a North-East coordinate frame projection with the origin at the starting point of the mission. Figure 15 displays the vehicle's telemetry recorded during the mission. It includes vehicle's velocity, compass heading, altitude, depth measurements and processed bathymetry over time. Figure 16 shows subset of the variables being monitored by the status monitor agent that were relevant to this experiment. These variables include direction of water current, remaining battery power, the availability of the transducers in the sidescan sensor and the mission execution status. Figure 17 represents the system activity of the payload computer recorded during the mission. The system activity logs show percentage of processor usage, memory usage, network activity and disk usage. Each of the symbols , , ♦, △ and on the aforementioned figures represents a point during the mission where an event occurred. Symbol represents the point where SAMP takes control of the vehicle. Note a change on the host platform mission status binary flag that becomes 0x05, i.e. the mission is active (0x01) and the payload is in control (0x04) (Figure 16.e).
Also, a peak on the CPU usage can be noted as this is the point where the mission partial plan gets generated (Figure 17.a). Symbol represents the point where the vehicle arrives to perform the survey of the area. At this point, the action survey gets instantiated based on the properties of the internal elements and external factors. Although the Loch waters where the trials were performed were very still (see Figure 16.b), note how the vehicle heading during the lawnmower pattern performed to survey the areas follows the water current direction sensed at the arrival (approx. 12 o , Symbol - Figure 16.a) in order to minimize drag and maximise battery efficiency. The heading of the vehicle during the survey can be observed in Figure 14 and Figure 15.b. The link between the vehicle heading in relation to the water current direction and its effect on the battery consumption was expert orientation knowledge captured by a relationship property between the two concepts in the Core Ontology. Symbol ♦ represents the point when the status monitor agent detects and reports a critical status in the starboard transducer of the sidescan sonar (Figure 16.d). It can be seen how the lawnmower pattern was adapted to cope with the change and to use the port transducer to cover the odd and even spacing of the survey. This pattern avoids gaps in the sidescan data under the degraded component configuration and maximises sensor coverage for the survey while the transducer is down. Symbol △ indicates the point where the starboard transducer recovery is diagnosed. It can be observed how the commands executing the action are modified in order to optimise the survey pattern and minimise distance travelled. Although also being monitored, the power status does not report any critical status during the mission that requires modification of the actions (Figure 16.c). Symbol shows the location where all the mission goals are considered achieved and the control is given back to the mission control of the host vehicle (see Symbol - Figure 16.e shows the mission is still active but the payload is not longer in control (0x01) ). From this point the host vehicle's control module takes the control back and drives the vehicle to the loiter at the recovery location.

Conclusion and future work
The underwater domain is a challenging environment in which to maintain the operability of an UUV. Operability can be improved with the embedded adaptation of the mission plan. We implement a system capable of adapting mission plans autonomously in the face of events while during a mission. We do this by using a combination of ontological hierarchical representation of knowledge and adaptive mission plan repair techniques. The advantage of this approach is that it maximises robustness, system performance and response time.
The system performance has been demonstrated in simulation. Additionally, the mission adaptation capability is shown during an in-water field trial demonstration. In our fully integrated experiments we achieved the following: • Knowledge based framework: We have presented a semantic-based framework that provides the core architecture for knowledge representation for service oriented agents in UUVs. The framework combines the initial expert orientation and the observations acquired during mission in order to improve the situation awareness in the vehicle. This is currently unavailable in UUVs.
• Goal-oriented plan vs. waypoint-based plan: The system uses a goal-oriented approach in which the mission is described in terms of 'what to do' instead of a 'how to do' it. The mission is parametrised and executed based on the available knowledge and vehicle capabilities. This is the first time that an approach to goal-based planning is applied to the adaptation of an underwater mission in order to maintain platform's operability.
• Adaptation to environmental parameters and internal issues: The approach shows adaptability to environmental elements, such as water current flows in order to improve mission performance. The approach is also capable of dealing with the critical status of certain components in the platform and can react accordingly.
• Platform agnostic: The approach is platform independent making it readily applicable to other domains, such as ground or air vehicles.
SAMP is open to event detections coming from other embedded service-oriented agents. We are planning to apply the approach to more complex scenarios involving other embedded agents, such as agents for automatic target recognition. We are also planning to extend it to a team of vehicles performing a collaborative mission. In this scenario, agents are distributed across the different platforms. We are currently working towards a shared situation awareness for a team of vehicles to which every team member possess the awarenessrequired for its responsibilities. The main advantage of our semantic-based approach is the low bandwidth required to share id-coded ontological concepts and, therefore, to cope with the underwater acoustic communication limitations. Autonomous Underwater Vehicles (AUVs) are remarkable machines that revolutionized the process of gathering ocean data. Their major breakthroughs resulted from successful developments of complementary technologies to overcome the challenges associated with autonomous operation in harsh environments. Most of these advances aimed at reaching new application scenarios and decreasing the cost of ocean data collection, by reducing ship time and automating the process of data gathering with accurate geo location. With the present capabilities, some novel paradigms are already being employed to further exploit the on board intelligence, by making decisions on line based on real time interpretation of sensor data. This book collects a set of self contained chapters covering different aspects of AUV technology and applications in more detail than is commonly found in journal and conference papers. They are divided into three main sections, addressing innovative vehicle design, navigation and control techniques, and mission preparation and analysis. The progress conveyed in these chapters is inspiring, providing glimpses into what might be the future for vehicle technology and applications.