Improvement of RISE Mobile Robot Operator Training Tool

In the chapter the framework for RISE (Risky Intervention and Environmental Surveillance) mobile robot operator training design is presented. The basic idea of the framework is to provide advanced software tools to improve the performance of the mobile robot simulation design that are applicable for operator training. The simulator is the end product of the framework. To achieve realistic rigid body simulation and realistic rendering the NVIDIA PhysX engine and the OGRE (Open Graphic Rendering Engine) are used. The compatibility between the framework software and CAD modeling tools such as SolidWorks was essential, therefore virtual models are exchangeable using COLLADA format. The simulator can be integrated with control panel of the mobile robot. It is possible to simulate the behavior of several types of robots such as caterpillar or wheeled. The improvement presented in this chapter is related to the problem of automatic environment model generation based on autonomous mobile robot observations. The approach related to semantic mapping is used and in consequence semantic simulation engine is implemented. Semantic simulation engine is composed by data registration module, semantic entities identification module and environment model generation module. Presented result is a new approach related to mobile robots applications and potentially can improve the process of advanced mobile robot application design and professional training of the operators. Training of the mobile robot operators is very difficult task not only because of the complexity of the mobile robot but also because of several factors related to different task execution. The study of the human-robot interactions (HRI) during a real rescue is presented in Casper & Murphy (2003) Bedkowski, Kowalski & Piszczek (2009). Several robotic systems are developed for rescue tasks Wei et al. (2009) Zhang et al. (2009). The research on simulation and training systems for mobile robots is shown in Xuewen et al. (2006). In the field of robotics applied for medical purposes such as surgery the simulation and training environments are developed and described in Schlaefer et al. (2008). The problem of learning via Web is common application used in several cases and as remote experiment system for distance training is discussed in Hong et al. (2007). Virtual labs have been meaningful and very popular in distance education in engineering oriented fields of study, since they allow to perform interesting experiments with the equipment in the labs remotely available through Internet Masar et al. (2004). An advance computer techniques such augmented reality approach applied in training robotics is shown in Kowalski et al. (2008)Bravo & Alberto (2009). The simulation environment used as a testbed for simulation based approach for 17


Introduction
In the chapter the framework for RISE (Risky Intervention and Environmental Surveillance) mobile robot operator training design is presented. The basic idea of the framework is to provide advanced software tools to improve the performance of the mobile robot simulation design that are applicable for operator training. The simulator is the end product of the framework. To achieve realistic rigid body simulation and realistic rendering the NVIDIA PhysX engine and the OGRE (Open Graphic Rendering Engine) are used. The compatibility between the framework software and CAD modeling tools such as SolidWorks was essential, therefore virtual models are exchangeable using COLLADA format. The simulator can be integrated with control panel of the mobile robot. It is possible to simulate the behavior of several types of robots such as caterpillar or wheeled. The improvement presented in this chapter is related to the problem of automatic environment model generation based on autonomous mobile robot observations. The approach related to semantic mapping is used and in consequence semantic simulation engine is implemented. Semantic simulation engine is composed by data registration module, semantic entities identification module and environment model generation module. Presented result is a new approach related to mobile robots applications and potentially can improve the process of advanced mobile robot application design and professional training of the operators. Training of the mobile robot operators is very difficult task not only because of the complexity of the mobile robot but also because of several factors related to different task execution. The study of the human-robot interactions (HRI) during a real rescue is presented in Casper & Murphy (2003)  . Several robotic systems are developed for rescue tasks Wei et al. (2009) Zhang et al. (2009). The research on simulation and training systems for mobile robots is shown in Xuewen et al. (2006). In the field of robotics applied for medical purposes such as surgery the simulation and training environments are developed and described in Schlaefer et al. (2008). The problem of learning via Web is common application used in several cases and as remote experiment system for distance training is discussed in Hong et al. (2007). Virtual labs have been meaningful and very popular in distance education in engineering oriented fields of study, since they allow to perform interesting experiments with the equipment in the labs remotely available through Internet Masar et al. (2004). An advance computer techniques such augmented reality approach applied in training robotics is shown in Kowalski et al. (2008) Bravo & Alberto (2009). The simulation environment used as a testbed for simulation based approach for 17 www.intechopen.com unmanned system command and control is demonstrated in Drewes (2006). The research related to the training and the hypotheses through interactions with unmanned systems using computer mediated gesture recognition is shown in Varcholik et al. (2008), where the presented methodology employs the Nintendo Wii Remote Controller (Wiimote) to retrieve and classify one-and two-handed gestures that are mapped to an unmanned system command set. The Modeling and simulation for the mobile robot operator training tool was presented in Bedkowski, Piszczek, Kowalski & Maslowski (2009). Semantic information Asada & Shirai (1989) extracted from 3D laser data Nüchter et al. (2005) is recent research topic of modern mobile robotics. In Nüchter & Hertzberg (2008) a semantic map for a mobile robot was described as a map that contains, in addition to spatial information about the environment, assignments of mapped features to entities of known classes. In Grau (1997) a model of an indoor scene is implemented as a semantic net. This approach is used in Nüchter, Surmann, Lingemann & Hertzberg (2003) where robot extracts semantic information from 3D models built from a laser scanner. In Cantzler et al. (2002) the location of features is extracted by using a probabilistic technique (RANSAC RANdom SAmple Consensus) Fischler & Bolles (1980). Also the region growing approach Eich et al. (2010) extended from Vaskevicius et al. (2007) by efficiently integrating k-nearest neighbor (KNN) search is able to process unorganized point clouds. The improvement of plane extraction from 3D Data by fusing laser data and vision is shown in Andreasson et al. (2005). The automatic model refinement of 3D scene is introduced in  where the idea of feature extraction (planes) is done also with RANSAC. The semantic map building is related to SLAM problem Oberlander et al.  Thrun et al. (2000) or even registered 3D laser data Magnusson, Andreasson, Nüchter & Lilienthal (2009). Concerning the registration of 3D scans described in Magnusson et al. (2007) Andreasson & Lilienthal (2007) we can find several techniques solving this important issue. The authors of Besl & Mckay (1992) briefly describe ICP algorithm and in Hähnel & Burgard (2002) the probabilistic matching technique is proposed. In Magnusson, Nüchter, Lörken, Lilienthal & Hertzberg (2009) the comparison of ICP and NDT (Normal Distributions Transform) algorithm is shown. In Rusu et al. (2008) the mapping system that acquires 3D object models of man-made indoor environments such as kitchens is shown. The system segments and geometrically reconstructs cabinets with doors, tables, drawers, and shelves, objects that are important for robots retrieving and manipulating objects in these environments. In this paper the application of framework for RISE mobile robot operator training design is presented. The framework is developed to improve the training development using advanced software tools. The semantic simulation engine is proposed to automatic environment model (composed by walls, door, stairs) generation and task execution with mobile robot supervision. As a result, it is demonstrated the task execution of training example of RISE robot, also examples of automatically generated scenes are shown. It should be noticed that the automatic generation of robot environment is still open problem and needs further developments.

Framework overview
The platform is going to support specialized tools for designing physical models, spatial models and others. The standard implemented and used in the platform accepts Collada and PhysX files for designing physical models. For spatial models 3ds format is considered as the best choice. Physical models can be developed using commercially available tools such as Solid Works with Collada plug-in, or Maya with ColladaMaya plug-in. It is also possible to use freeware tool under GNU/GPL license such a Blender with Collada plug-in. Similarly for designing spatial models one can use Autodesk 3ds Max as a non-free product which native file standard is 3ds or free competitor on the market Blender which can handle 3ds file as well. Utilization of presented tools ensures long term support for the designed platform. The framework consists of several software tools for training components development. Figure  1 shows the scheme of the expected end product of the framework -RISE robot operator training. The training is composed from n nodes. Each node consist of robot model, console model, environment model, and task model. Framework provides software tools for model design and integration such as: Robot Builder, Environment Builder, Console Builder, Task Builder, Training Builder. The main concept and the achievement is the limitation of the programmer effort during training design (all activities can be done using the framework advanced GUI applications), therefore the process of models design and integration is improved and the time and potential costs are decreased. The training can be performed simultaneously for several operators. It is possible to construct training classroom for multiple operator training. To demonstrate this, the single training architecture has to be discussed. Figure 2 shows the software architecture of single training. To perform the single training 2 PCs with framework software has to be taken into account. PC (operator) consists of: task manager and task executor programs. These programs are responsible for proper task execution and sending the result to the PC (instructor) where the training process is supervised with training manager program. Instructor can observe the operator behavior using training viewer software that visualizes the simulated scene. For Improvement of RISE Mobile Robot Operator Training Tool www.intechopen.com the multiple training PC (instructor) can use separate training manager and training viewer programs to perform single training on several operator PCs simultaneously.

Model of a robot
Virtual model of robot is the basic element of a simulation. The figure 3 presents main elements of the robot model. Virtual model of robot is composed of the following models: a) physical (represents mechanical properties of a designed robot), b) spatial (represents what is visualized), c) motors/actuators (represents all driven parts of a robot), d) sound (represents a set of sounds connected to associated to the model of motors/actuators), e) sensors (represents a set of sensors mounted onto a robot). Each listed model is described in a single file, therefore full virtual model of robot requires five files and the information about their interaction. Framework offers a tool called Robot Builder for designing a robot. The window of the application is presented in the

Model of an environment
Virtual model of an environment similarly as a models of control panel and robot consists the following elements: physical model, spatial model and sound model. The scheme shown in figure 5 presents main elements of an environment. The environment in the simulator is represented by Virtual Model of Environment. It is accepted that this model will be designed in similar manner as virtual model of robot described previously. The platform supplies such a tool which can support a designer in the process of creating the environment. It will be possible to utilize components from the Components Base. This Base will consists a set of sample models such as buildings, furniture (chairs, closets, tables, etc.) , vehicles, trees, different kinds of basis (i.e. soil, road, grass) (see figure 6). All objects from the Base will have physical and spatial properties. Apart from the physical properties, there will be a spatial model attached and visualized. Also special effect such as particle system, fog, animation and water can be integrated into the scene.

Model of a control panel
Virtual model of a control panel is a second basic element of a simulation. The figure 7 presents the main elements of a control panel. It is assumed that control panel consists of joysticks, buttons, switches and displays. The control panel communicates with a robot via communicating module or directly dependently whether a real or a virtual panel is in use. Control panel can be virtual or real thus the trainee can accustom to real device controlling the virtual robot using real control panel. Described technology is presented in the figure 8.

Task
In order to design Task, the following models are required: robot model, control panel model and environmental model. A training designer has the ability to set the beginning position of  the robot and after that define all possible mission events he concerns as important. Next, he can define time thresholds for the mission. Such e-Task is then exported and can be used to prepare the full multilevel training. The task designing steps are shown in figure 9. For this purpose, framework supplies e-Tasks Generator. The tool supplies the base of events. The Task designer will be able to move about the scene with the robot and environment read in then chose one actor or group of actors and attach an event with specific parameters. The possible events defined for the single Task are: a) robot path to chosen localization, b) time exceeded, c) move an object to chosen localization, d) touch an object, e) damage/disable robot, f) neutralize/destroy an object (e.g. shoot with water gun, put an explosives).

Training
When a designer get through the previous steps the last what is required is to prepare the training. Using previously defined Tasks one can prepare a graph of the training. The sample graph is presented in figure 10. The training starts with the Mission 1 and leads through two or three subsequent missions. In the first case when the condition C 3 is satisfied it means that Mission 2 is non requisite otherwise if the condition C 2 is satisfied the training path will lead through the Mission 2. Under conditions C 1 C 4 C 5 C 8 appropriate missions will be repeated. At the end of each training the summary is presented on the screen and saved to file. The file is imported to the training manager. Figure 11 shows the Training manager GUI. Training manager GUI informs the user about currently task report, also the training report is available for further tasks execution. Training manager can visualize the simulated scene for analyses purposes. Figure 12 shows executed task taskMETRO. The main objective was to transport hazardous object into defined place of neutralization. Fig. 10. Scheme of the training graph. Each node must be one of three: start, end or mission; transition to the next mission is represented by arrow and is triggered under specific condition denoted as C 1 ... C 9 Fig. 11. Training manager GUI.

Framework improvement
Framework for robot operator training design is improved by semantic simulation engine. Semantic simulation engine J. Bedkowski (2011b) is a project that main idea is to improve State of The Art in the area of semantic robotics Asada & Shirai (1989) Nüchter & Hertzberg (2008) Grau (1997) with the focus on practical applications such as robot supervision and control, semantic map building, robot reasoning and robot operator training using augmented reality techniques Kowalski et al. (2011) J. Bedkowski (2011a. It combines semantic map with rigid body simulation to perform supervision of its entities such as robots moving in INDOOR or OUTDOOR environments composed by floor, ceiling, walls, door, tree, etc. Fig. 12. Task taskMETRO execution. The goal is to transport hazardous object to appropriate location.

Semantic simulation engine
Semantic simulation engine is composed of data registration modules, semantic entities identification (data segmentation) modules and semantic simulation module. It provides tools to implement mobile robot simulation based on real data delivered by robot and processed on-line using parallel computation. Semantic entities identification modules can classify several objects in INDOOR and OUTDOOR environments. Data can be delivered by robot observation based on modern sensors such as laser measurement system 3D and RGB-D cameras. Real objects are associated with virtual entities of simulated environment. The concept of semantic simulation is a new idea, and its strength lies on the semantic map integration with mobile robot simulator.

Data registration
Data registration module provides accurate alignment of robot observations. Aligning two-view range images with respect to the reference coordinate system is performed by the ICP (Iterative Closest Points) algorithm. Range images are defined as a model set M and data set D, N m and N d denotes the number of the elements in the respective set. The alignment of these two data sets is solved by minimization with respect to R,t of the following cost function: w ij is assigned 1 if the i-th point of M correspond to the j-th point in D. Otherwise w ij =0. R is a rotation matrix, t is a translation matrix. m i and d i corresponds to the i-th point from model set M and D respectively. The key concept of the standard ICP algorithm can be summarized in two steps Segal et al. (2009): 1) compute correspondences between the two scans (Nearest Neighbor Search). 2) compute a transformation which minimizes distance between corresponding points. Iteratively repeating these two steps should results in convergence to the desired transformation. Because we are violating the assumption of full overlap, we are forced to add a maximum matching threshold d max . This threshold accounts for the fact that some points will not have any correspondence in the second scan. In most implementations of ICP, the choice of d max represents a trade off between convergence and accuracy. A low value results in bad convergence, a large value causes incorrect correspondences to pull the final alignment away from the correct value. Classic ICP is listed as algorithm 1 and the ICP algorithm using CUDA parallel programming is listed as algorithm 2. Parallel programming applied for ICP computation results the average time 300ms for 30 iterations of ICP. The proposed solution is efficient since it performs nearest neighbor search using a bucket data structure (sorted using CUDA primitive) and computes the correlation matrix using parallel CUDA all-prefix-sum instruction. We can consider data registration module as a component executing 6D SLAM -Simultaneous Localization and Mapping, where 6D is related to vector describing robot position in 3D (x,y,z,yaw,pitch,roll). The 6D SLAM algorithm executes three steps: a) ICP alignment of 2 robot observations, b) loop closing, c) map relaxation. The example result of data registration module is shown in figure 13 Algorithm 1 Classic ICP INPUT: Two point clouds A = {a i },B={b i }, an initial transformation T 0 OUTPUT: The correct transformation T, which aligns A and B  Academy building (Brussels, Belgium) using PIONEER 3AT robot equipped with 3DLSN unit (rotated LMS200).

Semantic entities identification
The procedure of prerequisites generation using image processing methods is used. The set of lines obtained by Hough transform from projected 3D points onto 2D OXY plane is used to obtain segmentation of cloud of points, where different walls will have different labels. For each line segment the orthogonal plane orth to plane OXY is computed. The intersection between this two planes is the same line segment. All 3D points which satisfy the condition of distance to plane orth have the same label. In the first step all prerequisites of walls were checked separately -it is data segmentation. To perform the scene interpretation semantic net is used ( figure 14). The feature detection algorithm is composed of cubes generation method, where each cube should contain measured 3D point after segmentation (see figure 15). In the second step of the algorithm wall candidates are chosen. From this set of candidates, based on relationships between them, proper labels are assigned and output model is generated (see figure 16 left). The image processing methods are also used for stairs prerequisites generation. The set of parallel lines (obtained by projected single 3D scan onto OXY plane) in the same short distance between each other is prerequisite of stairs. Possible labels of the nodes are L = {stair}. The relationships between the entities are R = {parallel, above, under}. Figure 16 right shows resulting model of stairs generated from 3D cloud of points. In this spatial model each stair (except first and last one obviously) is in relation r=above&parallel with the previous one and in relation r=under&parallel with next one.

Environment model generation
Model of the environment is automatically generated using semantic net from set of identified semantic entities. between the entities R rs = {connected via joint, position...}, semantic map events E sm = robot simulator events E rs = {movement, collision between two entities started, collision between two entities stopped, collision between two entities continued, broken joint...}. Robot simulator is implemented using NVIDIA PhysX library. The entities from semantic map correspond to actors in PhysX. L sm is transformed into L rs based on spatial model generated based on registered 3D scans. R sm is transformed into R rs . All entities/relations R sm has the same initial location in R rs , obviously the location of each actor/entity may change during simulation, therefore accurate object tracking system is needed. The transformation from E sm to E rs effects that events related to entities from semantic map correspond to the events related to actors representing proper entities. Following events can be noticed during simulation: robot can touch each entity (see figure 17), open/close the door and enter empty space of the door (see figure 18), climb the stairs (see figure 19), damage itself -broken joint between actors in robot arm (see figure 20), brake joint that connects door to the wall.

Conclusion and future work
The chapter presents the framework for RISE mobile robot operator training design with an improvement based on semantic simulation engine. The framework is composed by advanced software tools to improve the performance of the mobile robot simulation design that are applicable for operator training. The simulator is the end product of the framework. The effort to build the simulation software is decreased by developed programs that allow to build several model such as robot model, and environment model. The framework can help in RISE robot training classroom building for several task execution in simultaneous mode, therefore multiple training can be performed. We believe that developed software can help also in multi robotic system design and development by providing advanced techniques for simulation process, therefore the framework can be applicable in several mobile robotics applications. New concept of semantic simulation engine, composed of data registration modules, semantic entities identification modules and semantic simulation module is demonstrated. The implementation of parallel computing applied for 6D SLAM, especially for data registration based on regular grid decomposition is shown. Semantic simulation engine provides tools to implement mobile robot simulation based on real data delivered by robot and processed on-line using parallel computation. Semantic entities identification modules can classify door, walls, floor, ceiling, stairs in indoor environment. Semantic simulation uses NVIDIA PhysX for rigid body simulation. Future work will be related to AI techniques applied for semantic entities identification (furnitures, victims, cars, etc...), localization and tracking methods.
Concerning 6D SLAM, semantic loop closing techniques will be taken into the consideration as promising methods delivering conceptual reasoning. The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations.