Robotic Search and Rescue through In-Pipe Movement

Abstract

So far, we have been engaged in the research and development of various kinds of robots that could be applied to in-pipe inspections that existing methods (screw-drive type, parallel multi-modular type, and articulated wheeled type) cannot perform. In this chapter, we categorized each in-pipe inspection robot depending on its configuration and structure, which includes the design of the propulsive mechanism, steering mechanism, stretching mechanism, and the locations of the wheel and joint axes. On the basis of this classification and from a developer’s point of view, we also discussed the various kinds of robots that we have developed, along with their advantages and disadvantages.

1. Background

The progressive deterioration of aging social infrastructures in urban areas around the world has led to the occurrences of serious accidents one after another. Risks of accidents are mainly hidden, especially in aging bridges, pipelines, ports, and airports, to name a few, all over the country. In particular, water and gas pipe bursts and leaks, explosion, and fire accidents at complexes are growing into a serious problem. A piping accident, for instance, not only cuts the lifeline but also is associated with potential ignition of leaked gas, which necessitates urgent repair and replacement of deteriorated parts. In the process of repairing and replacing pipelines, the most important issues include how to prioritize the repairing place, how to efficiently identify the deteriorated parts in advance, and how to perform the work with minimum necessary cost and personnel.

The common method of inspection practiced up to the present is manual wall thickness measurement from outside of pipes using ultrasonic and magnetic equipment. Practical-wise, such approach consumes time and could be difficult to employ when reaching pipes installed at high places or underground. In addition, some pipelines contain toxic/explosive carbon monoxide (CO) and silane and combustible/flammable gases, which may cause a health hazard to inspection workers. These setbacks suggest the need for cost and effort reduction in maintaining and managing pipelines and in securing safety. Under these circumstances, the recent development of mechanical and electronic technologies, robotic nondestructive inspection technology (NDT) with cameras, and thickness measurement sensors (ultrasonic and magnetic methods) are receiving attention.

So far, a number of methods called smart pipe inspection gage (PIG) have been reported to utilize fluid force in the pipe to push out and move the camera or inspection device. Owing to this passive movement, a route cannot be selected at the branch sections and cannot propel unless the internal pressure of the pipeline is sufficient. In the pipeline business, PIG is not suited to “unpiggable pipelines.” Instead, industrial endoscopes with a camera attached to the tip are widely employed. Nevertheless, as the endoscopes require being pressed in with hands, they are not suitable for inspection in long winding pipelines.

To solve this problem, companies, universities, and research institutions have been working on a large number of self-mobile in-pipe inspection robots. The robot’s movement can be roughly classified into legged type [1], peristaltic type [2], serpentine type [3], and infinite rotation type [4, 5, 6, 7, 8, 9, 10]. The legged-type robot walks in pipes while extending its legs against the inner wall. However, multiple degrees of freedom cause complicated control systems and an increase in the entire robot size. The peristaltic-type robot produces propagating contractive waves found in earthworms and leeches to move as it pushes out its multiple segments in order. Any of the segments always comes in contact with the inner wall of the pipe to support the body; thus, it can move upward at vertical sections. The serpentine type moves in pipes by sending a waveform to an elongated structure consisting of multiple segments as seen in snakes. Unlike conventional planar snake-like robots with passive rollers at their bottom, the directions of the wave and the travel are the same.

Those types are very interesting and important in the sense of scientific investigation on how animal locomotion adapts to tubelike narrow environments. However, the infinite rotation type, such as in drive wheels and crawler mechanisms (belt-driven), was the one substantially studied as it provides a significantly faster and more efficient motion than the abovementioned animal locomotion schemes despite its simple structure and low cost. Thus, this is expected to contribute in checking buildings or infrastructures before and after disasters, especially in entering into a collapsed building through pipes to search for human casualties.

2. Essential mechanisms for in-pipe inspection robots

For each of the in-pipe robots described above, the body structure consists of three essential components: (1) a propulsive mechanism for moving forward and backward, (2) a steering mechanism for turning at bent and branch sections, and (3) an extending mechanism for avoiding slipping and falling at vertical sections. The propulsive and steering mechanisms are very common in the mobile robot field, whereas the extending mechanism is specific to in-pipe mobile robotic applications. A general in-pipe mobile mechanism is shown in Figure 1.

Figure 1.

General in-pipe mobile mechanism.

We believe that a key point in designing a small and highly adaptable in-pipe robot is its functional complex. If three components (propulsive, steering, and extending) are installed separately, then an increase in size is inevitable. In a sense, the legged-type, peristaltic-type, and serpentine-type locomotion can be regarded as the common principle because the propulsive mechanism works simultaneously as an extension and as a steering component. Moreover, the radial size of the snake and peristaltic robots may be reduced because the robot body itself generates a propulsive force by shifting its body shape, which suggests the nonnecessity of additional motion mechanisms. As mentioned above, animallike locomotion in pipes is slower than wheel-driven locomotion. Therefore, it is important to develop a scheme that combines the advantage of the wheeled mechanism (faster movement) and the snake and peristaltic mechanism (small size).

Structures with several components generally conflict with downsizing. Nonetheless, this issue is solved to some extent by combining multiple functions in one part of the robot (a functional complex). In this study, we tackle two approaches for the functional complex, namely, a differential mechanism and arranging multiple degrees of freedom (DoFs) on a common axis. Conceptually, the differential mechanism approach is applied to a steering mechanism of a screw-drive robot [11] and a step adaptation mechanism of a three-modular robot, whereas the idea of arranging multiple DoFs on a common axis is applied to an articulated wheeled robot.

3. Functional complex by a differential mechanism

An overview of the screw-drive-type in-pipe robot that we first introduced [12] is illustrated in Figure 2. This in-pipe robot consists of a front rotator that generates thrust and a rear stator that supports the reaction of the rotator. The rotator has several tilted passive wheels arranged on its circumference and can move forward and backward while tracing a spiral curve.

Figure 2.

Screw-drive in-pipe mobile robot for 5-in pipelines [12].

By arranging a motor and a gear reduction along the pipe axis, the output drive axis can be connected directly to the rotator without changing the direction of rotation through a transmission mechanism, such as a miter. This implies that the screw-drive type can be miniaturized easily, although it would face difficulty passing through T-branches with only a drive mechanism. To solve this challenge, an active steering joint with a simple miter-geared differential mechanism is installed between the rotator and the stator. The rotator can be swung by only a single actuator in both the longitudinal and lateral directions depending on the in-pipe constraint condition.

Accordingly, the robot can be steered by only a single actuator in both the longitudinal and lateral directions depending on the constraint condition in pipes. Owing to friction, the passive wheels of the middle unit maintain their position during rotation of the steering motor, and the front unit can be swung. Nonetheless, the robot can change its direction of navigation in pipes where steering movement is constrained by the inner wall, e.g., in straight sections. Driven by the orbiting miter gear, the entire middle unit rotates around the central axis; simultaneously, the wheels of the middle unit rotate in the circumferential direction as casters (Figures 3 and 4).

Figure 3.

Schematic of the screw-drive in-pipe mobile robot.

Figure 4.

Steering mode and rolling mode using a miter-geared differential mechanism.

Meanwhile, we also developed an in-pipe robot called multi-module parallel arrangement type [13], which has a structure in which multiple belt-driven crawler mechanisms are arranged parallel to the pipe axis and on the circumference. Although it tends to increase in size, a large traction force can be generated by coupling each propulsion force, and the orientation can be changed omnidirectionally by adjusting the speed balance among each module. In this study, we propose a new mechanism called an underactuated parallelogram crawler. We confirm its ability to cope with changes in internal pipe diameter without necessarily an increase in the number of motors (Figure 5).

Figure 5.

Three-module parallel arrangement type in-pipe mobile robot for 8-in pipelines [13].

To achieve differential motion, a pair of spur gears is mounted on the front flipper of each parallelogram crawler module. With the motion of the front flipper constrained by gravity and the pantograph-spring combining expansion mechanism in a normal driving mode, the motor torque is transmitted to the front driving pulley (Figure 6a). The front flipper is lifted up once the motion of the robot is stopped (Figure 6b). An additional timing belt in these modes enables the simultaneous rotation of the front and rear flippers. To avoid an endless rotation of the flippers, stopper pins are attached to stop at 30°.

Figure 6.

Driving mode and parallelogram mode using a spur-geared differential mechanism. (a) Driving mode and, (b) Parallelogram mode (arm-lifting).

4. Functional complex by arranging multiple DoFs on a common axis

On one hand, the screw-drive type can be easily downsized but with an associated limit of travel to pipelines without any junction. On the other hand, the multi-module parallel arrangement type can generate large propulsion by coupling each force but tends to increase in diameter. We thought that the differential mechanism could be one solution for downsizing; however, it leads to the complexity of the whole robot mechanism and eventually causes an increase in size and weight.

As introduced in the earlier sections, the key point for downsizing is combining the three components (propulsive, steering, and extending) in a common component. To achieve this compact design, we have been working on a multi-link-articulated wheeled-type in-pipe robot whose wheel shaft (as propulsive) and joint (as steering and extending) are all arranged on the same axis. This configuration leads to a drastic miniaturization to 3–4 in. in the inner diameter of pipes and is even adaptable to winding pipelines and T-branch [14, 15, 16, 17, 18, 19].

An overview of the multi-link-articulated wheeled-type in-pipe robot [20] is shown in Figure 7. This robot consists of four links and joints connecting them and moves back and forth using actively rotatable omni wheels installed on each joint axis. A torsional coil spring mounted in each joint allows the robot to form a zigzag shape, making the robot move up in vertical pipes by pressing the omni wheels to the inner wall of pipes. When the robot enters into a bent pipe, the joints can be opened and closed passively according to the shape of the curved section, thus making the robot easily pass through winding pipelines.

Figure 7.

A multi-link-articulated wheeled-type in-pipe robot named AIRo-2.2 [20].

Another major feature of this robot is that the rotational axes of all joints are parallel to each other (Figure 8). As the positions of all joints move only on the same single plane, the robot cannot pass through bent pipes if the bending direction of the joints does not match the pathway direction of the pipes. However, this is not a disadvantage to the robot. For example, in a situation where the inner wall of pipelines has obstacles, such as holes and dents, the robot can avoid them by displacing the trajectory of the wheels and the obstacle.

Figure 8.

Two robot orientations depending on the passability to the bent pipe.

To align the bending direction of the robot joints and the pathway direction of the pipe, we proposed a method of changing the robot’s orientation around the pipe axis by rolling spherical wheels [21, 22, 23] installed at its head and tail ends (Figure 9). The spherical wheel rotates freely in the direction of the robot movement; thus, it does not disturb the movement of the omni wheels. Similarly, the omni wheel has several small free rollers arranged on its circumference; thus, they do not interfere with the rolling motion of the robot by the spherical wheels.

Figure 9.

Experiment in a 15-m pipeline with 12 bent and 1 T-branch pipes.

We confirmed the robot’s ability to pass through a 15-m-length and 4-in-diameter vinyl chloride pipeline, including vertical sections with 12 bent and 1 T-branch pipes, as shown in Figure 10. Here, the operator operates the robot using a gamepad while only watching the camera images. It took approximately 6 min for the robot to reach the end of the pipeline and back to the entrance.

Figure 10.

Active and passive degrees of freedom of the AIRo-2.2 [20].

We have been in pursuit of a year-by-year improvement of the robot. The multi-link articulated wheeled-type in-pipe robots that we have been developing so far and their extended versions with some modifications (called AIRo-series) are displayed in Figure 11.

Figure 11.

Multi-link-articulated wheeled-type in-pipe robots in the AIRo-series developed by the authors.

AIRo-2.2 mini is specially designed for cleaning inside flexible ducts. As it does not have to generate a large traction force, the robot is only composed of two links. By rotating the head brush, the inner surface of the duct can be cleaned. AIRo-3.0 [24, 25] has the same multi-link structure as the AIRo-2.0 series. However, each joint has a differential mechanism to generate two movements: moving back and forth and twisting the body. AIRo-2.4 is a downsized version; from a 4-in diameter, its size was shrunk to 3 in. AIRo-2.3s is the latest version of the in-pipe robot and is equipped with an active joint with both angle and torque control systems (Figure 11).

5. Functional complex of camera and operation assistant systems

The robots that we developed do not only assume a straight motion but also roll around the pipe axis through the spherical wheels mounted on the head and tail. However, operators need to select the direction in which the robot orientation should be rotated from only camera images, which normally requires a practiced skill. In addition, this reduces difficulty in operating the robot as it can detect the orientation relative to the pathway direction of the bent pipe by itself.

Regardless of the design, in-pipe inspection robots need at least one illuminator and one camera to view its environment. For this matter, we proposed an anisotropic shadow-based operation assistant method using only a single LED and a camera (Figure 12) [26]. By displacing the position of the LED relative to that of the camera, a crescent-shaped shadow appears in the images captured in a bent pipe as illustrated in Figure 13 [27, 28]. The size, position, and orientation of the shadow depend on the robot’s orientation around the pipe axis, and it disappears in a certain robot’s orientation (anisotropic shadow). Generally, as for shadow-based navigation systems, shadow disappearance should be avoided to prevent the robot from losing its way. As exclusion, AIRo-2.2 is designed so that it could adapt to a bent pipe without any control when the robot’s orientation and the pathway direction of the bent pipe are aligned. By aligning those two specific orientations, the robot can select the optimal orientation to adapt to the bent sections.

Figure 12.

AIRo-2.2 with a shadow-based operation assistant system [26].

Figure 13.

Principle of crescent shadow appearance in camera images.

Even though the shadow that appears in the bent pipe can be clearly extracted by binarization alone after the threshold is tuned, the shadow that appears in the straight pipe is also detected in straight sections. Therefore, the robot may mistakenly recognize that it is in a bent pipe even if it is in a straight pipe. If the straight pipe and the bent pipe can be distinguished beforehand, then the operation assistant system used to adjust the robot roll orientation can be executed only in bent sections.

In our research, a monochrome image histogram (the relationship between the number of pixels and the brightness value of a camera image) is used to automatically distinguish a bending part and a straight pipe part (Figure 14). In straight pipes, the LED brightens the inner pipe wall around the robot. However, the light does not reach the far-off portion of the pipe (the center of the camera image), consequently leading to an even distribution in the luminance values from low to high on the image histogram (Figure 14a).

Figure 14.

Image histograms captured in a straight section (a) and a bent section (b).

In bent sections, as the LED light is brightly reflected in many areas (Figure 14b) in the camera images (the distance between the LED and the wall of the pipe is close), the luminance value is concentrated at the high brightness on the image histogram (Figure 15). This difference can be distinguished by the calculation of the variance of the image histogram. Apparently, the value of variance increases in bent pipes, and it decreases in straight pipes.

Figure 15.

Variance value depending on the distance between the robot and the entrance of the bent pipe.

We set the threshold of variance to 1.4 million and found that the robot recognizes the bent pipe when it exceeded about D = 0.4 m. There is a portion where the variance value increases rapidly near D = 1.0 m in the graph mainly because of the influence of reflected light from the step part of the connecting pipe. Although the robot incorrectly recognizes it as a bent pipe in a straight section, traveling is not inhibited even if roll rotation is performed. At D = 0 m, the variance value decreases because the shadow image of the bent pipe is detected.

We confirmed from the experiments that the robot with the shadow-based operation assistant system could travel by approximately 7.5 m in length, including three vertical pipes and seven bent pipes. At this time, the operator used only one button to make the robot move forward or stop (Figure 16).

Figure 16.

Experimental result of the robot with the shadow-based operation assistant system.

6. Conclusions

Herein, we introduced our researched and developed in-pipe inspection robots and their shadow-based operation assistant system (orientation adjustment). The key point to designing such robot for various pipelines available is downsizing and simplification by the functional complex. Our approach for this functional complex included a differential mechanism, arrangement of multiple DoFs on a common axis, and usage of a camera not only for inspection but also for the operation assistant system. At the present stage, we are testing such approach in simulated pipelines installed in our laboratory. Nonetheless, we will continue to improve the development while collaborating with the user company and are planning to carry out experiments on the actual site.

Acknowledgments

This work was supported by JSPS KAKENHI Grant Number 17K14632, Gesuido Academic Incubation to Advanced Project (GAIA Project) of the Ministry of Land, Infrastructure, Transport and Tourism and many other cooperative companies as commissioned research. We would like to thank all those who supported.

Conflict of interest

There is no conflict of interest regarding the publication of this article.