Automatic Visual Inspection and Condition-Based Maintenance for Catenary

Abstract

Defects on catenary components are a major part of device faults as a result of a much higher tension on high-speed catenary, such as looseness of bolts, component broken, and component missing. Traditional inspection on catenary components has to be performed only at night with human eyes. Not only the inspection speed is very slow but also the inspection results are not reliable, as a result of the poor lighting environment and long-time working tiredness. In this chapter, we present an automatic visual inspection system for checking the status of components on catenary. A dedicated designed camera system is mounted on an inspection car, which covers almost all the components to be checked and gives great details of each component. Considering the great data storm at each catenary post, high-performance servers with GPU acceleration are used, and technologies of multi-thread and parallel computing are exploited. Furthermore, an intelligent analysis framework is proposed, which uses structural analysis to localize each component in the image and perform automatic detection based on different features such as geometry, texture, and logic rules. The system has been successfully used in China’s high-speed railways, which shows great advantages in the catenary inspection application.

1. Introduction

Compared with traditional defects on stagger, arc, contact force, etc., defects on catenary components become more and more important on high-speed railways [1]. Defects on catenary components are a major part of device faults as a result of a much higher tension on high-speed catenary, such as looseness of bolts, component broken, and component missing. Figures 1 and 2 illustrate some typical defects on catenary components.

Figure 1.

Component missing.

Figure 2.

Looseness of bolts.

Traditional inspection for catenary components on high-speed railways has to be performed only at night with human eyes (Figure 3). Not only the inspection speed is very slow but also the inspection results are not reliable, as a result of the poor lighting environment and long-time working tiredness.

Figure 3.

Traditional inspection.

Due to the great advances in machine vision technologies [2, 3], some visual inspections or visual measurement systems have been developed in the field of infrastructure inspection on track, catenary, etc. [4, 5, 6, 7, 8]. However, the development of a visual inspection system for catenary components is not an easy task. First, catenary is composed of many types of components, and these components are mounted in a large area and vary greatly in size and location, which brings great difficulties in the design of camera system. Second, most components concentrate at catenary posts, which means a great data storm for the image capturing, processing, and saving. Finally, the automatic data analysis is also of great challenge, as a result of the variety and large number of defect types.

In this paper, we present an automatic visual inspection system for checking the status of components on catenary. A dedicated designed camera system is mounted on an inspection car, which covers almost all the components to be checked with an image resolution up to 0.5 mm/pixel. A post detection module is integrated in order to trigger the cameras and lighting device at a proper location. Technologies of GPU acceleration, multi-thread and parallel computing, are exploited in the image acquisition, image compression, and data storage. Furthermore, an intelligent analysis framework is proposed to perform the automatic analysis of image data.

The rest of paper is organized as follows. Section 2 gives an overview of the system architecture. Section 3 describes the methodology of each system part in detail. Section 4 reports some application results of the system. Then we draw the conclusions in the last section.

2. System architecture

Catenary is composed of contact wire, cantilever, additional wire, hanging post (in tunnels), etc. (Figure 4). As a result, different kinds of catenary components are mounted in a large area and vary greatly in size and location.

Figure 4.

Structure of catenary.

Considering that most components concentrate at catenary posts, we use groups of super-high-resolution area-scan cameras to capture image data of each region at a specific distance relative to each catenary post. A post detection module is applied to generate an accurate trigger signal for cameras and lighting device. Finally, image data from different cameras are paralleled processed and saved in a common data sever.

The system architecture is illustrated in Figure 5.

Figure 5.

System architecture.

3. Methodology

3.1 Camera system

Catenary components to be checked are mounted in a large area. It covers [−3500, 3500 mm] in the horizontal direction and [4800, 8100 mm] in the vertical direction relative to the track center. High-resolution image data are needed in order to perform image detection of each component.

In our system, the candidate detection area is divided into several detection regions. A dedicated designed camera system is mounted on an inspection car, which is composed of many area-scan cameras with different directions and fields of view. Super-high-resolution cameras are used which have a resolution up to 29 megapixel.

Table 1 gives details of the camera system. The camera system is carefully designed so that it covers almost all the components to be checked with an image resolution up to 0.5 mm/pixel, which gives great details of each component.

Detection modules	Camera resolution (megapixel)	Number of cameras	View directions
Cantilever	29	14	Forward and backward
Contact wire	25	2	Left and right
Additional wire	16	4	Forward and backward
Hanging post	16	4	Forward and backward
Catenary video	4	1	Forward

Table 1.

Camera system.

In order to get a brighter light and better image quality at night, groups of strobe light are applied in the system.

Figure 6 illustrates the design for part of the camera system.

Figure 6.

Camera system.

3.2 Post detection

The work of the system relies on an accurate generation of trigger signals at a specific distance relative to each catenary post. As a result, a reliable post detection module is very important in the system. The post detection module should be adaptive to different detection environments and types of catenary.

In order to achieve a reliable detection, several laser distance sensors are used which are mounted upward. The range of distance between the sensors and steady arms is utilized as the criterion in the data analysis of post detection. Measurement data are captured and analyzed in real time. The trigger signals are generated and distributed to each camera and strobe light.

The principle of the post detection module is illustrated in Figure 7.

Figure 7.

Post detection.

Laser sensors in the 1# group are used when the inspection car runs forward, and those in the 2# group are used when the inspection car runs backward. Finally, an accurate photographic distance at each post can be achieved for each camera as expected.

3.3 Image acquisition, processing, and saving

In the camera system, up to 25 super-high-resolution area-scan cameras are used to capture image data of each catenary component. In the maximum inspection speed of 160 km/h, the overall original image data is up to 10 Gb/s! It is a great challenge for the image acquisition, compression, and data storage.

Considering the great data storm at each post, high-performance servers are used in the system. The architecture of multi-thread processing is shown in Figure 8. The tasks of image acquisition, image compression, and image saving are distributed in different threads for each camera and are performed based on the technologies of parallel computing and GPU acceleration.

Figure 8.

Multi-thread processing.

Although the huge amount of image data are captured and compressed on several acquisition platforms, the final compressed JPEG images are saved in a common data server, as illustrated in Figure 9.

Figure 9.

Data storage.

There are labels in the name of each image file, which indicates the camera, post number, detection region, and frame number. The image data are well aligned, and all the image data for the same catenary post are saved in a common file folder, which is helpful for the data management and analysis.

3.4 Image detection

Technologies of visual inspection and image detection have already been used in the detection of track components such as fasteners and track slabs [7, 8]. However, it is quite different for the detection of catenary components. Not only the number of components and defect types to be checked is much larger, but also catenary components vary greatly in size and location, which brings more difficulties in the image detection.

In this paper, we proposed an intelligent analysis framework. Firstly, as shown in Figures 10 and 11, structural analysis is applied in order to localize each component in the image. Technologies of matching are exploited here, based on pair-wise constraints [9] between the main catenary components such as steady arm, registration arm, cantilever tube, etc.

Figure 10.

Structural analysis.

Figure 11.

Structural analysis.

Secondly, according to the analysis on the characteristics of different components and typical defects, elaborate detection algorithms are separately developed for each component, based on different features such as geometry, texture, and logic rules (Figures 12–15).

Figure 12.

Detection based on geometry feature.

Figure 13.

Detection based on geometry feature.

Figure 14.

Detection based on texture feature.

Figure 15.

Detection based on texture feature.

Finally, only a few of candidate defects are automatically selected and need to be manually rechecked by the system operator, which greatly improves the inspection efficiency (Figure 16).

Figure 16.

Manual rechecking for candidate defects.

4. Applications

The proposed system has been installed on several catenary inspection cars which have a maximum inspection speed of 160 km/h. Figure 17 shows the inspection equipment on one of the inspection car.

Figure 17.

Equipment on the inspection car.

The graphic user interface is shown in Figure 18. All the camera control and image display of different cameras can be done in the common software, and we can also enlarge a specific part of image to see much detail of any component or region that we are interested in.

Figure 18.

Graphic user interface.

Images of different inspection regions are shown in Figures 19–22.

Figure 19.

The whole cantilever region.

Figure 20.

The contact wire region.

Figure 21.

The additional wire region.

Figure 22.

The hanging post region.

The camera system is designed with an image resolution up to 0.5 mm/pixel. Figures 23–25 show details of some detection regions which are enlarged from the original images. As shown below, the images of catenary components are clear enough to perform the subsequent detection.

Figure 23.

Enlarged image.

Figure 24.

Enlarged image.

Figure 25.

Enlarged image.

As for the post detection, there are almost no missing detections in the system test. The detection performance is reliable between railway stations. However, a few of false detections exist in some stations, as a result of the complex structure of crossing wires and electric connection wires at stations.

Comparison between images from two inspection runs at the same post is shown in Figure 26. We can see the two images are almost the same, which means the accurate post detection and trigger signal generation have been achieved.

Figure 26.

Images from two inspection runs.

Some defects that have been automatically detected by the system are shown in Figures 27–30.

Figure 27.

Component missing.

Figure 28.

Looseness of bolts.

Figure 29.

Relaxation of tightening wires.

Figure 30.

Broken of insulators.

5. Conclusions

A visual inspection system for catenary components is proposed in this chapter. An image resolution of up to 0.5 mm/pixel is achieved, which covers almost all the components to be checked. An intelligent analysis framework is also proposed, which is used to perform automatic image detection for component defects.

In the past, the component status of catenary is inspected periodically; workers have to go on the railway lines and perform the inspection by human eyes. The infrastructure maintenance will be arranged based on the inspection results. The in-time catenary maintenance is no easy as a result of the poor inspection efficiency. Thanks to the inspection technologies and system proposed in this chapter, the inspection efficiency for component status of catenary is greatly improved, and it is of great use for the catenary maintenance. The status of catenary is well managed based on the automatic visual inspection, and the manner of catenary maintenance is improved from periodically maintenance to condition-based maintenance.

Acknowledgments

This research was partly supported by the National Key Basic Research Program of China (973 Program) (Grant No. 2013CB329406), China Railway Corporation Program on Scientific Research and Development Project (Grant No. P2018G002), and China Academy of Railway Sciences Research Program (Grant No. 2014YJ052).