Monitoring the Condition of Railway Tracks Using a Convolutional Neural Network

Abstract

Condition monitoring of railway tracks is effective for the sake of an increase in the safety of regional railways. This study proposes a new method for automatically classifying the type and degradation level of track fault using a convolutional neural network (CNN), which is a machine learning method, by imaging car body acceleration on a time-frequency plane by continuous wavelet transform. As a result of applying this method to the data measured in regional railways, it was possible to classify and extract the sections that need repair according to the degree of deterioration of the tracks, and to identify the track fault in those sections.

1. Introduction

Maintenance of railway tracks is essential for the safe operation of trains. Railway operators conduct track inspections using track geometry cars and track maintenance crews. However, regional railway operators, who carry fewer passengers, often lack the personnel and funds to conduct adequate track inspections. The monitoring of railway track geometry from an in-service vehicle has become increasingly attractive over the past decade [1].

To address this problem, a system that can monitor the track condition inexpensively and frequently using a device incorporating sensors and a global navigation satellite system (GNSS) unit, which is installed on in-service trains, has been developed [2, 3]. The system calculates root mean square (RMS) values from the vertical acceleration, lateral acceleration, and roll angular velocity of the car body. To select sites for repair, we adopt the method of prioritizing sites with the highest numerical values.

The acceleration RMS is closely related to the general health of the track [4]. In Ref. [5], RMS values are used to identify track irregularities for longitudinal level, alignment, cross-level. However, monitoring based on RMS values alone is not sufficient. Without frequency information, it is difficult to identify the type of track fault. Furthermore, since the amount of data generated by constant measurement is enormous, it is necessary to automate the analysis in order to monitor and predict the track condition efficiently.

In this study, we propose a method to classify the types of track faults automatically by means of machine learning, using a CNN trained on images created via a CWT from the vibration acceleration on the time-frequency plane. A continuous wavelet transform (CWT) is a transformation technique that emphasizes certain portions of the waveform by suppressing other portions as it proceeds by multiplying a target waveform using a mother wavelet [6]. A convolutional neural network (CNN) is a class of deep neural networks. It is widely used for image recognition.

To verify the effectiveness of the algorithm we developed, we first describe the results of simulating the vibration of a car body when passing over a faulty track. Next, we describe the results of diagnosing track faults from the vertical vibration acceleration data of a car body measured by a regional railway.

2. Literature review of track condition monitoring using machine learning techniques

It should be necessary for railway operators to control track irregularity, such as vertical rail profiles, lateral alignment, gauge, cross-level, twist (depicted in Figure 1) properly. Track irregularities cause vehicle vibrations that degrade the rider’s comfort and increase the risk of derailments. Track irregularities are strongly correlated with vehicle vibrations. Thus, it can be possible to estimate general trends of the track condition by analyzing vehicle vibrations.

Figure 1.

Track structure and irregularities.

Although track geometry measurement systems using in-service vehicles are becoming increasingly attractive around the world [2, 7, 8, 9], the repeated checking of the same track provides the information regarding track geometry degradation, which can be fed back to the track maintenance section for taking essential actions. The use of vehicle responses in the track geometry assessment process allows identifying of critical defects, which could not have been identified from geometry parameters, and thus, improve the maintenance operations.

Tsunashima et al. proposed techniques of condition monitoring of railway tracks based on time-frequency analysis [10]. They compared the performance of Hilbert-Huang transforms (HHT) and CWT for identifying track faults from car body vibration. It is shown that the feature of track fault can be identified in time-frequency plane.

Tsunashima proposed a classifier based on a machine learning technique for identifying track faults automatically from measured car body vibration [5]. It is shown that the degradation of track can be classified in the feature space consisting of car body vibration RMS.

Faghih-Roohi et al. proposed a deep convolutional neural network for the analysis of image data for the detection of rail surface defects [11]. They explored the efficiency of the proposed deep convolutional neural network for the detection and classification of rail surface defects.

Zheng et al. proposed a multi-object detection method based on a deep convolutional neural network that can achieve non-destructive detection of rail surface and fastener defects [12]. A defect detection model based on Mask R-CNN and ResNet framework was utilized to detect the surface defects.

Jin et al. proposed a machine learning framework based on wavelet scattering networks and neural networks for identifying railhead defects [13].

Alvarenga et al. proposed an embedded system for online detection and location of rails defects based on eddy current [14]. They proposed a new method to interpret eddy current signals by analyzing their wavelet transforms through a convolutional neural network.

3. Effect of track faults on time-frequency plane

3.1 Overview of the simulation

When a train runs on a track, vibrations that correspond to the track geometry are generated [15, 16]. Therefore, in this study, to verify the relationship between the type of track fault and the car body vibration acceleration, and to evaluate the effectiveness of time-frequency analysis in detecting track faults, we simulated the occurrence of track faults, calculated the vertical vibration acceleration of the car body, and then applied a CWT, a method of time-frequency analysis, to the results.

3.2 Continuous wavelet transform (CWT)

A CWT is a method that simultaneously detects the frequency and time characteristics of an unsteady signal, by comparing the original signal with dilated and translated versions of a small wavelike function called the mother wavelet. Using this method, it is possible to view the amplitude and frequency information of the vibration acceleration as an image. In this study, we used the Morlet wavelet, which offers a relatively good balance between localization of time and frequency, as the mother wavelet [17] (see Appendix A).

This technique is well suited for analyzing unsteady signals, such as $x\left(t\right)$ those that exhibit sudden variation, and is defined as follows:

$$W_{\psi }\left(a,b\right)={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{a}}{\psi }^{\ast }\left(\frac{t-b}{a}\right)x\left(t\right)\mathit{dt},$$E1

where, variables $a$ and $b$ correspond to the dilatation and location parameters, respectively, they translate the mother wavelet $\psi \left(t\right)$ by a time shift $b$ in time, and by $1/a$ in frequency. indicates the complex conjugate of $\psi$.

3.3 Vehicle model used in the simulation

The vehicle model used in the simulation is shown in Figure 2 [10]. The vehicle model consists of a total of seven rigid bodies: one car body, two bogies, and four wheelsets. The car body and bogie were assigned two degrees of freedom (DOF) for bounce and pitch, and the wheelset was assigned one DOF for the bounce. The vehicle’s parameters were obtained from measurement data from a regional railway vehicle equipped with an onboard sensing device.

Figure 2.

Vehicle model [10].

3.4 Simulation conditions

In the simulation, the vehicle model was run at 60 [km/h] for 500 [m], and the results were output for the section between 100 [m] and 350 [m]. We set rail joint faults (joint depressions) at 4 points; otherwise, the track was assumed to be straight. To set the rail joint faults, we used the function model shown in Figure 3 [18].

Figure 3.

Track fault model.

The geometry of the modeled track are represented by

$$y={\mathit{Ae}}^{-\left(\frac{1}{2}\right){\left(\frac{x}{k}\right)}^2},$$E2

and

$$y_2=\frac{1}{2}{\left(\frac{x}{k}\right)}^2-A.$$E3

The track geometry used in the simulation is shown in Figure 4. At the 150 [m] and 200 [m] points, we set depths of $A=15\left[\mathrm{mm}\right]$ and $30\left[\mathrm{mm}\right]$, respectively, to simulate joint depressions, which are depressions in the top surface of the railhead that occur at track joints.

Figure 4.

Track geometry with different faults.

In both cases, we set the depression length $k=83.5\left[\mathrm{mm}\right]$. For comparison, at the 250 [m] and 300 [m] points, we set a gentler dip in track geometry by setting depths of $A=15\left[\mathrm{mm}\right]$ and $30\left[\mathrm{mm}\right]$, respectively, but with length $k=1670\left[\mathrm{mm}\right]$. The values of $A$ and $k$ were determined with reference to generally occurring track displacement. The car body vibration acceleration was assumed to occur directly above the center of the front bogie of the vehicle model. The simulation was performed at a sampling frequency of 200 [Hz].

3.5 Simulation results

The simulated vertical vibration acceleration of the car body is shown in Figure 5a. The figure shows that characteristic vibrations corresponding to the track geometry are generated at the points where the track faults were set. Figure 5b shows the result of the CWT of the simulated vertical vibration acceleration. The color bar indicates the magnitude of the amplitude in the time-frequency plane.

Figure 5.

Simulated car body vertical acceleration and its CWT image.

At 150 [m] and 200 [m], the points where the joint depressions were simulated, vibrations in the frequency band of 15–30 [Hz] were detected due to the impulse-like track geometry, and variations depending on depth $A$ can be seen in the CWT images. In addition, at 250 [m] and 300 [m], the points where gentler dips in the track geometry were set, vibrations in the frequency band of 0–5 [Hz] were detected, and variations depending on depth $A$ can be seen in the CWT images. These results demonstrate that the CWT images are effective for identifying track faults, since the features of the CWT images change markedly depending on the type and level of degradation.

4. Monitoring the condition of railway track using a convolutional neural network

4.1 Track condition monitoring system

Figure 6 shows the track condition monitoring system developed and applied for regional railway lines in Japan [2].

Figure 6.

Track condition monitoring system [2].

Accelerometers and rate gyros in the onboard sensing device measure the car body vibration. A GNSS receiver detects the location and speed of the train. Collected data are transmitted to the data server in the monitoring center continuously via a mobile phone network.

The diagnostic software analyses the collected data and results are fed back to the railway operators through online channels via tablet computers. The diagnostic results are used to facilitate the maintenance work of railway operators.

Convolutional neural networks are a method used in the field of machine learning called deep learning and are particularly suitable for image recognition. In this study, we examined the effectiveness of classifying longitudinal level irregularities and joint depressions automatically, using a diagnostic algorithm, we constructed based on a convolutional neural network trained on CWT images generated from vertical vibration acceleration data from a car body. The diagnostic procedure is shown in Figure 7.

Figure 7.

Diagnostic procedure.

5. Identification of the condition of railway track using vertical vibration acceleration data measured from an actual car body

5.1 Overview of diagnosis

The car body’s vertical acceleration with track faults was collected in a regional railway line using the track condition monitoring system. The input data for the classifier consists of vertical vibration acceleration measurements from an onboard sensing device in a car body, which are then converted into images using a CWT. Figure 8 shows an example of converting the measurements into a CWT image.

Figure 8.

Measured car-body vertical acceleration and its CWT image.

The vibration characteristics of the joint depression at the distance of 25.82 [km] appear in the 10–30 [Hz] frequency range. The vibration characteristics of the longitudinal level irregularities around 25.95 [km] appear in the 0–5 [Hz] frequency range.

5.2 Images used for training and evaluation

In this study, we investigated the following three types of diagnoses:

• Classification of images into three types: longitudinal level irregularity, joint depression, and normal.

• Classification of the degradation level of longitudinal level irregularity into normal, medium, and large.

• Classification of the degradation level of joint depression into normal, medium, and large.

Examples of images used for each task are shown in Figures 9–11. The images were created with an aspect ratio of 1:1 (150 × 150 pixels), which is optimal for training.

Figure 9.

CWT images of faulty track.

Figure 10.

CWT images of the different levels of a degraded track (track irregularity).

Figure 11.

CWT images of the different levels of a degraded track (joint depression).

For diagnosing the level of degradation of longitudinal level irregularities, in cases where the one-side amplitude of the vibration acceleration was normal, images of car body acceleration of $0–0.5\left[\mathrm{m}/{\mathrm{s}}^2\right]$ were used. To diagnose medium degradation, images of $0.8–1.2\left[\mathrm{m}/{\mathrm{s}}^2\right]$ were used, and to diagnose large degradation, images of $1.5\left[\mathrm{m}/{\mathrm{s}}^2\right]$ or greater were used.

For diagnosing the level of degradation of joint depressions, in cases where the one-side amplitude of the vibration acceleration was normal, images of body acceleration of 0 to were used. To diagnose medium degradation, images of $2.5–3.5\left[\mathrm{m}/{\mathrm{s}}^2\right]$ were used, and to diagnose large degradation, images of $4.5\left[\mathrm{m}/{\mathrm{s}}^2\right]$ or greater were used.

5.3 Identification of longitudinal level irregularities and joint depressions

5.3.1 Configuration of the trained convolutional neural network

We prepared a total of 300 images: 100 normal images, 100 images with a longitudinal level irregularity, and 100 images with a joint depression. We set aside 80% of the images for training and 20% for evaluation as shown in Figure 9.

Figure 12 shows the configuration of the trained convolutional neural network (see Appendix B). In the figure, the name of the process and the size (vertical × horizontal × channels) before processing are indicated above each layer, and the size after processing is indicated below the layer.

Figure 12.

Network configuration.

The Convolution layer applies the convolution operation to the image, representing it in matrix form; the Max pooling layer performs information compression; the Affine layer combines information from different layers, and the Output layer outputs a set of probabilities indicating how well the image matches the three types of training image data. The number of training sessions was set to 50.

5.3.2 Diagnosis results

Figure 13 shows the results of using images for evaluation to discriminate longitudinal level irregularity track faults versus joint depression track faults versus normal track. The overall accuracy rate was 98.3%, demonstrating that convolutional neural networks are effective for the classification of track faults.

Figure 13.

Detection accuracy for the type of track fault.

5.4 Identification of the degradation level of longitudinal level irregularities

5.4.1 Configuration of the trained convolutional neural network

In order to classify the degradation level of longitudinal level irregularities into three types: normal, medium, and large, we prepared a total of 300 images: 100 normal, 100 medium, and 100 large. We set aside 80% of the images for training and 20% for evaluation. The network configuration and the number of training sessions were the same as in Section 5.3.

5.4.2 Diagnosis results

Detection results using the trained model are shown in Figure 14. The overall accuracy rate was 98.3%, demonstrating that the level of longitudinal level irregularity can be classified with high accuracy into normal, medium, and large.

Figure 14.

Detection accuracy for the different levels of a degraded track (track irregularity).

5.5 Classification of the degradation level of joint depression

5.5.1 Configuration of the trained convolutional neural network

In order to classify the degradation level of joint depression into three types: normal, medium, and large, we prepared a total of 300 images: 100 normal, 100 medium, and 100 large. We set aside 80% of the images for training and 20% for evaluation. The network configuration and the number of training sessions were the same as in Section 5.3.

5.5.2 Diagnosis results

Detection results using the trained model are shown in Figure 15. Some incorrect diagnoses were made in the images of normal and medium joint depression. However, the overall accuracy was 96.7%, which was sufficient to classify the level of joint depression, demonstrating that the diagnostic algorithm we developed is effective for the diagnosis of joint depression.

Figure 15.

Detection accuracy for the different levels of a degraded track (joint depression).

6. Investigation of CWT images that were diagnosed incorrectly

Figure 16 shows an example of an image that was diagnosed incorrectly. The right side of Figure 16a was diagnosed as normal, even though it shows joint depression. Conversely, the left side of Figure 16a shows an image that was diagnosed correctly as a joint depression. Comparing those, the feature representing the joint depression is extremely small in the incorrectly diagnosed image. This reveals that an incorrect diagnosis can occur when the features are extremely small.

Figure 16.

CWT images that were diagnosed incorrectly.

The right side of Figure 16b was diagnosed as normal, even though it shows a large track irregularity. Conversely, the left side of Figure 16b shows an image that was diagnosed correctly as a large track irregularity. The reason for the incorrect diagnosis was that the large amplitude of the vertical acceleration, shown in red color, was appeared at the bottom of the CWT image.

7. Conclusion

In this study, we proposed a method to classify the type and level of track faults automatically using a convolutional neural network trained on car body vibration acceleration measurements converted into images using a CWT, a well-known method of time-frequency analysis. The algorithm we developed was used to perform the diagnosis of track conditions on actual measurements.

The results demonstrated that it is possible to diagnose the type and level of degradation of track faults with high accuracy.

In future work, we plan to improve the algorithm to estimate the locations of track faults accurately in actual measurements and monitor the condition of railway tracks in more detail.

Acknowledgments

This research was funded by Nihon University Research Grant for Social Implementation (19-006) (2019). We would like to thank Editage (www.editage.jp) for English language editing.

Conflict of interest

The authors declare no conflict of interest.

A CWT is a method that simultaneously detects the frequency and time characteristics of an unsteady signal, by comparing the original signal with dilated and translated versions of a small wavelike function called the mother wavelet. The CWT computes the inner products of a continuous signal with a set of continuous wavelets according to the following equation

$$W_{\psi }\left(a,b\right)={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{a}}{\psi }^{\ast }\left(\frac{t-b}{a}\right)x\left(t\right)\mathit{dt},$$EA1

where, variables $a$ and $b$ correspond to the dilatation and location parameters, respectively, they translate the mother wavelet $\psi \left(t\right)$ by a time shift $b$ in time, and by $1/a$ in frequency. ${\psi }^{\ast }$ indicates the complex conjugate of $\psi$.

In this study, we used the real-valued Morlet wavelet (Figure 17) as the mother wavelet $\psi \left(t\right)$.

Figure 17.

Real-valued Morlet wavelet.

$$\psi \left(t\right)=e^{-\frac{t^2}{2}}cos\left(5t\right).$$EA2

A Convolutional Neural Network (CNN) is a well-known deep learning architecture. There are numerous variants of CNN architectures. The basic components of CNN consist of convolutional layer, pooling layer, and fully-connected layers [19].

B.1 Convolution Layer

The objective of the convolution operation is to extract the significant features from the input image. The convolution layer is composed of several convolution kernels which are used to compute different feature maps. The feature maps are generated by the convolution operation with the filter that acts as the feature extractor as follows.

$$I_2\left(x,y\right)=\sum _{j=-N}^N\sum _{i=-N}^{N}F\left(i,j\right)I_1\left(x-i,y-j\right).$$EA3

where $I_1\left(x,y\right)$: pixel value of input image at $\left(x,y\right)$, $I_2\left(x,y\right)$: pixel value of output image at $\left(x,y\right)$, $F\left(i,j\right)$: filter coefficient.

B.2 Pooling layer

The Pooling layer is responsible for reducing the spatial size of the feature maps. This is to decrease the computational power required to process the data through size reduction. It is useful for extracting dominant features. There are two types of Pooling: Max Pooling and Average Pooling. Max Pooling returns the maximum value from the portion of the image. On the other hand, Average Pooling returns the average value. In this study, Max Pooling were used. Figure 18 shows the example of the Max Pooling operation.

Figure 18.

Max pooling.

B.3 Activation function

Rectified linear unit (ReLU) is one of the most famous activation functions. In this study, the following function is used to adjust the output of the Pooling Layer.

$$y=\left\{\begin{array}{cc}0& \left(x\le 0\right)\\ x& \left(x>0\right)\end{array}\right.$$EB1

where $x$ is the input of the activation function. The simple operation of the activation function makes the faster computation than sigmoid or hyperbolic tangent functions.

Softmax function defined by

$$y_i=\frac{e^{x_i}}{{\sum }_{k=1}^n{e}^{x_k}}\left(i=1,2,\cdots ,n\right),$$EB2

was used in output layer. Where $n$ indicates the number of classification.

B.4 Fully connected layer

In fully connected layers, the neuron applies a linear transformation to the input vector through a weights matrix. In this study, an Affine transformation was used in fully connected layer.

B.5 Loss function

The loss function is the function that computes the distance between the current output of the algorithm and the expected output. In this study, we employed the categorical cross-entropy, which is well suited to classification tasks.

Abbreviations

CWT	continuous wavelet transform
RMS	root mean square
CNN	convolutional neural network
HHT	Hilbert–Huang transform
GNSS	global navigation satellite system