Condition monitoring of railway tracks is effective for the sake of an increase in the safety of regional railways. This study proposes a new method for automatically classifying the type and degradation level of track fault using a convolutional neural network (CNN), which is a machine learning method, by imaging car body acceleration on a time-frequency plane by continuous wavelet transform. As a result of applying this method to the data measured in regional railways, it was possible to classify and extract the sections that need repair according to the degree of deterioration of the tracks, and to identify the track fault in those sections.
- condition monitoring
- convolutional neural network
Maintenance of railway tracks is essential for the safe operation of trains. Railway operators conduct track inspections using track geometry cars and track maintenance crews. However, regional railway operators, who carry fewer passengers, often lack the personnel and funds to conduct adequate track inspections. The monitoring of railway track geometry from an in-service vehicle has become increasingly attractive over the past decade .
To address this problem, a system that can monitor the track condition inexpensively and frequently using a device incorporating sensors and a global navigation satellite system (GNSS) unit, which is installed on in-service trains, has been developed [2, 3]. The system calculates root mean square (RMS) values from the vertical acceleration, lateral acceleration, and roll angular velocity of the car body. To select sites for repair, we adopt the method of prioritizing sites with the highest numerical values.
The acceleration RMS is closely related to the general health of the track . In Ref. , RMS values are used to identify track irregularities for longitudinal level, alignment, cross-level. However, monitoring based on RMS values alone is not sufficient. Without frequency information, it is difficult to identify the type of track fault. Furthermore, since the amount of data generated by constant measurement is enormous, it is necessary to automate the analysis in order to monitor and predict the track condition efficiently.
In this study, we propose a method to classify the types of track faults automatically by means of machine learning, using a CNN trained on images created via a CWT from the vibration acceleration on the time-frequency plane. A continuous wavelet transform (CWT) is a transformation technique that emphasizes certain portions of the waveform by suppressing other portions as it proceeds by multiplying a target waveform using a mother wavelet . A convolutional neural network (CNN) is a class of deep neural networks. It is widely used for image recognition.
To verify the effectiveness of the algorithm we developed, we first describe the results of simulating the vibration of a car body when passing over a faulty track. Next, we describe the results of diagnosing track faults from the vertical vibration acceleration data of a car body measured by a regional railway.
2. Literature review of track condition monitoring using machine learning techniques
It should be necessary for railway operators to control track irregularity, such as vertical rail profiles, lateral alignment, gauge, cross-level, twist (depicted in Figure 1) properly. Track irregularities cause vehicle vibrations that degrade the rider’s comfort and increase the risk of derailments. Track irregularities are strongly correlated with vehicle vibrations. Thus, it can be possible to estimate general trends of the track condition by analyzing vehicle vibrations.
Although track geometry measurement systems using in-service vehicles are becoming increasingly attractive around the world [2, 7, 8, 9], the repeated checking of the same track provides the information regarding track geometry degradation, which can be fed back to the track maintenance section for taking essential actions. The use of vehicle responses in the track geometry assessment process allows identifying of critical defects, which could not have been identified from geometry parameters, and thus, improve the maintenance operations.
Tsunashima et al. proposed techniques of condition monitoring of railway tracks based on time-frequency analysis . They compared the performance of Hilbert-Huang transforms (HHT) and CWT for identifying track faults from car body vibration. It is shown that the feature of track fault can be identified in time-frequency plane.
Tsunashima proposed a classifier based on a machine learning technique for identifying track faults automatically from measured car body vibration . It is shown that the degradation of track can be classified in the feature space consisting of car body vibration RMS.
Faghih-Roohi et al. proposed a deep convolutional neural network for the analysis of image data for the detection of rail surface defects . They explored the efficiency of the proposed deep convolutional neural network for the detection and classification of rail surface defects.
Zheng et al. proposed a multi-object detection method based on a deep convolutional neural network that can achieve non-destructive detection of rail surface and fastener defects . A defect detection model based on Mask R-CNN and ResNet framework was utilized to detect the surface defects.
Jin et al. proposed a machine learning framework based on wavelet scattering networks and neural networks for identifying railhead defects .
3. Effect of track faults on time-frequency plane
3.1 Overview of the simulation
When a train runs on a track, vibrations that correspond to the track geometry are generated [15, 16]. Therefore, in this study, to verify the relationship between the type of track fault and the car body vibration acceleration, and to evaluate the effectiveness of time-frequency analysis in detecting track faults, we simulated the occurrence of track faults, calculated the vertical vibration acceleration of the car body, and then applied a CWT, a method of time-frequency analysis, to the results.
3.2 Continuous wavelet transform (CWT)
A CWT is a method that simultaneously detects the frequency and time characteristics of an unsteady signal, by comparing the original signal with dilated and translated versions of a small wavelike function called the mother wavelet. Using this method, it is possible to view the amplitude and frequency information of the vibration acceleration as an image. In this study, we used the
This technique is well suited for analyzing unsteady signals, such as those that exhibit sudden variation, and is defined as follows:
where, variables and correspond to the dilatation and location parameters, respectively, they translate the mother wavelet by a time shift in time, and by in frequency. indicates the complex conjugate of .
3.3 Vehicle model used in the simulation
The vehicle model used in the simulation is shown in Figure 2 . The vehicle model consists of a total of seven rigid bodies: one car body, two bogies, and four wheelsets. The car body and bogie were assigned two degrees of freedom (DOF) for bounce and pitch, and the wheelset was assigned one DOF for the bounce. The vehicle’s parameters were obtained from measurement data from a regional railway vehicle equipped with an onboard sensing device.
3.4 Simulation conditions
In the simulation, the vehicle model was run at 60 [km/h] for 500 [m], and the results were output for the section between 100 [m] and 350 [m]. We set rail joint faults (joint depressions) at 4 points; otherwise, the track was assumed to be straight. To set the rail joint faults, we used the function model shown in Figure 3 .
The geometry of the modeled track are represented by
The track geometry used in the simulation is shown in Figure 4. At the 150 [m] and 200 [m] points, we set depths of and , respectively, to simulate joint depressions, which are depressions in the top surface of the railhead that occur at track joints.
In both cases, we set the depression length . For comparison, at the 250 [m] and 300 [m] points, we set a gentler dip in track geometry by setting depths of and , respectively, but with length . The values of and were determined with reference to generally occurring track displacement. The car body vibration acceleration was assumed to occur directly above the center of the front bogie of the vehicle model. The simulation was performed at a sampling frequency of 200 [Hz].
3.5 Simulation results
The simulated vertical vibration acceleration of the car body is shown in Figure 5a. The figure shows that characteristic vibrations corresponding to the track geometry are generated at the points where the track faults were set. Figure 5b shows the result of the CWT of the simulated vertical vibration acceleration. The color bar indicates the magnitude of the amplitude in the time-frequency plane.
At 150 [m] and 200 [m], the points where the joint depressions were simulated, vibrations in the frequency band of 15–30 [Hz] were detected due to the impulse-like track geometry, and variations depending on depth can be seen in the CWT images. In addition, at 250 [m] and 300 [m], the points where gentler dips in the track geometry were set, vibrations in the frequency band of 0–5 [Hz] were detected, and variations depending on depth can be seen in the CWT images. These results demonstrate that the CWT images are effective for identifying track faults, since the features of the CWT images change markedly depending on the type and level of degradation.
4. Monitoring the condition of railway track using a convolutional neural network
4.1 Track condition monitoring system
Accelerometers and rate gyros in the onboard sensing device measure the car body vibration. A GNSS receiver detects the location and speed of the train. Collected data are transmitted to the data server in the monitoring center continuously via a mobile phone network.
The diagnostic software analyses the collected data and results are fed back to the railway operators through online channels via tablet computers. The diagnostic results are used to facilitate the maintenance work of railway operators.
Convolutional neural networks are a method used in the field of machine learning called deep learning and are particularly suitable for image recognition. In this study, we examined the effectiveness of classifying longitudinal level irregularities and joint depressions automatically, using a diagnostic algorithm, we constructed based on a convolutional neural network trained on CWT images generated from vertical vibration acceleration data from a car body. The diagnostic procedure is shown in Figure 7.
5. Identification of the condition of railway track using vertical vibration acceleration data measured from an actual car body
5.1 Overview of diagnosis
The car body’s vertical acceleration with track faults was collected in a regional railway line using the track condition monitoring system. The input data for the classifier consists of vertical vibration acceleration measurements from an onboard sensing device in a car body, which are then converted into images using a CWT. Figure 8 shows an example of converting the measurements into a CWT image.
The vibration characteristics of the joint depression at the distance of 25.82 [km] appear in the 10–30 [Hz] frequency range. The vibration characteristics of the longitudinal level irregularities around 25.95 [km] appear in the 0–5 [Hz] frequency range.
5.2 Images used for training and evaluation
In this study, we investigated the following three types of diagnoses:
Classification of images into three types: longitudinal level irregularity, joint depression, and normal.
Classification of the degradation level of longitudinal level irregularity into normal, medium, and large.
Classification of the degradation level of joint depression into normal, medium, and large.
For diagnosing the level of degradation of longitudinal level irregularities, in cases where the one-side amplitude of the vibration acceleration was normal, images of car body acceleration of were used. To diagnose medium degradation, images of were used, and to diagnose large degradation, images of or greater were used.
For diagnosing the level of degradation of joint depressions, in cases where the one-side amplitude of the vibration acceleration was normal, images of body acceleration of 0 to were used. To diagnose medium degradation, images of were used, and to diagnose large degradation, images of or greater were used.
5.3 Identification of longitudinal level irregularities and joint depressions
5.3.1 Configuration of the trained convolutional neural network
We prepared a total of 300 images: 100 normal images, 100 images with a longitudinal level irregularity, and 100 images with a joint depression. We set aside 80% of the images for training and 20% for evaluation as shown in Figure 9.
Figure 12 shows the configuration of the trained convolutional neural network (see Appendix B). In the figure, the name of the process and the size (vertical × horizontal × channels) before processing are indicated above each layer, and the size after processing is indicated below the layer.
The Convolution layer applies the convolution operation to the image, representing it in matrix form; the Max pooling layer performs information compression; the Affine layer combines information from different layers, and the Output layer outputs a set of probabilities indicating how well the image matches the three types of training image data. The number of training sessions was set to 50.
5.3.2 Diagnosis results
Figure 13 shows the results of using images for evaluation to discriminate longitudinal level irregularity track faults versus joint depression track faults versus normal track. The overall accuracy rate was 98.3%, demonstrating that convolutional neural networks are effective for the classification of track faults.
5.4 Identification of the degradation level of longitudinal level irregularities
5.4.1 Configuration of the trained convolutional neural network
In order to classify the degradation level of longitudinal level irregularities into three types: normal, medium, and large, we prepared a total of 300 images: 100 normal, 100 medium, and 100 large. We set aside 80% of the images for training and 20% for evaluation. The network configuration and the number of training sessions were the same as in Section 5.3.
5.4.2 Diagnosis results
Detection results using the trained model are shown in Figure 14. The overall accuracy rate was 98.3%, demonstrating that the level of longitudinal level irregularity can be classified with high accuracy into normal, medium, and large.
5.5 Classification of the degradation level of joint depression
5.5.1 Configuration of the trained convolutional neural network
In order to classify the degradation level of joint depression into three types: normal, medium, and large, we prepared a total of 300 images: 100 normal, 100 medium, and 100 large. We set aside 80% of the images for training and 20% for evaluation. The network configuration and the number of training sessions were the same as in Section 5.3.
5.5.2 Diagnosis results
Detection results using the trained model are shown in Figure 15. Some incorrect diagnoses were made in the images of normal and medium joint depression. However, the overall accuracy was 96.7%, which was sufficient to classify the level of joint depression, demonstrating that the diagnostic algorithm we developed is effective for the diagnosis of joint depression.
6. Investigation of CWT images that were diagnosed incorrectly
Figure 16 shows an example of an image that was diagnosed incorrectly. The right side of Figure 16a was diagnosed as normal, even though it shows joint depression. Conversely, the left side of Figure 16a shows an image that was diagnosed correctly as a joint depression. Comparing those, the feature representing the joint depression is extremely small in the incorrectly diagnosed image. This reveals that an incorrect diagnosis can occur when the features are extremely small.
The right side of Figure 16b was diagnosed as normal, even though it shows a large track irregularity. Conversely, the left side of Figure 16b shows an image that was diagnosed correctly as a large track irregularity. The reason for the incorrect diagnosis was that the large amplitude of the vertical acceleration, shown in red color, was appeared at the bottom of the CWT image.
In this study, we proposed a method to classify the type and level of track faults automatically using a convolutional neural network trained on car body vibration acceleration measurements converted into images using a CWT, a well-known method of time-frequency analysis. The algorithm we developed was used to perform the diagnosis of track conditions on actual measurements.
The results demonstrated that it is possible to diagnose the type and level of degradation of track faults with high accuracy.
In future work, we plan to improve the algorithm to estimate the locations of track faults accurately in actual measurements and monitor the condition of railway tracks in more detail.
This research was funded by Nihon University Research Grant for Social Implementation (19-006) (2019). We would like to thank Editage (www.editage.jp) for English language editing.
Conflict of interest
The authors declare no conflict of interest.
A CWT is a method that simultaneously detects the frequency and time characteristics of an unsteady signal, by comparing the original signal with dilated and translated versions of a small wavelike function called the mother wavelet. The CWT computes the inner products of a continuous signal with a set of continuous wavelets according to the following equation
where, variables and correspond to the dilatation and location parameters, respectively, they translate the mother wavelet by a time shift in time, and by in frequency. indicates the complex conjugate of .
In this study, we used the real-valued
A Convolutional Neural Network (CNN) is a well-known deep learning architecture. There are numerous variants of CNN architectures. The basic components of CNN consist of convolutional layer, pooling layer, and fully-connected layers .
B.1 Convolution Layer
The objective of the convolution operation is to extract the significant features from the input image. The convolution layer is composed of several convolution kernels which are used to compute different feature maps. The feature maps are generated by the convolution operation with the filter that acts as the feature extractor as follows.
where : pixel value of input image at , : pixel value of output image at , : filter coefficient.
B.2 Pooling layer
The Pooling layer is responsible for reducing the spatial size of the feature maps. This is to decrease the computational power required to process the data through size reduction. It is useful for extracting dominant features. There are two types of Pooling: Max Pooling and Average Pooling. Max Pooling returns the maximum value from the portion of the image. On the other hand, Average Pooling returns the average value. In this study, Max Pooling were used. Figure 18 shows the example of the Max Pooling operation.
B.3 Activation function
Rectified linear unit (ReLU) is one of the most famous activation functions. In this study, the following function is used to adjust the output of the Pooling Layer.
where is the input of the activation function. The simple operation of the activation function makes the faster computation than sigmoid or hyperbolic tangent functions.
Softmax function defined by
was used in output layer. Where indicates the number of classification.
B.4 Fully connected layer
In fully connected layers, the neuron applies a linear transformation to the input vector through a weights matrix. In this study, an Affine transformation was used in fully connected layer.
B.5 Loss function
The loss function is the function that computes the distance between the current output of the algorithm and the expected output. In this study, we employed the categorical cross-entropy, which is well suited to classification tasks.
|CWT||continuous wavelet transform|
|RMS||root mean square|
|CNN||convolutional neural network|
|GNSS||global navigation satellite system|
Weston P, Roberts C, Yeo G, Stewar E. Perspectives on railway track geometry condition monitoring from in-service railway vehicles. Vehicle System Dynamics. 2015; 53(7):1063-1091. DOI: 10.1080/00423114.2015.1034730
Tsunashima H, Mori H, Ogino M, Asano A. Development of track condition monitoring system using onboard sensing device. In: Zboinski K. editors. Railway Research. London: IntechOpen; 2015. p. 145. DOI: 10.5772/61077
Mori H, Ohno H, Tsunashima H, Saito Y. Development of compact size onboard device for condition monitoring of railway tracks. Journal of Mechanical Systems for Transportation and Logistics. 2013; 6(2):142-149. DOI: 10.1299/jmtl.6.142
Vinberg EM, Martin M, Firdaus AH, Tang Y, Qazizadeh A. Railway Applications of Condition Monitoring. Stockholm, Sweden: KTH Royal Institute of Technology; 2018. p. 147. DOI: 10.13140/RG.2.2.35912.62729
Tsunashima H. Condition monitoring of railway tracks from car-body vibration using a machine learning technique. Applied Sciences. 2019; 9(13):2734. DOI: 10.3390/app9132734
Daubechies I. Ten Lectures on Wavelets. Society for Industrial and Applied Mathematics: Philadelphia, Pennsylvania, United States. 1992. p. 441 DOI: 10.1137/1.9781611970104
Weston P, Ling C, Goodman C, Roberts C, Li P, Goodall R. Monitoring vertical track irregularity from in-service railway vehicles. Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit. 2007; 221:75-88. DOI: 10.1243/0954409JRRT65
Weston P, Ling C, Goodman C, Roberts C, Li P, Goodall R. Monitoring lateral track irregularity from in-service railway vehicles. Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit. 2007; 221:89-100. DOI: 10.1243/0954409JRRT64
Wei X, Liu F, Jia L. Urban rail track condition monitoring based on in-service vehicle acceleration measurements. Measurement. 2016; 80:217-228. DOI: 10.1016/j.measurement.2015.11.033
Tsunashima H, Hirose R. Condition monitoring of railway track from car-body vibration using time–frequency analysis. Vehicle System Dynamics. 2020; 1:1-18. DOI: 10.1080/00423114.2020.1850808
Faghih-Roohi S, Hajizadeh S, Nunez A, Babuska R, De Schutter B. Deep convolutional neural networks for detection of rail surface defects. In: Proceedings of the 2016 International Joint Conference on Neural Networks. Vancouver, Canada: IJCNN; 2016. pp. 2584-2589
Zheng D, Li L, Zheng S, Chai X, Zhao S, Tong Q, et al. A defect detection method for rail surface and fasteners based on deep convolutional neural network. Hindawi, Computational Intelligence and Neuroscience. 2021; 2565500:15. DOI: 10.1155/2021/2565500
Jin Y. Wavelet scattering and neural networks for railhead defect identification. Materials. 1957; 2021:14. DOI: 10.3390/ma14081957
Alvarenga TA, Carvalho AL, Honorio LM, Cerqueira AS, Filho LMA, Nobrega RA. Detection and classification system for rail surface defects based on Eddy current. Sensors. 2021; 21:7937. DOI: 10.3390/s21237937
Kraft S, Causse J, Coudert F. Vehicle response based track geometry assessment using multi-body simulation. Vehicle System Dynamics. 2018; 56(2):190-220. DOI: 10.1080/00423114.2017.1359418
Karis T, Berg M, Stichel S, Li M, Thomas D, Dirks B. Correlation of track irregularities and vehicle responses based on measured data. Vehicle System Dynamics. 2018; 56(6):967-981. DOI: 10.1080/00423114.2017.1403634
Le T. Use of the Morlet mother wavelet in the frequency-scale domain decomposition technique for the modal identification of ambient vibration responses. Mechanical Systems and Signal Processing. 2017; 95:488-505. DOI: 10.1016/j.ymssp.2017.03.045
Garg V, Dukkipati R. Dynamics of Railway Vehicle Systems. Cambridge, Massachusetts: Academic Press; 1984. p. 407
Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, et al. Recent advances in convolutional neural networks. Pattern Recognition. 2018; 77:354-377. DOI: 10.1016/j.patcog.2017.10.013