Human errors of train drivers may cause serious damage. Therefore, research on human error prevention has been conducted by many researchers. In this context, brain activity measurement of train drivers using near-infrared spectroscopy (NIRS) has been conducted to monitor the condition of train drivers. In this study, we developed a compact wireless wearable NIRS that can be used in natural environments. The wearable NIRS has been used to measure train drivers’ brain function using a train driving simulator. Experimental results showed that brain activity of the dorsolateral prefrontal cortex (DLPFC) increased when the driver made braking operation. The experiment for train driving with an accidental event was carried out to evaluate the relation between drivers’ attention and the brain activity. As a result, there was a difference in brain activity between with and without prior notice. Results showed that the increased attention of the train driver can be shown in the NIRS signal from the outer part of the prefrontal cortex.
- train driver
- human error
- condition monitoring
Accidents due to human error while driving trains are rare but can cause severe and extensive damage when they do occur. Human error for train drivers refers to driving in an unusual condition, for example, when drowsy, exhausted, or in a hurry, which may lead to ignoring signals or speeding. Signaling systems such as automatic train protection (ATP) are used to prevent accidents caused by human error, yet not all accidents can be avoided even when these signaling systems are in place. Thus, research is being conducted to enable the monitoring of the status of train drivers, the early detection of abnormalities, and the prevention of accidents.
Functional near-infrared spectroscopy (fNIRS) was used to measure subjects’ cerebral blood flow to investigate higher-order human brain function activity associated with cognition and attention while operating a vehicle [1, 2]. Lohani et al. provides a selective review of the psychophysiological measures that can be utilized to assess cognitive states in real-world driving environments . Brain activity of train driver was measured using a 128-channel high-density electroencephalogram (EEG) while participants drove in a train driving simulator . Kojima et al. analyzed the characteristics of brain activity measured in the prefrontal area with functional near-infrared spectroscopy (fNIRS) [5, 6]. However, there are many challenges to consider when applying these to the real world.
Conventional multichannel NIRS employs several cables to obtain readings. Furthermore, the probes placed on the head are heavy and are not suited for long-term monitoring. This has posed a problem for measuring brain activity in train drivers in real time. Recently, a lightweight and wireless wearable NIRS device was developed and has been used to measure brain function in car drivers. It is possible that this NIRS technology can also be applied to measure brain activity in train drivers.
While a wearable NIRS device places only a small burden on participants, can be used for assessment over long periods, and is applicable in the real world, it has fewer channels and cannot create a brain functional image, and thus is not suitable for the identification of activation areas. Meanwhile, multichannel NIRS device has several channels and is suitable for such identification, but places a burden on participants and is not a good match for measurement in the real world.
In this study, we first identified activation areas from a brain functional image using multichannel NIRS (Spectratech, OEG-16 ). We then measured brain activity in these areas using a wearable NIRS device (Astem, Hb131S ) and examined brain activity measurements from the channel signals, including the activation areas [9, 10].
This paper is an extended version of the published paper .
2. Principles of NIRS
NIRS is a method of measuring changes in hemoglobin concentration using near-infrared light (wavelength: 700–900 nm) which can penetrate deep into the living tissue. When nerve activity occurs, there is a local increase in blood flow and a change in hemoglobin concentration in the blood. It is possible to measure changes in concentration of oxygenated hemoglobin (oxy-Hb) and deoxygenated hemoglobin (deoxy-Hb) from the transmitted near-infrared light and attenuation of the diffused light [12, 13].
When measuring brain activity, near-infrared light is beamed from optical fibers placed on the surface of the head. This light reaches the surface of the brain while being absorbed and diffused by the scalp, cranium, and cerebrospinal fluid. The amount of transmitted light that returns to the surface of the head is then measured by the detector fibers, which can be used to calculate changes in the hemoglobin concentration on the surface of the brain.
Brain activity accounts for roughly 5% of local intracerebral oxygen consumption. In contrast, local intracerebral blood flow increases by 30–50%, far surpassing consumption. Consequently, an area in which there is brain activity will typically demonstrate an increase in oxy-Hb and a decrease in deoxy-Hb, and this change is seen in the surrounding brain regions as well. Thus, evaluating the amount of change in oxy-Hb makes it possible to understand the status of brain activity, and this feature is used by NIRS devices as the parameter which best reflects brain activation. In the present study, brain activity was analyzed using the amount of oxy-Hb change as the primary evaluation criteria.
NIRS also includes a method of measurement known as spatial resolved spectroscopy (SRS). SRS is a method of measuring oxygen saturation (StO2), which utilizes two detectors to measure two different optical path lengths. The wearable NIRS (Hb131S) used in the present study is equipped with SRS, thereby enabling the measurement of oxygen saturation, which is an effective indicator highly correlated with oxy-Hb and is not likely to be influenced by cutaneous blood flow. However, its sensitivity is low compared to oxy-Hb, and there are presently a few cases in which it is used as a primary evaluation index for brain activity. Therefore, deoxy-Hb and oxygen saturations were used as supplementary indicators to discern the existence of artifacts Figure 1 shows a schematic of changes in hemoglobin concentration and oxygen saturation due to neural activity.
3.1 Experimental equipment
In this experiment, brain activity was measured while driving a train using multichannel NIRS (OEG-16, Spectratech, 16ch, sampling rate: 0.66 s) and wearable NIRS (Hb131S, Astem, 4ch, sampling rate: 0.5 s). The multichannel NIRS was fitted so that the lowest row of channels was 40 mm from the base of the nose. The wearable NIRS was mounted so that 1ch corresponded to the vicinity of the multichannel NIRS 13ch.
Figure 2 shows the NIRS channels for OEG-16 and Hb131S, while Figure 3 shows the Hb131S device. Fpz in 10–20 electrode system corresponds to the center of the NIRS cap. The Hb131S is completely wireless and lightweight, meaning that it can be used for measurement while driving a train without impeding operations. Furthermore, in addition to oxy-Hb and deoxy-Hb, which can be measured by normal NIRS, Hb131S is able to measure oxygen saturation.
The brain activity of train drivers was also measured using a train driving simulator which records the driving position, speed, acceleration, and handle notch position of master controller. Figure 4 shows the experimental setup. Figure 5 shows the handle notch position in the train driving simulator.
3.2 Station stopping task
Previous studies have confirmed increases in oxy-Hb in the DLPFC in areas in which drivers performed a deceleration operation (hereafter referred to as a braking area) with the goal of stopping [5, 6]. However, measurement using multichannel fNIRS (OMM-3000, Shimadzu, 42ch) places a large burden on participants and is difficult to use for long-term experiments. Therefore, we took measurements using a multichannel NIRS device (OEG-16, Spectratech, 16ch), which is able to measure more easily and has a smaller burden on participants when used to identify activation areas. Next, we considered whether measurement was possible with the smaller wearable NIRS which can be worn constantly, even in the real world.
Nine healthy men between the ages of 20 and 29 with an experience of driving a car, and who gave informed consent, participated in the station stopping task. Brain activity was measured during operation of the train driving simulator. In addition to the wearable NIRS Hb131S, measurement of the entire prefrontal area was conducted with the multichannel NIRS OEG-16 (16ch) for the station stopping task. We investigated whether the brain activity of train drivers could be measured using wearable NIRS technology through a comparison with the multichannel NIRS. In this study, we particularly focused on the 1ch in wearable NIRS and 13ch in multichannel NIRS, for which the channel positioning was the same.
The experimental design for the station stopping task is shown in Figure 6. For the rest of the periods during the station stopping task, participants rested for 30 s with their eyes closed. For the task periods, participants were instructed to drive from departure to designated stop, using only acceleration and deceleration and following the speed limit on a roughly 940 m section. A total of four trials were performed with one trial consisting of a rest period and a task period. The experiment was performed with a 2-day interval between the respective devices.
3.3 Obstacle avoidance task
Nakagawa et al. assessed the brain activity of drivers when they were shown an obstacle on the tracks in a train driving simulator using a 128-channel high-density EEG . Similarly, in this study we showed participants an obstacle on the tracks and examined whether brain activity while driving could be evaluated from the cerebral blood flow. In particular, we evaluated whether drivers could appropriately process external environmental information while driving and maintain appropriate driving behaviors when abnormalities appeared (hereafter referred to as attentional state). Generally, individuals driving casually tend to have lower brain activity than those paying appropriate attention to the external environment.
Eleven healthy men between the ages of 20 and 29 with an experience of driving a car, and who gave informed consent, participated in the obstacle avoidance task. We examined whether measurement of the attentional state was possible with wearable NIRS during an accident event, which involved placing a car on the tracks as the obstacle. The experimental design is shown in Figure 7.
The obstacle appearance was positioned such that participants needed to apply the emergency brake promptly when the obstacle was discovered in order to safely stop the train. One experiment in which the participant was not informed that an obstacle would appear and one in which they were informed were performed with each participant.
The experiment with no advanced notice of the obstacle was performed four times: the first three times were similar to the station stopping task, but on the fourth trial, the obstacle was presented. The experiment with advanced notice of the obstacle was performed next. For this experiment, participants were informed in advance that an obstacle would appear in one of the four trials and that they should promptly stop when the obstacle appeared during the task. In the task period of the obstacle avoidance task, a driving distance of approximately 1400 m between stations was selected. The rest periods lasted 60 s.
4. NIRS signal analysis method
NIRS signals include sounds from the body such as breathing and heartbeat. The signals also include trends across the entire experiment when measured over long periods. Therefore, signal processing was performed through multi-resolution analysis (MRA) [14, 15] using discrete wavelet transformation to extract only the brain activity components relevant to the research design .
Wavelet transform expresses the local shape of the waveform to be analyzed, , by shifting and dilating the waveform called the mother wavelet, , and then analyzes the waveform:
which is dilated with a scale parameter and translated by as
The continuous wavelet transform of signal is computed with as
Here, denotes the complex conjugate of .
One can construct wavelets such that the dilated and translated function
is an orthonormal base.
Discrete wavelet transform can be computed by
In the continuous wavelet transform, information is duplicated, requiring many calculations. Discrete wavelet transform handles a smaller volume of information than continuous wavelet transform but is able to transform signals more efficiently. Furthermore, the use of an orthonormal base facilitates complete reconstruction of original signals without redundancy. The following section describes decomposition and reconstruction of signals using MRA.
MRA decomposes signals into a tree structure using the discrete wavelet transform. In the case of the object time series signals, , it decomposes the signals into an approximated component (low-frequency component) and multiple detailed components (high-frequency components).
A signal can be expressed as follows by discrete wavelet transform using an orthonormal base as
Here, is the scaling function as defined by the following equation:
The coefficient of the approximated component is calculated by
The detailed components of the signals on level m can be expressed by
Thus, the signal can be expressed as
Task-related components can thus be reconstructed from detailed components .
In the wavelet transform, the choice of a mother wavelet is important. We employed a
Further, NIRS signals can be used to measure the relative amount of change in oxy-Hb and deoxy-Hb with the value at the start of measurement as the criteria. However, with the relative amount of change, comparisons between participants and statistical processing are difficult. As such, the data from oxy-Hb and deoxy-Hb that had undergone multi-resolution analyses were, respectively, -scored such that the mean would be 0 and the standard deviation would be 1, according to the following equation:
Here, represents the oxy-Hb or deoxy-Hb signal after reconstruction, while and refer to the mean and standard deviation, respectively.
The NIRS signals for each participant were resampled using linear interpolation as there was a variation in the periods between driving, starting, and stopping between the participants due to the nature of the experiment. Thereafter, time series data was converted to distance data, allowing it to be directly compared with the data obtained from the driving simulator. Furthermore, the arithmetic mean was calculated for the NIRS signals which had been converted into distance data for each participant and statistical processing performed.
5.1 Station stopping task
Multi-resolution analysis and -scoring were performed for the multichannel NIRS data for each of the 9 participants (9 males, aged 21–27 years (mean SD 22.4 1.7)).
Figure 9 shows the time course of 16 channels for OEG-16. It can be seen that the neuronal activity in the braking task increases in the wide area of the prefrontal cortex. This result is supposed to be due to the fact that all participants are not familiar enough with train driving.
The arithmetic mean () results of the time series data converted into distance data for each trial are shown in Figure 10. Figure 10(a) shows the speed and notch data and Figure 10(b) the oxy-Hb and deoxy-Hb data of 13ch. Figure 10(a) demonstrates that the notch value became negative when the participant started braking in order to stop. Thus, Figure 10(b) confirmed that oxy-Hb signals tended to increase once braking had commenced.
This further designated the areas that should be focused on with wearable NIRS which has fewer channels. Therefore, brain functional images were created and evaluated for when participants first rested with their eyes closed during the experiment, during the acceleration operation (from the start of driving to the end of acceleration), and during the deceleration operation (from the deceleration start position to the stopping position), respectively. Figure 11(a) shows the functional brain image when resting with the eyes closed, Figure 11(b) during the acceleration operation, and Figure 11(c) during the deceleration operation.
Brain activation in the dorsolateral areas was confirmed from the multichannel NIRS used in this study. Thus, for the wearable NIRS, analysis of the NIRS signals for each participant was performed with a focus on these areas. The left lateral area (1ch) of the wearable NIRS was analyzed in the same way as the multichannel NIRS. These results are shown in Figure 12. The arithmetic mean () results for the notch data and speed for each distance are shown in Figure 12(a). The arithmetic mean () results for oxygen saturation and oxy-Hb and deoxy-Hb signals in the left lateral area for each distance are shown in Figure 12(b).
In Figure 12(a), the position at which the notch value becomes negative and deceleration begins was set as the deceleration start position. Figure 12(b) confirms that oxy-Hb and oxygen saturation tended to increase between the deceleration start position and stopping the train. The same trend was observed in the right lateral area. This confirms that the measurement of the drivers’ brain activity using wearable NIRS was successful.
5.2 Obstacle avoidance task
Like the station stopping task, multi-resolution analysis and -scoring were performed for the wearable NIRS data for each of the 11 participants. Then the arithmetic mean () was calculated for each trial. The same analysis was performed regardless of whether there was an advanced notice of the obstacle on the tracks. In this manuscript, analyses were performed for the lateral area that had been found to be activated during attention.
The notch, train speed, and position of the obstacle on the tracks for the trial in which it was presented without advanced notice are shown in Figure 13(a). The time history of NIRS signals (1ch) is shown in Figure 13(b). The obstacle car was presented without prior notice for all participants. As a result, it was not possible to quickly begin a deceleration operation and a collision with the obstacle occurred. The participant who was able to stop closest to the obstacle on the tracks (at 965 m) was used as a standard for the arithmetic mean.
Our finding that both oxy-Hb and StO2 decreased drastically when running a train under normal conditions is an indication that there are few driving operations required in such conditions, which cause a decline in concentration. We also confirmed that oxy-Hb and StO2 tended to increase when participants discovered an obstacle and performed a drastic deceleration. This trend implies that brain activity levels rose after detecting an obstacle on the tracks in order to be able to react to it.
Next, the average notch and average train speed for each participant in the trials with advanced notice of the obstacles appearance are shown in Figure 14(a). The averaged NIRS signals (1ch) from this time are shown in Figure 14(b). As with the experiment with advanced notice, the participant who stopped the nearest (865 m) was used as the standard for analysis.
When drivers were informed before the appearance of the obstacle, it can be concluded that they were focused and, therefore, were able to prevent a collision. Furthermore, the results of the measurement of brain activity in Figure 13(b) show that oxy-Hb and StO2 tended to decrease when accelerating. However, the change in oxy-Hb was lower than for Figure 14(b) in which there was advanced notice.
A previous study by Sakai and Kato confirmed that oxy-Hb value tends to increase when attention levels are high . As such, it can be concluded that participants were driving with higher attention during the obstacle avoidance task. Furthermore, the measurement was regarded as successful and was not influenced by artifacts due to the high correlation confirmed between oxygen saturation and oxy-Hb.
From the brain function images in Figure 11, no major changes in the brain are visible while at rest in normal driving task. When driving, there was a monotone decreasing trend in oxy-Hb only for the acceleration operation. The stopping operation during the deceleration operation was particularly complex. Thus, it is considered that there was bilateral activation of the lateral area due to learning the operation.
In a previous study, it was confirmed that bilateral activation of the dorsolateral prefrontal area is seen when learning an operation, while activation of the medial area is not, in a large-scale multichannel NIRS study of train drivers . The increases of the neuronal activity in the dorsolateral prefrontal area can be related to the connection between the dorsolateral prefrontal area and the cerebellum. In the cerebellum, an internal model is constructed so that the brain manages driving effectively. The evidence of this fact has been shown by the experiment of train driving training for 6 months for a beginner driver. As the time goes by and driving performance improved, the neuronal activity in the dorsolateral prefrontal area decreased.
In the trial performed without advanced notice of the obstacle in the obstacle avoidance task, no participant was able to safely stop the train after discovering the obstacle. In contrast, all participants were able to stop directly before the obstacle when given advanced notice, and oxy-Hb was higher in this situation. This is likely because participants were on alert for the appearance of an obstacle, in addition to normal driving operations. This trend was seen in all participants, which demonstrates that the attentional state of train drivers can be evaluated using wearable NIRS.
7. Field test
We investigated whether the wearable NIRS (Hb131S) can be applied in real train operations. A field test was carried out at a train depot of a railway operator. The maximum speed of the test line was 15 km/h. Figure 15 shows the train operation in the test line.
Measured raw data using wearable NIRS (Hb131S) was obtained from a train driver and is shown in Figure 16. The measured signals were processed to remove high-frequency noise, and the baseline correction between the tasks was performed. The signal processed oxy-Hb and handle notch position are shown in Figure 17.
It can be seen from Figure 16 that oxy-Hb and StO2 increase at braking area. As we can see a good correlation between oxy-Hb and StO2, the measurement using wearable NIRS (Hb131S) was successful. It should be noted from Figure 17 that oxy-Hb increases when the train driver operates the handle notch for train speed control particularly in the braking area.
This study aimed to examine the feasibility of monitoring the brain activity of train drivers using wearable NIRS. We investigated whether their brain activity could be measured using wearable NIRS, as well as whether their attentional states could be evaluated.
First, a general train-stopping task was carried out and multichannel NIRS compared to wearable NIRS. It was found that both NIRS devices confirmed elevation of oxy-Hb with a deceleration operation, which is consistent with earlier research. The wearable NIRS also confirmed elevation in oxy-Hb and StO2. Therefore, it was verified that the brain activity of train drivers can be measured using wearable NIRS.
Next, we carried out an obstacle avoidance task using wearable NIRS. Brain activity was measured in a condition with advanced notice of the obstacle and a condition without advanced notice to examine whether the attentional state could be evaluated. It was found that oxy-Hb was low prior to detecting an obstacle in the condition without notice. Oxy-Hb was higher in the condition with advanced notice. Therefore, it was proven possible to monitor attention levels from the brain activity of drivers using wearable NIRS.
Finally, we investigated whether the wearable NIRS can be applied in real train operations. A field test was carried out at a train depot of a railway operator. Field test results showed that the wearable NIRS can be applicable for measuring train drivers’ brain function in real world.
Most of the related studies have been carried out in laboratories using a train driving simulator. The purpose of this study is to evaluate the applicability of wearable NIRS for train drivers in real world. As the wearable NIRS system has been recently developed, we need to collect many NIRS data in real train driving so that we can evaluate the train drivers’ condition from NIRS data as our future work.
We are grateful to Ms. S. Makishima and Ms. M. Satozono for helping us with the field test and data analysis. We would like to thank Editage (www.editage.jp) for the English language editing.
Conflict of interest
The authors declare no conflict of interest.
|DLPFC||dorsolateral prefrontal cortex|
|fMRI||functional magnetic resonance imaging|
|fNIRS||functional near-infrared spectroscopy|
|SRS||spatial resolved spectroscopy|