Equivalent-Current-Dipole-Source-Localization-Based BCIs with Motor Imagery

1. Introduction

This chapter will propose a new paradigm for single-trial-electroencephalogram (EEG)-based Brain-Computer Interfaces (BCIs) with motor imagery (MI) [1] tasks. Among such BCIs, the sensorimotor rhythm (SMR)-based ones, when using common spatial patterns (CSPs), require features over broad frequency bands, such as mu, beta and gamma rhythms [2]. Therefore, very high-dimensional feature vectors and continuous-valued patterns necessary for spatiotemporally checking the features [3,4] could yield an enormous amount of data and much computational time [5]. So, various data reduction such as downsampling [6,7] and optimal EEG channel configuration [8,9,10] have been investigated for the BCIs.

The present method consists of 1) the categorization of single-trial EEGs as data reduction, and 2) the classifiers for the categorical data. 1) is realized by equivalent current dipole source localization (ECDL) after independent component analysis (ICA). For 2), we have been applying both Hayashi’s second method of quantification (H2MQ) and Bayesian network model (BNM) to the ECDL-based categorical data. For the former, we have obtained the good accuracy, for example, the accuracy average across all the ten subjects for left- and right-hand imageries in each 10-trial validation was more than 90 % [11].

This chapter addresses itself to the single-trial-EEG-based BCI using the BNM and to the generalization to dynamic BNM (DBNM) because of the time-varying functional networks in the brain. For the purposes, two experiments were conducted to obtain single-trial EEGs scalp-recorded during the MI tasks and movement-related potentials (MRPs) including the Bereitschaftspotential (BP) [12].

Recently, neuroscience has been attempting to take in various methodologies in network science, because the brain could be considered to be a kind of complex systems forming networks of interacting components, and the collective actions of the components, that is, individual neurons, linked by a dense web of intricate connectivity [13]. In addition to the network approach, one of the applied researches in neuroscience, the BCI, has extensively received probabilistic approaches whose aims are mainly two. One is to cope with non-stationarities in EEG signals such intertrial and intersubject variations, and the other to incorporate time-varying brain states and uncertainties into BCI design. For the former aim, adaptive classifications were executed by Kalman filtering [14,15], while the DBN achieved the latter one [16]. Micheloyannis et al. [17] analyzed multi-channel EEGs using graph theory. However, because the nodes are electrode positions, they have few functional meanings. In addition, all the above methods had been throughout applied to continuous-valued data.

A BN could be one of graphical models in neuroscience, where brain connectivity would be quantified by conditional probabilities. However, the existing graphical models require a large-scale anatomical data [18] and huge quantities of diffusion MRI data [19,20]. In this study, the brain connectivity will be calculated from the neural activity data even less than that in the above graphical models.

Figure 1 shows a typical BN, showing both the topology and the conditional probability tables (CPTs), given the joint probability distribution:

where X_i (i=1,…,5) (nodes) are random variables whose values could be 0 or 1, and B_S represents the BN topology, and Table 1 depicts 20 sample data generated from the BN model (BNM) [21]. The BN construction refers to that the topology of BNMs is estimated from such data in Table 1. In this study, BNM and DBNM consist of functionally distinct sites of the brain as nodes and directed relationships among these sites as edges, each of which is accompanied by conditional probabilities.

In order to predict the tasks which would have been executed by the subjects, in particular to discriminate between left- and right-hand imageries, the conditional probabilities at all the nodes in the BNM must be calculated for each trial. For the purpose, the probabilistic inference is made by the belief propagation [21], where the ECDL results for each trial correspond to the evidences. Hereafter, node activitiesare defined to be the summation of conditional probabilities at each node. Based on the node activities, a rule is proposed to classify into left- and right-hand imageries. This classification rule will be examined by the statistical tests.

Moreover, our method will be validated and compared with the existing one with the best performance, called common spatial pattern (CSP) [22]. In the section 3, MRPs will be modeled by the DBNM. Finally, we will mainly mention future perspectives.

Figure 1.

An example for causal model of Bayesian Network [21].

node number
	1	2	3	4	5
1	1	0	1	1	1
2	0	0	0	1	0
3	1	1	1	1	1
4	1	1	1	1	0
5	1	1	1	1	0
6	1	1	0	0	0
7	1	0	0	0	0
8	0	0	0	0	0
9	0	0	0	0	0
10	0	1	1	1	1
11	0	0	0	0	0
12	1	1	1	1	1
13	0	0	1	1	1
14	1	1	1	1	1
15	1	0	1	1	1
16	1	1	1	0	0
17	1	1	0	1	0
18	1	1	1	0	0
19	1	1	0	1	0
20	1	1	1	0	1
ave	.7	.6	.6	.6	.4

Table 1.

Data example [21] generated by the BN shown in Figure 1.

2. Bayesian network models for single-trial-EEG-based BCI

2.1. Materials and methods

Subjects.Ten healthy subjects (two females and eight males; mean age: 28.4 ± 4.27 years) participated in this experiment. All the subjects were right-handed according to the Edinburgh Inventory [23]. Some of the subjects were paid volunteers and received 2000 Yen (about US 25$).

EEG data acquisition.The subjects were seated inside an electrically shielded room with sound attenuation, and gazed at a monochromatic monitor of an AV tachistscope (IS-701B, IWATSU ISEL) 0.9 m away from their eyes. They were requested to relax their both hands on a table and with their chins on a chinrest (Figure 2A). The present experiment used a visual oddball paradigm, and three kinds of line drawings of hands were presented on the monitor: (1) right-hand stimulus to imagine being shaken with the subject’s right hand, (2) left-hand one for the subject’s left hand imagery and (3) open-right-hand one as control (Fig.2B). These stimuli were sequentially and randomly presented with probabilities of 0.20, 0.20 and 0.60, respectively. That is, (1) and (2) are rare targets or rare non-targets, and (3) is frequent non-target. We tried four conditions consisting of movement imagery of right hand, left one and the actual movements, where each condition was separately carried out. Names of the conditions and the instruction to subjects were as follows: R-MIRP (right-hand-movement-imagery-related potential) condition is to imagine grasping the right-hand stimulus with her or his own right hand as soon as possible when the stimulus was displayed; L-MIRP (left-hand-MIRP) condition to image grasping the left-hand stimulus with her or his own when the stimulus was presented; R-MRP (right-hand-movement-related potential) condition to actually grasp and loosen her or his own right palm as soon as possible if right-hand stimuli as the rare targets were displayed; L-MRP (left-hand-MRP) condition to grasp and loosen her or his palm when left-hand stimuli as the rare targets were presented. Both hands were hidden under a black coverlet so that it is easier for the subjects to imagine the hand movement. There was the following training session. At first, each subject was instructed to actually reach the monitor for the line drawing, then to shake (the stimulus) by her or his hand, and to practice the task scores of times. Then, after covering both of the hands with the coverlet, the subject was requested to imagine being shaken and to practice the MI tasks scores of times. One test session includes all the four conditions with a five-minute break between the conditions, where each condition contains 130, 130 and 400 trials of rare targets, rare non-targets and frequent non-targets, respectively. Therefore, it took about 90 minutes to finish one session. Note that different subject had different order of the conditions. This study addressed itself to only the L- and R-MIRP conditions, while the L- and R-MRP ones will be used in our another research in future.

Figure 2.

Experimental design: (A) EEG, EOG, EMG and electrode position measurement and stimulus presentation; (B) stimulus contents; (C) time-scheduling of the stimulation and the measurement of EEG, EOG and EMG.

With an electro cap (ECI, Electrocap International), EEG was from 32 electrodes (FP1, FPz, FP3, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T3, C3, Cz, C4, T4, CP5, CP1, CPz, CP2, CP6, T5, P3, Pz, P4, T6, PO3, POz, PO4, O1, Oz, O2) defined on the basis of the International 10-20 System [24]. All the electrodes were referred to A1, the ground electrode was attached to FPz and their impedances were kept below 5kΩ. Vertical and horizontal eye movements were monitored with two electrodes placed directly above the nasion and the outer canthus of the right eye as electrooculogram (EOG). Another two electrodes were placed at both the medial antibrachiums to record arm electromyogram (EMG) so that EEGs could be excluded when mistakenly grasping during the movement imagery.

The 32 signals of the EEGs were amplified by a Biotop 6R12-4 amplifier (GE Marquette Medical Systems Japan, Ltd.), and filtered a frequency bandwidth of 0.01-100 Hz. The amplified signals were sampled at a rate of 1 kHz during an epoch of 100 ms preceding and 700 ms following the stimulus onset. The inter-stimulus interval (ISI) was 1600 ms (Figure 2C). The on-line A/D converted EEG signals were immediately stored on a hard disk in a PC-9821Xt personal computer (PC) (NEC Corporation). The EOG and EMG data were also amplified by a Polygraph 360 amplifier (GE Marquette Medical Systems Japan, Ltd.), and sent to the same PC.

Independent component analysis (ICA).Fast ICA [25] was applied to each single-trial EEG in the L- and R-MIRP conditions, using ICALAB [26]. After ICA for each trial, among 32 ICs, we removed those associated with eye movement and line noise, and having a broader high frequency (50-100 Hz) spectrum that might be likely to be generated by scalp muscles [27], according to the spectra of all the ICs for each trial. Consequently, the following ECDL was applied to about 20 ICs containing only neural activity.

Equivalent current dipole source localization (ECDL).Independent EEG sources obtained by ICA are dipolar [28]. ECDL was applied to the reconstructed EEGs, namely the projection of each IC on the scalp surface by the deflation procedure, using “SynaCenterPro” (PC-based commercial software for multiple ECDL) (NEC Corporation). This software estimates unconstrained dipoles [29] at any timepoint, using the three-layered concentric sphere head model by the nonlinear optimization methods [30]. An unconstrained dipole was estimated at any timepoint with maximal peak or trough in the reconstructed EEGs for each IC. Here, we searched for appropriate and reliable dipole solutions that had goodness of fit (GOF) of more than 90 % and the simplified confidence limit [31] of less than 1 mm and had stable localization to the same brain site around the peak or trough. The brain sites, where dipoles were located, were determined by inspection with reference to the textbook of neuroanatomy (e.g., [32]). The inspectors had a good knowledge of neuroanatomy. Thus, one brain site was assigned to each IC.

Bayesian network model (BNM) construction.The present BNM consists of functionally distinct sites of the brain as nodes and directed relationships among these sites as edges. Nodes of the BNM are the brain sites where ECDs were located by the ECDL method. The BNM structure was constructed by the conditional independency (CI) test [33]. The BNMs obtained for each subject had fifteen nodes corresponding to the brain sites such as the frontal, temporal, occipital and cingulate gyri, hippocampus, insula, left and right parietal cortices, left and right motor areas, left and right cerebellum, left and right somatosensory areas, and others. The BNM initialization was made so that there are edges from the mesial prefrontal areas to the premotor and primary motor cortices because the early BP begins in the pre-supplementary motor area (preSMA) and the SMA proper and then in the premotor cortex, and the late BP (NS’) occurs in the primary motor and premotor cortices [12].

Probabilistic inference for classification rule.In order to discriminate between left- and right-hand imageries, the conditional probabilities at all the nodes in the BNM must be calculated for each trial. For the purpose, the probabilistic inference is made by the belief propagation using the clique tree algorithm [34], where the ECDL results for each trial correspond to the evidences. A free software, “MSBNx” (Microsoft Research) [35] enables this inference. On the basis of the node activities, a rule was proposed to classify into left- and right-hand imageries.

2.2. Results

Figure 3 shows an example for a series of our results for one trial by one subject: (A) 32-channel raw EEG data; (B) ICA results (showing the first 10 ICs); (C) deflation for the 10^th IC; (D) ECDL result, where when to be estimated is depicted by a black arrow at the 11^th reconstructed EEG in (C) and the same time is for the rest EEGs, and one localized dipole is represented by a blue arrow in (D). This figure exemplifies that the dipole was located at the left pre-central gyrus, for the single-trial EEG recorded during the right-hand movement imagery task. Table 2 illustrates a summary for these categorized ECDL results, including the same data as in Figure 3. That is, this table indicates that the 10^th IC, after the deflation procedure, from the single-trial EEGs recorded during the 7^th right-hand movement imagery task, was localized to the left motor area (the pre-central gyrus), where n1 to n15 correspond to the above 15 brain sites, in particular n1 the left motor area.

2.3. BN structure

Figure 4 shows a representative BNM constructed for subject 1, who satisfied the classification rule mentioned later. The BNM construction required each about 30 trials of the left-hand and right-hand-movement imageries. This number will be confirmed later. The BNM for subject 1 directs the link from the “occipital” node to the “frontal” one via the cingulate gyrus, and that from the “frontal” node to directly to the “left motor area” one and, to the “right motor area” one via the somatosensory cortex. The “frontal” node contains the superior and inferior frontal gyri. The “motor area” node includes the primary motor and premotor cortices. This BNM might reveal the neural network involving from the visual stimulus input to the movement imageries.

Figure 3.

A series of the present results for one trial by one subject: (A) raw EEG data; (B) ICA; (C) deflation; (D) ECDL.

2.4. Node activities

A BN topology could be determined from the ECDL results on all the trials for each subject. For all the nodes, however, the connectivity between any two nodes, that is, the conditional probability cannot be estimated from even all the ECDL results for each subject. Especially, for any one trial, the ECDL results do not necessarily enrich all the nodes in the subject’s BN. However, the probabilistic inference could enrich all the nodes for each trial in terms of the conditional probabilities.

Assuming that the summation of these conditional probabilities in each node reflects the neural activity of the node, the node activity was calculated for each trial. Moreover, we focused on the “left and right motor areas” nodes, because there have been many findings about the involvement of the primary and/or premotor cortices during the motor imagery [36-41]. The t-test concerning the mean of the node activities across the trials for each subject was as follows. For the right-hand imagery, there were significant differences in the node activities between the “left and right motor areas” nodes for subjects 1, 2, 3, 4, 5, 6 and 9 (t(51)=2.41, p=0.0193; t(59)=-2.81, p = 0.00672; t(60)=-7.27, p=1.05E-09; t(54)=-2.18, p=0.0336; t(58)=-2.77, p=0.00754; t(57)=5.11, p=3.90E-06; t(50)=2.19, p=0.0330, respectively). On the other hand, for the left-hand imagery, there were no differences for subjects 1, 2, 4, 5, 7, 8 and 10 (p>0.05). These findings might lead us to one possibility of classification rules to discriminate between left and right hand to be imagined for our single-trial-EEG-based BCI. That is, for right-handed subjects, there is a significant difference in the node activities between left and right “motor area” nodes during the right-hand-movement imagery, while no difference during the left-hand-movement one.

Figure 4.

Bayesian network model of motor imagery for subject 1

Table 2.

A summary for categorized ECDL results. “L” and “R” represent the left- and right-hand movement imageries, respectively. “IC”s independent components and n1 to n15 the 15 brain sites (see the text in more details).

2.5. Discussion

2.5.1. The present BNMs supported by the topological map of human cortical network

For subjects 1, 3 to 6

The BNMs of subjects 3 to 6 are not here.

, the paths to left and right “motor area” nodes were consistent with the existing topological map of human cortical network. For example, “frontal” → “right somatosensory” and “occipital” → “right parietal” are exemplified by MFG.R – PoCG.R and SOG.R – ANG.R in [20] (p.530, Fig.4), respectively.

2.5.2. Classification rule

Subjects 1, 2, 4 and 5 perfectly met the classification rule proposed above. This rule, which might show bilaterally non-symmetrical event-related desynchronization (ERD) patterns [42], contrary to that of Qin et al. [43], is strongly supported by Bai et al. [44].

2.5.3. Number of trials for BNM learning

Figure 5 shows the number of trials necessary to satisfy with the rule for subject 1, plotting the p-values in the t-test, as functions of the number of trials, for subject 1. The t-test examines difference in node activities between the “left and right motor areas” nodes for the left- and right-hand imageries. Figure 5 depicts the p-values for every 5 trials. This figure demonstrates that the above rule is effective after about 25 trials. That is, the present BNM learning for BCI requires about 25 trials.

Figure 5.

P-values in the t-test concerning the differences in conditional probabilities between the “left and right motor areas” nodes, as a function of number of trials.

2.5.4. Comparison with CSP

The present BCI based on the classification rule was validated and compared with the common spatial pattern (CSP) method [22]. The test data includes 20 trials with left- and right-hand movement imageries. In our BCI after the probabilistic inference for each trial, on the basis of the classification rule, if “the left motor areas” node activity are significantly different from “the right motor areas” one, the trial was judged to be a right hand imagery, while both of the node activities are not so different, the trial a left hand imagery.

The CSP is an algorithm for obtaining a spatial filter to transform multi-channel EEG data with two conditions into the surrogate space enabling the optimal discrimination of the conditions. This filtering is achieved by solving the generalized eigenvalue problem for the estimates of the covariance matrices of the band-pass filtered EEG signal. For each trial, 1-dimensional feature is calculated after operating the spatial filter on the single-trial EEG. From these features, the threshold is determined so that all the trials for the learning are optimally discriminated between the two conditions as exemplified in Figure 6. Thus, for each of α, μ, β and γ frequency bands, we conducted one CSP classifier with its threshold, using the same EEG data as in the subject 1’s BNM (Figure 4) construction. Finally, for subjects 1 and 2 among ones who satisfied with the proposed classification rule, the accuracy of the present BNM and the 1-CSP classifier was 90 % and 75 %, and 85 % and 70 %, respectively.

Figure 6.

Principle of 1-CSP classifier. ζ_i is a feature value for thei^th trial, andbis the threshold.

3. Generalization to dynamic BNM

3.1. Materials and methods

Acquisition of single-trial EEGs.This experiment was carried out to obtain MRPs with BP in term of single-trial EEGs. Subjects reclined in an EEG chair in an electrically shielded room, and gazed at a display 90 cm away from their eyes (Figure 7). For any one trial, on the display, “choice” was presented at first, “+” was presented 3.5 s after “choice” and “report” was presented 2 s after “+” in turn. The duration of “choice” was 2 s, that of “+” was 0.2 s and that of “report” was 2.5 s (Figure 8). In case of “left” or “right” selection in “choice”, the subjects were requested to grasp their hand of the same side as the selection as soon as possible when “+” was presented, and then to speak aloud the selection in “report”. Otherwise, there was no grasping and no speaking. During this task, 32-channel single-trial EEGs, EMG and EOG were recorded. This section reports results on only the same subject as in the section 2 (subject 1).

Division of the EEGs.According to Shibasaki and Hallett [12], the EEG data was divided into three intervals. That is, assuming that the time when the onset of the EMG is 0 ms, the interval between -1700 and -400 ms refers to early-BP (E-BP), that between -400 and 0 ms NS’ (negative slope) and that between 0 and 200 ms MP (motor potential).

ICA, ECDL then BNM and DBNM construction and probabilistic inference.For each interval, ECDL was applied to single-trial EEGs after ICA. For each interval, using the categorical data concerning the ECDL results on both the “left” and “right” tasks, each including 10 trials, a BNM structure was determined by the CI test. Moreover, connecting with the frontal node on the top of each BN led to a DBNM. Then, the probabilistic inference enriched all the nodes of the DBNM for each trial, in terms of conditional probabilities.

Figure 7.

EEG measurement and stimulus presentation system.

Figure 8.

Time-scheduling of the stimulus presentation, the task and the EEG measurement.

Figure 9.

E-BP BNM, where open rectangles and arrows depict nodes and edges, respectively.

3.2. Results and discussion

Figure 9 shows a BNM obtained for the E-BP. This BNM is also consistent with the topological map of human cortical network (for example, “cingulate gyrus” →“left motor area” was found in ACG.L-SMA.L-PreCG.L [20]), and reflects the previous neurophysiological findings (for example, “hippocampus”→“left/right cerebellum” might be explained by that lesions of the cerebellar nuclei abolish conditioned increases in hippocampal CA1 neural activity evoked by the tone-conditioned stimulus [45]). The DBNM, shown in Figure 10, contains the neural generators for the MRPs. Namely, the neural generator of the E-BP is the pre-SMA at first, next the SMA and then the premotor cortex, that of the NS’ the premotor cortex at the first and next the motor cortex and that of the MP the somatosensory cortex [8]. In the E-BP BNM, there is no difference in node activities between the “left- and right-motor area” nodes for the left- (t(14)=-2.0018, p=0.06507) and right-hand (t(16)=-1.0849, p=0.294) movement. There was also the same tendency (left-hand movement: t(14)=-1.0832, p=0.297; right-hand one: t(16)=-0.1059, p=0.917) in the NS’ BNM. In the MP-BNM, however, there were significant differences between the two nodes both for the right-hand movement (t(16)=-3.9817, p=0.001072) and for the left-hand one (t(14)=-2.2532, p=0.04081). These findings suggest that there may be differences in neural network connectivity between the MP BNM and the others. On the other hand, also in the present DBNM, there were significant differences between the two nodes both for the left-hand movement (t(46)=-2.9645, p=0.004791) and for the right-hand one (t(52)=-2.6109, p=0.01177). The difference between the DBNM and the BNM obtained in the section 2 might not reflect only that in the neural connectivity but also that in the tasks, that is, the MI and the actual movement.

Figure 10.

DBNM including (A) E-BP, (B) NS’ and (C) MP.

Figure 11.

An example of a system for automatically specifying the brain sites where estimated ECDs are located.

4. Consideration and conclusion

In this chapter, we have proposed a new framework for single-trial-EEG-based BCIs with the MI tasks. This framework consists of the categorization of the EEGs, which could lead us to data reduction, and the classifiers for the categorical data. The ECDL after ICA enabled us the former. For the latter, the classifier using Hayashi’s second method of quantification has yielded the accuracy of more than 90 % [11]. This chapter has concentrated on another classifier for the categorical data, called Bayesian networks. The present BCI learning required 25 trials at least. Although for the results on only two subjects, the accuracy in 20-trial validation was higher than that by the CSP, in addition to exceeding the existing ECDL-based BCIs [43,46]. If any subject met the classification rule, the subject would be expected to achieve the good accuracy.

To our knowledge, Shenoy and Rao [16] made the first report of the application of DBNM to BCIs. Although the DBN allowed continuous tracking and prediction of the brain states over time, it included hidden but ambiguous state variables. Our DBNM has revealed the difference among the E-BP, NS’ and MP in MRPs and that between the MI and the actual movement tasks. However, the DBNMs for BCIs are still in the data-processing stage, not in the validation one.

When obtaining ECDL results on 20 ICs for each of 50 (=25x2) trials by one subject, inspection with reference to the textbook of neuroanatomy must be iterated 1000 (=20x50) times, at least. Even if the inspector has the full knowledge of neuroanatomy, different inspectors might yield different ECDL results. In order to cope with this problem, we are developing a computer system for automatically specifying the brain sites for ECDL results on each individual with MR images (Figure 11) [47]. However, we cannot directly utilize the Talairach-Tournoux brain atlas [48], because there are big differences in the brain shape between Westerners and Asians, in particular Japanese. Therefore, we are now constructing the brain atlas for Japanese.

Acknowledgments

This research was partly supported by a Grants-in-Aid for Scientific Research on Scientific Research (B) (20300196) - The Japan Society for the Promotion of Science.