A Self-Paced Two-State Mental Task-Based Brain-Computer Interface with Few EEG Channels

Abstract

A self-paced brain-computer interface (BCI) system that is activated by mental tasks is introduced. The BCI’s output has two operational states, the active state and the inactive state, and is activated by designated mental tasks performed by the user. The BCI could be operated using several EEG brain electrodes (channels) or only few (i.e., five or seven channels) at a small loss in performance. The performance is evaluated on a dataset we have collected from four subjects while performing one of the four different mental tasks. The dataset contains the signals of 29 EEG electrodes distributed over the scalp. The five and seven highly discriminatory channels are selected using two different methods proposed in the paper. The signal processing structure of the interface is computationally simple. The features used are the scalar autoregressive coefficients. Classification is based on the quadratic discriminant analysis. Model selection and testing procedures are accomplished via cross-validation. The results are highly promising in terms of the rates of false and true positives. The false-positive rates reach zero, while the true-positive rates are sufficiently high, i.e., 54.60 and 59.98% for the 5-channel and 7-channel systems, respectively.

1. Introduction

Brain-computer interfaces (BCIs) aim at providing an alternative means of communication for motor-disabled people suffering from diseases such as brain injury, brainstem stroke, high-level spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS, also known as Maladie de Charcot or Lou Gehrig’s disease), muscular dystrophies, multiple sclerosis (MS), cerebral palsy (CP), or locked-in syndrome (sometimes called ventral pontine syndrome, cerebromedullospinal disconnection, pseudocoma, and de-efferented state). Well-developed BCI systems are used by motor-disabled people to control their environment. They can also be used by healthy individuals for entertainment purposes such as playing computer games.

Existing BCI systems are categorized in two major classes: system-paced (or synchronous) and self-paced (or asynchronous). In system-paced BCIs, the user can only control the BCI during specific time intervals that are predefined by the system and not by the user. A self-paced BCI, on the other hand, can be available for control by the user at all times. It is clear that the second class is better and more efficient in terms of practicality and applicability to real-life applications.

Two types of states (or modes) are usually assumed for the output of a self-paced BCI: the no-control (NC) state and the intentional control (IC) state. This type of BCIs is in the NC mode most of the time. However, when the user issues a mental command that would lead, for example, to switching the light on, moving the computer cursor to the right, etc., the system changes its state from the NC mode to the IC mode. After that, the BCI returns to the NC state.

Two measures that can properly evaluate the performance of a self-paced BCI are the true-positive rate (TPR) and the false-positive rate (FPR). A true-positive outcome results from correctly classifying a command as an IC state, and a false-positive outcome results from the BCI misclassifying a no control as an IC state. The ratios of true positives and false positives to the total number of classifications yield the TPR and FPR, respectively. Further details on BCIs can be found in [1, 2, 3, 4, 5].

Due to the high false activation rates, BCI systems are deemed unsuccessful for use in real-life applications. This is because false activations are a major cause of user frustration. To further illustrate this point, suppose that the output rate of a self-paced BCI is 5 Hz (i.e., five outputs/s, as in the BCI designed in this paper) and the FPR value is 1%. This FPR of 1% means one false positive in every 100 outputs of the BCI. As the BCI generates 100 outputs in 20 s, there would be three false activations in every minute, which is too high for practical purposes. Considering the fact that a self-paced BCI is in the no-control mode for most of the time, even a low FPR would greatly annoy and frustrate any user. This is why for BCI systems, lowering the FPR is of extreme importance.

Mental tasks are a class of neurological phenomena that can be exploited in BCI systems. They generally refer to intentional cognitive tasks that are done by the brain. Mental tasks can be mental mathematical calculations (such as multiplication and counting), mental rotation of a two- or three-dimensional object, motor imagery, visualization, etc.

Motor imagery is a task that has been investigated by a large majority of BCI studies [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64]. The use of other types of mental tasks in BCI studies has received little attention in the literature. The papers that have studied non-motor imagery mental tasks along with the motor imagery tasks include [65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84]. Studies [85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96] have only considered non-motor imagery mental tasks.

The mental tasks investigated in [65] are motor imagery of opening and closing of the users’ hand(s) and serially subtracting seven from a large number. In [66], four motor imagery tasks of (the users) left hand movement, right hand movement, foot movement, and tongue movement along with a simple calculation task (i.e., repeated subtraction of a constant number from a randomly chosen number) are considered as the mental tasks. The mental tasks used in [67] are the imagination of left and right hand movements, cube rotation, and subtraction. Four imagery tasks, i.e., spatial navigation around a familiar environment, auditory imagery of a familiar tune, and right and left motor imageries of opening and closing the hand, are investigated in [68]. In [69, 70], the imagination of left and right hand (or arm) movements, cube rotation, subtraction, and word association are studied. In [71, 72, 75, 76, 77, 79, 81], the imagination of repetitive self-paced left or right hand movement and the generation of words beginning with the same random letter are investigated. The EEG data used in these studies are those provided by the IDIAP Research Institute in Switzerland [70]. The studies in [77, 81] consider the imagination of the left or right hand movement as well using the data collected by the BCI laboratory at Graz University of Technology in Austria [16]. In [73], the mental tasks are auditory recall, mental navigation, sensorimotor attention of the left hand, sensorimotor attention of the right hand, mental calculation, imaginary movement of the left hand, and imaginary movement of the right hand. The mental tasks used in [74] are the exact calculation of repetitive additions, imagination of left finger movement, mental rotation of a cube, and evocation of a nonverbal audio signal. In [78], the right and left hand extension motor imageries, subtraction, navigation imagery, auditory imagery, phone imagery, and idle task are investigated. The mental tasks considered in [80] include the right and left hand flexion motor imageries, subtraction, navigation imagery, auditory imagery, phone imagery, and idle task. The mental tasks considered in [82] are subtraction, navigation imagery, auditory recall, phone imagery, and motor imageries of the left and right hands. Hand movements and word imagination are the mental tasks used in [83]. Imagination of left and right hand movements, mental rotation of a 3D geometric figure, and mental subtraction of a two-digit number from a three-digit number are considered in [84].

The old and small datasets of Keirn and Aunon [85] that contain non-motor imagery mental tasks are employed in [86, 87, 88, 89, 90, 92, 94, 95, 96]. Vowel speech imagery (i.e., imaginary speech of the two English vowels /a/ and /u/) is proposed as a control scheme for the BCI system in [91]. In [93], the mental arithmetic and spatial imageries are investigated. Real fist rotation and imagined reverse counting are investigated in [97].

From all BCI systems designed in these studies, only the systems in [19, 24, 32, 39, 42, 46, 57, 61, 67, 69, 70] and [76, 78, 80, 82] are self-paced. The FPR values are not reported in [57, 69, 70, 76, 78, 80, 82]. Even though the number of FPs and TPs is mentioned in [32, 46], the rates of FPs and TPs are not given.

The FPRs are given in [19, 24, 42, 61]. In [19], the given FPR values are in the 10–77% range. In [24], the BCI system was evaluated in terms of FPs during only one 3-minute interval. No FPs were generated during this interval; however, since the designed BCI is too slow, it is deemed impractical for the real-life applications. The minimum time period between two subsequent active states of the system is 4 s. In [42], the FPRs of the BCI systems are between 3.8% and 32.5%. In [61], the reported false activation rate is in the range of 0–3.25 activations/minute.

In [39], the specificity rates (i.e., 100–FPR%) are given. Based on the specificity rates, the FPR values are between 0.38 and 14.38%. Based on the confusion matrices given in [67], the FPRs are in the range of 0–9%.

The ultimate and first goal of conducting this study is to develop a self-paced two-state mental task-based BCI with a zero or near-zero false activation rate using EEG signals. The mental tasks investigated are the visualization of some words as they are written, multiplication, mentally rotating a 3D object, and motor imagery. We collected the EEG signals of these four tasks as they were being performed mentally and also during the baseline state, i.e., when the subjects were not performing any of the four mental tasks as will be explained in Section 2. The number of EEG channels used was 29. The details on each mental task and the dataset are provided in the next section where the experimental protocol is described.

The second goal is to design a BCI with few channels. Such a BCI has few electrodes to collect the EEG signals and would be significantly more efficient computationally, leading to BCIs that operate in real time. Thus, for practical applications, the number of EEG channels should be small. In Section 3.1, we discuss how we choose five or seven channels that would yield acceptable performance.

For each subject, four different BCIs are developed. Each BCI is based on one of the four mental tasks mentioned above, i.e., in each BCI, one mental task is considered as the IC task (i.e., the user is indeed issuing a command). The other three mental tasks are considered as NC tasks. The BCI system should remain in the NC mode during the NC tasks and the baseline.

Even though the system performance is evaluated off-line, the EEG signals are analyzed in a self-paced manner. A signal trial is divided into overlapping segments. Each segment is labeled as either IC or NC, depending on whether or not it belongs to the IC task. The performance of the system is then evaluated in terms of TPR and FPR.

Classification is based on the quadratic discriminant analysis due to its simplicity and accuracy. The features to be classified are the scalar autoregressive (AR) coefficients of the EEG signals. The feature extraction and the classification methods employed are efficient in terms of computational complexity. The cross-validation process is performed so as to obtain the optimal order of AR coefficients as well as the best EEG channels for every mental task of every subject.

2. Dataset

The EEG signals of four subjects were collected while they were seated in a chair approximately 75 cm in front of an LCD monitor in a 4 $\times$ 4 m² room. The subjects were asked to keep their eyes open during the recordings. Using an electrode cap, the signals were captured from 29 channels located at Fpz, AF3, AF4, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, PO3, PO4, and Oz, according to the 10–10 system [98, 99]. Electrodes were properly distributed over the scalp to study the signals of the different brain regions. Refer to Figure 1 to see the electrode positions. The earlobes were electrically linked together and used as the reference. The EEG signals were amplified and digitized using a 12-bit analog-to-digital converter. The sampling rate was 500 Hz.

Figure 1.

EEG signals were recorded from 29 electrodes distributed over the scalp according to the 10–10 system.

Every subject attended three recording sessions on 3 different days. They were asked to perform four mental tasks. Each session started with the preparation and setup and consisted of six recording runs. Twelve minutes of EEG signals were approximately recorded in a run. The subjects were instructed to complete the six runs one after the other at their own pace. Each run consisted of signals of 20 epochs (i.e., five epochs for each of the four mental tasks). An epoch was 32.5 $\pm$ 2.5 s long. To avoid possible adaptation, the order of epochs belonging to different mental tasks was changed randomly from one run to the other.

The timing of the epochs was as follows. At the beginning of each epoch, there was a break with a length of 15 $\pm$ 2.5 s. The break had a variable length in different epochs so as to avoid possible adaptation. A “Start” cue was displayed after the break on the screen to prompt the subject to perform a specific mental task. The task to be performed was shown on the screen. The length of the Start cue was 4 s. The subjects had been instructed to start to perform the mental task approximately 1 s after the disappearance of the Start cue and to keep performing the task until a “Stop” cue appeared on the screen. The 1-s delay was used to avoid any possible effects of visual evoked potentials. The time interval between the Start cue and the Stop cue was 10 s. The Stop cue lasted for 2.5 s. After the Stop cue, the break of the next epoch started. Figure 2 illustrates the timing of each epoch.

Figure 2.

Epoch timing. A “Start” cue was displayed on the screen for 4 s after a break of length of 15 $\pm$ 2.5 s. The subject was told to wait about 1 s after the cue disappeared before performing a mental task for about 10 s. The “Stop” cue was displayed on the screen for 2.5 s, informing the subject of the end of the 10-s interval. The next epoch then started.

The background of the screen was always black. During the break interval of each epoch, “Break” was written on the screen in white and in a size which could be easily read from a distance of 75 cm. The name of a specific mental task written in white was the Start cue. The size of the Start cue was the same as Break. The word “STOP” written in a green circle was the Stop cue.

The mental tasks were:

1. Visualizing some words being written on a board: subjects were told to imagine a board on which they were writing their full names.

2. Non-trivial multiplication: the subjects performed multiplication of two two-digit numbers. The numbers to be multiplied were given to them as the Start cue.

3. Mentally rotating a 3D object: the subjects imagined that they were rotating a laptop mentally.

4. Motor imagery: the subjects imagined extending his/her right hand.

The subjects were asked to be in the baseline state during the break interval of each epoch, i.e., they should not be performing any of the four mental tasks of the experiment and were supposed to remain looking at the screen and not move. They should attain the same physical condition as that assumed when they performed the mental tasks.

For each of the four mental tasks, 300 s (30 10-s epochs) of EEG signals and for the baseline, about 1800 s (120 epochs) of EEG signals were recorded in each session. Therefore, at the end of the last session, we had 90 epochs of each mental task and 360 epochs of the baseline for each subject.

The experimental protocol had been approved by the Behavioral Research Ethics Board of the University of British Columbia, and all subjects signed the required consent form.

3. Methodology

3.1 EEG channel selection

The dataset is formed by the signals from 29 electrodes. However, a BCI system with 29 channels is impractical for use in real-life applications. For practical applications, the number of channels should be as small as possible. We thus select a smaller set of the channels that together yield the best performance for the final design of our system.

Suppose that we have a BCI system with n channels and we need to select the BCI that has m channels (m < n) and yields the best performance. The ideal but not always computationally practical way is to consider all the possible m-channel combinations of the n channels and select the combination that yields the best system performance. For instance, if we want to decrease the number of channels from 29 to 7, the system performance needs to be evaluated for all 1,560,780 different 7-channel combinations and then compared. Moreover, in order to make the results more robust, the performance evaluation of the system is usually carried out for different training and testing sets via a cross-validation process. If we assume that the number of evaluation which runs in cross-validation is 5, the number above (1,560,780) should be multiplied by 5. This forms a prohibitively large amount of computations as the processing time will take several days. It is thus impractical for implementation.

In this study, two approaches are performed for selecting the best system that has m channels. The first approach deletes the channels (from among the 29-channel system) that results in the least reduction in the performance of the remaining system. The channels are deleted one by one. The second approach builds a new system by adding channels one by one to the newly built system. A channel added to the new system is selected from the 29-channel system so that the performance of the new system is maximal. These two approaches result in two methods that we denote as MDelete and MForm. Even though these methods are not optimal as the method mentioned above (i.e., considering all possible combinations), they still reach the goal to a certain extent.

Channel selection method one: in channel selection method one (MDelete), all 29 channels are first considered. The resultant FPR and TPR values after deleting each channel are obtained. The BCI system with 28 channels that yields the best performance is selected. That is, the channel whose removal results in the best 28-channel system is detected and deleted from the list. This task is repeated on the remaining list (i.e., on 28 channels), and the best 27-channel system is found. This is repeated again until all but m channels are omitted.

Channel selection method two: channel selection method two (MForm) is similar to the first one except that it is carried in the reverse direction and the selection of the channels differs as explained below. In the first iteration, the BCI system is assumed to have one channel only. The channel with the best performance among the 29 existing channels is thus detected. This will form the best BCI that has one channel only. In the second iteration, the BCI system is assumed to have two channels only. One of these two channels is the one already selected in iteration 1. Thus the performance of each channel in the remaining 28 channels, together with the channel selected in the first iteration, is obtained. Among these 28 possibilities, the channel (together with the already selected one) that yields the best performance is selected and added to the list of the best channels. In the third iteration, three channels are considered. These are formed by each channel from the remaining list (i.e., 27 channels) and the two channels already selected in the previous iterations. The channel that together with the two already selected channels yields the 3-channel BCI system with the best performance is added to the list. This procedure is repeated until m channels are added to the list of the best channels.

3.2 Procedure

The length of each mental task epoch is 10 s. The baseline epochs have a variable length in the range of 15 $\pm$ 2.5 s. Since the sampling frequency is 500 samples/s, each mental task epoch consists of 5000 samples, and the number of samples in a baseline epoch varies between 6250 and 8750.

To process the data, every epoch is divided into overlapping segments. Each segment is of length 1 s (i.e., 500 samples) and overlaps with the previous segment by 400 samples. In other words, the BCI system generates an output every 100 samples using the last 500 samples of the signals. Since 100 samples are equivalent to 0.2 s, the output rate of the BCI is 5 Hz.

Feature selection and classification: autoregressive (AR) modeling is used to obtain the features from the segments. Based on the results in [100, 101], 1 s of the EEG signal is sufficiently long for the AR model estimation. The feature vector is formed by concatenating the AR coefficients (estimated from the segments of the selected channels) into a single vector. This vector is then fed to the classifier for classification purposes. Classification is performed using quadratic discriminant analysis. The AR modeling and quadratic discriminant analysis are briefly explained in Appendices A and B of this paper.

Custom designing: custom designing the system for every subject yields improvements in the overall BCI performance [102, 103]. In this study, the BCI system is customized for each subject and for each mental task by selecting the channels and AR orders during cross-validation.

Cross-validation: to perform the cross-validation, we randomly divide the whole set of segments into five equal-sized sections. Four of the data sections are used to train and validate the system. Testing is carried on the remaining section. The four data sections assigned to training and validation are further divided randomly into five data partitions of equal size. Four partitions are used for training and one is used for validation.

Selecting the best five and seven channels: we select the top five and also the top seven best performing channels for future processing using MDelete and MForm with the AR model order of 40. We then compare the results of the 5-channel cases with those of the 7-channel cases to figure out the final design for the BCI system. Channel selection is accomplished separately for different subjects and different mental tasks. Tables 1 and 2 list the best channels selected using MDelete and MForm, respectively. For each subject and each mental task, each channel selected by both MDelete and MForm is shown in bold in the tables.

	Selected channels (from best to worst)
	1	2	3	4	5	6	7
Subject 1
Sentence visualization	T8	F8	F3	AF3	Oz	CP6	P7
Non-trivial multiplication	FC5	FC6	PO3	T7	C4	CP6	C3
3D object rotation	FC6	T8	F3	AF3	CP6	AF4	F8
Motor imagery	CP6	T8	P8	AF3	F7	FC2	T7
Subject 2
Sentence visualization	P8	FC6	T7	F3	PO4	P7	P3
Non-trivial multiplication	Oz	Fpz	Fz	T7	FC6	P3	C3
3D object rotation	Oz	T7	P3	FC5	Fz	C4	Cz
Motor imagery	CP2	Oz	F7	Fpz	P7	CP6	F4
Subject 3
Sentence visualization	PO3	P8	P4	AF4	T7	P7	C4
Non-trivial multiplication	Oz	Fpz	Pz	T7	CP1	PO4	CP5
3D object rotation	FC2	CP1	PO4	P8	Cz	F4	FC6
Motor imagery	T7	P7	CP2	Fpz	CP6	F7	FC6
Subject 4
Sentence visualization	P8	Cz	PO3	CP5	T7	PO4	AF3
Non-trivial multiplication	F4	CP5	T8	P7	P8	Oz	F3
3D object rotation	AF3	FC5	AF4	P7	C3	T8	PO4
Motor imagery	Cz	P8	P7	FC6	T7	F3	CP5

Table 1.

Channels selected for different subjects and mental tasks using channel selection method one (MDelete).

	Selected channels (from best to worst)
	1	2	3	4	5	6	7
Subject 1
Sentence visualization	T8	FC5	F8	P4	T7	P7	Oz
Non-trivial multiplication	F7	Oz	FC6	T8	F8	P7	CP1
3D object rotation	T7	FC6	P8	CP2	P7	Oz	F4
Motor imagery	Oz	P7	C3	CP6	T8	P8	FC6
Subject 2
Sentence visualization	P8	F7	FC6	T7	FC5	AF3	T8
Non-trivial multiplication	CP6	P4	T8	Fpz	C4	F7	FC6
3D object rotation	P7	P4	C4	Oz	FC6	AF4	F7
Motor imagery	CP5	Fz	FC6	T7	FC1	Oz	C4
Subject 3
Sentence visualization	PO3	Oz	Fpz	P8	P7	Pz	CP1
Non-trivial multiplication	Oz	P8	Fpz	P7	Cz	T8	AF3
3D object rotation	CP5	FC2	P7	P4	T7	Oz	Fz
Motor imagery	T8	Oz	P3	PO3	P7	AF4	AF3
Subject 4
Sentence visualization	CP6	P8	FC5	AF4	T7	CP5	Oz
Non-trivial multiplication	AF4	F7	T8	P8	P7	T7	Oz
3D object rotation	F3	F7	AF4	PO3	FC5	Oz	C3
Motor imagery	CP2	P3	AF4	F8	P8	F3	F7

Table 2.

Channels selected for different subjects and mental tasks using channel selection method two (MForm).

MDelete and MForm give BCI systems with different channel combinations. This is because each of these methods obtains a channel set which is locally optimum. The globally optimum set can be obtained by the exhaustive method mentioned earlier in which the best combination is selected by considering all possible combinations of channels. Finding the global optimum is computationally impossible for our application. We hence have to be satisfied with the local optima.

Finding the optimum AR model order: after selecting the best five and the best seven channels for each subject and each mental task, we find the optimum AR model order for each of these two cases in the cross-validation process. The initial AR model order is equal to 41. If the FPR for an order reaches zero, that order is selected as the optimum order; if not, the order is increased by one, and the FPR of the new order is then calculated. Increasing the AR order is terminated once the FPR is zero or a maximum order of 136 is reached. The order corresponding to the minimum FPR is chosen as the optimum AR order for the latter case.

4. Experimental results

For each subject and each mental task, there are two sets of five best channels (one is obtained using MDelete and the other is obtained using MForm). The performance of the 5-channel set that yielded the better performance is summarized in Table 3. For each subject and each mental task, the table shows whether the channels obtained using MDelete (1) or MForm (2) are selected. This is indicated under the channel selection method (CSM) column. It also shows the mean values of the TPR and FPR obtained from the cross-validation and testing processes in two separate rows. The optimum AR model order is also included in the table. In Table 4, the difference in the performance between the 5-channel BCIs using MDelete and MForm is given. Table 5 shows the results of the t-test between any 2 of the four mental tasks for each subject in the 5-channel BCIs.

Subject	Process	Visualization				Multiplication				Object rotation				Motor imagery
		CSM	AR	TPR	FPR	CSM	AR	TPR	FPR	CSM	AR	TPR	FPR	CSM	AR	TPR	FPR
1	Validation	1	87	59.82	0.00	2	98	61.52	0.00	2	89	59.82	0.00	2	93	56.56	0.00
	Testing			57.78	0.00			61.24	0.00			59.71	0.00			55.30	0.00
2	Validation	1	91	57.37	0.00	1	90	58.29	0.00	2	84	57.23	0.00	1	90	54.68	0.00
	Testing			58.56	0.00			57.93	0.00			56.00	0.00			54.38	0.00
3	Validation	2	81	61.98	0.00	2	82	63.25	0.00	2	95	57.23	0.00	1	90	57.25	0.00
	Testing			62.68	0.00			63.72	0.00			55.37	0.00			58.51	0.00
4	Validation	1	101	55.21	0.00	1	129	42.89	0.00	1	136	33.03	0.20	1	134	41. 62	0.01
	Testing			53.89	0.00			42.61	0.01			33.50	0.18			42.37	0.00

Table 3.

Cross-validation and testing results of the better channel selection method for 5-channel systems.

The table shows the mean values of the TPR and FPR measures. The results of the cross-validation and testing processes are given in two separate rows for each subject and each mental task. The selected CSM and AR orders for each case are also included in the table.

Subject	Process	Visualization				Multiplication				Object rotation				Motor imagery
		dAR†	dTPR†	dFPR†	p-value‡	dAR	dTPR	dFPR	p-value	dAR	dTPR	dFPR	p-value	dAR	dTPR	dFPR	p-value
1	Validation	–8	1.31	0.00	0.2063	7	−5.61	0.00	0.0004	5	−2.04	0.00	0.0593	16	−5.11	0.00	0.0029
	Testing		0.29	0.00	0.7615		−5.24	0.00	0.0005		−1.78	0.00	0.2160		−4.57	0.00	0.0201
2	Validation	−2	1.59	0.00	0.0121	4	2.40	0.00	0.1009	5	−0.19	0.00	0.8667	−5	4. 82	0.00	0.0000
	Testing		1.63	0.00	0.3012		2.88	0.00	0.0654		−0.43	0.00	0.7867		4. 16	0.00	0.0095
3	Validation	6	−1.37	0.00	0.3534	9	−4.96	0.00	0.0005	6	−3.07	0.00	0.1215	−3	0.91	0.00	0.1889
	Testing		−2.19	0.00	0.0620		−6.10	0.00	0.0001		−3.19	0.00	0.1863		1.51	0.00	0.1076
4	Validation	−26	6.36	0.00	0.0004	−7	2.43	0.00	0.1970	0	0.67	0.01	0.5523	−2	2.90 –	0.01	0.0878
	Testing		6.41	0.00	0.0021		1.98	0.00	0.0146		1.83	−0.01	0.0756		2.27 –	0.01	0.0777

Table 4.

The difference in performance between MDelete and MForm for the 5-channel systems.

^†

dV = V_MDelete − V_MForm, V $\in$ {AR,TPR,FPR}.

^‡

$p<0.05$ means “there is a significant difference.” These cases are shown in bold.

Subject	Mental task	Mental task
		Multiplication	Object rotation	Motor imagery
1	Visualization	0.0146	0.1759	0.1427
	Multiplication		0.2472	0.0053
	Object rotation			0.0251
2	Visualization	0.7203	0.1695	0.0330
	Multiplication		0.2499	0.0408
	Object rotation			0.2949
3	Visualization	0.3236	0.0049	0.0040
	Multiplication		0.0034	0.0001
	Object rotation			0.1062
4	Visualization	0.0000	0.0000	0.0000
	Multiplication		0.0000	0.8060
	Object rotation			0.0000

Table 5.

p-values calculated using Welch’s t-test between the four mental tasks for each subject (5-channel systems).

$p<0.05$ means “there is a significant difference.” These cases are shown in bold.

The performance of the 7-channel BCIs using the two channel selection methods are compared with each other, and the performance of the better method is given in Table 6 for each subject and each mental task. The performance difference between the two channel selection methods is shown in Table 7.

Subject	Process	Visualization				Multiplication				Object rotation				Motor imagery
		CSM	AR	TPR	FPR	CSM	AR	TPR	FPR	CSM	AR	TPR	FPR	CSM	AR	TPR	FPR
1	Validation	2	63	63.23	0.00	2	73	65.91	0.00	1	64	65.45	0.00	1	69	63.35	0.00
	Testing			63.70	0.00			64.51	0.00			66.11	0.00			62.51	0.00
2	Validation	2	62	64.59	0.00	2	61	63.07	0.00	2	60	61.55	0.00	1	64	59.39	0.00
	Testing			64.77	0.00			62.73	0.00			61.97	0.00			60.68	0.00
3	Validation	1	64	63.32	0.00	2	56	69.94	0.00	1	65	66.33	0.00	1	65	64.57	0.00
	Testing			64.90	0.00			70.51	0.00			64.73	0.00			65.32	0.00
4	Validation	2	79	61.56	0.00	2	95	51.90	0.00	2	129	27.58	0.01	1	97	49.30	0.00
	Testing			59.71	0.00			50.21	0.00			27.36	0.01			50.00	0.00

Table 6.

Cross-validation and testing results of the better channel selection method for 7-channel systems.

The table shows the mean values of the TPR and FPR measures. The results of the cross-validation and testing processes are given in two separate rows for each subject and each mental task. The selected CSM and AR orders for each case are also included in the table.

Subject	Process	Visualization				Multiplication				Object rotation				Motor imagery
		dAR†	dTPR†	dFPR†	p-value‡	dAR	dTPR	dFPR	p-value	dAR	dTPR	dFPR	p-value	dAR	dTPR	dFPR	p-value
1	Validation	2	−1.77	0.00	0.1354	7	−6.70	0.00	0.0017	3	0.58	0.00	0.5591	1	1.71	0.00	0.3435
	Testing		−2.56	0.00	0.0084		−5.12	0.00	0.0047		0.80	0.00	0.5357		2.36	0.00	0.1876
2	Validation	2	−1.58	0.00	0.0954	4	−2.25	0.00	0.1739	6	−0.70	0.00	0.6486	4	1.25	0.00	0.4317
	Testing		−2.24	0.00	0.0960		−0.44	0.00	0.7226		−2.48	0.00	0.2445		2.58	0.00	0.0045
3	Validation	−2	1.00	0.00	0.4971	6	−4.86	0.00	0.0007	4	0.73	0.00	0.7233	4	−0.06	0.00	0.9542
	Testing		2.22	0.00	0.0455		−5.42	0.00	0.0009		0.91	0.00	0.6633		1.52	0.00	0.1110
4	Validation	−3	−1.59	0.00	0.2486	−5	1.62	0.00	0.1907	−3	4.26	0.01	0.0173	5	0.00	0.00	1.0000
	Testing		−0.37	0.00	0.7741		2.26	0.01	0.0037		4.02	0.00	0.0012		0.54	0.00	0.7501

Table 7.

The difference in performance between MDelete and MForm for the 7-channel systems.

^†

dV = V_MDelete − V_MForm, V $\in$ {AR,TPR,FPR}.

^‡

$p<0.05$ means “there is a significant difference.” These cases are shown in bold.

In Tables 3 and 6, the values related to the highest performance are shown in bold, while those related to the lowest performance are underlined.

The Welch’s t-test [104], as a statistical significance test, is performed on the TPR values for measuring the performance difference between MDelete and MForm (for each subject and each mental task) and between every pair of the four mental tasks (for each subject). The null hypothesis is that there is no difference between the TPR values of the two groups (these two groups can be the two channel selection methods or any two mental tasks). We assume a 5% significance level. The null hypothesis is rejected if the resultant p-value is less than $0.05$ . This means that the TPR values of the two test groups are significantly different. If $p\ge 0.05$ , the null hypothesis cannot be rejected at the 5% significance level. This implies that there is no significant difference between the TPR values of the test groups.

The p-value between MDelete and MForm for each mental task of each subject is given in Tables 4 and 7 under the p-value column. Table 8 shows the results of the t-test between any 2 of the four mental tasks for each subject in the 7-channel BCIs.

Subject	Mental task	Mental task
		Multiplication	Object rotation	Motor imagery
1	Visualization	0.4889	0.0519	0.3890
	Multiplication		0.2553	0.2243
	Object rotation			0.0405
2	Visualization	0.0809	0.1582	0.0035
	Multiplication		0.6683	0.0328
	Object rotation			0.4601
3	Visualization	0.0007	0.9199	0.6681
	Multiplication		0.0135	0.0013
	Object rotation			0.7321
4	Visualization	0.0001	0.0000	0.0002
	Multiplication		0.0000	0.8619
	Object rotation			0.0000

Table 8.

p-values calculated using Welch’s t-test between the four mental tasks for each subject (7-channel systems).

$p<0.05$ means “there is a significant difference.” These cases are shown in bold.

5. Summary and discussion of results

Table 4 shows that the performance of the 5-channel BCI systems obtained using MDelete is close to those obtained using MForm for the majority of the cases. The same is true for the 7-channel systems (Table 7). From Tables 3 and 6, it is shown that MDelete yields better results in nine out of the 16 cases of the 5-channel BCIs and in seven out of the 16 cases of the 7-channel BCIs.

5.1 System performance of 5-channel BCIs

From Tables 3 and 5, we find the following:

1. The FPR values are zero for all 5-channel BCIs of Subjects 1, 2, and 3 during the cross-validation and testing processes, irrespective of the task type. For Subject 4, this is also true for the sentence visualization task-based BCI: the BCI based on the multiplication task has 0.01% FPR for the testing process, the BCI based on the motor imagery task has an FPR of 0.01% for the cross-validation process, and the BCI based on the object rotation task has FPR values of 0.20 and 0.18% for cross-validation and testing, respectively.

2. Among all 5-channel BCIs, the highest performance (TPR = 63.72%) is reached by the multiplication task of Subject 3. The object rotation task-based BCI of Subject 4 has the lowest performance with TPR and FPR values of 33.50 and 0.18%, respectively.

3. For Subject 1: multiplication is the best in performance (TPR = 61.24%) although it is not significantly different from object rotation. Motor imagery is the poorest in performance (TPR = 55.30%) although it is not significantly different from sentence visualization.

4. For Subject 2: sentence visualization, multiplication, and object rotation have statistically similar performance with TPRs in the range of 56.00–58.56%. Motor imagery has the poorest performance (TPR = 54.38%) although it is not significantly different from object rotation.

5. For Subject 3: multiplication and sentence visualization have similar and the highest performance (TPRs = 63.72 and 62.68%). Object rotation and motor imagery have similar and the lowest performance (TPRs = 55.37 and 58.51%).

6. For Subject 4: sentence visualization has the best performance (TPR = 53.89%), and object rotation has the poorest performance (TPR = 33.50% and FPR = 0.18%).

5.2 System performance of 7-channel BCIs

From Tables 6 and 8, the following are found:

1. The FPR values reach zero for 15 out of the 16 cases. That is for all cases except for the object rotation task of Subject 4, which has 0.01% FPR for each of the cross-validation and testing processes.

2. The results as to which BCIs yield the best and worst performance are exactly as those in the 5-channel systems. Among all the 7-channel BCIs, the best performance with a TPR of 70.51% is reached by the BCI based on the multiplication task of Subject 3 (which is better than the TPR = 63.72% of the corresponding 5-channel BCI). The object rotation task-based BCI of Subject 4 has the worst performance with TPR and FPR values of 27.36% and 0.01%, respectively (which is also better than the FPR = 0.18% of the corresponding 5-channel BCI).

3. For Subject 1: motor imagery is significantly different from object rotation. All other cases are statistically similar to each other. The TPR varies in the range 62.51–66.11% for different mental tasks.

4. For Subject 2: sentence visualization, multiplication, and object rotation have statistically similar performance with TPRs in the range of 61.97–64.77%. Motor imagery has the least performance (TPR = 60.68%) although it is not significantly different from that of object rotation.

5. For Subject 3: multiplication has the highest performance (TPR = 70.51%). Sentence visualization, object rotation, and motor imagery have similar performance with TPRs in the range of 64.73–65.32%.

6. For Subject 4: sentence visualization has the best performance (TPR = 59.71%), and object rotation has the poorest performance (TPR = 27.36% and FPR = 0.01%).

5.3 Discussion of results

Comparing Tables 3 and 6, it can be noticed that increasing the number of channels from five to seven enhances the performance of every BCI by an increase of 5.38% in TPR on average (see Table 9). Therefore, there is a trade-off between using fewer channels and having better system performance. The choice should be made depending on the applications, the situations in which the BCI system is used, and the computational power available.

	5-channel design			7-channel design			29-channel design†
	AR	TPR	FPR	AR	TPR	FPR	AR	TPR	FPR
Average over tasks
Subject 1	92*	58.51	0.00	67*	64.21	0.00	24	68.54	0.00
Subject 2	89*	56.72	0.00	62*	62.54	0.00	19*	70.38	0.00
Subject 3	87	60.07	0.00	63*	66.37	0.00	20*	70.80	0.00
Subject 4	125	43.09	0.05	100	46.82	0.00	29	59.31	0.00
Average over subjects
Visualization	90	58.23	0.00	67	63.27	0.00	21*	68.95	0.00
Multiplication	100*	56.38	0.00	71*	61.99	0.00	24	67.38	0.00
Object rotation	101	51.15	0.05	80*	55.04	0.00	24*	65.71	0.00
Motor imagery	102*	52.64	0.00	74*	59.63	0.00	23*	67.00	0.00
Total average	98*	54.60	0.01	73*	59.98	0.00	23	67.26	0.00

Table 9.

Average performance of the BCI systems.

^†

Results of study [110].

^*

The value is rounded to the nearest integer.

In terms of the AR model orders, the 5-channel BCIs need higher orders than the 7-channel BCIs. According to Table 9, the overall mean of the AR model order is 98 and 73 for the 5-channel and 7-channel BCI systems, respectively.

Studies [105, 106, 107, 108, 109] have also used high AR model orders in their analyses. The AR model order depends on the sampling frequency [106]. Since our sampling frequency is 500 Hz (which is high), then the AR order can also be high. In other words, if the AR order belonging to the sampling frequency 100 Hz is 25, then the AR order belonging to sampling frequency 500 Hz is 125. This is because:

1. The AR order is the number of previous samples of the signal that represents the current sample of the signal. Please refer to Eq. (1) in Appendix A.

2. An AR order = 25 at freq = 100 Hz means that we need the last 0.25 s of the signal to represent the current sample. For freq = 500 Hz, we will need the last 0.25 s of the signal, and thus the AR order would be 0.25 $\times$ 500 = 125.

From Table 9, it can be easily recognized that the system performance depends on the subject and the mental task. Among all subjects, Subject 3 has the BCIs with the highest average performance with average TPRs of 60.07 and 66.37% for the 5-channel and 7-channel systems, respectively. Subject 4 has the least average performance with an average TPR of 43.09% for the 5-channel systems and 46.82% for the 7-channel systems.

Among the mental tasks, sentence visualization and multiplication are the best tasks overall. Object rotation and motor imagery yield the least performance on average.

Table 9 also shows the average performance of the 29-channel systems over the subjects and mental tasks for comparison purposes [110]. It can be seen that by decreasing the number of EEG channels from 29 to 7 and 5, the system performance degrades by a decrease of 7.28 and 12.66%, respectively, in TPR on average. This degradation in the system performance is the trade-off one has to make to have a simpler system that can be easily set up and requires less computational power.

6. Conclusion

This study shows that it is feasible to design self-paced mental task-based BCIs with a zero false activation rate using very few (i.e., five or seven) EEG channels. The system performance was evaluated on a dataset we collected from four subjects. Although the evaluation was carried off-line, the methodology can be used in real-time self-paced systems after slight modifications. This is because the feature extraction and the classification processes are not computationally demanding, and there is no need for the timing information of the EEG signals after training the classifier with the training data.

The best performance of the BCI systems described above has zero FPRs and sufficiently high TPRs (i.e., 53.89–63.72% for the 5-channel systems and 59.71–70.51% for the 7-channel systems). Hence, in terms of the system performance, they are acceptable for use in real-life applications. As reported in the study [110], the best performance of the BCI systems with 29 channels has zero FPRs and 65.06–72.65% TPRs.

The frequency domain analysis of the recorded EEG signals is left for future work. By finding the frequency bands with the high amounts of information, we may be able to decrease the sampling frequency from its current value of 500 Hz.

In this study, the EEG trials are divided into 1-s segments for classification, and the BCI system gives an output every 0.2 s. Finding the optimum values for the segment length and the system output rate is the other directions for future work. The use of these values might result in a more accurate system.

Running online experiments is the most important step that needs to be taken in the future in order to evaluate the performance of our BCI system in real time.

Acknowledgments

The authors would like to thank the subjects who came and devoted their time to attend the EEG recording sessions.

A.

The autoregressive (AR) model of the signal $s$ [ $t$ ] is defined as

$$s\left[t\right]=\sum _{j=1}^F{b}_{j}s\left[t-j\right]+n\left[t\right]$$ E1

where $F$ is the order of the model, $b_j$ is the $j$ th coefficient, and $n$ [ $t$ ] is the noise (or error) signal. The noise signal is assumed to be a random process with a zero mean and a finite variance.

In this study, the AR model is estimated using the Burg algorithm [111] which is a popular and widely used algorithm in the field.

There are some methods based on the reflection coefficient [112], the final prediction error criterion [113], the autoregressive transfer function criterion [113], and the Akaike information criterion [113] to find the order of the AR model; however, none of them are efficient in BCI applications [100]. In our previous study [100], it is suggested that instead of using the existing methods, it is better to select the model order by cross-validation based on the system performance. The same approach is taken in this study.

Quadratic discriminant analysis (QDA) [114] is the quadratic version of linear discriminant analysis (LDA). Like LDA, normal distributions are assumed for the classes. The only difference between QDA and LDA relates to the covariance matrices of the classes. In QDA, unlike LDA, the covariance matrices of the classes are not assumed to be the same. Therefore, QDA is more general than LDA.

Suppose we have $k$ -dimensional vectors of $x$ to be classified into one of the $M$ classes with the normal distributions

$${\mathrm{\Omega }}_m\sim N_k\left({\mu }_m,{\mathrm{\Sigma }}_m\right)$$ E2

where $m\in \left\{1,2,\dots ,M\right\}$ , ${\mu }_m$ is the $k$ -dimensional mean vector and ${\mathrm{\Sigma }}_m$ is the $k\times k$ covariance matrix of Class ${\mathrm{\Omega }}_m$ .

The probability density function of Class ${\mathrm{\Omega }}_m$ can be expressed as

$$\begin{array}{l}{f}_{m,X}\left(x\right)=\frac{1}{{\left(2\pi \right)}^{k/2}{\left|{\mathrm{\Sigma }}_m\right|}^{1/2}}\\ \mathit{exp}\left(-\frac{1}{2}{\left(x-{\mu }_m\right)}^T{\mathrm{\Sigma }}_m^{-1}\left(x-{\mu }_m\right)\right)\end{array}$$ E3

Based on the Bayes discriminant rule, the input vector $x$ is classified into Class ${\mathrm{\Omega }}_i$ if

$$C_i{\pi }_i{f}_{i,X}\left(x\right)=\underset{j}{\text{max}}\left(C_j{\pi }_j{f}_{j,X}\left(x\right)\right),j\in \left\{1,2,\dots ,M\right\}$$ E4

where ${\pi }_i$ is the a priori probability of Class ${\mathrm{\Omega }}_i$ and $C_i$ is the total cost of misclassifying a member of Class ${\mathrm{\Omega }}_i$ to the other classes. Note that

$$\sum _{j=1}^M{\pi }_j=1.$$ E5

The decision rule can be simplified as follows if only two classes exist:

$$x\in \left\{\begin{array}{l}{\mathrm{\Omega }}_1:C_1{\pi }_1{f}_{1,X}\left(x\right)\ge C_2{\pi }_2{f}_{2,X}\left(x\right)\\ {\mathrm{\Omega }}_2:C_1{\pi }_1{f}_{1,X}\left(x\right)<C_2{\pi }_2{f}_{2,X}\left(x\right)\end{array}\right.$$ E6

This is equivalent to

$$x\in \left\{\begin{array}{l}{\mathrm{\Omega }}_1:\text{ln}{f}_{1,X}\left(x\right)-\text{ln}{f}_{2,X}\left(x\right)\ge \text{ln}\left(\frac{{\pi }_2}{{\pi }_1}.\frac{C_2}{C_1}\right)\\ {\mathrm{\Omega }}_2:\text{ln}{f}_{1,X}\left(x\right)-\text{ln}{f}_{2,X}\left(x\right)<\text{ln}\left(\frac{{\pi }_2}{{\pi }_1}.\frac{C_2}{C_1}\right)\end{array}\right.$$ E7

Using (3) in (7), the discriminant rule becomes

$$x\in \left\{\begin{array}{l}{\mathrm{\Omega }}_1:F_{\mathit{qd}}\left(x\right)\ge 0\\ {\mathrm{\Omega }}_2:F_{\mathit{qd}}\left(x\right)<0\end{array}\right.$$ E8

where the discriminant function, $F_{\mathit{qd}}\left(x\right)$ , is defined as

$$\begin{array}{cc}{F}_{\mathit{qd}}\left(x\right)=& -\frac{1}{2}{x}^T\left({\mathrm{\Sigma }}_1^{-1}-{\mathrm{\Sigma }}_2^{-1}\right)x+\left({\mu }_1^T{\mathrm{\Sigma }}_1^{-1}-{\mu }_2^T{\mathrm{\Sigma }}_2^{-1}\right)x\\ & -\frac{1}{2}\text{ln}\left(\frac{\mid {\mathrm{\Sigma }}_1\mid }{\mid {\mathrm{\Sigma }}_2\mid }\right)-\frac{1}{2}\left({\mu }_1^T{\mathrm{\Sigma }}_1^{-1}{\mu }_1-{\mu }_2^T{\mathrm{\Sigma }}_2^{-1}{\mu }_2\right)\\ & -\text{ln}\left(\frac{{\pi }_2}{{\pi }_1}.\frac{C_2}{C_1}\right)\end{array}$$ E9

The mean vectors (i.e., ${\mu }_1$ and ${\mu }_2$ ) and the covariance matrices (i.e., ${\mathrm{\Sigma }}_1$ and ${\mathrm{\Sigma }}_2$ ) are estimated from the data samples. In this study, the same value for the a priori probabilities ${\pi }_1$ and ${\pi }_2$ and the same value for the cost parameters $C_1$ and $C_2$ are assumed.