Open access peer-reviewed chapter

Evaluating Steady-State Visually Evoked Potentials-Based Brain-Computer Interface System Using Wavelet Features and Various Machine Learning Methods

Written By

Ebru Sayilgan, Yilmaz Kemal Yuce and Yalcin Isler

Submitted: 09 April 2021 Reviewed: 11 May 2021 Published: 15 June 2021

DOI: 10.5772/intechopen.98335

From the Edited Volume

Brain-Computer Interface

Edited by Vahid Asadpour

Chapter metrics overview

487 Chapter Downloads

View Full Metrics

Abstract

Steady-state visual evoked potentials (SSVEPs) have been designated to be appropriate and are in use in many areas such as clinical neuroscience, cognitive science, and engineering. SSVEPs have become popular recently, due to their advantages including high bit rate, simple system structure and short training time. To design SSVEP-based BCI system, signal processing methods appropriate to the signal structure should be applied. One of the most appropriate signal processing methods of these non-stationary signals is the Wavelet Transform. In this study, we investigated both the effect of choosing a mother wavelet function and the most successful combination of classifier algorithm, wavelet features, and frequency pairs assigned to BCI commands. SSVEP signals that were recorded at seven different stimulus frequencies (6–6.5 – 7 – 7.5 – 8.2 – 9.3 – 10 Hz) were used in this study. A total of 115 features were extracted from time, frequency, and time-frequency domains. These features were classified by a total of seven different classification processes. Classification evaluation was presented with the 5-fold cross-validation method and accuracy values. According to the results, (I) the most successful wavelet function was Haar wavelet, (II) the most successful classifier was Ensemble Learning, (III) using the feature vector consisting of energy, entropy, and variance features yielded higher accuracy than using one of these features alone, and (IV) the highest performances were obtained in the frequency pairs with “6–10”, “6.5–10”, “7–10”, and “7.5–10” Hz.

Keywords

  • steady-state visually-evoked potentials (SSVEP)
  • brain-computer interfaces (BCI)
  • wavelet transform (WT)
  • mother wavelet selection
  • pattern recognition
  • machine learning (ML)

1. Introduction

Electroencephalogram (EEG) signals are one of the most widely used types of biomedical signals for Brain-Computer Interfaces (BCIs), owing to their portability, high time resolution, ease of acquisition, and cost-effectiveness as compared to other brain activity monitoring techniques [1, 2, 3]. There are four typical EEG-based BCI paradigms: steady-state visual-evoked potentials (SSVEP), slow cortical potentials (SCP), the P300 component of evoked potentials, and sensory-motor rhythms (SMR) [4, 5, 6].

The SSVEP signal is a periodic response to a visual stimulator modulated at a frequency greater than 6 Hz [7] or 4 Hz [8]. The amplitude and phase characteristics of the SSVEP depend on stimulus intensity and frequency. SSVEP events can be repeatedly produced if the stimuli are provided under controlled conditions [9]. For instance, staring at a flickering light that flashes at a constant frequency stimulates the human visual pathway. The flickering frequency is radiated throughout the brain. This stimulation produces electrical signals in the brain at the base frequency of the flashing light, as well as at its harmonics [10]. Practically, there is a marked reduction in the power of the SSVEP signals from the second harmonics onwards. This has been attributed to the low signal-to-noise ratio of the SSVEP signals at high frequencies and can be accounted for the brain dynamics that act as a low pass filter [11].

The analysis of EEG signals using machine learning (ML) methods is developed to help physicians in accurate diagnosis and provides fast and valid tools in assistive applications designed for individuals. Among the various approaches available in the literature, the Wavelet Transform (WT) has proven to be an effective time-frequency analysis tool for analyzing transient signals [12, 13]. Various wavelet families are available to define and adapt signal characteristics [14]. However, choosing an appropriate mother wavelet is very important for the analysis of these signals. Research studies to date for EEG-signal classification using the wavelet technique have mostly been done using the Daubechies (Db) family. The maximum accuracy achieved in this study was 95.00% [15]. However, in this study, although the signal was suitable for Discrete Wavelet Transformation (DWT), analysis was performed using the Continuous Wavelet Transformation (CWT) method. Furthermore, in the same study, analysis was performed for a single frequency. In this chapter, a detailed analysis was performed using multiple frequencies. Also, in Ref. [16], the SSVEP signal was used for a single wavelet type (Db40), but no mother wavelet selection was made. Thus, the mother wavelet selection for SSVEP is still an unanswered question.

The research presented in this chapter is especially about selecting the most suitable wavelet function for signal analysis of SSVEP signals, detailed investigation of energy, entropy, and variance attributes, and examining the appropriate frequency(s) for SSVEP based BCI design.

There is not any, to our knowledge, in-depth study on the selection of stimulation frequencies. It was noticed that higher accuracy rates could be obtained for pattern recognition by examining the frequency selection and the differences between the frequencies. The frequency or frequencies that might result in higher higher accuracy rates and time advantages are considered to help design user-friendly BCI systems. Due to the shortcomings in the literature mentioned above, this study was considered to be conducted.

Advertisement

2. Materials and methods

2.1 Data recording process and users

In this study, the dataset (AVI SSVEP Dataset) containing SSVEP signals designed and recorded by Adnan Vilic was used [17]. The data set contains data that include EEG measurements of healthy individuals (three men and one woman having ages range from 27 to 32) looking at the flickering target to trigger responses of SSVEP signals at different frequencies, and the data set used for this study is publicly available. Using the standard international 10–20 system for electrode placement, the reference electrode is positioned in Fz with the signal electrode in Oz and Fpz in the ground electrode. In this experiment, individuals had been seated 60 cm away from a monitor staring at a single flashing target whose color changed rapidly from black to white. The test stimulus was a flashing box at seven different frequencies (6–6.5 - 7 - 7.5 - 8.2 - 9.3 - 10 Hz) presented on the monitor. The data set comprises of four sessions with four different participants. Each trial in a session lasts 30 seconds and participants take a short break between trials. Experiments were repeated at least three times for each frequency.

In Figure 1, a) the raw signal stimulated at a frequency of 10 Hz and b) the power spectrum density computed signal (with its 1st and 2nd harmonics) are shown.

Figure 1.

a) SSVEP raw signal b) power spectrum of the 10 Hz stimulated SSVEP signal and topography.

2.2 Feature extraction

It is possible to define the neurophysiology of the human visual system, the neuronal activity of the visual cortex is replaced by visual stimulation, and variations of the brain response related to the features of the visual stimulus such as brightness, contrast and frequency [18]. Neurons in the visual cortex synchronize their flickering to the frequency of blinking of the visual stimulus. SSVEP signals are generated when visual stimuli are repeatedly presented, creating almost sinusoidal oscillations [19]. Applying a visual stimulus flashing at a constant frequency increases the energy of brain activities comparing to the case of applying a constant visual stimulus [7]. The strongest response occurs in the visual cortex of brain (occipital), but other areas of brain are also activated to different degrees [8, 9]. SSVEP marks can be detected even for narrow frequency bands around the visual stimulation frequency with signal processing methods that take advantage of the specific features of the signal such as timing, frequency, and rhythm [20]. For this reason, this study is designed on accepted signal processing strategies that validate the comprehensive scenarios analyzed.

2.2.1 Time-domain based feature extraction

The SSVEP time-domain features are extracted from available literature information in the original field of the EEG signal. Table 1 describes the relevant and distinctive SSVEP time-domain features we identified. These features are based on the amplitude (e.g. average amplitude change value, root mean square, interquartile ranges, etc.) and statistical changes of the EEG signal (e.g., mean, variance, skewness, and kurtosis, etc.) [20].

EEG Time-domain features Fit
No.FeaturesNo.Features
1.EEG minimum value14.Kurtosis of EEG signal
2.EEG maximum value15.Skewness of EEG signal
3.EEG mean value16.Hjorth identifiers: 1) Activity
4.EEG standard deviation value17.Hjorth identifiers: 2) Mobility
5.Integrated EEG value18.Hjorth identifiers: 3) Complexity
6.Mean absolute value19.Signal range (max-min.)
7.Simple square integral value20.Inter-quarter intervals 1st Quartile
8.EEG variance value21.Inter-quarter intervals 2nd Quartile (Median)
9.Root mean square value22.Inter-quarter intervals 3rd Quartile
10.Waveform length value23.Zero-crossing
11.Average amplitude change value24.Slope-change value
12.Absolute difference in standard deviation25.Mode value of the signal

Table 1.

EEG time-domain features (EEG signal is represented by x, and Fitstands for the EEG features computed from x).

2.2.2 Frequency-domain based feature extraction

SSVEP signals are identified by oscillations with frequencies synchronized with the stimulus frequency [6, 21]. For this reason, many SSVEP-based BCI systems use frequency information embedded in the signal in the feature extraction process [22, 23]. Within the scope of this chapter, SSVEP frequency features were extracted from the frequency domain representation of the SSVEP signal using a Fourier transform. The relevant and distinctive SSVEP frequency characteristics we detected are based on the spectral information of SSVEP signals for each EEG rhythm, such as energy, variance and spectral entropy.

These features explain how power, variance, and irregularity (entropy) change in certain related frequency bands. In practice, this implies that these features will use their power in certain frequency bands [24].

Features based on power spectrum, energy of each frequency band,

F1f=Energyf=k=1Myk2E1

Here is the Fourier transform of the analytic signal y of a real discrete time EEG signal x.F1f=Energyf stands for the EEG features computed from y, and M corresponds to the maximum frequency.

Features based on variance of each EEG frequency band,

F2f=Variancef=1M1k=1Myky¯2E2

y¯” in the formula stands for the average of the “y” signal.

Feature based on entropy of each EEG frequency band: Spectral entropy measures the regularity of the power spectrum of EEG signal,

F3f=Entropyf=1logMk=1MPyklogPykE3

2.2.3 Wavelet transform based feature extraction

2.2.3.1 Wavelet decomposition

SSVEP signal is non-stationary [18]. Consequently, WT has been used to examine not only spectral analysis of the signal but also the spectral behavior of the signal over time. This method is characterized by a smooth and fast oscillating function that is well localized in frequency and time [12]. WT can be applied as a specially designed dual Finite-Impulse Response (FIR) filter. The frequency responses of FIR filters separate the high frequency and low-frequency components of the input signal. The point of dividing the signal frequency is usually between 0 Hz and half the data sampling rate (Nyquist frequency). In the Multi-resolution Algorithm (MRA) of the WT, the identical wavelet coefficients are used in both low-pass (LP) and high-pass (HP) filters. The LP filter coefficients are associated with scaling parameter, which will determine the oscillatory frequency and the length of the wavelet. At the same time, the HP filter is associated with the wavelet function. The outputs of the LP filters are called the approximation (a) coefficients, and the outputs of the HP filters are called the detail (d) coefficients. In MRA of WT, any time-series signals can be entirely decomposed in terms of a and d coefficients based on decomposition level. Implementation of DWT on raw signal produces an MRA of various statistical and non-statistical parameters across time and frequency [24]. The subsets of the wavelet coefficients of the decomposition tree were selected as input vectors to the classifier. The SSVEP signals are decomposed into 9 decomposition levels, and i = 1, 2,. .., 9 for 512 Hz sampling frequency.

2.2.3.2 Parameters for feature extraction

Using different DWT functions (Haar, Db2, Sym4, Coif1, Bior3.5, Rbior2.8), SSVEP signals are subdivided into frequency bands (delta, theta, alpha, beta, gamma), and the energy, entropy and variance were calculated for each band [13, 14]. Every DWT frequency band is associated with one or two EEG rhythms. Thus, a number of features represented in the frequency bands were obtained.

Energy at each decomposition level was calculated using the following Equations [24]:

F1w=Edi=j=1Ndij2,i=1,2,3,,lE4
F1w=Eai=j=1Naij2,i=1,2,3,,lE5

where dij and aij represent detail and approximate coefficients, respectively, formed by the wavelet level corresponding to each EEG band (delta, theta, alpha, beta, gamma).i=1,2,3,,l is the wavelet decomposition level from levels 1 to l. Finally, N stands for the number of detail and approximate coefficients at each decomposition level.

Another feature, the entropy at each decomposition level is calculated using the following Equation [25]:

F2w=Enti=j=1Ndij2logdij2,i=1,2,3,,lE6

The variance at each decomposition level was calculated using the following Equation [24]:

F3w=Vari=1N1j=1Ndijμi2,μi=1Nj=1Ndij,i=1,2,3,,lE7

Extracted features, which consist of different combinations, (l +1) dimensional are used as input vectors. In other words, for an ‘l’ level decomposition, the feature vector of any parameter can be represented as Feature = [xd1, xd2, …, xdl, xal], where x stands on energy, entropy, and variance.

2.3 Machine learning classification algorithms

The most important use of machine learning (ML) methods is classification [26]. After feature extraction, classification is performed to recognize an SSVEP signal and convert it to command, that is, to use it as output [27]. For the classification process, the “datasets” formed by a certain number of feature vectors, of which class it belongs, are passed through the training period required by the classification type. As a result of this training, a decision mechanism algorithm is created, which is used to assign the unknown signal to the appropriate class [28, 29].

The extracted feature vectors have been tested with seven well-known and commonly-used basic classifiers. These selected classifier algorithms are Decision Trees, Discriminant Analysis, Logistic Regression, Naive Bayes, Support Vector Machines, k-Nearest Neighbors, and Ensemble Learner. The classifier performances were examined to determine which combination of mother wavelet function, wavelet features, and classifier algorithm gives the highest accuracy.

2.4 Evaluation of machine learning algorithms performance

While training ML algorithm to classify SSVEP signals is an important step, it is essential to consider how the algorithm is generalized on unprecedented data (test set) [30]. We need to know if the algorithm works correctly and whether we can trust its predictions. The machine learning algorithm can only memorize the training set. Therefore, it can make reasonable predictions about future examples or examples that it has not seen before. Thus, it is one of the essential steps for BCI systems to know and apply the techniques used to evaluate how well a ML model generalizes to new, unprecedented data [31, 32]. For this goal the “k-fold cross-validation” and “confusion matrix” evaluation criteria were used to evaluate the performance of the ML algorithms used in this study.

2.4.1 k-fold cross-validation

In this method, the data set is randomly divided into k segments. Among these segments, k-1 parts are used for the training, and the remaining part is used for the testing. This process is repeated until all parts are used for testing separately. The test errors are recorded each time, and the average of the errors after the last part is reported. The performance of each classifier algorithm used is measured by carrying out this approach [30, 31]. In this study, the data set is divided into five equal parts.

2.4.2 Confusion matrix

Confusion matrix is, at first, calculated to evaluate the classifier performance. The confusion matrix is generated by comparing the responses of the classification algorithm to the test set with the actual values in the data set. In case of two-state problems, it is a table consisting of four different situations [26]. These are True Positive (TP) value, True Negative (TN) value, False Positive (FP) value, and False Negative (FN) value.

Accuracy value (ACC) is calculated as classifier performance based on these values [27]:

ACC=TP+TNTP+FN+FP+TNE8

2.5 Experimental design and implementation details

In accordance with the objective of our study, we have designed it in a two-fold manner for time-frequency domain features. First, we measured the accuracy of each (feature, mother wavelet function) pair. As the second part, we combined the set of three features with each mother wavelet function in order to discover which mother wavelet function yields the best performance in terms of accuracy. Three important features (i.e. energy, variance, and entropy) have been extracted for EEG bands (i.e. delta, theta, alpha, beta, and gamma) using six different mother wavelet families (Haar, db, sym, coif, bior, rbio). To this purpose, algorithms were implemented using Signal Processing Toolbox and Wavelet Toolbox in Matlab 2019a. All the classifiers and performance analyses were implemented using the Classifier Learner App tool from Matlab version 2019a.

Advertisement

3. Results and discussion

Characterized as an increase in the amplitude of the stimulating frequency, the photic driver response results in significant baseline and harmonics [33]. Thus, it is possible to determine the stimulus frequency based on the SSVEP measurement. For this purpose, 115 feature vectors were extracted from the SSVEP signals recorded using seven different frequencies. The extracted feature vectors were run with seven basic ML algorithms. Simultaneously, the frequencies that constitute the SSVEP data set were evaluated with multiple, selected three-class, and binary classifications. Also, the effect of the increase in the difference between frequencies on the accuracy criterion was investigated, and the results are shown in detail between Figures 217, and Tables 25.

Figure 2.

Binary classification performance of the time-domain features.

Figure 3.

Percentage of classifier where the best result is the most often obtained as a result of running the algorithms 2,520 times in total.

Figure 4.

Results of selected 3-class classifications for frequency-domain features.

Figure 5.

Binary classification performance of the frequency-domain features.

Figure 6.

Percentage of successful classifiers that give the highest accuracies from 2,520 runs in total.

Figure 7.

Classification performance of energy, entropy, and variance as separate features.

Figure 8.

Classification performance of energy, entropy, and variance together as a feature set (all features together).

Figure 9.

Percentage of classifier where the best result is the most often obtained as a result of running the algorithms 2,520 times in total a) energy, entropy, and variance as separate features, b) energy, entropy, and variance as a feature set.

Figure 10.

Binary classification performance of the features for bior 3.5 mother wavelet function.

Figure 11.

Binary classification performance of the features for coif 1 mother wavelet function.

Figure 12.

Binary classification performance of the features for Db 4 mother wavelet function.

Figure 13.

Binary classification performance of the features for Haar mother wavelet function.

Figure 14.

Binary classification performance of the features for Rbio 2.8 mother wavelet function.

Figure 15.

Binary classification performance of the features for Sym 4 mother wavelet function.

Figure 16.

Change of accuracy value according to the differences between frequencies for mother wavelet functions.

Figure 17.

Percentage of classifier where the best result is the most often obtained as a result of running the algorithms 2,520 times in total (for Haar wavelet function).

SubjectsACCClassifier
Subject 125.90LDA
Subject 250.00Ensemble
Subject 352.40Ensemble
Subject 442.90Ensemble
Mean42.80

Table 2.

Results of multiple classification for time-domain features.

SubjectsACCClassifier
Subject 129.20Ensemble
Subject 250.00Ensemble
Subject 357.10Ensemble
Subject 447.60Ensemble
Mean45.98

Table 3.

Results of multiple classification for frequency-domain features.

Subject 1Subject 2Subject 3Subject 4
Mother waveletACCClassifiersACCClassifiersACCClassifiersACCClassifiers
Coif 129.20KNN34.60Ensemble33.30Ensemble33.30LDA
Bior 3.555.60LDA23.10Ensemble42.90Ensemble28.60Naive Bayes
Db 437.50SVM23.10SVM33.30Naive Bayes33.30Ensemble
Sym 429.20LDA30.80Decision Tree38.10Ensemble28.60LDA
Haar37.50KNN23.10LDA42.90LDA23.80LDA
Rbio 2.833.30Naive Bayes23.10SVM38.10Ensemble28.60Ensemble
Mean37.0526.3038.1029.37

Table 4.

Multiple classification results of wavelet features.

Frequency pairEnergyEntropyVarianceMeanAll features together
6–1095.8394.45100.0096.7697.23
6.5–8.292.8595.8395.8394.83100.00
6.5–10100.0084.5095.8393.44100.00

Table 5.

Classification results of the most successful frequency pairs of the Haar mother wavelet.

3.1 Time-domain features results

The multiple and binary classification results of 25 feature vectors extracted from SSVEP signals using time-domain properties are given below, respectively.

3.1.1 Multiple classification results

Presented in Table 2 are accuracy results for multiple classification. In regard to these results, the highest performance was shown by the Ensemble Learning classifier with 52.40%.

3.1.2 Binary classification results

According to the binary classification results shown in Figure 2, the best performance was obtained with an accuracy value of 91.68% in 6–10 Hz frequency pairs based on the average of the subjects. Simultaneously, when the subjects are considered separately, a classification performance up to 100% were obtained. In addition, there is no definitive finding related to the increase in the accuracy value parallel to the difference between frequencies for the time-domain.

The results of classifiers to be expressed in the pie chart in Figure 3 are the number of hits of the classifiers obtained. These numbers were obtained by running all algorithms 2,520 times in total. The best classification performance is shown by the Ensemble learning classifier.

3.2 Frequency-domain features results

For the frequency-domain characteristics used in the problem of determining seven different frequencies, firstly, spectrum analysis was performed to detect the stimulus frequencies more clearly than the signal. This analysis is often used to obtain frequency information in evoked SSVEP responses. The power spectrum of SSVEP signals was determined by FFT using MATLAB software to calculate its power, entropy, and variance for each band in the frequency range corresponding to the frequencies. For this purpose, the signal received FFT is divided into EEG bands (delta, theta, alpha, beta, gamma), and energy, entropy, and variance values of each band are calculated. A total of 15 feature vectors are generated.

3.2.1 Multiple classification results

According to the multiple classification results of the seven frequencies presented in Table 3, it was determined that the best performance was in the Ensemble Learning classifier with an accuracy value of 57.10%. Another remarkable finding here is that the results of the classifier from all individuals are the same. This shows us that, like the time-domain, the Ensemble Learning classifier performs better than others. In addition, when multiple classification results of frequency-domain features are compared with multiple classification results of time-domain features, it has been determined that there is an increase of 4.70% on an individual basis and 3.18% on average.

3.2.2 Selected three class classification results

In this part, three frequencies (6 Hz - 8.2 Hz - 10 Hz), which are considered to increase the classification performance, were chosen among the seven frequencies present in the data set, during the feature extraction phase. The reason for choosing these frequencies are the results of the study done in Ref. [12, 13, 20].

According to the results obtained (Figure 4), the highest classification performance for the first participant was 83.30% in the Ensemble Learning classifier, the highest 100% classification performance for the second participant was in the KNN and SVM classifiers, and 88.90% for the third participant in the KNN classifier. Finally, in the fourth participant, it was seen again in the Ensemble Learning classifier with 77.80%.

When the results are evaluated considering the classifiers, the performance of the six different classifiers was calculated by taking the average of the four participants and the highest performance was found in the Ensemble Learning classifier with an accuracy of 79.73%.

3.2.3 Binary classification results

Considering the averages of the binary classification results of frequency features, the performances obtained vary between the lowest 70.85% and the highest 100%. Accordingly, the highest performance was determined with 100% accuracy value in 7.5–10 frequency pairs.

When the results are evaluated in terms of classifiers, it is clearly seen in Figure 6 that the classifier with the highest accuracy rate is the Ensemble Learning classifier. Runner-up classifier is the SVM classifier. Other classifiers following Ensemble Learning and SVM were identified as KNN, Logistic Regression and Naive Bayes classifiers, in order. It is also seen that no successful results have been obtained in the LDA and Decision Tree classifiers.

3.3 Wavelet transform features results

This section aims to analyze three crucial features, such as energy, variance, and entropy, which are frequently used in DWT studies, have been extracted from the bands (delta, theta, alpha, beta, and gamma) of the EEG signal. These features were generated for six different mother wavelets (Haar, db4, sym4, coif1, bior3.5, rbio2.8) commonly used in the literature. The results of each were evaluated in detail for multiple, binary, and three selected frequencies.

3.3.1 Multiple classification results

On the basis of mother wavelet selection, the results in (Table 4), reveal that Bior3.5 and Coif1 mother wavelets were relatively successful, although there is no dominant wavelet type. Experimenting with a larger sample size (number of subjects), in order to generalize, can help obtain more precise results.

In contrast to the mother wavelet selection, when the classifiers are evaluated, the success of Ensemble learning and LDA classifiers is clearly seen.

3.3.2 Classification results for three selected frequencies

In this analysis, as in the classification of frequency-domain features (Section 3.2.2), multiple classification was made by selecting 3 selected frequencies (6 Hz - 8.2 Hz – 10 Hz) where the differences between the frequencies were higher among the seven frequencies. However, unlike the analysis made in the frequency-domain, the selected features are classified and evaluated both they are used together, that is, when energy, variance and entropy features are used as a single feature vector (all features together, and they are used as separate features. Thus, detailed information about the power, irregularity and bias of the signal was obtained. At the same time, it is learned how to use these three features, which have the indispensable properties of the signal, more effectively. And the contribution of these features, which are frequently used in the literature, as a new form of features is wanted to be shown.

In Figure 7, the ACC values obtained by classification of the energy, entropy, and variance features extracted using each wavelet family are presented. Mean, minimum and maximum values of the classification results were also shown. According to these results, the values given by the Haar wavelet function for energy, entropy, and variance feature groups, which yield more successful results than other wavelet functions, were 75.85%, 73.08%, and 73.75%, respectively. There were no major differences between the mean values of the features extracted based on the Haar wavelet. However, it was seen that the entropy feature group had a 100% success rate compared to the others.

In Figure 8, the extracted features based on wavelet were used as a feature set, and the successful performances of the wavelet families were compared in this way. It was seen that the most successful wavelet family was the Haar wavelet function. The ranking of success in other wavelet families has not changed. The accuracy values are as follows: 75.85% with Haar mother wavelet, 67.53% with bior3.5 mother wavelet, 60.85% with db4 mother wavelet, 56.25% with coif1 mother wavelet, 52.35% with rbio2.8 mother wavelet and 44.73% with sym4 mother wavelet obtained. It was seen that some mother wavelet performances increased when compared with the ACC values in which the features in Figure 7 were handled separately. Mean values of coif1, db4, and sym4 mother wavelet functions increased.

As a result of the classification processes performed separately for each subject, when the performances of both feature groups were examined, the most successful wavelet function was found as the Haar wavelet. When the average accuracy values of the feature groups are examined, the results in the case that the three features are used as a single feature vector gave higher results for all wavelet functions than the other feature group. Although there is no dominant result in the comparison of energy, entropy, and variance features among themselves, the highest result was seen in the entropy feature in Subject 3 with 100%.

The results of classifiers to be expressed in the pie chart in Figure 9 are the number of hits of the classifiers obtained. With reference to results obtained, it is obvious that the most successful and also the most frequent classifier in the classification was obtained as the Ensemble classifier.

3.3.3 Binary classification results

In this analysis, feature vectors are treated as a single feature vector and individual (separate) feature vectors, similar to those in Section 3.2.3. The resulting feature vectors were then evaluated by binary classification in order to analyze frequencies in detail. As the results of the experimental design, the classification performances are obtained for:

  • three features separately (energy, entropy and variance),

  • average of the three features separately (Mean),

  • the extracted features were grouped as a single feature set (All features together).

Each feature (energy, entropy, variance and all features together) extracted using each wavelet family. All values of the classification results are presented in Figures 1015 for each mother wavelet, respectively.

According to these results, features obtained from the Haar wavelet function yielded higher accuracies than those obtained from the other wavelet functions. Maximum accuracy performances were obtained in the frequency pairs “6–10”, “6.5–8.2”, “6.5–10” in the Haar wavelet (Table 5). When the features are evaluated, it is realized that the “All features together” feature generally has better results for all mother wavelet functions.

And another researched hypothesis results are presented in Figure 16 for each mother wavelet, respectively. The purpose here is to show the change in the accuracy value according to the increase in the difference between the frequencies.

Finally, classification results obtained are presented in Figure 17. Since the classification results of all the features ranking are similar for all the wavelet functions, the classification result of the “All features together” for Haar wavelet function is presented. According to these results, the most successful classifier was obtained as the Ensemble classifier.

Advertisement

4. Conclusions

This chapter aimed to achieve significant optimization of cortical visual responses, signal processing methods, and ML algorithms, as well as the accuracy and reliability of the superior multi-command SSVEP-based BCI system. New approaches have been explored using existing methods to develop an accurate, reliable, comfortable SSVEP-based BCI that can offer people with severe motor neuron diseases a communication alternative using attention modulation without requiring neuromuscular activities or eye movements.

As a result, the following research objectives were achieved in this study:

  • When the results of the time-domain features are evaluated first, it can be seen that these features give usable (noteworthy) results in the classification of SSVEP signals. However, given the natural structure of the SSVEP signal, it is a fact that the results obtained are not sufficient for a real-time SSVEP-based BCI design, since the time-domain properties do not reflect the characteristics of the signal alone.

  • According to the classification results of the frequency-domain features, were evaluated alone, satisfactory results were obtained. Higher accuracy values were obtained in both multi-classification and binary classification compared to time-domain.

  • And when the last feature group time-frequency domain features are used, using mother DWT functions, SSVEP signals are divided into frequency bands and energy, entropy and variance values of each band are calculated. In this way, feature vectors were created and feature vectors were used as, both separately and also together. Extracted feature vectors were tested with a binary, multiple and three selected classes classification method to see the relationship between seven different classifiers and each frequency in detail. Although multiple classification results seem to be low for all feature groups, there is no study with 7 frequencies (by command) when the literature is searched according to the best knowledge of the author, but high results were obtained compared to studies with 3 and 4 frequencies.

  • For stimulation frequency detection in the SSVEP signal, a new form has been proposed that has been proven to be more effective with respect to the use of energy, entropy and variance features than the properties derived from the frequency domain and time-frequency domain. According to this form, instead of the energy, entropy and variance properties used separately, the feature vector, which is all features together, gave better results than the others.

  • By conducting detailed research on stimulation frequencies, frequency pairs estimated with the highest accuracy were determined. Although this result showed small differences between the mother wavelet functions, the highest performance was obtained in the frequency pairs in which the difference was generally high (6–10, 6.5–10, 7–10, and 7.5–10 Hz).

  • In the literature, the performances of the classifier types that were not compared before were evaluated in terms of SSVEP detection and the most successful classifier was found to be the “Ensemble Classifier”.

  • Also, does system performance increase in parallel with the differences between frequencies? Based on this hypothesis, the relationship between frequencies was investigated in pairs. A decrease in “Sym4” function was observed, where only the lowest performances were obtained.

  • Finally, the most successful mother wavelet selection was made. Accordingly, it was the Haar wavelet function that gave the best results compared to others.

Advertisement

Conflict of interest

The authors declare no conflict of interest.

Advertisement

Thanks

We would like to thank Adnan Vilic for his support in providing SSVEP records.

References

  1. 1. Wolpaw JR, Boulay CB. Brain signals for brain–computer interfaces. In: Graimann B., Pfurtscheller G., Allison B, editors. Brain-Computer Interfaces. The Frontiers Collection. Springer: Heidelberg; 2009. p. 29-46. DOI: 10.1007/978-3-642-02091-9_2
  2. 2. Graimann B, Allison B, Pfurtscheller G. Brain–computer interfaces: A gentle introduction. In: Graimann B., Pfurtscheller G., Allison B, editors. Brain-Computer Interfaces. The Frontiers Collection. Springer: Heidelberg; 2010. p. 1-27. DOI: 10.1007/978-3-642-02091-9_1
  3. 3. Mason SG, Birch GE. A general framework for brain-computer interface design. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2003;11(1): 70-85. DOI: 10.1109/TNSRE.2003.810426
  4. 4. Ramadan RA, Vasilakos AV. Brain computer interface: Control signals review. Neurocomputing. 2017;223: 26-44. DOI: 10.1016/j.neucom.2016.10.024
  5. 5. Abiri R, Borhani S, Sellers E, Jiang Y, Zhao X. A comprehensive review of EEG-based brain-computer interface paradigms. Journal of Neural Engineering. 2019;16 011001. DOI: 10.1088/1741-2552/aaf12e
  6. 6. Basar E. EEG-brain dynamics: relation between EEG and brain evoked potentials. 1st ed. Brain Lang Elsevier; 1980. 411 p
  7. 7. Wang Y, Gao X, Hong B, Jia C, Gao S. Brain-computer interfaces based on visual evoked potentials. IEEE Engineering in Medicine and Biology Magazine. 2008;27(5): 64-71. DOI: 10.1109/MEMB.2008.923958
  8. 8. Regan D. An effect of stimulus colour on average steady-state potentials evoked in man. Nature. 1966;210:1056–1057
  9. 9. Gao S, Wang Y, Gao X, Hong B. Visual and auditory brain-computer interfaces. IEEE Transactions on Biomedical Engineering. 2014;61(5):1436–1447. DOI: 10.1109/TBME.2014.2300164
  10. 10. Zhang Y, Xie SO, Wang H, Zhang Z. Data analytics in steady-state visual evoked potential-based brain–computer interface: A review. IEEE Sensors Journal. 2021;21(2):1124-1138. DOI: 10.1109/JSEN.2020.3017491
  11. 11. Huang X, Xu J, Wang Z. A novel instantaneous phase detection approach and its application in SSVEP-based brain-computer interfaces. Sensors. 2018; 18(12):4334. DOI:10.3390/s18124334
  12. 12. Sayilgan E, Yuce YK, Isler Y. Evaluation of wavelet features selected via statistical evidence from steady-state visually-evoked potentials to predict the stimulating frequency. Journal of the Faculty of Engineering and Architecture of Gazi University. 2021;36(2):593-605. DOI:10.17341/gazimmfd.664583
  13. 13. Sayilgan E, Yuce YK, Isler Y. Evaluation of mother wavelets on steady-state visually-evoked potentials for triple-command brain-computer interfaces. Turkish Journal of Electrical Engineering & Computer Sciences. 2021;29(3). DOI:10.3906/elk-2010-26
  14. 14. Sayilgan E, Yuce YK, Isler Y. Investigating the effect of flickering frequency in steady-state visually-evoked potentials on dichotomic brain-computer interfaces. Innovation and Research in BioMedical Engineering. 2021;Under Review
  15. 15. Zhang Z, Li X, Deng Z. A CWT-based SSVEP classification method for brain-computer interface system. In: 2010 International Conference on Intelligent Control and Information Processing; 13-15 Aug. 2010; Dalian, China. 2010. pp. 43-48. DOI: 10.1109/ICICIP.2010.5564336
  16. 16. Bian Y, Li H, Zhao L, Yang G, Geng L. Research on steady state visual evoked potentials based on wavelet packet technology for brain-computer interface. Procedia Engineering. 2011;15:2629-2633. DOI: 10.1016/j.proeng.2011.08.494
  17. 17. Vilic A. AVI steady-state visual evoked potential (SSVEP) signals dataset 2013 [Internet]. Available from: https://www.setzner.com/avi-ssvep-dataset/. [Accessed 15th August 2018]
  18. 18. Sutter EE. The brain response interface-communication through visually induced electrical brain responses. Journal of Microcomputer Applications. 1992;15(1):31-45
  19. 19. Bisht A, Srivastava S, Purushothaman G. A new 360° rotating type stimuli for improved SSVEP based brain computer interface. Biomedical Signal Processing and Control. 2020;57:101778. DOI:10.1016/j.bspc.2019.101778
  20. 20. Sayilgan E, Yuce YK, Isler Y. Prediction of evoking frequency from steady-state visual evoked frequency. Natural and Engineering Sciences. 2019;4(3): 91-99
  21. 21. Sayilgan E, Yuce YK, Isler Y. Estimation of three distinct frequencies using fourier transform of steady-state visual-evoked potentials. Duzce University Journal of Science and Technology. 2020;8(4):2337-2343. DOI: 10.29130/dubited.716386
  22. 22. Liu W, Zhang L, Li C. A method for recognizing high-frequency steady-state visual evoked potential based on empirical modal decomposition and canonical correlation analysis. In: 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC); 15–17 March 2019; Chengdu, China. 2019. p. 774-778. DOI:10.1109/ITNEC.2019.8729005
  23. 23. Chen YF, Atal K, Xie SQ, Liu Q. A new multivariate empirical mode decomposition method for improving the performance of SSVEP-based brain-computer interface. Journal of Neural Engineering. 2017;14(4):046028. DOI:10.1088/1741-2552/aa6a23
  24. 24. Gandhi T, Panigrahi KB, Anand S. A comparative study of wavelet families for EEG signal classification. Neurocomputing. 2011;74(17):3051-3057. DOI: 10.1016/j.neucom.2011.04.029
  25. 25. Cao Z, et al. Extraction of SSVEPs-based inherent fuzzy entropy using a wearable headband EEG in migraine patients. IEEE Transactions on Fuzzy Systems. 2020;28(1):14-27. DOI:10.1109/TFUZZ.2019.2905823
  26. 26. Alpaydin E. Introduction to Machine Learning: MIT Press; 2004. 712 p
  27. 27. Duda RO, Hart PE, Stork DG. Pattern Classification: John Wiley & Sons;2001
  28. 28. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F. A review of classification algorithms for EEG-based brain-computer interfaces: A 10-year update. Journal of Neural Engineering. 2018;15(3):1-28. DOI: 10.1088/1741-2552/aab2f2
  29. 29. Sayilgan E, Yuce YK, Isler Y. Determining gaze information from steady-state visually-evoked potentials. Karaelmas Science and Engineering Journal. 2020;10(2):151-157. DOI: 10.7212/zkufbd.v10i2.1588
  30. 30. Narin A, Isler Y, Ozer M. Comparison of the effects of cross validation methods on determining performances of classifiers used in diagnosing congestive heart failure. DEÜ Mühendislik Fakültesi Mühendislik Bilimleri Dergisi. 2014;16(48):1-8
  31. 31. Jung Y, Hu J. A k-fold averaging cross-validation procedure. Journal of Nonparametric Statistics. 2015;27:1-13. DOI: 10.1080/10485252.2015.1010532
  32. 32. Jiao Y, Du P. Performance measures in evaluating machine learning based bioinformatics predictors for classifications. Quantitative Biology. 2016;4(4): 320–330. DOI: 10.1007/s40484-016-0081-2
  33. 33. Cetin V, Ozekes S, Varol HS. Harmonic analysis of steady-state visual evoked potentials in brain computer interfaces. Biomedical Signal Processing and Control. 2020;60(2020):101999. DOI: 10.1016/j.bspc.2020.101999

Written By

Ebru Sayilgan, Yilmaz Kemal Yuce and Yalcin Isler

Submitted: 09 April 2021 Reviewed: 11 May 2021 Published: 15 June 2021