SSVEP-Based BCIs

Abstract

This chapter describes the method of flickering targets, eliciting fundamental frequency changes in the EEG signal of the subject, used to drive machine commands after interpretation of user’s intentions. The steady-state response of the changes in the EEG caused by events such as visual stimulus applied to the subject via a computer screen is called steady-state visually evoked potential (SSVEP). This feature of the EEG signal can be used to form a basis of input to assistive devices for locked in patients to improve their quality of life, as well as for performance enhancing devices for healthy subjects. The contents of this chapter describe the SSVEP stimuli; feature extraction techniques, feature classification techniques and a few applications based on SSVEP based BCI.

1. Introduction

1.1. Evoked potential

Evoked potentials (EP) are the electrical signals measured from the scalp after the stimulation rendered by some external stimulus. Corresponding to various stimuli, evoked potentials are distinguished as visual, auditory and somatosensory evoked potentials.

Event related potential (ERP) implies both EP and brain responses prompted by cognitive processes evolved by external stimuli or precursory mechanisms for motor action [1, 2].

1.2. Visual evoked potential

Visual evoked potentials (VEPs) are the brain activity modulations occurring in the visual cortex after encountering visual stimulus [3]. They are easy to detect as the movement of stimulus closer to the central visual field immensely enhances the amplitude of VEPs [4].

Based on following criteria, VEPs are classified into different categories [5]:

1. Morphology of the optical stimuli

   1. VEPs caused by flash stimulation

   2. VEPs caused by graphic patterns like checkerboard lattice, gate

2. Frequency of visual stimulation

   1. Transient VEPs (TVEPs): VEPs with visual stimulation frequency below 6 Hz

   2. Steady-state VEPs (SSVEPs): VEPs with visual stimulation frequency above 6 Hz [3, 6]

3. Field stimulation

   1. Whole field VEPs

   2. Half field VEPs

   3. Part field VEPs

While the user needs to gaze at the screen and keep his eyes fixed on one particular point. These exogenous signals are not suitable while dealing with advanced level amyotrophic lateral sclerosis (ALS) patients or users with uncontrollable eye or neck movements [7].

1.3. Steady-state visual evoked potential

Regan experimented with long trains of stimuli that comprised of sinusoidally modulated monochromatic light [8]. Small amplitude stable VEP were generated which were entitled “steady-state” visually evoked potentials (SSVEPs) of the human visionary system. There hence, steady-state visual evoked potentials (SSVEPs) are defined as the potential elicited by the change in the visual field with the frequency higher than 6 Hz.

When a user is presented with some periodic stimuli, SSVEP is generated strongly at the occipital areas of the brain [10]. SSVEP is usually acquired from various electrode sites like Oz, O1, O2, Pz, P3, P4, and some surrounding locations to occipital region. While the most commonly used SSVEP frequency range is 4–60 Hz, the resonance phenomenon is generally observed around 10, 20, 40 and 80 Hz [11].

1.4. Feature extraction

Various features are procured from the properties of the brain signals that have discriminative information embedded in them. Various feature extraction techniques are used to extract such features when overlapped in time and space by several brain signals.

The feature extraction in SSVEP signals was often done with the study of amplitude in the target frequency [12, 13, 14], followed by independent component analysis (ICA) [16], the fast Fourier transform (FFT) [15], continuous wavelet transform (CWT) [17, 18], Hilbert-Huang transform (HHT) [19, 20] or the PSD [21] can be used.

1.4.1. Independent component analysis (ICA)

ICA follows a statistical procedure for separating a set of mixed signals into its sources without any presumptuous information regarding the nature of the signal. The only criteria that need to be followed are that the unspecified underlying sources must be statistically mutually independent. ICA can express an EEG signal as following:

x(t) = f(s(t)) + n(t)E1

where, f is some unknown mixer function, s(t) is the source vector, n(t) is the additive arbitrary noisy vector and x(t) is the resultant EEG signal. ICA mainly follows two approaches: spatial ICA that extricates out independent spatial maps and temporal ICA that extricates out independent time courses.

EEG over the visual cortex was fragmented into SSVEP signals and background noise using ICA in the study by Wang et al. [16].

1.4.2. Fast Fourier transform (FFT)

Fourier transform (FT) comprises fast Fourier transform (FFT) and fiscrete Fourier transform (DFT). Wang et al. used 256-point FFT for transforming EEG signals to corresponding frequency domain representation. This was represented in terms of five frequencies: 9, 11, 13, 15 and 17 Hz [21]. Mouli et al. considered the maximum amplitudes of the FFT as the prime parameter for differentiating various target stimuli of 7, 8, 9 and 10 Hz [22].

1.4.3. Spatial filtering

The most commonly endorsed EEG recording methods emanated monopolar signals or Laplacian filtered signals. For example, channel Oz consisted as follows:

Monopolar:$${\mathrm{y}}_{\mathrm{Ox}}^{\mathrm{M}}={\mathrm{y}}_{\mathrm{Ox}}-{\mathrm{y}}_{\mathrm{A}2}$$

Laplacian:$${\mathrm{y}}_{\mathrm{Ox}}^{\mathrm{L}}={\mathrm{y}}_{\mathrm{Ox}}^{\mathrm{M}}-\frac{{\mathrm{y}}_{\mathrm{O}1}^{\mathrm{M}}+{\mathrm{y}}_{\mathrm{O}2}^{\mathrm{M}}}{2}$$

Laplacian signals are extracted considering both the sides (like Cz utilizes C3 and C4) [23].

1.4.4. Continuous wavelet transform (CWT)

Wavelet transform (WT) is best suited to extract information from nonstationary signals as it extends a versatile method for representation of time-frequency of a signal [24]. CWT is basically the convolution of signal with the wavelet function [25]:

$$w\left(s,\tau \right)=\underset{-\infty }{\overset{\infty }{\int }}x\left(t\right)\psi \ast \left(t\right)\mathit{dt}$$ E2

where, ψ*(t) is the complex conjugate wavelet function, x(t) is the particular function and w(s,τ) is the wavelet coefficient corresponding to frequency related with scale s and time τ of the involved wavelet function. CWT works like template matching just like matched filter where cross variance is calculated for the signal and some predefined waveform [26].

Zhang et al. established the use of CWT technique for extracting features and classifying them in SSVEP-based BCI [16]. Kumari et al. transformed the CWT coefficients into feature vectors for tracing out the site of high frequency SSVEP components [17].

1.4.5. Hilbert-Huang transform (HHT)

HHT is a self-adaptive data analysis technique comprising empirical mode decomposition (EMD) and Hilbert spectral analysis (HAS) [27]. It can opt stationary and non-stationary signals analysis. An intrinsic mode function (IMF) is an oscillator function with time-varying frequencies capable of depicting the local properties of non-stationary signals [28].

Huang et al. identified high frequency SSVEP signals using HHT in SSVEP-based BCI [18]. HHT remodeled the original signals into 11-order IMF with the help of EMD.

1.5. Feature classification

The classification algorithms designate boundaries between various targets in the feature space on the basis of feature vectors involved considering the as independent variables.

The classification methods for SSVEP signals are heterogeneous in nature, like Bayesian classifier [14], linear discriminant classifier (LDA) [29, 30, 31], support vector machine (SVM) [32, 33, 34, 35], k-nearest neighbor (k-NNC) [31, 36].

1.5.1. Bayesian classifier

Bayesian statistical classifier obtains the posterior probability P(y|x) as per prior probability of a feature vector for belonging to some particular class. The class that has got the maximum probability is the one to which the particular feature vector belongs.

$$\mathrm{P}\left(\mathrm{y}|\mathrm{x}\right)=\frac{\mathrm{P}\left(\mathrm{y}\right)\mathrm{P}\left(\mathrm{x}|\mathrm{y}\right)}{\mathrm{P}\left(\mathrm{x}\right)}$$ E3

1.5.2. Linear discriminant classifier (LDA)

LDA or Fisher’s LDA (FLDA) classifies the data into various classes using hyper planes [36]. This classifier is successfully applied in BCI community despite of high computational time involvement. This classifier traces out an optimal projection that maximizes the distance between the classes. The decision hyper plane that divides the feature space into various classes is perpendicular to the projection direction [37]. The hyper plane is expressed as:

m(x) = w^T_x + w_oE4

where, w, x and w_o implies the weight vector, the input feature vector and the threshold, respectively.

1.5.3. Support vector machine (SVM)

SVM classifies the feature vectors into various classes by the concept of construction of one or more hyper planes. This classifier differs from LDA, as in this, the decision boundary or hyper plane escalates the margins that implies, the distance between the decision boundary and the training sample nearest to it [38]. While the hyper plane separates the training data set with maximal margin, it also maps them to a higher dimensional space [39]. The decision boundary followed up in SVM may be linear as well as non-linear depending upon the choice of kernel function (linear, cubic, polynomial, Gaussian or radial basis (RBF)) [40].

1.6. k-Nearest neighbor (k-NNC)

The classification principle of k-NNC is that the features belonging to different classes get flocked up in different clusters while keeping the adjacent neighbors in one cluster. It considers k metric distances between the testing dataset features and those of the nearest classes for classifying a test feature vector. Although classification with k-NNC reduces the error probability in the decision but still it is not so commonly used in BCI community [41].

2. SSVEP in BCIs

A BCI is an artificial intelligence system that has the ability to identify particular set of patterns in the brain signals to provide an additional output channel for the control of artificial devices like restoring motor function, robot arm, communication program, etc. [43, 44].SSVEPs based BCIs are classified into following categories:

1. Time modulated VEP (t-VEP) BCI: In this BCI, the follow up of flash sequences of various stimuli are orthogonal in time, that is, they are strictly non overlapping or stochastic in nature.

2. Frequency modulated VEP (f-VEP) BCI: In this BCI, stimuli are made to flash at some exclusive frequency and the potential evoked is generated with the fundamental frequency same as that of the stimuli as it harmonics.

3. Pseudorandom code modulated (c-VEP) BCI: In this BCI, a pseudorandom sequence defines the duration of ON and OFF states of each stimulus. This mode yields highest communication speed.

2.1. Stimulus types

In SSVEP-based experiments, the user is asked to identify the target with eye-gaze. The attention of the user is supposed to be visually fixed on the target and the target is identified by feature extraction and its analysis [42]. In case of single graphic stimuli, stimulus appears and disappears at some particular rate just like displayed in Figure 1. In case of pattern reversal stimuli, at least two graphical patterns are displayed by alternative oscillations like shown in Figure 2. Such stimulus maybe of checkerboard or grating type.

Figure 1.

Single graphic stimuli.

Figure 2.

Pattern reversal stimuli.

With flashing stimulus, SSVEP appears as a sinusoidal-like waveform with fundamental frequency as that of blinking frequency of the stimulus. With graphic pattern stimulus, SSVEP appears at the reversal rate and their harmonics [8]. The SSVEP discrete frequency components stay intently constant in terms of amplitude and phase for long time [9].

2.2. Applications of SSVEP in BCI’s

2.2.1. SSVEP for BCI based wheelchair

Singla in 2014 spearheaded the research on the effects of stimuli color, of the flickering targets, on the accuracy of decision making to drive a wheelchair. In the study, SSVEPs were selected as compared to VEP because they are less vulnerable to artifacts produced by the eye blinks, eye movements as well as EMG noise [44].

SSVEP data was acquired, which originated due to four different flickering target frequencies, from the occipital region of the brain. The frequency features of the data were extracted using fast Fourier transform (FFT) and wavelet transform (WT). Three different classification methods were tried, two based on ANN with back propagation algorithm and one based on SVM with One against All (OAA) strategy. The control signals were assigned for each of the five classes detected (7, 9, 11, 13 and rest signal). Corresponding to five classes, five movement positions such as forward (F), backward (B), left (L), right (R) and stop (S) were obtained.

The SSVEP stimulus produces a response in the EEG signal, which is characterized by oscillations of the order of the stimulation frequency and sometimes at harmonics or sub harmonics of it. The visual system can be divided into three subsystems [45].

(i) Low frequency subsystem (near 10 Hz). It gives the greatest SSVEP amplitudes.

(ii) Medium frequency subsystem (16–18 Hz). It gives medium amplitude.

(iii) High frequency subsystem (40–50 Hz). It shows the smallest response.

The ability of the human eye to distinguish colors is based upon the varying sensitivity of cone cells to the light of different wavelengths [46]. There are three kinds of cone cells and are conventionally labeled as short (S), medium (M), and long (L) cones according to the wavelengths of the peaks of their spectral sensitivities. S, M and L cone cells are therefore sensitive to blue (short-wavelength), green (medium-wavelength) and red (long-wavelength) light respectively. The brain combines the information from each cone cells to give different perceptions for different colors and as a result the SSVEP strength elicited with different colors of the stimuli will be different [46]. In this work blue, green, red and violet were selected as stimuli colors to explore how different colors influence the elicited SSVEPs and the performance of SSVEP based system.

The research used, repetitive visual stimuli (RVS) with four different flickering frequencies was designed by using LabVIEW software (National Instrument Inc., USA). The front panel of RVS is shown in Figure 3. RVS with violet, red, green and blue flickering bars were designed as four different sets. The back ground color of the RVS was selected as black. The visual stimuli were square (4 × 4 cm) in shape and were placed on the four corners of the LCD screen. Four frequencies 7, 9, 11 and 13 Hz, i.e., in the low frequency range were selected by considering 60 Hz refreshing rate of LCD monitor [45]. In order to select any particular stimuli the four visual stimuli were separated in pair of two each, i.e., 7, 11 and 9, 13. Further in an interval of 2 s if eye blink once then first pair is selected, i.e., 7, 11 and if eye is blinked twice then the next pair is selected, i.e., 9, 13. Once a pair of stimuli is selected then again in next interval of 2 s if eye blink once then upper stimuli is selected and if it is blinked twice then the lower stimuli is selected in that pair of stimuli.

Figure 3.

Visual stimuli with four different flickering frequencies.

The research used, repetitive visual stimuli (RVS) with four different flickering frequencies was designed by using LabVIEW software (National Instrument Inc., USA). The front panel of RVS is shown in Figure 3. RVS with violet, red, green and blue flickering bars were designed as four different sets. The back ground color of the RVS was selected as black. The visual stimuli were square (4 × 4 cm) in shape and were placed on the four corners of the LCD screen. Four frequencies 7, 9, 11 and 13 Hz, i.e., in the low frequency range were selected by considering 60 Hz refreshing rate of LCD monitor [45].

In order to select any particular stimuli the four visual stimuli were separated in pair of two each, i.e., 7, 11 and 9, 13. Further in an interval of 2 s if eye blink once then first pair is selected, i.e., 7, 11 and if eye is blinked twice then the next pair is selected, i.e., 9, 13. Once a pair of stimuli is selected then again in next interval of 2 s if eye blink once then upper stimuli is selected and if it is blinked twice then the lower stimuli is selected in that pair of stimuli.

The EEG signals recorded from each channel were digitized and segmented into 1-s time window in every 0.25 s. The coefficients of first (fundamental frequency) and second harmonic of all the four target frequencies were considered as the feature vector for classification. It can be seen from Table 1 that for SSVEP input of 7 Hz, maximum values of amplitude exists at 7, followed by 14.

7	14	9	18	11	22	13	26	Stimulus frequency (Hz)
27.62	8.01	9.2	5.25	5.50	1.81	4.61	1.40	7
18.21	7.40	6.97	1.42	7.92	2.34	0.99	2.38	7
2.65	4.17	23.02	9.91	9.2	1.15	1.00	2.22	9
3.57	6.02	20.4	7.83	4.04	2.52	0.70	1.13	9
11.72	3.62	2.25	2.92	19.91	5.20	3.91	2.24	11
6.83	4.60	4.7	2.40	14.22	3.40	1.42	1.40	11
3.27	6.82	11.83	4.85	9.19	2.02	16.63	4.83	13
8.81	3.82	12.7	5.25	3.62	0.91	14.22	5.66	13
4.75	6.60	5.00	1.09	2.55	1.42	6.48	1.53	Relax
2.44	3.14	5.06	2.34	3.62	1.65	6.36	3.11	Relax

Table 1.

Samples of extracted feature components of different frequencies and relax state for two subjects.

In case of ANN, there were total eight parameters (first and second harmonics of all the four frequencies) so the input vector contains eight rows. Another set of Q target vectors (the correct output vectors in four digits for each of the input vectors) formed a second matrix. They developed wheelchair prototype to control in forward, backward, left, right and stop positions. The schematic representation of the BCI wheelchair control is shown in Figure 4. The wheelchair prototype is shown in Figure 5. Motor driver IC, L293D (www.instructables.com) was used. By changing the polarity of the signal given to the motors, it moves each of the motors in both forward and backward directions [32].

Figure 4.

Schematic representation of BCI-based wheelchair control.

Figure 5.

Wheelchair prototype for SSVEP-based BCI control.

2.2.2. SSVEP based BCI as independent application for locked-in syndrome

Lesenfants et al. in [47] conducted studies with a basic aim of developing independent SSVEP based BCI applications for locked in patients. Lesenfants et al. used the covert attention of healthy as well as locked-in patients by developing an independent, covert two-class paradigm of flashing targets. The study was divided over two groups of subjects. Group A consisted of 12 healthy subjects and Group B consisted of 12 healthy and 6 Locked-in Syndrome (LIS) patients. For both the groups 12 channels of EEG were recorded (P3, P1, P2, P4, PO7, PO3, POz, PO4, PO8, O1, Oz, and O2).

The visual stimulation was delivered via a custom made stimulus device, which had two subsystems: a control unit and a stimulation panel, based on the paradigm introduced in [48]. The panel, placed at 30 cm from subject’s head, was a 7 × 7 cm² “interlaced square” made of red and yellow 1 × 1 cm² light emitting diode (LED) - squares with a white fixation cross in the middle (Figure 6).

Figure 6.

Electronic visual stimulation unit.

The yellow squares (represented by white squares here) flicker at the frequency of 10 Hz. The red squares (represented by grey squares here) flash at 14 Hz.

The interlaced square pattern showed a 10% improvement in accuracy in comparison with a “line” pattern [49]. The control unit was designed to precisely control the red and yellow flickering frequencies independently between 1 and 99 Hz by microcontroller based circuit. The yellow and red squares were programmed to flicker at 10 and 14 Hz, respectively. The pattern was composed of two 2 × 2 cm² blocks made of 1 × 1 cm² LED squares separated by 12 cm with a white fixation cross in between (Figure 7).

Figure 7.

Overt block pattern.

The subjects were asked 33 yes/no questions (e.g., “is your name Paul?”). To answer “yes,” the subjects had to focus their attention over yellow flashes for 7 s or over the red for “no.” Epochs of 7 s were used as a unique window, where after four different feature extraction algorithms like DFT, multitaper spectral analysis (PMTM) [53, 54], CCA, lock-in analyser system (LAS) [49, 50, 51]. A automatic channel selection algorithm (ACSA) based on distinction sensitive learning vector quantization (DSLVQ) [52] selected an optimal channel set specific to each subject out of the 12 available channels. Classification was performed using LDA or a SVM (linear kernel), and assessed with a 10 × 10 fold cross validation.

PMTM obtained maximum accuracy of 77.0 ± 3.4% averaged over subject population, while LAS produced a similar mean accuracy of 74.4 ± 3.2% (Tables 2 and 3). DFT and CCA gave worse results as compared to PMTM and LAS (respectively, 69.4 ± 3.4% and 58.4 ± 3.9%).

Subject (Group A)	Nharm = 1		Nharm = 2		Nharm = 3
	AC	ACSA	AC	ACSA	AC	ACSA
Subject 1	78.4 ± 3.4	85.7 ± 1.8	73.9 ± 4.7	94.3 ± 1.8	—	94.4 ± 2.2
Subject 2	92.3 ± 2.4	94.8 ± 1.0	76.5 ± 4.9	91.3 ± 1.6	—	92.6 ± 1.1
Subject 3	78.4 ± 3.4	84.4 ± 2.1	73.9 ± 4.7	84.9 ± 2.6	—	80.6 ± 3.0
Subject 4	67.8 ± 3.5	76.0 ± 2.3	51.7 ± 4.7	76.7 ± 3.7	—	78.6 ± 3.2
Subject 5	62.4 ± 3.9	78.9 ± 2.3	51.6 ± 4.8	77.4 ± 2.4	—	70.5 ± 3.2
Subject 6	71.6 ± 3.8	85.8 ± 2.6	62.6 ± 5.0	81.9 ± 3.3	—	85.6 ± 2.9
Subject 7	89.5 ± 2.4	94.5 ± 1.4	76.0 ± 3.9	92.8 ± 2.4	—	92.6 ± 1.9
Subject 8	82.7 ± 3.3	89.2 ± 2.3	77.3 ± 4.3	94.4 ± 1.9	—	87.2 ± 1.7
Subject 9	91.1 ± 2.3	94.6 ± 1.6	84.0 ± 4.3	91.7 ± 1.1	—	91.8 ± 1.3
Subject 10	56.6 ± 4.4	63.5 ± 1.8	58.3 ± 4.7	67.5 ± 3.0	—	68.4 ± 2.4
Subject 11	57.0 ± 3.8	—	43.1 ± 5.0	—	—	—
Subject 12	44.5 ± 3.8	—	43.2 ± 5.8	—	—	—
Total	77.0 ± 3.4	84.7 ± 2.0	68.6 ± 4.6	85.3 ± 2.5	—	84.2 ± 2.4

Table 2.

Mean and standard deviation of classification accuracy (in percent) obtained with the Thomson multitaper method (PMTM) for different numbers of harmonics with (ACSA) and without (AC) the use of automatic channel selection algorithm.

Subject (Group A)	Nharm = 1		Nharm = 2		Nharm = 3
	AC	ACSA	AC	ACSA	AC	ACSA
Subject 1	76.7 ± 3.1	85.0 ± 1.9	73.2 ± 4.9	93.9 ± 1.8	—	94.7 ± 1.2
Subject 2	86.5 ± 2.1	93.5 ± 0.8	74.8 ± 5.2	95.4 ± 0.9	—	95.0 ± 1.7
Subject 3	76.7 ± 3.1	82.5 ± 2.7	73.2 ± 4.9	79.4 ± 2.7	—	74.6 ± 2.6
Subject 4	61.0 ± 4.2	73.6 ± 3.5	58.3 ± 4.5	73.6 ± 2.2	—	77.6 ± 1.9
Subject 5	69.7 ± 2.9	80.2 ± 2.4	54.2 ± 4.6	76.1 ± 2.8	—	72.8 ± 1.9
Subject 6	61.4 ± 3.6	76.2 ± 2.0	60.8 ± 4.7	79.0 ± 2.2	—	77.7 ± 3.3
Subject 7	85.2 ± 2.8	90.0 ± 2.2	76.5 ± 4.7	91.2 ± 2.4	—	94.0 ± 2.4
Subject 8	83.0 ± 2.9	87.4 ± 2.2	75.9 ± 4.4	94.3 ± 2.2	—	91.5 ± 2.6
Subject 9	90.5 ± 2.4	92.9 ± 1.7	75.9 ± 4.2	90.9 ± 2.1	—	91.5 ± 1.8
Subject 10	53.7 ± 3.8	70.1 ± 2.4	61.7 ± 4.7	70.9 ± 3.0	—	71.4 ± 2.9
Subject 11	57.8 ± 4.3	—	48.5 ± 5.1	—	—	—
Subject 12	49.7 ± 4.3	—	54.0 ± 5.0	—	—	—
Total	74.4 ± 3.2	83.1 ± 2.8	68.4 ± 4.7	84.5 ± 2.3	—	84.1 ± 2.3

Table 3.

Mean and standard deviation of classification accuracy (in per cent) obtained with the lock-in analyzer system (LAS) for different numbers of harmonics with (ACSA) and without (AC) the use of automatic channel selection algorithm.

Another comparison was done with the results obtained from the feature extraction methods using the ACSA as well as a single harmonic. PMTM and LAS produced significantly greater accuracy than DFT and CCA, with an accuracy of 84.7 ± 2.0 and 83.1 ± 2.3%, respectively. DFT obtained a 79.3 ± 2.7% accuracy and CCA was able to attain 72.4 ± 1.6% but in only five out of the 10 subjects. The performance with and without ACSA could therefore not be compared with CCA. For a single harmonic, a significant mean accuracy increase of 7.8% for PMTM, 7.9% for LAS and 7.6% for DFT was obtained.

2.2.3. SSVEP based virtual gaming application

Martišius and Damaševičius in 2016 [55] proposed an SSVEP based BCI gaming system. The researchers developed a 3-class BCI system based on SSVEP and emotive EPOC Headset. The game involved target shooting developed in the OpenVIBE environment which provided the user feedback. Emotive EPOC, a 16 electrode based gaming headset, was used in combination with the SSVEP paradigm. Raw EEG data from the head set was acquired with internal sampling of 2048 Hz. Signals from the O1, O2, P7, and P8 were taken.

At first, data was split into three groups, according to their corresponding class labels, LEFT, RIGHT, and CENTER. Each group of signals was subjected to band-pass filter centered on the target frequency of interest: for the LEFT class, 29.5–30.5 Hz; CENTER, 19.5–20.5 Hz; RIGHT, 11.5–12.5 Hz.

There have been studies [46] that analyzed how different colors of the targets influence classification quality. The user was presented with an LCD display, containing three blinking targets on a black background and a yellow arrow. On cue, the targets start blinking at different frequencies as shown in Figure 8(a).

Figure 8.

SSVEP BCI game interface: (a) training and (b) playing.

After classifier training, subjects were invited to participate in the game experiment. During the game, the subjects were presented with an interface from Figure 8(b). The “spaceship” with two “engines,” represented by two rectangles, and a “cannon,” represented by the triangle. The subject could rotate the spaceship by focusing his/her attention on one of the rectangular targets.

By focusing attention on the middle triangle, the user was able to fire the spaceship cannon. The aim of the game is to rotate the spaceship and fire its canon to hit the red target. Once the target was hit, it disappeared to reappear in another position.

An evaluation of the system was performed using two subjects, named S1 and S2, unfamiliar with the BCI technology. The first algorithm used was wave atom transform (WAT) coefficients and the second algorithm used the band power (BP) in the stimulation frequency bands.

The accuracy was measured for each subject, while performing classification with 4 different classifiers (LDA, sparse LDA (sLDA), SVM with linear kernel, and SVM with RBF kernel (with parameter values, gamma = 10)). The results are depicted in Table 4.

Classifier	Features	Accuracy, %		F1 Score
		S1	S2	S1	S2
LDA	WAT	71.5	78.2	0.64	0.67
	BP	66.2	73.2	0.56	0.62
sLDA	WAT	70.6	77.4	0.64	0.68
	BP	68.4	73.5	0.59	0.61
SVM, linear kernel	WAT	75.5	79.3	0.64	0.68
	BP	74.3	75.1	0.64	071
SVM, RBF kernel	WAT	78.7	82.2	0.68	0.71
	BP	74.0	77.4	0.63	0.67

Table 4.

Comparison of classification accuracy.

S1: subject number 1, S2: subject number 2, LDA: linear discriminant analysis, sLDA: sparse LDA, SVM: support vector machine, RBF: radial basis function, WAT: wave atom transform, and BP: band power.

2.2.4. SSVEP based communicator/speller enhancement

Nakanishi et al. in [56] designed a high speed speller based on SSVEP stimulus. The study was aimed at exploring the feasibility of mixed frequency and phase coding to form a high speed speller using a TFT monitor. A frequency and phase approximation approach was deployed to remove the limitation of the number of targets caused by the monitor refresh rate, resulting in a speller comprising 32 flickers specified by eight frequencies (8–15 Hz at an interval of 1 Hz) and four phases (0, 90, 180, and 270°).

Wang et al. [57] proposed an approach that generates visual flickers at a flexible frequency by approximating the frequency with variable number of frames in a stimulation cycle. For instance, a flicker at 11 Hz under a 60 Hz refresh rate can be realized by bridging five and six frames in a stimulation cycle as “1110001110011100011100111…” Based on this technique we can generate flicker frequencies up to 50% of the screen refresh rate, hence increasing the number of stimuli that can be presented. Generally, to render a visual flicker at frequency f with an initial phase φ, a stimulus sequence s(f, φ,i) can be generated by:

$$\mathrm{s}\left(\mathrm{f},\mathrm{\phi },\mathrm{i}\right)=\text{square}\left[2\mathrm{\pi f}\left(\frac{\mathrm{i}}{\text{refesh rate}}\right)+\mathrm{\phi }\right]$$ E5

where the function square [] generates a square wave of 50% duty cycle with levels 0 and 1, and i indicates the frame index. Nakanishi et al. used quad-phase coded flickering signals at phases or 0, 90, 180, and 270, for frequencies 8–15 Hz with an interval of 1 Hz, hence providing 32 unique targets instead of just 8 as indicated in Figure 8.

The subjects were instructed to gaze at one out of the 32 visual stimuli (a target stimulus) for 4 s, and the other 31 targets were indicated in a random order in a run. At the beginning of each trial, a red rectangle marker (Figure 9) appeared for half a second highlighting the target stimulus. Subjects were asked to shift their gaze to the target within the same duration. After which, all the stimuli started to flicker simultaneously for 4 seconds. Seven runs were carried out for each subject EEG data were recorded by 16 electrodes over the parietal and occipital areas (FPz, F3, F4, Fz, Cz, P1, P2, Pz, PO3, PO4, PO7, PO8, POz, O1, O2, and Oz).

Figure 9.

Presentation of the 32-target visual stimuli using mixed frequency and phase coding.

The entire data epochs were correlated using common average reference (CAR) and then subjected to a band-pass filter with cut off frequencies 7–50 Hz with an infinite impulse response (IIR) filter. Zero-phase forward and reverse IIR filtering were implemented.

Canonical correlation analysis (CCA) was used for target identification which used the reference from the SSVEP training data ( ${\widehat{\mathrm{X}}}_{\mathrm{k}})$ to identify the user’s intention. The study developed an ensemble classifier that correlates the test (X) and training ( ${\widehat{\mathrm{X}}}_{\mathrm{k}})$ set signals with sine-cosine reference signals Y. A correlation vector ρ is defined as follows:

$$\mathit{\rho }=\left[\begin{array}{c}{\mathrm{\rho }}_1\\ \begin{array}{c}{\mathrm{\rho }}_2\\ {\mathrm{\rho }}_3\end{array}\\ {\mathrm{\rho }}_4\end{array}\right]$$=$$\left[\begin{array}{l}\mathrm{\rho }\\ \begin{array}{l}\mathrm{\rho }\left({\mathrm{X}}^{\mathrm{T}}{W}_{X\widehat{X}},{\widehat{X}}^T{W}_{X\widehat{X}}\right)\\ \mathrm{\rho }\left({\mathrm{X}}^{\mathrm{T}}{W}_{\mathit{XY}},{\widehat{X}}^T{W}_{\mathit{XY}}\right)\end{array}\\ \mathrm{\rho }\left({\mathrm{X}}^{\mathrm{T}}{W}_{\widehat{X}Y},{\widehat{X}}^T{W}_{\widehat{X}Y}\right)\end{array}\right]$$ E6

To validate the efficiency of the combined method, this study compared classification performance of the following five methods: (M1) a standard CCA-based method; (M2) a correlation analysis using a spatial filter derived from test set and training reference signals; (M3) a correlation analysis using a spatial filter derived from test set and since-cosine reference signals; (M4) a correlation analysis using a spatial filter derived from training reference signals and sine-cosine reference signals; and (M5) the combined method using the ensemble classifier described in Eq. (6).

Figure 10 shows the averaged accuracy (Figure 10(a)) and ITR (Figure 10(b)) across all subjects for the offline experiments. Results for different CCA-based methods were calculated with different data lengths from 1 to 4 s. It is evident that the four methods (M2, M3, M4, and M5) outperformed M1 under all conditions with different data lengths.

Figure 10.

(a) Target identification accuracy, (b) ITRs as functions of data length (from 1 to 4 s) calculated by different methods.

2.3. Information transfer rate (ITR)

The dynamic performance of any BCI can be evaluated by the information transfer rate (ITR) as introduced in [58]. This parameter depends upon three factors regarding the BCI

• Speed

• Accuracy

• Number of unique commands

ITR (B) is defined as

$$\mathrm{B}=\mathrm{V}\left[{\text{log}}_2\mathrm{N}+\mathrm{P}{\text{log}}_2\mathrm{P}+\left(1-\mathrm{P}\right){\text{log}}_2\left(\frac{1-\mathrm{P}}{\mathrm{N}-1}\right)\right]$$ E7

where, V = application speed in trials per second.

P = classifier accuracy, i.e., how accurately the thoughts are translated into command.

N = number of mental tasks used in the BCI application under consideration.

3. Conclusion

SSVEP proves to be the most widely used paradigm for BCI used for various different application for healthy as well as locked in patients due to the fact that SSVEP-BCI’s require minimum user training. This is because the subject does not have to regulate his/her own brain activity to provide controlling input for the task at hand. The subjects have to merely focus their attention towards the flickering targets which is then converted to machine command by a trained classifier.

The accuracy of SSVEP based BCI’s is fairly high for most subjects with substantial amount of visual capabilities. However some subjects were not able to produce a significant change in the EEG with respect to the SSVEP stimuli. This condition is termed as BCI illiteracy [59]. This phenomena cause the failure of BCI for such subjects as the task is not performed due to minimal EEG activity. To counter this problem a novel approach of hybrid brain computer interfacing (hBCI) was proposed [60, 61]. The hBCI combines a standard BCI paradigm (SSVEP, P300, slow cortical potential (SCP) or event related synchronisation/de-synchronisation (ERS/ERD)), with another BCI signal or some other physiological signal. hBCI’s are an emerging area of research where all possible combinations are being explored to increase system accuracy as well as eliminate the phenomena of BCI illiteracy. The hBCI’s also address the problem of subject fatigue due to fixing of gaze at flickering targets for a longer duration, this fatigue is known to reduce the accuracy of the BCI due increase in the number of False Positive (FP) outcomes.