Cognitive Load Measurement Based on EEG Signals

Abstract

Measurement of cognitive load should be advantageous in designing an intelligent navigation system for the visually impaired people (VIPs) when navigating unfamiliar indoor environments. Electroencephalogram (EEG) can offer neurophysiological indicators of perceptive process indicated by changes in brain rhythmic activity. To support the cognitive load measurement by means of EEG signals, the complexity of the tasks of the VIPs during navigating unfamiliar indoor environments is quantified considering diverse factors of well-established signal processing and machine learning methods. This chapter describes the measurement of cognitive load based on EEG signals analysis with its existing literatures, background, scopes, features, and machine learning techniques.

1. Introduction

To understand cognitive load, we must first understand the working memory and to understand working memory we have to know what memory is. The memory is the aptitude of brain that deals with encoding, storing and retrieving information as needed. These information are received and transmitted from external environment by sensory nervous system in the form of chemical or physical stimuli and processed by memory in central nervous system. Now memory is classified into two categories: (1) short term or working memory that holds information temporarily [1] and allows manipulation of stored information for reasoning, decision making, guiding behavior etc., (2) long term memory that can stock data for a long period.

Cognitive load is the quantity of working memory in use. There is another notion called “long term working memory” that is a set of rescue of parts of long term memory enabling continuous access to data required for daily activities [2]. Cognitive load is the basis of problem solving and learning [3].

Now comes the question, why do we need to measure cognitive load? Cognitive load is associated with learning new things whether be it a study matter or a new skill. So, to design a lesson plan it is important to understand how can a learner learn or memorize it easily and quickly where the measurement of cognitive load comes handy. The constructions and roles of human cognitive built have been used to develop a range of instructional means aiming for the reduction of load of working memory in learners and encouragement of diagram construction [4]. Furthermore, there are many other purposes of measuring cognitive load, such as, to know how a disease (neurodegenerative diseases, carcinomas etc.) or it’s treatment (chemotherapy, radiotherapy, immunotherapy etc.) affects human cognition. This measurement is also very important for different researches including age related cognitive declines, learning task performances and multiple document handling [5], Designing navigation aid for blind people [6] etc.

Researchers formulated many ways of measuring cognitive load such as subjective scales, task-invoked pupillary response [7, 8], EEG signals [9], fMRI etc. Distraction causes increase in cognitive load [10].

Subjective scales vary with varying perception in different individuals; thus, such scales are less reliable. Again, pupillary responses is equivalent to a range of events requiring psychological efforts which may be perceptual, cognitive and/or response related, thus it is not indicative of cognitive load being linked to task performance [11]. EEG and fMRI give more precise results regarding cognitive load related to task performance although they are expensive and complicated to operate. Compared to fMRI, EEG is easier to perform and read as various software are available now for EEG interpretation.

In this chapter Measurement of cognitive load based on EEG signals is discussed taking references from a study of learning processes of visually impaired people (VIP) while navigating through unfamiliar indoor environment using EEG signal [12].

2. Literature review

First, we review the prior research in the area of cognitive load measurement using EEG signals. Much work has been devoted in this area by extracting various features from EEG signals and then machine learning approaches have been applied to quantify cognitive load over years.

Fraser et al. [13] argued that intrinsic cognitive load needs to be adjusted to level of the apprentice, extraneous cognitive load needs to be abridged, and germane load needs to be augmented until the boundaries of working memory are not exceeded.

Gevins et al. [14] studied cortical activity throughout working memory tasks and found that a sluggish (low-frequency), parieto-central, alpha signal lessened as working memory load amplified.

Anderson et al. [15] designed user study performance based on measurement of cognitive load using EEG signals and these trials are used for quantitative evaluation the efficiency of visualizations.

Chandra et al. [16] extracted different features such as root mean square value, sub band energy, power spectral density, and engagement index and then used neural network for classification of the workload.

Kumar et al. [17] postulated that the cognitive load is possible to be quantified by measuring alpha and beta band events in the frontal, temporal and front-central regions of the cortex of cerebellum.

Antonenko et al. [9] computed Event-related desynchronization (ERD)/event-related synchronization (ERS) principles for individual partaker’s alpha and theta rhythms under respective experimental conditions. Results illustrated that attainment of abstract and physical knowledge was meaningfully better in the lead-facilitated hypertext state.

Fournier et al. [18] illustrated that alpha ERD is insensitive to various workload weights in multi-task situations, it is operative in measuring variances in processing difficulties in the solitary interactive mission condition. Moreover, theta ERS is insensitive to workload and exercising the interactive of multiple task condition.

Klimesch et al. [19] illustrated the volume of power of theta and alpha frequency range in EEG is certainly associated with cognitive performance and memory in specific, if a binary dissociation amid absolute and event-related fluctuations in alpha and theta wave is considered.

Ryu et al. [20] established technique based on several functional indices for evaluating the mental load all through arithmetic and tracking tasks. The suppression of alpha delivered appropriate data to deduce exertions for math task, but not in case of tracking task. On the other hand, blink intermission and heart rate variability allowed comprehensive readings about workload during tracking task, but not in case of arithmetic task.

Roy et al. [21] evaluated psychological exhaustion, ascending from mounting time-on-task (TOT), can pointedly affect the dispersal of the band power features. They exposed contradictory variations in alpha power spreading between Workload (WKL) and TOT settings, and reduction in WKL level discriminability by means of growing TOT in both figure of statistical alterations in band power and cataloguing act.

Krigolson et al. [22] demonstrated that bigger cognitive load lessens the functional efficiency of a recompence processing structure inside human medial–frontal cortex.

The performance of the quantification of cognitive load using EEG in real time seems to be more challenging due to diverse factors such as different environments, age, sex, social status, therefore, more research will be conducted.

3. Background study

In this section EEG system, event related synchronization/event related desynchronization (ERS/ERD), as well as different bands from EEG signals are discussed.

3.1 EEG system

The foremost International EEG assembly was held in 1947 when it recognized that standard electrodes appointment technique is required for the EEG recording [23]. This recognition resulted in the introduction of 10–20 electrode structure by H.H. Jasper in 1958. In this 10–20 system, 21 electrodes are placed utilizing the scalp size conferring to the external landmarks on cranium and their locations at distance of 10% and 20% of measurement of coronal, sagittal and circumference arcs from the nasion to inion. Electrodes are marked for identification according to their relative site on scalp. For instance, F, T, C, P and O represents frontal, temporal, central, parietal, and occipital lobes successively. Odd figures mean the electrodes of left side while even figures mean the right-sided ones, whereas the ground electrode is positioned at a unbiased site of the head such as midline of forehead. The two reference electrodes namely A1 and A2 are sited in dynamic zone (ear lobes) on left and right side respectively. The 10–10 system or International 10–20 system is an globally recognized technique for relating and localizing scalp electrodes in the context of EEG based experiments, which was established to guarantee standardized duplicability so that subjective researches as well as subjects could be compared to each other over time. This scheme is founded on the basis of association amid the position of an electrode and its underlying region of cerebral cortex. As for the designation of the scheme, the “10” and “20” actually refer to the real distances among adjacent electrodes which are either 10% or 20% of the entire front–back or right–left dimension of the skull. The 10–10 system, also acknowledged as the 10% system, is not only presented but also recommended by the standard of the American Electroencephalographic Society and the International Federation of Societies for Electroencephalography and Clinical Neurophysiology [23] where electrodes’ situation occur at interval of 10% of measured coronal, sagittal and circumference arcs amid the two points explicitly nasion and inion. The other stretched variety that is 10–5 system allows more than 300 electrodes to be placed [24]. Intracranial source of EEG signals is applied to facilitate the high-density EEG applications.

3.2 Event related synchronization/event related desynchronization

Cognitive load theory (CLT) refers to a hypothetical agenda which is founded upon human reasoning construction of long-term and short term (working memory) built [25]. CLT is a theory about information processing that relates working memory manacles to the effectiveness of instruction. Learning procedure is performed in the working memory which has restricted in both terms of capacity and duration as it can hold only upto 7 ± 2 portions of data at a specified time while novel data can be stored within about 15 to 30 seconds [26]. There are the three commonest categories of cognitive load, specifically intrinsic, extraneous and germane and these are defined by present CLT [27]. Intrinsic load is mainly imposed according to the intrinsic intricacy of the mission, while extraneous and germane loads are forced in accordance with the methods by which things are needed to be learnt. Event-related synchronization and desynchronization (ERSD) are measured to estimate the experimental participants’ cognitive load. Antonenko et al. [9] demonstrated that the alpha band power is augmented in event-related synchronization (ERS) and diminished in event-related desynchronization (ERD) of the task interlude by means of baseline intermission. Hence, cognitive index (CI) of ERSD is designed via the following equation:

$$\partial =\left(\frac{{\in }_b-{\in }_t}{{\in }_b}\right)\ast 100$$E1

where $\partial$ is cognitive index, ${\in }_b$ is the base-line intermission of band power, and ${\in }_t$ is the task internal of band power.

3.3 Different bands signals

In typical scenarios, the breadth of clinical EEG signals is from 10 to 100 μv as the frequency ranges from 1 to 100 Hz. EEG signal is catagorized into five broad rhythmic categories conferring to their frequency bands as clarified underneath:

Delta waves (δ): The frequency range is below 4 Hz with amplitude ranging from 20–200 μv. It arises at the time of deep sleep, infancy and grave organic brain disorders (He, 2013). It is documented from posterior brain in children and from front brain in grown-ups.

Theta waves (θ): The frequency ranges from 4 to 7 Hz as it is prominent in situations like sleep, psychological strain, and awakening in case of both children and grownups. It can be documented from both temporal and parietal regions of the scalp with an breadth ranging from 5–10 μv (He, 2013).

Alpha waves (α): It is periodic wave that is found in fit grownups during wakefulness, relaxation with eyes kept closed. Its frequency ranges from 8 to 13 Hz with usual voltage array of about 20–200 μv. It disappears during disease conditions like coma or normally in sleep.

Beta waves (β): Its frequency ranges from 13–30 Hz, although their breadths are lesser, ranging from 5–10 μv. It surfaces in situations like excessive excitement of the central nervous system when upsurge in alertness and watchfulness occur. It substitutes the alpha wave if cognitive damage ensues for any reason. It can be recorded placing electrodes on parietal and frontal regions of the scalp.

Gamma waves (γ): Frequency of gamma waves arrays from 30 to 100 Hz. It can be documented from the somatosensory cortex in instances of cross model sensory processing, short term memory processing to identify objects, sounds, palpable sensation and in some pathologies including cognitive deterioration, mainly when it relates to the θ band.

4. Cognitive load measurement

Measurement of cognitive load is typically consisting bands extraction, features extraction, calculation of ERS/ERD, and classification (see in Figure 1).

Figure 1.

Basic diagram for cognitive load measurement using EEG signals and machine learning approach.

4.1 EEG signals

Nine VIPs with different degree of vision lose partaken this experiment wandering through a complex route of the University building which included students’ units, class rooms, reading rooms, a book store and two restaurants [6]. The route was about 200 meters in length whereas the walking distance was roughly 5 minutes. Various environmental condition such as door, elevator, moving people and object, open and narrow space, as well as stairs were considered.

The study got its approval from the National Bioethics Committee of Iceland and data set was anonymized prior to analysis. The data set comprises of EEG recordings from nine healthy VIPs (6 female; average duration of visual damage = 30 yrs., range = 2–52 yrs) with different degrees of eyesight loss (see Table 1) as they strolled separately through a multifaceted course in an educational institute. The participants were asked to abstain from smoking, consuming coffee and sugar roughly 1 hour before the initiation of the experiment. The partakers were instructed to walk through the charted route thrice (i.e. trial 1, trial 2, and trial 3) as drills. Directions were provided for the first time only (i.e. trial 1) to aid the VIPs acquaint with the course. They were also prohibited from needless head activities and hand movements. They also asked to avoid using their O&M instructor except in case of emergencies.

Brain Location	Sensitivity	Specificity	Accuracy
Entire Brain	58.37	76.23	73.08
Left hemisphere	56.44	74.32	72.31
Right hemisphere	54.72	73.64	71.45
Left hemisphere and frontal lobe	56.98	72.61	71.64
Right hemisphere and frontal lobe	55.46	72.25	71.45
Left hemisphere and theta and alpha bands	70.11	71.35	70.65
Right hemisphere and theta and alpha bands	56.15	72.20	71.29

Table 1.

Calculate performance using signal processing feature and machine learning approach for cognitive load measurement in the indoor environment based different band from EEG signals.

EEG data was attained by the Emotiv EPOC+ system which is a portable headset containing 16 dry electrodes with 128 Hz sampling rate registering capacity over the 10–20 system locations specifically AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4, GYROX and GYROY (i.e. information about how the head accelerates during leaning sessions according to x-axis and y-axis, respectively). This system encompasses a sum of interior signal training phases. Analogue signals are at first filtered by high-pass filter with a 0.16 Hz cut-off, pre-amplified, then low-pass filtered with a 83 Hz cut-off, and tried at 2048 Hz. Digital signals are subsequently notch-filtered at 50/60 Hz and down-sampled to 128 Hz before broadcast. The EEG signal of each participant are firstly smothered by his own base signal and then bandpass butter worth filter is used on the signal for mining of gamma (30–60 Hz), beta (13–30 Hz), alpha (7–13 Hz), theta (4–7 Hz), and delta (0.5–4 Hz) bands respectively.

4.2 Bands extraction

Different bands like gamma, beta, alpha, theta, and delta are extracted to get appropriate features. The power of alpha band is grown in event-related activation and declined in event-related deactivation considering task interval with baseline interval [7].

4.3 Features extraction

Features extract play an important role for detection of cognitive load based on EEG signals. Different feature extraction techniques such as time, frequency, and time-frequency domains are considered for features extraction [28]. To analysis of the cognitive load, we have extracted entropy feature based on frequency domain analysis.

4.4 Labelling of cognitive load

Though alpha band (i.e., 7–13 Hz) power rises in event-related synchronization (ERS) and lessens in event-related desynchronization (ERD) of the task intermission by means of reference line interval. So, alpha band power is used for the labelling of cognitive load based on measurement of cognitive index.

4.5 Classification

In this section, we discuss classification and cognitive load measurement. Classifier intent to categorize the EEG signals by applying machine learning method. Support vector machine (SVM) is implemented on spatio-temporal feature namely entropy that we extracted using frequency domain analysis. Classification processed revolving cross validation which execute its technique involving the test set and training set of data into complimentary subset and analyze how accurately the predicted model will perform [29]. In our case, we used five-fold cross validation (i.e., N = 5). One set is randomly chosen and reserved for testing and remaining N-1 is used for training and then average the results. Five-fold cross validation is operated upon training set to produce an ideal model of the SVM classifier. N-1 which means 80% of the training set is randomly selected to establish the SVM model and remaining 20% is observed to fit the model.

5. Results and discussion

Entropy features extracted from EEG signals using 16 electrodes and different bands. There performance metrics such as sensitivity, specificity, and accuracy are considered to evaluate and measure the performance [30]. The formulation of sensitivity, specificity, and accuracy are as follows:

$$\mathit{Psen}=\frac{{\tau }_p}{{\tau }_p+{\phi }_n}$$E2

$$\mathit{Pspe}=\frac{{\tau }_n}{{\tau }_n+{\phi }_p}$$E3

$$\mathit{Pacc}=\frac{{\tau }_p+{\tau }_n}{{\tau }_p+{\tau }_n+{\phi }_p+{\phi }_n}$$E4

where ${\tau }_p$ is the true positive, ${\tau }_n$ is the true negative, ${\phi }_p$ is the false positive, and ${\phi }_n$ is the false negative.

To observe the measurement of cognitive load, the performance metrices are considered. In the Table 1, the results demonstrated that accuracy is high when considering entire brain compared to consider partial brain locations.

6. Conclusion

In this study the chief aim of the measurement of cognitive load with emphasis on working memory load was to design a smart motion system for the visually impaired people (VIPs) for their navigation. To achieve this goal an experiment consisting of different phases such as bands extraction, features extraction, labelling of cognitive index using well-established metric, and finally use of the machine learning techniques was conducted to measure cognitive load. The performance of the prediction and/or detection of cognitive load using EEG signals in real time is more challenging, therefore, further research should be conducted for improved performance.