Brain Computer Interfaces for Cerebral Palsy

1. Introduction

Cerebral Palsy (CP) is a group of disorders that affect movement and posture, causing activity limitation to the person who suffers from it. It is caused by a lesion that occurred in the developing brain, usually before birth but also during or after. Cerebral palsy manifests itself early in life, during infancy or preschool years with delayed or aberrant motor progress and it is non-progressive, which means that at the time of the diagnosis, the disturbance that incited the cephalic lesion is no longer active. At the moment there is no cure for cerebral palsy (Bax 2005).

Cerebral Palsy is a condition which affects approximately 2 out of every 1000 newborns. The total number of children with cerebral palsy has remained stable since 1970, but at the same time there has been a consistent rise in the risk of cerebral palsy associated with preterm infants (Thornhill 2009). Since it was first reported by Little in 1861, it has been widely documented and it has attracted research interest.

According to the World Health Organization (WHO), more than one billion people of the world’s population lives with a disability and this number is rising as the population grows, the increase of chronic health conditions, and the life expectancy becomes higher (World report on disability, 2011).

There are 3 possibilities to restore function:

• Using remaining muscular pathways as substitute for paralyzed muscles. The use of eye or hand movement can give control over communication devices.

• Using EMG, above the level of lesion, as control for paralyzed muscles.

• Provide the brain with a new form of communication and control that uses no muscular paths.

Despite the motor limitations of the physically disabled, most of the time the brain activity remains intact. Brain-Computer Interfaces (BCI) are communication devices that translate signals from the brain or nervous system (e.g. Electroencephalogram (EEG)) into electrical signals For the control of devices, allowing people to regain some form of control and regain interaction with the environment (Rao 2010, Wolpaw 2002). Even though, BCIs where initially developed as assistive devices for people with severe neuromuscular disorders, such as brain or spinal cord injury, cerebral palsy, muscular dystrophies, multiple sclerosis, and numerous other conditions; the increasing interest in non-medical application looking for an improved technology for Human-Computer Interaction (HCI) such as exoskeletons, robot or wheelchair control, or augmented reality (Lotte 2013), has generated clinical, scientific and commercial interest in the use of BCI’s for an augmentative communication and control technology.

2. Examples of success

Despite, that BCIs have shown their possibility as communication and control device through spelling devices (Donchin 2000), used as control of prosthesis (Tenore 2008), web browsing (Mugler 2010), for control in a virtual reality environment (Lotte 2009) and for entertainment (Rao 2010), there are still more possible applications and room for improvement, using combined technology (e.g. Hybrid BCIs), improving or creating new classification algorithms, and better recording technology. In this chapter we give a brief description of the recording technology, pattern selection, current classification algorithms used for BCI and the state of the art as well as future technology.

Figure 1.

Basic components of a BCI. The image illustrates the map between the input and output through the translating algorithm. Signals are acquired by electrodes and then translated into a control signal for an external device (e.g. wheelchair, neuroprosthesis or exoskeleton) using a sequence of processing steps.

3. Evaluation and classification

3.1. Neuromotor examination of neonates and infants

The diagnosis of CP is made largely through clinical observations. The natal history is of vital importance for the identification of reasons for concern and the determination of the cases which merit closer monitoring. Failure to meet gross motor milestones is often the initial concern of parents. Significantly delayed motor milestones, persistence of primitive reflexes, and abnormal postural reactions are additional reasons for concern and referral to a neurologist or expert in neurodevelopment for evaluation. Clusters of symptoms or evolving abnormal movement patterns may be indications of CP and thus should be explored further with diagnostic instruments.

Instruments like the Hammersmith Neurological Examination (Dubowitz 1999), the Amiel Tison Neurological Assessment (Tison 2002) or the INFANIB (Infant Neurological International Battery for the Assessment of Neurological Integrity in Infancy) (Ellison 1994) have proven extremely valuable in the earlier identification of the difficulties that at-risk neonates and, as a result, a better targeted, early intervention.

These instruments offer a neurological or neuromotor exploration of the neonate and the infant, assessing the existence of primitive reflexes, the automatic system or any other involuntary movements, that appear in normal infants and should be integrated by the 9th month of life. Their persistence past that age is a reflection of abnormalities in terms of control in the central nervous system and may indicate cerebral palsy. The persistence of primitive reflexes causes changes in muscle tone and the position of limbs, which makes it interfere with the development of voluntary motor movements by causing changes in muscle tone and the position of the limbs. Failure to develop protective reflexes such as the parachute response or an asymmetrical response is also taken into consideration.

The instruments also take into consideration the age when the infant met the motor milestones (head control, sitting, voluntary grasp, ability to kick, rolling, crawling, standing and walking) and may include some items of attention, sensory function or self regulation, as well as muscle tone and posture are considered.

4. Nature and typology of the motor disorder

Cerebral Palsy is a symptom complex with various types and degrees of motor impairment. Depending on the area of the brain that has been affected, according to the SCPE (Elison P 2004, SCPE working group 2002) we may identify the predominant motor characteristics of the condition as of three types: Spastic, dyskinetic and ataxic.

Spastic CP results from defects or damage occurring in the brain's corticospinal pathways, also described as upper motor neuron damage. Spastic CP accounts for almost 84% of all cases of CP, with cognitive impairments seen in approximately 30% of the cases with CP (SCPE working group 2000, Palisano 1997). Although increased , as well as muscle tone is the predominant feature observed, hyperreflexia, clonus, extensor Babinski response, and persistent primitive reflexes are commonly seen.

Dyskinetic and ataxic CP are caused by damage to nerve cells outside of the pyramidal tracts in the basal ganglia or the cerebellum. Dyskinetic CP is then further divided into athetoid and dystonic. It accounts for 15% to 20% of all cases of CP, with dyskinetic accounting for 10% to 15% and ataxic approximately 5%. The resulting disability is global with abnormal tone regulation, postural control, and coordination (SCPE Working Group, Palisano 1997)

It is actually quite common to see many different combinations of types of CP, since this depends on the area of brain damage; the types overlap very frequently, which can make it very difficult to precisely label the resulting disability within the typical subtypes. As a result, when not one type dominates we make reference to a “mixed” category.

5. Functional motor abilities

What is of particular interest for the parents of the children affected is evaluating the functional consequences of the condition. The Gross Motor Function Classification System (GMFCS) (Eliasson 2006) was developed as an evaluation tool in order to offer a prognosis or to assess differences in motor functions after an intervention. It recognizes motor function as dependent on age due to the expected change of the developing child. It separates clusters of periods (0-2, 2-4, 4-6 and 6-12 years of age).

Degree of functionality	Gross Motor Function Classification System (GMFCS)	Manual Ability Classification System (MACS)
Level I	Child´s ability to walk is not affected	Child handles objects easily and successfully
Level II	Child´s ability to walk is slightly affected	Child handles objects with somewhat reduced quality
Level III	Child walks with assistive device	Child handles objects with difficulty
Level IV	Limited self-mobility with assistive device	Child handles only a few, adapted objects
Level V	No self-mobility	Child cannot handle objects

Table 1.

Classification of gross motor function and manual ability in children with cerebral palsy.

In this chapter we have cited the classification of a child´s gross motor function between 6 and 12 years of age, which is divided into five levels, based on functional mobility or activity limitation. Particular emphasis is made on the function of sitting and walking. Children in level I are the most independent (motor function) and children in Level V are the least according to the Gross Motor Function Classification System for Cerebral Palsy.

The Manual Ability Classification System for Children with Cerebral Palsy (MACS) (Bottcher 2010) is widely used to evaluate and classify how children with cerebral palsy use their hands to handle objects in daily activities. Like the Gross Motor Function Classification System, MACS describes five levels. The levels are based on the children’s self-initiated ability to handle objects and their need for assistance or adaptation to perform manual activities in everyday life.

6. Accompanying impairments

A child with cerebral palsy often has other conditions related to developmental brain abnormalities, such as intellectual disabilities. Almost 50% of children with CP have an average intelligence, 20% have an intelligence slightly lower than average (borderline intelligence). The rest 30% its not mentioned if its more inteligent or not than average. Most patients that have spastic tetraparetic, discinetic and ataxic have a severe mental discapacity (SCPE Working Group 2002)

There have been studies that prove that children with CP with average intelligence have attentional deficits or problems with the executive functions, which may partially account for the behavioral problems that sometimes present. (Guzzetta 2001) They might have deficits in visioperceptive functioning. The child has difficulties recognizing the spatial relations between objects, as well as between objects and his own body. This results frequently in a constructive dyspraxia. The saccadic movement of the eye to focus on an object that appears periferically at the previous point of focus are slow and dyspraxic, which constitutes an added difficulty in order to achieve the perceptive integration. The proprioceptive-visual integration of the parietal lobe is necessary in order to orient the movements and postures of the upper limbs to reach for and manipulate the surrounding objects and starting the proceeding automatic movement that experience and repetition offers. These deficits are completely independent from the vision problems that may coexist (Guzzetta 2001).

Language problems are also common and their severity depends on the timing that the lesion took place, in the prelinguistic period or later, when the linguistic function has already started to form.

7. Implications for everyday life

Although there are many compelling reasons to give the diagnosis as early as possible (parents frustration of handling a child with abnormal tone such as feeding, sleep, and temperament problems, plan in advance for long-term treatments and management options that may be needed by the child, possible increased insurance benefits and in some cases federal assistance, benefits that come from an early intervention) the diagnosis should not be formally made until the second year of age. For the SCPE in Europe minimum age of 4 years old is required to make a diagnosis so that transitory alterations of neurodevelopment or degenerative diseases may not be confused with CP (SCPE Working Group 2002).

The diagnosis has an impact on the life of the family and, of course, the child. The major issue of concern is, for most parents, walking. Once confronted with the diagnosis, the first question that parents ask their child´s health care specialist if the child is going to walk. Children will CP will experience some degree of difficulty with movement. This can range from problems like clumsiness that does not disrupt everyday life activities all the way to difficulties with walking. The child may move slowly, may need to use a walking aid or a wheelchair.

Simple activities like dressing, bathing, eating can be a real challenge to the child with CP and their family. The activities can take longer, especially if the child needs more assistance, physical help or specialized equipment.

Language problems are common among children with CP. Children may have difficulties with both verbal and non verbal aspects of language. The expression and understanding of the formal aspects of language can be affected (for example articulation or denomination) which may eventually lead to problems with reading and writing or even interfere with the child´s ability to communicate verbally. The other aspect of language that can be impaired is pragmatics, which refers to the ability to place words in the context of one´s own mind and the interlocutor´s, which creates problems in the child´s social adjustment.

8. Attentional processes

The aim is for every child with CP to achieve their potential. Depending on the child´s individual characteristics decisions must be taken that will determine whether he would benefit more from mainstream placement in a school or from a placement in a more specialized environment that could tend to his needs.

In order for the interface to be able to read the brain signal, the child needs to be focused. Not always is it possible for all children to emit a signal strong enough so that it can be captured by the interface. The emission of a strong signal depends on the attention of the child which can be negatively affected by a variety of factors which have no relation with the interface but which affect its ability to read the brain signals.

The attention of the child can be hindered by three main factors which are at constant interplay and affect the prefrontal cortex and the ability of the child to focus on a particular task. The three main factors are:

1. Cognitive

2. Emotional

3. Behavioral

Although the cognitive function of children with CP has not been systematically studied, and more research is needed, there is evidence suggesting that children with CP and normal intelligence present impairments in executive functions. Executive functions are the brain functions that regulate and control impulse, anticipate consequences, put attention, regulate emotion, allow flexibility, plan and monitor results. Executive functions are highly fragile because they are the last cognitive area to mature. They involve the prefrontal cortex and they rely on an extensive interconnectivity with other parts of the brain. Damage to that area results in slower information processing, and a decrementation in sustained attention performance, which is necessary for the reading of the signal by the interface (Guzzetta 2001).

In the case of an intellectual disability, which as we have seen affects almost half of the children with CP, we cannot speak of attention problems. The degree of cognitive impairment is such that the attention processes cannot reach the required level so that it can be captured by the interface.

The attentional processes are also going to be affected by the emotional problems that the child may be experiencing (Parkes 2008). This is also an issue that has not been researched but there is enough evidence to suggest that children with CP, like children with some sort of a disability in general, are more likely to suffer from depression, anxiety and low self esteem. This is associated with the severity and visibility of the condition, which affects the child´s ability to control his own body and the way his peers may perceive him as being different from them. (McDermott 1996) The lack of social support, the anxiety of the parents, the child´s inability to use words to express his emotions are all factors that put the child at increased risk to experience emotional problems. Emotional problems hinder the ability of a child to focus and pay enough attention so as to send a strong signal to the interface.

Bottom of Form

Children with CP have behavioral problems like being defiant and disobedient. The behavior problems reported by parents were 5 times more likely in children with cerebral palsy compared with children having no known health problems. Behavioral problem are associated with some kind of combination of the impairment, the environment and interpersonal relationships. Damage to the prefrontal cortex affects, as we have seen, cognitive flexibility, the abilities for strategic planning, tolerance to frustration, behavioral inhibition (hyperactivity-impulsivity) as well as the associated impairment of inattention. The child that has trouble maintaining his attention on the signal is more likely to refuse to try or abandon the task.

It is important to mention epilepsy as one of the factors that cause behavior and attention difficulties in children with cerebral palsy.

Epilepsy affects 7 to 50% children with cerebral palsy. Epilepsy, in itself, takes away part of the vitality of the brain, with the frequent crises affecting the cognitive abilities of the child. Behavior and attention difficulties are highly common in children with cerebral palsy who have epilepsy. Furthermore, the crises are frequently a motive for the child to stop receiving education. Medical treatment for epilepsy can be helpful, keeping in mind that although antiepileptic drugs may impair the cognitive functions of the child, with the careful monitoring of the physician and the new medical intervention, this side effect would be very infrequent.

9. Communication system

There are some limits that can be solved using the brain activity. This activity allows the communication between the processor and the person.

9.1. Brain signals

Through the recording and processing of direct brain electrical activity via signal processing and machine learning algorithms, BCIs enables communication and control to assistive devices. Although the aim of a BCI is to identify and translate brain electrical signals into commands, it is not a thought-reading device or systems able to literally translate arbitrary cognitive activities. BCIs are design for translation of well characterized a priori defined brain activity patterns through the use of machine learning techniques and patterns recognition methods into commands.

Considered as a control system, a BCI has an input (e.g. EEG), an output (e.g. control signal), and components that translate input into output, a protocol that determines the timing operation and in some cases some feedback is provided to the user (Figure 1.).

Figure 2.

Exemplification on EEG (a), ECoG (b) and Single-neuron recording (c) electrode placement over the head

9.2. Functional neuroimaging

Invasively or noninvasively brain activity is recorded either from recording electrical activity through electrodes (EEG, Electrocorticography (ECoG) or from single-neuron recordings within the brain), recording magnetic fields using magnetoencephalography (MEG)), or recording metabolic activity reflected in changes in blood flow (positron emission tomography (PET), functional magnetic resonance imaging (fMRI) and functional Near Infrared (fNIR)). Despite the fact that MEG, PET, fMRI and fNIR have shown success for BCI applications these techniques are still technically demanding and expensive technologies that require sophisticated equipment that can be operated only in special facilities. Furthermore, PET, fMRI and fNIR techniques depend on metabolic processes, such as blood flow, having long latencies and thus less suitable for the control of BCIs.

On the other hand, the non-invasive EEG and the invasive ECoG and single neuron recordings (Figure 2.), are methods that have relative low costs, are simpler to use and have higher temporal resolutions, making them more practical to the use with BCIs.

Invasive techniques such single-neuron recording and ECoG take recordings over the cortex; while single-neuron recording records the activity within the cortex, ECoG records the activity over the cortical surface of the brain. Single-neuron recordings and ECoG does not record single neuron activity but records activities over small regions of the brain giving them a high spatial resolution, and as it is implanted directly over the cortex, they have a high bandwidth, high SNR and high amplitude. Since ECoG electrodes do not penetrate the cortex, recorded signals are also not subjected as heavily to immune response, possess lower risk to implant as well. Furthermore, maintaining long term reliable recording with implantable electrodes is difficult.

Although, ECoG has a higher spatial resolution compared to EEG (i.e. 1.25 - 1.4mm vs centimeters) higher frequency bandwidth ([19, 11] (i.e. 0−500Hz vs. 0−40Hz), have higher signal amplitude (50 − 100µV maximum vs. 10 − 20µV maximum), and being less susceptible to artifacts (i.e. EMG, EOG or electrical devices), EEG has become the most common source for brain activity due to its none invasiveness (requiring no craniotomy (surgical incision of the skull)), being more practical for everyday situations. EEG measures the potential over the scalp, reflecting the collective activity over large population of neurons located underneath the sensor position.

9.3. Recording and processing

Brain signal recordings, like EEG or ECoG, are obtained with electrodes attach from the surface of the skull or to the surface of the brain measuring difference over the potential that reflect the activity within the brain. The electrodes are connected to biosignal amplifier where they are amplified and go through an analog-digital conversion. These signals are sent to the signal processing system that is in charge to perform the feature extraction and classification. Finally, a signal will be send to the control system as final output. The BCI can be design to present feedback that is beneficial to learn the BCI control faster.

The electrodes measure a difference in potential (i.e. the voltage) between two electrodes. The difference in potential reflects neural activity below the electrode. There are different EEG electrode montages. Usual EEG recordings use unipolar montage rather than bipolar electrodes, meaning that they use a common reference for all electrodes. A ground is added to keep the voltage levels close to the amplifier ground voltage level. The reference and ground can be positioned everywhere within the array of electrodes, but they are normally placed either over the ear or the mastoids (the temporal bone behind the ear). There also exist a bipolar and Laplacian montage that each electrode represents the difference between the electrode and its surrounding electrodes (see Figure 3).

EEG recordings electrodes usually use small metal plates made out of gold or Ag/AgCl. Alternative materials such as Tin have been used, but they present drifting noise below 1HZ, making them unsuitable for some applications, such as Slow Cortical Potentials. The electrodes could be either passive or active (i.e. pre-amplified with gain 1-10) disks that are connected through a cable to the biosignal amplifier. Active electrodes are less susceptible to environmental noise, and can work with higher skin impedance than passive electrodes. There exist also dry and wet electrodes. As the dry electrodes normally use an array of pins to go through the hair have contact with the skin, the wet electrodes use a gel that reduces the impedance and make a better connection to the skin. Even though dry electrodes have the advantage that require less preparation and cleaning time (not requiring conductive gel) and They are proven to be an alternative for EEG recordings (Zander 2002), More in-depth research is necessary for their successful dailybased application. daily-base application.

Figure 3.

Examples of non-invasive BCI, with visual stimuli and virtual control of ball movement

9.3.1. Electrode distribution

The standard EEG electrodes naming and position on the scalp are according to the international 10-20 electrode system (Jasper 1958). The system ensures that different laboratories share the same names over electrode positioning. It is based on arcs dividing the scalp in an array, using the Nasion and Inion as longitudinal reference points (i.e. front and back respectively) and the left and right Pre-Auricular points as lateral reference points (Figure 4a.). The intersection between the longitudinal line and the lateral is called the Vertex, and at this point it is located over the center. The 10-20 system identify each point using each lobe (Frontal-F, Temporal-T, Central-C, Parietal-P and Occipital-O) and each hemisphere (Left-Odd, Right-Even numbers and z or zero over the midline) as a marking. An extension to this configuration is using 70 electrodes (Figure 4b.), subdividing in between the 10-20 arcs (using the combinations of the letters for reference). In addition, the letters A and Fp are used to identify the earlobes and frontal polar sites respectively

There is also the AF marking the subdivision in between the Frontal and Frontal polar site

. The electrode can either be placed directly over the scalp, Which requires practice and is time consuming. A second technique is using caps, Which already have the marking of the electrodes and their position, some even have the electrodes pre-mounted making them easier to work with. These caps come in different sizes and can be adjusted to different persons.

Currently electrode caps are mainly intended to be used in a laboratory environment, therefore being expensive, the electrode and cap placement is difficult and need special preparation, making them not practical for an everyday use. Current commercial caps, such as the Emotive Epoc, they present an alternative for an everyday use being more economically accessible, they have the drawback that they use wet electrodes that dry quickly, and they are difficult to position and remain stable, which is problematic as it gives a high variance over recordings, as well as the same recording, and they are not possible to use in a different configuration. For the remaining of this section we will only focus on laboratory caps and EEG recording, as these are the most commonly used for BCI.

Figure 4.

EEG Electrode Montage (Jasper 1958)

9.3.2. Artifacts

Since EEG is further away from the neurons it has low spatial resolution and very noisy overview of ongoing brain activity. There are mainly two sources of noise while performing EEG recordings: Environment and Physiological artifacts.

Environment artifacts: Electrical power lines and/or surrounding electrical equipment become a problem while recording EEG. Their frequencies can overlap with the EEG feature the classification algorithm is working on. For laboratory settings these artifacts are normally solved by using some isolation from environmental signals, avoiding the interference with EEG recordings. A notch pass filter over the frequency over the power line (50 or 60 Hz for America or Europe recordings respectively) may also be applied suppressing signals in a narrow band.

Physiological artifacts: Although, muscle activation, eye movement or eye blinking can serve as communication signals for HCI, they can mask EEG frequencies and mislead researchers by mimicking EEG-based control and/or hide EEG features. Some EEG recording incorporate these signals as either control signals, or they used to filter the desired signal. A different way of control is to remove the EEG recordings that have been contaminated by artifacts, leaving only the trials that were not contaminated for the training of the classifier.

9.4. BCI signal processing

To design a BCI, we need to decide on the type of signal, the location, the desired feature and the appropriate classification technique.. In this section a description of the different types of signals, the different types of feature extraction that has been used, and finally a brief description of the different machine learning algorithms available is presented.

9.5. Evoked potentials and oscillatory activity patterns

The two major types of EEG signals used in BCI are Evoked Potentials (EPs) and changes in the spontaneous oscillatory EEG activity, also known as event-related desynchronization (ERD), and event-related synchronization (ERS) (Pfurtscheller 1999(2)).

EPs are electrical potential shifts that are time-locked to perceptual events, such as a rare visual or audio stimulus. Time-locked implying here that the time between the event and the time potential shift is approximately constant. Due to its low Signal-to-Noise Ratio (SNR) are typically analyzed by averaging EEG data over time beginning of the perceptual event for duration over 1s. There are different types of EPs based on the source of stimulus (e.g. visual, auditory or tactile)

On the other hand, oscillatory activity can be voluntary induced by the user (e.g. imagination of kinesthetic body movement, aka motor imagery, Neuper 2005). Such imagery usually generates a decrease or increase in power in a particular frequency band (ERD or ERS (ERDS) respectively). ERS are normally associated with an ERD appearing either after the termination of the movement or simultaneously to the ERD, but in other areas of the cortex. Although, Oscillatory patterns detection is less robust and reliable compare to EP, which as synchronous signal (i.e. knowing its time and shape) requires little adaptation and its detection is robust, as an asynchronous BCI it allows the user to send information at their own pace, unlike synchronous BCIs that require to follow the cues or prompts from the system.

Figure 5 illustrates the use of EP and ERDS for achieving brain-computer interaction in physical and virtual environment.

Figure 5.

Examples of evoke potentials with either lights or screen squares flickering at different frequencies

9.6. BCI control commands

Within these two ways of brain signal extraction there are four main strategies to consider for input at a BCI system. extraction there exist mainly 4 common strategies are considered for input of a BCI system, a) Motor imaginary. b) Slow Cortical Potentials, c) the P300 wave of visual evoke potentials, and d) Steady State Visual Evoked Potential (SSVEP)

9.6.1. Motor imaginary

It is believed that the mechanisms of brain operations are characterized by groups of neurons synchronizing themselves to a certain physiological frequency (Engel 2001). These oscillations has been divided into different frequency bands and are referred as brain rhythms (Delta [0.1 − 4Hz], Theta [4 − 7Hz], Alpha [8 − 12Hz], Mu [8 − 13Hz]

Although the Alpha and Mu rhythm occur over the same frequency, one is located over the resting visual cortex at the back of the scalp, while the second is found over the motor cortex

, Beta [12 − 30Hz], Gamma wave [30 − 100Hz] Figure 5a.). Movements are normally accompanied by changes of the Mu and Beta rhythm over the motor/sensory cortical areas (see Figure 5b.). For example movements of the hand are associated with decrease of power (or desynchronization) over the Mu rhythms and associated with a decrease over the Beta rhythm, particularly contralateral to the movement. The same effect occur with motor imagery (Neuper 2005), making the Mu/Beta rhythm as base for a BCI. The most common approach used for classification is to calculate the bandpower in a specific frequency band and then use discrimination via some machine learning technique (e.g. Fisher linear discriminant analysis).

9.6.2. Slow cortical potentials

Slow cortical potentials (SCPs) are slow voltage changes generated over the cortex. These changes in potential occur over 0.5−10s. These potentials can be divided in Negative SCPs, typically associated with movement and other typical cortical activation, and Positive SCPs that is associated with a reduce cortical activation, the viability of the use of SCPs that after a period of learning user has gain control selecting words or pictograms from a computerized language (Birbaumer 2000) or used with patients Suffering from a locked-in condition such as amyotrophic lateral sclerosis (ALS) (Kübler 1999). The drawback of using SCPs is that It requires a long training process that allows the user to gain control, a normal training can go for several weeks or even months. The normal training for a SCP based BCI users first learn to move a cursor vertically on a monitor selecting targets at the top or the bottom of the screen. Next, a split keyboard into two where an area is selected, the selected characters are once more split into two and once more selected, and this is done until the final choice is made.

Figure 6.

Example of different types of normal EEG rhythms (Lotte 2009) and Primary Motor and somatosensory cortical homunculus

9.6.3. The P300 wave of visual evoke potentials

The P300 (P3) wave of visual evoke potentials (henceforth referred only as P300) is a positive wave that appears 300ms after a stimulus is presented (Figure 7a.). It was first described by Sutton (Sutton 1965). The most known paradigm for P300 is the one described by Farwell-Donchin 1988, where characters and numbers are represented in a six by six grid (Figure 7b.) where the rows and columns are repeatedly flashed and when the character containing the chosen character is flashed a P300 is evoked. These characters (number or letters) can be replaced with different symbols and used not only as spelling device, but for navigation or for control tasks, using symbols as arrows or object selection. P300 can be used as a lie detector, providing certain stimuli (e.g. picture, phrase or word related to the lie) that P300 is generated if the subject has knowledge of the stimuli presented (Farwell 2001, Farwell 2012).

As a normal procedure, the P300 of different repetitions are averaged, to reduce the effect of artifacts, or the presence of different mental activity that could masked the ongoing P300. Different features and classification techniques (e.g. Linear Discriminant Analysis or Supported vector machines, Krusienski 2006) has been used for P300 based systems, these techniques will be described in a following section.

Figure 7.

P300 wave and the classical P300 spelling paradigm described by Farwell-Donchin 1988. Figure 6a show a change of potential occur approximately 300ms after the stimulus is presented (Picture adapted from Scherer 2013). In Figure 6b shows the classical spelling paradigm, where a P300 potential is generated if either the row or column flashes over the letter desired.

9.6.4. Steady State Visual Evoked Potential

Steady State Visual Evoked Potential (SSVEP) are brain responses to visual stimulus (e.g. flickering LEDs or phase-reversing checkerboards), flashing at constant frequencies between approximately 6 − 100Hz. SSVEP is a frequency-locked signal that manifests itself as an increase of the EEG amplitude of the stimulated frequency over the occipital lobe. Classification of SSVEP is done either using FFT-spectrum analysis or by the use of canonical correlation analysis (CCA) or finally by using of the minimal energy approach.

SSVEP have shown to be independent to eye movement, making them a good alternative for people with well preserve eye acuity but are incapable to moving their eyes (Brendan 2008). Some drawbacks while using SSVEP is that if a computer screen is used only frequencies that entire division over its base refresh rate (e.g. a 60Hz screen only 30, 20, 15, 12, 10 or lower are possible). A second drawback is that SSVEP are usually developed with a short number of flickering channels trying to avoid distraction and hence lower performance.

9.7. Feature extraction

Even though the amount of electrodes, the number of tasks performed, the high sample frequency required, the classes and the different patterns make that the amount of data recorded large, the normal training data set is short. Identifying, Selecting and extracting the relevant properties or features of the signals that better describe the EEG signals are essential steps in the design of a BCI. The correct selection of the features is crucial, if the features extracted from EEG are not relevant and do not accurately describe the EEG signals employed, the classification algorithm will have trouble selecting the class or label the user intended. The feature extraction could be divided in two main groups: temporal and frequential methods, a third group can be added as hybrid between temporal and frequency techniques.

9.7.1. Temporal methods

Features that present a time dependent variation can be treated using a temporal method. The changes can be as the ones that occur on P300 wave, which depend of the flashing of the selected command to 300ms later to be generated. The main temporal methods are the parametric models (e.g. AR or AAR) modeled the signal using a weighted sum of values, the Hjorth parameters that describe the dynamics of the signal by the use of three measures (activity, mobility and complexity) and finally the signal amplitude method that concatenates the electrodes amplitude into a feature vector that is used as input into the classification algorithm.

9.7.2. Frequency methods

The different oscillations or rhythms that characterize the EEG signals present variations while performing a mental task (e.g. motor imagery) or with a steady state evoked potential that a change in the oscillation is highly related to the stimulus frequency. Frequency methods are commonly used for the ease of application and computational speed. The most commonly used methods are power spectral densities and band powers. The third method uses a feature that can be located both in time and frequency domain. This method uses the Short Time Fourier Transform or the Wavelet transform to have a time-frequency representation of the signal.

9.8. Classification techniques

After the features have been selected the next step is to translate them into a command. This translation can use regression/classification methods. There are different classification methods and they can be divided using their classifier properties into: Linear classifiers, neural networks, non-linear Bayesian classifiers, nearest neighbor classifiers and combinations of classifiers (Lotte 2007).

9.8.1. Linear classifiers

Linear classifiers are discriminant algorithms that use linear functions to separate between classes. The most common used for BCI are Linear Discriminant Analysis (also known as Fisher’s LDA) and supported vector machines. These two methods separate the data using hyperplanes, for two-classes they are divided depending on the side of the hyperplane they are located (see Figure 8.). For LDA and SVM the popular method to solve a multiclass situation (N-number of classes) is selecting a class and separating it from the rest, this technique is referred “One Versus the Rest” (Schlögl 2005). This technique is very computational efficient and suitable for online classification. One drawback of LDA is when it deals with complex nonlinear EEG data (Garcia 2003). Even though SVM is originally linearly, it can be expanded using the “kernel trick”. The trick consists of mapping the data into another space, using a kernel function. For BCI usually the Gaussian or radial basis function K(x,y)=exp[−||x−y||/ (2σ 2)]. This trick gives a better generalization, but has lower speed execution (Lotte 2007).

Figure 8.

LDA and SVM hyperplanes that separate between two classes (circles and crosses).

LDA has been used for motor imagery (Pfurtscheller 1999), for a multiclass asynchronous motor imagery (Scherer 2004), as well as for P300 (Congedo 2006).

9.8.2. Neural networks

The second most used for classification method for BCI is using Neural Networks (NN). NN are non-linear classifiers that use assembly of neurons to produce the boundaries. The most used technique is the Multilayer Perceptron (Bishop 1995), that uses an input layer where the features are inserted, some hidden layers for processing and finally an output that defines the class (Figure 9.). Even though NNs can adapt to any number of classes and composed with enough neurons they can approximate any function, they are susceptible to over training and noise (Bishop 1995).

A conventional feed-forward artificial neural network (ANN´s) is a system constructed by a finite number of basic elements called neurons, which are grouped in layers. Every neuron is highly interconnected in the whole topology; the structure has a number of inputs and outputs that depends on the system that will be approximated.

A neuron is the basic element in an artificial neural network that simulates biological neurons which receives electrical impulses which are received through its dendrites, from other neuron’s axons. Those electrical impulses are added in order to have a final potential. This potential must exceed a certain level to have the neuron generate an electrical impulse on its axon. If the level required is not met, then the axon of that neuron doesn’t fire its axon. Neurons can be divided as: dendrites which are channels of input signals, core cell that processes all these signals and axons that transmit output signals of the processed information came from dendrites.

The ANN´s are applied to approximate normally a non-linear system as universal approximations. The first step to design an ANN´s is to train the neural network in order to fix the interconnection namely weights which are between the neurons. The training process can take a lot of time in the case of the back-propagation algorithm. After training the ANN´s the response could show a high-quality behavior, when a new input signals is presented to the system.

In other words the ANN´s could generalize any input signal. These ANN´s mimic the human brain, on the basic process of learning and generalization. Normally the process of training the ANN´s is slow and defining the correct topology could be complicated. The main advantages of artificial neural networks are:

• Ability to generalize and learn.

• Acquire knowledge from internal and external parameters.

• Ability to learn from examples and adapt to situations based on its findings.

• Generalization of knowledge to Production of adequate responses to unknown situations.

• Artificial neural networks solve complex problems that are difficult to manage by approximation.

• Produce linear or non-linear relationships

• Fault Tolerance

An extension to this technique is the Restricted Boltzmann Machines (RBM) that have a bidirectional connection between the layers (see Figure 9b.), this quality allows the RBMs to be train as normal NN and retrained using back propagation (Hinton 1986). Success of NN can be seen in (Kalcer 1993, Pfurtscheller 1996,

Hsu 2012) while for RBM in (Balderas 2011).

9.8.3. Non-linear Bayesian classifiers

There are mainly types Bayesian classifiers used for BCI systems: Bayes quadratic and Hidden Markov Models (HMM). Both these classifiers produce nonlinear decision boundaries. Furthermore, they are generative, which allows them to reject uncertain samples more efficiently than discriminative classifiers (Lotte 2007). While Bayesian assign the class to the feature vector with the highest probability, HMM is probabilistic automaton that can provide the probability of observing a given sequence of feature vectors (Rabiner 1989, Lotte 2007).

Figure 9.

Neural Networks architectures having Multilayer Perceptron and RBM

9.8.4. Nearest Neighbor classifiers

Nearest Neighbor classifiers are also used in BCIs with the k Nearest Neighbor (kNN) and Mahalanobis Distance (MDist) as preferred Nearest Neighbor classifier methods. kNN assign to an unseen point the dominant class among its kNN within the training set. kNN algorithms are sensitive to the curse of dimensionality making them fail in several BCI experiments, however they may perform efficiently with low-dimensional feature vectors (Lotte 2007). Mahalanobis Dististance based classifiers use Mahalanobis distance to assign a class to a feature vector to the nearest prototype. Mahalanobis Distance has been used to detect motor imagery of the hand giving accuracies over the 80% (Ming 2009).

9.8.5. Combinations of classifiers

Combinations of classifiers are proposed trying to reduce the variance and thus increase classification accuracy. Voting, Boosting, Stacking and Random subspaces. Voting consists of assign different classifiers the input feature vector and select the class with the higher majority of votes (hence the name). Boosting uses several classifiers in cascade where the errors committed by previous classifier are focus by each classifier. Stacking uses several classifiers (level-0 classifiers) running through the input vector. The output of the different classifiers is then use as input vector for a meta-classifier (or level-1 classifier) which is responsible for the final decision. Lastly Random subspaces uses subsets of the original feature vector as training set for different classifiers and the final decision is made by majority voting.

9.9. State of the art

The design of a BCI comes with two major challenges, the non-stationary and inherent variability of the EEG signals. Data from the same experimental paradigm but recorded at different instances are likely to exhibit differences due to; for instance; shift of the electrodes positions between sessions or changes in the sensor mechanical properties of the electrodes (e.g. change in the impedances). Adding to this problem the noisy nonlinear superposition of the measured EEG activity can mask underlying neural patterns and hamper their detection. The user current mentally state (e.g. due to tiredness, workload or stress) may impact in the ability to focus and generate specific mental events. Due to these factors, statistical signal processing and machine learning techniques play a crucial role in recognizing EEG patterns and translating them into control signals.

9.10. Co-adaptive training

A normal training of a BCI uses information from a first or previous sessions EEG recordings are used to pre-train the pattern recognition algorithms for classification or regression. On a posterior session user uses the trained algorithm for control. One of the drawbacks is that the variability of brain activity requires that the system is robust enough to handle the changes. Adding to this, the high adaptability of the brain gives the problem on how much has to be relegated for the system and how much left for the brain. It has been shown that using invasive over single neuron or a population of neurons the can rapidly learn to generate an appropriate pattern for a fix task. The same adaptation using EEG could take months to have a similar level of performance (Kübler 1999). Adding to this normal neuromuscular activity depends of feedback to have a successful control. A strategy to improve the control over the BCI system has to have a control that uses feedback. A good strategy is to use co-adaptive training, with a self-optimizing pattern detector and user adaptation, using new data to update the system. So new data is collected in different session and used to update the classifier to user’s most recent brain patterns. Feedback can be provided during the new session to helping to generate more distinct EEG patterns, which increases detection performance. An Online adaptation can be included to provide a faster update of the training parameters and have a faster co-adaptation.

9.11. Hybrid BCI

There exists almost no reason why different technology could be combined, combining different patterns (e.g. EP and motor imaging) or different recording technology, combining invasive and non-invasive, or using different electrophysiological signals with the combination of BCI technologies.

9.12. HCI recording

A case study was made using a commercial HCI, the amplifier Emotiv EPOC, which can record EEG signals as well as movement from the head with an incorporated gyroscope. The interface was created using the amplifier gyroscope signals as control for the displacement and the direction for an electric wheelchair (see Figure 10.).

Figure 10.

Example of the control with the movements of the head, translated with a gyroscope into the control of an electric wheelchair.

The gyroscope counts with two rotation axes that were used for displacement and turn. Also the velocity of displacement and turn was control depending on the amount of rotation the gyroscopes detect from the movement of the head.

The wheelchair counts with the displacement control of the two back wheels, giving it advance and turn control. This control was adapted to be controlled directly from a DAQ that has a direct interface with Labview.

9.13. Methodology

The case study was divided in three areas: Signal acquisition, Signal processing, and control signal (Figure 11.). For the signal acquisition we use the amplifier driver connected with Simulink (Matlab). Both signals were filtered and amplified using the rotation left-right for the turn and the rotation frontal-backward for forward or backward motion. The online process was done in the same interface that was used for recording in Simulink (Figure 12a.), which finally send the signals to Labview using a UDP protocol. Labview was finally in charge of the control signal (Figure 12b.) that had control over the wheelchair wheels.

Figure 11.

Control Process

Figure 12.

Matlab-Simulink and Labview interfaces

The control was first tested on a free environment and later on a simple maze (Figure 13.). Testing the manageability to make turns and understand the commands.

Figure 13.

Interfaces de Matlab-Simulink y Labview

10. Conclusion

Even though the control with the Emotiv EPOC was limited, the viability of developing an HCI was shown using the gyroscope signals as control signals for a wheelchair.