A User Centred Approach for Bringing BCI Controlled Applications to End-Users

1. Introduction

In the past 20 years research on BCI has been increasing almost exponentially. While a great deal of experimentation was dedicated to offline analysis for improving signal detection and translation, online studies with the target population are less common. Although BCIs are also developed for entertainment and thus potentially for healthy users, the main focus for BCI applications that are aiming at communication and control are people with severe motor impairment. There is a great need for translational studies that test BCI at home with the target population. Further, long-term studies with users in the field are required to improve reliability of BCI control. The user centred approach appears suitable to foster such studies.

In this chapter we will first define the needs and the gaps for bringing BCI to end-users and explain the model of BCI control which guides our interventions. Then we will describe the user-centered design and report first results of studies that adopted this approach for evaluating BCI applications. Those results led us to develop novel BCI components which we then tested with healthy and severely ill end-users. More specifically, we will introduce the optimized communication interface, the face speller, and remotely supervised BCI controlled brain painting with a locked-in patient in the field. We will end the chapter with summarizing the requirements for improvement and reasons for cautious optimism that the BCI community will be successful in providing end-users in need with reliable and independent BCI controlled applications.

1.1. The needs and the gaps

In 1973 J.J. Vidal posed the question whether “electrical brain signals” can “be put to work as carriers of information in man-computer communication or for the purpose of controlling such external apparatus as prosthetic devices…?“ (p. 157 [1]). Already in those days Vidal answered the question with a clear Yes and time has proved him right. Since the early nineties, when only few articles on brain-computer interfacing were available, publication activity has increased almost exponentially [2]. We performed a coarse search in Pubmed and PsychInfo with the terms BCI OR brain computer interface for 2011 through Sept 12 and received 461 hits. Thus, we may expect at least 700 publications by the end of 2013 indicating unbowed research activity, and thus funding. However, the amount of studies including the major target population, namely severely motor impaired individuals were 39 only. Less than 10 percent of the papers published, which refer to BCI in one way or another, deal with motor impaired individuals, although many authors mention those as target of their research [3, 4]. This illustrates quite overwhelmingly the gap between prosperous and active research in BCI laboratories with healthy participants and the transfer of the gained knowledge to the main target population of BCI, namely patients with severe motor impairment.

We are thus, facing a translational gap, i.e. a lack of translational studies that investigate the problems and obstacles that emerge when BCIs are to be applied to severely ill patients in their home environment. Such studies would include a thorough quantitative and qualitative evaluation of BCI. We argue and will describe that a user-centered design may be suitable to bridge this gap.

Further, we are confronted with a reliability gap, i.e. intra- and inter-individual performance varies tremendously when controlling an application with a BCI in the short-term and even more so in the long-term use. Many studies exist that introduce one or the other more or less small improvement in accuracy, bit rate or error rate – the main outcome measures of performance in BCI research. However, only few of them deal with targeted end-users in the field, where multiple sources of artefacts exist including changes of the health status of the user, such as altered brain responses due to neuronal degeneration in the brain. Thus, the reliability gap can only be bridged with longitudinal studies that include end-users in the field. Such studies need to take into account the several aspects that may contribute to successful BCI control. An integration of these aspects leads to a neuro-bio-psychological, data analytical, and ergonomical model of BCI-control (Fig.1) [5], which will be defined in the next section.

2. A model of BCI control

A BCI acquires input from the human brain, mostly its electrical activity recorded with electroencephalography (EEG), which is filtered, classified and transferred to an output signal. This output signal relates to the brain response or pattern of the BCI users and conveys the respective intention of the user. Importantly, the user receives feedback of his or her action and thus, BCIs imply a closed-loop between the system and the user. The output signal can be used to control an application – ideally, one that meets the desire of the user. Four aspects can be identified that contribute to BCI control: (1) individual characteristics of the BCI user, (2) characteristics of the BCI, (3) type of feedback and instruction, and (4) the BCI-controlled application [5]. The individual characteristics of the user include psychological, physiological and neurobiological factors. For example, visuo-motor coordination and motivation have been identified to predict performance with BCI controlled by sensorimotor rhythms [6] and event-related potentials [7]. Better inhibitory control, i.e. ability to allocate attention and inhibit distracting stimuli, measured as heart rate variability was related to better ERP-BCI performance [8]. The amplitude of the SMR peak at rest and the P300 amplitude evoked in an auditory oddball paradigm were also related to performance with the respective BCI [9, 10]. Further, the location and quantity of neuronal loss due to accident or disease may deteriorate performance. Besides the hardware used, the software components, namely the classifier of the input signal further determines BCI control (for review [11]). Common spatial pattern technique and stepwise linear discriminant analyses proved to perform well in SMR- and ERP-BCIs [12, 13].

Little research is available on how the type of feedback and instruction provided in a BCI setting may influence performance. From early neurofeedback studies it is known that immediate feedback is superior to delayed feedback which held also true in a BCI context [14]. It may also be the case that a more ecologically valid feedback in a virtual environment outperforms traditional two-dimensional feedback on a computer screen [15-17]. A quite robust finding across BCI types is that visual feedback is superior to auditory feedback [18-20]. In the SMR based BCI instruction to imagine motor imagery kinaesthetically leads to increased performance as compared to visual motor imagery [21].

Finally, the complexity of the application influences performance. Usually simple spelling tasks are mastered more accurately and faster than environmental control or control of information technology, such as internet [22, 23].

As can be seen, the model offers multiple toeholds for improvement and user feedback. In the following sections we will introduce novel achievements for BCI that improve and facilitate BCI use and are based on feedback provided by end-users within the user-centred approach. Before we detail the novel approaches, the user-centred design and its application to BCI will be outlined.

Figure 1.

A model of BCI-control comprised of 4 aspects: individual characteristics, BCI characteristics, feedback and instruction, BCI-controlled application. Colours serve for distinction of categories only. Boldness of black arrows indicates possible strength of influence on BCI control [5].

3. The user centred design and its application to BCI

BCI development demands for close investigation of the end-users’ needs and requirements and of the restrictions that come along with their diseases. The latter restrictions may range from small artefact contamination of the recorded brain signal up to loss of perception modalities, e.g. loss of ocular control as often the case with progression of neurodegenerative diseases. Furthermore, attention allocation may be limited and long lasting training sessions may be too demanding. BCIs are required to accommodate for such restrictions and to offer appropriate solutions, such as switching to auditory or tactile modalities when vision is impaired. Many of these restrictions are not evident when testing systems with healthy users. Furthermore, a system in daily use has to meet other requirements than a system developed for research purpose only, e.g. with regard to hardware setup, software handling and technical support. Bringing BCI technology to end-users’ homes thus, inevitably requires involving them into this developmental processes.

More recently the potential user of a BCI came more into the focus of BCI development and user-centred approaches were adopted [22, 24, 25]. A user-centred approach implies early focus on users, tasks and environment; the active involvement of users; an appropriate allocation of function between user and system; the incorporation of user-derived feedback into system design; and an iterative process whereby a prototype is designed, tested and modified [26]. The user-centred approach was standardized with the International Organization for Standardization (ISO) 9241-210 (Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems). According to this approach three kinds of requirements have to be taken into account: (1) Business requirements: Here, typically, a specific number is set in terms of how many systems should be sold in a defined time frame. Although our face speller and brain painting (see below) have already been adopted by a company (http://www.intendix.com/) and are thus, available on the market, these products are not yet suitable for daily use in the field. (2) User requirements and functional specification: BCI requirements need to be specified from a user’s point of view, including the functions required to support a user’s tasks, the user-system and interfaces. Usability goals that must be achieved and the approach for system maintenance at the user’s home need to be defined. (3) Technical requirements: It has to be specified how the system will achieve the required functions and what data structure must be available for internal processing for the approach to be successful. Technical constraints need to be defined, such as the maximum data communication speed over a network or the trade-off between good EEG measurement and comfort with regards to the EEG cap. On the basis of these requirements Zickler and colleagues asked experts in using assistive technology (AT), i.e. people with severe motor impairment, what they would consider the most important requirements for BCI [25]. Those requirements were functionality, independent use, and easiness of use (see section on “User-centred improvements of BCI controlled applications”).

Two different approaches to BCI control were subject of evaluation following these standards: BCIs dependent on modulation of sensorimotor rhythms, referred to as SMR-BCI, and on detection of event-related potentials, referred to as ERP-BCI. To better understand the applications and their evaluation, we provide a condensed description of the SMR- and ERP-BCI as implemented for control of the specific applications described below.

3.1. SMR-BCI

BCIs can be established by detecting an active modulation of sensorimotor rhythms (SMR) over sensorimotor areas of the brain. In a resting state, these rhythms are highly synchronised in the alpha (10-12 Hz) and beta (12-30 Hz) bands. When moving or imagining a movement, these rhythms desynchronise, i.e. the power of these frequency bands can actively be modulated by the user. Thus, SMR modulation constitutes a signal for BCI control [27, 28]. Different classes of motor imagery can be selected depending on a user’s individual brain signals and the degrees of freedom that are required for control of an application. In a typical SMR-BCI, users trigger control signals for two classes by either imagining movement of the right or the left hand. Feedback is provided during the imagery tasks to enhance participants’ performance thereby reinforcing correct behaviour. As hand areas are largely separated in the sensorimotor cortex, the evoked patterns are usually well distinguishable. Importantly, it has been shown that people with amyotrophic lateral sclerosis can utilize such modulations of the SMR to operate a BCI [29]. One of the remaining issues, however, is that a large number of participants is not able to achieve sufficient SMR-BCI performance [7, 9, 30, 31]. BCI systems that do not rely on such active modulations of brain signals are available. The most frequently used system is described in the next section.

3.2. Event-related potential (P300) BCI

A typical BCI based on event-related potentials is the so called P300-Speller, providing muscle independent communication on a character-by-character basis [32]; for recent reviews: [33] and [34]. A character matrix is displayed on a computer screen and groups of characters (usually rows and columns in a matrix) are highlighted (flashed) in random order. Users focus their attention on the desired field of the matrix (the target) by counting the number of flashes whilst ignoring all other characters (non-targets). This pattern constitutes an oddball-paradigm as target flashes are rare (odd) as compared to the high amount of non-target flashes. For example, in a 6x6 matrix one row and one column contains the target character whereas 5 rows and 5 columns are to be ignored. Each stimulus triggers distinct event-related potentials among which the P300 usually is the most prominent. It is a positive deflection in the EEG which occurs roughly around 300 ms post stimulus. Its latency may strongly vary with paradigms and across individuals (for review [35]). Yet other ERPs are also elicited, therefore a time window of up to 1000 ms post stimulus (typically 800 ms) is recommended to investigate users’ individual ERPs (i.e., negative and positive deflections at distinct latencies). The characteristic sequence of event-related potentials is identified for each row and each column. The row and column with the most prominent ERPs are selected and the respective letter appears on the screen. It has been shown that 72.8% of N=81 healthy BCI users were able to communicate with 100% accuracy by means of such an ERP-BCI and that less than 3% could not achieve any control [30]. Importantly, these results transfer to individuals with severe motor impairment, e.g. due to neurodegenerative disease, in that the speller can be utilized as a muscle independent tool for communication (e.g., [22, 36-39]; for review [40]). Since its first description in 1988, the P300-Speller has been used intensively, further investigated and modified in a plethora of research publications leading to new applications for communication and device control (for review, e.g. [34]).

4. Evaluation of BCI controlled applications

The ISO 9241-201 (2010) defines usability as the “extent to which a … product … can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (ISO 9241-201, 2010, p. 3). Effectiveness refers to how accurate and complete the users accomplish the task. Efficiency relates the invested costs, i.e. users’ effort and time, to effectiveness. User satisfaction refers to the perceived comfort and acceptability while using the product. Context of use refers to users, tasks, equipment (hardware, software and materials) and the physical and social environments in which a product is used (ISO 9241-201, 2010, p. 2) [22].

To accommodate for these aspects when evaluating newly developed BCI driven applications, a set of measures has been compiled to assess effectiveness, efficiency and satisfaction [22]. Effectiveness refers to how accurately end-users can communicate with the BCI and is operationalized by the numbers of intended and thus, correct selections in relation to the total number of selections. This measure is also often referred to as accuracy. Efficiency comprises the amount of information transferred (bit rate), which expresses speed and accuracy with one value, and the workload experienced by the end-user. A measure to assess subjective workload is the NASA task load index (TLX) which quantifies the workload for each task and identifies its sources [41]. Workload is defined as physical, mental, and temporal demands, and performance, effort, and frustration. User satisfaction can be addressed with the Quebec User Evaluation of Satisfaction with assistive technology (QUEST 2.0) which is the only standardized satisfaction assessment tool that was designed specifically for AT-devices [42]. It explicitly allows for deleting inadequate and adding informative questions with respect to a specific AT so that BCI specific items could be integrated. Reliability, speed, learnability, and aesthetic design were added to accommodate for specific aspects of BCI and the resulting questionnaire was referred to as Extended-QUEST [22]. Possible ratings range from 1 to 5 with 5 indicating best possible satisfaction.

As another measure of device satisfaction the ATD PA Device Form was used. The Assistive Technology Device Predisposition Assessment (ATD PA) is a set of questionnaires based on the Matching Person and Technology Model (MPT) of Scherer (2007) [43]. It addresses characteristics of an AT-device and asks respondents to rate their predisposition for using the AT under consideration. The questionnaire rates the AT-person match and the expected support in using the device, in other words the expected technology benefit [44].

As a coarse measure for overall satisfaction with the device, a visual analogue scale (VAS) ranging from 0 to 10 (not at all – absolutely satisfied) was included in the evaluation procedure. An open interview allowed participants to state their opinion about the BCI and its application and recommendations for further development.

To date, with this instrumentation three studies were performed with severely impaired end-users [22, 45], which we will describe in the following subsections.

4.1. Extended communication

Zickler and colleagues investigated the first prototype in which BCI was integrated into a commercially available AT software [22]. Control of AT was realized by means of the ERP-BCI described above. Participants tested the text entry, emailing and internet surfing options (Fig 2). The oddball paradigm had to be implemented such that these applications provided by the standard software could be controlled. Instead of rows and columns flashing red dots were assigned to each possible selectable item. The red dots then flashed in random order. Participants were able to write a text, send an email and surf the internet for a specific website.

Figure 2.

Emailing and internet surfing with the Qualilife software. Possible items to select are indicated with a red frame. The red dots appear randomly at every item which can be selected. Thus, the to-be-selected item again constitutes a rare target within frequently appearing irrelevant items, and hence, the oddball paradigm is realized (Figure 1 from [22] with permission).

Selection accuracy (effectiveness) ranged between 70 and 100% correct responses and for all participants internet surfing was the most difficult task. Information transfer rate (efficiency) was between 4.5 and 8 bits per minute. Experienced workload (efficiency) was quite different among users. While one user rated workload on all dimensions between 9 and 12 (of 100 possible, with 100 being the maximum possible workload experienced), two participants were always between 34 and 46 indicating moderate workload for all tasks. In one user, who was confronted with BCI for the first time, workload decreased with every session from 49 to 15, which was encouraging as it demonstrated that workload can be decreased with practice. Satisfaction was high for safety of the device and the professional services and low for adjustment. With regards to the BCI specific items, reliability and learnability were rated high while speed and aesthetic design were only moderate. Obstacles for use in daily life were (1) low speed, (2) time needed to set up the system, (3) handling of the complicated software and the (4) demanding strain that accompanies EEG recordings (washing hair, etc.). Overall satisfaction ranged from 4 to 9 indicating substantial variance and considerable room for improvement. In the interview participants stated that the greatest obstacle for use in daily life would be the EEG cap, there should be no cables, no gel and it should look less eye catching. Hardware should be within one device (instead of an amplifier, a laptop and a screen) and wheelchair control should be integrated. None of the participants could imagine using the BCI in daily life unless substantially improved.

The above described BCI controlled application already goes beyond simple verbal communication and may constitute a step toward inclusion via the world wide web. Some of our patients have been participating in BCI studies for a long time [46] and stated that they would also like to control other, more entertaining applications such as playing games or painting.

4.2. Brain painting

Together with an artist (Adi Hösle www.retrogradist.com) the letter matrix controlled by the ERP-BCI was transformed into a painting matrix which allowed the user to select shapes, size, colours, and contours and to move a brush on a virtual canvas (Fig 3). One participant stated “Everyone talks about freedom, but the worst oppression is to be locked into my own body. This art form allows me to break from the prison…”. With his painting (see Fig 4) he wanted to illustrate that there is a light at the end of a tunnel.

Figure 3.

Brain Painting matrix. For painting an object and its shape, location and transparency have to be defined. Only after the selection of “color” the object is transferred to the “canvas”. In the toolbox at the top of the screen the latest selections are shown (from left to right in this figure): grid size (3), brush size (1), transparency of color (100%), object shape (rectangle), color (black). In the last square of the toolbox the latest selection is shown, which in this example is “black”.

Figure 4.

Painting “Who” by a brain painter with locked-in syndrome.

Four severely motor impaired potential end-users participated in the evaluation study which comprised seven daily sessions. In five of those sessions, participants could freely paint pictures of their choice. Effectiveness ranged between 80 and 90%, i.e. in 80 to 90% of the time participants selected the item they intended to. With an average around five bits per minute the information transfer rate (efficiency) was relatively low. This was due to an extended break between selection of items, to provide the user with sufficient time to think about what to select next (“creative pause”), and users explicitly appreciated this adaptation of the selection speed. Workload varied considerably between 20 and 50 and was sometimes due to disease related physical problems experienced by the users, and thus, independent of the specific BCI application. Like in the communication application described above, reliability and learnability were rated high (4.2 and 5.0) whereas users were not so satisfied with speed, adjustment and dimensions [44]. For two users the ATD PA Device Form indicated a good match between the system and the user (4.3 and 4.2 of 5 possible), but for the other two only 3.4 and 3.8 indicating that the match could be improved [44]. Overall satisfaction ranged between 5 and 8 also leaving room for improvement.

Taken together, users enjoyed painting and painted up to one picture per session. Three users would have liked to use Brain Painting in daily life once or twice a week. They reported high satisfaction with the learnability, ease of use, and reliability of the device. The EEG-cap and system operability clearly required improvement if the BCI application was to be used in daily life [44].

4.3. Gaming

Four severely disabled end-users - two in the locked-in state – evaluated the gaming application Connect-Four (http://en.wikipedia.org/wiki/Connect_Four) [45]. Connect-Four is a SMR-BCI based prototype, enabling end-users to select either a row or column and setting a coin by regulating their brain activity. In six BCI sessions end-users were trained to regulate their brain activity in copy-tasks (location of coins were pre-defined by the experimenter), which were followed by free mode game playing. Effectiveness in the copy-task was low to medium in three of four end-users, with accuracies varying between 47% and 73%, and only one end-user, in the locked-in state, achieved high BCI control with up to 80% accuracy. With an ITR ranging between 0.05 and 1.44 bits/min, efficiency was low. The end-users rated their subjective workload moderate (on average between 28 and 52 of 100), with mental and temporal demand contributing most to their workload (efficiency). Two end-users reported high frustration which first increased and then decreased again with sessions. Nevertheless, the BCI game was accepted well by the end-users. On average end-users were moderately to highly (3.8 for the total Quest score and 3.9 for the added BCI items total score; ratings ranging between 1 and 5 with 1 indicating “not satisfied at all” and 5 “very satisfied”) satisfied with the BCI (satisfaction). End-users were highly satisfied with weight, safety and learnability (4.3, 4.5 and 4.8). Reliability and speed were rated moderately (3.5). Main obstacles were the EEG-cap and electrodes, time-consuming and complex adjustment, difficulty to handle BCI equipment and low effectiveness. Like in the other two BCI controlled applications, the evaluation by the end-users implied that there is need for improvement. It seems to be more challenging to implement an SMR-BCI in activities of daily living of end-users as compared to an ERP-BCI controlled application [22, 47]. Two end-users (one of them locked-in), however, stated that they could imagine using Connect Four in their daily life. The other end-user in the locked-in state could imagine using the BCI in his daily life provided substantial improvement. The fact that both locked-in end-users were highly motivated throughout the BCI sessions and did not report any frustration, even when BCI control was low, implies the need and hope of these patients that BCI may provide better communication and control opportunities.

Taken together, such evaluation studies are first steps toward bridging the translational gap experienced in BCI research and development. Based on these evaluation results we state that to date ERP-BCIs are more effective and efficient for communication and interaction as compared to SMR-BCIs (Table 1). End-users indicated that the speed of the BCI controlled application was too low. Users would have liked to use the Brain Painting application several times a week, but none could imagine to use the BCI for emailing and internet surfing unless substantially improved. Somewhat surprisingly two end-users could imagine playing Connect Four in daily life despite low control. Table 1 summarizes the evaluation results for all applications.

Table 1.

Summarized evaluation results for the three applications. Clearly, all of them leave room for improvement. However, end-users would have liked to use the Painting and Gaming applications in their daily life.

5. User-centred improvements of BCI controlled applications

As outlined above, functionality, independent use, and easiness of use were rated by expert users of assistive technology (AT) as most important for BCI use in daily life. In the next sections we will describe how we addressed and improved these three aspects.

5.1. Functionality

In an effort to bridge the reliability gap and to address speed of the BCI, we changed the stimulation mode of the widely used P300 spelling matrix. In the commonly used ERP-BCI, characters are light flashed and attention to one of the characters will usually elicit a distinct P300 [32] and sometimes other ERP components such as N100 or N200 (e.g., [48-50]). One option to increase reliability of the system is to enhance signal to noise ratio of the recorded ERPs. It is well known that familiar faces elicit characteristic ERPs, among which the N170 and N400f (f for faces, Figure 5) are very reliable ERPs. Thus, instead of flashing the letters of the matrix we overlaid row- and column-wise a famous face (the face of Albert Einstein or Ernesto Che Guevara, [51]). Figure 5 provides a screenshot from such modified BCI matrix and illustrates the grand average event-related potentials across N=20 healthy participants. Increasing the signal-to-noise ratio by eliciting more target specific ERPs, significantly boosted offline BCI performance. Importantly, these findings were replicated online in a group of possible end-user of BCI with severe motor impairment, e.g. users with amyotrophic lateral sclerosis or spinal muscular atrophy [38]. They benefited to such an extent that even some users who were unable to operate the traditional ERP-BCI, reached an online accuracy of 100% due to the face stimulation. As such it was possible to decrease the number of stimulation cycles without negatively affecting performance, i.e. bit rate was strongly increased. In six online runs, the number of stimulation cycles was decreased from 10 to 6, 3, 2 and 1 (i.e. single trial) stimulation sequences. Performance in N=9 users with neurodegenerative disease was significantly increased in all runs when exposed to the face speller as compared to the classic ERP-BCI. Furthermore, we compared their single trial performance to the online performance of N=16 healthy participants. As usual, performance was significantly worse in the classic ERP-BCI, however, no difference was found for the face speller. These results clearly underline how modifications to the system can diminish performance drops in end-user samples. Zhang and colleagues (2012) reported that inversion of faces may further increase the N170 component and thus, performance in the BCI task. Face motion, face emotion and face familiarity, however, did not affect BCI performance [38, 52]. We conclude that investigating stimulus material other than the classical character highlighting is a very promising direction for addressing speed and reliability of the system.

Figure 5.

left: Instead of flashing letters in the rows and columns, rows and columns are overlaid with the face (Einstein is not shown due to copyright). Right: Averaged evoked potentials as response to targets and non-targets. In the face condition prominent N170 and N400f appear in addition to the P300. The ERP amplitude is depicted as a function of time [51].

5.2. Easiness of use

We developed a so-called optimized communication interface which allows for auto-calibration and word completion and is controlled with a user-friendly graphical interface [47]. After the subject is set up with the electrode cap and connected to the BCI by an expert, the calibration process for parameterizing the classifier can be started by pressing a single button on the screen. No familiarity with technical or scientific details of the BCI is required. Data from calibration is automatically analysed in the background, invisible to the user who only receives a feedback on successful or unsuccessful outcome of the calibration. In the latter case, calibration can be performed again with one click. Yet, if successfully calibrated, communication with the P300-BCI can be initiated with another button press. We tested if such a user-friendly BCI implementation can be handled independently by naïve users. All healthy subjects (N=19) handled the BCI software completely on their own and stated that the procedure was easy to understand and that they could explain it to a third person. A text completion option significantly decreased communication speed. We conclude that from a software perspective, a BCI system can easily be integrated into an automated application that allows caregivers, friends or relatives to control such complex systems without prior knowledge at the end-user’s home or bedside.

5.3. Independent use

Finally, to bridge the reliability gap we implemented BCI controlled brain painting for long-term use at the home of a 72-years old locked-in patient diagnosed with amyotrophic lateral sclerosis (ALS) who used to be a painter [53]. The brain painting application, which was successfully tested and evaluated by healthy subjects [23], as well as patients ([44] and see above) was embedded into an easy-to-use interface enabling to use the application after a few steps only. Family was trained to set-up the 8-channel EEG-cap and amplifier and to control the brain painting interface. The brain painting software automatically saved the duration of painting time, number of runs, and the paintings, and transfered them to our lab for remote supervision. After every session, satisfaction was rated. In a separate window familiy and caregivers can comment on the session. In doing so, occurring problems can be noticed and remotely solved by our experts via remote internet access. Figure 6 shows the end-user in a brain painting session at her home.

Figure 6.

ALS patient at her home, after finishing her brain painting. While painting, the brain painting matrix appears on one screen while on an additional monitor, placed on the table in the background, she can follow the progress of her painting. The brain painting software is operated by the family or caregivers and requires few steps only for set up.

After each session the end-user is asked to rate her satisfaction with the visual analogue scale (VAS) (Figure 7) and after approximately 10 sessions the workload and device satisfaction are assessed with the NASA TLX [41] and the Extended QUEST 2.0 [22, 42]. Her responses as well as her data can be observed by our experts remotely to allow for system modifications or other interventions if necessary (e.g. advise for recalibration of the system).

Figure 7.

Ratings of satisfaction (VAS Satisfaction; VAS = visual analogue scale) after each of 86 sessions with the brain-painting application with 0 indicating „not satisfied at all” and 10 indicating “very satisfied”. Satisfaction ratings vary strongly between very low satisfaction (rating between 0 and 3) and very high satisfaction (rating between 7 and 10). The low ratings in the first 20 sessions were always due to malfunction of the BCI which was still in the set-up state. Continuous remote access to these data allowed for in-time modifications to the system by our experts (Holz et al., in preparation).

In more than 8 months the end-user has been painting in more than 86 BCI sessions with an average paiting duration of 66.2 minutes. Satisfaction with device strongly depended on functioning of the BCI (Figure 7). When implementing a remote-controlled BCI application problems of malfunctioning arise which are immediately visible in the satisfaction rating (e.g., sessions 9 and 17 in Figure 7). Three types of sources for her dissatisfaction could be identified: In most of the cases dissatisfaction was due to technical problems (software/hardware; especially in the first sessions after set-up of the BCI system at the end-user’s home); second, due to problems from the end-user’s side, e.g. low concentration or exhaustion or not being able to realize the desired painting; and third, due to bad control (e.g. due to false cap placement, insufficient electrode gel) or loss of control over time (e.g. due to electrode gel drying).

Also for this locked-in BCI end-user effectiveness, reliability and easiness of use were the most important aspects for device satisfaction. Additionally, she mentioned professional support, specifically during times in which the system was not running properly. With a mean VAS satisfaction score of 6.2, her overall satisfaction is moderate to high. However, there is high variability with lowest satisfaction when the system was not working (early sessions) and when the painting was not as she expected it to be (later sessions). Highest ratings indicate that the system worked properly and that she was satisfied with her painting. Despite initial problems with the BCI, her motivation to continue brain painting has remained high even after more than 80 sessions. The end-user is currently painting 2-3 times a week, but stated that she would like to paint every day, if she could do so. The limiting factor is the available time of the family setting up the BCI. But currently also caregivers and friends are willing to learn the set-up and control the application to enable her to paint more often. In conclusion, our results demonstrate that expert-independent BCI use by end-users in the field is possible and illustrate the important role of family and caregivers when transferring BCI technology from the research environment to the end-user’s daily life. Figure 8 depicts some of her brain paintings.

Figure 8.

Example Brain Paintings of the BCI -user with locked-in syndrome. All paintings were painted with the BCI in her daily life, independent of BCI expert’s control, (with friendly permission from the brain painting artist).

6. Conclusions

Taken these results together, we can state that milestones were achieved in bringing BCIs to end-users. BCIs were combined with standard assistive technology, set up of the system including handling of software was facilitated tremendously and spelling speed was increased whilst maintaining high accuracy levels by altering the stimulation mode. In one exemplary end-user with severe motor impairment an application was installed at home such that family and caregivers can set up the system and maintenance and support is provided remotely. With innovative applications to be set-up at the end-users’ home and long-term studies first steps have been undertaken to bridge the translational and reliability gaps encountered when bringing BCIs to end-users. The user-centred iterative process between developers and end-users proved successful and the results are powerful demonstrators that BCIs are well coming of age and can face the transfer out of the lab to the end-users’ home.