Open access peer-reviewed chapter

Improving the Brain-Computer Interface Learning Process with Gamification in Motor Imagery: A Review

Written By

Chongli Gao, Mingyue Xia, Zhehao Zhang, Yongpeng Han and Yaxin Gu

Submitted: 08 May 2022 Reviewed: 06 June 2022 Published: 13 October 2022

DOI: 10.5772/intechopen.105715

From the Edited Volume

Gamification - Analysis, Design, Development and Ludification

Edited by Ioannis Deliyannis

Chapter metrics overview

182 Chapter Downloads

View Full Metrics

Abstract

Brain-computer-interface-based motor imagery (MI-BCI), a control method for transferring the imagination of motor behavior to computer-based commands, could positively impact neural functions. With the safety guaranteed by non-invasive BCI devices, this method has the potential to enhance rehabilitation and physical outcomes. Therefore, this MI-BCI control strategy has been highly researched. However, applying a non-invasive MI-BCI to real life is still not ideal. One of the main reasons is the monotonous training procedure. Although researchers have reviewed optimized signal processing methods, no suggestion is found in training feedback design. The authors believe that enhancing the engagement interface via gamification presents a potential method that could increase the MI-BCI outcome. After analyzing 2524 articles (from 2001 to 2020), 28 pieces of research are finally used to evaluate the feasibility of using gamified MI-BCI system for training. This paper claims that gamification is feasible for MI-BCI training with an average accuracy of 74.35% among 111 individuals and positive reports from 26 out of 28 studies. Furthermore, this literature review suggests more emphasis should be on immersive and humanoid design for a gaming system, which could support relieving distraction, stimulate correct MI and improve learning outcomes. Interruptive training issues such as disturbing graphical interface design and potential solutions have also been presented for further research.

Keywords

  • motor imagery
  • brain-computer interface
  • gamification
  • video game
  • training

1. Introduction

World Health Organization (WHO) claims that over 1 billion people suffer types of disability worldwide [1]. Neurological disease is accepted as one of the reasons causing disability. For instance, stroke attacks more than 20 million people each year and causes 45% of the total to become permanent upper-body disabled [2], without mentioning a significant number suffer from lower limb disability or temporary limited mobility. Although there have been several advanced therapies for neurological rehabilitation, the range of compatible users and the rehabilitation efficiency are still limited. For example, in 2017, Langhorne et al. [3] applied a very early rehabilitation trial (AREAT) among patients who suffered a stroke after 24 hours. However, in the inclusion criteria, patients must have at least a capacity to react with verbal commands. Patients with lock-in syndrome [4] would discover it tough to gain such interventions since injury risks might occur if patients could not timely express whether their bodies are suitable for the training intensity or not.

Brain-computer interface based motor imagery (MI-BCI) has the potential to unlock the potential interaction between these patients with their environment. Motor imagery (MI), imagining kinaesthetic movements of parts of the body [5], is widely suggested to utilize as a training method to improve physical capacities and rehabilitation outcomes. Articles claim that MI training could enhance physical performances in tennis [6], basketball [7], and Water polo [8]. Furthermore, this therapy could also positively affect functional network efficiency [9] and motor learning of locomotion [10]. Brain-computer interface (BCI), also called Brain-Machine Interface (BMI), is an advanced technology that plays an intermediary role in sharing information between a human’s brain and external devices. With a non-invasive BCI device (wearable head cap with sensors attached for signal monitoring, rather than invasively inserting sensors into the skull), users could safely modulate their brain activity to interact with external devices, such as a robotic arm [11]. The interaction actively navigated by a human is called active BCI mode. This novel control strategy, combined with motor imagery [2, 12, 13], would widen the range of potential beneficiaries of assistive or rehabilitative robotics, such as diagnosed lock-in patients. For example, using MI-BCI for exoskeleton navigation would help lock-in patients to have accessibility for walking assistance by mere imagination [14]. Combining with this principle of robotic therapy, such as residual cortical and subcortical neuronal group facilitation [15], a more significant number of patients can gain positive outcomes.

Dating back to one classical piece of research in 1993, one of the earliest MI-based BCI applications, called motor planning at that moment, was developed and reported at the University Technology of Graz [16]. Afterward, an increasing number of research groups began to focus on potential MI-BCI applications.

1.1 Challenges

Although MI-BCI has been given great attention compared to other BCI applications [17], the gradual reduction in research interest hints at the bottleneck that academia is encountering in MI development. One of the main reasons, claimed by Hwang et al. [17], is that other BCI paradigms could have a shorter training duration and higher information transfer rates (ITR) than MI-BCI. The performance issue mentioned above is not an exceptional case. Several errors are repeatedly diagnosed when applying non-invasive MI-BCI for further research with humans. For instance, one BCI group [18] identifies that distraction around the environment might cause insufficient attention failing to output accurate results and promoting frustration. Therefore, the MI-BCI control system in users learning outcomes seems less competitive than its counterparts.

The most commonly used MI-BCI training modes [19] might not help overcome those challenges above since participants would hardly engage with the graphical interface design [20]. Thus, a new solution for improving the learning outcome of MI-BCI users is demanded.

1.2 Potential solution

Gameplay-based learning, gamification, is a potential method to level up the non-invasive MI-BCI learning outcome. The advantages of gamification in training are the upgraded level of users’ engagement and happiness, thanks to its real-world simulation and exciting game content [21]. These might prolong the users’ time of willingness to engage in the therapeutic MI-BCI activity. Using gamification for training is also supported by a consensus [22] that playing a PC game has motivations and benefits for users to learn skills. These pieces of evidence increase the possibility of using gamification in MI-BCI.

In 2003, the Graz group designed one of the initial motors imagery-controlled BCI games. Users could imagine their left or right-hand movement to navigate the falling ball inside the screen to the same side basket to finish this gamified task [23]. The interaction between users and the game could teach the participants to adjust to performing MI correctly. Almost at the same time, a 3D first-person shooting game (3D ShT) is created to declare the feasibility of binary gaming MI-BCI control [24]. These findings support the technical feasibility of gaming solutions in MI-BCI.

1.3 Aim of the study

Although pieces of gamified MI-BCI research have published, no literature has mainly and specifically analyzed the existing gamified MI-BCI training methods. This research gap reveals the essence of accomplishing one literature review for the above purpose. Thus, this paper will review the current non-invasive MI-BCI game in academia. We expect the result could help identify the feasibility of existing MI-BCI game development. This finding probably helps optimize the future research-based game design in testing whether gamified MI-BCI training mode would have a better outcome than the traditional mode.

1.4 Research question

This paper designs research questions (RQ) and a search plan using a topic-relevant review for the template [25]. One main RQ is: whether gamification of MI-BCI is feasible as a training method. Specifically, this review answers 2 RQs presented below (see Table 1).

Research questionNote
RQ1: What games have been used in MI based BCI?
  • What is the genre that each game belongs to?

  • What type of subject (healthy or disabled) has participated in each game testing?

  • What are the learning outcomes of each game study?

Hybrid BCI would not be included
RQ2: What are the factors that researchers should be focusing on when designing a gamified experiment.
  • What are the main metrics that have been used

  • What are the training issues that have been reported

This research presents main quantitative and qualitative metrics.

Table 1.

Two RQs with their detailed aspects of questions.

1.5 Outline

The organization of this review is as follows: The research technique used for reviewing and analyzing literature is presented in Section 2. The findings will be arranged in Section 3, followed by a discussion that focuses on a recommendation of MI-BCI game preferences in Section 4, ended with a conclusion.

Advertisement

2. Method

An MI-BCI review could provide instructions for better game design in the future. We finished a pre-literature review to ensure the scheduled literature review method is achievable. After pre-reviewing, this paper believes that although existing MI-BCI games are various, the research fields regarding what MI-BCI games have been used and what game genres are more preferred for MI-BCI training than others are still virgin land. Thus, a systematic review (SR) based selection strategy could be used in this review to present data objectively. However, a meta-analysis is challenging because of the limited number of published articles and reported trials. Thus, the following part will introduce how this review attempts to utilize PRISMA [26] for scientific analysis.

2.1 Eligibility criteria

Out of variables controlling, this review merely focuses on studies asking subjects to use MI to control a BCI game. In detail, we select experiments with all healthy and disabled human participants using MI solely to control a BCI device to play a game (actively non-invasive MI-BCI). The BCI device must be non-invasive. Additionally, metrics cover both quantitative and qualitative learning outcomes. A Hybrid MI-BCI control system, BCI for assessing MI function, or no human experiment would exclude. Summarization of all the critical aspects of the eligibility criteria is presented in Table 2.

Aspect in the criteriaEligibility
Control methodUsing only motor imagery to control a BCI game
BCI device type (invasive?)Non-invasive BCI
BCI device type (passive, reactive, active?)Active
BCI learning outcomeQuantitative and qualitative outcome
BCI experiment typeHuman experiments only
Participant typeAll healthy and disabled cohort

Table 2.

Summarization of eligibility criteria.

2.2 Information sources and searching plan

We apply three searching databases: IEEE Xplore, Scopus, and PubMed/MEDLINE. This review includes three strategies (see Table 3) with two additional plans for finding articles. The pre-searching plan is designed relatively broad so that a brief idea of the current gaming MI-BCI systems could help revise the main-researching plan. In contrast, the main-searching plan is designed relatively specifically after referring to one current published topic-relevant SR [25] and the pre-searching result. We design a compensatory plan so that targeted articles with a slightly expressive difference are unlikely missed. One additional search plan utilizes Google scholar, and another technique is to dig topic-related articles presented as references in the articles already reviewed. The key searching terms are separated into three groups based on the MESH library in Pubmed, template SR [25], and another relevant SR discussing BCI games [27]. All search plans are based on title, abstract, and keywords (TAK), and duplicative articles would be identified and removed via EndNote X9 and a relatively deep read.

PlanGroup1Group2Group3
Pre-searchBrain-Computer InterfaceVideo game
Main-searchBCIGameMotor imagery
Brain-Computer InterfaceEntertainment
Brain Machine InterfaceVideogame
NeurofeedbackGaming
EEGGamification
ElectroencephalogramGamified
Mind controlledCompetition
Brain Controlled
Biofeedback
compensatoryMotor imagerygame
gamification

Table 3.

Three searching plans with separate groups.

2.3 Study selection

Three main types of criteria have been used in this review: Selection Criteria (SC), Exclusion Criteria (EC), and Inclusion and Quality Criteria (IQC). SC is for broadly filtering articles. After determining the scope of literature via SC, EC will determine this scope more precisely. IQC is adopted to decide whether the articles could be suitable after deep reading. The specific SC, EC, and IQC can be seen below:

2.3.1 Selection criteria

For objectiveness, the chosen articles must be written in English. Furthermore, these articles must be peer-reviewed, covering a designed MI-BCI game. There is no restriction for the published years as some early MI-BCI game systems, such as the first-person shooting game [24], are still highly valued in the academic area.

  • Written in English

  • Peer-reviewed

  • No searching duration is limited

  • TAK is discussing about a non-invasive MI-BCI game.

2.3.2 Exclusion criteria

The remains still need to be under consideration before being used as a definitive reference. For example, some titles or abstracts, including the typed terms but different prior research aims, should be excluded. We would not recommend any literature review without one own experiment. Additionally, reports aiming at pure MI therapy without BCI, or invasive MI-BCI research, would be withdrawn. Other factors such as a hybrid control method, which might positively influence the feasibility of an MI-controlled BCI game, should be excluded. The lists below are the detailed EC.

  • The outcome of the articles does not cover gaming MI-BCI

  • If the paper is a review, there is no MI-BCI research or outcome but just the result of literature analysis.

  • Motor imagery is not for BCI.

  • Gamified MI-BCI system with other control or feedback methods as a factor that could influence the result, such as a hybrid BCI control system with MI and P300 together [28].

2.3.3 Inclusion and quality criteria

Rather than distinguishing between inclusion criteria and quality criteria, this paper combines these two as IQC (see Table 4). This combination is because the inclusion criteria could also be qualified (totally fulfilled or partially fulfilled) and become one part of the article evaluation. We will withdraw the study if it cannot match any one of the two compulsory questions (CQs). Six main quality questions (QQs) are subsequently presented to assess the level of the selected article. If the article failed to be higher than the four scores, this work would also be rejected because of the low report quality.

Criteria typeNo.ScoreCriteria requirementLevel of result
CompulsoryCQ1Y1Are the goals of the paper clearly stated and include gamified MI-BCI system?If No, the article should be ignored
P0.5
N0
CQ2Y1Is there any clear outcome related to gamified MI-BCI system presented?
P0.5
N0
Non-CompulsoryQQ1I5Is this article mainly talking about game (2) training (3) improvement (5)Satisfactory >= 4
T3
G2
QQ2N1Is there any main focus on one of the steps in the whole loop of MI-BCI
(recording brain activity, preprocessing, feature extraction, classification, translation, and receiving feedback) [29]
P0.5
Y0Good >= 7
QQ3Y1Is this paper presenting the gamification metrics of the research?
P0.5
N0
QQ4Y1Is there a clear explanation of the result?
P0.5Important >= 9
N0
QQ5Y1Is there a scientific evaluation for the outcome?
P0.5
N0Excellent >= 11
QQ6Y1Is there a useful discussion for recommending further research?
P0.5
N0

Table 4.

ICQ form. Yes (Y) for 1, Partially (P) for 0.5, and No (N) for 0.

QQ1 is estimated with different scores because different research aims reveal different study repeatability. For example, 5-scores are rewarding for the studies’ attempt to improve the gamification outcome. These studies are probably continuous studies with a developed system under a reproducible experience. These articles might therefore be convincing for result reviewing. In contrast, if the research aims to design and test an MI-BCI game, it would be scored two since this game system might need more evidence for reproducible testing. MI-BCI applied for subjects training research will gain 3-scores. That is because a training environment is more challenging for MI-BCI robustness than just a game test. However, the lower level of challenge and number of supportive studies than the ‘improvement’ study positions them in 3 scores. We then divide the quality of articles into four levels: satisfactory, good, important, and excellent.

Advertisement

3. Results

3.1 Study selection

This review finds 2524 articles after using the TAK searching strategy above. The exact numbers of each database and term can be seen in Table 5. After removing the duplication and first reviewing with SC, 549 articles are saved in three plans, and seven more articles are included with two additional searching methods. Sixty-one articles remained after a second review with EC. Three main reasons for high elimination (approximately 90% of the articles) are no gamification, hybrid BCI, and BCI for assessment but not control. After detailed reading with IQC, we finally marked 28 articles. This reduction is due to insufficient reporting data. The whole search process is presented in Figure 1

Plan1Plan2Plan3
Scopus192719357
IEEE6734421
Pubmed82172563

Table 5.

The number of the articles found in 3 plans of stage 1.

Figure 1.

The whole process of searching strategy using PRISMA.

The means of ICQ of 28 articles are 9.3, while the standard deviation is 2.1. These parameters show that the quality of the chosen articles is relatively high, with an average level of ‘important’ with slight fluctuation.

3.2 What games have been used in MI-based BCI?

In this study, we refer to the book written by Ernest [30] to classify different games. The seven main genres are action, strategy, role-playing, real-world simulation, construction and management, adventure, and puzzle games. Additionally, we use subgenres for a better description. This subgenres classification strategy is also agreed by Ernest, who claims the classification change constantly with different circumstances.

Thus, this review develops a subgenre classification for the chosen 28 MI-BCI games depending on three aspects of the graphical interface: ① whether it is the first-person view, ② which visual type (2D, 3D, VR) it is, and ③ what the game content type is. For content-type differentiation, this paper pays strong attention to task-orientate separation, i.e., what targets the gamers are asked to achieve, or what functions they are expected to present. Games with these similar contents would be linked. For instance, one study [31] describes itself as a cue movement game. Nevertheless, the game aims to pick a falling parachutist via a controllable platform. These contents, such as control method, feedback, and mission, are similar to the game [23], whose participants attempt to control a failing ball to land on a fixed basket. Thus, we identify the saving parachutist game as a ball-to-basket (B2B) game but not a cue movement-based searching coins game [32] (Table 6).

Game subgenreCategory
First-personDimensionContentClassic GenreAbbreviation
No2DBall to basketAction2D B2B

Table 6.

An example of game differentiation.

After game classification, four genres with 11 subgenres of MI-BCI games have been identified (see Appendix 1). We present each research group with its summarized game content type in this detailed form. In Appendix 1, the game vision type is shown. For easy understanding, we match both the content and vision of the games with the abbreviation (Ab.).

The four main genres of MI-BCI games are action, puzzle, adventure, and simulation. Table 7 presents the quality of each genre. The following paragraphs explain each genre with its typical subgenre games in order.

GenreTotal scoreTotal studyAverage
Action264199.428571
Puzzle173.579.131579
Adventure16.528.25
Simulation1628

Table 7.

Four genres total score, number of studies and average score.

3.3 Action game genre

Most games (18/28) belong to action games whose players try to control the gaming character to present physical activity, such as jumping or moving to one side, to overcome the challenges and gain rewards. The following paragraphs introduce four main subgenres of action games (2D avoiding obstacles, 2D ball to basket, 3D Cybathlon brain-running, VR first-person action) and other games.

2D avoiding obstacles (2D AO) games (5/18) are a series of games where players use different imaginations to control the character's direction to avoid obstacles and collect rewards. In 2007, one study [33] presented a “Jump and Run” game whose ten healthy players used their adept imagination (right hand, left hand, or feet) to send jump commands to avoid the intermittent obstacles. Two out of five subjects presented a high accuracy of more than 90%. An ‘arcade game’ with similar up and down movement was present with an accuracy of 70% among ten subjects [34]. In 2011, a spaceship control game was provided [35]. Gamers used two out of four motor imageries (left hand, right hand, both feet, or tongue) to control the spaceship to move left or right to avoid falling asteroids. The synchronized game score also rises when the character passes asteroids successfully. Two out of three subjects reported increasing classification accuracy during this gameplay. A similar game is a design where a spaceship is replaced by a car, adding a coin reward [36]. Feet MI was linked with left movement, and right-hand MI could control the car to the right. An average multiple event accuracy (ME_Acc) could reach 78.3% among four participants. One game [37] provided an environment where three classes of MI could be presented. Gamers could move to the left or right by ipsilateral hand imagery to collect coins. A jump command is sent by feet of imagination when the subject has to avoid the snake on the lane. After three gameplay sessions, an increasing average classification rate (53% to 63%) and game score (1600 to 1900) are presented among 14 BCI naïve healthy participants.

2D ball to basket (2D B2B) games (3/18) are games where players use one side of their body image to control a falling ball landing on the ipsilateral side of the basket. In 2003, one article indicated the feasibility of applying this game to 4 paraplegic patients [23]. 3 out of 4 participants succeeded in controlling the falling ball to an exact color basket by using two optimized MI from left hand MI, right hand MI and feet MI. One study in 2010 revealed the feasibility of using this type of game for stroke patients with a moderate classification accuracy of 60%-75% among five novices BCI subjects [2]. This result is similar to a previous study [38] with six healthy users (average classification accuracy of 69.2%). In 2016, nine healthy users were trained to use left or right-hand MI to control a platform (basket demo) to save a falling parachutist (ball demo) [31]. All the accuracy performances are higher than 70%.

3D Cybathlon brain-running (3D CyR) is a game (3/18) designed for the first Cybathlon competition held for a disabled cohort in 2016 in Switzerland [39]. Cybathlon competitors, called pilots, in BCI Race, present four different MIs depending on the areas the role needs to pass through. The result is ranked from the shortest runtime in order. In the MIRAGE team, whose pilot is a post-stroke patient, the average runtime could reduce from 178s to 143s during training and 196s (rank 11/11) in competition [40]. The optimized 2 MI classes are right hand MI, both feet MI, with other two mental subtraction, and auditory imagery. In the same competition, two tetraplegic pilots in the Brain Tweakers team [41] present 90s (rank 1/11) and 123s (rank 2/11) runtimes in the first race. In the final round, these two pilots present 125s (1/8) and 190 (4/8). Their control strategy is a 3-class MI (right hand, left hand, and both feet) combined with resting. Those two groups of pilots are training regularly for months before participating Cybathlon competition. In contrast, one research [42] shows that a healthy cohort trained with only two sessions could reach an accuracy of 68.62% with 2-classes of MI (right hand MI and stomach MI).

VR first-person games (VRFP, 4/18) is a group of action games with visual reality (VR) devices to present a first-person vision gamified feedback. From the players’ view, no realistic but misleading interruption such as a passing-by researcher would occur so that players could fully be engaged in the game. In 2013 [43], 12 healthy subjects with one post-stroke patient accomplished a VR B2B game. Participants tried to use the ipsilateral hand MI to control the virtual hands to the right side for picking the falling basketball. Results show that although the disabled users only trained for one session, the final accuracy could reach 77%, whereas the highest accuracy among two-session healthy users is 70.67%. A similar VR B2B game designed in 2019 with 4-class MI (left hand MI, right hand MI, both feet MI, and resting) presents a mean accuracy of 70% with 10 participants [44]. One game with not only visual but also auditory and haptic feedback, called “NeuRow,” was present in 2016 [45]. 13 healthy users use their left- or right-hand MI to navigate the boat towards the same side. Two high-fidelity arms enhance the MI of users by linking the imagery environment with the real world. The classification score (70.7%) in NeuRow compared with the other six studies with the same classification method indicates that the NeuRow has scored the best. Another VR MI-BCI game designed in 2019 is a simplified shooting game [46]. Nineteen users try to destroy the asteroid (VR DA) with the same side hand MI. After receiving the MI commend, the embodied visual hand would move to switch on the trigger of the weapon to shoot the asteroid. The average peak accuracy among these users could reach 75.84%.

In 2003, one 3D first-person shooting game (3DShT, 1/18) was tested among four healthy participants [24]. Since it is the earliest MI-BCI game, researchers did not limit the users to control the game in a certain fixed MI way. Instead, participants are allowed to discover how to adjust mu rhythm, which is related to sensorimotor control by themselves. Result reveals that mu activity could be actively adjusted to control the scenery to the left or right. Researchers subsequently indicated the potential that mu rhythm becomes a binary signal for MI-BCI control. In 2013, researchers developed BrainArena, a simplified football game (2D FB, 1/18), where two users could play in a collaborative or competitive mode [47]. Eight healthy subjects in 4 pairs show an average classification accuracy of 75.4% (collaborative manner) and 74.6% (competitive manner). In 2018, a 3D ball balance game (3D BL, 1/18) was designed. Ten healthy subjects use left or right-hand MI to control the platform to slightly rotate clockwise or anti-clockwise so that the planet on the platform would not move to the low end and fall off [48]. Final accuracy could reach 70% with the concentration improvement.

3.3.1 Adventure game genre

Adventure games (2/28) are a series of a game whose target is mainly task-orientated exploration. For example, adventure gamers usually need to explore the map with specific targets, such as finding a particular non-player character (NPC) or finishing a specific mission.

We only found one type of adventure game for the MI-BCI system. Scherer et al. [49] designed a 3D first-person searching coins game (3D SC) called “freeSpace” in which two out of three healthy players succeeded in collecting all three coins in three minutes. Three out of four MI classes (right hand MI, left hand MI, tongue MI, foot MI) are used for left, right, and forward navigation to find three hidden coins on the whole map. After a few trials, two participants could see improvement in covered distance and searched coin numbers. In another report of this game experiment with a different angle, research indicates that the whole runtime would be 110s with 100% classification accuracy [27].

3.3.2 Puzzle game genre

Puzzle games (7/28) usually offer enjoyable logic or cognitive training tasks. Gamers need to identify the link among the provided items, answer specific questions or find a way out of a maze in a limited duration.

2D fix route maze games (2D FxM, 2/7) are a group of games whose characters should walk following a pre-designed line. In 2007, a research group redesigned the 2D “Pacman” game to be compatible with MI-BCI users [50]. Users could navigate the Pacman with left hand MI or right-hand MI to finish the instructed route in the maze with the highest score. Additional credit will be rewarded if the user could pick the apple on the lane, and the game score will decline if the Pacman hits the wall. A similar maze game was designed in 2017 [51] where users could use 4-class MI (left hand, right hand, both feet, and tongue MI) to finish the instructed route. The highest gaming accuracy is 48.7% among the four subjects, while the average classification accuracy is 68.5%.

2D Voluntary route maze games (2D VM,3/7) are maze games without a fixed path but only a start point and an endpoint. In 2009, 2 healthy players performed well in a maze-like cursor control game [52]. Gamers are asked to move to the target area and avoid one fixed trap with left- and right-hand MI. One gamer could reach an offline classification accuracy of 73%. This game is then used among one amyotrophic lateral sclerosis (ALS) patient and one primary lateral sclerosis (PLS) [53]. ALS user presents a receiver operating characteristics (ROC) rate of 81.8%, and the ROC rate for PLS user is 86.7% in two visits. A similar game was developed in 2012, and 4-classes MI (left hand MI, right hand MI, both hand rise MI and both hand fist MI) are used for navigation in four directions [54]. Healthy gamers present an accuracy of 60%-70% in this study.

2D Jigsaw puzzle (2D JP) games (2/7) are a group of games where players try to finish collecting the jigsaw by specific tasks. One case study reports that Arm MIs could help play a jigsaw puzzle game [55]. The cerebral palsy (CP) participant uses left or right-arm MI to accomplish tasks and collect puzzles. The result shows that the acquired MI skill could help the participant gain additional MI-BCI skills such as choosing, playing, and pausing videos. Furthermore, the skill could last for more than six weeks, as proven by the subject’s successful control of the MI-BCI virtual cube. An afterward study in 2017 with the same game but a larger number of subjects (eight patients with CP) showed the potential for CP patients to interact with this MI-BCI puzzle game [20]. However, the quantitative result in performance score for all users is relatively negative with several training issues such as head cap discomfort.

3.3.3 Simulation

Simulation (1/28), or real-world simulation, is a group of games, including vehicle driving or particular physical challenges. However, unlike action games, simulation is often more focused on navigation experience and related tasks. In 2019, one VR drone racing game called “Brain eRacing” (VR DR) was reported with 30 healthy participants [56]. However, unlike other games using MI for navigation control, MI is only used to accelerate the drone’s speed. The final average game time is 54.88s, while the group with the last average time is 69.6s.

3.4 What are the metrics of MI-BCI game?

3.4.1 Performance metrics

24 out of 28 articles use quantitative measurement to test the learning outcomes, whereas four use qualitative metrics to report the participants’ performance (see Appendix 2). One article has not tested their result [50]. One study [54] only presents a description indicating that a user who was not an expected volunteer could control the mental task easily in a short duration. We will not consider these two articles in metrics recommendations. Therefore, the following paragraph will indicate the commonly used performance metrics in the qualitative and quantitative groups.

Appendix 2 lists several performance metrics used for learning outcomes measurement (see Appendix 2). Twenty-four articles used quantitative metrics for MI-BCI gaming mode measurement. The two most frequently used metric is accuracy (16/24) and performance score, or game score/rate (5/24). In four articles using qualitative evaluation, the commonly used metric is the game experience questionnaire (GEQ) (2/4). In the discussion part, we will present a detailed introduction and analysis.

3.4.2 Training issue

The existing issues identified in the experiment are reported from the first designed MI-BCI game till now (see Appendix 2). We found six main issues: ① environmental background and distraction (3/28), ② trial length relevant fatigue (4/28), ③ performance variation (4/28), ④ increasing false positive rate (2/28), ⑤ performance decrease (2/28), ⑥ toughness to complete game (2/28). Table 8 below shows the summarization of each issue with its consideration. We gave a detailed explanation in the discussion part.

Training issueFactors/potential solutions need to consider
Environmental distractionBalance between distraction and engagement
FatigueShorten trial length and test in the morning
Performance variationReal-world experimental environment but consider the difficulty
Increasing False Positive rateMethod eliminates the expectation of next MI, and find optimal threshold for better classification
Performance decreaseReducing physical demand
ToughnessReforming environment and consider classification

Table 8.

Summarization of training issue and its considerations.

Advertisement

4. Discussion

4.1 What are the factors that researchers should be focusing on when designing a gamified experiment

4.1.1 Metrics for MI-BCI game mode

Quantitative metrics: Researchers usually identify the learning outcome by comparing the accuracy of each experiment in sequence [43]. The testable and quantitative characteristics keep accuracy as the top used metric. Performance score, or game score/rate, is another commonly used quantitative metric equally ranked with consuming time. The game score could reflect real-time performance intra-trials and quantitative improvement cross trials objectively. One study [37] uses online game scores to motivate BCI users to improve their next performance. Moreover, line graphs for describing performance score fluctuation could be more visible for presenting learning outcomes [55]. The same Auckland group used the score-line curve one year later in another research with ADHD patients [20].

Based on the writers’ experience in the rehabilitation equipment market, the scoring method is more intuitive for patients and therapists to identify the rehabilitation outcome. In contrast, accuracy is more reliable for the technical measurement of outcomes from both users and the system. Therefore, we suggest using accuracy for research aims, such as identifying whether gamification could enhance the learning outcome more than non-game MI-BCI. Synchronized gaming score, in contrast, could be applied to potentially enhance the performance of users or check the rehabilitation stage.

Qualitative metrics: The commonly used qualitative metric is the game experience questionnaire (GEQ) (2/4). GEQ is the response from MI-BCI gamers. Researchers use GEQ for measuring several experiences, such as flow experience. A guideline is recommended to quantify all the qualitative answers and group them into a specific experience [57]. For example, Tezza et al. [26] use a 1–5 Likert scale to evaluate the experience.

4.1.2 Existing issues

Environmental Background and Distraction: Distraction is an environmental or feedback issue that would disturb the users from performing a correct MI. In one of the earliest MI-BCI gaming experiments [24], researchers assumed distraction caused the no-growth low mu rhythm used for game control. In a simplified football game [47], 7 out of 20 subjects argue similar disturbing feedback. One article reports the same disturbing issue [35], but they claim this issue offers more engagement in the game, in contrast. Serious consideration to balance the augmented engagement and distraction, thus, is required when applying the idea of multi-player MI-BCI games.

Trial Length, Experimental Time and Fatigue: Fatigue is another issue that generally happens during MI-BCI training. The length of trials and the time for experiments are linked to fatigue. Significant variability in performance is found in different lengths of trials [23], which might be due to fatigue. In one 2D B2B game [2], researchers discover similar performance variability due to evident fatigue and depression cross-subjects. In detail, two participants complain that the sessions are too long to keep them performing well. Researchers report similar fatigue issues and lack of concentration in one subject whose performance is not ideal after four sessions [36].

Authors from the 2D B2B game research [2] also believe that the chosen time (afternoon) for an experiment is one factor that increases the tiredness providing low performance. One puzzle game case study [55] supports this argument, in which authors suggest a morning experiment to avoid fatigue.

Performance Variation: Performance variation is when outcomes of distinct trials are different. It often happens when different individuals use the same MI-BCI device with all other factors being the same. For example, in one gamified training mode [27], two participants showed improvement in the game performance while one subject showed a decrease in performance.

However, variation would not only be found in cross-subjects but also found in the same subject when they failed to provide reproducible MI effort in each trial. This variation is called intra-subject performance variation. Psychological states, such as nervousness and motivation, correspond with intra-subject performance variation [40]. Another study [31] reports the same issue and claims that it is possible to solve this problem by using adaptive assistance in changing the trial length. Consistency and personalization are two issues causing cross-subject and intra-subject performance variability [42]. Researchers predict that a competitive training environment might help maintain users’ excitement [35]. However, this reviewer would consider whether this environment could have side effects, such as distraction, on learning outcomes in a short time.

The Increasing False Positive Rate: The false-positive rate is the proportion when the classified command is against the imagination of participants. Scherer et al. [27] reveal that the increasing false positive rate might be related to premature brain activation. Researchers also believe this non-stationary brain wave links with the unsatisfactory robustness of the system.

An optimal threshold is also a factor that impacts the false positive rate. The threshold of MI-BCI is the digital criteria to identify whether the amplitude of Electroencephalogram (EEG, one type of brain signal) could be a task-relevant mental task. One study shows the rate declined when adjusting the decision boundary from 0.6 to 0.8 [44].

Performance decrease: Performance decrease is the declining quality of quantitative performance metrics, such as decreased accuracy, ITR, and increased time. One study indicates its link with increasing workload [45]. To solve this, reducing the physical demand is recommended [53]. Furthermore, to keep a good performance, minimum eyes and body movement are required [44].

Toughness in playing games: Toughness is the difficulty for participants to finish a goal in the game. One study [42] argues that though the gaming concentration is satisfied, the controlling difficulty is still visibly raised when the number of MI classes grows. This evidence informs that the number of MI control types impacts toughness, which could then cause performance variability.

4.2 What games have been used in MI based BCI?

4.2.1 Feasibility of MI-BCI gaminification

Appendix 3 indicates the evidence of the feasibility analysis. For research reporting with performance accuracy, the criterion of feasibility is that the average accuracy (accuracy/no. person) could finally reach 70% of accuracy, based on the criteria of BCI application [58, 59]. Although the articles concluded that 233 subjects successfully went through the MI-BCI experiments, only 111 individuals have examined the feasibility of the MI-BCI game with its gaming accuracy.

This review chooses the maximum average accuracy (MAA) for quantified feasibility as one metric. For example, Djamal et al. [34] presented two groups of average accuracy (a non-FFT group with 50% versus an FFT group with 70%). This review picked the 70% FFT group as an MAA since this accuracy is reachable with existing system upgrade technologies, such as FFT. After analysis, 28 chosen articles present an average accuracy of 74.35%, higher than the threshold. Additionally, 26 out of 28 articles (92.86%) possess positive responses to the feasibility of the MI-BCI game. These two results illustrate that it could be feasible to apply gamified MI-BCI training mode for further experiment.

4.2.2 Game recommendation

From the analysis above, it is explicit that 2D AO is the commonly used game type (5/28, 17.86%). The evidence to support this argument is generally high (46.5 total scores in AQ with a mean of 9 and a standard deviation of 2.17). These statistic results advise 2D AO to become a high-priority recommended game. The advantage of the 2D AO game is the high level of motivation. The coming obstacles would increase the tension and stimulate the users to accomplish the correct MI to rescue the character intuitively.

However, except one study uses 3 MI classes (left- and right- hand MI and feet MI), others do not use more than two MI types for gaming navigation. This circumstance is a limitation of current 2D AO games. Future 2D AO game design should cater to real-world complexity by providing more MI class on the one hand and a reliable user experience with guaranteed accuracy on the other hand.

A potential solution is to add more controllable gaming factors in the 2D AO game. For example, forward and backward, acceleration and deceleration, and even shooting a bullet against the obstacle could become controllable by additional MI types more than just left and right hands' imagination. Furthermore, different MI classes could depend on environmental color changes. 3D CyR games apply this idea to reality [40, 41, 42], where competitors produce four MI classes based on the above mission colors.

2D B2B game is ranked equally with 3D CyR in report number (3/28, 10.71%) and similar quality (30.5 total scores in 3D CyR vs. 29.5 in 2D B2B). Like a 2D AO game, game designers should notice the limited MI control classes since all three studies using 2D B2B games only present the learning outcome of two classes of MI (Left and Right hand). The failure in recommending a 2D B2B game is because of its lower level of motivation than a 2D AO game [35].

We advise the 3D CyR game since this game could be not only for training but also for the Cybathlon competition. These applications encourage researchers to further the development of rehabilitation and patients to open out to build relationships and share their life with their competitors from other nations or territories. Nonetheless, an advanced challenge for the CyR game requires an excellent feature selection method to afford the upgraded complex data calibration and classification due to the increasing number of MI classes and highly competitive interruptions.

One limitation in all three games above is the lack of immersion, i.e., players feel difficulty linking their MI with their motor execution (ME) when playing a game. This condition raises challenges for presenting correct MI since no environment could work as humanoid neurofeedback to motivate the gamer to produce the related MI. Therefore, humanoid factors should be a focus when utilizing these three types of games.

First-person VR action games (VR FP) do not need to consider humanoid environment issues. Advantages of VR application are imaginative immersion and a high level of sensation. [56] Karácsony et al. [44] share the comments from participants who played VR B2B games: gamers feel they are using their own hands to catch the falling balls via VR-based MI-BCI game. Therefore, these humanoid advantages could improve the participants' performance [44]. Furthermore, one study reports a high gaming accuracy in the VR B2B game. However, the VR DR game does not reproduce a similar extent of learning outcome. This review believes a lower level of immersion, such as a non-first-person humanoid control environment and no visible link between MI and navigation, causes the game to become relatively distracting. This phenomenon prevents the VR DR game from recommendation.

The interest and the difficulty of the game design are required to consider [27]. Although VR games could reach a high level of learning outcome, the complexity of designing the platform and synchronized interaction with numerous MI-BCI data is not as easy as a 2D PC game. The immersion provided by a VR game might be lower than a 2D AO game if designers sacrifice the quality of a game system to reduce complexity. That is because potential high delay, game bugs, and illogical graphical switches might occur as distractions reducing the MI-BCI experience. A 2D MI-BCI game with good design could also present well in tactical, strategy, or narrative immersion, which are not specific to VR [30]. These facts prevent VR FP action games from becoming the only recommended game.

In summary, if there are temporal and spatial limitations, we advise researchers to choose game content close to 2D avoiding obstacles. The insufficient degree of immersion and the number of MI classes are two main issues that need to consider. 3D Cybathlon running game is another choice for researchers due to its futuristic maturity in competition, stakeholders, and technique. In contrast, we recommend a VR first-person action game with first-person vision when there is no limitation in designing and experimenting with an upgraded gamified MI training system. In particular, the VR ball to the basket game and destroying asteroids game are suggested as two existing first-person VR examples.

Another game genre, puzzle game, is not recommended in this review because motor imagery is still easily disturbed by other factors, as mentioned in both current issues and Appendix 2. A hybrid game with an action factor to motivate correct MI and a puzzle factor to enhance the strategy immersion is still potential for improving learning outcomes of MI-BCI. However, we suggest designing the game navigated by action-dominated factors. For example, the game story could cover puzzle game questions. However, for choosing an answer and continuing the story, the graphical interface of the MI-BCI game could use a humanoid hand for assistance.

4.3 Limitation

No article has discussed the comparison between gaming MI-BCI with a control group. The main reason is that majority of the studies are still focusing on the feasibility of using different feature selection methods in the MI-BCI game, aiming to improve the performance outcome with more advanced techniques. Even though three existing articles [32, 43, 47] claimed the improved learning outcome with comparison tests, their research process is not sufficiently scientific as randomized control trial (RCT). This limitation shows a research gap that could be filled shortly: a randomized control experiment for evaluating whether gamification could improve the learning outcome of MI-BCI is required.

Another limitation is the objectiveness of this review. Since gamification in MI-BCI is an interdisciplinary study covering tremendous academic fields, this review cannot present an in-depth analyzed review and recommendation. This research group hopes to complete a systematic review with video game experts when enough RCTs have tested an optimal training mode.

Furthermore, we advocate more significant attention to users' cohorts. Only 7 out of 28 articles test the feasibility of games in a disabled cohort. One study in this group using a commercial wireless BCI device reports a highly negative result: all participants failed to complete the initially designed trial. None of them shows a learning outcome increase after the full training [55]. This situation absorbs attention on an appropriate task for different users. A relatively short training duration with a more comfortable experiment setting is probably more friendly to patient-participants. Additionally, one study reports the possibility that subjects with video game experience would have a better MI-BCI performance [60]. Gender is also a probable factor related to performance [46]. Therefore, when separating the participants and analyzing the results, these participants-relevant factors should consider.

Real-World Study (RWS) [61] is a research type covering the data collected widely and randomly. Researchers pick the evidence (Real World Evidence) without strictly classifying one controlled particular pattern. For MI-BCI, existing data is often gained from healthy cohorts or individuals with specific neurological impairments. These data are not sufficient for testing the robustness and reliability of MI-BCI for real-world users and applications. More influential factors would occur than subjects whose patterns are controlled in an RCT. For applying MI-BCI to these real-world users with a reliable outcome, this review recommends having an additional RWS after a sufficient number of RCT studies show positive results.

4.4 Insisting pure MI-BCI control strategy in game

This review studied gaming BCI systems controlled by only MI. Plenty of research has tested the feasibility of hybrid BCI control [62] with games. One article successfully used a hybrid technique with MI to control a well-known but highly complex game, World of Warcraft, surprisingly seven years before [63]. Although these combination control methods might accomplish more remarkable improvement than mere MI control, this review still insists on the importance of exploring the maximum outcome of pure MI.

The reason for insisting on single MI control is that the source of performance issues still needs to be researched. On the one hand, well-direct optimization of the pure MI control could efficiently improve the hybrid control system. This review is not denying the better performance that hybrid MI devices could achieve. In contrast, after digging out and solving the problem caused by MI-related factors, a combination of other methods might have higher performance growth in another dimension.

On the other hand, even if a hybrid MI system has been designed, the outcome would still not be ideal if we have not sufficiently explored and solved the potential performing issue. Like a disease, low MI performance is a symptom of MI-BCI, and unsuitable MI design is one of the root causes. Using merely hybrid MI control seems like treating the low-performance symptom, but not the root causes. This assumption could be supported by one study [62] that participants still need to be trained for a long duration with a hybrid control system that has been applied in their Tetris game. In contrast, discovering suitable MI-BCI gamification to help users is a metaphor for treatment for the root cause.

As a product manager in Fourier Intelligence, our regular marketing survey shows that patients requiring long training duration would still get bored of the game provided by our rehabilitation equipment. However, a number of games have been included in our game library for different rehabilitation equipment. Therefore, providing MI-BCI games to patients requiring several weeks or even months of training would still challenge keeping patients engaged. Designing other MI-BCI games for satisfying patients might not be a long-term solution (Figure 2).

Figure 2.

An example of training game in game library.

4.5 Future work

This literature review presents several research gaps. For achieving the target of using an RCT and an existing game to test the improvement of MI-BCI gamification in a short time, we advise designing a 2D AO game first. A simulation could be done first to test the feasibility of this theoretical recommendation via OpenVibe software. Suppose the classification accuracy is higher than 70%. In that case, the participant-based study will try the designed game training model, the non-game control group model, and the third-party outcome testing mode. Also, if no significant variation can be observed between the game and control groups, the game type will be optimized. For example, a 2D first-person humanoid game might replace the 2D AO game to identify whether the negative result is due to the lack of a humanoid environment. The argument claiming gamification would not improve learning outcomes would be reported if even a VR first-person humanoid game could not present a significant difference from its control group.

Advertisement

5. Conclusion

Presently, researchers and clinicians expect to apply the combination of MI and BCI to a great range of users for rehabilitation purposes. Thus, existing challenges in current MI-BCI works must be solved, such as training issues. This review has utilized the majority of SR to present a scientific and objective review of one potential solution, gamification, to optimize MI-BCI training outcomes. Our finding through evidence (28 articles, 111 individuals with accuracy test, 74.35% average accuracy, 26 out of 28 positive responses) shows that using the MI-BCI game for training could be feasible.

The result also recommends 2D avoiding obstacles game, 3D Cybathlon game, and VR humanoid first-person action game as prior MI-BCI games for further research. Moreover, this review maps the cautions of the current issues of gamified MI-BCI training mode and the potential methods to overcome them. The literature review is predicted to provide a new vision of the appropriate gamified MI-BCI training modes and what elements, such as metrics and issues, are suggested to consider for scientific research.

Advertisement

StudyGameCategory
ICQAuthorYearFirst-personDimensionContentClassic GenreAbbreviation
11Pineda et al. [24]2003Yes3DShooting gameAction3D ShT
9.5Graz(Krausz et al.) [23]2003N/A2DBall to basketAction2D B2B
7.5Graz(Müller-Putz et al.) [33]2007N/A2DJump obstaclesAction2D AO
6Berlin(Krepki et al.) [50]2007N/A2DPac-manPuzzle2D FxM
6.5Graz(Scherer et al.) [49]2007Yes3DSearching coinsAdventure3D SC
10Graz(Scherer et al.) [27]2008Yes3DSearching coinsAdventure3D SC
6.5USA(Huang et al.) [52]2009N/A2DSokobanPuzzle2D VM
7.5USA(Ou Bai et al.) [53]2010N/A2DSokobanPuzzle2D VM
8.5Prasad et al. [2]2010N/A2DBall to basketAction2D B2B
11Coyle et al. [35]2011N/A2DAvoiding obstaclesAction2D AO
9Bordoloi et al. [54]2012N/A2DMaze gamePuzzle2D VM
10Anopas et al. [43]2013YesVRBall picking gameActionVR FP
12Bonnet et al. [47]2013N/A2DFootball gameAction2D FB
11Asensio-Cubero et al. [37]2016N/A2DRunning gameAction2D AO
11.5Kreilinger et al. [36]2016N/A2DCar gameAction2D AO
11.5Switzerland(Saeedi et al.) [26]2016N/A2DBall pickingAction2D B2B
11Auckland (Taherian et al.) [55]2016N/A2D+audioPuzzlePuzzle2D JP
8Vourvopoulos et al. [45]2016YesVR+vibrationRowing game(NeuRow)ActionVR FP
5.5Djamal et al. [34]2017N/A2DArcade avoides obstaclesAction2D AO
8Graz (Statthaler et al.) [40]2017N/A3DBrain running(Cybathlon)Action3D CyR
9Auckland (Taherian et al.) [23]2017N/A2DPuzzle gamePuzzle2D JP
11Zhou et al. [42]2017N/A2DMaze gamePuzzle2D FxM
12Switzerland (Perdikis et al.) [41]2018N/A3DBrain running(Cybathlon)Action3D CyR
10.5Ponferrada et al. [42]2018N/A3DBrain running(Cybathlon)Action3D CyR
7Yang et al. [48]2018N/A3DBalance gameAction3D BL
11Karácsony et al. [44]2019YesVRCatching&kicking falling itemsActionVR FP
12Škola et al. [46]2019YesVRDestroying asteroidsActionVR FP
6.5Tezza et al. [56]2019N/AVRDrone competitionSimulationVR DR

AuthorYearPerformance metricPerformance Issue
Pineda et al. [24]2003Low / high mu power changes learning curve (10H in total)Non-increase of low mu might be due to feedback distraction.
Krausz et al. [23]2003Information transfer rate(ITR)Significant variability in ITR with different trial lengths might be because MI and attention effort cannot be kept at the same level over the session.
Müller-Putz et al. [33]2007Accuracy
Krepki et al. [50]2007Not test yet
Scherer et al. [49]2007Time for finish mission(maximum time 3 mins)
Scherer et al.[27]2008Time for finish mission(maximum time 4 mins), AccuracyTraining: the expectation of the next cue to come might change the brain activity and produce False-positive command; although a higher detection threshold could reduce the false positive rate, motivation would also be decreased
Huang et al. [52]2009Accuracy
Ou Bai et al. [53]2010AccuracyVividness: participants reported difficulty imagining wrist extension; it might be improved by VMIQ (vividness of movement imagery questionnaire) or by teaching efficient motor imagery.
Prasad et al. [2]2010Qualitative1+AccuracyFatigue and Depression: a higher level of fatigue can contribute to more considerable variability in the BCI performance among subjects; two find treatment sessions are excessively lengthy and tiring (mainly in the afternoon); subjects want the game to be more exciting and challenging.
Coyle et al. [35]2011AccuracyChallenging is due to background distraction, but this is also engaging
Bordoloi et al. [54]2012Only a description
Anopas et al. [43]2013Accuracy
Bonnet et al.[47]2013AccuracyFeedback disturbing 7/20; purely informative 8/20; positive feeling 5/20; 4/8 users find multiuser feedback helpful, 3/8 find they were disturbed. None prefers single-user conditions among the best players.
Asensio-Cubero et al. [37]2016Accuracy, Kappa, and game scoreHigh accuracy and less amount of analyzing data could be achieved by the best basis selection method; The difference between features in the calibration stage and game stage could be identified (this difference could be due to the mental state such as frustration or stress)
Kreilinger et al. [36]2016Success rate, error rateFatigue and lack of concentration is found after 4 runs in one user (S04)
Saeedi et al. [31]2016Accuracy, success ratePerformance variation in different trials for the same subject (intra-subject performance variation). Solved by adaptive assistance (that is, a user-dependent time out in a single trial)
Taherian et al. [55]2016Game score learning curveMotivation, fatigue, and concentration influence performance. Subjects bored with a puzzle game loss are interesting. Performance decrease might be due to fatigue, increased concentration, and curiosity to identify new videos. The workload might need to be decreased by some methods such as meditation practice. Recommendation: BCI training needs to show users application of their learned skills from early on to increase a sense of self-efficacy and confidence; experiment should be done in the morning due to fatigue; also, a small office for less distraction.
Vourvopoulos et al.[45]2016Qualitative2Low physical demand increases effort and good classification performance. Increasing workload causes worse performance.
Djamal et al. [34]2017Accuracy, time for finish mission
Statthaler et al.[40]2017Running timeFeature distribution had changed considerably between training and the game. This distribution might be due to a long time of rest (45min gap between sessions) and nervousness in competition (different heart rates). There are limited BCI systems in new environments; the pilot's performance fluctuations might be due to intra-subject performance variation, and intra-subject performance variation is related to psychological states such as motivation. Races with human competitors and a sizeable audience in training help the pilot himself to better prepare.
Taherian et al. [20]2017Game scoreThe unique head shape prevents subject three from using BCI; linear reduction could be seen in subject one since he has interval illness during the training and visual impairment, making him unable to focus on the screen. Subject2 feels discomfort when researchers touch his head; subject 5 is interested in doing it but is upset about it at home. Subject 7 feels frustrating in playing puzzles with sometimes no reaction; all 6 participants found EPOC uncomfortable wearing for more than 15 mins; EPOC cannot be used among patients with head support. Hearing the auditory feedback could improve their performances (comment from a special educator).
Zhou et al. [51]2017Accuracy
Perdikis et al.[41]2018AccuracyUnsatisfactory robustness might be related to non-stationary brain signals; longitudinal mutual learning could help increase robustness.
Ponferrada et al. [42]2018AccuracyThree commends control is significantly more complicated than two commends. Although enough concentration could be provided, high variability could be seen across different trial subjects: consistency and personalization are significant challenges. An ideal solution is to run multiple sessions to identify which motor imageries each subject can best control.
Yang et al. [48]2018Concentration(TBR,SMR)Portability is required
Karácsony et al.[44]2019Accuracy, game scoreThe 0.6 decision boundary (low activation threshold) causes high false positives (after increasing to 0.8, the false positive is reduced). For good performance, subjects are asked to minimise their eye and head movements; immersiveness in the game could increase the deep engagement.
Škola et al. [46]2019Accuracy + Qualitative3Females perform better than males on average (in only the first run). Fatigue is higher in BCI-naïve subjects in the first run than inexperienced but not replicated in any remaining runs. People are alert and motivated sufficiently to improve their results by fast but not long MI. No evident fatigue influencing the performance has been confirmed.
Tezza et al. [56]2019Qualitative4

AuthorYearNo.ParticipantsFinal AccuracyFinal recommendation
Pineda et al. [24]20034positive
Graz (Krausz et al.) [23]20034positive
Graz (Müller-Putz et al.) [33]20071065%,72%,51%,70%,70%(5 subjects finally tested)possible
Berlin (Krepki et al.) [50]20071positive
Graz (Scherer et al.) [49]20073positive
Graz (Scherer et al.) [27]2008383%, 88%, 80%positive
USA (Huang et al.) [52]2009573%, 59.2%(2 subjects finally tested)positive
USA (Ou Bai et al.) [53]2010681.1%, 86.7%(2 subjects finally tested)positive
Prasad et al. [2]2010570%(average acc for 5 subjects)positive
Coyle et al. [35]2011392.6%, 75.3%, 79.8%positive
Bordoloi et al. [54]201214positive
Anopas et al. [43]20131270%,72%,70%(3 subjects finally tested)positive
Bonnet et al. [47]20132073.94%(average acc for 20 subjects)positive
Asensio-Cubero et al. [37]20161463%(average acc for 14 subjects)positive
Kreilinger et al. [36]201610positive
Switzerland (Saeedi et al.) [31]2016985%,88%,91%,92%,89%,75%,82%,75%,96%positive
Auckland (Taherian et al.) [55]20161positive
Vourvopoulos et al. [45]201613positive
Djamal et al. [34]20171070%(average acc for 10 subjects)positive
Graz (Statthaler et al.) [40]20171positive
Auckland (Taherian et al.) [20]20176negative
Zhou et al. [51]2017445%(average acc for 4 subjects)negative
Switzerland (Perdikis et al.) [41]2018293.8%,96.8%positive
Ponferrada et al. [42]2018268.62%(one subject finally tested)positive
Yang et al. [48]201810positive
Karácsony et al. [44]201910100%(average acc for 10 subjects)positive
Škola et al. [46]20191975.84%(average acc for 18 subjects)positive
Tezza et al. [56]201930positive

References

  1. 1. “Disability and health,” Who.int, 2021. [Online]. Available from: https://www.who.int/news-room/fact-sheets/detail/disability-and-health [Accessed: 10 July, 2020]
  2. 2. Hettiarachch I, Nguyen T, Nahavandi S. Motor Imagery Data Classification for BCI Application Using Wavelet Packet Feature Extraction. Cham: Springer International Publishing; 2014. pp. 519-526
  3. 3. Langhorne P, Wu O, Rodgers H, Ashburn A, Bernhardt J. A Very Early Rehabilitation Trial after stroke (AVERT): A Phase III, multicentre, randomised controlled trial. Health Technology Assessment. 2017;21(54):1-120
  4. 4. Smith E. Locked-in syndrome. BMJ. 2005;330:7488
  5. 5. Malouin F, Jackson P, Richards C. Towards the integration of mental practice in rehabilitation programs. A critical review. Frontiers in Human Neuroscience. 2013;7(536):1-20
  6. 6. Guillot A, Genevois C, Desliens S, Saieb S, Rogowski I. Motor imagery and placebo-racket effects in tennis serve performance. Psychology of Sport and Exercise. 2012;13(5):533-540
  7. 7. Zhang L, Pi Y, Zhu H, Shen C, Zhang J, Wu Y. Motor experience with a sport-specific implement affects motor imagery. PeerJ. 2018;6:e4687
  8. 8. Napolitano S, Rosa R, Cusano P. The analysis of technical movement of the shooting in water polo. Sport Science. 2019;12:93-98
  9. 9. Lee D, Jang C, Park H. Neurofeedback learning for mental practice rather than repetitive practice improves neural pattern consistency and functional network efficiency in the subsequent mental motor execution. NeuroImage. 2019;188:680-693
  10. 10. Nicholson V, Keogh J, Low Choy N. Can a single session of motor imagery promote motor learning of locomotion in older adults? A randomized controlled trial. Clinical Interventions in Aging. 2018;13:713-722
  11. 11. Bhattacharyya S, Konar A, Tibarewala D. Motor imagery and error related potential induced position control of a robotic arm. IEEE/CAA Journal of Automatica Sinica. 2017;4(4):639-650
  12. 12. Prasad G, Herman P, Coyle D, McDonough S, Crosbie J. Applying a brain-computer interface to support motor imagery practice in people with stroke for upper limb recovery: A feasibility study. Journal of NeuroEngineering and Rehabilitation. 2010;7(1):1-17
  13. 13. Aggarwal S, Chugh N. Signal processing techniques for motor imagery brain computer interface: A review. Array. 2019;1-2:100003
  14. 14. Choi J, Kim K, Jeong J, Kim L, Lee S, Kim H. Developing a motor imagery-based real-time asynchronous hybrid BCI controller for a lower-limb exoskeleton. Sensors. 2020;20(24):7309
  15. 15. Dobkin B et al. Methods for a randomized trial of weight-supported treadmill training versus conventional training for walking during inpatient rehabilitation after incomplete traumatic spinal cord injury. Neurorehabilitation and Neural Repair. 2003;17(3):153-167
  16. 16. Pfurtscheller G, Flotzinger D, Kalcher J. Brain-Computer Interface—A new communication device for handicapped persons. Journal of Microcomputer Applications. 1993;16(3):293-299
  17. 17. Hwang H, Kim S, Choi S, Im C. EEG-based brain-computer interfaces: A thorough literature survey. International Journal of Human Computer Interaction. 2013;29(12):814-826
  18. 18. Vourvopoulos A, Liarokapis F, Chen M. The effect of prior gaming experience in motor imagery training for brain-computer interfaces: A Pilot Study. In: VS-Games 2015 –7th International Conference on Games and Virtual Worlds for Serious Applications. Skovde, Sweden: IEEE; 2015
  19. 19. Pfurtscheller G, Neuper C. Motor imagery and direct brain-computer communication. Proceedings of the IEEE. 2001;89(7):1123-1134
  20. 20. Taherian S, Selitskiy D, Pau J, Claire Davies T. Are we there yet? Evaluating commercial grade brain–computer interface for control of computer applications by individuals with cerebral palsy. Disability and Rehabilitation: Assistive Technology. 2017;12(2):165-174
  21. 21. Kapp K, Blair L, Mesch R. The Gamification of Learning and Instruction Fieldbook: Ideas into Practice. 1st ed. Pfeiffer; 2013. pp. 13-67
  22. 22. Peterson M. Computer Games and Language Learning. Basingstoke: Palgrave Macmillan; 2016
  23. 23. Krausz G, Scherer R, Korisek G, Pfurtscheller G. Critical decision-speed and information transfer in the Graz Brain-Computer Interface. Applied Psychophysiology and Biofeedback. 2003;28:233-240
  24. 24. Pineda J, Silverman D, Vankov A, Hestenes J. Learning to control brain rhythms: Making a brain-computer interface possible. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2003;11(2):181-184
  25. 25. Vasiljevic G, de Miranda L. Brain–computer interface games based on consumer-grade EEG devices: A systematic literature review. International Journal of Human Computer Interaction. 2020;36(2):105-142
  26. 26. Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. BMJ. 2009;339(211):b2535-b2535
  27. 27. Ahn M, Lee M, Choi J, Jun S. A review of brain-computer interface games and an opinion survey from researchers, developers and users. Sensors. 2014;14(8):14601-14633
  28. 28. Ahn M, Lee M, Choi J, Jun S. Brain-computer interface speller using hybrid P300 and motor imagery signals. Sensors. 2014;14:14601
  29. 29. Clerc M, Bougrain L, Lotte F. Brain-computer interfaces 1. In: Foundations and Methods. Hoboken, New Jersey: ISTE Ltd/John Wiley and Sons Inc; 2016. pp. xvi-xvii
  30. 30. Adams E. Fundamentals of Game Design. Second ed. New Riders. 2010. pp. 26-80
  31. 31. Saeedi S, Chavarriaga R, Leeb R, Millan J. Adaptive assistance for brain-computer interfaces by online prediction of command reliability. IEEE Computational Intelligence Magazine. 2016;11(1):32-39
  32. 32. Scherer R, Lee F, Schlogl A, Leeb R, Bischof H, Pfurtscheller G. Toward self-paced brain–computer communication: Navigation through virtual worlds. IEEE Transactions on Biomedical Engineering. 2008;55(2):675-682
  33. 33. Müller-Putz G, Scherer R, Pfurtscheller G. Game-like Training to Learn Single Switch Operated Neuroprosthetic Control. Graz University of Technology; 2007. Available from: elsevier.com
  34. 34. Djamal EC, Abdullah MY, Renaldi F. Brain computer interface game controlling using fast fourier transform and learning vector quantization. JTEC. 2017;9(2-5):71-74
  35. 35. Coyle D, Garcia J, Satti AR, McGinnity TM. EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game. In: 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB). 2011. pp. 1-7
  36. 36. Kreilinger A, Hiebel H, Muller-Putz G. Single versus multiple events error potential detection in a BCI-controlled car game with continuous and discrete feedback. IEEE Transactions on Biomedical Engineering. 2016;63(3):519-529
  37. 37. Asensio-Cubero J, Gan J, Palaniappan R. Multiresolution analysis over graphs for a motor imagery based online BCI game. Computers in Biology and Medicine. 2016;68:21-26
  38. 38. Herman P, Prasad G, Mcginnity TM. Design and on-line evaluation of type-2 fuzzy logic system-based framework for handling uncertainties in BCI classification. In: Conference proceedings:. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. 2008. pp. 4242-4245
  39. 39. Zurich E. CYBATHLON—bewegt Mensch und Technik. 2021. [Online]. Available from: http://www.cybathlon.ethz.ch/ [Accessed: July 10, 2020]
  40. 40. Statthaler K et al. Cybathlon Cybathlon experiences of the Graz BCI racing team discipline. Journal of NeuroEngineering and Rehabilitation. 2017;14(1):129, 1-16
  41. 41. Perdikis S, Tonin L, Saeedi S, Schneider C, Millán J. The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users. PLOS Biology. 2018;16(5):e2003787
  42. 42. Ponferrada E, Sylaidi A, Aldo Faisal A. Data-efficient motor imagery decoding in real-time for the cybathlon brain-computer interface race. In: NEUROTECHNIX 2018—Proceedings of the 6th International Congress on Neurotechnology, Electronics and Informatics. 2018. pp. 21-32
  43. 43. Anopas D, Horapong M, Wongsawat Y. BCI-based neurorehabilitation and prediction system for stroke patients. In: i-CREATe 2013—International Convention on Rehabilitation Engineering and Assistive Technology Conjunction with SENDEX 2013. 2013
  44. 44. Karácsony T, Hansen J, Iversen H, Puthusserypady S. Brain computer interface for neuro-rehabilitation with deep learning classification and virtual reality feedback. In: ACM International Conference Proceeding Series. 2019
  45. 45. Vourvopoulos A, Ferreira A, Badia S. NeuRow: An immersive VR environment for motor-imagery training with the use of Brain-Computer Interfaces and vibrotactile feedback. In: Proceedings of the 3rd International Conference on Physiological Computing Systems. 2016. pp. 43-54
  46. 46. Škola F, Tinková S, Liarokapis F. Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment. Frontiers in Human Neuroscience. 2019;13(329):1-16
  47. 47. Bonnet L, Lotte F, Lecuyer A. Two brains, one game: Design and evaluation of a multiuser BCI video game based on motor imagery. IEEE Transactions on Computational Intelligence and AI in Games. 2013;5(2):185-198
  48. 48. Yang C, Ye Y, Li X, Wang R. Development of a neuro-feedback game based on motor imagery EEG. Multimedia Tools and Applications. 2018;77(12):15929-15949
  49. 49. Scherer R, Schloegl A, Lee F, Bischof H, Janša J, Pfurtscheller G. The self-paced Graz brain-computer interface: Methods and applications. Computational Intelligence and Neuroscience. 2007;79826:1-9
  50. 50. Krepki R, Blankertz B, Curio G, Müller K. The Berlin Brain-Computer Interface (BBCI)—Towards a new communication channel for online control in gaming applications. Multimedia Tools and Applications. 2007;33(1):73-90
  51. 51. Zhou T, Chen A, Chen K. A brain computer interface based on motor imagery for maze game. In: 2017 10th International Symposium on Computational Intelligence and Design (ISCID). 2017. pp. 398-401
  52. 52. Huang D, Lin P, Fei D, Chen X, Bai O. EEG-based online two-dimensional cursor control. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2009. pp. 4547-4550
  53. 53. Bai O, Lin P, Huang D, Fei D, Floeter M. Towards a user-friendly brain–computer interface: Initial tests in ALS and PLS patients. Clinical Neurophysiology. 2010;121(8):1293-1303
  54. 54. Bordoloi S, Sharmah U, Hazarika SM. Motor imagery based BCI for a maze game. In: 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI). 2012. pp. 1-6
  55. 55. Taherian S, Selitskiy D, Pau J, Davies T, Owens R. Training to use a commercial brain-computer interface as access technology: A case study. Disability and Rehabilitation: Assistive Technology. 2016;11(4):345-350
  56. 56. Tezza D, Garcia S, Hossain T, Andujar M. Brain eRacing: An Exploratory Study on Virtual Brain-Controlled Drones. In: Chen J, Fragomeni G, editors. Virtual, Augmented and Mixed Reality. Applications and Case Studies. HCII 2019. Lecture Notes in Computer Science(). Vol. 11575. Cham: Springer; 2019. DOI: 10.1007/978-3-030-21565-1_10
  57. 57. Norman KL. GEQ (Game Engagement/Experience Questionnaire): A review of two papers. Compute. 2013;25(4):278-283
  58. 58. Kübler A, Neumann N, Kaiser J, Kotchoubey B, Hinterberger T, Birbaumer N. Brain-computer communication: Self-regulation of slow cortical potentials for verbal communication. Archives of Physical Medicine and Rehabilitation. 2001;82(11):1533-1539
  59. 59. Pfurtscheller G, Neuper C, Birbaumer N. Human brain-computer interface. In: International Conference of the IEEE Engineering in Medicine & Biology Society. 2005
  60. 60. Vourvopoulos A, Bermudez S, Badia I, Liarokapis F. EEG correlates of video game experience and user profile in motor-imagery-based brain–computer interaction. The Visual Computer. 2017;33(4):533-546
  61. 61. FDA. Real-World Evidence to Support Regulatory Decision-Making for Devices. U.S. Food and Drug Administration. 2021. [Online]. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-world-evidence-support-regulatory-decision-making-medical-devices [Accessed: 15 January 2021]
  62. 62. Wang Z, Yu Y, Xu M, Liu Y, Yin E, Zhou Z. Towards a hybrid BCI gaming paradigm based on motor imagery and SSVEP. International Journal of Human Computer Interaction. 2019;35(3):197-205
  63. 63. Scherer R et al. Brain–computer interfacing: more than the sum of its parts. Soft Computing. 2013;17(2):317-331

Written By

Chongli Gao, Mingyue Xia, Zhehao Zhang, Yongpeng Han and Yaxin Gu

Submitted: 08 May 2022 Reviewed: 06 June 2022 Published: 13 October 2022