Improving the Brain-Computer Interface Learning Process with Gamification in Motor Imagery: A Review

Abstract

Brain-computer-interface-based motor imagery (MI-BCI), a control method for transferring the imagination of motor behavior to computer-based commands, could positively impact neural functions. With the safety guaranteed by non-invasive BCI devices, this method has the potential to enhance rehabilitation and physical outcomes. Therefore, this MI-BCI control strategy has been highly researched. However, applying a non-invasive MI-BCI to real life is still not ideal. One of the main reasons is the monotonous training procedure. Although researchers have reviewed optimized signal processing methods, no suggestion is found in training feedback design. The authors believe that enhancing the engagement interface via gamification presents a potential method that could increase the MI-BCI outcome. After analyzing 2524 articles (from 2001 to 2020), 28 pieces of research are finally used to evaluate the feasibility of using gamified MI-BCI system for training. This paper claims that gamification is feasible for MI-BCI training with an average accuracy of 74.35% among 111 individuals and positive reports from 26 out of 28 studies. Furthermore, this literature review suggests more emphasis should be on immersive and humanoid design for a gaming system, which could support relieving distraction, stimulate correct MI and improve learning outcomes. Interruptive training issues such as disturbing graphical interface design and potential solutions have also been presented for further research.

1. Introduction

World Health Organization (WHO) claims that over 1 billion people suffer types of disability worldwide [1]. Neurological disease is accepted as one of the reasons causing disability. For instance, stroke attacks more than 20 million people each year and causes 45% of the total to become permanent upper-body disabled [2], without mentioning a significant number suffer from lower limb disability or temporary limited mobility. Although there have been several advanced therapies for neurological rehabilitation, the range of compatible users and the rehabilitation efficiency are still limited. For example, in 2017, Langhorne et al. [3] applied a very early rehabilitation trial (AREAT) among patients who suffered a stroke after 24 hours. However, in the inclusion criteria, patients must have at least a capacity to react with verbal commands. Patients with lock-in syndrome [4] would discover it tough to gain such interventions since injury risks might occur if patients could not timely express whether their bodies are suitable for the training intensity or not.

Brain-computer interface based motor imagery (MI-BCI) has the potential to unlock the potential interaction between these patients with their environment. Motor imagery (MI), imagining kinaesthetic movements of parts of the body [5], is widely suggested to utilize as a training method to improve physical capacities and rehabilitation outcomes. Articles claim that MI training could enhance physical performances in tennis [6], basketball [7], and Water polo [8]. Furthermore, this therapy could also positively affect functional network efficiency [9] and motor learning of locomotion [10]. Brain-computer interface (BCI), also called Brain-Machine Interface (BMI), is an advanced technology that plays an intermediary role in sharing information between a human’s brain and external devices. With a non-invasive BCI device (wearable head cap with sensors attached for signal monitoring, rather than invasively inserting sensors into the skull), users could safely modulate their brain activity to interact with external devices, such as a robotic arm [11]. The interaction actively navigated by a human is called active BCI mode. This novel control strategy, combined with motor imagery [2, 12, 13], would widen the range of potential beneficiaries of assistive or rehabilitative robotics, such as diagnosed lock-in patients. For example, using MI-BCI for exoskeleton navigation would help lock-in patients to have accessibility for walking assistance by mere imagination [14]. Combining with this principle of robotic therapy, such as residual cortical and subcortical neuronal group facilitation [15], a more significant number of patients can gain positive outcomes.

Dating back to one classical piece of research in 1993, one of the earliest MI-based BCI applications, called motor planning at that moment, was developed and reported at the University Technology of Graz [16]. Afterward, an increasing number of research groups began to focus on potential MI-BCI applications.

1.1 Challenges

Although MI-BCI has been given great attention compared to other BCI applications [17], the gradual reduction in research interest hints at the bottleneck that academia is encountering in MI development. One of the main reasons, claimed by Hwang et al. [17], is that other BCI paradigms could have a shorter training duration and higher information transfer rates (ITR) than MI-BCI. The performance issue mentioned above is not an exceptional case. Several errors are repeatedly diagnosed when applying non-invasive MI-BCI for further research with humans. For instance, one BCI group [18] identifies that distraction around the environment might cause insufficient attention failing to output accurate results and promoting frustration. Therefore, the MI-BCI control system in users learning outcomes seems less competitive than its counterparts.

The most commonly used MI-BCI training modes [19] might not help overcome those challenges above since participants would hardly engage with the graphical interface design [20]. Thus, a new solution for improving the learning outcome of MI-BCI users is demanded.

1.2 Potential solution

Gameplay-based learning, gamification, is a potential method to level up the non-invasive MI-BCI learning outcome. The advantages of gamification in training are the upgraded level of users’ engagement and happiness, thanks to its real-world simulation and exciting game content [21]. These might prolong the users’ time of willingness to engage in the therapeutic MI-BCI activity. Using gamification for training is also supported by a consensus [22] that playing a PC game has motivations and benefits for users to learn skills. These pieces of evidence increase the possibility of using gamification in MI-BCI.

In 2003, the Graz group designed one of the initial motors imagery-controlled BCI games. Users could imagine their left or right-hand movement to navigate the falling ball inside the screen to the same side basket to finish this gamified task [23]. The interaction between users and the game could teach the participants to adjust to performing MI correctly. Almost at the same time, a 3D first-person shooting game (3D ShT) is created to declare the feasibility of binary gaming MI-BCI control [24]. These findings support the technical feasibility of gaming solutions in MI-BCI.

1.3 Aim of the study

Although pieces of gamified MI-BCI research have published, no literature has mainly and specifically analyzed the existing gamified MI-BCI training methods. This research gap reveals the essence of accomplishing one literature review for the above purpose. Thus, this paper will review the current non-invasive MI-BCI game in academia. We expect the result could help identify the feasibility of existing MI-BCI game development. This finding probably helps optimize the future research-based game design in testing whether gamified MI-BCI training mode would have a better outcome than the traditional mode.

1.4 Research question

This paper designs research questions (RQ) and a search plan using a topic-relevant review for the template [25]. One main RQ is: whether gamification of MI-BCI is feasible as a training method. Specifically, this review answers 2 RQs presented below (see Table 1).

Research question	Note
RQ1: What games have been used in MI based BCI?; What is the genre that each game belongs to? What type of subject (healthy or disabled) has participated in each game testing? What are the learning outcomes of each game study?	Hybrid BCI would not be included
RQ2: What are the factors that researchers should be focusing on when designing a gamified experiment.; What are the main metrics that have been used What are the training issues that have been reported	This research presents main quantitative and qualitative metrics.

Table 1.

Two RQs with their detailed aspects of questions.

1.5 Outline

The organization of this review is as follows: The research technique used for reviewing and analyzing literature is presented in Section 2. The findings will be arranged in Section 3, followed by a discussion that focuses on a recommendation of MI-BCI game preferences in Section 4, ended with a conclusion.

2. Method

An MI-BCI review could provide instructions for better game design in the future. We finished a pre-literature review to ensure the scheduled literature review method is achievable. After pre-reviewing, this paper believes that although existing MI-BCI games are various, the research fields regarding what MI-BCI games have been used and what game genres are more preferred for MI-BCI training than others are still virgin land. Thus, a systematic review (SR) based selection strategy could be used in this review to present data objectively. However, a meta-analysis is challenging because of the limited number of published articles and reported trials. Thus, the following part will introduce how this review attempts to utilize PRISMA [26] for scientific analysis.

2.1 Eligibility criteria

Out of variables controlling, this review merely focuses on studies asking subjects to use MI to control a BCI game. In detail, we select experiments with all healthy and disabled human participants using MI solely to control a BCI device to play a game (actively non-invasive MI-BCI). The BCI device must be non-invasive. Additionally, metrics cover both quantitative and qualitative learning outcomes. A Hybrid MI-BCI control system, BCI for assessing MI function, or no human experiment would exclude. Summarization of all the critical aspects of the eligibility criteria is presented in Table 2.

Aspect in the criteria	Eligibility
Control method	Using only motor imagery to control a BCI game
BCI device type (invasive?)	Non-invasive BCI
BCI device type (passive, reactive, active?)	Active
BCI learning outcome	Quantitative and qualitative outcome
BCI experiment type	Human experiments only
Participant type	All healthy and disabled cohort

Table 2.

Summarization of eligibility criteria.

2.2 Information sources and searching plan

We apply three searching databases: IEEE Xplore, Scopus, and PubMed/MEDLINE. This review includes three strategies (see Table 3) with two additional plans for finding articles. The pre-searching plan is designed relatively broad so that a brief idea of the current gaming MI-BCI systems could help revise the main-researching plan. In contrast, the main-searching plan is designed relatively specifically after referring to one current published topic-relevant SR [25] and the pre-searching result. We design a compensatory plan so that targeted articles with a slightly expressive difference are unlikely missed. One additional search plan utilizes Google scholar, and another technique is to dig topic-related articles presented as references in the articles already reviewed. The key searching terms are separated into three groups based on the MESH library in Pubmed, template SR [25], and another relevant SR discussing BCI games [27]. All search plans are based on title, abstract, and keywords (TAK), and duplicative articles would be identified and removed via EndNote X9 and a relatively deep read.

Plan	Group1	Group2	Group3
Pre-search	Brain-Computer Interface	Video game
Main-search	BCI	Game	Motor imagery
	Brain-Computer Interface	Entertainment
	Brain Machine Interface	Videogame
	Neurofeedback	Gaming
	EEG	Gamification
	Electroencephalogram	Gamified
	Mind controlled	Competition
	Brain Controlled
	Biofeedback
compensatory	Motor imagery	game
		gamification

Table 3.

Three searching plans with separate groups.

2.3 Study selection

Three main types of criteria have been used in this review: Selection Criteria (SC), Exclusion Criteria (EC), and Inclusion and Quality Criteria (IQC). SC is for broadly filtering articles. After determining the scope of literature via SC, EC will determine this scope more precisely. IQC is adopted to decide whether the articles could be suitable after deep reading. The specific SC, EC, and IQC can be seen below:

2.3.1 Selection criteria

For objectiveness, the chosen articles must be written in English. Furthermore, these articles must be peer-reviewed, covering a designed MI-BCI game. There is no restriction for the published years as some early MI-BCI game systems, such as the first-person shooting game [24], are still highly valued in the academic area.

• Written in English

• Peer-reviewed

• No searching duration is limited

• TAK is discussing about a non-invasive MI-BCI game.

2.3.2 Exclusion criteria

The remains still need to be under consideration before being used as a definitive reference. For example, some titles or abstracts, including the typed terms but different prior research aims, should be excluded. We would not recommend any literature review without one own experiment. Additionally, reports aiming at pure MI therapy without BCI, or invasive MI-BCI research, would be withdrawn. Other factors such as a hybrid control method, which might positively influence the feasibility of an MI-controlled BCI game, should be excluded. The lists below are the detailed EC.

• The outcome of the articles does not cover gaming MI-BCI

• If the paper is a review, there is no MI-BCI research or outcome but just the result of literature analysis.

• Motor imagery is not for BCI.

• Gamified MI-BCI system with other control or feedback methods as a factor that could influence the result, such as a hybrid BCI control system with MI and P300 together [28].

2.3.3 Inclusion and quality criteria

Rather than distinguishing between inclusion criteria and quality criteria, this paper combines these two as IQC (see Table 4). This combination is because the inclusion criteria could also be qualified (totally fulfilled or partially fulfilled) and become one part of the article evaluation. We will withdraw the study if it cannot match any one of the two compulsory questions (CQs). Six main quality questions (QQs) are subsequently presented to assess the level of the selected article. If the article failed to be higher than the four scores, this work would also be rejected because of the low report quality.

Criteria type	No.	Score	Criteria requirement	Level of result
Compulsory	CQ1	Y1	Are the goals of the paper clearly stated and include gamified MI-BCI system?	If No, the article should be ignored
		P0.5
		N0
	CQ2	Y1	Is there any clear outcome related to gamified MI-BCI system presented?
		P0.5
		N0
Non-Compulsory	QQ1	I5	Is this article mainly talking about game (2) training (3) improvement (5)	Satisfactory >= 4
		T3
		G2
	QQ2	N1	Is there any main focus on one of the steps in the whole loop of MI-BCI; (recording brain activity, preprocessing, feature extraction, classification, translation, and receiving feedback) [29]
		P0.5
		Y0		Good >= 7
	QQ3	Y1	Is this paper presenting the gamification metrics of the research?
		P0.5
		N0
	QQ4	Y1	Is there a clear explanation of the result?
		P0.5		Important >= 9
		N0
	QQ5	Y1	Is there a scientific evaluation for the outcome?
		P0.5
		N0		Excellent >= 11
	QQ6	Y1	Is there a useful discussion for recommending further research?
		P0.5
		N0

Table 4.

ICQ form. Yes (Y) for 1, Partially (P) for 0.5, and No (N) for 0.

QQ1 is estimated with different scores because different research aims reveal different study repeatability. For example, 5-scores are rewarding for the studies’ attempt to improve the gamification outcome. These studies are probably continuous studies with a developed system under a reproducible experience. These articles might therefore be convincing for result reviewing. In contrast, if the research aims to design and test an MI-BCI game, it would be scored two since this game system might need more evidence for reproducible testing. MI-BCI applied for subjects training research will gain 3-scores. That is because a training environment is more challenging for MI-BCI robustness than just a game test. However, the lower level of challenge and number of supportive studies than the ‘improvement’ study positions them in 3 scores. We then divide the quality of articles into four levels: satisfactory, good, important, and excellent.

3. Results

3.1 Study selection

This review finds 2524 articles after using the TAK searching strategy above. The exact numbers of each database and term can be seen in Table 5. After removing the duplication and first reviewing with SC, 549 articles are saved in three plans, and seven more articles are included with two additional searching methods. Sixty-one articles remained after a second review with EC. Three main reasons for high elimination (approximately 90% of the articles) are no gamification, hybrid BCI, and BCI for assessment but not control. After detailed reading with IQC, we finally marked 28 articles. This reduction is due to insufficient reporting data. The whole search process is presented in Figure 1

	Plan1	Plan2	Plan3
Scopus	192	719	357
IEEE	67	344	21
Pubmed	82	172	563

Table 5.

The number of the articles found in 3 plans of stage 1.

Figure 1.

The whole process of searching strategy using PRISMA.

The means of ICQ of 28 articles are 9.3, while the standard deviation is 2.1. These parameters show that the quality of the chosen articles is relatively high, with an average level of ‘important’ with slight fluctuation.

3.2 What games have been used in MI-based BCI?

In this study, we refer to the book written by Ernest [30] to classify different games. The seven main genres are action, strategy, role-playing, real-world simulation, construction and management, adventure, and puzzle games. Additionally, we use subgenres for a better description. This subgenres classification strategy is also agreed by Ernest, who claims the classification change constantly with different circumstances.

Thus, this review develops a subgenre classification for the chosen 28 MI-BCI games depending on three aspects of the graphical interface: ① whether it is the first-person view, ② which visual type (2D, 3D, VR) it is, and ③ what the game content type is. For content-type differentiation, this paper pays strong attention to task-orientate separation, i.e., what targets the gamers are asked to achieve, or what functions they are expected to present. Games with these similar contents would be linked. For instance, one study [31] describes itself as a cue movement game. Nevertheless, the game aims to pick a falling parachutist via a controllable platform. These contents, such as control method, feedback, and mission, are similar to the game [23], whose participants attempt to control a failing ball to land on a fixed basket. Thus, we identify the saving parachutist game as a ball-to-basket (B2B) game but not a cue movement-based searching coins game [32] (Table 6).

Game subgenre			Category
First-person	Dimension	Content	Classic Genre	Abbreviation
No	2D	Ball to basket	Action	2D B2B

Table 6.

An example of game differentiation.

After game classification, four genres with 11 subgenres of MI-BCI games have been identified (see Appendix 1). We present each research group with its summarized game content type in this detailed form. In Appendix 1, the game vision type is shown. For easy understanding, we match both the content and vision of the games with the abbreviation (Ab.).

The four main genres of MI-BCI games are action, puzzle, adventure, and simulation. Table 7 presents the quality of each genre. The following paragraphs explain each genre with its typical subgenre games in order.

Genre	Total score	Total study	Average
Action	264	19	9.428571
Puzzle	173.5	7	9.131579
Adventure	16.5	2	8.25
Simulation	16	2	8

Table 7.

Four genres total score, number of studies and average score.

3.3 Action game genre

Most games (18/28) belong to action games whose players try to control the gaming character to present physical activity, such as jumping or moving to one side, to overcome the challenges and gain rewards. The following paragraphs introduce four main subgenres of action games (2D avoiding obstacles, 2D ball to basket, 3D Cybathlon brain-running, VR first-person action) and other games.

2D avoiding obstacles (2D AO) games (5/18) are a series of games where players use different imaginations to control the character's direction to avoid obstacles and collect rewards. In 2007, one study [33] presented a “Jump and Run” game whose ten healthy players used their adept imagination (right hand, left hand, or feet) to send jump commands to avoid the intermittent obstacles. Two out of five subjects presented a high accuracy of more than 90%. An ‘arcade game’ with similar up and down movement was present with an accuracy of 70% among ten subjects [34]. In 2011, a spaceship control game was provided [35]. Gamers used two out of four motor imageries (left hand, right hand, both feet, or tongue) to control the spaceship to move left or right to avoid falling asteroids. The synchronized game score also rises when the character passes asteroids successfully. Two out of three subjects reported increasing classification accuracy during this gameplay. A similar game is a design where a spaceship is replaced by a car, adding a coin reward [36]. Feet MI was linked with left movement, and right-hand MI could control the car to the right. An average multiple event accuracy (ME_Acc) could reach 78.3% among four participants. One game [37] provided an environment where three classes of MI could be presented. Gamers could move to the left or right by ipsilateral hand imagery to collect coins. A jump command is sent by feet of imagination when the subject has to avoid the snake on the lane. After three gameplay sessions, an increasing average classification rate (53% to 63%) and game score (1600 to 1900) are presented among 14 BCI naïve healthy participants.

2D ball to basket (2D B2B) games (3/18) are games where players use one side of their body image to control a falling ball landing on the ipsilateral side of the basket. In 2003, one article indicated the feasibility of applying this game to 4 paraplegic patients [23]. 3 out of 4 participants succeeded in controlling the falling ball to an exact color basket by using two optimized MI from left hand MI, right hand MI and feet MI. One study in 2010 revealed the feasibility of using this type of game for stroke patients with a moderate classification accuracy of 60%-75% among five novices BCI subjects [2]. This result is similar to a previous study [38] with six healthy users (average classification accuracy of 69.2%). In 2016, nine healthy users were trained to use left or right-hand MI to control a platform (basket demo) to save a falling parachutist (ball demo) [31]. All the accuracy performances are higher than 70%.

3D Cybathlon brain-running (3D CyR) is a game (3/18) designed for the first Cybathlon competition held for a disabled cohort in 2016 in Switzerland [39]. Cybathlon competitors, called pilots, in BCI Race, present four different MIs depending on the areas the role needs to pass through. The result is ranked from the shortest runtime in order. In the MIRAGE team, whose pilot is a post-stroke patient, the average runtime could reduce from 178s to 143s during training and 196s (rank 11/11) in competition [40]. The optimized 2 MI classes are right hand MI, both feet MI, with other two mental subtraction, and auditory imagery. In the same competition, two tetraplegic pilots in the Brain Tweakers team [41] present 90s (rank 1/11) and 123s (rank 2/11) runtimes in the first race. In the final round, these two pilots present 125s (1/8) and 190 (4/8). Their control strategy is a 3-class MI (right hand, left hand, and both feet) combined with resting. Those two groups of pilots are training regularly for months before participating Cybathlon competition. In contrast, one research [42] shows that a healthy cohort trained with only two sessions could reach an accuracy of 68.62% with 2-classes of MI (right hand MI and stomach MI).

VR first-person games (VRFP, 4/18) is a group of action games with visual reality (VR) devices to present a first-person vision gamified feedback. From the players’ view, no realistic but misleading interruption such as a passing-by researcher would occur so that players could fully be engaged in the game. In 2013 [43], 12 healthy subjects with one post-stroke patient accomplished a VR B2B game. Participants tried to use the ipsilateral hand MI to control the virtual hands to the right side for picking the falling basketball. Results show that although the disabled users only trained for one session, the final accuracy could reach 77%, whereas the highest accuracy among two-session healthy users is 70.67%. A similar VR B2B game designed in 2019 with 4-class MI (left hand MI, right hand MI, both feet MI, and resting) presents a mean accuracy of 70% with 10 participants [44]. One game with not only visual but also auditory and haptic feedback, called “NeuRow,” was present in 2016 [45]. 13 healthy users use their left- or right-hand MI to navigate the boat towards the same side. Two high-fidelity arms enhance the MI of users by linking the imagery environment with the real world. The classification score (70.7%) in NeuRow compared with the other six studies with the same classification method indicates that the NeuRow has scored the best. Another VR MI-BCI game designed in 2019 is a simplified shooting game [46]. Nineteen users try to destroy the asteroid (VR DA) with the same side hand MI. After receiving the MI commend, the embodied visual hand would move to switch on the trigger of the weapon to shoot the asteroid. The average peak accuracy among these users could reach 75.84%.

In 2003, one 3D first-person shooting game (3DShT, 1/18) was tested among four healthy participants [24]. Since it is the earliest MI-BCI game, researchers did not limit the users to control the game in a certain fixed MI way. Instead, participants are allowed to discover how to adjust mu rhythm, which is related to sensorimotor control by themselves. Result reveals that mu activity could be actively adjusted to control the scenery to the left or right. Researchers subsequently indicated the potential that mu rhythm becomes a binary signal for MI-BCI control. In 2013, researchers developed BrainArena, a simplified football game (2D FB, 1/18), where two users could play in a collaborative or competitive mode [47]. Eight healthy subjects in 4 pairs show an average classification accuracy of 75.4% (collaborative manner) and 74.6% (competitive manner). In 2018, a 3D ball balance game (3D BL, 1/18) was designed. Ten healthy subjects use left or right-hand MI to control the platform to slightly rotate clockwise or anti-clockwise so that the planet on the platform would not move to the low end and fall off [48]. Final accuracy could reach 70% with the concentration improvement.

3.3.1 Adventure game genre

Adventure games (2/28) are a series of a game whose target is mainly task-orientated exploration. For example, adventure gamers usually need to explore the map with specific targets, such as finding a particular non-player character (NPC) or finishing a specific mission.

We only found one type of adventure game for the MI-BCI system. Scherer et al. [49] designed a 3D first-person searching coins game (3D SC) called “freeSpace” in which two out of three healthy players succeeded in collecting all three coins in three minutes. Three out of four MI classes (right hand MI, left hand MI, tongue MI, foot MI) are used for left, right, and forward navigation to find three hidden coins on the whole map. After a few trials, two participants could see improvement in covered distance and searched coin numbers. In another report of this game experiment with a different angle, research indicates that the whole runtime would be 110s with 100% classification accuracy [27].

3.3.2 Puzzle game genre

Puzzle games (7/28) usually offer enjoyable logic or cognitive training tasks. Gamers need to identify the link among the provided items, answer specific questions or find a way out of a maze in a limited duration.

2D fix route maze games (2D FxM, 2/7) are a group of games whose characters should walk following a pre-designed line. In 2007, a research group redesigned the 2D “Pacman” game to be compatible with MI-BCI users [50]. Users could navigate the Pacman with left hand MI or right-hand MI to finish the instructed route in the maze with the highest score. Additional credit will be rewarded if the user could pick the apple on the lane, and the game score will decline if the Pacman hits the wall. A similar maze game was designed in 2017 [51] where users could use 4-class MI (left hand, right hand, both feet, and tongue MI) to finish the instructed route. The highest gaming accuracy is 48.7% among the four subjects, while the average classification accuracy is 68.5%.

2D Voluntary route maze games (2D VM,3/7) are maze games without a fixed path but only a start point and an endpoint. In 2009, 2 healthy players performed well in a maze-like cursor control game [52]. Gamers are asked to move to the target area and avoid one fixed trap with left- and right-hand MI. One gamer could reach an offline classification accuracy of 73%. This game is then used among one amyotrophic lateral sclerosis (ALS) patient and one primary lateral sclerosis (PLS) [53]. ALS user presents a receiver operating characteristics (ROC) rate of 81.8%, and the ROC rate for PLS user is 86.7% in two visits. A similar game was developed in 2012, and 4-classes MI (left hand MI, right hand MI, both hand rise MI and both hand fist MI) are used for navigation in four directions [54]. Healthy gamers present an accuracy of 60%-70% in this study.

2D Jigsaw puzzle (2D JP) games (2/7) are a group of games where players try to finish collecting the jigsaw by specific tasks. One case study reports that Arm MIs could help play a jigsaw puzzle game [55]. The cerebral palsy (CP) participant uses left or right-arm MI to accomplish tasks and collect puzzles. The result shows that the acquired MI skill could help the participant gain additional MI-BCI skills such as choosing, playing, and pausing videos. Furthermore, the skill could last for more than six weeks, as proven by the subject’s successful control of the MI-BCI virtual cube. An afterward study in 2017 with the same game but a larger number of subjects (eight patients with CP) showed the potential for CP patients to interact with this MI-BCI puzzle game [20]. However, the quantitative result in performance score for all users is relatively negative with several training issues such as head cap discomfort.

3.3.3 Simulation

Simulation (1/28), or real-world simulation, is a group of games, including vehicle driving or particular physical challenges. However, unlike action games, simulation is often more focused on navigation experience and related tasks. In 2019, one VR drone racing game called “Brain eRacing” (VR DR) was reported with 30 healthy participants [56]. However, unlike other games using MI for navigation control, MI is only used to accelerate the drone’s speed. The final average game time is 54.88s, while the group with the last average time is 69.6s.

3.4 What are the metrics of MI-BCI game?

3.4.1 Performance metrics

24 out of 28 articles use quantitative measurement to test the learning outcomes, whereas four use qualitative metrics to report the participants’ performance (see Appendix 2). One article has not tested their result [50]. One study [54] only presents a description indicating that a user who was not an expected volunteer could control the mental task easily in a short duration. We will not consider these two articles in metrics recommendations. Therefore, the following paragraph will indicate the commonly used performance metrics in the qualitative and quantitative groups.

Appendix 2 lists several performance metrics used for learning outcomes measurement (see Appendix 2). Twenty-four articles used quantitative metrics for MI-BCI gaming mode measurement. The two most frequently used metric is accuracy (16/24) and performance score, or game score/rate (5/24). In four articles using qualitative evaluation, the commonly used metric is the game experience questionnaire (GEQ) (2/4). In the discussion part, we will present a detailed introduction and analysis.

3.4.2 Training issue

The existing issues identified in the experiment are reported from the first designed MI-BCI game till now (see Appendix 2). We found six main issues: ① environmental background and distraction (3/28), ② trial length relevant fatigue (4/28), ③ performance variation (4/28), ④ increasing false positive rate (2/28), ⑤ performance decrease (2/28), ⑥ toughness to complete game (2/28). Table 8 below shows the summarization of each issue with its consideration. We gave a detailed explanation in the discussion part.

Training issue	Factors/potential solutions need to consider
Environmental distraction	Balance between distraction and engagement
Fatigue	Shorten trial length and test in the morning
Performance variation	Real-world experimental environment but consider the difficulty
Increasing False Positive rate	Method eliminates the expectation of next MI, and find optimal threshold for better classification
Performance decrease	Reducing physical demand
Toughness	Reforming environment and consider classification

Table 8.

Summarization of training issue and its considerations.

4. Discussion

4.1 What are the factors that researchers should be focusing on when designing a gamified experiment

4.1.1 Metrics for MI-BCI game mode

Quantitative metrics: Researchers usually identify the learning outcome by comparing the accuracy of each experiment in sequence [43]. The testable and quantitative characteristics keep accuracy as the top used metric. Performance score, or game score/rate, is another commonly used quantitative metric equally ranked with consuming time. The game score could reflect real-time performance intra-trials and quantitative improvement cross trials objectively. One study [37] uses online game scores to motivate BCI users to improve their next performance. Moreover, line graphs for describing performance score fluctuation could be more visible for presenting learning outcomes [55]. The same Auckland group used the score-line curve one year later in another research with ADHD patients [20].

Based on the writers’ experience in the rehabilitation equipment market, the scoring method is more intuitive for patients and therapists to identify the rehabilitation outcome. In contrast, accuracy is more reliable for the technical measurement of outcomes from both users and the system. Therefore, we suggest using accuracy for research aims, such as identifying whether gamification could enhance the learning outcome more than non-game MI-BCI. Synchronized gaming score, in contrast, could be applied to potentially enhance the performance of users or check the rehabilitation stage.

Qualitative metrics: The commonly used qualitative metric is the game experience questionnaire (GEQ) (2/4). GEQ is the response from MI-BCI gamers. Researchers use GEQ for measuring several experiences, such as flow experience. A guideline is recommended to quantify all the qualitative answers and group them into a specific experience [57]. For example, Tezza et al. [26] use a 1–5 Likert scale to evaluate the experience.

4.1.2 Existing issues

Environmental Background and Distraction: Distraction is an environmental or feedback issue that would disturb the users from performing a correct MI. In one of the earliest MI-BCI gaming experiments [24], researchers assumed distraction caused the no-growth low mu rhythm used for game control. In a simplified football game [47], 7 out of 20 subjects argue similar disturbing feedback. One article reports the same disturbing issue [35], but they claim this issue offers more engagement in the game, in contrast. Serious consideration to balance the augmented engagement and distraction, thus, is required when applying the idea of multi-player MI-BCI games.

Trial Length, Experimental Time and Fatigue: Fatigue is another issue that generally happens during MI-BCI training. The length of trials and the time for experiments are linked to fatigue. Significant variability in performance is found in different lengths of trials [23], which might be due to fatigue. In one 2D B2B game [2], researchers discover similar performance variability due to evident fatigue and depression cross-subjects. In detail, two participants complain that the sessions are too long to keep them performing well. Researchers report similar fatigue issues and lack of concentration in one subject whose performance is not ideal after four sessions [36].

Authors from the 2D B2B game research [2] also believe that the chosen time (afternoon) for an experiment is one factor that increases the tiredness providing low performance. One puzzle game case study [55] supports this argument, in which authors suggest a morning experiment to avoid fatigue.

Performance Variation: Performance variation is when outcomes of distinct trials are different. It often happens when different individuals use the same MI-BCI device with all other factors being the same. For example, in one gamified training mode [27], two participants showed improvement in the game performance while one subject showed a decrease in performance.

However, variation would not only be found in cross-subjects but also found in the same subject when they failed to provide reproducible MI effort in each trial. This variation is called intra-subject performance variation. Psychological states, such as nervousness and motivation, correspond with intra-subject performance variation [40]. Another study [31] reports the same issue and claims that it is possible to solve this problem by using adaptive assistance in changing the trial length. Consistency and personalization are two issues causing cross-subject and intra-subject performance variability [42]. Researchers predict that a competitive training environment might help maintain users’ excitement [35]. However, this reviewer would consider whether this environment could have side effects, such as distraction, on learning outcomes in a short time.

The Increasing False Positive Rate: The false-positive rate is the proportion when the classified command is against the imagination of participants. Scherer et al. [27] reveal that the increasing false positive rate might be related to premature brain activation. Researchers also believe this non-stationary brain wave links with the unsatisfactory robustness of the system.

An optimal threshold is also a factor that impacts the false positive rate. The threshold of MI-BCI is the digital criteria to identify whether the amplitude of Electroencephalogram (EEG, one type of brain signal) could be a task-relevant mental task. One study shows the rate declined when adjusting the decision boundary from 0.6 to 0.8 [44].

Performance decrease: Performance decrease is the declining quality of quantitative performance metrics, such as decreased accuracy, ITR, and increased time. One study indicates its link with increasing workload [45]. To solve this, reducing the physical demand is recommended [53]. Furthermore, to keep a good performance, minimum eyes and body movement are required [44].

Toughness in playing games: Toughness is the difficulty for participants to finish a goal in the game. One study [42] argues that though the gaming concentration is satisfied, the controlling difficulty is still visibly raised when the number of MI classes grows. This evidence informs that the number of MI control types impacts toughness, which could then cause performance variability.

4.2 What games have been used in MI based BCI?

4.2.1 Feasibility of MI-BCI gaminification

Appendix 3 indicates the evidence of the feasibility analysis. For research reporting with performance accuracy, the criterion of feasibility is that the average accuracy (accuracy/no. person) could finally reach 70% of accuracy, based on the criteria of BCI application [58, 59]. Although the articles concluded that 233 subjects successfully went through the MI-BCI experiments, only 111 individuals have examined the feasibility of the MI-BCI game with its gaming accuracy.

This review chooses the maximum average accuracy (MAA) for quantified feasibility as one metric. For example, Djamal et al. [34] presented two groups of average accuracy (a non-FFT group with 50% versus an FFT group with 70%). This review picked the 70% FFT group as an MAA since this accuracy is reachable with existing system upgrade technologies, such as FFT. After analysis, 28 chosen articles present an average accuracy of 74.35%, higher than the threshold. Additionally, 26 out of 28 articles (92.86%) possess positive responses to the feasibility of the MI-BCI game. These two results illustrate that it could be feasible to apply gamified MI-BCI training mode for further experiment.

4.2.2 Game recommendation

From the analysis above, it is explicit that 2D AO is the commonly used game type (5/28, 17.86%). The evidence to support this argument is generally high (46.5 total scores in AQ with a mean of 9 and a standard deviation of 2.17). These statistic results advise 2D AO to become a high-priority recommended game. The advantage of the 2D AO game is the high level of motivation. The coming obstacles would increase the tension and stimulate the users to accomplish the correct MI to rescue the character intuitively.

However, except one study uses 3 MI classes (left- and right- hand MI and feet MI), others do not use more than two MI types for gaming navigation. This circumstance is a limitation of current 2D AO games. Future 2D AO game design should cater to real-world complexity by providing more MI class on the one hand and a reliable user experience with guaranteed accuracy on the other hand.

A potential solution is to add more controllable gaming factors in the 2D AO game. For example, forward and backward, acceleration and deceleration, and even shooting a bullet against the obstacle could become controllable by additional MI types more than just left and right hands' imagination. Furthermore, different MI classes could depend on environmental color changes. 3D CyR games apply this idea to reality [40, 41, 42], where competitors produce four MI classes based on the above mission colors.

2D B2B game is ranked equally with 3D CyR in report number (3/28, 10.71%) and similar quality (30.5 total scores in 3D CyR vs. 29.5 in 2D B2B). Like a 2D AO game, game designers should notice the limited MI control classes since all three studies using 2D B2B games only present the learning outcome of two classes of MI (Left and Right hand). The failure in recommending a 2D B2B game is because of its lower level of motivation than a 2D AO game [35].

We advise the 3D CyR game since this game could be not only for training but also for the Cybathlon competition. These applications encourage researchers to further the development of rehabilitation and patients to open out to build relationships and share their life with their competitors from other nations or territories. Nonetheless, an advanced challenge for the CyR game requires an excellent feature selection method to afford the upgraded complex data calibration and classification due to the increasing number of MI classes and highly competitive interruptions.

One limitation in all three games above is the lack of immersion, i.e., players feel difficulty linking their MI with their motor execution (ME) when playing a game. This condition raises challenges for presenting correct MI since no environment could work as humanoid neurofeedback to motivate the gamer to produce the related MI. Therefore, humanoid factors should be a focus when utilizing these three types of games.

First-person VR action games (VR FP) do not need to consider humanoid environment issues. Advantages of VR application are imaginative immersion and a high level of sensation. [56] Karácsony et al. [44] share the comments from participants who played VR B2B games: gamers feel they are using their own hands to catch the falling balls via VR-based MI-BCI game. Therefore, these humanoid advantages could improve the participants' performance [44]. Furthermore, one study reports a high gaming accuracy in the VR B2B game. However, the VR DR game does not reproduce a similar extent of learning outcome. This review believes a lower level of immersion, such as a non-first-person humanoid control environment and no visible link between MI and navigation, causes the game to become relatively distracting. This phenomenon prevents the VR DR game from recommendation.

The interest and the difficulty of the game design are required to consider [27]. Although VR games could reach a high level of learning outcome, the complexity of designing the platform and synchronized interaction with numerous MI-BCI data is not as easy as a 2D PC game. The immersion provided by a VR game might be lower than a 2D AO game if designers sacrifice the quality of a game system to reduce complexity. That is because potential high delay, game bugs, and illogical graphical switches might occur as distractions reducing the MI-BCI experience. A 2D MI-BCI game with good design could also present well in tactical, strategy, or narrative immersion, which are not specific to VR [30]. These facts prevent VR FP action games from becoming the only recommended game.

In summary, if there are temporal and spatial limitations, we advise researchers to choose game content close to 2D avoiding obstacles. The insufficient degree of immersion and the number of MI classes are two main issues that need to consider. 3D Cybathlon running game is another choice for researchers due to its futuristic maturity in competition, stakeholders, and technique. In contrast, we recommend a VR first-person action game with first-person vision when there is no limitation in designing and experimenting with an upgraded gamified MI training system. In particular, the VR ball to the basket game and destroying asteroids game are suggested as two existing first-person VR examples.

Another game genre, puzzle game, is not recommended in this review because motor imagery is still easily disturbed by other factors, as mentioned in both current issues and Appendix 2. A hybrid game with an action factor to motivate correct MI and a puzzle factor to enhance the strategy immersion is still potential for improving learning outcomes of MI-BCI. However, we suggest designing the game navigated by action-dominated factors. For example, the game story could cover puzzle game questions. However, for choosing an answer and continuing the story, the graphical interface of the MI-BCI game could use a humanoid hand for assistance.

4.3 Limitation

No article has discussed the comparison between gaming MI-BCI with a control group. The main reason is that majority of the studies are still focusing on the feasibility of using different feature selection methods in the MI-BCI game, aiming to improve the performance outcome with more advanced techniques. Even though three existing articles [32, 43, 47] claimed the improved learning outcome with comparison tests, their research process is not sufficiently scientific as randomized control trial (RCT). This limitation shows a research gap that could be filled shortly: a randomized control experiment for evaluating whether gamification could improve the learning outcome of MI-BCI is required.

Another limitation is the objectiveness of this review. Since gamification in MI-BCI is an interdisciplinary study covering tremendous academic fields, this review cannot present an in-depth analyzed review and recommendation. This research group hopes to complete a systematic review with video game experts when enough RCTs have tested an optimal training mode.

Furthermore, we advocate more significant attention to users' cohorts. Only 7 out of 28 articles test the feasibility of games in a disabled cohort. One study in this group using a commercial wireless BCI device reports a highly negative result: all participants failed to complete the initially designed trial. None of them shows a learning outcome increase after the full training [55]. This situation absorbs attention on an appropriate task for different users. A relatively short training duration with a more comfortable experiment setting is probably more friendly to patient-participants. Additionally, one study reports the possibility that subjects with video game experience would have a better MI-BCI performance [60]. Gender is also a probable factor related to performance [46]. Therefore, when separating the participants and analyzing the results, these participants-relevant factors should consider.

Real-World Study (RWS) [61] is a research type covering the data collected widely and randomly. Researchers pick the evidence (Real World Evidence) without strictly classifying one controlled particular pattern. For MI-BCI, existing data is often gained from healthy cohorts or individuals with specific neurological impairments. These data are not sufficient for testing the robustness and reliability of MI-BCI for real-world users and applications. More influential factors would occur than subjects whose patterns are controlled in an RCT. For applying MI-BCI to these real-world users with a reliable outcome, this review recommends having an additional RWS after a sufficient number of RCT studies show positive results.

4.4 Insisting pure MI-BCI control strategy in game

This review studied gaming BCI systems controlled by only MI. Plenty of research has tested the feasibility of hybrid BCI control [62] with games. One article successfully used a hybrid technique with MI to control a well-known but highly complex game, World of Warcraft, surprisingly seven years before [63]. Although these combination control methods might accomplish more remarkable improvement than mere MI control, this review still insists on the importance of exploring the maximum outcome of pure MI.

The reason for insisting on single MI control is that the source of performance issues still needs to be researched. On the one hand, well-direct optimization of the pure MI control could efficiently improve the hybrid control system. This review is not denying the better performance that hybrid MI devices could achieve. In contrast, after digging out and solving the problem caused by MI-related factors, a combination of other methods might have higher performance growth in another dimension.

On the other hand, even if a hybrid MI system has been designed, the outcome would still not be ideal if we have not sufficiently explored and solved the potential performing issue. Like a disease, low MI performance is a symptom of MI-BCI, and unsuitable MI design is one of the root causes. Using merely hybrid MI control seems like treating the low-performance symptom, but not the root causes. This assumption could be supported by one study [62] that participants still need to be trained for a long duration with a hybrid control system that has been applied in their Tetris game. In contrast, discovering suitable MI-BCI gamification to help users is a metaphor for treatment for the root cause.

As a product manager in Fourier Intelligence, our regular marketing survey shows that patients requiring long training duration would still get bored of the game provided by our rehabilitation equipment. However, a number of games have been included in our game library for different rehabilitation equipment. Therefore, providing MI-BCI games to patients requiring several weeks or even months of training would still challenge keeping patients engaged. Designing other MI-BCI games for satisfying patients might not be a long-term solution (Figure 2).

Figure 2.

An example of training game in game library.

4.5 Future work

This literature review presents several research gaps. For achieving the target of using an RCT and an existing game to test the improvement of MI-BCI gamification in a short time, we advise designing a 2D AO game first. A simulation could be done first to test the feasibility of this theoretical recommendation via OpenVibe software. Suppose the classification accuracy is higher than 70%. In that case, the participant-based study will try the designed game training model, the non-game control group model, and the third-party outcome testing mode. Also, if no significant variation can be observed between the game and control groups, the game type will be optimized. For example, a 2D first-person humanoid game might replace the 2D AO game to identify whether the negative result is due to the lack of a humanoid environment. The argument claiming gamification would not improve learning outcomes would be reported if even a VR first-person humanoid game could not present a significant difference from its control group.

5. Conclusion

Presently, researchers and clinicians expect to apply the combination of MI and BCI to a great range of users for rehabilitation purposes. Thus, existing challenges in current MI-BCI works must be solved, such as training issues. This review has utilized the majority of SR to present a scientific and objective review of one potential solution, gamification, to optimize MI-BCI training outcomes. Our finding through evidence (28 articles, 111 individuals with accuracy test, 74.35% average accuracy, 26 out of 28 positive responses) shows that using the MI-BCI game for training could be feasible.

The result also recommends 2D avoiding obstacles game, 3D Cybathlon game, and VR humanoid first-person action game as prior MI-BCI games for further research. Moreover, this review maps the cautions of the current issues of gamified MI-BCI training mode and the potential methods to overcome them. The literature review is predicted to provide a new vision of the appropriate gamified MI-BCI training modes and what elements, such as metrics and issues, are suggested to consider for scientific research.

A.

Study			Game		Category
ICQ	Author	Year	First-person	Dimension	Content	Classic Genre	Abbreviation
11	Pineda et al. [24]	2003	Yes	3D	Shooting game	Action	3D ShT
9.5	Graz(Krausz et al.) [23]	2003	N/A	2D	Ball to basket	Action	2D B2B
7.5	Graz(Müller-Putz et al.) [33]	2007	N/A	2D	Jump obstacles	Action	2D AO
6	Berlin(Krepki et al.) [50]	2007	N/A	2D	Pac-man	Puzzle	2D FxM
6.5	Graz(Scherer et al.) [49]	2007	Yes	3D	Searching coins	Adventure	3D SC
10	Graz(Scherer et al.) [27]	2008	Yes	3D	Searching coins	Adventure	3D SC
6.5	USA(Huang et al.) [52]	2009	N/A	2D	Sokoban	Puzzle	2D VM
7.5	USA(Ou Bai et al.) [53]	2010	N/A	2D	Sokoban	Puzzle	2D VM
8.5	Prasad et al. [2]	2010	N/A	2D	Ball to basket	Action	2D B2B
11	Coyle et al. [35]	2011	N/A	2D	Avoiding obstacles	Action	2D AO
9	Bordoloi et al. [54]	2012	N/A	2D	Maze game	Puzzle	2D VM
10	Anopas et al. [43]	2013	Yes	VR	Ball picking game	Action	VR FP
12	Bonnet et al. [47]	2013	N/A	2D	Football game	Action	2D FB
11	Asensio-Cubero et al. [37]	2016	N/A	2D	Running game	Action	2D AO
11.5	Kreilinger et al. [36]	2016	N/A	2D	Car game	Action	2D AO
11.5	Switzerland(Saeedi et al.) [26]	2016	N/A	2D	Ball picking	Action	2D B2B
11	Auckland (Taherian et al.) [55]	2016	N/A	2D+audio	Puzzle	Puzzle	2D JP
8	Vourvopoulos et al. [45]	2016	Yes	VR+vibration	Rowing game(NeuRow)	Action	VR FP
5.5	Djamal et al. [34]	2017	N/A	2D	Arcade avoides obstacles	Action	2D AO
8	Graz (Statthaler et al.) [40]	2017	N/A	3D	Brain running(Cybathlon)	Action	3D CyR
9	Auckland (Taherian et al.) [23]	2017	N/A	2D	Puzzle game	Puzzle	2D JP
11	Zhou et al. [42]	2017	N/A	2D	Maze game	Puzzle	2D FxM
12	Switzerland (Perdikis et al.) [41]	2018	N/A	3D	Brain running(Cybathlon)	Action	3D CyR
10.5	Ponferrada et al. [42]	2018	N/A	3D	Brain running(Cybathlon)	Action	3D CyR
7	Yang et al. [48]	2018	N/A	3D	Balance game	Action	3D BL
11	Karácsony et al. [44]	2019	Yes	VR	Catching&kicking falling items	Action	VR FP
12	Škola et al. [46]	2019	Yes	VR	Destroying asteroids	Action	VR FP
6.5	Tezza et al. [56]	2019	N/A	VR	Drone competition	Simulation	VR DR

B.

Author	Year	Performance metric	Performance Issue
Pineda et al. [24]	2003	Low / high mu power changes learning curve (10H in total)	Non-increase of low mu might be due to feedback distraction.
Krausz et al. [23]	2003	Information transfer rate(ITR)	Significant variability in ITR with different trial lengths might be because MI and attention effort cannot be kept at the same level over the session.
Müller-Putz et al. [33]	2007	Accuracy	—
Krepki et al. [50]	2007	Not test yet
Scherer et al. [49]	2007	Time for finish mission(maximum time 3 mins)
Scherer et al.[27]	2008	Time for finish mission(maximum time 4 mins), Accuracy	Training: the expectation of the next cue to come might change the brain activity and produce False-positive command; although a higher detection threshold could reduce the false positive rate, motivation would also be decreased
Huang et al. [52]	2009	Accuracy	—
Ou Bai et al. [53]	2010	Accuracy	Vividness: participants reported difficulty imagining wrist extension; it might be improved by VMIQ (vividness of movement imagery questionnaire) or by teaching efficient motor imagery.
Prasad et al. [2]	2010	Qualitative1+Accuracy	Fatigue and Depression: a higher level of fatigue can contribute to more considerable variability in the BCI performance among subjects; two find treatment sessions are excessively lengthy and tiring (mainly in the afternoon); subjects want the game to be more exciting and challenging.
Coyle et al. [35]	2011	Accuracy	Challenging is due to background distraction, but this is also engaging
Bordoloi et al. [54]	2012	Only a description	—
Anopas et al. [43]	2013	Accuracy	—
Bonnet et al.[47]	2013	Accuracy	Feedback disturbing 7/20; purely informative 8/20; positive feeling 5/20; 4/8 users find multiuser feedback helpful, 3/8 find they were disturbed. None prefers single-user conditions among the best players.
Asensio-Cubero et al. [37]	2016	Accuracy, Kappa, and game score	High accuracy and less amount of analyzing data could be achieved by the best basis selection method; The difference between features in the calibration stage and game stage could be identified (this difference could be due to the mental state such as frustration or stress)
Kreilinger et al. [36]	2016	Success rate, error rate	Fatigue and lack of concentration is found after 4 runs in one user (S04)
Saeedi et al. [31]	2016	Accuracy, success rate	Performance variation in different trials for the same subject (intra-subject performance variation). Solved by adaptive assistance (that is, a user-dependent time out in a single trial)
Taherian et al. [55]	2016	Game score learning curve	Motivation, fatigue, and concentration influence performance. Subjects bored with a puzzle game loss are interesting. Performance decrease might be due to fatigue, increased concentration, and curiosity to identify new videos. The workload might need to be decreased by some methods such as meditation practice. Recommendation: BCI training needs to show users application of their learned skills from early on to increase a sense of self-efficacy and confidence; experiment should be done in the morning due to fatigue; also, a small office for less distraction.
Vourvopoulos et al.[45]	2016	Qualitative2	Low physical demand increases effort and good classification performance. Increasing workload causes worse performance.
Djamal et al. [34]	2017	Accuracy, time for finish mission	—
Statthaler et al.[40]	2017	Running time	Feature distribution had changed considerably between training and the game. This distribution might be due to a long time of rest (45min gap between sessions) and nervousness in competition (different heart rates). There are limited BCI systems in new environments; the pilot's performance fluctuations might be due to intra-subject performance variation, and intra-subject performance variation is related to psychological states such as motivation. Races with human competitors and a sizeable audience in training help the pilot himself to better prepare.
Taherian et al. [20]	2017	Game score	The unique head shape prevents subject three from using BCI; linear reduction could be seen in subject one since he has interval illness during the training and visual impairment, making him unable to focus on the screen. Subject2 feels discomfort when researchers touch his head; subject 5 is interested in doing it but is upset about it at home. Subject 7 feels frustrating in playing puzzles with sometimes no reaction; all 6 participants found EPOC uncomfortable wearing for more than 15 mins; EPOC cannot be used among patients with head support. Hearing the auditory feedback could improve their performances (comment from a special educator).
Zhou et al. [51]	2017	Accuracy	—
Perdikis et al.[41]	2018	Accuracy	Unsatisfactory robustness might be related to non-stationary brain signals; longitudinal mutual learning could help increase robustness.
Ponferrada et al. [42]	2018	Accuracy	Three commends control is significantly more complicated than two commends. Although enough concentration could be provided, high variability could be seen across different trial subjects: consistency and personalization are significant challenges. An ideal solution is to run multiple sessions to identify which motor imageries each subject can best control.
Yang et al. [48]	2018	Concentration(TBR,SMR)	Portability is required
Karácsony et al.[44]	2019	Accuracy, game score	The 0.6 decision boundary (low activation threshold) causes high false positives (after increasing to 0.8, the false positive is reduced). For good performance, subjects are asked to minimise their eye and head movements; immersiveness in the game could increase the deep engagement.
Škola et al. [46]	2019	Accuracy + Qualitative3	Females perform better than males on average (in only the first run). Fatigue is higher in BCI-naïve subjects in the first run than inexperienced but not replicated in any remaining runs. People are alert and motivated sufficiently to improve their results by fast but not long MI. No evident fatigue influencing the performance has been confirmed.
Tezza et al. [56]	2019	Qualitative4	—

C.

Author	Year	No.Participants	Final Accuracy	Final recommendation
Pineda et al. [24]	2003	4	—	positive
Graz (Krausz et al.) [23]	2003	4	—	positive
Graz (Müller-Putz et al.) [33]	2007	10	65%,72%,51%,70%,70%(5 subjects finally tested)	possible
Berlin (Krepki et al.) [50]	2007	1	—	positive
Graz (Scherer et al.) [49]	2007	3	—	positive
Graz (Scherer et al.) [27]	2008	3	83%, 88%, 80%	positive
USA (Huang et al.) [52]	2009	5	73%, 59.2%(2 subjects finally tested)	positive
USA (Ou Bai et al.) [53]	2010	6	81.1%, 86.7%(2 subjects finally tested)	positive
Prasad et al. [2]	2010	5	70%(average acc for 5 subjects)	positive
Coyle et al. [35]	2011	3	92.6%, 75.3%, 79.8%	positive
Bordoloi et al. [54]	2012	14	—	positive
Anopas et al. [43]	2013	12	70%,72%,70%(3 subjects finally tested)	positive
Bonnet et al. [47]	2013	20	73.94%(average acc for 20 subjects)	positive
Asensio-Cubero et al. [37]	2016	14	63%(average acc for 14 subjects)	positive
Kreilinger et al. [36]	2016	10	—	positive
Switzerland (Saeedi et al.) [31]	2016	9	85%,88%,91%,92%,89%,75%,82%,75%,96%	positive
Auckland (Taherian et al.) [55]	2016	1	—	positive
Vourvopoulos et al. [45]	2016	13	—	positive
Djamal et al. [34]	2017	10	70%(average acc for 10 subjects)	positive
Graz (Statthaler et al.) [40]	2017	1	—	positive
Auckland (Taherian et al.) [20]	2017	6	—	negative
Zhou et al. [51]	2017	4	45%(average acc for 4 subjects)	negative
Switzerland (Perdikis et al.) [41]	2018	2	93.8%,96.8%	positive
Ponferrada et al. [42]	2018	2	68.62%(one subject finally tested)	positive
Yang et al. [48]	2018	10	—	positive
Karácsony et al. [44]	2019	10	100%(average acc for 10 subjects)	positive
Škola et al. [46]	2019	19	75.84%(average acc for 18 subjects)	positive
Tezza et al. [56]	2019	30	—	positive