Near-Infrared Optical Technologies in Brain-Computer Interface Systems

Abstract

This chapter presents a comprehensive review of near-infrared spectrometry (NIRS) and related methods, implemented in brain-computer interfaces (BCI). Basic physical principles of such devices are described. Reviews supply readers with summary of recent development in dynamics and perspectives of the field in question. Examples of NIRS usage in BCI systems are provided and different experimental paradigms are described. Review not only deals mainly with noninvasive NIRS-BCIs but also covers some instances of usage of neighboring fields methods (such as EEG, for instance) for the sake of their importance in so-called hybrid BCI systems and/or in fundamental research, which may be less relevant in case of separate application of different encephalographic methods. As potentially beneficial for NIRS-BCIs, the phenomena of fast optical signals (FOS) are described, and some research on connectivity, including those based on NIRS, is covered. Some attention is paid to the perspective for future BCI’s construction using optogenetics.

1. Introduction

Most of us routinely manipulates of objects in the real world on an everyday basis. But, some people, for example, subjected to severe nervous system damage, may lack this necessary and basic ability. On the other hand, we may not limit our objects manipulations only with hands, but imagine some external actuator, not a part of a human body, but still controlled by one’s will and suitable for the specific target manipulation. Such a substitution or addition of a human’s abilities is certainly welcomed in the modern world. And although mentioned, the idea seems to be tempting, and its development was suspended for a long time because of difficulty in solving key component of the problem. Key component here is as precise as possible interpretation and transmitting commands born in human mind to external device of any sort. This task can be accomplished at some level by brain-computer interface technology (or BCIs for short). BCI problem was stated several decades ago. Since then, a lot of ground has been covered, yet a lot of discoveries are still ahead.

In present time, there are many “types” of BCIs build on different principles. Among many varieties of BCI, we are willing to emphasize on one, based on registration, interpretation, or classification of activity patterns of human cerebral cortex, registered by the means of one of the existing encephalographic methods, as the most popular. There are other paradigms, but only one is mentioned that allows to build BCI realization based on most encephalographic methods, such as electroencephalogram (EEG), magnetoencephalogram (MEG), fMRI, and so on (for example, see [1] of MEG realization). Not all of such realizations are of practical value due to usually bulky design. Additionally, we need to consider a family of new encephalographic methods, “fully” developed relatively recently, based on light propagation and interaction with living tissues. Light here must be considered in a wide sense—as radiation of different wavelengths (i.e., visible, infrared and—something in the middle—near infrared). As such, the near-infrared spectrometry method can be introduced, providing yet another way to achieve scientific and practical goals and get additional fundamental results on nervous system.

2. Theoretical bases of near-infrared spectroscopy in biological measurements

Measurements of a substance’s characteristics and its complex properties by the means of observing radiation passed through mentioned substance are not a new concept—X-ray imaging is common in medical practice nowadays. Similarly to imaging in visible and plain infrared range of electromagnetic (EM) spectrum, NIRS imaging method utilizes light of near-infrared range to obtain information on tissue properties. In present, there are widely used “optical” (mostly near-infrared—2100–2400 nm wave length range) pulsometer for measurements of heart rate and measurements of glucose level [2, 3, 4, 5]. Probing to obtain more “deep” biological parameters is beginning to be routinely implemented, e.g., measuring cerebral oxygenation during cardiac surgeries and in BCIs as a next step.

In general, all infrared range of EM spectrum can be conditionally divided in three subranges: near (λ = 740–2500 nm), middle (λ = 2500 nm–50 μm), and far (λ = 50–2000 μm). Relatively, low radiation absorption by tissues of human body is a distinctive feature of near-infrared range. As a result, for NIR light, one may observe bigger penetration depths—up to several centimeters (maximum 3–5 cm) [6, 7, 8, 9, 10]. Thus, some types of biological tissue, to a certain extent, are virtually transparent for named spectrum range. Nevertheless, during light propagation in tissues, elastic scattering processes are very strong and that sets limits to penetration depths on the other hand. In these conditions, beam weakening can be explained predominantly with isotropization and measurements on adult humans, for example, possible only in “reflected light.” “Permeate light” measurements are still possible in some cases. For instance, in infants, NIRS method can be applied to diagnose birth brain damage and detectors of NIRS equipment can be located on the opposite (in relation to light sources) side of infant head [11]. This is possible because of the higher optical transparency of infants’ bones, skin, and skull covers. One must admit, however, that case mentioned above is certainly not a “BCI application” for several reasons.

All those factors and frequently relatively large value of optical density of biological matter limits our ability to obtain sharp, contrast, and precise images of small (such as mm-size voxels in fMRI) probing volumes. We must, however, stress the fact that this is true only for today’s technique and technology.

Nevertheless, one can obtain the estimation of chromophore concentration distribution in a little bit large volumes (usually several cubic centimeters) and thus obtain image of chemicals distribution in targeted tissue [10, 12]. That is sufficient for many applications, including various types of BCIs.

Selection of chromophores (i.e., physiologically relevant substances) depends on light sources’ wavelength built in specific piece of equipment. Certain wavelength guarantees preferred interaction of light with chromatophore, structurally important for specific biological tissue. From the physiological point of view, among most important chromophores, one can name hemoglobin, glucose, myoglobin, and cytochrome-c-oxidase [13].

Overwhelming majority of NIRS medical devices, also applicable for BCIs, are designed for cerebral hemodynamics measurement purposes. These devices use 735–760 and 810–860 nm wavelength light sources for target deoxygenated hemoglobin and oxy-hemoglobin correspondingly and utilizes so-called continuous wave experimental paradigm, meaning measuring only power reduction of light beam at detectors, passing through highly scattering mediums, in comparison with initial power, generated by sources [14]. Time delays, phase, and frequency parameter changes are neglected, although there are exceptions [9, 15]. “Continuous wave” device allows building “images” or distribution maps of oxy- and deoxyhemoglobin concentration changes, and measures tissue oxygenation index (TOI) and normalized total hemoglobin index (nTHI) [16].

One must notice, how close such a hemodynamics activity observations make NIRS related to fMRI BOLD. Both methods measure blood oxygenation levels in their own way [6, 9].

NIRS devices itself, usually, consists of one and up to several tens of light sources and detectors. Each possible pair “source-detector” forms a “channel” informative or not; i.e., whether it passes through zone where intensive neural activity is occurring or not and whether emitted by the source-optode light fades away passing through tissues or not. Analog-digital converter (ADC) read detectors’ output and after filtering and preprocessing, usually conducted in the form of moving average filter to filter out heart beating, respiratory slow waves, and other nonphysiological artifacts and information about hemodynamics of volume, located “in between and a little bit in depth” in relation to selected source-detector optode position, ready for analysis. Inevitably on the path from source to detector, a portion of radiation will be lost in tissues and will never get in the detector. The other portion diffusely reflected from target volume will be weakened and get in to detector, where it can be quantitatively estimated [17, 18].

Layout of sources and detectors in a manner, when the distance between them is approximately 3 cm on the scalp surface, allows ensuring probing depth of 3–5 cm, sufficient for detecting an activation of human brain cortical areas, although this occurs indirectly, through metabolic effects. Intensive neural activity (usually well differs from background activity) is a process accompanied by oxygen delivery, its absorption by neurons, and evacuation of metabolic products [18, 19, 20]. Interpretation of such activity is practically basic of any BCI.

On the other hand, such a set up allows conducting EEG measurements in parallel with NIRS and in the direct proximity of NIRS channels. This is a key for building so-called hybrid BCIs. In principle, in hybrid BCIs, not only and not exclusively EEG can be applied, but EEG is most popular, simple, accessible, and technically usually does not disturb functioning of NIRS devices in any way and vice versa.

Among all methods to describe light propagation in matter, one must notice at least two. First, radiative transfer equation (RTE) (and its various approaches, like diffusion approach) is precise, but difficult to handle and almost impossible to solve for relatively complex problem types [21]. Second, on the other hand, is the phenomenological “modified Beer-Lambert law,” specifically “designed” for simplified description of light and radiation, in general propagation in turbid media [22, 23]. Brain and its covers from the NIR-light viewpoint can be considered as possessors of strong turbid properties. As such, the law states that weakening on initial beam depends on value of extinction coefficients, treated as constants, specific for every chemical or chromophore, like both oxy- and deoxyhemoglobin and other [24]. Due to individual scattering acts of spreading photons, their path from entry point to leaving point from turbid media is curvilinear. The so-called differential pathway factor (DPF) serve the purpose of description of this phenomenon in modified Beer-Lambert law. The law in question must be used with caution, and some attention must be paid to borders in which mentioned coefficients variate [25]. Neural tissue has extremely rich blood microcirculation [19], which makes modified Beer-Lambert law at the scale of those blood flows, desired for observation, look like rather a gross approach. In reality, NIR-light interaction process with hemoglobin molecules is much more complicated. For instance, hemoglobin itself has a complex molecular structure and thus influences light scattering and absorption (scattering particles form factor). In addition, hemoglobin molecules are not free blood elements, but are included in dynamically changing structured particles—erythrocytes, which chaotically move in blood stream and in turn have their own form features [20, 22, 26]. All these factors on the microlevel play important role in scattering and absorption processes, but on today’s observable scales allowed by modern equipment (1–5 cm in linear dimensions), they hardly can be called significant. This makes modified Beer-Lambert law applicable to described class of problems.

Tomography of cranium in NIR spectrum at present virtually is not widely implemented, especially in context of application of BCIs considered here. Among the reasons of such state of affairs are poor quality, images resolution, with demanding of opposite, and some other general restrictions. Although technology is itself promising and some pieces of optical tomography equipment exist, it can be explained by significant complexity and yet not sufficiently developed by mathematical methods of tomographic image processing, acquired in NIR wave length and as well by some technical difficulties. Nevertheless, some approaches to the problem’s solution exist (for example, see diffuse optical tomography or DOT [27, 28, 29, 30]).

Ongoing perfection process of scientific equipment, mathematical, and technical measurement methods allows now to detect some signal features, which were impossible to observe before. In particular, there is some research conducting on so to speak “portability” on NIRS, methods earlier developed for detecting evoked potentials essentially by the means of EEG. Among such researches, one may notice [31, 32, 33, 34, 35] reports on recording of so-called fast optical signals (FOS), which in terms of authors exactly are optical analogue of low latency EEG that evoked potentials. Reliable registration technique of such phenomenon will allow creating NIRS BCI based on evoked potential paradigm.

It is also necessary to mark fundamental research in the field. A good example of such makes research on connectivity, conducted with NIRS [36, 37]. It may give an additional data always needed during constructing applied equipment for specific tasks, for instance in clinics for poststroke or neurotrauma patients, where brain activity patterns suffer serious changes, and in healthy BCI’s users. Generally, research on connectivity of different cortex regions allows to detect not only groups of NIRS channels registering activity specific to the BCI task, but also allows to assume channels, say, with relatively high noise levels as informative, and thus produces more information to identify activity pattern and hence increases BCI system performance. This is also true regardless to modality of registration: NIRS, EEG, etc. Information on connectivity at the stage of development (testing and adaptation to some group of individuals or to a single user) of particular BCI system, can, for example, ease such calculatively difficult process as channels selection. This means inclusion in and/or exclusion from consideration, channels with informative or less informative data output. Otherwise, channels selection procedure require of execution of some usually complex searching algorithm, often time-consuming. Such process of relevant channel selection leads to increase BCI target state classifiability for it is sampling most informative channels, related to task at hand [38].

Resting-state connectivity research is also beneficial as it allows developing approaches and methods, needed for understanding of cortex regions’ interrelations [37]. Similarly, functional connectivity research allows understanding interrelations during some task performance. Registered activity patterns are not stable in their nature. They undergo a fluent and perpetual change. Thus, understanding of their spatial and temporal interrelations; i.e., connectivity, probably will allow predicting or at very least mark their changing borders during mental task execution. As a result, such consideration may allow increasing BCI functionality and stability in general. As a good firm method for connectivity research, one must name independent component analysis (ICA) [39, 40]. ICA capable to separate whole data to a set of spatially and temporarily independent components, thus, allow separating useful, informative data from noise and artifacts (such as oculographic artifacts produced by eyes movements) and increase BCI performance. It can be used for better or more precise relating electrical or metabolic, i.e., NIRS-registered activity to a specific cerebral cortex region and hence to a mental state. For example, ICA component with localization mainly focused in motor regions, by association can be viewed as a reflection of some motor act or motor act imagination [41]. Alternatively, localized in Broca area, ICA components signalize of acoustic perception [42]. Among others, processing methods may be useful methods for signal separation in spatially and temporal localized components or for specific process-driven components, one should mention empirical mode decomposition method and its modifications [43, 44].

In relation to connectivity problem in research, one must also notice usage of repetitive transcranial magnetic stimulation as a more invasive measure [45].

Speaking of fundamental research and role of optics in brain research, one must mention optogenetics—quickly developing field utilizing light (of NIR range in some cases) to activate and/or observe genetically modified neural tissues [46, 47].

It will not be far from reality to say that besides those interesting and rapidly developing fields, there are plenty of room for evolution in conventional NIRS-based and hybrid BCI. NIRS-based BCI and imaging technology have many advantages. Among them, one may name noninvasiveness, safety for users, portability and not in the last turn—fairly low price in comparison, for example, with fMRI setups.

3. Material’s analysis basics

In order to evaluate present day level of research in the field in question, one must in some way characterize ongoing work, information on which conveys for scientific community through existed papers. To accomplish such a task, one must bring in some form of classification for those objects. Thus, carried out by the general analysis of texts and analysis of keywords, we come up with publications’ hierarchical classification system. This classification system forms framework in which the further information is organized (see Table 1). This classification system was constructed by the analysis of available papers and their keywords, most often used in them. This system is “emergent, “i.e., was formed on precedents. In event of occurrence of publication, which is not related to one of already existing categories in hierarchy, the new category was reserved. Roots of a tree of hierarchy are the basic types of the publications, for example, whether article is the review, whether it represents the description of real experiments in ERS/ERD- or EP paradigms (event-related synchronization/desynchronization or evoked potentials), or it represents the description of a new unique method of research.

General type (subtype #1)	Experiment type (subtype #2)	Hybrid type (subtype #3)	Observation area (subtype #4)	Result type (subtype #5)	Additional type (subtype #6)
Reviews	Real BCI experiments description	NIRS only	Motor area	Review’s statements	Nothing special (reviews or methodical works)
Real functioning prototype of BCI over ERD paradigm	Classical experiments (finger tapping, memory tasks)	EEG only	Frontal & prefrontal area	Success!	Only one type of experiment described
Real functioning prototype of BCI over EP paradigm	No experiments described in detail	EEG + NIRS hybrid	Temporal area	Failed to achieve success	Several types of experiments considered (motor area, Broca area, etc.)
Unique method description	Nonclassical experiments	Other hybrids (not an EEG + NIRS)	Occipital area	Failed to achieve success, but there is a hope for the future	One type of experiment described only, and indirect fundamental conclusions were drawn by results
	FOS	EEG only, but some hybrid declared (+NIRS)	Something else (For instance, EEG over motor area, NIRS—over frontal area)		Robot control
		Just about everything in existence			Article about a choice of classifiers, usage of exotic classifiers, research directed on improvement of work of classifiers or algorithms of a choice of features
					Paper about technical aspects of NIRS of equipment designing
					Paper about experiment with a big deviation toward technical details of instrument and equipment
					Modeling of neural activity/hemodynamics

Table 1.

Hierarchical emergent classification system.

The second subtype was formed by types of described experiments, such as: real experiments on functioning model of BCI, the description of “classical” for NIRS experiments (for instance, various memory tasks, finger or a palm tapping, mental arithmetic), etc. The last category was related to “not classical” experiment, i.e., mental tasks or problems solving and were rarely or not published at all earlier. A separate category of the given subtype has been allocated as fast optical signals (FOS) or “optical evoke potentials” registration with or without signal averaging and so on. Researches on connectivity and optogenetics were not included in classification system and were treated separately, for they constitute merely a support value in the BCI context.

The third subtype was whether the considered one in article BCI was a “hybrid.” Along with separate EEG and NIRS measurements, there are papers on hybrids of EEG + NIRS and hybrids of other nature, for example NIRS + fMRI out there.

The fourth criterion of classification was the brain area over which measurements were conducted (for example, motor areas, frontal and prefrontal areas of human cerebral cortex, temporal area, an occipital cortex or something else, for instance EEG-measurements were conducted over motor areas, and NIRS measurements—over frontal and prefrontal areas).

The fifth criterion was whether there is an achievement of the goal or objective declared in the paper, i.e., whether work can be treated or treated by authors as success.

At last, the sixth subtype of hierarchical emergent classification was devoted to papers in which some special characteristics or unique feature were presented. Among such features were “how many types of experiments were described and conducted in work, reported by the paper” or whether BCI was used in work for management of the robot or external assistive actuator, etc.

Certainly, the given classification is not complete and a closed system. Also, it is not unique. However, its application is quite justified, since, at least, it is “to some extent” stable, i.e., robust and stable to the new information addition. It is possible to come to such a conclusion, having taken into consideration fact, that actually the classification structure has ceased to change after processing only of about 25 publications from the considered pool, chosen in a random manner. Also, it is necessary to notice that there were some categories which by their nature have not been presented in a considered pool of publications during analysis, but were included in (or better to say drawn into) classification system. For example, a class of articles telling about distribution of light propagated through a biological tissue and physic properties of this process, FOS, which only indirectly connected to BCI, and so on. Focus of such articles is displaced aside from practical applications of BCI to general physical laws on the basis of which any of NIRS devices are constructed and to fundamental research of nervous system. Nevertheless, some processed papers can be additionally classified with the last category, yet bearing in mind the fact of their distant relations to the main subject (i.e., distant from the roots branches) of classification system described.

4. Latest achievements in constructing BCI over NIRS and “hybrid” BCI

Research presented here provides with a detailed analysis of over 100 articles with various time of publishing. Total number of papers under consideration, possibly only indirectly connected with BCI subject, but connected to the related fields, was 178. Some of these papers used NIRS equipment in BCI applications, or for the comparative analysis and test for reliability of NIRS data versus fMRI-BOLD, or for other fundamental or technical purposes. The earliest publication that has come into the view was dated by 1993, the latest by 2017. Thus, this affirms that research of a human brain and nervous system utilizing NIRS is a new and fast developing field of science. Data confirm this statement as shown in Figure 1, where it is well visible, that the dynamics of number of publications on past years shows general growth. The year 2017 and beyond, of course, at the moment is not indicative and can be analyzed only in future in retrospective. Despite this, due to the increasing interest to NIRS, to BCI technology, and to related fields, the tendency will most certainly remain.

Figure 1.

Number of publications on “NIRS” by years.

Among types of experiments, leading position (by number) is occupied by papers in which experiments in both ERS/ERD—and EP paradigms were described. Among them, there was some number of papers about classical type experiments (i.e., accepted and well known in cognitive research, for instance mental arithmetic). Smaller, but nevertheless the noticeable share constitutes publications in which there were no detailed descriptions of experiments, or they were not conducted at all. Still smaller portion describes “nonclassical” experiments. See [48] for experiments, where motor cortex activity was recorded, in which movements done by the person peeling apples were described. Paper [49] describes BCI, one of which states the intention of the user to carry out “speech activity, “i.e., intention to say something. Such state was utilized as a sign of examinee’s wish to use the BCI system. Recognition accuracy of such mental state reaches 73%. Segments of NIRS records, in which the examinee pronounced words, can be distinguished from “rest state.”

Smallest party was formed by works published on an FOS, connectivity, and optogenetics, but last two were viewed more like as an addition here.

Despite the obvious advantages widely described in the scientific periodic literature, hybrid BCIs are just not exclusively popular. BCIs over NIRS-only are leading by quantity of publications (at least in considered pool till 2017 year). EEG + NIRS hybrids actually take only the second place. The third occupies BCI over EEG only, still containing a considerable quantity of references to work with NIRS. The remaining numbers are made by articles with description of BCI over EEG in which the future research in BCIs over NIRS was declared or with the comparison works of two technologies. Papers in which hybrid BCI over NIRS and some another “modality” (other than EEG) was described constitutes the smallest pool. Distribution of publications by quantity according to these properties is resulted in Figure 2.

Figure 2.

BCI hybrid type (papers types & count).

Areas over which usually NIRS optodes and/or EEG electrodes were situated correspond to type of a mental task, which is planned to work with in particular experiment.

So for, imagination of movements and tapping the motor areas corresponds, for cognitive tasks—frontal and prefrontal cortex areas, to visual recognition, naturally—an occipital cortex. Experiments with auditory system, for example, definition of the fact of audibility of a sound, pronouncing of internal speech, demand registration in temporal area and prefrontal cortex. All these ways of registration are traced in articles of a considered pool. In all those experiments, hybrid registration also took place. Usually, EEG recording was conducted over motor cortex, and NIRS recording was conducted over frontal and prefrontal areas. Most papers fell into this category—almost 44% from the general number of publications. Proceeding from experience of NIRS registration, it is supreme strategy of BCI construction since it gives possibility of simultaneous EEG registration of event-related desynchronization of brain rhythms, providing advantages in time resolution, and, reliability of NIRS cognitive answers. Preferences of registration areas, and, hence, to experiment types are illustrated in Figure 3.

Figure 3.

Registration areas (observation area type & paper count).

By this analysis, one can say that the most number of papers represents publications of work results upon only one—“pilot” experiment with NIRS or with hybrid modality. Equivalently, a big share constitutes the works, which do not have any outstanding properties (mainly its reviews, and also the works, results of which have descriptive or methodical properties). Third place by frequency of occurrence is divided by works in which different types of experiments were conducted and by works in which some fundamental conclusions were drawn from those results. Articles about mathematical features of BCI application or about classifiers of NIRS signals form another category. Here, papers on choice of signals’ features selection algorithms are also included (including automatic feature selection).

The remaining small part is made by articles devoted to technical aspects of BCI over NIRS devices and systems construction, control of assistive robots and external actuators, and artificial limbs. Hereby, NIRS begins to connect with innovation process in area of anthropomorphic mechanisms designing in addition to medicine. One also must note articles on hemodynamics studying. Such situation can be generalized by phrase: “NIRS only starts to extend to those directions.” Mentioned features of a considered pool of articles are illustrated in Figure 4 (for explanations of section names, see Table 1).

Figure 4.

Special features of considered papers.

5. Particular examples of BCI on NIRS and hybrid BCI

Let us now introduce description of some interesting specific examples on considered device type. One should see [50, 51, 52] to know about technical features and designs of NIRS devices and applications. It is possible to read about potential of commercial application of considered technologies in [53]. Paper [54] represents the review of the most recent achievements in the field of BCI uses in rehabilitation of poststroke patients with a stress on methodology, which pointed out that results of such rehabilitation led to improvement of the patients’ motor act. Some challenges and unresolved problems are also discussed. Efficiency of NIRS-BCI was compared in relation to implementation of visual feedback and placebo as a feedback. The first case has shown motility substantial improvement. In [55], NIRS research of Broca area for purpose of Chinese language vowels internal pronouncing recognition was conducted. Paper represents an attempt to introduce a measure of consciousnesses presence for the paralyzed patients and patients with a locked in syndrome (LIS). The study was constructed around concept of “consciousness index” and attempted to introduce objective numerical criterion of consciousness presence in the patient.

Work [56] was devoted to various types of mental task usage efficiency estimation in NIRS-BCI. Till now, studies in the field of NIRS-BCI have been focused on increase in accuracy of various mental tasks’ classification. In the given paper, search of mental states pairs which would be the best from the point of view of BCI customization, i.e., best classifiable in given conditions. In event-related hemodynamic response on eight channels, NIRS system had been recorded for various mental tasks in seven subjects. The beginning of a condition simulation for the examinee was designated by sounds followed by a 15-s pause for elimination “aftershock” in hemodynamics caused by sound. The pause between trails varied from 10 to 15 s for elimination of accustoming effect. Mental conditions or states under investigation were the following: LMI—left motor imagining (i.e., imagining of motor activity by the left arm), RMI—right motor imagining, FMI—foot motor imagining, SING—mental singing, SUB—sequential subtraction of small numbers, MUL—mental multiplication, ROT—mental rotation of a given three-dimensional (3-D) geometric figure, and WRT—mental character writing. Based on this approach, a set of “recommended” mental tasks with rather big classifications accuracy represents the following list: “mental multiplication,“ “mental rotation,“ and “motor imagination of the right hand.“ Pairs, formed of any two mental states from three listed above, show the highest mean accuracy of classification by utilizing method of linear discriminant analysis (LDA), most used in BCI applications, as authors stated. Authors of article expect that their results will be useful to reduction of time spent in research, technical, and methodical works, on definition of the best individually specific mental conditions, and their combinations. LDA, as classification algorithm, utilized three features: oxy-, deoxy-, and full hemoglobin. Naturally, choice of the best mental state was made upon the best distinguish ability of LDA algorithms. Also, it was stated that selection was influenced by searching on channel position dependencies, i.e., searching the most informative channel.

Often, the most natural to the examinee was the imagination of movement of their own hands and that is applied in many BCI systems based on an ERD/ERS paradigm. Work [57] represents research and the proof of fact that imagination of right and left hand movements produces distinguishable patterns of hemodynamic activity, which can be classified with the linear classifier and, thus, applicable in BCI systems. Conditions of experiments are rather a standard scheme nowadays: 10 healthy examinees, kinesthetic imagination, inflections of the left and right hands, and command indications were presented on the computer screen. Signals from right- and left-lateral motor cortex have been registered simultaneously with the use of multichannel NIRS system of a continuous wave type. Linear discriminant analysis was used as the classifier, which has allowed achieving an average on examinees classification accuracy of 73.35 and 83.0% for the right and left hands, accordingly.

In works [58, 59], classification of multichannel “NIRS patterns” is considered for states of motor imagination in order to construct BCI for people with the limited abilities. In work [59], the previous studies of other authors were mentioned in which other paradigms were also considered, for example, imagination of movements and recall of specific emotions, concentrations, and electric stimulation (reaction to it). The general scheme of classical experiments, information gathering, signal reception, choice and extraction of features, and a command/classification estimation is described. The detailed description of NIRS signals processing is resulted; trends, typical for NIRS signal, and ways to obviate them by means of averaging were also mentioned, and two types of averaging were applied. Some theoretical aspects based on NIRS were given, such as explanation of nature of changes in hemodynamics, accompanied by neurons activity. Problems of influence of respiratory cycles, Mayer’s waves, and heart responses described were also considered. As reliable methods of a noise reduction in NIRS, general linear model (GLM) and ICA were offered and used. Main objective of this paper was to find a difference between an “active” (i.e., “key-”) and “passive” mental states using NIRS.

It would be desirable to note also the work [60] in which working out hand-movement-direction-detection methods were described, proceeding from the NIRS hemodynamic response of the motor acts induced by the examinee and registered from motor area. The paper pursues the aim of perfection applications of assistive technologies. In total, 64 specially distributed optodes were used in experimental setup. Examinees had to produce “free” (i.e. with no constraints) hands’ movements’ in orthogonal directions (x- and y-directions in a horizontal plane). As features, changes in concentrations of oxy-, deoxyhemoglobin, their sum, and their difference were considered. Full delivery of oxygen and its evacuation have been calculated for local neural populations in the motor cortex, which underlaid the optode positions. The analysis of these signals has shown that such movements can be distinguished in space and time depending on a directions of movements. Thus, by analyzing of existed profiles of brain activation, it is possible to identify unique directions of movement of a hand in real time. This work can be considered as a precursor of BCI in which states can be changed slowly or “gradually”; thus, such a device can be termed gradual BCI.

The idea of rehabilitation with the use of the robotized artificial limb has been presented more than 10 years ago. Since then, their clinical reliability and efficiency have been reported in a considerable quantity of works (see [61, 62, 63]).

As one can note, branch of research connected with a robotics is a rapidly developing direction inside the basic stream of research in the field related to BCI. In [64], the idea of NIRS usage in robot management and possibility of its technical realization is considered. Actually, it is the review of the hardware and software complexes realizing the considered function.

Probably, the principal cause of frequent use of the evoked potentials is the simplicity of their registration and the simplicity of mathematical methods of their processing. But, this is not the case with other paradigms. On the other hand, as an example of elaborate methods for signal processing, one may consider the paper [65], where methods of chaotic dynamics were used in order to analyze fNIRS signal in BCI application. The aim was to recognize left and right hands’ motor imagery. Authors also used principal components analysis for the exclusion of high-frequency noise signals and mutual information criterion for some windows of signal. The aim of this paper is to investigate the chaotic property of hemoglobin changes of the blood within motor cortex by Lyapunov analysis. Such a result was achieved—the paper stated that NIRS signals have chaotic properties.

In [66], the noninvasive BCIs for use in neuroprosthesis are described. Works in the field of EEG indicate that the big accuracy and stability for such BCI are essential. The question of whether NIRS method is capable to improve BCI based on EEG was discussed in the paper. Both the methods have been applied simultaneously to record sensomotor rhythm. Research includes work with real and imagined movements. Results of work say that simultaneous EEG and NIRS records can essentially improve classification of motor imagination accuracy up to about 90%, on the average, having an improved accuracy indicator on 5% (p < 0.01) in comparison with the use of EEG only. Thus, the concept of hybrid BCI had been introduced. Nevertheless, the long delay of the hemodynamic answer can interfere with the improvement of the general accuracy of classification. Moreover, authors have found out that NIRS and EEG complement each other in sense of the information capacity. Therefore, those methods are applied together and are reliable multimodal imaging techniques. The 24-channel NIRS was used in which readings were averaged in time. As features, classifier LDA was used. It appeared to be that results of classification for EEG and NIRS correlate, but with some time lag that is connected with the NIRS nature.

The low time resolution is an essential lack of NIRS, and it is underlined in number of works, including with orientation to practical application of BCI. In [67], it is noticed that work of any BCI over NIRS is accompanied by a delay in several seconds that limits practical application of this system in a real world. Here, support vector machine (SVM) classification method for definition of a true mental state (finger tapping and rest) was used. Estimations of various spatial features’ efficiency, such as signal history, history of a gradient of a signal, and spatial distribution of oxy- and deoxyhemoglobin, are resulted. It is revealed that the delay for decoding of changes in a behavioral condition can be reduced to 50% (from 4.8 down to 2.4 s) that essentially improves indicators of BCI over NIRS. Results of classification in terms of accuracy that reached depending on a set of features applied were considered. Maximum achieved accuracy was 87%.

It is necessary to note the work [68] in which NIRS was used with fundamental purposes. The quality of the evoked potentials in the visual cortex was studied in dependence of age of the examinee. Two groups of examinees were considered by age about 21 and about 71 years. Both groups have shown increase in change of concentration of oxygenated hemoglobin and in reduction of deoxygenated hemoglobin during stimulation by a visual chess pattern. However, people in their 70s, on the average, gave more variable hemodynamic answer and often had comparable level of hemoglobin concentrations during time of stimulation versus rest condition—a base line. More young people had essentially high concentration of oxygenated and deoxygenated hemoglobin in each test without dependence from type of stimulation (p < 0.05). Average variability associated with effect of age has made 88% on oxygenated and 91% on deoxygenated hemoglobin. Experiments with visual stimuli has shown rapid falling of cortical hemodynamic answer with age, independently from stimulus’ parameters. Thus, authors do the conclusion that hemodynamic answer can be treated as age characteristic. Area V1 was studied. Results were analyzed with ANOVA.

Works [69, 70, 71, 72, 73] were devoted to cognitive research, namely “memory” group; besides in [74], methods of psychology for improvement of BCI performance were used. In [70], patterns of hemodynamic activity build-up estimation during perception of numbers by examinees (“mental arithmetic “task) were investigated. Researches used NIRS with high optode density in scalp positioning (348 channels), and standard mathematical methods of preliminary signals processing were applied. For features’ selection, applied algorithms were used, which governs the choice of the best channels or a best feature set. Thus structure of classification features in array of channels were reduced and varied for better BCI performance. One can name this as “greedy” algorithm; also, the combined algorithm with cutting out superfluous features is considered. The effective number of channels for classification, therefore, was reduced. Classification was made by SVM; and accuracy of distinction of “difficult mental arithmetic tasks” from a rest condition reached about 100%, “easy mental arithmetic tasks”—also about 100%. Check of reliability concluded the result.

Ref. [75] is a typical example of papers concerning methodical maintenance of NIRS experiments. Research focuses on developing a method for choosing effective training data sets for BCI training. In particular, BCI was considered for definition of concentration of examinees during experiment. This research also was devoted to integration of EEG and NIRS for their joint usage in “cognitive level” BCI applications. This term designates level of the interest of the examinee in experiment process. For construction of such interface, the paradigm based on P300 evoked potential was used. In this work, two experiments are presented: first—the mathematical task with NIRS-only measurements took place. Hybrid EEG + NIRS was used under EP paradigm (P300), and BCI system of type “ON-OFF machine” was constructed on its basis. Experiments had shown that with the use of NIRS, it is possible to differentiate concentration conditions (mental activity) and to distinguish them from level of mental rest. The second experiment, however, has revealed essential and statistically significant result only in EEG.

As for the struggle against artifacts, Ref. [76] brings it on a new level. The work sets as its purpose working out methods of artifacts cutout and implementation of those methods. Not only artifacts of true or “internal” physiological nature were considered (such as relatively stable heart beating waves, appearing on HbR and HbO concentrations’ changes curves), but also artifacts of physiological nature, caused by external factors. For example, there were considered changes in signal, conditioned by sudden attraction of attention to external objects or conditions. A number of external distracting factors, in particular, sharp sounds, distracting noise, are considered. The whole purpose of this work was to develop compensation for distracting factors, for NIRS-BCI to satisfy practical conditions. In the article, the system of filtration of the mentioned types of the artifacts based on hidden Markov chains (HMM) is considered. In [72], struggle against artifacts was carried out by medicamentous methods—by the local application of vasodilator drug.

One of the techniques directed to improvement of classifiers functioning within BCI is the utilization of so-called hybrid BCI in which signals about a condition of the examinee were received by means of at least two devices of various modalities [51, 66, 76, 77, 78, 79, 80, 81]. In [81], the concept of hybrid BCI in general—not only in sense of various modalities, but also in sense of various paradigms of record (ERD/EP), was introduced. The basic criteria of such device with reference to practice were given. The necessary quality standards were discussed. Various types of BCI, for example “the brain switch,“ were considered, basically focusing on an artificial hand limb, which operates by means of EP.

One can consider [82] as a good “head first” text about nervous processes studying techniques. Here, it is noticed that NIRS possesses the low time resolution and can even interfere with transitions between conditions. To cut out this lack, it was recommended to use it together with EEG. In general, the article affirms that the BCI based on NIRS are inexpensive, but does not show high-quality results. Various methods of preprocessing (CAR, SL, ICA, CSP, PCA, SVD, CSSP, Freq-Norm, LAT, LKF, and CSSD) are considered and estimated. Most often used are ICA, CAR, LS, PCA, CSP, and adaptive filtering. In this paper, the hybrid BCI on the basis of NIRS and MEG were also mentioned and also stated that they are disproportionately expensive, considering achievable results.

FOS responses [31, 32, 33, 34] do not directly relate to BCI, but nevertheless were considered for a possibility to build optical EP BCI based on such phenomena. FOS usage can increase “reaction time” of hemodynamic responses and hence can improve NIRS-FOS BCI performance. FOS registration demands extreme temporal resolution on NIRS equipment. Analogically, research of correlation with fMRI also difficult in that sense that fMRI-BOLD signal with “reception” time about 2.5 s has no sufficient time resolution to obtain information on “fast” reactions. Nevertheless, it is a very interesting problem, which can be solved only with the use of a multichannel and high-frequency NIRS with superb parameters.

In [33], authors demonstrate that optical methods can be used to detect rapid changes in functional connectivity during cognitive processes.

In experiments, described in [31], animal photos were presented to examinee, who needs to press button, when a picture was “well known.” That is so-called Go/no-Go paradigm. After the experiment was carried out, the map of correlations between EEG and FOS was built. In this paper, averaging of a signal was applied to allocate FOS responses. Thus, Ref. [31] stated that NIRS method is sensitive to hemodynamics of a brain (in the article, the term “slow signals” was used), and as well to fast responses of neural activity, or so-called fast optical signals or FOS for short. Registration of the FOS is difficult due to their nature and assumes low level of signal/noise ratio of experimental setup. Authors managed to register authentically FOS for 11 examinees, simultaneously with EEG registration.

At present, the necessity for more “fine” research on hemodynamic response’s features registered in the NIR modality in order to obtain more detailed description of its features from the point of view of physiology is obvious. In [83], experimental confirmation presented that there is a time difference between hemodynamic answers in oxy- and deoxyhemoglobin. Authors revealed that a time profile of hemoglobin-concentration-change curve is desynchronized; i.e., their levels do not fall, and their sum and the difference do not rise simultaneously. Such measurements can reveal a difference in brain activity patterns, while hand movements in directions left-right and forward-back considered in experiments were described in mentioned paper.

“Connectivity” research can be treated as another example of theoretical research, beneficial to BCI system improvement. NIRS data acquisition systems were introduced here just recently. For basic information about connectivity, see [84, 85].

For information about connectivity modeling and NIRS/EEG phantom construction for testing connectivity models, see [86]. Authors offer “testbed,” i.e., phantom for simultaneous NIRS/EEG recordings for rapid model testing for plausibility, although biological interpretation of connectivity is simplified. NIRS processing software “NAVI” as a part of testbed design was used [87, 88].

For comprehensive evaluation of utilizing NIRS in connectivity research, resting state, and functional as well and especially their dynamic characteristics, one should see [89]. Authors also point to disadvantage of NIRS usage in this type of studies—“low” penetration depth limits research only to cerebral cortex. On the other hand, this may allow excluding off the most low-level nervous functions from the consideration.

Ref. [90] is devoted to the research of functional connectivity of neuronal mechanisms underlying the reactions occurring during experiments in Go/No-Go paradigm, in relation to human development problem. The paper compares such reactions of children and adults examinees, and in particular, comes to the conclusion that motor-related activation did not differ between age groups. This means that at least within the limits of mentioned paradigm, there is virtually no difference in movement realizations and in their control patterns of cortical activity. This statement brings such patterns in position of more universal detectable target, independent from age of examinee.

This can be summarized as ongoing process of initial accumulation of facts and potentially useful information. Their implementation in BCI technology may lead to productive results.

Finally, let us point the reader to optogenetics research [46]. Papers on the subject do not directly relate to BCI technology directly, but more like to neurobiology as a whole. In the context of problem at hand, it makes sense to concentrate on papers, which aims to use some kind of NIRS or optical equipment.

In this way, Refs. [91, 92] describe methods, which utilize NIR radiation in confocal microscope in order of correction and control of neural cell growth. Further developments of this direction can be found in [93], where laser microsurgery of cellular membrane was described.

Ref. [47] was devoted to optogenetics methods of neural cells activation with high spatial precision by making those cells produce reaction when irradiated with light. This is achievable by the means of transfection of cell genome with gene constructs, which corresponds to channelrhodopsin 2 protein. Activation of transfected cells develops upon exposure to light of NIR range (860–1028 nm). Such manipulations may provide a researcher with method of external activation of nerve cells’, deeply situated in experimental animals’ brain and with high spatial resolution. Experiments in the field also features ability to “read” neurons’ activity patterns, induced on purpose. This can be achieved by variation of wavelength and intensity of excitatory radiation. This direction of thought is interesting because it represents fully optical method for monitoring, excitation, and detection of neural activity on the cellular level. One may hypothesize that in distant enough perspective, such a research may lead to obtaining new and more precise data on brain structure, functioning, and connections. Also, new generation of genetically coded voltage sensitive dyes can yield state-of-the-art methods and techniques of acquiring new data on functioning of neuron cells’ population level [94]. Alas, in their present state, optogenetics methods are very suitable and useful for fundamental research (usually not even in vivo), but they have not yet leveled up with practical demands of problem discussed here.

At last, it is necessary to notice that despite essential successes in the field of construction of BCI and machine interfaces (these are researches that generate about thousands of publications every year), progress in the field of creation of devices would allow patients with a full paralysis and “locked-in” syndrome to interact with an external world, which is, for now, possible to characterize only as moderate [95].

6. Conclusion

Objectively, the quantity of works in dynamics for more than 10 years grows, and the considered area of researches extends. Also, the subjects of papers in the field extend in more details, growing with new data.

The fact that the considerable part of papers in the field is reviews attracts attention. It can testify to the big number of scientific personnel, which began or begin at the moment the work on a considered problem. Also from this number, it is possible to draw indirect conclusions about quantity of scientific personnel.

The offered system of classification of publications was given here in hope that it will be useful for researchers’ choice of a direction in considered area and in order to aid more effective development of this direction as a whole.

There are a big number of articles in which only the near-infrared range spectrometry is used (without EEG and other means of neuroimaging) and also the ERD/ERS experimental paradigm most frequent and limited enough circle of cognitive effects was measured.

Researches, in which hybrid interfaces were used in comparison with aforementioned, are scarce. Even less papers concerning fast optical signals, fundamental physiological questions with the use of NIRS or connection of nervous activity, and optical properties of a living tissue and physical features of distribution of near-infrared range radiation in biological substances possess complex structure. Hybrid interfaces are focused mainly on conducting NIRS cognitive experiments (in which some cognitive effects or conditions exploited and responses were acquired with NIRS) in synchrony or in parallel with registration of motor activity with the EEG usage only. There are exceptionally few publications, concerning the technical details and improvement of characteristics of the spectrometer equipment. Technical features are not addressed in detail even in papers that were using devices of “own manufacture.” The field of NIRS devices with technical features, obviously, is assigned exclusively to manufacturers of the equipment. The insignificant part of papers, reviewed here, was devoted to the usage of NIRS/EEG BCI in one setup with assistive robotic devices, artificial limbs, and exoskeletons. Due to pressing nature of this matter and under a condition of extraordinary results, occurrences in the near future in this field, obviously the share of such publications, will only grow, probably, involving in the pool of papers on the technical details of such installations as a whole, including NIRS and robotics.