Introductory Chapter: Multilevel Representational Content in BCI Therapy - Extending Syntactic and Semantic Architectures

1. Introduction

The often expressed, but usually trite cliché about history duplicating fiction, nonetheless, reflects a deeper reality, about the human penchant for mystery behind modern technological marvels like brain-computer interfacing (BCI). Indeed, by combining the elusiveness of mental representations with unseen links to motor movements, BCI seemingly appealed to fictional accounts of unlimited mobility and teleportation. This mystique behind the mechanism has lessened somewhat since Jacque Vidal first coined the term in the 1970s [1]. Nevertheless, there remains ongoing excitement over therapeutic prospects that continue to drive interest in advancing BCI applications. Recent domains for example have included the rehabilitation of stroke victims, improved learning with artificial sensory feedback, and real-time control over fine motor movements, as well as the traditional mobilization of external devices usually associated with BCI.

As a strategic response to cognitive and CNS impairments, BCI is a theoretical outgrowth of several generations of endogenous devices that have as a prime strategy the direct replacement of lost neural function. Devices like pacemakers, cochlear implants, and vagal stimulators for example have all been successfully deployed in the relatively simpler anatomical substrate of sensorial and motor nerves where nerve transmission is largely unidirectional and composed of sequences of transmitting signals [2, 3]. In these applications the premise of administering therapy by replacing lost function has been limited to the restoration of signal-generating capacity [4]. Cochlear implants, for instance, transduce pitch vibrations that occur outside the ear to coded electrical signals within the cochlea in order to elicit action potentials in the frequency to place receptors that form the auditory nerve. Implants sited more internally are similarly designed but require the presence of a bidirectional interface for nerve signals, that is, one that can both receive electrical impulses from the intact nerve tissue and yield an equivalently spaced temporal output beyond the point of lesion. For these devices the replacement of action potentials is akin to the restoration of language syntax, here linearly related to temporal spiking sequences.

In building on these earlier devices, BCI has appropriated not only a similar premise but also a similar design and has, therefore, been largely sequence based and output driven. One consequence of this approach, for example, has been the search for an electrical feature that can be used in a fashion analogous to that of spiking in implant devices for peripheral nerves, such as the local field potential [5]. The premise of a temporally defined syntax is increasingly challenged, however, as knowledge of the anatomical recurrency of the brain is made manifest and the need to distinguish transmitted signals from a dominant background of noise becomes evident [6, 7, 8]. How the brain resolves the challenges posed by its complex operation is now thought to occur through the structuring of temporally independent and cyclically repetitive activity, that is, nonlinear dynamical elements that, while using spiking activity as a fundamental mechanistic feature, nonetheless relate only indirectly to it for communication. This is to say that the brain employs a very different type of coding syntax from that of the peripheral nerves. Such fundamentally distinct conditions for communicating information in turn require a different premise on which to base BCI therapy.

Qualitatively different premises for technology, in fact, are hardly new in science, often exerting profound influences on the subsequent course a field may take. The difference in the way information content is represented, when transitioning from peripheral to central nervous tissue resembles, for example, the transition made in computational programming architectures before and after the introduction of autonomous robotic design [9]. Attempts to endow field-situated robotic agents with autonomous mobility initially employed basic program planning formats where decision-making points were encoded in a series of steps telling the robot how to respond. In the field however, it became apparent that programmed contingencies were incapable of responding to the vast array of circumstances that could act as input variables. The need to accommodate this nearly unlimited variability resulted in a new approach to program planning that adopted a more interactive format where plans comprised only one among several input resources that autonomous robots could call upon [10]. In their formatting, these plans adopted a parallel architecture to accommodate multiple and simultaneous inputs. World information was thus assimilated and assembled as blocks of knowledge rather than temporally consecutive incidents.

An analogous shift is now needed for conceiving of BCI as a therapeutic medium, that is, as one that no longer entails only the restoring of signal transmission capacity but also the repairing of processes that structure basic functions. The direction in which this shift will need to evolve, therefore, is not merely in duplicating how the brain transmits information but also in a larger grasp of organismal design that is mediated globally. This becomes apparent when analogized to a linguistic hierarchy, which is used to structure multilevel representational content.

2. Syntactical generation for cognitive representations

In the distinct circumstances of brain cognition, this is apparent, first, at a syntactical level. Given a prevailing background of noise, signal preservation is prioritized, by use of recurrent connectivities that possess reciprocal inhibitory and excitatory contributions [11, 12, 13]. The emphasis of this anatomical architecture is to create circumstances of signal stability, to enable information-bearing signals to persist, thereby minimizing any corruption of information content. Hence, the physical architecture of the brain is anatomically configured to create patterns of cyclical flow, where the pattern of the cycle contains the information representation. Current estimates indicate that nearly 95% of brain neurons exhibit some form of feedback, with some zones noted for especially dense innervation [14, 15]. The physiological consequence of this arrangement is the generation of energetically favored zones where signal propagation is retained. Such persistent activity is a necessity to enable the brain to monitor ongoing bodily activity. However, persistent activity also makes brain operation susceptible to the pervasive influence of a noisy background. This susceptibility is overcome by structuring flow within energetically favorable zones, which minimizes the influence of noise and maximizes signal retention.

The dynamical motifs that are generated adapt spiking activity to exhibit a periodicity that frees syntactical expression from its temporal dependence. This periodicity fundamentally restructures the representation of information content. Hence, basic elements of syntax in the brain are not pulsed sequences, but blocked patterns.

Critically, these stabilized patterns are unique outcomes determined by the resolution of numerous physical forces; that is, they emerge from a high-dimensional state space within the global activity of the brain. They can therefore potentially assume an indefinite number of mathematical configurations that are defined by these physical circumstances. In a simple model, like a fixed point attractor, the rate of change of the attractor back to its original configuration is linearly related to the brain state, which is typically represented by a signal feature related to that state. More complex models entail the continuous and repetitive traversal of brain states by the attractor, which are described mathematically by a second derivative function, while still other models are complex and multiparameterized [16, 17]. The result of this variation is a significant expansion of syntactical range that is likely to substantially differ from that in peripheral nerves.

For BCI therapy the use of a different syntactical expression can be expected to have several consequences. The transposition of one syntax for another means, first, that an interfacial medium relying only on the original syntax introduces gaps in syntactical interpretation, with the immediate consequence of interpretive redundancy [18, 19]. That is, the mapping from one coding structure to the second is not one to one, but instead elicits multiple readouts. For a therapy premised on signal restoration, this overextends the intended output range and diminishes if not obviates therapeutic effectiveness. Hence, bidirectional interfacing premised on duplicating spiking sequences alone is likely to be inadequate for information transfer.

By acquiring temporal independence additionally, the manner in which syntactical elements are assembled is also altered. As cyclical patterns it is only through their modular assembly into larger architectures that they can yield representational variation, a feature that is seen, for instance, in cases of stable heteroclinic channels [20, 21]. Such variation is potentially amenable to exploitation for constructing extended symbolical architectures [22]. Rodrigues et al., for example, have shown that simple combinations of dynamical elements can be exploited to significantly expand the range of syntactical elements [23]. Using an attractor and repellor, they were able to demonstrate that networks generating these elements not only variably combine in specific ratios but also generalize from external inputs; that is, they learned to represent external input information. This is significant for relating the structure of the network in its connectivity features to a dynamical generation of symbolical structures that establish equivalency with external representation, that is, as codes that map content.

3. Feature-specific representation and semantic construction in BCI therapy

Yet, the generation of symbolical content is not the only consequence of acquiring temporal independence. The manner in which syntactical elements are assembled is also altered. As cyclical patterns it is only through their modular assemble into larger architectures that they can yield representational variation. Significantly, this change offers the immediacy of parallel-based representation. Hence, the role of syntax as representational sign, that is, as in a symbolical, Peircean coding, is itself transformed, linked instead to semantic elements that duplicate through self-organization feature-specific content of the external world [24]. For BCI, information extraction premised on symbolical articulation alone and not accounting for such modular assembly reduces structural content, diminishing the capacity for representation.

The complexity and magnitude of dynamical variation encountered in the state space of the brain, moreover, is a capacity amenable to environmental exigencies, in much the manner that field-situated robotic artifacts become amenable to local input by transferring responsivity from programmatic architectures to distributed processing. Here, sensorial input can elicit motor responsivity directly, structuring forms that directly respond to molding stimuli [25].

Some of the essence of this process of feature-specific duplication can be seen in the motor image, a covert action that is a representation of a non-executed action. The concept of the motor image itself evolved from several experimental legacies. Classical observations made by Lashley [26, 27] in a subject with a deafferented limb showed that humans, and animals, were able to generate actions without sensorial input, in contrast to the broadly assumed hypothesis prevalent in the nineteenth century. Later, experiments in monkeys showed that with deafferentation of spinal dorsal motor roots the animals nonetheless could execute pointing movements in all the phases of motion [28]. This indicated that the movement was predetermined centrally. How this was done and how executed became apparent in studies of ongoing motion. Held [29] observed that limb movements in such circumstances usually do not correspond to their expected trajectories, but entail a misreaching followed by progressive compensatory movements. To explain his finding he proposed Von Holst and Mittelstaedt [30] hypothesis that the command for the executed movement was stored as an efference copy, sent to the sensory cortex, where it was then compared with the actual movement undertaken so as to correct the misaligned motions. The experimental observation of misalignment and correction seen experimentally served as evidence of the memorized storage. A corollary of this hypothesis was that self-made motions could be contextualized to the individual who initiated the actions, a conclusion drawn by Frith in his comparator model [31, 32]. This is to say that the comprehension of the actions as those of one’s own was a necessary feature of movement; while the actions could be initiated without afferences, they nonetheless required them for motor cognitions in order to be understood as self-executed functions.

In continuous motions the sensory cues are coupled to motor execution in a mutually reciprocal and sustained process [33]. This is necessary, since as the body undergoes motion, its spatiotemporal position is continually changing and so also the sensory cues that reference it. While these cues entail contributions from all the senses, those having the greatest influence are of somatotopic origin due to their capacity to delimit the three-dimensional topological perimeter of the body [34]; this is also to say that it is necessary to know where the body is situated in space and time in order to know where next to move it. Linda Smith has described this as a point of criticality, analogous to a phase transition in a material substance, where the body is framed as a stable reference that is transitioning to a fluid and behaviorally flexible state [35].

The validity of this observation, and also as a demonstration of the need to frame the whole body, is well documented in the Piaget A not B error where a young infant continues to perseverate toward an object goal despite having been informed of its prior displacement. This error is explained by the delay in development of maturational processes of the brain needed to formulate and execute goal-directed actions [36]. From these, and other experimental studies, it is intuitive to see why the observed events and processes hypothesized by Von Holst and Mittelstaedt and by Frith require a “predictive processing” to engage motion [37]. Predictions are needed if one is to engage in actions, that is, actions that are intended to be carried out by the self, and are not merely passive responses to external events. Since all external contingencies cannot be known beforehand, like the field-situated autonomous artifact, neither can all consequences of the intended actions. The expectation of the action, its prediction, affords a first approximation that is open to correction that can structure the sequence that follows and that is energetically efficient.

This interplay between predictive actions, goals, and a holistic bodily sense point, further, to the presence, indeed need of mechanisms that involve a simulation of intended actions. Covert actions are thus a motor planning stage needed for subsequent motor execution. In this, the motor image is the key element. The construction of the image, its contextualization to the whole, and its traversal of stability flexibility bifurcations are all basic elements that entail feature duplications of the projected events. That is, they constitute semantic representation of objective events directly and not coded symbols of what is intended.

Hence, at deeper levels, linguistic primitives function as determinants for assimilating semantic content. That is, the assembly of these elements creates the semantic content of what is communicated through the action. For BCI therapy, this expands the role of therapy from interpretive assessment to the construction of semantic form, like that occurring when coupling sensorial input to the elicitation of motor imagery [25]. Here, semantic content is added by combining the specific motions that are undertaken to their semantic representation in the whole form of the individual, a process likely to the precision of motor processing primitives of the cerebellum [38, 39].

4. BCI therapy and biological design

Taken together, what is made apparent in analogizing from a linguistic perspective is the strategical implementation of multilevel representational content to structure goal-oriented motor actions. By extension, there is thus also the implicit subordination of this strategy to ontological demands, that is, actions undertaken for the good of the organism. Hence, they entail more than the execution of actions, a traditional objective performed in BCI, and so also include the formulation of organismal goals. For BCI therapy, accordingly, this formulation of representational content will be a critical objective for therapeutic strategy, encompassing diagnosis and therapy, and dictated at syntactic and semantic levels.

For the motor image, notably, it is apparent that representational content is articulated at multiple levels, built upon a dynamical syntax that acquires semantic content by binding representational, feature-specific, i.e., simulated, forms together. Distinguishing the level of functional disturbance therefore is an objective needed in order to administer therapy adequately. Yet, in decoding approaches that have evolved to date, the central technical concern is that of classification, that is, the mapping of a brain state in its activity patterns to an external object or event. Older techniques like mass univariate analysis sequentially evaluate brain regions for a specific activity at a specific location. Measuring covariance between multiple single units is thereby taken as a diagnostic feature of how select images are encoded, like the activation of long regions of the occipital cortex on presentation of a single object. Discerning the underlying structure of the representational content, therefore, remains unknown and an obstacle to focal BCI therapy [40].

In more recently developed multivariate classification approaches, previously determined activity patterns are linked to specific object features that can assess or predict the content of a specific activity. While this approach can be employed without the presentation of an object, many potential representations are left unclassifiable. These limitations have led to current model-based classification approaches that use models to predict patterns not elicited by training data. Such promising efforts seek to extract greater information content from patterned activity than obtained from linear mapping strategies alone. These latter strategies are likely to be strengthened by expanding the capacity to extract information content by combining deep neural learning with wavelet analysis, like that seen in Chapter 2. Hence, they can be expected to extrapolate from syntactical structure to simulated actions; that is, they will be better capable of extracting how meaning is formulated in the assembly of simulated executable sequences. Enlisting technological methods that can optimize distinctions between signal and noise, like that of Chapter 3, can be expected to further this capacity and particularly evident where discerning the syntactical expression of dynamical architectures is key, in order to communicate the motor image, as in Chapters 6, 7, and 8 of this text.

Crucially, issues of deciphering multilevel representational content and formulating semantic architectures for action-oriented goal seeking enter into primitive motor assembly levels, where, for example, the capacity for assimilating meaningful content is impaired. These will require new therapeutic paradigms where BCI may be one among several adjunct approaches used together to restore the functional modalities needed for simulated motor articulation. In practice, these paradigms will need to recreate the multilevel, brain-based operation that occurs in motor planning, like that used in sensory motor coupling. Models of such therapy, for example, are presented in Chapters 4 and 5 of the current volume.

5. Conclusion

Novel insights into the multilevel construction of representational content promise a new phase of BCI therapy, embracing not only the restoration of executable actions but also the formulation of the motor image and motor planning sequences. Built upon the fundamentally distinct syntactic and semantic architecture of dynamic cognition, new forms of therapy will undertake to simulate the brain’s approach to information transfer and to attain goal-directed planning. These will likely entail enhanced information extraction in classification and predictive technology, dynamically structured command and communication methodologies, and integrative, mixed-mode BCI approaches that can restructure motor semantics.