Neural recording and stimulation technologies.
Analytical methods are crucial to advance the field of brain sciences, and efficient and effective methods of data analysis are required. Early from the last century, the neural signals have been used in the engineering sphere to discover mechanisms by which neural activity is generated and corresponding behavior is produced. The function of the neural system was detected and studied using engineering methodologies, and meanwhile, the engineering methodologies helped to understand, repair, replace, enhance, or otherwise use the properties and functions of neural systems. The neural signals are recorded by advanced neural recording technologies, and the information is extracted to be used for the understanding of neural representations of behavior. The external devices are designed to assist signal acquisition, signal processing, or provide neural feedback to humans.
Since movement is an essential activity of daily life, some of the major applications of neural engineering in the field of motor control typically involve motor function compensation, movement restoration, rehabilitation, disorder detection, etc. A movement process is integrated and translated from the higher levels of the control system, and it involves a series of transmissions to multi-structure musculoskeletal coordination. The central nervous system (CNS) works as a computational controller structure in motor behavior characterization and reorganization . Multiple structures in the brain contribute to motor control by connecting, integrating, and coordinating the motor-related information. Each structure is utilized in formulating a motor command when a particular action is performed, and the CNS switches the command between multiple motor-related structures . The mechanism of coordination and cooperation of these structures in the brain could be determined as “black box” models, providing the neural representations of relationships between motor command input and predicted behavior output. These models may represent multiple brain structures, especially the regions with synaptic plasticity that can receive and send out information.
2. Neural recording and stimulation
Populations of neurons exhibit time-varying fluctuations in their aggregate activity. Currently, various invasive or noninvasive recordings exist that can record large amounts of spatial and temporal information from the human and nonhuman brain. In order to investigate how the motor-related information is generated, and what kind of patterns could be found in certain areas during a specific action, many engineering methodologies are applied.
In the human brain, neurons communicate with each other through connections known as synapses. Synapses can be electrical or chemical, and the excitatory or inhibitory nature of synapses contributes to information transmission—the influx and outflux of sodium and potassium causing the membrane potential to rise and fall rapidly. The rapid changes of membrane potentials are called spikes, which can be recorded by intercellular or extracellular recordings. Valuable information can be discovered from the rate of spikes, namely, the firing rate. The deep brain implanted electrodes allow the recording from individual neurons and can present significant results in awake animals but not in humans. Multielectrode arrays can record the voltage oscillations from multiple neurons. Simultaneously recording from a large population of local neurons increases spatial resolution benefits to the extraction of complex information in contrast with single-unit recordings. The aforementioned invasive recording technologies provide considerably less vulnerability to artifacts and relevantly higher resolution and larger amplitudes (voltages), and thereby the performance relies much more on the technologies of electrodes. However, there are several limitations of these invasive recording technologies including restricted to clinical environments and the risks of surgery and implantations.
As an alternative to the constrained invasive technologies, several noninvasive recording technologies such as electroencephalography and magnetoencephalography have been used in human studies. Advanced computational algorithms promise to promote signal processing and signal filtering; thus, more and more noninvasive recording technologies are being considered in human studies. Some techniques record neuronal potentials from the scalp, and such recordings capture the population activity of thousands of neurons depending on the level of recording. Multiple layers restrict information transmission from the cerebral cortex to the scalp leading to lower amplitudes of the signal and lower spatial resolution. Additionally, the electrodes are sensitive to the surrounding interferences like eye movements, facial movements, chewing, swallowing, etc. Therefore, it is necessary to apply robust and efficient signal processing technologies to amplify the neural activity and filter out the ambient and transducer noise, thus improving the signal-to-noise ratio.
Under noninvasive technologies, there are imaging methods that focus on the metabolic activity in the brain rather than the activity of neurons or the population of neurons. When performing a specific task, the activation of the brain neurons is enhanced and thereby more oxygen is required and absorbed from surrounded blood vessels. An increased inflow and higher oxygenated level can be detected. This hemodynamic response is comparatively slow that it reaches the peak in a few seconds and takes a longer time to fall back to the original level. Therefore, this kind of recording technology provides good spatial resolution but very poor temporal resolution.
In addition to neural recording technologies, there are also neural stimulation technologies that are used in clinical treatments (cochlear implants and deep brain stimulators) and emerging neuroprosthetics. This involves giving electrical or magnetic stimulation to a particular region of the brain to mimic sensorimotor feedback. Most recording electrodes can also be used for stimulations. Brain stimulations have proven effective in clinical treatments. These methodologies also involve the use of signal processing methodologies in determining ideal stimulation patterns. Table 1 summarizes neural recording and stimulation technologies.
|Electrical recordings||Single-unit recordings (spikes)||Microelectrodes insert into neurons or placed between adjacent neurons|
|Local field potential (LFP) recordings||Multielectrode arrays placed inside the brain|
|Electrocorticography (ECoG)||Implanted electrodes placed on the upper layers of cerebral cortex|
|Electroencephalography (EEG)||Electrodes placed on the surface of the scalp|
|Magnetic recordings||Magnetoencephalography (MEG)||Measures the magnetic field produced by electrical activity in the brain|
|Neuroimaging recordings||Functional near-infrared recordings (fNIR)||Detects near-infrared light absorbance of hemoglobin in the blood with/without oxygen|
|Functional magnetic resonance imaging (fMRI)||Measures the changes in oxygenated and deoxygenated hemoglobin concentrations in the blood|
|Positron emission tomography (PET)||Detects the radioactive compound as a result of metabolic activity caused by brain activity|
|Brain stimulations||Transcranial magnetic stimulation (TMS)||Current-passed coil of wire paced next to the skull to produce a rapidly change magnetic field|
|Transcranial direct current stimulation (tDCS)||Stimulates specific parts of the brain using low-intensity direct electrical currents|
|Deep brain stimulation (DBS)||Electrodes are implanted in target regions of the brain|
3. Neural signal processing
3.1 Spike sorting
Spikes from closer neurons produce larger amplitude deflections in the recorded signal. The goal of signal processing methods for such an input signal is to reliably isolate and extract the spikes being emitted by a single neuron per recording electrode. This procedure is usually called spike sorting. The simplest spike sorting method is to classify spikes according to their peak amplitude. Sometimes, the peak amplitudes may be the same for different neurons, making the method not feasible. A better approach is the window discriminator method in which the experimenter visually examines the data and places windows on aligned recordings of spikes of the same shape. The recent trend has been toward clustering spikes automatically into groups based on shape, where each group corresponds to spikes from one neuron. The shape of a spike is characterized by features extracted using wavelets or dimensionality reduction techniques.
3.2 Temporal and spatial feature extraction
Key temporal and spatial features can represent and help us understand the neural activity from the underlying oscillations. The neural signals recorded from the brain are typically mixture potentials resulting from network activity of a large population of neurons around the local neighborhood. Thus, applying appropriate feature extraction methods can isolate and extract significant features in both temporal and spatial domains.
3.2.1 Spatial filtering
For some methods that record brain signals using multi-electrodes, the signals are recorded from multiple regions of the brain. With the large variance of global noise, the local signals appear diminished. Therefore, spatial filtering or re-referencing methods are applied to enhance the local activity and filter out the common noise. For individual electrodes, the averaged activity from surrounded electrodes (Laplacian filtering) or from global electrodes (common averaged referencing) is subtracted. Spatial filtering methods can also be used to estimate the variance of the neural data.
3.2.2 Temporal analysis
The quality of the recorded brain signals primarily depends on recording techniques. However, the recorded time-series signals contain lots of noise that can be filtered using time-domain filtering methods. Numerous filtering techniques like moving average smoothing, exponential smoothing, etc. are used to preprocess raw signals in the time domain.
In addition to filtering, temporal analysis can also be used to extract significant features that represent behavior. These significant features can be extracted out from a series of time signals using computational models. Some neural signals tend to be correlated over time, and thus, the following time samples are possible to be predicted based on the previous samples using autoregressive models (for stationary signals) or adaptive autoregressive models (for nonstationary signals). Such methods depend on the model built up from the characteristic internal relationships between the previous signal samples and the subsequent samples. The coefficients of the model can be considered as neural features for the subsequent pattern recognition or classification procedure utilized for real-time decoding or estimation.
3.2.3 Frequency analysis
While temporal analysis methods are useful, there are some signals for which these methods may not result in extracting meaningful features. For example, noninvasive methods such as EEG are based on signals that reflect the activity of several thousands of neurons. Poor spatial and temporal resolution challenges the feature extraction in the time domain. The recorded signal thus can capture only the correlated activities of large populations of neurons, such as oscillatory activity.
The intrinsic property of the brain signals is neuronal oscillations . Theoretically, these oscillations can be decomposed with a set of basis functions, such as sinusoid functions using Fourier transform (FT) for periodic signals. For each cycle, the amplitude, the period, and the waveform symmetry are measured and oscillatory bursts are algorithmically identified, allowing us to investigate the variability of oscillatory features within and between bursts. Usually, for neural signals, short-time Fourier transform (STFT) provides better results by performing FT with sliding short-time windows. For the nonperiodic signals, the wavelet transform is applied for signal decomposition. A variety of scaled and finite-length waveforms can be selected according to the shape of the raw neural signals. The wavelet coefficients sometimes contain unique information which can be considered as neural features. Additionally, the power spectrum of neural signals usually reflects lots is important information, such as power spectral density (PSD).
3.2.4 Time-frequency analysis
By combining the advantages of temporal and frequency analyses, the researchers realized the power of time-frequency analysis. As an example, using decomposition techniques, a signal can be decomposed into intrinsic mode functions (IMF) and instantaneous frequencies over time can be obtained by applying methods such as Hilbert spectral analysis. The most significant advantage of this technique is that the nonlinear, nonstationary recorded neural signals can be transformed into linear and stationary components. These components are usually physically meaningful since the special features are localized in their instantaneous frequencies and represent meaningful behavioral information in the time-frequency domain. Time-frequency analysis is extensively implemented in neural signal processing since the individual analysis in the time or frequency domain comes with respective disadvantages, and time-frequency analysis trades off time and frequency resolution to get the best representation of the signals. Other techniques such as spectrogram and STFT are most performed by segmenting a signal into short periods and estimating the spectrum over sliding windows.
3.3 Dimensionality reduction
A critical procedure in neural signal processing is to reduce the high dimensionality of the recorded neural data. These data could be brain images, multi-electrode signals, network potentials, or high-dimensional neural features. Several algorithms can be applied linearly or nonlinearly to preserve the most useful components and remove redundancies. Principal component analysis (PCA) is to find the direction of maximum variance and thereby build the principal components (weighted linear combinations) based on the observed variance. Linear discriminant analysis (LDA) performs similar to PCA but tends to minimize the variance within a group of neural data and maximize the distance between groups of neural data. Thus, PCA is described as an unsupervised algorithm implemented for feature extraction, and LDA is described as a supervised algorithm that uses training based on labels for groups of data. Other methods that are used most often are CCA and ICA. Canonical correlation analysis (CCA) is yet another method for exploring the relationships between two multivariate sets of variables allowing us to summarize the relationships into a lesser number of variables while preserving the essential features of the relationships. Independent component analysis (ICA) is a blind source separation method rather than a dimensionality reduction method. Neural signals consist of recordings of potentials that are presumably generated by mixing some underlying components of brain activity. ICA can theoretically isolate these underlying components of brain activity by computing independent components. Additionally, ICA can also be used as a filtering method to remove signal artifacts generated by eye blinks or other artifacts in EEG signals.
3.4 Machine learning algorithms
Machine learning or deep learning algorithms have become increasingly popular and are being implemented in many fields. They can be broadly divided into unsupervised learning and supervised learning. Unsupervised learning methods aim to extract hidden structures within the neural data, commonly used for feature extraction, pattern recognition, clustering, and dimensionality reduction. Supervising learning methods train the neural data using underlying functions to map to a given output and automatically discover the relationships between input data and output labels. The most common applications of supervised learning are classification and regression. Quantitative models of machine learning algorithms provide incredibly powerful implementations in neuroscience. Some traditional methods such as LDA, PCA, and support vector machine (SVM) are also regarded as machine learning algorithms. Other algorithms (neural networks, autoencoders, and logistic regression) train batches of input data using basis transformation function to match the output adaptively.
4. Applications of neural signal processing
Neuroprostheses, neurostimulators, or human-machine interfaces are devices that record from or stimulate the brain to help individuals with neurological disorders, restore their lost function, and thereby improve their quality of life. Neural signal processing methodologies are used extensively in all these applications.
Neurostimulators that have demonstrated decades of success are cochlear implants that are designed for those who have dysfunctional conduction of sound waves from the eardrum to the cochlea. These implants can also help elderly individuals who have age-related hearing loss . There is an external speech processor to capture and convert the sound from the surrounded environment to digital signals. The internal implants turn the digital signals into electrical signals to stimulate the hearing nerve by the electrodes inside the cochlea. Once the brain receives the signals, one can hear and interpret the sound.
Another successful neurostimulator is the deep brain stimulator (DBS) system used for individuals with Parkinson’s disease. The DBS has been available as a reliable treatment for decades for individuals with Parkinson’s disease. The implanted impulse generator placed under the collarbone provides continuous electrical impulses by giving a certain frequency of stimulation to the subthalamic nucleus and makes it possible to minimize the uncontrolled tremors. During the DBS surgery, electrodes are inserted into a targeted area of the brain, and the whole procedure is monitored and recorded using MRI. After the treatment, symptomatic improvement was durable for at least 10 years .
4.2 Neuroprostheses or human-machine interfaces (HMIs)
Stroke, spinal cord injury, and traumatic brain injury may lead to long-term disability, and an increased number of individuals are suffering from severe motor impairments, resulting in loss of independence in their daily life. Recovery of motor function is crucial in order to perform activities of daily living. Human-machine interfaces (HMIs) can enable dexterous control of exoskeletons that could be used as a rehabilitative device or an assistive device to restore lost motor functions poststroke or spinal cord lesions [6, 7, 8], thus promoting long-lasting improvements in motor function of individuals with movement disorders.
Additionally, significant applications in neural engineering are HMI-based systems to restore or compensate the lost limb functions for individuals with amputation or paralysis. Cortical control of prosthetics has been studied both in animals  and humans [10, 11]. Movement-related cortical potentials used to assess cortical activation patterns provide interesting information, as they are associated with the planning and execution of voluntary movements. Recently, HMI-based research has stressed on the development of algorithms for movement decoding using noninvasive neural recordings [12, 13, 14]. In order to understand neural intent before or during the movement, it is necessary to extract the characteristics accurately using efficient algorithms. The adaptability and reliability over the long-term are current challenges that are being addressed using advanced and adaptive signal processing methodologies. Months to years of training are essential to operate prosthetic or exoskeleton skillfully. This training time can possibly be reduced by increasing the burden on machine learning algorithms that are currently being addressed by advanced signal processing methods.
4.3 Neurological disorders
Epilepsy is a common neurological disorder characterized by an enduring predisposition to generate epileptic seizures . These seizures may cause disturbances in movement, loss of control of bowel or bladder function, loss of consciousness, or other disturbances in cognitive functions. Currently, the signal processing algorithms can detect ongoing seizures and provide clinicians with detailed information such as localization of seizure foci useful for the treatment of epilepsy. The ability to detect seizures rapidly and accurately could promote therapies aimed at rapid treatment of seizures.
Skilled neurophysiologists visually examine the neural signals and detect epilepsy. Apart from the single-channel signals, other contextual information such as spatial and temporal data are vital to neurophysiologists for recognizing spikes [16, 17]. Currently, the epileptic seizures can be detected and predicted from EEG or ECoG signals by extracting the hidden features using machine learning algorithms [15, 18, 19, 20].
Neural signal processing has become an increasingly important tool in neuroscience and neural engineering. This chapter provides a general overview of the widely implemented neural recording and stimulation technologies, neural processing methodologies, and how these techniques can be practically executed in some practical applications. The understanding of the neural representations of human behavior in the brain can be significantly enhanced using neural signal processing methodologies. With the advances in neural recording techniques and parallel advances in neural signal processing, we believe that several unanswered questions and challenges in neuroscience and neural engineering will be solved in the near future.
Wonham W. The internal model principle of control theory. Automatica. 1976; 12(5):457-465
Kawato M, Wolpert D. Internal models for motor control. In: Sensory Guidance of Movement; 1998. pp. 291-307
Fransen AMM, van Ede F, Maris E. Identifying neuronal oscillations using rhythmicity. NeuroImage. 2015; 118:256-267
Cabral Jr F, Pinna MH, Alves RD, Malerbi AFDS, Bento RF. Cochlear implantation and single-sided deafness: A systematic review of the literature. International Archives of Otorhinolaryngology. 2014; 20(1):69-75
Hitti FL, Ramayya AG, Mcshane BJ, Yang AI, Vaughan KA, Baltuch GH. Long-term outcomes following deep brain stimulation for Parkinson’s disease. Journal of Neurosurgery. 2019; 132(1):1-6
Monge-Pereira E, Ibañez-Pereda J, Alguacil-Diego IM, Serrano JI, Spottorno-Rubio MP, Molina-Rueda F. Use of electroencephalography brain-computer interface systems as a rehabilitative approach for upper limb function after a stroke: A systematic review. PM & R: The Journal of Injury, Function, and Rehabilitation. 2017; 9(9):918-932
Frolov AA, Mokienko O, Lyukmanov R, Biryukova E, Kotov S, Turbina L, et al. Post-stroke rehabilitation training with a motor-imagery-based brain-computer interface (BCI)-controlled hand exoskeleton: A randomized controlled multicenter trial. Frontiers in Neuroscience. 2017; 11(JUL)
Burns MK, Pei D, Vinjamuri R. Myoelectric control of a soft hand exoskeleton using kinematic synergies. IEEE Transactions on Biomedical Circuits and Systems. 2019; 13(6):1351-1361
Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for self-feeding. Nature. 2008; 453(7198):1098-1101
Yanagisawa TT, Hirata M, Saitoh Y, Kishima H, Matsushita K, Goto T, et al. Electrocorticographic control of a prosthetic arm in paralyzed patients. Annals of Neurology. 2012; 71(3):353-361
Bright D, Nair A, Salvekar D, Bhisikar S. EEG-based brain controlled prosthetic arm. Conference on Advances in Signal Processing (CASP) 2016. 2016.pp. 479-483
Pei D, Burns M, Chandramouli R, Vinjamuri R. Decoding asynchronous reaching in electroencephalography using stacked autoencoders. IEEE Access. 2018; 6:52889-52898
Pei D, Patel V, Burns M, Chandramouli R, Vinjamuri R. Neural decoding of synergy-based hand movements using electroencephalography. IEEE Access. 2019; 7:18155-18163
Patel V, Burns M, Pei D, Vinjamuri R. Decoding synergy-based hand movements using electroencephalography. In: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2018. pp. 4816-4819
Elger CE, Hoppe C. Diagnostic challenges in epilepsy: Seizure under-reporting and seizure detection. Lancet Neurology. 2018; 17(3):279-288
James CJ, Jones RD, Bones PJ, Carroll GJ. Detection of epileptiform discharges in the EEG by a hybrid system comprising mimetic, self-organized artificial neural network, and fuzzy logic stages. Clinical Neurophysiology. 1999; 110(12):2049-2063
Tzallas AT, Oikonomou VP, Fotiadis DI. Epileptic spike detection using a Kalman filter based approach. Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2006. pp. 501-504
Ramgopal S, Thome-Souza S, Jackson M, Kadish NE, Fernández IS, Klehm J, et al. Seizure detection, seizure prediction, and closed-loop warning systems in epilepsy. Epilepsy & Behavior. 2014; 37:291-307
Naghsh-Nilchi AR, Aghashahi M. Epilepsy seizure detection using eigen-system spectral estimation and multiple layer perceptron neural network. Biomedical Signal Processing and Control. 2010; 5(2):147-157
Chakrabarti S, Swetapadma A, Pattnaik PK. A Review on Epileptic Seizure Detection and Prediction Using Soft Computing Techniques. Vol. 374. Springer International Publishing; 2019