Open Access is an initiative that aims to make scientific research freely available to all. To date our community has made over 100 million downloads. It’s based on principles of collaboration, unobstructed discovery, and, most importantly, scientific progression. As PhD students, we found it difficult to access the research we needed, so we decided to create a new Open Access publisher that levels the playing field for scientists across the world. How? By making research easy to access, and puts the academic needs of the researchers before the business interests of publishers.
We are a community of more than 103,000 authors and editors from 3,291 institutions spanning 160 countries, including Nobel Prize winners and some of the world’s most-cited researchers. Publishing on IntechOpen allows authors to earn citations and find new collaborators, meaning more people see your work not only from your own field of study, but from other related fields too.
To purchase hard copies of this book, please contact the representative in India:
CBS Publishers & Distributors Pvt. Ltd.
www.cbspd.com
|
customercare@cbspd.com
Technological integration of Artificial Intelligence (AI) and machine learning in the Prosthetic and Orthotic industry and in the field of assistive technology has become boon for the Persons with Disabilities. The concept of neural network has been used by the leading manufacturers of rehabilitation aids for simulating various anatomical and biomechanical functions of the lost parts of the human body. The involvement of human interaction with various agents’ i.e. electronic circuitry, software, robotics, etc. has made a revolutionary impact in the rehabilitation field to develop devices like Bionic leg, mind or thought control prosthesis and exoskeletons. Application of Artificial Intelligence and robotics technology has a huge impact in achieving independent mobility and enhances the quality of life in Persons with Disabilities (PwDs).
Pt. Deendayal Upadhyaya National Institute for Persons with Physical Disabilities (Divyangjan), India
Rajesh Kumar Das
Artificial Limbs Manufacturing Corporation of India, India
*Address all correspondence to: smitank7@gmail.com
1. Introduction
Human is the most intelligent creature in the planet for their brain power and neural network. The human brain is extremely complex with more than 80 billion neurons and trillion of connections [1]. Simulation scales can array from molecular and genetic expressions to compartment models of subcellular volumes and individual neurons to local networks and system models [2]. Deep Neural Network nodes are an over simplification of how brain synapses work. Signal transmission in the brain is dominated by chemical synapses, which release chemical substances and neurotransmitters to convert electrical signals via voltage-gated ion channels at the presynaptic cleft into post-synaptic activity. The type of neurotransmitter characterizes whether a synapse facilitates signal transmission (excitatory role) or prevents it (inhibitory role). Currently, there are tenths of known neurotransmitters, whereas new ones continuously emerge with varying functional roles. Furthermore, dynamic synaptic adaptations, which affect the strength of a synapse, occur in response to the frequency and magnitude of the presynaptic signal and reflect complex learning/memory functions, (Spike time dependent plasticity) [3, 4]. Recently, evidence has found that surrounding cells, such as glia cells that are primarily involved in ‘feeding’ the neurons, can also affect their function via the release of neurotransmitters. This new vision of “tripartite synapses,” composed of perisynaptic glia in addition to pre- and postsynaptic terminals certainly makes this one of the most exciting discoveries in current neurobiology [5]. The functional loss due to amputation, spinal cord injury, brachial plexus injury or traumatic brain injury resulting loss of connection from brain to extremity and those residual/weakened extremities are not able to function as of healthy/intact limb. These lost structure & functions of extremities were being replaced by fitment of prosthetics and orthotic devices or rehabilitation aids. The conventional prosthesis which is a mechanical device only provide the basic function, similarly Orthosis provides the support to weaken parts not fully with out completely mimicking the lost section. The concept of biomechatronic is a sub-discipline of mechatronics. It is related to develop mechatronics systems which assist or restore to human body gave the prosthetics and orthotics concept to a new direction. A biomechatronic system has four units: Biosensors, Mechanical Sensors, Controller, and Actuator [6]. Biosensors detect intentions of human using biological reactions coming from nervous or muscle system. The controller acts as a translator among biological and electronic structures, and also monitors the activities of the biomechatronic device. Mechanical sensors measure data about the biomechatronic device and relay to the biosensor or controller. The actuator is an artificial muscle (robot mechanism) that produces force or movement to aid or replace native human body function. The areas of use of biomechatronic are orthotics, prosthesis, exoskeleton and rehabilitation robots, and neuroprosthesis. Robots are the intelligent devices that easily fulfill the requirements of cyclic movements in rehabilitation, better control over introduced forces; accurately reproduce required forces in repetitive exercises and more precise in different situations [7].
2. History of artificial intelligence (AI) in prosthetics and orthotics
The first intelligent prosthesis developed by Chas. A. Blatchford & Sons, Ltd. in 1993 [8] and the improved version in 1995 named as Intelligent Prosthesis Plus [9] Blatchford in 1998 developed Adaptive prosthesis combining three actuation mechanisms of hydraulic, pneumatics and microprocessor. The fully microprocessor control knee developed in 1997 by Ottobock known as C-leg [10]. Rheo knee and power knee both developed by OSSUR in 2005 and 2006 subsequently uses onboard AI mechanism [11]. In late 2011 Ossur introduced the world first bionic leg with robotics mechanism known as “symbionic leg” and this time period the Genium X3 was lunched by Ottobock which allow backward walking and provide intuitive and natural motion during gait cycle [12]. On 2015 Blatchford group introduced Linx the world’s first fully integrated limb has seven sensor and four CPU throughout the body of Leg. It allows coordination and synchronization of knee and ankle joint by sensing and analyzing data on user movement, activities, environment and terrain making standing up or walking on ramp more natural. The iwalk BiOM is the world first bionic foot with calf system commercially available from 2011 developed by Dr. Hugh Herr uses robotics mechanism to replicate the function of muscle and tendon with proprietary algorithm [13, 14]. The commercially available microprocessor control foot are Meridium (OttoBock, Germany), Elan (Blatchford, UK), Pro-prio (Össur, Iceland), Triton Smart Ankle (hereinafter referred as TSA) (Otto Bock, Germany), and Raize (Fil-lauer, USA) etc. available from 2011 in the market [15].
The first commercially available bionic hand lunched by Touch bionics in 2007 with individually powered digits and thumb has a choice of grip. The design again embedded with rotating thumb known as i- limb ultra and i- limb revolution designs implanted with Biosim and My i- limb app [16]. Bebionic was commercially available in the market in 2010 manufactured by RSL steeper and lunched by World congress, in 2017 it owned by Ottobock. Bebionic3 allows 14 different hold with two thumb position [17]. Michelangelo hand is the fully articulated robotic hand with electronically actuated thumb first fitted in the year 2010 developed by Ottobock [18]. The concept of brain computer interface (BCI) implemented neuroprosthesis or mind control prosthesis which can able to recognize the real time data and a gadget to get nearly normal function is the demand of the day. The EEG based mind controlled smart prosthetic arm was presented in 2016 IEEE conference but till now this concept is not commercialized [19]. Researchers are on the path of developing more complex devices that mimic the natural brain by implementing artificial intelligence to on board computer that read and reply the nerve signal that transmitted to robotic prosthesis and Orthosis which enhance the function of amputated and paralyzed part of the body.
Machine learning contains elements of mathematics, statistics, and computer science, which is helping to drive advances in the development of artificial intelligence. It is the study of computer algorithms which expands and develops through experiences. This is a subset of AI as shown in Figure 1. The ML algorithm methods generally categorized two types supervised and unsupervised learning [20, 21].
Figure 1.
Relationship between artificial intelligence (AI), machine learning (ML) and deep learning (DL).
3.1.1 Supervised learning
The method of predicting a model on a trained range of inputs learning function to maps the known output, which discover the pattern of new sets of data.
Example 1: To predict the model for microprocessor knee joint which is trained with numerous input or labeled data of the knee angle variation in different sub phase of gait cycle and apply on new amputee to predict the new data by the phase dependent pattern recognition approach.
Example 2: Intuitive myoelectric prosthesis or pattern recognition control prosthesis, FES.
Pattern recognition is an automatically recognition of pattern applied in data analysis, signal processing etc. when the pattern of algorithm trained from labeled data that is supervised learning. When the model of algorithm is fruitfully trained, the model can be used for the prediction of a new data. The ultimate goal of this ML is to develop a successful predictor function. The models of discrete or categorical categories of dependent variables are known as classification algorithm and with continuous value known as regression algorithm. Three basic steps followed to finalize a model are training, validating and application of algorithm to new data. Algorithm used for supervised learning are support vector machines, linear regression, linear discriminant analysis (LDA) etc. This is error based learning.
Example: Prediction of a model to relate the patient’s energy consumption using Trans femoral prosthesis with the function of walking velocity in level surfaces.
The linear regression model for the above statement is:
Y=b+aX+eE1
Y = Energy consumption (dependent variable)
b = Y intercept
a = Slope of the Line
X = Walking velocity (Independent variable), e = Error
The Logistic regression model is used to model the probability of a certain class or event such as pass fail, win/loss, healthy/sick etc. This is fall between 0 and 1 with categorical dependent variables.
Example: To predict a model for the successful or failed prosthetic rehabilitation within the categories of 50 meter walk test in level surface with combatable use of any assistive devices for successful and considered as fail if they could not complete the 50 meter walk test.
The model is predicted in terms of the probability (p) which are passing the 50 meter test are pass and could not cross 50 meter as fail.
ThemodelofthisStatementis:lnp1−p=a+bXE2
p = No of patient cross the level of 50 meter
1-p = No of patient could not able to cross
The dependent variable Y (predictive) = p/(1-p)
Independent Variable x = Type of prosthesis
3.1.2 Unsupervised learning
The algorithm of unsupervised learning finds a solution to unknown or unlabeled data which is not required any kind of supervision from human. It works of its own to gather information and allow performing more complex task compared to supervised learning. Cluster analysis and k means are the methods used for pattern formation for the new data.
Example: Intent detection algorithm with unlabeled data based on reference pattern is an unsupervised learning method used in microprocessor knee.
3.1.3 Reinforcement learning (RL)
This is concerned with how a software agent must take action in an environment to maximize the cumulative reward. The agent learns from the consequences of its actions and selects the choice from its past experiences and the new choices by the trial and error learning. This is generally output based learning. The components of the RL are agent and environment. The agent (Learner) learns about a policy (π) (strategy or approach that the agent uses to determine the next action based on the current state) by observing or interacting with the environment. All the possible steps followed by the agent during the process of learning are known as the “action” and current condition returned by the environment is “state”. The approach that the agent uses to determine the next action based on the current state is known as “policy”. The artificial intelligence gets either reward or penalties for the action the agent performs. The reward is an instant return from the environment to appraise the last action. The goal of an agent to maximize the reward based on the set of actions. The agent follows the concept of exploration and exploitation to get the optimal action value or rewards. The exploration is about exploring and capturing more information from the environment and exploitation uses the already known information to get the reward.
Example: Learning from demonstration (LfD) of myoelectric prosthesis. In this method the policy to determine the next action is learned by different methods i.e. demonstration provided by the Prosthetist, learned from the action of similar prosthetic user or intact limb movement of prosthetic user. During process of demonstration the sequence of state action pairs are recorded for the training of prosthetic limb. The learning process for movement of amputated side with intact limb happens simultaneously. The intact limb considered as training limb and the amputated side prosthetic limb as control limb. During training procedure the agent or learner or amputee asked to perform same motion for both the limb the information from training limb create a prosthetic policy that map the state of action of the control limb. Robotic prosthesis can use its learned and state conditional policy for user during post training use. The training arm demonstrated the desired movement, position and grasp pattern to robotic or control arm. During initial training process the opening of the prosthetic arm may not be the similar to the training limb but when the training preceded the gradual opening of the hand work as a reward to the agent to pick up the appropriate movement and position for required opening of the prosthetic hand and proportional control for graded prehension. The schematic diagram of Bento arm using reinforcement learning shown in Figure 2 [22]. Another example to understand the strategy of exploring and exploitation is to find out the exact position for placement of surface electrode in the residual limb of amputee. This is a trial and error method where surface electrodes are placed in different locations around the residual limb of the amputee to get the desired action potential to operate the prosthetic hand. The simultaneous activities of residual muscle EMG signal and operation of connected Prosthetic hand provide a visual feedback to amputee and Prosthetist. Based on the feedback the Prosthetist keeps on exploring new site of the electrode in the residual limb until optimization is achieved. This technique helps the amputee to learn about the amount of muscle contraction which operates the prosthesis. The opening and different grasping pattern in sequence acts as a reward to perform more complex activities. In some cases many old user or experienced Prosthetist use the strategy of the exploitation rather than exploring the new site for electrode placement based on their past learning and experiences. Other examples are adaptive switch control myoelectric prosthesis, Power leg Prosthesis, etc.
Figure 2.
Schematic diagram of flow of information with bento arm [22].
3.2 Deep learning
This is a form of machine learning uses both supervised and unsupervised and subset of machine learning and AI. It uses the method of artificial neural network (ANN) with representation learning. ANN is inspired by the human brain neural network system whether human brain network is dynamic (Plastic) and analog at the same time the ANN is static and symbolic. It can learn, memorize, generalized and prompted modeling of biological neural system. ANNs are more effective to solve problems related to pattern recognition and matching, clustering and classification. The ANN consist of standard three layer input, output and hidden layer, the output layer can be the input layer for the next output the simple network of neural system shown in Figure 3 [23], if there many hidden layer are present that ANN known as Deep Neural Networks”, or briefly DNN, can be successfully expert to solve difficult problems. Deep learning models yield results more quickly than standard machine learning approaches. The propagation of function in ANN through input layer to output layer and the mathematical representation for this is:
Figure 3.
Layers of ANN (artificial neural network) [23].
s=fφwxE3
(s = output, x = Input, w = corresponding weight of link between input and transfer function, φwx = linear combination of w and x, f(.) = transfer function.)
Example: EEG based pattern recognition which uses brain computer Interface (BCI) to control prosthetic arm, Neuroprosthesis etc.
3.3 Other artificial intelligence (AI) techniques
Artificial Intelligence is the intelligence of machine that simulates the human intelligence which programmed in such way that it thinks and act like human. It includes; reasoning, knowledge representation, planning, learning, natural language processing, perception, the ability to move and manipulate objects and many more subjects. AI has four main components Expert systems, Heuristic problem solving, Natural Language Processing (NLP) and Vision. In human the intelligent agents like eyes, ears, and other organs act as sensors, and hands, legs, mouth, and other body parts act as per instruction known as effectors similarly the robotic agent substitutes cameras and infrared range finders for the sensors and various motors for the effectors. A software agent has encoded bit strings as its precepts and actions. Similarity between human and artificial intelligence is shown in Table 1. AI can be divided into two categories as per its function as symbolic learning (SL) and machine learning (ML). SL is perform the functions like image processing through computer vision and understands the environment through robotics. ML computes the large amount of data to get a solution to the problem in terms of pattern recognition. Statistical machine learning embedded with speech recognition and natural language processing. Deep learning recognizes objects by computer vision through convolution neural network (CNN) and memorize past by recurrent neural network (RNN). The schematic diagram of AI and its functions are shown in Figure 4.
Human can perform
AI can perform
Speak and Listen
Speech recognition based on statistical learning system
Write and learn
Natural Language processing (NLP)
Eye can see
Computer vision or symbolic vision
Recognize the scene and create image
Image processing by symbolic learning
Understand the environment
Robotics
Ability to recognize pattern
Pattern recognition by Machine learning
Human brain formed by the networks of neurons
Artificial neural networks
Human memorize the past
Recurrent neural network (RNN) can use previous output as the input, so it remembers the data.
Recognize objects
Convolutional neural network (CNN) recognizes the object and also differentiates from others.
Table 1.
Similarity between human intelligence and artificial intelligence (AI).
Figure 4.
AI and its functions.
The methods or techniques used for the AI are classifier and prediction. Classifier is an algorithm that implements classification; the classifiers are Perceptron, Naïve Bayes, Decision trees, Logistic regression, K nearest Neighbor, AANN/DL and support vector machine [24]. Perceptron is the basic building block of the neural network it breakdown the complex network to smaller and simpler pieces. The classifier used in the myoelectric prosthetic hand is LDA classifier, Quadratic discriminant classifier and Multilayer perceptron neural network with linear activation functions etc. LDA (linear discriminant classifier) is a simple one that helps to reduce the dimension of the algorithm for application of neural network model. Prediction is a method to predict a pattern an output noise free data with a model from input data in hidden layer.
Examples: EMG CNN based prosthetic hand, EGG based Mind controlled prosthesis with sensory feedback, robotic arm, exoskeleton Orthosis.
Implementation of artificial intelligence in controlling prostheses has increased drastically and thus enables the amputee to operate the prosthesis more desirably. Adaptive controlling would enable a system to perform closer to the desired output by adjusting the input with the help of a feedback system. Recently, a mind-controlled limb (type of myoelectric controlling) was introduced as the latest advancement in the artificial intelligence-aided control system. A joint project between the Pentagon and Johns Hopkins Applied Physics Laboratory (APL) has come up with a modular prosthetic limb which would be fully controlled by sensors implanted in the brain, and would even restore the sense of touch by sending electrical impulses from the limb back to the sensory cortex [25]. Chang et al. (2009) proposed a multilayer artificial neural network (ANN)-based model to discover the essential correlation between the intrinsic impaired neuromuscular activities of people with spina bifida (SB) and their extrinsic gait behaviors [26]. The application of AI in prosthetics and orthotics is divided into various subparts according to the involvement of the region that get affected i.e. Lower extremity prosthesis and Orthosis, Upper extremity Orthosis and prosthesis, and rehabilitation aids like motorized mobility devices.
4.1 AI in upper extremity prosthesis and orthosis
The artificial Intelligence in upper extremity prosthesis used as direct control and indirect control from the neural network by various signal, sensor, controller and algorithm. The control signals are coming from the human in the two form for operation of upper extremity prosthesis i.e. electromyography (EMG) and Electroencephalogram (EEG). Prior attempts at voluntary control of the elements of prosthesis have focused on the use of electromyography (EMG) signals from muscle groups that remain under voluntary control. Most of this work has centered on control systems for upper extremity prostheses. The first commercialized powered hand myoelectric prosthesis was introduced by USSR in 1960 [27]. The advancement in EMG control myoelectric prosthesis was with use of EMG pattern recognition based control strategy [28]. This approach allows the user to control the prosthesis with multiple degrees of freedom. The most advanced and developed neural machine interface technology was TMR or targeted muscle reinnervation [29].
The conventional Electromyography (EMG) technique uses bipolar surface electrodes, placed over the muscle belly of the targeted group of muscles. The electrodes are noninvasive, inexpensive, and readily incorporated into the socket of the prosthesis. These surface electrode have limitations like inability to record the signal from different muscle group at a time, inconsistency in signal magnitude and frequency, due to change in skin electrode interface associated in physiological and environmental modifications and also the EMG signals may encounter noise and interference from other tissues. Apart from these limitations it is easy to use by amputee and risk free. The amplitude of the EMG signal is mostly proportional to the contraction of the remaining muscle. To enhance the quality of the signal the Myoelectric control of prosthesis or other system utilizes the electrical action potential of the residual limb’s muscles that are emitted during muscular contractions. These emissions are measurable on the skin surface at a microvolt level. The emissions are picked up by one or two electrodes and processed by band-pass filtering, rectifying, and low-pass filtering to get the envelope amplitude of EMG signal for use as control signals to the functional elements of the prosthesis. The myoelectric emissions are used only for control. In simultaneous control (muscle co contraction) and proportional control (fast and slow muscle contraction) controls the two different mode from wrist to terminal device and vice versa.
The advance method over the conventional technique of EMG signal which replace the complicated mode of switching is the pattern recognition. This new control approach is stranded on the assumption that an EMG pattern contains information about the proposed movements involved in a residual limb. Using a technique of pattern classification, a variety of different intended movements can be identified by distinguishing characteristics of EMG patterns. Once a pattern has been classified, the movement is implemented through the command sent to a prosthesis controller. EMG pattern-recognition-based prosthetic control method involves performing EMG measurement (to capture reliable and consistent myoelectric signals), feature extraction (to recollect the most important discriminating information from the EMG), classification (to predict one of a subset of intentional movements), and multifunctional prosthesis control (to implement the operation of prosthesis by the predicted class of movement) [30]. EMG pattern recognition block diagram of Trans radial prosthesis shown in Figure 5.
Figure 5.
Process of EMG pattern recognition control.
In pattern recognition control for a multifunctional prosthesis, multi-channel myoelectric recordings are needed to capture enough myoelectric pattern information. The number and placement of electrodes would mainly depend on how many classes of movements are demanded in a multi-functional prosthesis and how many residual muscles of an amputee are applicable for myoelectric control. For myoelectric transradial prostheses, the EMG signals are measured from residual muscles with a number of bipolar electrodes (8-16) which are placed on the circumference of the remaining forearm in which 8 of the 12 electrodes were uniformly placed around the proximal portion of the forearm and the other 4 electrodes were positioned on the distal end. A large circular electrode was placed on the elbow of the amputated arm as a ground [31].
For acquisition of EMG signal 50 Hz-60 Hz can be used to remove or reduce more low-frequency to increase the control stability of a multifunctional myoelectric prosthesis [32]. EMG feature extraction is performed on windowed EMG data, all EMG recordings channels are segmented into a series of analysis windows either with or without time overlap (WL (window length) is 100-250 ms) shown in Figure 6 [33].
Figure 6.
Windowing techniques, time to process each window analysis is ‘t’ and decisions (d1, d2, d3). In adjacent windows the processing time is less and the classifier is idle most of the time but in overlapping windows increase frequency of class decision because the analysis window slides with small increment (inc), the amount of overlap is equal to processing time which help the controller to process next class decision before the previous decission has been completed [33].
Overlapping analysis windows are used to maximally utilize the continuous stream of data and to produce a decision stream, for analysis, the duration of the overlapping (e.g., 50 ms) due to data buffering is the operational delay in real-time control and 50% of overlapping is suitable for the real time embedded system. The features are categorized as time domain (TD), frequency domain (FD) and time- frequency domain (TFD). The EMG features are extracted from each analysis window as a representation of EMG signal pattern. A feature set is extracted for each analysis window and all the recording channels, producing an L-dimensional feature vector. After computing the feature sets of all the channels, the entire EMG feature matrix (L × C × W, where L, C, and W are the number of features, the number of channels, and the number of analysis windows, respectively) from the training set is provided to a classifier for training shown in Figure 7. Example: The features extracted from four channels of surface EMG in each window is 44 and the data analyzed for the three windowed length, the EMG feature matrix for this situation (L × C × W = 44 × 4 × 3 i.e. L = 44, C = 4, W = 3).
Figure 7.
EMG windowing in continuous feature extraction. Size of successive window for analysis is L, the sEMG data for classification is divided into C segments for every L that is the length of integrated samples as a feature extraction and the start point is shifted every S.
The aim of pattern recognition based classifier is to discriminate the intended movements from the EMG recordings as accurately as possible. Many classification techniques have been investigated, including linear discriminate analysis, Bayesian statistical methods, artificial neural networks, and fuzzy logic [34, 35]. The LDA classifier is much simpler to implement and much faster to train without compromising the accuracy (>93%). Then the performance of a trained classifier in identifying a movement is evaluated using the testing data set and measured by the classification accuracy, which is defined as:
The classification accuracies in identifying all the classes of movements are averaged to calculate the overall classification accuracy for a subject uses convolutional neural network (CNN). Block diagram for classification and regression pattern shown in the Figure 8 [36].
Figure 8.
a. Pattern recognition is able to classify different movement patterns, but only in sequence, which limits multifunctional control. b. Regression control is able to identify different movements at the same time, leading to more intuitive prosthetic control [36].
EMG pattern recognition based prosthesis control strategy is not suitable for people with shoulder disarticulation amputations because few muscles remain in their residual arm from which to extract myoelectric control signals. To address this challenge, a new neural machine interfacing (NMI) technology called targeted muscle reinnervation (TMR) have been proposed and developed at Rehabilitation Institute of Chicago (RIC), which has the ability to improve control performance of multifunctional myoelectric upper-limb prostheses shown in Figure 9 [37].
Figure 9.
Targeted muscle reinnervation (TMR) [37].
TMR uses the remaining nerves from an amputated limb and transfers them onto substitute muscle groups that are not biomechanically functional because they are no longer attached to the missing arm. During this transfer procedure, target muscles are denervated so that they can be reinnervated by the residual arm nerves that previously traveled to the arm prior to amputation. The reinnervated muscles then assist as biological amplifiers of the amputated nerve motor commands. During the surgery subcutaneous tissue is removed that, surface EMG signals are optimized for power and focal recording.
Another advanced technique to control the multifunctional limb is Virtual reality (VR) based platforms have been developed for the purposes of development and performance quantification of multifunctional myoelectric prosthesis control system These VR platforms are designed to create an efficient, flexible, and user-friendly environment for prosthetic control algorithm development in the laboratory, application in a clinical setting, and eventual use in an embedded system. The major function modules of this platform include multi-electrode EMG recording (up to 16 channels), classifier training and testing in offline, virtual and physical prosthesis control in real time to regulate performance shown in Figure 10 [38].
Figure 10.
Virtual reality system (VR), subjects can operate a simulated prosthetic arm to interact with virtual objects. Multiple input modalities such as motion tracking systems and EMG/EEG electrodes provide maximum flexibility when evaluating different control approaches. Figure shows a subject operating a prosthetic arm prototype in VR (right side). Subject controls the arm via real-time motion tracking (left side), and 3-D visual feedback is provided via stereoscopic goggles for closed loop operation [38].
Apart from EMG signal the Electroencephalography (EEG) is the widely used non-invasive method by placing the electrode on the scalp for picking brain signal that has been utilized in brain machine interface (BCI/BMI) applications. It has high temporal resolution (about 1 ms) in comparison with other brainwave measurements such as electrocorticograms (ECoGs), magneto encephalograms (MEGs), functional magnetic resonance imaging (fMRI) and near-infrared spectroscopy (fNIRS). The advanced prostheses may best control by EEG signal with BCI, connected by ANN. The neural signals associated with arm movements as control signals of artificial neuroprosthesis collected from either the cortex of brain directly or from residual nerves. The diagram of EEG based control and EMG pattern recognition based control in utilized in upper extremity prosthesis is shown in schematic Figure 11.
Figure 11.
Brain computer Interface (BCI), controlling prosthetic and orthotics devices.
Examples: Ottobock Dynamic Arm Plus is a combination of Myo Hand Vari Plus Speed terminal device and Wrist rotator with custom TMR socket which control the six DOF [39]. Mind or thought controlled prosthesis uses EEG signal and ANN.
4.1.1 Recent advancement in control strategy in upper extremity prosthetics and orthotics
Jafarzadeh M (2019) uses the novel deep convolutional neural network (6 convolutional layers and 2 deep layers) and FIFO memory for operation of prosthetic hand in real time. The novel CNN was implemented in Python 3.5 using tensor flow library [40].
Chih-Wei-Chen et al. (2009) developed BCI based hand Orthosis used cursor control interface with a simple LDA classifier, that classify the EEG signals to control the hand orthosis in to three state right, left and nil and the corresponding command as +1, −1 and 0. The four states of activities like grasp, open, holding and standby can control by these three commands. The +1 and − 1 command signifies grasp and open, command ‘0’ is for standby mode depending on the feedback signals which are grasping force (F) and angular position (Ꝋ) collected from FSR and encoder [41].
4.2 AI in lower extremity prosthesis and orthosis
The first Artificial intelligence method used in the lower extremity as Intelligence prosthesis which is a knee joint that replace the hydraulic mechanism by combination of microprocessor controlled and hydraulic or pneumatic actuator.
The microprocessor as name suggests process the signal received by the first sensor known as knee angle sensor provides information about the knee’s angle of flexion and extension and velocity of lateral and angular movement, unlike the human body, the sensor determines the direction of movement because of a magnetic implant and second sensor gathers information about weight placement.
Microprocessor receives the data or signals by the motion employed by the amputee and that data are analyzed and interpret to get the closer approximation to natural gait. This data provides information to the microprocessor about the device’s position and the extent of its motion, which are essentially proprioceptive sensations. The data are stored in the memory of the microprocessor for the future use like a recurrent neural network (RNN). A series of wire networks which are similar in function to the body’s nervous system. That is, it enables the sensors, microprocessor, servo motors, and hydraulic cylinder to communicate with each other. These networks connect the two sensors to the microprocessor, which transmits sensory data much like the ascending sensory pathways send information to the brain. The wires exiting the microprocessor leading to the servo motors carry “motion commands,” mimicking the descending motor pathways which instruct muscles to contract and produce a desired movement.
As in the human nervous system, these wires are dedicated to specific communication circuits between the sensors, microprocessor, servo motors, and hydraulics. This computed data are used to control the resistance generated by the hydraulic cylinders through the small valve passes into and out of the cylinder which regulate extension and flexion of the knee joint in different sub phases of gait cycle. It controls knee joint motion from 0° to maximum 60-700. This mechanism helps the amputee to do various activities like stair climbing, jogging, running and walking in uneven terrain.
The microprocessor knee joint uses various algorithms to achieve gait symmetry, motion analysis, stumble control and comfort. These algorithm are control logic, Intent detection algorithm, Genetic algorithm, Fuzzy logic based classifier, Expectation maximization algorithm and Impedance control algorithm [42, 43]. The operation principle of a smart leg or intelligent prosthesis is shown in block diagram (Figure 12).
Figure 12.
Block diagram of controller based intelligent prosthesis.
The Prosthetic knee joints uses this microprocessor control mechanism with machine learning Artificial Intelligence are Otto Bock’s C leg (1997), OssurRheo knee (2005), Power knee by Ossur (2006), Self-learning knee by DAW Industries, Plie knee from freedom Innovation, Intelligent Prosthesis (IP) (Blatchford, United Kingdom), Linx (Endolite, Blatchford Inc. United Kingdom), Orion 2 (Endolite, Blatchford Inc. United Kingdom), X2 prostheses (Otto Bock Orthopedic Industry, Minneapolis, MN), X3 prostheses (Otto Bock Orthopedic Industry, Minneapolis, MN) etc.
The volitional EMG control robotics Transtibial prosthesis was developed in 2014 by Baojun Chen et al., which adapt the amputee to walk on slope with different angles. The combination of myoelectric and intrinsic controller reduces the fatigue of muscle and attention during walking [44]. The prototype design of prosthesis and schematic diagram of this mechanism showed in Figure 13.
Figure 13.
(a) Schematic diagram of prosthesis control by integrating the proposed myoelectric controller with the intrinsic controller. (b) Strategy of extracting amputee users’ movement intention with a 200-ms window in swing phase [44].
To mimic the normal foot and ankle motion several prosthetic feet uses AI mechanism are élan Foot (Blatchford, United Kingdom), iPED (developed by Martin Bionics LLC and licensed to College Park Industries), Proprio Foot (Össur, Iceland), Power Foot BiOM (developed at MIT and licensed to iWalk) and Meridium foot (Ottobock) etc. These feet are integrated with foot and ankle sensor to sense the terrain, angle and force required in different phases to mimic the normal foot.
Apart from EMG Control lower extremity prosthesis can be controlled by EEG signal using BCI, the example of EEG based control prosthesis is BiOM.
Lower Extremity Orthosis is a supportive device to the patients those have lost their function due to traumatic, neurologic and congenital abnormalities. The working principle of the Orthosis for the patient like hemiplegia, paraplegia and traumatic brain injury is changed vigorously with the implementation of artificial intelligence like functional electrical stimulation, Brain computer Interface and myoelectric controller. The concept of machine learning implemented in some sensor embedded stance control Orthosis which help the paraplegic to achieve near to normal gait with some limitations. The concept of functional electrical stimulation (FES) started in the year of 1960. This is used in case of damage of brain or spinal cord, stroke, Multiple Sclerosis (MS) and cerebral palsy.
The Functional electrical stimulation (FES) is the application of electrical stimulus to a paralyzed nerve or muscle to restore or achieve function. FES is most often used in neuro rehabilitation and is routinely paired with task-specific practice. Neuroprosthesis is a common example in orthotic substitution [45]. Control system can be open loop or Feed forward control, closed-loop or Feed backward control and adaptive control can be applied to both Feed forward and Feed backward controller. In open-loop controlled FES, the electrical stimulator controls the output and closed-loop FES employs joint or muscle position sensors to facilitate greater responsiveness to muscle fatigue, or to irregularities in the environment [46].
Electrodes act as interfaces between the electrical stimulator and the nervous system. The FES utilizes electrical current to stimulate muscle contraction so that the paralyzed muscles can start functioning again. The desired purpose is to stimulate a motor response (muscle contraction) through activation of a specific group of nerve fibers, typically using fibers of peripheral nerves. This may be achieved by the activation of motor efferent nerve fibers showed in Figure 14. FES uses Adaptive logic Network (ALN) and Inductive Learning Algorithm (IL) [47]. ALN is a type of artificial neural network for supervised learning which produces binary decision tree. This is a special type of feed forward multilayer perceptron the signal restricted to the Boolean logic. IL is a supervised learning produces decision tree in the form of IF, THEN, ELSE, etc. [48, 49].
AI implemented Gait Orthosis for spinal cord injury patients are powered ankle foot Orthosis (PAFO) and Exoskeletons. PAFO is incorporated with EMG controller to control the activity of soleus muscle to perform the actions of plantar flexion and inhibit the artificial dorsiflexion. Exoskeletons are uses BCI or EMG controller to control the orthotic devices [50].
4.3 AI in mobility devices
Wheel chair and walking aids is the important gadget for the disable to perform daily activities and transfer. In this robotic world the smart wheel chairs and intelligent walking aid reduced the area of work limitation. Application of artificial neural network in state of art robotics and AI technologies in smart wheels enhances the quality of life with ease in performance. The smart wheeler robotic wheelchair was developed by using Inverse Reinforcement Learning (IRL) techniques which was able to achieve maximum safety and set of tasks easily as compared to joystick control wheel chair [51]. Visual joystick control intelligent wheel chair is most advanced wheelchair prototype control by “Hand Gesture’ incorporate recurrent neural network (RNN) in joystick control makes it a smart joystick having driving flexibility to different kind of disability [52]. The schematic diagram of virtual simulation for visual joystick control showed in Figure 15.
Figure 15.
Virtual simulation of visual joystick control wheelchair.
Smart cane is a boon for the visually impaired persons; it incorporated with raspberry PI 3 microcontroller, HC-SRC04 ultrasonic sensor for obstacle detection, WTV-SR IC recognition module for record and fix voice playback and GPS/GSM module to save different locations [53].
Human being is the most intelligent and complex engineered structure created by almighty. It is really a tough challenge for the Prosthetist & Orthotist to replicate its lost anatomical structure and function. However with advancement in the field of AI and robotics has created a ray of hope for millions of persons with disabilities. The application of AI in the field of prosthetics and orthotics are in the initial stage and not so widely being practiced. Many projects using AI are in prototype Stage and not yet commercialized. High costs of these devices are being major limitations as many Persons with disabilities cannot afford it. Government bodies, manufacturing unit and funding agencies must come forward and invest in this field so that the highest quality and latest technology must reach to larger population of disabled in an affordable cost.
The author does not have any conflict of interest.
References
1.Jernigan TL, Stiles J. Construction of the human forebrain. Wiley Interdiscip Rev Cogn Sci. 2017;8(1-2):10.1002/wcs.409.
2.Colombo M. Why build a virtual brain? Large-scale neural simulations as jump start for cognitive computing. Journal of Experimental & Theoretical Artificial Intelligence. 2017;29(2):361-70.
3.Kandel ER, Schwartz JH, Jessell TM. Essentials of neural science and behavior. Norwalk, CT: Appleton & Lange; 1995.
4.Rothwell J. From Neuron to Brain (Second Edition). A Cellular Approach to the Function of the Nervous System. J Neurol Neurosurg Psychiatry. 1985;48(2):196-7.
5.Bezzi P, Volterra A. A neuron-glia signalling network in the active brain. Curr Opin Neurobiol. 2001;11(3):387-94.
6.Demir MH, editor 1 Rehabilitation Technologies : Biomechatronics Point of View 2018.
7.Akdogan E, Adli MA, Taçgin E, Bennett N, editors. A Human Machine Interface Design to Control an Intelligent Rehabilitation Robot System 2010.
8.Taylor MB, Clark E, Offord EA, Baxter C. A comparison of energy expenditure by a high level trans-femoral amputee using the Intelligent Prosthesis and conventionally damped prosthetic limbs. Prosthet Orthot Int. 1996;20(2):116-21.
9.Awad M, Dehghani-Sanij A, Moser D, Zahedi S. Inertia Properties of a Prosthetic Knee Mechanism 2015.
10.Burner M, Program MUH. The C-Leg® and Proprioception: How a Microprocessor-controlled Knee Prosthesis Mimics the Human Body’s System of Awareness of Position and Production of Movement 2011.
11.Kyriazi NE, Pujol AO, Barcelona FdId, Universitat Politècnica de Catalunya. Departament d’Enginyeria de Sistemes AiII. AI and Prosthetics: Universitat Politècnica de Catalunya. Facultat d’Informàtica de Barcelona. Departament d’Enginyeria de Sistemes, Automàtica i Informàtica Industrial, 2016 (Master in Artificial Intelligence - MAI); 2016.
12.Grabowski AM, D’Andrea SC, Herr HM, editors. BIONIC LEG PROSTHESIS EMULATES BIOLOGICAL ANKLE JOINT DURING WALKING 2011.
13.Grabowski AM, D’Andrea SC. Effects of a powered ankle-foot prosthesis on kinetic loading of the unaffected leg during level-ground walking. Journal of NeuroEngineering and Rehabilitation. 2013;10:49 -
14.Herr HM, Grabowski AM. Bionic ankle-foot prosthesis normalizes walking gait for persons with leg amputation. Proc Biol Sci. 2012;279(1728):457-64.
15.Ernst M, Altenburg B, Bellmann M, Schmalz T. Standing on slopes - how current microprocessor-controlled prosthetic feet support transtibial and transfemoral amputees in an everyday task. J Neuroeng Rehabil. 2017;14(1):117.
16.Clement RG, Bugler KE, Oliver CW. Bionic prosthetic hands: A review of present technology and future aspirations. Surgeon. 2011;9(6):336-40.
17.Belter JT, Segil JL, Dollar AM, Weir RF. Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review. Journal of rehabilitation research and development. 2013;50 5:599-618.
18.Luchetti M, Cutti AG, Verni G, Sacchetti R, Rossi N. Impact of Michelangelo prosthetic hand: Findings from a crossover longitudinal study. J Rehabil Res Dev. 2015;52(5):605-18.
19.Beyrouthy T, Kork SKA, Korbane JA, Abdulmonem A. EEG Mind controlled Smart Prosthetic Arm. 2016 IEEE International Conference on Emerging Technologies and Innovative Business Practices for the Transformation of Societies (EmergiTech). 2016:404-9.
20.Sathya R, Abraham A. Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification. International Journal of Advanced Research in Artificial Intelligence. 2013;2.
21.Sidey-Gibbons JAM, Sidey-Gibbons CJ. Machine learning in medicine: a practical introduction. BMC Medical Research Methodology. 2019;19(1):64.
22.Vasan G, Pilarski PM, editors. Learning from demonstration: Teaching a myoelectric prosthesis with an intact limb via reinforcement learning. 2017 International Conference on Rehabilitation Robotics (ICORR); 2017: IEEE.
23.Spiers DL, editor Facial emotion detection using deep learning 2016.
24.Sarker IH, Kayes ASM, Watters P. Effectiveness analysis of machine learning classification models for predicting personalized context-aware smartphone usage. Journal of Big Data. 2019;6(1):57.
25.Bridges MM, Para MP, Mashner M, editors. Control System Architecture for the Modular Prosthetic Limb 2011.
26.Yanbo H. Advances in Artificial Neural Networks – Methodological Development and Application. Algorithms. 2009;2.
28.Momen K, Krishnan S, Chau T. Real-time classification of forearm electromyographic signals corresponding to user-selected intentional movements for multifunction prosthesis control. IEEE Trans Neural Syst Rehabil Eng. 2007;15(4):535-42.
29.Li G. Electromyography Pattern-Recognition-Based Control of Powered Multifunctional Upper-Limb Prostheses. 2011.
30.Yang Z, Chen Y. Surface EMG-based Sketching Recognition Using Two Analysis Windows and Gene Expression Programming. Front Neurosci. 2016;10:445.
31.Yonghong H, Englehart KB, Hudgins B, Chan ADC. A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Transactions on Biomedical Engineering. 2005;52(11):1801-11.
32.Li G, Schultz AE, Kuiken TA. Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehabil Eng. 2010;18(2):185-92.
33.Hakonen M, Piitulainen H, Visala A. Current state of digital signal processing in myoelectric interfaces and related applications. Biomedical Signal Processing and Control. 2015;18.
34.Hargrove LJ, Englehart K, Hudgins B. A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans Biomed Eng. 2007;54(5):847-53.
35.Li G, Li Y, Yu L, Geng Y. Conditioning and Sampling Issues of EMG Signals in Motion Recognition of Multifunctional Myoelectric Prostheses. Annals of Biomedical Engineering. 2011;39(6):1779-87.
36.Roche AD, Rehbaum H, Farina D, Aszmann OC. Prosthetic Myoelectric Control Strategies: A Clinical Perspective. Current Surgery Reports. 2014;2(3):44.
37.Ajiboye AB, Weir RF. A heuristic fuzzy logic approach to EMG pattern recognition for multifunctional prosthesis control. IEEE Trans Neural Syst Rehabil Eng. 2005;13(3):280-91.
38.Hauschild M, Davoodi R, Loeb GE. A virtual reality environment for designing and fitting neural prosthetic limbs. IEEE Trans Neural Syst Rehabil Eng. 2007;15(1):9-15.
39.Vujaklija I, Farina D, Aszmann O. New developments in prosthetic arm systems. Orthopedic Research and Reviews. 2016;20168:31-9.
40.Jafarzadeh M, Hussey D, Tadesse Y. Deep learning approach to control of prosthetic hands with electromyography signals2019.
41.Chen CW, Lin CC, Ming S. Hand Orthosis Controlled Using Brain-computer Interface. J of Med and Biol Eng. 2008;29:234-41.
42.Stinus H. [Biomechanics and evaluation of the microprocessor-controlled C-Leg exoprosthesis knee joint]. Z Orthop Ihre Grenzgeb. 2000;138(3):278-82.
43.Dedić R, Dindo H, editors. SmartLeg: An intelligent active robotic prosthesis for lower-limb amputees. 2011 XXIII International Symposium on Information, Communication and Automation Technologies; 2011 27-29 Oct. 2011.
44.Chen B, Wang Q , Wang L. Adaptive Slope Walking With a Robotic Transtibial Prosthesis Based on Volitional EMG Control. IEEE/ASME Transactions on Mechatronics. 2014;20:1-12.
45.Martin R, Sadowsky C, Obst K, Meyer B, McDonald J. Functional electrical stimulation in spinal cord injury:: from theory to practice. Top Spinal Cord Inj Rehabil. 2012;18(1):28-33.
46.Bhatia D, Bansal G, Tewari R, Shukla KK. State of art: Functional Electrical Stimulation (FES). Int J of Biomedical Engineering and Technology. 2011;5:77-99.
47.Schauer T. Sensing motion and muscle activity for feedback control of functional electrical stimulation: Ten years of experience in Berlin. Annual Reviews in Control. 2017.
48.Martinez J. Neuroprostheses: Significance in Gait Rehabilitation. 2018. p. 427-46.
49.Kostov A, Andrews B, Popović D, Stein R, Armstrong W. Machine learning in control of functional electrical stimulation systems for locomotion. IEEE transactions on bio-medical engineering. 1995;42:541-51.
50.Rosen J, Fuchs MB, Arcan M. Performances of Hill-Type and Neural Network Muscle Models - Toward a Myosignal-Based Exoskeleton. Computers and biomedical research, an international journal. 1999;32 5:415-39.
52.Rabhi Y, Mrabet M, Fnaiech F. Intelligent Control Wheelchair Using a New Visual Joystick. Journal of Healthcare Engineering. 2018;2018:1-20.
53.Ilag BN, Athave Y. A Design review of Smart Stick for the Blind Equipped with Obstacle Detection and Identification using Artificial Intelligence. International Journal of Computer Applications. 2019;182:55-60.
Written By
Smita Nayak and Rajesh Kumar Das
Submitted: 10 May 2020Reviewed: 06 September 2020Published: 07 October 2020
Open access peer-reviewed chapter
