1. Introduction

The human history has been accompanied by accidental trauma, war, and congenital anomalies. Consequently, amputation and deformity have been dealt with, one way or another, throughout the ages. More than one million individuals in the United States today are living with limb amputations (Adams et al., 1999), in which there are approximately 100,000 patients with an upper limb amputation. The wars in Iraq and Afghanistan have added to this number. According to the survey results of the Second China National Sample Survey on Disables (SCNSSD 2006) led by the National Statistics Bureau in 2006, approximately 8% of physical disables, or 2.26 million people, live with limb amputations in China alone. Natural disasters and accidents have been making this number increase. The massive earthquakes that occurred in May 2008, Sichuan Province, China, recently increased about 20 thousand of new limb amputees. Expectations for control of upper limb prostheses have always been high because of the standard established by able-bodied dexterity.

Most commercially available upper limb prostheses are either body-powered or electrical motor powered. The body-powered prostheses are operated by certain movements of the amputees’ body through a system of cables, harnesses, and sometimes, manual control. In order to operate a body-powered prosthesis, the upper limb amputees have to possess significant strength and control over various body parts, including the shoulders, chest, and residual limb which must have sufficient residual limb length, musculature, and range of motion. Exaggerated movements of the body are captured by harness systems and are transferred through cables to operate the hand, wrist, or elbow movements of a prosthesis. With some advantages such as low cost, high reliability, and some kinesthetic feedback provided by the harness system, body-powered prostheses are still widely accepted by the upper limb amputees worldwide, especially in some developing countries. However, with this inappropriate control approach, body-powered upper limb prostheses are limited in utility, frustratingly slow to operate, awkward to maintain, and can operate only one joint at a time.

Myoelectric signals detected with electrodes placed on the skin surface overlying the muscles, well-known as electromyography (EMG), have been used in control of motorized upper-limb prostheses for several decades (Kay & Newman, 1975; Parker & Scott, 1986). The myoelectric control approach was proposed in the 1940s, but the myoelectric prostheses could not be viably made into clinical applications in that day due to the technical limitations. With the advances of technologies, especially electronic technologies, a significant progress has been made in the development of myoelectric prosthesis control during the 1960s. The first commercialized myoelectric prosthesis, a powered hand, was developed in the USSR (Kobrinski et al., 1960) in 1960, and later more myoelectric prostheses had been developed one after the other in different countries. A multifunctional myoelectric hand was developed in Japan (Kato et al., 1969), in 1969, and the first myoelectric elbow prosthesis was developed in the United States (Lyman et al., 1976), in 1970s. The three-state myoelectric controller was developed for the control of a three function device with a single muscle (Dorcas & Scott, 1966) in Canada. However, before several major commercial companies such as Otto Bock and Viennatone invested in the field, the production of powered myoelectric prostheses was pretty small. Beginning in the 1970s, the powered upper-limb myoelectric prostheses were clinically and routinely fitted to upper-limb amputees. Currently, most commercially available motorized upper-limb prostheses are controlled by using EMG signals from the residual muscles of an amputated arm. Commercial electronic prostheses and prosthetic components are sold by several companies, including Liberating Technologies, Inc, Otto Bock, Shanghai Kesheng Prostheses Co, and Touch Bionics.

The control strategies of upper-limb myoelectric prostheses use surface electromyogram (EMG) amplitude to control the prosthetic devices in either on/off or proportional mode. The EMG signals are recorded from one or two electrodes and processed by band-pass filtering, rectifying, and low-pass filtering to get the envelope amplitude of EMG signals, as shown in Fig. 1. Threshold of EMG signal amplitude is then applied to determine the minimum level of contraction necessary to initiate a movement. In the on/off control mode, the speed of prosthetic movements is constant. When two electrodes are used to control one degree of freedom (DOF), it is possible to use proportional control, in which the speed of movement is proportional to the amplitude of the myoelectric signal.

Figure 1.
Schematic diagram of conventional two-site EMG prosthesis control system

Most commercially available upper-limb myoelectric prostheses use a pair of muscles (usually an agonist/antagonist pair) to control one degree of freedom: one EMG signal from a flexor muscle and one from an extensor muscle, as shown in Fig. 1. Each of two movements in a joint DOF is assigned to a separate control muscle, such as hand opening to biceps and hand closing to triceps. When the EMG amplitude from one control muscle (such as biceps) is greater than a given threshold (T1 for biceps muscle), the associated prosthetic movement (hand opening) is selected and performed by an electric motor, as shown in Fig. 2. The logical circuitry in the controller of a myoelectric prosthesis allows only one of the DOF movements to be active at a time.

This control mode allows choosing the physiologically appropriate control muscles associated to the movement functions for an intuitive control of a prosthesis, but requires two control muscles for each prosthetic DOF. The control approach works fairly well if only one joint DOF is required such as for a transradial amputee, where two remaining forearm muscles (flexor and extensor) are used to control a powered hand DOF (hand opening/closing). If wrist rotation is desired, the users must activate an external switch or co-contract the two forearm muscles to shift from hand mode to wrist rotation mode. The same forearm flexion and extension EMG signals are then used to control the wrist rotator.

For higher level amputees, given a limited number of muscles available after amputation, it is difficult to control multiple DOF using this conventional control mechanism (Hudgins, 1993; Sears, 1992). For example, for a transhumeral amputee, the remaining arm muscles only have parts of biceps and triceps which can serve as EMG signal sites to control prosthetic movements. When all the three joint DOFs of elbow, wrist, and hand are required, the user must trigger a “mode switch” such as making a co-contraction of the agonist/antagonist muscle pair to sequentially select which of these joints is desired to be actuated. Obviously, switching to different modes is slow and cumbersome. Moreover, using a same agonist/antagonist pair to control different joint DOFs is non-intuitive and very difficult for users to learn the contraction/co-contraction of these muscles, because the applicable residual muscles may not be physiologically associated with the joint DOFs (such as using the residual biceps and triceps muscles to control hand opening and closing).

Figure 2.
Two-channel EMG amplitudes based prosthesis control

An alternative strategy is one-site EMG prosthetic control approach that has been used in a three-state myoelectric control system (Dorcas & Scott, 1966). In this control approach, the amplitude range of the one-site EMG signals that are generated from a relaxed muscle state to the full contraction is divided into three segments, as depicted in Fig. 3. Each segment has an associated amplitude threshold and corresponds to a specific prosthetic movement function (S1 for no movement, S2 for hand closing, and S3 for hand opening). In order to perform a specific movement function, the user must try to produce a constant muscle contraction to keep the EMG amplitude in the range of the associated segment. Theoretically, this approach can control a number of prosthetic functions. However, the number of functions that an amputee can control with acceptable accuracy is limited to two per control muscle (Vodovnik, 1967). In addition, like the two-site EMG control approach, using this method to control a prosthetic function also is non-intuitive and very hard for users to learn the contraction procedures of muscles.

Figure 3.
One-channel EMG amplitude based prosthesis control

A promising alternative to myoelectric control is to measure the actual muscle movement, since this method is impervious to external variables, yet captures the individual movement of the muscles. It is very difficult, however, to capture this movement. Zheng et al. (Zheng et al., 2005) have attempted to measure muscle movement using sonomyography. Although the technique has provided impressive results, it is not currently feasible to implement the required instrumentation in a form factor suitable for integration in a prosthesis. Miniature muscle tunnel cineoplasties (Marquardt, 1987; Beasley, 1966), in which the tendons of muscles are connected to external cables, offer a more accurate measurement of tendon excursion. However, it has not seen much clinical interest in the world due to the invasive nature of this method.

Electrically powered upper-limb myoelectric prostheses have several advantages over body-powered prostheses. The user of a myoelectric prosthesis is freed of cables and harnesses that are required in body-powered and mechanical switch control. The myoelectric signal is noninvasively recorded on the skin surface of the residual arm and the muscle activity required to generate prosthesis control signals is relatively small. However, with the limitations of currently available myoelectric prostheses discussed above, it is estimated that only 50% of patients with an upper limb amputation use a myoelectric prosthesis. These disabled people have always been expecting high performance artificial upper-limb systems to restore the motion functions involved in their lost arms.

Recently, a significant progress of the advanced physical prostheses or components with a number of degrees of freedom has been made worldwide. Several multifunctional hands and wrists are under development or even in clinical trial. Touch Bionics has released a prosthetic hand with individually driven fingers and thumb. The Otto Bock Michelangelo Hand, the Southampton Hand (Kyberd & Chappell, 1994; Kyberd et al., 2001) and Cyberhand (Carrozza et al., 2002; 2004; 2006) have been in development in Europe for many years. However, without a new control approach that allows the user to operate a multifunctional myoelectric prosthesis intuitively and easily, these newly developed prostheses or components could not be practically usable and clinically viable.

A significant improvement over the conventional control method of current myoelectric prostheses is the use of EMG pattern recognition based control strategy (Hudgins et al., 1993; Saridis & Gootee, 1982; Kang et al., 1995; Park & Lee, 1998; Englehart et al., 1999; Englehart & Hudgins, 2003; Chan & Englehart, 2005; Ajiboye & Weir, 2005; Sebelius et al., 2005; Hargrove et al., 2007; Momen et al., 2007). This new control approach is grounded on the assumption that EMG patterns contain rich information about the intended movements involved in a residual limb. Using a pattern classification technique, the distinguishing characteristics of EMG patterns can be used to identify a variety of different intended movements. Once a pattern has been classified, a command is sent to a prosthesis controller to implement the movement, as shown in Fig. 4. With this new control method the user elicits the contraction corresponding to the DOF that they want to control, and the classifier chooses the appropriate class of motion. As a result, the user has intuitive control and rapid selection of each function, as the intended movement matches the prosthesis function. This control approach may allow users to more easily operate their myoelectric prostheses with multiple degrees of freedom.

Figure 4.
Schematic diagram of EMG pattern-recognition-based prosthesis control system

In the next sections of this chapter, some important issues related to this new control strategy of multifunctional myoelectric prostheses, such as EMG signal processing and analysis and the performance of pattern recognition algorithms, will be introduced. Then, a newly proposed and developed neural-machine interface technology called Target Muscle Reinnervation (TMR) will be briefly described. TMR technology has the ability to provide additional myoelectric sources for improvement of control performance of a multifunctional prosthesis. Finally, the quantification of control performance of multifunctional myoelectric prostheses will be discussed and the real-time control performance in manipulating a virtual-reality arm and a powered physical transradial prosthesis by upper-limb amputees will be quantized and analyzed.

2. Pattern-recognition-based control approach

As explained in the previous section, current myoelectric control strategies use information from the EMG based on an estimate of the amplitude or the levels of EMG change for controlling a single device in a prosthetic limb, such as a hand, an elbow, or a wrist. These control methods have been commercially available and clinically viable to meet the need of upper-limb amputees for a powered prosthesis. However, the conventional control mode is not able to reliably control multiple functions, as required in high-level limb deficiencies. So a new control strategy is needed to deal with this difficult problem in control of a multifunctional myoelectric prosthesis. This section describes the newly proposed control strategy, EMG pattern-recognition-based control approach, which promises to deliver multifunction control of a myoelectric prosthesis. Although a full-fledged practical implementation is still awaited, many previous studies conducted to investigate the performance of this new control technology have shown its capabilities of developing the next generation of multifunction and microprocessor-driven myoelectric prosthetic systems.

In general, an EMG pattern-recognition-based prosthetic control approach involves performing EMG measurement (to capture more and reliable myoelectric signals), feature extraction (to retain the most important discriminating information from the EMG), classification (to predict one of a subset of intended movements), and multifunctional prosthesis control (to implement the prosthetic operation of the predicted class of movement), as illustrated in Fig. 4. The details of each stage of a pattern recognition based control strategy are discussed in the following sections.

2.1. Multi-channel EMG acquisition

EMG signal measurement: In EMG pattern-recognition-based control of a multifunctional prosthesis, multi-channel myoelectric recordings are needed to capture enough myoelectric pattern information for the accurate classification of multiple classes of intentional movements. This raises two primary concerns in practice: number of myoelectric channels and configuration of electrode placement (electrode positions). The number and placement of electrodes would mainly depend on how many classes of movements are demanded in a multi-functional prosthesis and how many residual muscles of an amputee are applicable for myoelectric control. It is obvious that the more the classes of movements are involved in a prosthesis, the more the myoelectric electrodes are required to get more myoelectric signals. Using more myoelectric electrodes may increase the number of myoelectric signals captured, but it simultaneously adds more complexity, weight, and cost to a prosthesis. For the amputees with different upper-limb amputation levels, the motion classes that they demand and their remaining arm muscles available for myoelectric control are highly variable. Thus an appropriate analysis must be performed to determine the number and placement configuration of myoelectric electrodes required to control multifunctional upper-limb prostheses, accordingly.

Pattern recognition has been used in different laboratories worldwide for development of transradial prosthesis control because the forearm contains the residual wrist muscles, allowing wrist function to be readily controlled, and some residual hand muscles, for limited multifunction hand control. For myoelectric transradial prostheses, the EMG signals are measured from residual muscles with a number of bipolar electrodes (8-16) which are generally placed on the circumference of the remaining forearm. In a recent study (Li et al., 2010), 12 self-adhesive bipolar electrodes were used to record EMG signals, in which 8 of the 12 electrodes were uniformly placed around the proximal portion of the forearm and the other 4 electrodes were positioned on the distal end. A large circular electrode was placed on the elbow of the amputated arm as a ground.

The primary motion classes that may be highly required by a transradial amputee are wrist flexion/extension, wrist rotation (pronation/supination), and hand open/close. The preliminary analysis that was recently performed (Li et al., 2010) shows that for the six basic motion classes, using six optimally selected electrodes could produce an average classification accuracy of around 92%. In addition, this study also showed that for different transradial amputees, the locations of the optimal electrode placement are variable. This study used a straightforward, exhaustive search algorithm to determine the optimal electrodes based on the 12-channel EMG recordings for each subject.

EMG signal conditioning and acquisition: EMG signals captured with surface electrodes are commonly filtered by a band-pass filter to improve signal quality. Properly choosing the frequency band for the band-pass filter would be of importance for improving the control performance of a myoelectric prosthesis. At the higher frequency side of signal spectrum, a low-pass filter is used to attenuate the unwanted high-frequency components in EMG signals and avoid aliasing signal distortion. Generally, the cut-off frequency of a low-pass filter is determined by the requirement of the Nyquist sampling theory, which should be equal or less than half of signal sampling rate. At the lower frequency side of signal spectrum, the cut-off frequency of a high-pass filter is determined by the need to remove slow variations in the signals caused by the motion artifacts such as electrode shift and cable movement. Almost all the previous studies of EMG pattern recognition based prosthesis controls adopted a high-pass cut-off frequency ranging from 5 Hz to 20 Hz. The lower frequency components of EMG spectrum mainly contain the information on the firing rates of active motor units, which may be important for some EMG studies. However, these components may not make a significant contribution to the movement classification in EMG-based movement analysis. It is known that the cable motion artifacts typically have a frequency range of 1-50 Hz and the power density of electrode motion artifacts is up to 20 Hz. Thus a high-pass filter of 5-20 Hz could not effectively attenuate the motion artifacts, which may impair control accuracy and stability of a myoelectric prosthesis. Therefore, a higher high-pass cut-off frequency will be expected to significantly reduce more motion artifacts in the captured EMG signals; this may enhance the control accuracy and stability of a myoelectric prosthesis. The results from our recent study (Li et al., 2011) showed that the accuracy for the classification of a number of classes of arm movements could not benefit much from acquiring more low frequency components of EMG signals. Including 20-100 Hz frequency band components of EMG signals only slightly increased the classification accuracy in both of able-bodied subjects (about 0.25%) and amputees (about 1.6%). This suggests that a higher high-pass cut-off frequency such as 50Hz-60Hz can be used to remove or reduce more low-frequency motion artefacts from EMG recordings for improving the control stability of a multifunctional myoelectric prosthesis.

With the exception of a few cases, the major power (about 95%) of surface EMG signals is accounted for by harmonics up to 400-500Hz (Clancy et al., 2002) and most of the EMG components with a frequency of more than 500 Hz are contributed by electrode and equipment noise or environmental interference. Thus, the widely used sampling rate in surface EMG studies (Clancy et al., 2002; Ives & Wigglesworth, 2003) is around 1,000 Hz. This sampling rate was also adopted in most studies of EMG pattern recognition prosthesis control (Ajiboye & Weir, 2005; Sebelius et al., 2005; Hargrove et al., 2007; Li et al., 2010). It is obvious that using a high sampling rate may involve more high-frequency contents in myoelectric signals captured with surface electrodes, but it simultaneously adds more processing and computational complexity to the controller of a prosthesis. With the limited computation capability of a microprocessor-based prosthetic controller embedded into the socket of a prosthesis, it would be desired in EMG signal acquisition to use a low sampling rate without compromising much with prosthesis control performance. Our recent investigations (Li et al., 2011) showed that using a 500-Hz sampling rate, the average classification accuracy for the subjects with upper-limb amputation only dropped around 2.0% in comparison of a 1-kHz sampling rate. Compared to a 1-kHz sampling rate, using a 500-Hz sampling rate can save about 50% storing memory and reduce 50% data processing time with a slight accuracy sacrifice; this will greatly simplify the design and implementation of a microprocessor-based prosthetic controller. In addition, fast data processing speed may allow us to use more sophisticated pattern recognition algorithms and additional control strategies such as prosthetic adaptive control and majority vote in decision making to further improve the control performance of multifunctional myoelectric prostheses.

2.2. EMG feature extraction

An EMG pattern associated to a limb movement is described with the features extracted from EMG recordings. The choice of a feature set has a significant influence on the performance of the EMG pattern classifier. Commercially available myoelectric controllers only use the smoothed amplitude myoelectric signals as their feature. With the need of providing more information about the EMG patterns in each channel, multivariate features sets have been proposed and used in EMG pattern-recognition-based control of multifunctional prostheses. The most intuitive features are based on time-domain statistics such as mean absolute value, mean absolute value slope, variance of the EMG signals, zero crossing, slope sign changes (Hudgins et al., 1993), which need less computational resources in comparison to frequency-domain features and time-frequency features such as autocorrelation coefficients, spectral measures, short-time Fourier transform, wavelet transform, and wavelet packet transform. Because of their relative ease of implementation and high performance, the time-domain features have been widely used in most previous studies.

EMG pattern recognition is performed on windowed EMG data. EMG recordings from all recording channels are segmented into a series of analysis windows either with or without a time overlap, as shown in Fig.5. The window length is usually 100-250 ms. Overlapping analysis windows are used to maximally utilize the continuous stream of data and to produce a decision stream that is as dense as possible, with regard to the available computing capacity (Englehart and Hudgins, 2003). For overlapping window analysis, the operational delay in real-time control due to data buffering would be the duration of the overlapping (e.g., 50 ms) instead of the length of the window (e.g., 150 ms). The EMG features are extracted from each analysis window as a representation of EMG signal pattern. For each analysis window, a feature set is extracted on each of all the recording channels, producing an L-dimensional feature vector (corresponding to the L features). After concatenating the feature sets of all the channels, the entire EMG feature matrix (L×C×W, where L, C, and W are the number of features, the number of channels, and the number of analysis windows, respectively) from the training set is provided to a classifier for training.

Figure 5.
Segmentation of analysis windows of EMG recordings

2.3. EMG pattern recognition classifier

The goal of a pattern recognition based classifier is to discriminate the intended movements from the EMG recordings as accurately as possible. Many classification techniques have been investigated, including linear discriminate analysis (Hargrove et al., 2007; Li et al., 2010), Bayesian statistical methods (Huang et al., 2005), artificial neural networks, and fuzzy logic (Ajiboye & Weir, 2005). All report similar classification accuracies (92-98% accuracy), and there is no statistical difference across a subject pool [38], provided the classifiers are properly tuned and use a good set of features. The linear discriminant analysis (LDA) classifier (Tou and Gonzalez, 1974) has been widely used in previous studies for classification of different movements. More complex and potentially more powerful classifiers may be constructed, but it has been shown in previous work (Hargrove et al., 2007) that the LDA classifier does not compromise classification accuracy. Compared with other classifiers, the LDA classifier is much simpler to implement and much faster to train.

It is worth noting that many previous studies have used able-bodied subjects to assess the feasibility and performance of pattern-recognition algorithms using EMG signals from forearm muscles. Using various pattern classification techniques, such as linear discriminant analysis (LDA), artificial neural networks, and fuzzy logic, high accuracies (>93%) for classification of six to ten wrist and hand movements were consistently achieved in many previous studies. This suggests that a variety of pattern-recognition algorithms can be used to predict the able-bodied subject’s actual hand or arm movements with high accuracies. Use of able-bodied subjects is reasonable with the simple goal of comparing classification accuracy of different pattern recognition algorithms in discriminating EMG patterns. However, the limb amputees are the final users of a myoelectric prosthesis. In some of these previous studies for able-bodied subjects, electrodes were placed on the proximal portion of forearm to mimic the case of people with transradial amputations. However, unlike the able-bodied subjects who could do a hand or arm movement physically, limb amputees perform an intended movement using their phantom hand or arm. Limited works have been done in subjects with a limb amputation. A recent study involved six subjects with transradial amputations (five transradial amputees and one congenital below-elbow failure of formation) and used 8 electrodes placed on the residual forearm for EMG recordings (Sebelius et al, 2005). This study showed a low average accuracy (approximately 70%) for classification of 10 arm classes (wrist flexion/extension plus 8 hand grasps) with an artificial neural network–based classifier. Another study (Li et al., 2010) that was conducted on five unilateral transradial amputees also found high pattern recognition accuracies (around 94%) on the intact limbs—in which EMG data was collected from both forearm and intrinsic hand muscles—and significantly lower accuracies (around 79%) on the amputated limb. It is obvious that the classification accuracy achieved with an amputated arm is significantly lower than that with an intact arm. Thus this suggests that the performance assessment of a classifier in identifying a number of movements for control of a multifunctional myoelectric prosthesis should use the people with limb amputations.

2.4. Evaluation of classification performance

Historically, investigators quantified the EMG pattern recognition performance with the simple goal of comparing the classification accuracy of different pattern recognition algorithms. In general, the EMG recordings from performing a movement are divided into two parts. One part of EMG data is used as the training data set and another part serves as the testing data set. For a subject, a specific classifier is built using the training EMG data set. Then the performance of a trained classifier in identifying a movement is evaluated using the testing data set and measured by the classification accuracy, which is defined as

Number correctly classified samples Total number of testing samples x 100 % E1

The classification accuracies in identifying all the classes of movements are averaged to calculate the overall classification accuracy for a subject.

2.5. Multifunctional prosthesis control

For multifunctional prosthesis control, a classifier is offline trained by having the user to perform repetitions of a number of motion classes that will be involved in the prosthesis. Then in the real-time application, the trained classifier sequentially determines which motion class the user is employing based on a set of EMG features. The duration of making a decision of the classifier would be the time increment of an overlapping analysis window. When a motion class is recognized, a motor control command is sent to the prosthesis controller for completion of the motion. The classification is repeated at overlapping intervals to provide continuous control of a myoelectric prosthesis.

3. Neural-machine interface for improvement of control performance

3.1. A paradox

As discussed above, EMG pattern recognition based control strategy seems highly promising in developing the novel myoelectric prosthetic systems that may allow users to more easily and intuitively operate their prostheses with multiple degrees of freedom. The usability and performance of the pattern recognition approach in control of a multifunctional myoelectric prosthesis are premised on whether the users have enough residual muscles as sources of myoelectric control signals. This premise may be true for people with a below-elbow amputation. Their remaining forearm contains the residual wrist muscles, allowing wrist function to be readily controlled, and some residual hand muscles for control of hand movements (Fig. 6(a)). Since their elbow joint is intact, there is no need to restore the movements associated with elbow. High accuracies are consistently achieved in different studies using different processing techniques, for six classes of basic wrist and hand movements, as illustrated in Fig. 7. Thus the clinical implementation of a pattern recognition control system with wrist movements and one hand grasp looks promising for people with transradial amputations, based on the results of these previous studied. However, this premise is hardly true for people with an above-elbow or shoulder disarticulation amputation. In a transhumeral amputee only portion of biceps and triceps muscles remains (Fig. 6(b)), which would provide enough myoelectric signals for control of elbow function, but there are no remaining muscles for control of wrist and hand functions. For a person with whole shoulder disarticulation amputations, there is no any arm muscle remained as control signal (Fig. 6(c)), whereas they have a need to restore whole arm joint functions (shoulder, elbow, wrist, and hand). The less the arm muscles remain after amputations, the more the arm joint functions need to be restored in a prosthesis. It is quite a paradox.

Figure 6.
Different level upper-limb amputations.

Figure 7.
Six basic wrist and hand movements.

3.2. Neural prostheses

It is obvious for people with high-level arm amputations that additional control information associated to arm movements is needed to see the realization of myoeletric prostheses with multiple degrees of freedom. An exciting concept called neuroelectric control has received considerable attention. Three so-called neural-machine interface techniques emerged and have been investigated for control of neural prostheses. They are brain-computer interface (BCI), peripheral never interface (PNI), and targeted muscle reinnervation (TMR), as shown in Fig. 8.

With BCI or PNI prosthetic control, neural sensors need to be directly connected to either the cortex or the residual nerves to capture the neural signals associated with arm movements as control signals of artificial neuroprostheses (DeLuca, 1978; Hoffer & Loeb, 1980; Edell, 1986; Hochberg et al., 2006). Although this concept offers the hope of improved control there are several inherent problems such as the mechanical sensitivity of nervous tissue, the permanence of sensor array fixation, and the fibrosis of sensor recording tips. In addition, the neural signal is very small, difficult to record and ease to be contaminated by various interference and noise in surrounding environment. An inherent challenge in the neural interface is that only a relatively small number of motor nerve fascicles may be sampled (with respect to all fascicles within a nerve bundle), making it difficult to construct a complete representation of motor intent. Motor nerves also atrophy when they are not connected to muscle, which could compound these problems. A further difficulty arises in transmitting the signals out of the body. This requires either chronic percutaneous wires (which tend to become infected) or complex transmitter-receiver systems. Finally, the durability of the implanted hardware is a critical issue. With these limitations of the BCI and PNI technologies, the works of many years have not yet resulted in any usable system of multifunctional neuroprosthesis. Prosthetic control systems are required to function for a long time (from several years to decades). The TMR technology may be considered as using muscle as a biological amplifier of the neural signal to circumvent many of the problems of BCI or PNI control and makes additional control signals accessible without implanted hardware into body.

Figure 8.
Three emerging neural-machine interface techniques for control of neuroprostheses.

3.3. Target muscle reinnervation

EMG pattern recognition based prosthesis control strategy is not applicable for people with above-elbow amputations because few muscles remain in their residual arm from which to extract myoelectric control signals. To address this challenge, a new neural machine interfacing (NMI) technology called targeted muscle reinnervation (TMR) have been recently proposed and developed at Rehabilitation Institute of Chicago (RIC), which has the ability to improve control performance of multifunctional myoelectric upper-limb prostheses (Kuiken et al., 2009; Zhou et al., 2007). Neural information that controlled the limb prior to amputation remains in the residual peripheral nerves. TMR uses the residual nerves from an amputated limb and transfers them onto alternative muscle groups that are not biomechanically functional since they are no longer attached to the missing arm. During the nerve transfer procedure, target muscles are denervated so that they can be reinnervated by the residual arm nerves that previously traveled to the arm prior to amputation. The reinnervated muscles then serve as biological amplifiers of the amputated nerve motor commands (Kuiken, 2003). During the surgery subcutaneous tissue is removed so that surface EMG signals are optimized for power and focal recording. Fig. 9 schematically shows the TMR technique in a person with shoulder disarticulations. TMR thus provides physiologically appropriate EMG control signals that are related to previous functions of the lost arm. Successful TMR allows voluntary motor control signals that used to activate muscles in the amputated limb to activate these newly reinnervated muscles.

TMR technique has been successfully performed in some dozens of people with transhumeral and higher upper-limb amputations worldwide. The relevant studies showed that TMR can provide a rich source of additional control data that are physiologically related to the missing limb. The high classification accuracy was consistent within subjects, demonstrating good repeatability. It was also high between subjects who had had different surgical procedures and had different remaining posttraumatic anatomy and geometry of their target muscle, demonstrating that the surgical concept can be applied to a broad array of injury levels (Zhou et al, 2007).

Figure 9.
Schematic diagram of TMR technique (Kuiken et al., 2009)

4. Quantification of real-time control performance

It is a challenge to evaluate the real-time control performance of EMG pattern recognition based prostheses, especially in the case that there are no multifunctional prosthetic systems available. Note that almost all of the previous studies used classification accuracy to evaluate the performance of pattern recognition algorithms. Classification accuracy is the ability of the algorithm to appropriately recognize the desired movements during each time window (usually 100-200 ms) while the subject holds different movements for several seconds. This accuracy is calculated by post-processing EMG recordings and is not a true measure of real-time function of a myoelectric prosthesis. Thus, in order to know whether the residual muscles following amputation can provide stable EMG information for accurate real-time control of multifunctional prostheses, the real-time performance metrics are required to examine the clinical robustness and accuracy of pattern recognition control.

4.1. Virtual prosthesis control

The controllable degrees of freedom are limited by the mechanical degrees of freedom available in the prosthesis. Currently, physical myoelectric prostheses with multiple degrees of motion freedom are not available yet, resulting in a challenge in quantitatively evaluating the real-time control performance. To deal with this challenge, the virtual reality (VR) based platforms have been developed for the purposes of development and performance quantification of multifunctional myoelectric prosthesis control system (Li et al, 2010; Kuiken et al., 2009). These VR platforms are designed to create an efficient, flexible, and user-friendly environment for prosthetic control algorithm development in the laboratory, application in a clinical setting, and eventual use in an embedded system. The major function modules of this platform include multi-electrode EMG recording (up to 16 channels), classifier training and testing in offline, virtual and physical prosthesis control in real time, real-time motion testing for quantification of control performance. Using this platform, we can choose an arbitrary number of motion classes (up to 22 upper-limb movements) as the targets of a virtual prosthesis. This platform has served as an important research platform to perform many lines of research works at RIC group and others.

A pilot work (Lock et al., 2005) has shown that offline classification accuracy across different classifiers has only a weak correlation with real-time performance in an objective task. This indicates that real-time performance metrics are required to examine the clinical robustness of various pattern recognition techniques and improvements. Towards this end, RIC has developed a protocol in which subjects must control a virtual arm. Experiments with the virtual prosthesis are performed immediately following classifier training. Subjects are instructed to follow visual prompts for each movement. A virtual arm which responded to the class decisions allows subjects to observe the real-time results of their movement commands. Subjects are asked to sequentially perform a series of motions and to maintain each muscle contraction until the virtual arm completed the movement. Dynamic data in performing each movement are recorded and used to quantitatively evaluate the speed and consistency of pattern recognition control in real time.

4.2. Real-time performance metrics

To assess important control parameters and gain insight into the feasibility of clinically implementing EMG pattern recognition based controllers for upper limb amputees, the three real-time performance metrics have been first proposed and used by the research group at RIC (Li et al., 2010). These metrics could also be used for comparing conventional myoelectric control and any new neural-machine-control systems that may evolve in the future. The three performance metrics are:

Motion-Completion Rate (MCR) is defined as the percentage of successfully completed motions. This metric is a measure of performance reliability. A motion trial will be considered completed if it is successfully performed through the full range of motion within the designated time limit. If the target movement is not completed within the time limit, the movement will be considered a failure.

Motion-Completion Time (MCT) is defined as the time taken to successfully complete a movement through the full range of motion. This metric is a measure of speed of use. MCT is measured as the time from the onset of movement to the completion of the intended movement.

Motion-Selection Time (MST) is defined as the time taken to correctly select a target movement. This quantity represents how quickly motor command information (here represented with myoelectric signals) could be translated into the correct motion predictions. MST is measured as the time from the onset of movement to the first correct prediction of the movement. The onset of movement was identified as the time of the last “no movement” classification; this corresponded to approximately a 5% increase in the mean absolute value of the baseline EMG signals.

4.3. Real-time performance in amputees

Recently, several studies have been conducted to use these real-time performance metrics for quantification of real-time control performance in amputees. The real-time performance metrics was first used by the RIC’s group to quantify the control performance of virtual prosthesis control in five TMR patients with transhumeral or shoulder disarticulation amputations. Ten classes of different elbow, wrist, and hand movements were included in the study (Kuiken et al., 2009). According to this study, the mean motion selection and motion completion times for hand grasp patterns were 0.38 seconds and 1.54 seconds, respectively. These patients successfully completed a mean of 96.3% of elbow and wrist movements and 86.9% of hand movements within 5 seconds, compared with 100% and 96.7% completed by controls. These results suggest that reinnervated muscles can produce sufficient EMG information for real-time control of advanced artificial arms.

Later, another study was done by the RIC’s group in five people with unilateral transradial amputees (Li et al., 2010). Same metrics were used to quantify the real-time performance of virtual prosthesis control in these amputees. Based on the results of this study, the wrist movements could be selected and completed quickly with both the amputated and intact limbs, with no difference between arms. Similarly, the motion-completion rates for wrist movements with both arms were close to 100%. When hand grasps were successfully performed in 5 s or less with the amputated arm, they were selected and completed just as quickly as with the intact arm, but fewer hand grasps were successfully performed with the amputated limb. From these findings, it appears that motion-completion rate was the most telling performance metric. It is obvious that a high completion rate will be needed for adequate prosthesis function and to prevent user frustration.

Note that Quantifying operation of a virtual arm allows measurement of some useful metrics in the laboratory. However, the ultimate goal is for amputees to operate more dexterous prosthetic arms. Controlling a real prosthesis introduces many practical challenges, such as stability of EMG signal recording, interference from muscles controlling remaining joints, and the effects of tissue loading and arm dynamics.

5. Summary

The limb muscle cells can be activated by an intentional limb movement to generate myoelectric signals which are able to be recorded using electrodes. The surface recordings of myoelectric signals are effective and important input signals in control of prostheses for people with limb amputations. EMG pattern-recognition-based control systems of myoelectric prostheses rely on the myoelectric signal to convey information regarding intent from the user to the prosthesis controller. The previous efforts have showed that using a pattern classification technique, an intentional movement can be predicted with the distinguishable characteristics of EMG patterns; this new method allows users to intuitively operate their myoelectric prostheses with multiple degrees of freedom.

Many encouraging progresses have been made in EMG pattern-recognition-based control of multifunctional prostheses. However, currently there is no any multifunctional myoelectric prosthesis system available for clinical use. The primary limitation may be lack of reliability and stability of current pattern recognition control, which have substantially hindered this technique from getting clinical applications. Further research and development need to be conducted before field trials can be performed. Improving EMG signal recording repeatability and stability are required to minimize or eliminate daily classifier training. Work is ongoing to develop more robust surface EMG recording systems and prosthetic interfaces. Adaptive pattern-recognition algorithms also may improve the stability of control. Various existing hierarchical control schemes may be more robust for some patients; customization of control hierarchy is an accepted practice in modern prosthetics. These early trials of TMR technique demonstrate its feasibility and realization in control of complex multifunction myoelectric prostheses.

Acknowledgments

The author would like to thank Dr. Todd Kuiken’s team at Neural Engineering Center for Artificial Limbs, Rehabilitation Institute of Chicago, USA, for supports. This work was supported in part by the Shenzhen Governmental Basic Research Grand #JC200903160393A and JC201005270295A the National Natural Science Foundation of China under Grant #60971076, Hong Kong Innovation and Technology Fund (ITF) #GHP/031/08, and the Shenzhen Engineering Laboratory of Motion Function Rehabilitation Technology.

Mili Mandal

Chapters authored

Related collaborators

Tomislav Benjak

Stephen Bustin

Gorka Vuletic

Ponzio

Stella Elkabes

Pan Zhang

Junichi Sadoshima

Sayantani Basak

Peiyong Zhai

Robert Donnelly