Three types of prosthetic hand are currently available: cosmetic, body-powered, and myoelectric (Laschi et al., 2000). Cosmetic prostheses are passive, and designed to look like the natural hand, with solely an aesthetic purpose. Body-powered prostheses are powered and controlled by body movements, generally of the shoulder or of the back. Myoelectric hands are electrically powered and controlled by electromyographic (EMG) signals;
Akazawa’s Lab has developed a myoelectric prosthetic hand (
The current design of the
An additional initial problem is that shortly after an amputation atrophy of the remnant muscles occurs, and their EMG signal becomes very weak. As that EMG signals are used to control the prosthesis, users wanting to wear the
In order to solve the two problems mentioned above, we developed a graphic simulator system for the
A number of works on prosthesis simulators have been already described, each of them fitting the specific requirements of a given prosthesis. Yamada et al. (1983) employed six different bi-dimensional (2D), fix images appearing on the screen depending on the frequency and amplitude pattern of three EMG signals in order to evaluate their proposed control method for a theoretical prosthetic hand. Daley et al. (1990) developed a simple 2D graphical simulator for operator performance comparison when using different myoelectric control strategies. Abul-Haj and Hogan (1987) performed an emulation with a combination of software and hardware for elbow-prosthesis prototypes evaluation. Perlin et al. (1989) developed a simulation program for their Utah/MIT 16-joint, four-finger
Several works describe simulators operated by shoulder movement; Zahedi and Farahani (1995), for example, used a graphical simulator for a fuzzy EMG classifier; Durfee et al. (1991) created a 2D graphic simulator to evaluate command channels trough which control an upper limb neural prosthesis; and Zafar and Van Doren (2000) employed a video-based simulator for a shoulder-activated neuroprosthesis for spinal cord injured persons. Lin and Huang (1997) made a computer simulation of a robotic hand to test its potential use as a prosthesis.
There are already some commercially available systems such as
However, all the abovementioned simulators can be used only with prosthetic hands with a switched or single proportional mode, not for those with a more complex control mode as the one of the
The goal of the present work was to develop an upper limb and hand graphic simulator system that solves the abovementioned problems, allowing amputee subjects to try virtually the myoelectric hand without needing the socket, and to perform the physical training required prior to use the real one. This simulator allows us also to easily identify the optimal electrode location for the EMG signals acquisition in each individual. In addition, the simulator allows physicians and related staff to recognize how easily the hand can be controlled and its advantages over other kinds of prosthesis.
The simulator that we have developed consists of a data acquisition system, a mathematical model that simulated the behavior of the
2. Materials and methods
2.1. Structure of the
A general overview of the system is shown in Figure 1. An exceptional feature of the
To obtain such a control, the
As shown in Figure 1, for each subject a pair of surface electrodes were put on the
From those two calculated torques, the desired finger angle of the end effector (the target angle the user wants to achieve) was calculated as
where is the grip force exerted by the fingers of the
where the time constants were calculated to be and , and the gain
in proportion to the contraction level of the extensor-flexor muscles pair. The user can regulate the stiffness of the hand fingers angle by varying the level of contraction of each of those muscles. The stiffness at resting state is 0.1 Nm/rad, and the coefficient is 0.98 rad-1. A software program implementing this model was introduced in the microprocessor that controls the end effector.
The position control system (see Figure 1) consists of a DC motor (MINIMOTOR SA, Croglio, Switzerland, type 2233), its servo controller (Figure 1(c2)), and a one degree-of-freedom end effector with three fingers (Figure 1(c3)). Index and middle fingers are bound between them and are endorsed with an open-close movement with respect to the thumb. This movement is produced by the DC motor, the servo controller of which works to nullify the difference between the commanded angle and the actual motor rotational angle as measured by an optical encoder.
2.2. Composition and operation of the simulator
Figure 2 shows the components of the simulator system, which can be divided into three main sub-systems: data acquisition (EMG and video), processing, and display. Ten light emitter diode (LED) markers and two pairs of surface electrodes are attached to the subject’s upper limb as shown in Figure 2(a) and 2(b). Those LEDs and electrodes provide the inputs for the processing sub-system, which is implemented in the graphic workstation (Figure 2(c)).
Figure 3 shows the block diagram of the simulator, illustrating how the processing system (Figure 3(c)) determines the position of the upper limb from the three-dimensional (3D) location of the markers on the shoulder, elbow, and wrist detected with an OPTOTRAKTM 3D camera (NORTHERN DIGITAL Inc., Ontario, Canada) (Figures 2(a) and 3(a)). The processing system determines the desired finger angle from the processed surface EMGs of both wrist flexor and extensor muscles of the subject (Figures 2(b) and 3(b)). The virtual upper limb and hand are ultimately presented on the 3D graphic workstation (display system, Figure 3(d)).
2.2.1. Data acquisition system
The OPTOTRAK 3D camera (Figure 2(a) and Figure 3(a)) detects the position of the LED markers attached to the user's shoulder, elbow, and wrist. The marker on the shoulder is attached to the point where the movement of the
To measure the rotational angle of the wrist during an external pronation of the arm, eight LEDs are placed on the external side of a bracelet-like device attached to the wrist (shown in the inset of Figure 2).
The EMG signals of wrist muscles are picked up with surface electrodes (Figure 2(b) and Figure 3(b)). These signals are then amplified (gain 58.8 dB, CMRR 110 dB) to the range ±5 V, full-wave rectified, smoothed with a second order low-pass filter (cut-off frequency 2.7 Hz), and then sampled at a frequency of 25 Hz, 12 bits per sample (resolution of ±2.4 mV, less than 0.01% of the maximum value) with an OPTOTRAK Data Acquisition Unit (NORTHERN DIGITAL Inc., Ontario, Canada).
2.2.2. Processing system
The location of the LEDs and the processed EMG signals are collected by a graphics workstation (GW, SILICON GRAPHICS, Inc., California, USA) that holds the processing system software (Figure 2(c) and Figure 3(c)).
The angle that the user wants to achieve with the prosthesis fingers (the target angle) is given by Eqs. (2) and (3) using the current value of user’s EMG signals and . Those equations are calculated in the real
Dynamics of the DC motor servo system of the actual prosthetic hand were calculated in terms of the relationship between target angle and rotational angle of the motor shaft (see Figure 1). In the steady case, we assumed , with zero time delay (Figure 3(c2)).
In order to model the relationship between and final finger angle of the real
2.2.3. Display system
The tasks depicted in Figure 3(c1) to (c4) were implemented in a program that used OpenGL graphical library to represent the virtual arm and prosthetic hand by the wire-frame drawing shown in Figure 3(d). The refresh rate was 25 frames/s, which was sufficient to give the impression of smooth motion. In addition, the GW displayed the processed EMG signals used as input. During the experiments, supplementary information was displayed to guide the subject to achieve the proposed goal.
2.3. Common experimental set-up
To test the performance, validity, and controllability of the simulated hand, several experiments were carried out with three male, able-bodied subjects aged 22, 24, and 32; and a 43-years-old male who had both hands amputated 18 years earlier after a traffic accident. All of them gave their informed consent.
The amputee subject uses a body-powered hook at the end of the right upper limb and a body-powered hand at the end of the left upper limb. He had worn a myoelectric prosthetic hand on the right upper-limb until four years before the experiment. Since then, he has not been actively using his forearms muscles; for this reason, he suffered from muscular atrophy (very weak muscles) in both forearms. Consequently, his EMG signals corresponding to the maximal voluntary contraction (MVC) had a value of less than 20% of the average MVC of the three non-amputee subjects (0.65V/3.51V). In addition, he exhibited a slightly higher level of involuntary co-contraction (simultaneous contraction of antagonist muscles) in his wrist flexor and extensor muscles.
All subjects performed the same protocol composed of four sessions. One session consisted of two different tasks: angle and position control. Subjects repeated these tasks from three to five times in each session.
The subject sat barefoot in a chair, with one foot on a steel sheet on the floor in front of the chair and with sleeves rolled up to expose the forearm. The steel sheet was used as reference voltage for the EMG processing unit. The subject was instructed to sit in a relaxed position in a chair, with the forearm and the arm forming an angle of about 15o. The forearm of the right-hand was cleaned with SkinPure skin abrasion gel (NIHON KOHDEN Corp., Tokyo, Japan) and ethanol. A pair of bipolar, surface electrodes (Ag-AgCl, 1 cm in diameter; NIHON KOHDEN Corp., Tokyo, Japan. Type NS-111U) was attached, with a centre-to-centre distance of about 2 cm, following the muscle fiber direction of the wrist flexor muscle (
Before starting the experiments, the subject was instructed to exert for 1 second his maximal contraction of each target muscle from which the EMG signals were taken. The simulator calculated the MVC amplitude value for each muscle as the average around its EMG peak (the maximum detected value). The EMG signals of each subject were normalized to the range of 0-1 by their respective MVC values.
To familiarize the subject with the equipment and functioning of the simulator, the subject was firstly instructed to freely move the virtual hand contracting his forearm muscles. When he felt comfortable with the system, the different sessions of experiments were performed. In order to avoid fatigue, a rest was scheduled between tasks, and the subject was not asked to keep any of the postures for more than a few seconds (Basmajian and Deluca, 1985; Kampas, 2001).
After the experiments, a short questionnaire was given to the amputee volunteer to gather feedback on the
3. Experiments and results
Two types of experiments were carried out; the ones of the first block (3.1) were oriented to check whether the behavior of the simulator corresponded to the behavior of the
3.1. Behavior of the simulator system
3.1.1. Validation of the input-output relationship
EMG signals were acquired from one subject as explained in the previous section and given as input to the simulator and, simultaneously, to the
Figure 5 shows the result of one of these experiments, carried out with a non-amputee subject freely moving the simulated hand. Figure 5(a) shows the inputs of the system: processed EMG signals of the wrist extensor muscle (dashed line) and those of the wrist flexor muscle (solid line). Figure 5(b) shows a comparison between the finger angle of the
3.1.2. Variable stiffness
As one of the main features of the
To simulate different levels of co-contraction, we fed the simulator with different levels of and under the condition () (see Figure 3(b) and (c1)). We sinusoidally modulated the applied grip force (see Figure 3(c4)) at a frequency of 0.2 Hz and a range between -0.08 V and 0.08 V, which corresponds to the actual output amplitude of the strain gauges. Figure 6 shows that as increased –that is, as the level of co-contraction increased-, in response to the same perturbation , the finger angle displacement decreased; that is, the stiffness increased. When there was no co-contraction (= 0V), the perturbation caused the total opening of the hand (110o is its maximal aperture), but when the level of co-contraction was maximum (= 10V), the perturbation had nearly no effect on the angle of the hand fingers. Therefore, the simulator behaves like the
3.1.3. Effect of force feedback
We carried out a preliminary experiment to study the simulator behavior when a subject grasped a virtual object (a sphere). The mechanical dynamics of the object were modeled in a simple fashion as a spring (Figure 3(c4)). The exerted force was calculated then as
where is the spring constant; is the finger angle when the contact with the object occurs, and is the current angle. The inputs and to the simulator were given as sinusoidal waves. Figure 7(a) compares the fingers angle when grasping the sphere: continuous line curve corresponds to the experiment carried out without feedback; and the dashed line curve when was calculated as explained above.
The first contact fingers-sphere occurred at (marked in the graph as A), when the fingers angle was around 43o. The maximal grasping force occurs when the fingers angle without was 3o. Therefore, there is a difference of approximately 40o in
Figure 7(b) shows the value of the calculated . When it reaches its maximal value, approximately 25 mV (roughly one third of its maximum), the difference between the fingers target angle with and without pressure feedback is nearly 10o. Therefore, as it happens with the real prosthetic hand (Okuno et al., 1999), gives self-control to the hand over the exerted force when grasping objects, producing a smoother grasping motion.
3.2. Control experiments
3.2.1. Finger angle control
We ran a control experiment to determine how accurately the subjects could control the finger angle of the simulator hand. The effects of using the simulator were also investigated by comparing the performance of the subjects before and after two trails. In this finger angle control task, the subject was asked to achieve a series of eight different angles (from 0o to 110o) showed on the screen of the GW.
Figure 8 shows the typical results obtained, where the target angle to achieve was 55o (thick horizontal line). Figures 8(a) and 8(c) (left column) show the results of the first trial of two different subjects; (a) a sound-limbed subject and (c) the amputee subject. Both subjects needed more than 4 s to be able to keep the angle within the acceptable range, and were able to maintain it there for only less than 2 s (period between points
Figure 8(b) shows the results obtained by the sound-limbed subject after several trials for a period of about 40 min, and Figure 8(d) for the amputee subject after a similar period. In this case, both achieved the angle in just approximately 1 s (point
To measure how accurately the subjects performed the task, we calculated the mean square error made while trying to keep a constant target angle as
where is the hand simulator finger angle in the sample
The average of the error was 1.19o (s.d. 0.67) for the three non-amputee subjects and 1.78o (s.d. 0.54o) for the amputee subject.
3.2.2. Grasping control
An additional control experiment was carried out to examine whether the subjects were able to grasp a virtual object using the simulator hand. In this grasping control task, six spheres with different diameters (from 2 mm to 10 cm) were depicted one at a time on a fix position on the simulator screen. The subject was instructed to grasp them with the virtual hand. To give some feedback to the subjects about the virtual force exerted over the sphere, a second index finger was drawn (with a very faint color) in the position the simulated index finger would have been if there was not a solid object in its way.
The results of this experiment were very similar to the ones shown in Figure 8. In this case, we defined the error as the distance between the index fingertip and the surface of the sphere. The average error while subjects tried to hold onto the sphere in the last two trails of each subject was 1.32 mm (s.d. 0.47). For the amputee subject, the average error was 1.77 mm (s.d. 0.63). In conclusion, subjects were able to grasp the object, and to do it in a smooth, natural way. These results prove that the simulator developed in this work is a valid tool for rehabilitation.
In this study, we have introduced a simulator of our biomimetic, myoelectric prosthetic hand (
We have demonstrated that the simulator output agrees sufficiently for practical use with the finger angle of the prosthetic hand when both are given the same input.
Usefulness of the simulator has been shown in the experiments of controlling angle and stiffness of the hand. After a short period of training, subjects were able to control quite accurately the simulated hand. The precision achieved by an amputee subject was nearly as good as the precision obtained by the three non-amputee subjects, even though the amputee had not actively used his forearm muscles for four years.
This kind of powered myoelectric prostheses is not yet widely known. For example, in Japan only 350 units have been sold in the last 30 years (report of the Ministry of Health, Labour and Welfare of Japan). Our simulator could be accessible to physicians and related staff and be used to offer the opportunity to a wider group of amputees to try a myoelectrically controlled prosthesis.
The simulator can also be used for EMG signal processing and modeling. For example, when new features are added to the
This simulator could be easily adapted to any myoelectric prosthesis, by performing just a few simple modifications on its software.
This work was partially funded by the Ministry of Education, Culture, Sports, Science, and Technology of Japan. G.A.G. was funded by a grant from the same Ministry (
G.A.G. thanks Professor Pedro García Teodoro (Granada University, Granada, Spain) for encouragement and scientific support during the first stages of this project.
Authors would like to thank as well Dr. Sandra Rainieri (AZTI Foundation, Bilbao, Spain) and Professor Antonio Peinado (Granada University, Granada, Spain) for useful comments and input on the original manuscript.