Control Strategy for Underactuated Multi-Fingered Robot Hand Movement Using Electromyography Signal with Wearable Myo Armband

Abstract

The main goal of this research is to develop a control strategy for an underactuated robotic hand, based on surface electromyography (sEMG) signal obtained from a wireless Myo gesture armband, to distinguish six, several hand movements. The pattern recognition system is employed to analyze these gestures and consists of three main parts: segmentation, feature extraction, and classification. A series of 150 trials is carried out for each movement and it is established which was most suitable for electromyography signals that can be later used in recognition systems. A backpropagation neural network was used as a classifier. The architecture has a hidden network and six output layers. The number of neurons of the hidden network (20) was determined based on the performance in training progress. The proposed system is tested on datasets extracted from five healthy subjects. A great accuracy (94.94% correct assessment). between the experimentally values and those predicted by the artificial neural network (ANN) was achieved. In addition, kinematic analysis of the proposed underactuated hand has been carried out to verify the motion range of the joints. Simulations and experiments are carried out to verify the effectiveness of the proposed fingers mechanism and the hand prosthesis to generate grasp or postures.

1. Introduction

Due to its use in majority of activities of daily living (ADL), the human hand is one of the most essential body parts, which enables humans to perform basic daily activities ranging from hand gestures to object manipulation [1]. Amputation results from accidents at work, armed conflicts, diseases, and malformations [2], and it is a severe mental and physical trauma with the loss of both motor and sensory perceptions [3].

High-performance prosthetic hands significantly improve the quality of life for upper limb amputees. Passive prosthetic hands are lightweight, robust, and quiet but can only perform a limited subset of activities. Therefore, researchers have investigated externally powered prosthetic hands for upper limb amputees for more than a century. Regarding the prostheses development, the main contributions are the robotic hand designs such as Vincent hand [4], iLimb hand [5], iLimb Pulse [6], Bebionic hand [7], Bebionic hand v2 [8], Michelangelo hand [9], Metamorphic hand [10], etc. (Figure 1). These devices, which emulate with human hand, are characterized by complex systems based on microprocessor technology.

Figure 1.

(a) Vincent hand (Vincent Systems), (b) iLimb hand (touch bionics), (c) iLimb pulse (touch bionics), (d) Bebionic hand (RSL steeper), (e) Bebionic hand v2 (RSL steeper), and (f) Michelangelo hand (Otto bock).

All these myoelectric prostheses harness the EMG signals of residual limb muscles to trigger the function of a robotic prosthetic arm or hand. While these devices have been greatly improved over the past decade, they are still limited due to their number of controllable degrees of freedom, intuitiveness, and reliability. Additional restrictions include weight, limited battery life, cost (there are very expensive due to the amount of actuators, sensors, and the electronics involved), and the user’s inability to control multiple degrees of freedom simultaneously and consistently [11]. Therefore, despite numerous advances in the field, rates of user abandonment for upper limb prosthetic systems remain high.

Underactuated robotic prosthesis (URP) is an emerging research direction in the field of robotic medical devices. The control input of the underactuated prostheses is less than the degree of freedom of the system. It has the advantages of lightweight, low energy consumption, and excellent performance. Compared with full-drive prostheses, URP has fewer drives while maintaining the same DOF, thus reducing mass, volume, and energy consumption. It is ideal for applications where quality, volume, low power consumption, and low cost are desirable.

Natural muscles provide mobility in response to nerve impulses. Electromyography (EMG) measures the electrical activity of muscles in response to a nerve’s stimulation. Surface electromyography (SEMG) is a noninvasive method of measurement of the bioelectrical activity of muscles. EMG signals have been used extensively in the identification of user intention to potentially control assistive devices such as smart wheelchairs, exoskeletons, and prosthetic devices [4, 7, 8]. Feature extraction and signals’ classification are essential subsystems of this approach.

Pattern recognition-based myoelectric control consists of feature extraction and feature classification of segmented data in signal processing to command to the motor controller. Some signal processing may include feature reduction or feature selection between extraction and classification, depending on the number of features. In general, various features are extracted in time, frequency, and time-frequency to identify the information content of EMG signals [12]. Surface electromyography (sEMG) signals are information bearers that correspond to the hand posture intention. Finite gestures could be identified [13], if considering that the characteristics extracted from each sEMG signal are directly related to the gesture and the number of gestures (including their positions and speeds).

2. Materials and methods

2.1 Data acquisition

Raw sEMG signals were collected with Myo armband placed on forearm’s transradial portion as usually performed in common applications. The Myo armband is a wireless wearable technology (Figure 2) and has eight medical grade stainless steel EMG sensors. Similar to other surface electrodes, the EMG signals returned by the sensors represent the electric potential of the muscles because of muscle activation. The Myo armband also has a nine-axis inertial measurement unit (IMU) which contains a three-axis gyroscope, three-axis accelerometer, and a three-axis magnetometer [14].

Figure 2.

MYO gesture armband device and its parts, Thalmic Labs.

From these units, the orientation and movement of a wearer’s arm can be determined through analyzing the spatial data provided. The angular velocity of the armband is provided in a vector format and the accelerometer represents the acceleration the Myo armband is undergoing at a given time. However, the Myo armband is better suited for determining the relative positioning of the arm rather than the absolute position, a consideration to be aware of when applying pattern recognition algorithms. Currently, the Myo armband is able to pull IMU data at sampling rate of 50 Hz. The system is supported on Windows, IOS. MAC and Android and has a Bluetooth 4.0 Smart Wireless connection and a 32-bit ARM Cortex M4 processor with a lithium battery. Signals extracted by Myo armband was processed using Matlab software.

The Myo gesture control armband was used to identify six hand gestures that are the basic ones to achieve the improvement of the grasp: power grasp, palm inward, palm out, open hand, grasp type grasp and hand in rest (Figure 3).

Figure 3.

Hand gestures identify with the MYO device.

Eight datasets were recorded of six hand gestures for five healthy subjects (three males, two females). All subjects did not have any experience in attending this kind of research before. The inclusion criteria adopted in this research were as follows: no evidence in their medical history of peripheral neuropathy, diseases of the central nervous system, and restricted mobility. The EMG signal has a typical amplitude of ±6 mV, and the useful frequency is in the range from 10 to 500 Hz with the greatest amount of concentrated energy up to 150 Hz [15]. According to Calderon et al. [16], the first 400 ms of a muscular activity are enough for the identification of the movement, so the signal was extracted considering this elapsed time as shown in Figure 4.

Figure 4.

(a) Acquisition of sEMG signals through the developed application in Matlab. (b) Signals obtained from the eight sensor channels.

The performed sequence to capture the myoelectric signals is as follows: 200 samples per second were taken for each grasp or hand gesture in an interval of 20 s, it means, there are 4000 samples per sensor. Between each one of the six proposed hand gestures transitions of 5 s were made as was recommended in [15]. The myoelectric signals capture of each sensor was executed at a frequency of 200 Hz [16].

An application for signal processing using a GUI, in the MATLAB® Classification Learner library was developed (Figure 4). The GUI shows nine graphs, one for each channel of the myoelectric signals with the Myo armband device and one where all the signals are displayed. The computed data of these myoelectric signals are stored as matrices format for later offline processing.

2.2 Pattern recognition system

The proposed system for sEMG processing that includes several blocks of preprocessing, segmentation, feature extraction and the neural network development is shown in the flowchart of Figure 5.

Figure 5.

A flowchart for the proposed extracting sEMG signals.

2.3 Preprocessing

Usually, the collected sEMG signals are normally noisy due to ambient noise, motion artifact, inherent noise in electronics equipment, and inherent instability of the sEMG signal. When using the Myo gesture armband, practically the noise ratio in sEMG signals is low and does not affect the sEMG.

However, in the preprocessing analyses signals was forced to pass through band pass filters or adaptive filtering to eliminate the undesirable frequency content [17]. The adaptive filter is a system that receives two signals: $x\left(n\right)$ and $e\left(n\right)$, the latter is called an error signal and becomes from the subtraction of a signal called the desired signal or reference, $r\left(n\right)$, and another that is the filter outlet $y\left(n\right)$ (see Figure 6).

Figure 6.

Flowchart of an adaptive filter.

$$e\left(n\right)=r\left(n\right)-y\left(n\right)$$E1

The filter coefficients are called $w\left(n\right)$, which are those that multiply the input $x\left(n\right)$ to obtain the output.

$$y\left(n\right)=w\left(n\right)\ast x\left(n\right)$$E2

The purpose of the device is to make the error signal to be zero. For this, the system must be configured so that, from the input signal $x\left(n\right)$, the output $y\left(n\right)$ is generated and it is equal to the signal reference $r\left(n\right)$. Each way to minimize that error is an example of implementing adaptive filters. For instance, it could be proposed to minimize the cost function $J=2\ast e\left(n\right)\ast x\left(n\right)$, applying the delta rule would obtain the new coefficients such as:

$$w\left(n+1\right)=w\left(n\right)-\alpha \nabla J$$E3

where the constant “$\alpha$” is used to adjust the convergence speed and avoid possible instabilities. Solving, then:

$$w\left(n+1\right)=w\left(n\right)-2\alpha \ast e\left(n\right)\ast x\left(n\right)$$E4

The algorithm implemented for learning an adaptive system could be:

1. Initialize the weights randomly

2. Choose an $“\alpha ”$ value

3. Calculate the output y(n)

4. Calculate the error e(n) Eq. (1)

5. Update the weights with the chosen cost function

6. Repeat a certain number of times from point 3

The differences between raw and filtered signals are presented in Figure 7.

Figure 7.

The difference between raw and filtered signals.

2.4 Segmentation

These are disjoint EMG segmentation and overlapped segmentation. In disjoint segmentation (a), different segments are used which have predefined length and these segments are used for feature extraction. In case of overlapped segmentation (b), a new segment is placed over the present segment with an increment. Thus, the disjoint segmentation deals only with the segment length while the overlapped segmentation deals with segment length and increment. Overlap technique is chosen for segmenting portion of the signal in this research work.

2.5 Feature extraction

The time domain (TD) feature is the feature extracted from EMG signal in time representation [16, 18]. TD features such as mean absolute value (MAV), zero crossing (ZC), Wilson Amplitude (WAMP), variance (VAR), and wavelength (WL) were most popular in EMG pattern recognition due to high processing speed in classification. MAV is defined as the average of total absolute value of EMG signal [6, 14]. It can be calculated as:

1. Mean absolute value: the appreciation of the absolute mean value of the x signal in segment i of N samples is given by equation:

   $${\overline{X}}_t=\frac{1}{N}\sum _{K=1}^N\mid x_k\mid ,\mathit{\text{para}}i=1,\dots ,I-1$$E5

2. Wavelength (WL): WL is an improvement of integrated EMG feature and is defined as a cumulative length waveform over the segment. It can be represented as:

   $$l=\sum _{K=1}^N\mid x_k-x_{k-1}\mid$$E6

3. Zero crossing (ZC): it has frequency related features, which represents counts of how much signal amplitude crosses the zero amplitude over time segment. It is measuring the frequency shift and shows the number of signal sign variations. The mathematical representation of ZC is as follows

   $$\mathit{ZC}=\sum _{K=1}^N\mathit{sgn}\left(\left[x_k-0.4\right]\left[x_{k+1}-0.4\right]\right)$$E7

   where

   $$\mathit{sgn}\left(x\right)=\left\{\begin{array}{c}1x>\mathit{\text{Limit}}\\ 0\mathit{\text{the rest}}\end{array}\right.$$

4. Wilson amplitude (WAMP): this is the number of times that the difference between two consecutive amplitudes in a time segment becomes more than threshold. It can be formulated:

   $$\mathit{\text{WAMP}}=\sum _{K=1}^{N}f\left(\left|x_k-x_{k+1}\right|\right)$$E8

5. Variance (VAR): it is used to determine thickness, density of EMG signal power.

$$\mathit{VAR}=\frac{1}{N-1}{X}_k^2$$E9

2.6 Classification

The EMG data taken from muscle myoelectric signals were grouped in a vector to be used as inputs in a creation of a back-propagation neural network as in [19, 20, 21], with 20 neurons in the occult layer and 6 outputs. Thirty characteristic vectors were taken for each of the six selected movement patterns (power grasp, inward palm, outward palm, open hand, pincer grasp and rest), which allowed the training of the network.

2.7 Creation of the artificial neural network

Neural networks are created using Matlab software. To train and test the network, the dataset is divided into three sets. Datasets were divided in 60% for training, 20% for testing and 20% for validation. This is the most common method of neural network validation [19, 22, 23]. The true error is calculated directly as the test set error, and bias can be calculated by subtracting the apparent error (training set error) from the test set error.

2.8 CAD design

The five-fingered underactuated prosthetic hand was design using Solidwork 2019 package. This hand has five fingers, driving each one by a DC motor, a worm gear transmission and a four bar mechanisms [24, 25, 26, 27] (Figure 8). Each finger has three joints. The manufacturing of the hand prostheses was carried out with an Ultimaker 3 printer for its good performance and excellent print quality of the figures with a thickness, height and amplitude from a design made by a computer. The polylactic acid or polylactide thermoplastic was used as a raw material.

Figure 8.

Design of the anthropomorphic under-performed hand. (a) Proposed prosthetic hand. (b) Underactuated finger.

2.9 Control system

The control system was implemented using the prosthetic hand, Arduino UNO microcontroller, laptop computer, and Myo armband. The Arduino has 14 pins that can be configured as input or output which have the advantage of being able to connect any device that is suitable for transmitting or receiving digital signals of 0 and 5 V. It also has analog inputs and outputs. By means of analog inputs, sensor data can be achieved in the form of continuous variations of a voltage. The analog outputs can normally be used to send control signals in the form of pulse width modulation (PWM) signals.

Labview software was used as a graphical development environment platform, ideal for designing systems with a friendly programming language. It allows to structure and perfect professional works of any system. Communication is made through the serial port between the Arduino and Labview. Through the Arduino UNO card using PWMs, the system controls the DC motors to achieve the different positions and grasps previously established.

Tests were carried out for each one of the actuators to be manipulated and it was verified that the position of each finger for the different types of grasps already proposed to be carried out were those established. For the handling of the robotic hand, a checking was made in the LabVIEW interface. To ensure that the time and angles of the servomotors are correct, tests and measurements were carried out to analyze the inverse and direct kinematics of the prototype, which are analyzed in specific details in the following section.

3. Results and discussion

3.1 Pattern recognition system

Feature extraction was carried out with each posture using the series of time domain, and processing with the Matlab software for six motions or positions of the hand. From Figure 9, it can be seen the system behavior trough the developed GUI. Below, eight myoelectric channels are shown. Likewise, at the bottom, on left, real-time motions or types of gestures can be observed.

Figure 9.

Feature extraction for the open hand gesture trough GUI.

Figure 10 displays the results of feature extraction, in this case for the power grasp gesture. The less useful features are preferably deleted to decrease the computational time, especially in real-time. It can be observed that characteristic with less accuracy (37.18%) was the Wilson amplitude for the identification of the pronation, and the best one with the better identification (96.45%) was the mean absolute value (MAV) in a 50 mS window.

Figure 10.

Feature extraction for the power grasp gesture.

Figure 11 and Table 1 explain feature effectiveness gesture hand. From Figure 11 and Table 1, it is concluded that the maximum percentage, 96.55% in this 50 mS window, was obtained with the characteristic of Wilson’s Amplitude in the grasp gesture. Similarly, the lowest percentage of 37.18% in this window was also obtained with the same characteristic but in the pronation gesture. Likewise, the AVM in this 50 mS window presents an average of 82.31%, which is higher in all characteristics.

Figure 11.

Features extraction of myoelectric signals for the six gestures at 50 mS window.

		Ventana de 50 mS
Caracteristica EMG	Repose	Grasp	Propination	Supination	Extension	Flexion
MAV	94.45%	85.54%	56.63%	85.94%	84.79%	86.48%
WL	83.35%	49.25%	92.24%	82.50%	83.87%	53.36%
ZC	89.83%	55.16%	60.61%	88.48%	72.91%	63.72%
WAMP	91.11%	96.51%	37.18%	92.01%	83.17%	83.65%
VAR	87.78%	89.65%	64.72%	92.13%	39.95%	51.22%

Table 1.

Feature records for selected gestures at 50 mS window.

Figure 12 and Table 2 identify the features effectiveness for 100 mS windows. It is concluded that the maximum percentage, 99.25% in this 100 mS window, was obtained with the characteristic of MAV in the repose gesture. Similarly, the lowest percentage of 42.68% in this window was obtained with the VAR characteristic in the extension gesture. Likewise, the MAV in this 100 mS window presents an average of 85.36%, which is higher in all characteristics.

Figure 12.

Features extraction of myoelectric signals for the six gestures at 100 mS window.

		Ventana de 100 mS
Caracteristica EMG	Repose	Grasp	Propination	Supination	Extension	Flexion
MAV	99.25%	87.45%	64.25%	87.37%	86.71%	87.13%
WL	84.67%	54.78%	93.18%	81.56%	85.81%	63.21%
ZC	81.23%	56.89%	63.23%	89.75%	75.36%	66.25%
WAMP	89.34%	95.34%	42.87%	91.34%	84.67%	84.83%
VAR	86.63%	87.58%	67.39%	91.67%	42.68%	65.24%

Table 2.

Feature records for selected gestures at 100 mS window.

Figure 13 and Table 3 identify the features effectiveness for 250 mS windows. It is concluded that the maximum percentage, 99.53% in this window, was obtained with the characteristic of MAV in the repose gesture. Similarly, the lowest percentage of 37.18% in this window was obtained with the Wilson’s Amplitude in the grasp gesture. Likewise, the MAV in this window presents an average of 86.06%, which is higher in all characteristics.

Figure 13.

Features extraction of myoelectric signals for the six gestures at 250 mS window.

		Ventana de 250 mS
Caracteristica EMG	Repose	Grasp	Propination	Supination	Extension	Flexion
MAV	99.53%	90.29%	56.63%	89.78%	90.31%	89.84%
WL	86.23%	60.45%	92.24%	82.50%	84.12%	64.21%
ZC	83.46%	57.81%	60.61%	88.48%	72.91%	65.05%
WAMP	90.26%	94.62%	37.18%	92.01%	83.17%	84.12%
VAR	86.48%	88.93%	66.13%	91.89%	41.78%	64.23%

Table 3.

Feature records for selected gestures at 250 mS window.

From the previous data, it can be concluded that from the probability matrices obtained for each of the characteristics, with the highest accuracy was the time domain features of mean absolute value (MAV). It guarantees the highest probabilities of success were obtained compared to the other characteristics. Similarly, it is conclusive as in [18, 28], that with just one measure such as the MAV, the different movements can be fully identified.

Likewise, such features show a greater percentage of accuracy as the windowing increases in the samples analyzed by the different features in the time domain. These results match others like [15, 16, 17].

3.2 Classification

Regarding the neural network, created by using the Levenberg-Marquardt algorithm, a root mean squared error (RMSE) of 2.9437 × 10⁻⁹ was obtained, which was inferior to what was expected (10⁻⁹) in only 16 training epochs (Figure 14).

Figure 14.

Training algorithm used in ANN validation.

The training correlation coefficient was 94.94% (Figure 15). The error in the training set (therefore the estimation of the real error) depends mostly on the exact sample, chosen for the training and the exact sample for the test (which are completely dependent on each other since they are mutually exclusive).

Figure 15.

Correlation between the experimental values and the ANN predicted values.

3.3 CAD design

The designed prototype was created as anthropomorphic hand and able to perform three types of hand grasping (tip, spherical, and cylindrical).

The direct and inverse kinematics of the prototype was developed based on the antrophormism of the robotic hand. Figure 16 shows index finger prosthesis prototype model in order to calculate. The Denavit Hartenberg (DH) parameters. The dimensions of the phalanges are as follows: $L_1=2\mathit{\text{inch}}$, $L_2=1,37\mathit{\text{inch}}$, and $L_3=1\mathit{\text{inch}}$.

Figure 16.

Index finger prosthesis prototype model of a robotic hand.

The Denavit Hartenberg (DH) parameters of the finger are shown in Table 4:

DH parameters	Finger joints
	1	2	3
qi	q1	q2	q3
di	0	0	0
ai	L1	L2	L3
αi	0	0	0

Table 4.

Denavit Hartenberg (DH) parameters of the finger.

Direct kinematics allows knowing the position and orientation of the distal phalanx, which is:

$$P_x=L_3\left(C_1{C}_{23}-S_1{S}_{23}\right)+L_2{C}_{12}+L_1{C}_1$$E10

$$P_y=L_3\left(S_1{C}_{23}-C_1{S}_{23}\right)+L_2{S}_{12}+L_1{S}_1$$E11

Donde

$$S_1=\mathit{Sin}\left(q_i\right),C_i=\mathit{Cos}\left(q_i\right),S_{\mathit{ij}}=\mathit{Sin}\left(q_i+q_j\right),C_{\mathit{ij}}=\mathit{Cos}\left(q_i+q_j\right)$$E12

The direct kinematics of the finger is

$$q_1={tan}^{-1}\left(\frac{P_y}{P_x}\right)-{tan}^{-1}\left(\frac{L_1\mathit{sin}\left(q_2\right)+L_3\mathit{sin}\left(q_2\right)}{L_1+L_2\mathit{cos}\left(q_2\right)+L_3\mathit{cos}\left(q_2\right)}\right)$$E13

$$q_2={cos}^{-1}\frac{\left(-2L_1{L}_2+2L_2{L}_3\pm \sqrt{{\left(2L_1{L}_2+2L_2{L}_3\right)}^2-16L_1{L}_3\left(L_1^2+L_2^2+L_3^2-P_x^2-P_y^2-2L_1{L}_3\right)}\right)}{8L_1{L}_2}$$E14

and

$$q_3=k{q}_2\left(14\right)k\approx \frac{7}{11}$$E15

The dynamic model of an n-articulation robot, which in Lagrange’s equation can be written as:

$$M\left(q\right)\ddot{q}+C\left(q,\dot{q}\right)+G\left(q\right)=\tau$$E16

Where q is the vector of articulated variables, τ is the vector of generalized forces acting as the manipulating robot, $M\left(q\right)$ is the inertia matrix, $C\left(q,\dot{q}\right)$ is the centripetal forces matrix, and $G\left(q\right)$ is the gravity vector [28]:

$$L\left(q,\dot{q}\right)=K\left(q,\dot{q}\right)-U\left(q\right)$$E17

$$\frac{d}{\mathit{dt}}\left(\frac{\partial L\left(q,\dot{q}\right)}{\partial \dot{q}}\right)-\frac{\partial L\left(q,\dot{q}\right)}{\partial q_i}={\tau }_{i}i=1,\dots 3$$E18

The kinetic energy of the finger can be expressed as

$$K_1=\frac{1}{2}{m}_1{c}_1^2{\dot{q}}_1^2+\frac{1}{2}{I}_1{\dot{q}}_1^2$$E19

$$\begin{array}{l}{K}_2=\frac{1}{2}{m}_2\left(C_2^2{\dot{q}}_2^2-2C_2^2{\dot{q}}_1{\dot{q}}_2-2C_2{l}_1\mathit{Cos}\left(q_2\right){\dot{q}}_1{\dot{q}}_2+C_2^2{\dot{q}}_1^2+2C_2{l}_1\mathit{Cos}\left(q_2\right){{\dot{q}}_1}^2+l_1{{\dot{q}}_1}^2\right)\\ +\frac{1}{2}{I}_2{\left({\dot{q}}_1-{\dot{q}}_2\right)}^2\end{array}$$E20

$$\begin{array}{c}{K}_3=\frac{1}{2}{m}_3(2C_3{l}_1{\dot{q}}_1({\dot{q}}_1-{\dot{q}}_2+{\dot{q}}_3\cos \left({\dot{q}}_2-{\dot{q}}_3\right)\\ +\left(2C_3{l}_2\cos \left(q_3\right)+2l_1{l}_2\cos \left(q_2\right)+C_3^2+l_1^2+l_2^2\right){\dot{q}}_1^2\\ +\left(\left(-4C_3{l}_2\cos (q_3\right)-2l_1{l}_2\cos \left(q_2\right)-2C_3^2-2l_2^2\right){\dot{q}}_2^2\\ +2C_3\left(C_3+l_2\cos (q_3\right)){\dot{q}}_3){\dot{q}}_1+2C_3{l}_2\cos \left(q_3\right)+C_3^2+l_2^2){\dot{q}}_2^2\\ -2C_3\left(l_2\cos \left(q_3\right)+C_3\right){\dot{q}}_2{\dot{q}}_3+C_3^2{\dot{q}}_3^2)+\frac{1}{2}{I}_3{\left({\dot{q}}_1-{\dot{q}}_2+{\dot{q}}_3\right)}^2\end{array}$$E21

The potential energy in the analyzed finger can be expressed as

$$U_1=-m_{1}g{C}_1\mathit{Cos}\left(q_1\right)$$E22

$$U_2=-m_{2}g{l}_1\mathit{Cos}\left(q_1\right)-m_{2}g{C}_2\mathit{Cos}\left(q_1-q_2\right)$$E23

$$U_3=-m_{3}g{l}_1\mathit{Cos}\left(q_1\right)-m_{3}g{l}_2\mathit{Cos}\left(q_1-q_2\right)+m_{3}g{C}_3\mathit{Cos}\left(q_1-q_2+q_3\right)$$E24

Finding solutions for Lagrange’s equations, the dynamic model of the system is

$$\begin{array}{c}{\tau }_1=m_3{l}_1{c}_3\cos \left(q_2-q_3\right)\left(\ddot{q_1}-\ddot{q_2}+\ddot{q_3}\right)-m_3{l}_1{C}_3\sin \left(q_2-q_3\right)\left(\dot{q_1}-\dot{q_2}+\dot{q_3}\right)\left(\dot{q_2}-\dot{q_3}\right)\\ +m_3{l}_1{c}_3\cos \left(q_2-q_3\right)\ddot{q_1}-m_3{l}_1{C}_3\sin \left(q_2-q_3\right)\left(\dot{q_2}-\dot{q_3}\right)\dot{q_1}\\ +\left[-2m_3{l}_2{C}_3\sin \left(q_3\right)\dot{q_3}-2l_1\left(m_2{c}_2-m_3{l}_2\right)\sin \left(q_2\right)\dot{q_2}\right]\dot{q_1}\\ +\left[2m_3{l}_2\cos \left(q_3\right)+2l_1\left(m_2{c}_2-m_3{l}_2\right)\cos \left(q_2\right)+m_3\left(l_1^2+l_2^2+l_3^2\right)+m_1{C}_1^2\right]+m_2{l}_1^2\\ +m_2{c}_2^2+I_1+I_2+I_3]\ddot{q_1}+\left[2m_3{l}_2{C}_3\sin \left(q_3\right)\dot{q_3}+l_1\left(m_2{c}_2-m_3{l}_2\right)\sin \left(q_2\right)\dot{q_2}\right]\dot{q_2}\\ +\left[m_3{l}_2{C}_3\cos \left(q_3\right)+m_3{C}_3{}^2+I_3\right]\ddot{q_3}-m_3{l}_2{C}_3\sin \left(q_3\right){\dot{q_3}}^2\\ -\left[2m_3{l}_2{C}_3\cos \left(q_3\right)+l_1\left(m_2{c}_2-m_3{l}_2\right)\cos \left(q_2\right)+m_3\left(l_2^2+C_3^2\right)+m_2{c}_2{}^2+I_2+I_3\right]\ddot{q_2}\\ -\left(m_1{c}_1-m_1{l}_1-m_3{l}_1\right)g\sin \left(q_1\right)+\left(m_2{C}_2-m_3{l}_2\right)g\sin \left(q_1-q_2\right)\\ +m_3{c}_{3}g\sin \left(q_1-q_2+q_3\right)\end{array}$$E25

$$\begin{array}{c}{\tau }_2=-m_3{l}_1{c}_3\cos \left(q_2-q_3\right)\ddot{q_1}+m_3{l}_1{c}_3\sin \left(q_2-q_3\right)\left({\dot{q}}_2-\dot{q_3}\right)\dot{q_1}\\ +\left[2m_3{l}_2{c}_3\sin \left(q_3\right)\dot{q_3}+l_1\left(m_2{c}_2+m_3{l}_2\right)\sin \left(q_2\right)\dot{q_2}\right]\dot{q_1}\\ -\left[2m_3{l}_2{c}_3\cos \left(q_3\right)+l_1\left(m_2{c}_2+m_3{l}_2\right)\cos \left(q_2\right)+m_3\left(l_2^2+c_3^2\right)+m_2{c}_2^2+I_2+I_3\right]\ddot{q_1}\\ -2m_3{l}_2{c}_3\sin \left(q_3\right)\dot{q_2}\dot{q_3}+\left[2m_3{l}_2{c}_3\cos \left(q_3\right)+m_3\left(l_2^2+c_3^2\right)+m_2{c}_2^2+I_2+I_3\right]\ddot{q_2}\\ -\left[m_3{l}_2{c}_3\cos \left(q_3\right)+m_3{c}_3^2+I_3\right]\ddot{q_3}+m_3{l}_2{c}_3\sin \left(q_3\right){\dot{q_3}}^2\\ +l_1\left(m_2{c}_2+m_3{l}_2\right)\sin \left(q_2\right)\left[{\dot{q_1}}^2-\dot{q_1}\dot{q_2}\right]-\left(m_2{c}_2+m_3{l}_2\right)g\sin \left(q_1-q_2\right)\\ +m_3{l}_1{c}_3\sin \left(q_2-q_3\right)\left(\dot{q_1}-\dot{q_2}+\dot{q_3}\right)\dot{q_1}-m_3{c}_{3}g\sin \left(q_1-q_2+q_3\right)\end{array}$$E26

$$\begin{array}{cc}{\tau }_3& =m_3{l}_1{c}_3\mathit{Cos}\left(q_2-q_3\right)\ddot{q_1}-m_3{l}_1{c}_3\mathit{Sin}\left(q_2-q_3\right)\left({\dot{q}}_2-\dot{q_3}\right)\dot{q_1}+\left[m_3{c}_3^2+I_3\right]\ddot{q_3}\\ & -m_3{l}_2{c}_3\mathit{Sin}\left(q_3\right)\left({\dot{q}}_1-\dot{q_2}\right)\dot{q_3}+\left[m_3{l}_2{c}_3\mathit{Cos}\left(q_3\right)+m_3{c}_3^2+I_3\right]\left(\ddot{q_1}-\ddot{q_2}\right)\\ & -m_3{c}_3\left\{-\mathit{\text{gSin}}\left(q_1-q_2+q_3\right)+\left[l_1\mathit{Sin}\left(q_2-q_3\right)\dot{q_1}-l_2\mathit{Sin}\left(q_3\right)\left({\dot{q}}_1-\dot{q_2}\right)\left(\dot{q_1}-\dot{q_2}+\dot{q_3}\right)\right]\right\}\end{array}$$E27

To follow the track of position trajectories, the PD control is used for gravity compensation (Figure 17) [22], where

Figure 17.

Proportional derivative (PD) control with gravity compensation.

$$\tau =K_p\widehat{q}+K_v\dot{\widehat{q}}+G\left(q\right)$$E28

Figure 18 shows the SimMechanics tool is used to generate the trajectories of each of the fingers. Once the trajectories of each of the fingers have been generated, the workspace of each of them is generated.

Figure 18.

PD control simulation in SimMechanics.

3.4 Arduino connections to generate the interface with LabVIEW

Figure 19 shows the connection Arduino-computer and Arduino-servomotors as the fundamental relationships to get intercommunication and produce grasp movements like cylindrical, spherical, and tip. Figure also shows the Proteus package simulation.

Figure 19.

Programming Arduino Uno, LabView, and DC motors.

Figure 20 shows the workspace of the index finger of the prototype robotic prosthesis generated from the SimMechanics. The implementation of the robotic hand prosthesis required an intensive learning of the different hand postures. A LabView application to perform the corresponding grasps will be carried out.

Figure 20.

Index finger workspace of the robotic prosthesis prototype.

Figure 21 shows the flowchart of the developed application in LabView. Several tests with the anthropomorphic subactuated robotic hand were carried out in order to confirm if modeling task was correctly performed by fingers joints for specific hand gesture.

Figure 21.

Flowchart of the developed application in LabView.

Figure 22 shows the trajectories that were generated with the developing software and implemented in LabView in order to reproduce the movement of the underactuated anthropomorphic robotic prosthesis.

Figure 22.

Anthropomorphic underactuated finger trajectories.

PD controller parameters tuning methods were implemented, which were tuned manually by trial and error, and then an automatic parameter adjustment was used minimizing the integral of the tracking error by minimum squares. The PD controller makes fingertip tracking possible, thus allowing flexion and extension motions in the finger, in a way that the finger gets to a point without an important overshoot on axis y of 1.85 mm in a shorter time than 0.05 s. As shown in Figure 23, the values obtained are son k = 1, ζ = 0.01, C = 0.12, γ = 0.0015, the gains found were Kp = 0.004 and Kd = 0.035, respectively.

Figure 23.

Cartesian error in the PD control.

Finally, Figure 24 shows the position of the fingers according to the different gestures according to the control system designed.

Figure 24.

Position of the fingers in different gestures. (a) Open hand, (b) precision grip, and (c) closed hand.

4. Conclusions

The Myo armband is a wireless portable device developed by Thalmic Labs, capable of recording EMG through eight surface stainless steel electrodes, and has a sampling rate of 200 Hz. In addition, the Myo armband has an inertial unit of measure sensor (IMU) nine-axis, haptic feedback, and Bluetooth communication. These characteristics combined with a compact design, as it easily adjusts to the forearm, maintaining the distance between electrodes, lead to an acquisition system that is easier to use for prosthetic systems.

Mean absolute value (MAV), wavelength (WL), zero crossing (ZC), Wilson amplitude (WA), and variance were used to accomplish feature extraction by using a Myo armband device in different time series. The features extraction of the characteristics through the MAV allows the comparison between the values determined experimentally and predicted by the ANN, with a high level of effectiveness (94.94%).

Multilayer network topology was used to carry out the design of the artificial neural network (feedforward and back-propagation). The mean square error value of $1.2041x{10}^{-19}$ was obtained by using the Levenberg-Marquardt training algorithm; however, this value was lower than desired (10⁻³) in only 16 training epochs.

A prototype of hand prostheses was developed based on the anthropometry, kinematics and dynamics of the hand joints, and the main dimensions of the joints as well as the fingers’ trajectories to guarantee the different hand gestures. The direct and inverse kinematic model of the prototype robotic hand prosthesis based on the DH parameters was obtained. The simulations of the workspace and the PD control of the hand were performed by finding the Cartesian error set on the axis and 1.85 mm in a time less than 0.05 s. This allows the tracking of objectives attainable by the fingertips, developing movements of flexion and extension of the finger so that the finger reaches the point without an important overshoot.

A control system was developed based on the Arduino microcontroller architecture. The design system allows generating the joint trajectories of the hand prostheses prototype.