Perspective Chapter: Classification of Grasping Gestures for Robotic Hand Prostheses Using Deep Neural Networks

Abstract

This research compares classification accuracy obtained with the classical classification techniques and the presented convolutional neural network for the recognition of hand gestures used in robotic prostheses for transradial amputees using surface electromyography (sEMG) signals. The first two classifiers are the most used in the literature: support vector machines (SVM) and artificial neural networks (ANN). A new convolutional neural network (CNN) architecture based on the AtzoriNet network is proposed to assess performance according to amputation-related variables. The results show that convolutional neural networks with a very simple architecture can produce accurate results comparable to the average classical classification methods and The performance it is compared with other CNN proposed by other authors. The performance of the CNN is evaluated with different metrics, providing good results compared to those proposed by other authors in the literature.

1. Introduction

Upper limb amputations are injuries that substantially limit a person’s quality of life by drastically reducing the number of independent activities they perform in daily life (ADL). Current myoelectric prostheses are electronically controlled by the user’s voluntary muscle contractions. A general scheme of how these and other devices that use biosignals work is presented in Figure 1. In this sense, the prostheses for amputees with higher performance follow this common pattern of development. There is a wide variety of very sophisticated myoelectric prostheses commercially available that use sEMG signals [1, 2, 3, 4, 5].

Figure 1.

Most common configuration of human–machine interaction [6].

A relevant limitation in the development of pattern recognition methods for myoelectric control is that their tests are mainly performed offline. It is now established that high offline precision, does not necessarily translate into accurate functional control of a physical prosthesis. In this sense, several recent studies have shown the discrepancy between “on and offline” in performance metrics [7, 8, 9, 10, 11]. However, very few studies have published validation results of pattern recognition in terms, and even fewer in clinical settings, relating the variability of the signal and the performance of the classifiers with those parameters related to amputation (disability index, length remaining limb, amputation time, phantom limb sensation, etc.) [12, 13, 14, 15].

By other hand, the number of features extracted also depends on the number of EMG sensors and the feature extraction strategy for each sensor. Many investigations in have implemented alternatively, dimensionality reduction has been shown to be an effective feature projection (PC) method [16]. Among the most used methods are: principal component analysis (PCA) [17, 18, 19], linear-nonlinear PCA composite analysis, self-organizing feature maps [16] and supervised discretization together with PCA [20, 21].

Convolutional neural networks have been applied for myoelectric control with interest in inter-sessions/subjects and inter-session performance, in addition to many other applications in biomedical signal processing [22, 23, 24]. Some authors have commented on the advantages of these deep neural networks and their ability to assimilate the recognition of hand gestures corresponding to groups of sEMG signals. Although the results obtained come from a small number of investigations, their employment possibilities are promising [25, 26, 27].

However, most of the research has been carried out on healthy subjects. In recent decades, different authors [28, 29, 30, 31] have shown that the variation of the signal over time in amputated patients is even greater than in healthy subjects. The EMG signal is weaker due to the amputation of certain muscle groups and as the amputation time elapses, the muscles become more atrophied and weak. There are also few databases of amputees, a situation that constitutes a significant obstacle for these researches. and for the gestures recognition at the international level [29, 30]. Additionally, amputees’ performance was found to be proportional to residual limb size, indicating that an anthropomorphic model might be beneficial [28, 29, 30, 31]. The previous findings motivated the study on the variance of results between amputee patients and fit populations under disturbances of dynamic factors such as the length of the remaining limb, age, level of atrophy, among others. That is why the results obtained in amputated patients are far from those presented.

2. Materials and methods

2.1 Databases subjects

The review of these databases allows knowing the characteristics of the population involved and the signal capture protocols. The literature review showed that there are few databases with sEMG data collection in a significant number of patients, with subjects without known prior deficiencies and whose data are heterogeneous, so the most used is the NINAPRO database [32, 33, 34], which contains the electromyography recording using the system of 8 sEMG sensors Thalmic Labs - MYO. The data in this repository is free to use and is intended for use in developing hand gesture movement classifiers [22]. The NINAPRO database in its DB3 section establishes the parameters with which the sEMG data of 11 subjects with transradial amputation were recorded [35].

In the DB3 dataset, as explained above, the transradial amputee wears two Myo cuffs side by side. The superior MYO cuff is placed closest to the elbow with the first electrode at the radio-humeral joint, following the configuration of the NINAPRO electrode. The lower MYO cuff is placed just below the first one, closer to the amputation region (Table 1).

Patient	Hand	Laterality	Age	Remained Forearm (%)	Years since amputation	Amputation cause	DASH Score	Time wearing prostheses (years)
1	Right	right handed	32	50	13	Traumatic injury	1.67	13
2	Left	right handed	35	70	6	Traumatic injury	15.18	6
3	Right	right handed	50	30	5	Traumatic injury	22.50	8
4	Right	right handed	34	40	1	Traumatic injury	86.67	0
5	Left	left handed	67	90	1	Traumatic injury	11.67	0
6	Left	right handed	32	40	13	Traumatic injury	37.50	0
7	Right	right handed	35	0	7	Traumatic injury	31.67	0
8	Right	right handed	33	50	5	Traumatic injury	33.33	0
9	Right	right handed	44	90	14	Traumatic injury	3.33	0
10	Right	right handed	59	50	2	Traumatic injury	11.67	0
11	Right	right handed	45	90	5	Cancer	12.50	0

Table 1.

Clinical characteristics of subjects with amputation. NinaPro DB3.

In order to build our own database, the subjects invited to participate in this stage are amputated subjects, without neurological deficiencies. Invited subjects followed the population parameters used in the NINAPRO Database [36, 37]. Ten male and female amputees ranging in age from 24 to 65 years participated in the experiments. The procedures were performed in accordance with the Declaration of Helsinki and were approved by the ethics committee of the Universidad del Tolima (approval number: N – 20,160,021). All subjects participated voluntarily, providing their written informed consent before the experimental procedures. Any amputee who has experience in the use of hand prostheses will be included in the study, registering in advance their experience in the use of passive or myoelectric prostheses.

Aspects and demographic data to be recorded: For each subject, age, sex, education level, related to the amputation: dominant hand, amputated side, year of amputation, cause, type of prosthesis if used or has been used, and level of amputation.

Inclusion criteria: Adults in an age range of 20–65 years, no history of neurological and/or psychiatric diseases, voluntary participation in the study and acceptance of the medical staff. Only the transradial level of amputation will be considered, amputations above the elbow or beyond the wrist will not be admitted to the study. Any non-compliance with these parameters becomes criteria for exclusion from the study. Table 2 shows the characteristics of the amputee patients who participated in the trials.

Patient	Age (years)	Gender	Remained Forearm Length (Below elbow)	Amputated; hand	Years since amputation	DASH Score
P01	36	M	10 cm	Dominant hand	1	45
P02	51	M	Wrist Disarticulation	Dominant hand	30	19
P03	62	M	Wrist Disarticulation	Dominant hand	36	39.16
P04	26	M	10 cm	Dominant hand	12	20
P05	60	M	Wrist Disarticulation	Not dominant Hand	41	26.6
P06	55	M	Wrist Disarticulation	Dominant hand	5	16.66
P07	28	M	10 cm	Dominant hand	9	24.16
P08	48	F	Wrist Disarticulation	Dominant hand	22	20.83
P09	65	M	Wrist Disarticulation	Dominant hand	29	42.5
P10	35	M	10 cm	Dominant hand	2	47.5

Table 2.

Clinical characteristics of subjects with amputation.

2.2 Sensor EMG MYO armband

Data was recorded using the commercial MYO armband (MYO). MYO is a portable EMG sensor developed by Thalmic Lab and has eight dry electrode channels with a sampling rate of 200 Hz. It is a low cost, consumer grade device with a nine inertial measurement unit (IMU) [22], that connects wirelessly with the computer via Bluetooth. It is a non-invasive device, easier to use compared to conventional electrodes [38, 39]. Despite the low sampling frequency, its performance has been shown to be similar to that of full-band EMG recordings using conventional electrodes [22, 40], and the technology has been used in many studies [29, 35, 38] (Figure 2).

Figure 2.

Signals acquisition through the application developed in Matlab 2020b. Author. Six movements were identified in MYO sensor to achieve grip improvement: power grip (AP), palm inward (PI), palm outward (PO), open hand (MO), pincer grip (AT) and rest (RE) (Figure 3).

Figure 3.

Gestures to identify with the MYO device.

sEMG recording: Prior to carrying out the tests, the patients will be instructed on the experimental procedure and as a first step. The sensor operation will be calibrated for both limbs. To make the records in each gesture, the subjects will be seated comfortably in front of the computer with both limbs with their elbows flexed at 90 degrees and will be instructed to perform the gestures that are reflected on the monitor, in the case of amputated patients with the contralateral limb and with the amputated limb (Figures 4 and 5).

Figure 4.

(a) Amputee patient in front of the computer with a graphic signal to perform the movements. (b) MYO device arrangement.

Figure 5.

User interface that indicates the imaginary movement to be performed and includes completion and rest time.

The graphic interface will provide the patient with the times for performing the tests and the state of rest (Figure 5). Amputee recordings were performed in repeated sessions for 1 week.

The procedure carried out to capture the myoelectric signals is as follows: for each grip or gesture of the hand, 200 samples are taken during an interval of 30 seconds. Transitions are made between each of the six proposed gestures for spaces of 1 minute as recommended in [41]. sEMG signals were captured during several sessions and on different days of the week. The data of these myoelectric signals are stored in dataset for later offline processing.

2.3 Signal pre-processing

The segmentation and overlay methods used in this work improved the training efficiency by increasing the number of training samples based on recent work such as [17, 20, 42].

2.4 Feature extraction

Each captured sEMG signal is subdivided into 200 ms windows. The signal captured by the MYO was obtained at a frequency of 200hz [11, 21]. In order to be analyzed, it is divided every 200 ms, leaving a total of 40 data in each sub-window. Each sub-window has a 50% overlap with the immediately previous window, which allows increasing the number of samples and thus expanding the database obtained. The extraction described here is applied to each of the MYO channels. These data obtained for each of the channels [19], are concatenated horizontally, thus allowing a database to be obtained with 10 data features for each channel and a column with the information on the grasping gestures that are performed.

Different kinds of features extracted are used by different researchers, such as mean absolute value (MAV), root mean square (RMS), autoregression coefficients (AC), variance (VAR), standard deviation (SD), crossover by zero (CC), the length of the waveform (LO), Amplitude of Wilson (AW), slope of mean absolute value (PVAM). Features in the time domain were treated in [42]. These extracted features are used in SVM and in ANN and the raw signals for the CNN classifier.

2.5 Classifiers

Artificial neural networks Artificial neural networks (ANN) are a nonlinear classifier that simulates brain information processing through a series of weighted nodes, called neurons. Neurons are organized in layers and interconnect with each other to create a network. ANNs use a nonlinear function of a linear combination of the inputs, where the coefficients of the linear combination are adaptive parameters. The basic model of an ANN will be described as a series of functional transformations. First, M linear combinations of the input variables are constructed x₁, x₂, …, x_D in a way:

$$a_j=\sum _{i=1}^D{w}_{\mathit{ji}}^{\left(1\right)}+w_{j0}^{\left(1\right)}$$E1

where $j=1,\dots ,M,$ y the superscript (1) indicates that the corresponding parameters correspond to the first layer of the network. Parameters $w_{\mathit{ji}}^{\left(1\right)}$ are weights and the parameters $w_{j0}^{\left(1\right)}$ are polarization constants. The magnitude a_j is named activation, and each activation is transformed using a nonlinear and differentiable activation function [22].

In ANN classifier (Table 3), the chosen hyper parameters are highlighted and the variations that were proposed for each of them. Among the hyper parameters that were varied are the weight optimizer, the activation method, the maximum number of iterations and the type of learning.

Parameters	Options
Optimizer	SDG, ADAM
Regularization	0.0001–0.00001
Activation Function	Tanh, ReLu, Logístico
maximum numbers of iterations	10, 30, 50, 100
Hidden layers	3, 4, 5 y 6
Learning rate	Constant, Adaptive

Table 3.

Hyper-parameters. ANN.

2.6 Support vector machine

Support vector machines are a very powerful method used to solve classification problems, it is highly efficient on multidimensional data sets. It consists of defining a hyperplane or decision limit that separates the samples into two groups, where those above the decision limit are classified as positive and those below it, are classified as negative. The fundamental objective is to maximize the margin M or distance between the neighboring samples called support vectors and the separating hyperplane (Figure 6) [43].

Figure 6.

Optimal separation hyperplane, for linearly separable classes.

For the SVM classifier, the following kernels were used: linear, polynomial, Gaussian and sigmoid as shown in Table 4. In the linear and Gaussian kernels, the Gamma parameter was used with a value of 0.5. In the polynomial and sigmoid nuclei, the Degree parameter was used: between: 0.5 and 3, respectively. For all nuclei, the regularization constant C was used with values of 0.1, 1, 10 and 100 as recommended in the literature [44, 45].

Parameters	Options
Kernel	Lineal, Gaussian, Polynomial, Sigmoid
Gamma	0.5
Degree	0.5–3
C	0.1, 1, 10 y 100

Table 4.

SVM classifier parameters.

2.7 Convolutional neural networks

An CNN is a deep learning algorithm able to collecting an input matrix of size M X N and attributing weights and biases in parallel under the constraints of a predictive problem [46], resulting in specific features. A convolutional layer performs a dot product between two arrays, where one array is the set of learnable parameters and the other is known as the kernel, producing an activation map, as shown below:

$$G\left[m,n\right]=\left(f,h\right)\left[m,n\right]=\sum _j^M\sum _k^{N}h\left[j,k\right],f\left[m-j,n-k\right],$$E2

Where:

The input matrix is f and the kernel is denoted as h.

m is number of rows in the input matrix

n is the number of columns in the input matrix

j is the index for the offset in the rows of the input

k is the index for the offset in the columns of the input (Figure 7)

Figure 7.

Architecture of the convolutional neural network used.

Table 5 shows the parameters used in the CNN that were selected for the experiment carried out. Highlighted in the text are the hyper parameters: the batch size and the number of epochs that yielded the best results. The batch sizes were worked with values of 64, 128 and 256 respectively. The CNN architecture has four hidden convolutional layers and one output layer (Figure 8).

Figure 8.

Proposed model of CNN.

The first two hidden layers consist of 32 filters of size 8x1 and 3 × 3. The third consists of 64 filters of size 5 × 5. The fourth layer contains 64 filters of size 5 × 1, while the last one is a convolutional layer with six possible outputs with 1 × 1 filters, corresponding to the six gestures to classify. Followed by Nonlinear Rectification Units (ReLU) and Dropout layer with a probability of 0.15 to put to zero the output of a hidden unit. Also, a subsampling layer performs maximum pooling in a 3 × 3 window after removal of the second and third layers. Finally, the last convolutional layer is followed by a Softmax activation function (Table 5).

Parameters	Selected choice
Lost function	Categorical cross entropy
Optimizer	ADAM (lr = 0.001)
Batch size	64, 128, 256
Epoch	100, 400, 500, 1000
Validation rate	30%

Table 5.

Hyper parameters tuning.

2.8 Metrics

The confusion matrix is used to calculate many common classification metrics. The diagonal represents correct predictions and the other positions of the matrix indicate incorrect predictions. If the sample is positive and is classified as positive, i.e. correctly classified positive sample, and it is considered as a true positive (TP); if it is classified as negative, it is considered a false negative (FN). If the sample is negative and is classified as negative, it is considered true negative (TN); If the sample is negative and is classified as negative, it is considered true negative (TN); if it is classified as positive, it is counted as a false positive (FP), false alarm. The most common metrics are sensitivity (Se), specificity (Sp), which indicate the ability of the CNN to identify hand gestures. Accuracy (ACC) is used to assess overall detection performance and Precision (Pr) is used to measure model quality in posture classification tasks. Likewise, the F1 score (F1) is a measure of the precision of a test, it is the average precision and sensitivity of the classification. It has a maximum score of 1 (perfect accuracy and sensitivity) and a minimum of 0. Overall, it is a measure of the accuracy and robustness of your model. This metric gives information about the quantity that the model is able to identify. In this work, six commonly used evaluation metrics were used: accuracy, precision, sensitivity, specificity and F1 to evaluate the performance of CNN:

$$\mathit{\text{Accuracy}}\left(\mathit{ACC}\right)=\frac{\mathit{TP}+\mathit{TN}}{\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN}}$$E3

$$\mathit{\text{Precision}}\left(Pr\right)=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FP}}$$E4

$$\text{Sensitivity}\left(\mathit{Se}\right)=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FN}}$$E5

$$\text{Specificity}\left(\mathit{Sp}\right)=\frac{\mathit{TN}}{\mathit{TN}+\mathit{FP}}$$E6

$$F1\mathit{\text{score}}\left(F1\right)=\frac{2^{\ast }\mathit{TP}}{\mathit{TP}+\mathit{FP}+\mathit{FN}}$$E7

3. Results

The analysis in amputated patients is preceded by the great variability of the sEMG signal. Figure 9 shows this behavior of the sEMG signals for the power grab gesture in amputated subjects. It is observed that the data are very scattered and do not have the same symmetry, either the same mean, either standard deviation between each one of patients.

Figure 9.

Box and Whisker Plot of the sEMG for the power grab gesture in amputees.

3.1 ANN classifier

The results obtained with this classifier are shown in Table 6 in the specific patients. The own database has been used for the test data set with the following configuration: Optimizer: ADAM, activation function: ReLu, L2 regularization with a constant of 0.0001, a constant learning rate of 0.001, with three hidden layers and all layers with eight neurons.

ANN
Patients	Accuracy	Lost
P01	85.71	0.1000
P02	80.22	0.1700
P03	56.04	0.0280
P04	82.00	0.8600
P05	76.92	0,4700
P06	79.12	0.1886
P07	85.71	0.1200
P08	72.53	0.1920
P09	67.03	0.2111
P10	81.32	0.1239

Table 6.

Accuracy results with the ANN classifier on the test data set at 100 epochs.

On both data sets, that is, the test and trial data set, the ANN classifier showed an increase in the accuracy metric, especially when increasing the number of training epochs, passing on average from 76.66% to 100. periods with a minimum of 56.04% and a maximum of 85.77%, respectively. The average accuracy was consistent with the results shown by other authors for this classifier [35, 44, 45, 47].

In Table 6, superior results can also be seen on subjects P01, P02, P04, P05, P07 and P10, which is outstanding, since their accuracy was above 80% with this classifier with a standard deviation of 9.23. This low standard deviation indicates that most of the accuracies obtained tend to be clustered close to their mean.

3.2 SVM classifier

As mentioned above, different kernels were used: linear, polynomial, Gaussian and sigmoid. Likewise, we worked on the regularization constant C with values of 0.1, 1, 10 and 100. On both data sets with the SVM classifier using the RBF and Sigmoid kernels, the best results were obtained when evaluating the accuracy metric, obtaining results up to 80%. in this metric. Table 7 shows the results obtained with all the kernels.

		C = 0.1				C = 1				C = 10				C = 100
Patients	Kernel	Accuracy	F1 score	Sensitivity	Precision	Accuracy	F1 score	Sensitivity	Precision	Accuracy	F1 score	Sensitivity	Precision	Accuracy	F1 score	Sensitivity	Precision
P01	Polynomial	47.5	41.25	51.39	48.23	47.5	51.62	58.06	51.64	25.00	23.94	28.94	29.12	25.00	23.94	28.94	69.12
	Linear	62.5	59.82	64.63	60.89	62.5	61.17	65.32	62.74	25.00	17.29	30.00	25.00	25.00	20.21	29.86	27.50
	RBF	12.5	2.78	16.67	2.08	22.5	19.36	27.08	52.38	65.00	62.66	67.55	66.44	65.00	59.82	64.63	60.89
	Sigmoid	12.5	2.78	16.67	12.5	12.5	18.21	33.33	27.5	60.00	59.07	63.24	61.55	60.00	62.39	65.09	64.60
P02	Polynomial	20.00	9.20	25	7.78	20	30.97	32.16	30.86	20.00	15.87	23.21	43.98	20.00	15.87	23.21	43.98
	Linear	52.5	52.78	52.46	59.76	52.5	49.77	51.03	57.13	2.50	1.74	3.33	2.50	2.50	2.15	6.11	5.00
	RBF	12.5	2.78	16.67	2.08	10	2.7	13.33	2.08	42.50	35.85	47.16	38.57	42.50	36.37	47.16	47.62
	Sigmoid	12.5	2.78	16.67	12.5	12.5	2.78	16.67	12.5	35.00	41.42	43.27	42.26	35.00	30.44	38.27	31.15
P03	Polynomial	32.50	24.25	36.30	33.65	32.5	37.23	42.22	34.72	20.00	13.21	25.00	19.12	20.00	13.21	25.00	19.12
	Linear	55	49.47	55.6	46.89	55	49.47	55.6	46.89	5.00	3.13	6.67	5.00	5.00	10.3	10.2	11.5
	RBF	12.50	2.78	16.67	2.08	20	13.21	25	19.12	55.00	50.33	55.60	48.28	55.00	47.09	49.49	44.44
	Sigmoid	12.50	2.78	16.67	12.5	12.5	2.56	13.33	10	50.00	44.78	49.63	45.69	50.00	46.61	52.27	43.22
P4	Polynomial	47.5	42.18	47.22	39.58	47.5	64.15	65.65	65.93	20.00	15.99	24.31	52.25	20.00	15.99	24.31	52.25
	Linear	65	64.36	64.07	69.83	65	64.36	64.07	68.83	2.50	3.91	2.78	2.50	2.50	2.80	2.70	2.30
	RBF	12.5	2.79	16.67	2.08	17.5	11.48	22.22	35.53	65.00	64.15	64.07	68.75	67.50	67.24	68.10	70.24
	Sigmoid	12.5	2.78	16.67	12.5	12.5	2.78	16.67	12.5	67.50	66.82	67.41	71.53	65.00	68.25	66.85	70.83
P05	Polynomial	17.5	7.06	22.22	5.65	17.5	27.37	38.33	28.84	20.00	14.75	25.00	33.16	20.00	14.75	25.00	33.16
	Linear	52.5	50.76	56.62	53.89	52.5	53.28	59.95	56.83	12.50	3.29	16.67	12.50	12.50	3.97	5.56	5.00
	RBF	12.5	2.78	16.67	2.08	20	14.66	25	33.08	55.00	53.74	59.95	57.45	55.00	50.80	57.87	54.99
	Sigmoid	12.5	2.78	16.67	12.5	12.5	2.78	16.67	12.5	40.00	37.70	44.35	40.08	40.00	51.51	56.62	55.40
P06	Polynomial	32.5	25.44	36.3	36.31	32.5	41.29	50.37	40.1	25.00	19.41	30.56	35.71	25.00	19.41	30.56	35.71
	Linear	65	65.85	65.32	74.63	65	65.85	65.32	74.63	2.50	4.17	3.33	2.50	2.50	3.10	4.20	3.70
	RBF	12.5	2.78	16.67	2.08	17.5	10.41	22.22	18.86	57.50	51.28	60.56	49.81	57.50	57.21	61.16	62.82
	Sigmoid	12.5	2.78	16.67	12.5	12.5	2.78	16.67	12.5	65.00	65.85	65.32	74.63	65.00	68.26	68.10	75.32
P07	Polynomial	75	70.18	71.51	63.07	75	79.18	76.51	79.54	27.50	31.06	29.64	54.44	27.50	31.06	29.64	54.44
	Linear	80	79.76	77.76	79.4	80	79.76	77.76	79.4	15.00	8.94	15.48	15.00	15.00	9.37	15.48	15.00
	RBF	12.5	2.78	16.67	2.08	17.5	14.5	20.87	35.19	80.00	65.77	64.56	66.05	80.00	66.04	63.87	69.44
	Sigmoid	12.5	2.78	16.67	12.5	12.5	19.36	30.95	27.5	62.50	63.13	60.40	68.06	62.50	63.35	60.54	63.33
P08	Polynomial	45	35.28	51.67	34.47	45	44.13	55.37	52.15	25.00	20.71	30.56	35.78	25.00	20.71	30.56	35.78
	Linear	45	46.43	46.71	50.89	45	49.32	49.49	55.51	17.50	10.06	21.67	17.50	17.50	9.47	18.33	15.00
	RBF	12.50	2.78	16.67	2.08	17.50	10.48	22.22	18.92	57.50	52.28	59.44	50.67	57.50	49.79	58.15	48.96
	Sigmoid	12.50	2.78	16.67	12.50	12.50	12.98	25	20	47.50	45.76	49.49	49.74	47.50	40.89	47.41	39.98
P9	Polynomial	40	38.25	43.33	48.57	40	41.75	42.78	44.7	17.50	11.19	22.22	25.00	17.50	11.19	22.22	25.00
	Linear	45	42.38	45	40.37	45	44	45	43.94	10.01	9.27	7.35	8.12	12.0	9.27	8.48	12.86
	RBF	12.5	2.78	16.67	2.08	15	6.81	19.44	8.24	45.00	42.72	45.00	40.42	45.00	47.82	53.33	46.52
	Sigmoid	12.5	2.78	16.67	12.50	12.50	3.01	10.00	7.50	47.50	43.71	47.22	41.94	47.50	38.56	38.89	36.55
P10	Polynomial	35	26.76	40.56	30.11	35	77.21	77.59	78.02	22.50	19.57	26.16	52.38	22.50	19.57	26.16	52.38
	Linear	72.5	72.9	71.25	72.02	72.5	73.07	71.25	72.32	12.50	7.86	14.44	12.50	12.50	9.79	14.44	12.50
	RBF	12.5	2.78	16.67	2.08	20	15.05	24.07	35.65	75.00	75.21	74.03	74.40	75.00	70.70	69.40	70.06
	Sigmoid	12.5	2.78	16.67	12.5	12.5	12.44	21.67	17.5	75.00	75.05	73.10	73.21	75.00	69.68	69.40	70.00

Table 7.

Results of the metrics with the SVM classifier in the specific patients.

3.3 CNN Classifier

Table 8 shows the comparative results of the CNN classifier evaluated in different patients using regularization techniques such as: early stopping, dropout and batch normalization. Which is a technique that tries to apply certain rules to know when it is time to stop training, so that there is no overfitting to the input data, nor under fitting. The average time required to train each convolutional neural network was 1 hour and 25 minutes. The average time needed to test the network was 15.2 s using a GPU NVIDIA Titan-V, 12 GB RAM HBM2 y 640 Tensor Cores.

CNN
	EPOCHS
	100	200	300	400	500	600	700	800	900	1000
Patients	Accuracy	Accuracy	Accuracy	Accuracy	Accuracy	Accuracy	Accuracy	Accuracy	Accuracy	Accuracy
P01	61.53	64.87	67.38	69.26	71.61	73.86	76.08	78.11	80.26	80.51
P02	54.15	58.23	60.16	60.80	63.65	67.76	69.57	72.98	76.13	76.52
P03	55.32	60.41	60.94	62.84	64.54	68.71	70.97	73.64	75.95	76.34
P04	67.28	70.46	72.20	72.91	74.09	76.24	79.39	81.54	83.12	83.69
P05	60.07	62.99	65.62	67.08	68.38	71.64	73.91	76.28	77.95	78.58
P06	55.53	59.51	62.64	62.51	67.37	69.61	71.89	74.32	76.54	78.85
P07	70.44	73.25	74.61	76.54	77.12	78.88	80.65	82.51	84.62	86.63
P08	56.51	60.80	63.30	64.81	65.87	67.97	71.11	74.25	77.39	80.53
P09	56.72	63.53	63.20	66.44	69.83	71.37	73.86	76.38	78.54	81.20
P10	62.04	67.71	70.01	71.39	73.27	74.85	76.58	78.31	80.14	81.77

Table 8.

Comparative results of the CNN classifier of the different patients at different epochs.

Table 9 shows a summary of the different classifiers by means of the accuracy metric in the different patients. In patients P01 and P02, the best classifier is ANN. Although the accuracy shown by the CNN is acceptable. These patients have a DASH index of 45 and 19, respectively. They also present different levels of amputation: one at 10 cm from the elbow and the other at the level of the wrist. Likewise, they present amputation times of 1 and 30 years, correspondingly. These clinical factors affect the performance of the classifiers.

Patients	SVM; (Accuracy %)	ANN; (Accuracy %)	CNN; (Accuracy %)
P01	65.00	85.71	80.51
P02	52.20	80.22	76.52
P03	55.00	56.04	76.34
P04	67.50	82.00	83.69
P05	55.00	76.92	78.58
P06	65.00	79.12	78.85
P07	80.00	90.11	86.63
P08	57.50	84.62	80.53
P09	47.50	67.03	81.20
P10	75.00	81.32	81.77
Average	61.97	78.30	80.46

Table 9.

Accuracy metric comparison between all classifiers. The values in bold represent the highest accuracy values recorded by each patient in the classifiers.

4. Discussion

From the results obtained, the following points are analyzed. Figure 10 presents the confusion matrix of each of the classifiers (SVM, ANN and CNN) corresponding to the patient (P03 0 P09), It is observed that both the SVM and ANN show a low number of hits in the different gestures in this patient whose cause of amputation is due to congenital factors that further affect the variability of the signal. Even though only one case is analyzed in this work, this type of behavior has been reported by other research works in patients of this type.

Figure 10.

Confusion matrices of the different classifiers for the patient P09 (a) SVM (b) ANN (c) CNN using the metric of the accuracy of the data corresponding to the training (session two).

Once again, the performance of the SVM and the ANN is significantly affected. The results of the present work show a significant accuracy rate for the classification of various classes of amputated subjects in comparison with other studies carried out (Table 10).

Methods	Accuracy	Authors
AtzoritNet, CNN classifier, healthy subjects and amputees	70.48±1.52%	Tsinganos et al., 2018 [39]
Time domain, regressive models, Bayesian Network, ANN, CNN, AD, SVM, healthy subjects	80.4±2.6%	Ramirez-Martinez, et al., 2019 [40]
CNN, multiclass classifier, input TWC, amputees subjects	68.98±29.33%	Cote-Allard et al., 2019 [22]
WeiNet, CNN classifier, NinaPro dataset, healthy subjects	99.1%	Wei et al., 2020 [27]
AD Classifier, ACP, MD, Multiclass Classifier, amputees patients	77.3±17.5%	Rabin Neta et al., 2020 [38]
SVM classifier, LDA and TWD, healthy subjects	94.73%	Lin Chen et al., 2020 [48]
CNN classifier, TWD, healthy subjects	98.82%	Tsinganos et al., 2020 [49]
Multiclass classifiers, SVM, ANN, CNC, amputees patients	80.46%	This research

Table 10.

Studies conducted using CNN as an EMG-based hand prosthesis movement classifier in healthy subjects and amputated subjects.

5. Conclusions

Over the past decade, deep learning and convolutional neural networks have revolutionized several fields of machine learning, including speech recognition and computer vision. Therefore, its use is promising for obtaining better classification indexes of the sEMG signals if the great variation of these is considered in accordance with clinical variables of the amputation, all of which would contribute to closing the gap between the prosthesis market (which requires fast and robust control methods) and the results of recent scientific research in disability support technologies.

The protocol for obtaining sEMG measurements in amputated patients was applied, as well as the extraction and classification of the signal, all of which is consistent with the proposal for the integrated design of the prosthesis. A database of 10 amputated patients according to the 6 defined hand gestures was constructed. The data is publicly available in the repository of the Huila-Corhuila University Corporation (CORHUILA).

The classification accuracy obtained with CNN using the proposed architecture is 80.46%, but the most significant thing is its ability to obtain a higher performance in the classification between subjects in relation to parameters such as length of the remaining limb, years of amputation or disability index, compared with the results obtained by conventional classifiers such as the support vector machine and artificial neural networks.