Open access peer-reviewed chapter

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games and Toys

Written By

Natallia Ivaniuk, Zahar Ponimash, Vladimir Karimov and Valentsin Shepanskiy

Submitted: 31 October 2017 Reviewed: 02 February 2018 Published: 24 October 2018

DOI: 10.5772/intechopen.74794

From the Edited Volume

Biofeedback

Edited by Mark Schwartz

Chapter metrics overview

1,298 Chapter Downloads

View Full Metrics

Abstract

The LLC Bionic Natali company is a startup and has been engaging in creation of bionic artificial limbs of hands for more than 2 years, and the LLC Bi-oN EMG was created one year ago on base of LLC Bionic Natali. From the first steps, projects had been directed on the solution of a problem of development of the domestic bionic functional artificial limb of the hand based on neural network and others algorithms. In projects, it had been created the functional system of management, system of tactile feedback which has increased controllability of a functional artificial limb is already realized and integrated, and also the functional bionic artificial limb of the hand. Based on this work it had been done the general representations and practical application of machine training, neural network and others algorithms. The technology of recognition of gestures of electromyographic activity based on neural network or an analog of network is the cornerstone. The bracelet is put on a hand (in case of disabled people, a stump), further noninvasive electrodes remove potential difference of neuromuscular activity; by means of an electric circuit there is data handling and their transmission to the processor where by means of a neural network there is a recognition of a gripper, further data are transferred for control of a bionic hand. This technology has also found so far mostly theoretical management, but undergoes testing practical, for control of the knee module of a bionic artificial limb of the lower limb. The technology of bracelet became the product of LLC Bi-oN EMG for virtual reality, games and rehabilitation. On the basis of this researches the important conclusion has been drawn: after carrying out operations within rehabilitation and the subsequent use to establish to the recommendation immediately bionic artificial limbs as eventually without trainings and also because of psychology of people “experience of management” the hand is lost, and without trainings of the muscle atrophy. But restoration is possible, alas, it will demand much more time, than in cases of immediate prosthetics by bionic/myoelectric artificial limbs. The current researches have also shown requirement of rehabilitation of disabled people before use of bionic artificial limbs of hands, use of an electromyographic bracelet of LLC Bi-oN EMG is offered as option.

Keywords

  • the bionic artificial limb
  • neuronal net
  • electromyographic signals
  • system of control
  • EMG
  • Bionic Natali
  • Bi-oN EMG
  • recognition the electromyographic signals
  • artificial intelligence
  • machine learning
  • feedback of the signals
  • Bi-oN

1. Introduction

Practical developments of LLC “Bionic Natali” and LLC “Bi-oN EMG” companies’ researches and creation of technology of management of limbs on recognition of signal EMG on the basis of neural network were presented in the current article, and also the use of the given technology at rehabilitation of people with loss of an extremity, after an acute stroke and even in the game sphere for children and young people, such as virtual reality, radio control by machine, quadcopters, etc. was presented. In the article the practical results of a research of feedback on the basis of EMG of signals at disabled people and people without loss of limbs are given.

Within the project of the LLC “Bionic Natali” company, the hi-tech bionic artificial limb which surpasses all domestic functional electromechanical and cosmetic artificial limbs in the Russian Federation has been developed and also will surpass foreign bionic artificial limbs in functionality by industrial production. Within the project the functional control system, the system of tactile feedback which has increased controllability of a functional artificial limb is realized and integrated and also the functional bionic artificial limb of the hand is developed.

Now, in most cases, in the absence of a hand, the cosmetic artificial limbs bearing in themselves only the cosmetic purposes are used, being, in fact, a hand model. Electromechanical artificial limbs which allow to replace partially functionality of a full-fledged hand are widespread poorly at present, have limited functionality, and also are available to still a limited circle of users. Especially, this situation is characteristic of Russia that is explained by the influence of several factors: the certain structure of schemes of social insurance causing an order of financing of prosthetics for the state account of low consumer ability of the people who have lost a hand and, of course, absence in the market of decisions in various price segments with the significant level of functionality.

In a prosthetic repair, active transition from static prostheses to robotic bionic is now observed; in the light of this, it is possible to allocate tendencies in creation of cheap and low-quality prostheses from the 3D press and on cheap accessories and also development of more expensive prostheses, but the second tendency is bound to larger problems available for a technological base, namely, small capacious accumulators, small drives, plateaus of processing, processors, and so on. But, despite all difficulties, the last conform to requirements which are exposed by the state at a prosthetic repair, namely, the long-term use of prostheses, 2–3 years.

As for the most electromyographic bracelet, a control system on the basis of EMG—that decision on his allocation in the separate project that has been made after the prototype has been created and the architecture of work of a bracelet became clear, namely, that the bracelet can work independently and he can be tied to any electronic product with Bluetooth for management of this product. By the results of the works done, the LLC “Bi-oN EMG” company has been open, and its application is found in the sphere of rehabilitation and game devices. It is important to note that the game sphere for children has been chosen in connection with market width and also the need to reduce the price of product cost.

The sensing technology of gestures of electromyographic activity on the basis of a neural network is the cornerstone. The bracelet is put on a hand (in case of disabled people, a stump); further noninvasive electrodes remove potential difference of neuromuscular activity, by means of an electric circuit there is data handling and their transmission to the processor where by means of a neural network there is a recognition of a gripper, further data are transferred for control of any electronic device with the set special software (the executive mechanism), back coupling is carried out from the executive mechanism to a bracelet by means of signal transmission of back coupling and on a bracelet the vibromotor is launched in action.

Advertisement

2. Implementation of the project based on the neural network for the recognition of the gripper for bionic artificial limbs and for virtual reality and game devices

As it was already noted earlier, within the project the hi-tech bionic artificial limb which surpasses all internal functional electromechanical and cosmetic artificial limbs is developed and also will surpass foreign bionic artificial limbs in functionality by industrial production. And, the LLC “Bionic Natali” company is faced by a task to make him available to most disabled people due to considerable reduction of cost in comparison with foreign bionic artificial limbs, about by five to six times. Also, it should be noted about a know-how the project, creation of innovative system of reading on bigger quantity of electrodes, than in the current foreign bionic artificial limbs which is also radio and it is constructed on the principles of neural network and other algorithms.

The technology of recognition of gestures of electromyographic activity based on neural network or an analog of network is the cornerstone. The bracelet is put on a hand (in case of disabled people, a stump), further noninvasive electrodes remove potential difference of neuromuscular activity; by means of an electric circuit, there is data handling and their transmission to the processor where by means of a neural network there is a recognition of a gripper or movement of a knee, further data are transferred for control of a bionic hand or leg.

Information on how usually artificial limbs are working can be found in article [1].

Within the carried-out scientific research works of more than 1 year, the basic divergences with data of muscular activity were found in disabled people and given to muscular activity at people without amputation. Distinctive feature is the range of frequencies at which the signal registers, and importance of maintenance of a tone of muscular activity at disabled people, otherwise there are artifacts of the movement, which at people without amputations are not registered. For this task now with Ilizarov’s center, the system of rehabilitation is studied, and also the separate product “an electromyographic bracelet” which is taken out in the separate company LLC “Bi-oN EMG” has been created. In Figure 1, the picture on a control system of any executive mechanism of the recognition of EMG based on the neural network is given below.

Figure 1.

The scheme of control of any executive mechanism of the recognition of EMG based on the neural network.

General explanation of EMG can be found in the book [2].

2.1. Technology of bionic artificial limbs

Technical tasks, realized within works and development, were the following:

  • Reliable functional intellectual bionic artificial limb of an extremity with return of tactile communication by means of neuromuscular signals

  • Way and control system of the auxiliary device, such as extremity artificial limb

The main problems, which have been put before LLC “Bionic Natali” company at implementation of the project, are the following:

  1. Increase in accuracy of positioning of engines

  2. Increase in accuracy of the obtained data at recognition of gripper

  3. Increase in accuracy and probability of recognition of gripper

  4. Making decision on the capture of a subject and increase in controllability of gripper

Emphasis on a project know-how, creation of innovative system of reading on bigger quantity of electrodes than in the current foreign bionic artificial limbs which also is wireless and constructed on the principles of the neural network, is worth placing. The matter is that in all foreign artificial limbs at the moment the management is based on the basis of management from two sensors and threshold value of a signal, grippers are programmed by means of phone or the button on an artificial limb. There is selection of an algorithm of data processing of electromyographic activity of muscles for the best recognition of the gripper and experimental confirmation of developments.

Materials are the knowledge of developers and experience of the advising experts (the mentor on algorithms—professor of Skoltech Victor Lempitsky; prosthetists Levon Kirakozov, S. Golovin, V. Golovin, and A. Dozhdev; experts of the Federal State Budgetary Institution SPb NCEPR of G.A. Albrecht of Ministry of Labour and Social Protection of the Russian Federation; experts of the center of Ilizarov; etc.) and also completely experimental data that had been used.

The method which was used in the course of selection of an algorithm is the experimental “trial-and-error method.”

It is important to mark that at the first stages of the project it was planned to select algorithms, processing collected these electromyograms of muscular activity of disabled people from open sources, but as it was clarified later, in open access information was practically not, and what managed to be found, was very far from reality and is poorly described regarding process of removal of data that does not exclude existence of an error in data. As a result, the decision to create an own experimental database that was successfully realized was made.

As for the solution of a task of optimization of gripper, decrease in cognitive load of the person and increase in effective management of an artificial limb is reached due to use of a hybrid control system which combines ways of decoding electro-neuromyosignals with elements of autonomous robotic manipulations on capture of a subject at achievement of threshold value of distance to him by means of the decision-making which is not demanding participation of the user on the beginning or the termination of implementation of the gripper by means of information from additional sensors (temperature and distance), and about force of compression of a subject by means of information from sensors.

Also, in a picture, the example of an arrangement of sensors on the hand artificial limb has been shown below (Figure 2).

Figure 2.

Example of the arrangement of sensors on the hand artificial limb.

Example of the scheme of realization of the data acquisition in the form of the myoelectric system (poses 1—system of reading, poses 2—sensor of registration of biopotential, poses 3—elastic cuff) (Figure 3).

Figure 3.

The scheme of realization of the data acquisition in the form of the myoelectric system.

The use of wireless data transmission between the artificial limb and the system of control provides usability of system and interchangeability of components, besides a possibility of their use independently of each other.

It is important to notice that in the course of the analysis of putting off data it was revealed that a bracelet for management of a prosthesis of the hand and a forearm in a case with the use of neural network, it is possible to wear not only on a forearm and to take off data from a forearm, but also for management of the hand to take off data from a brachium that considerably dilated opportunities at a prosthetic repair by bionic prostheses of arms (of course, there is a restriction on number of gripper and features of management, at some disabled people different grippers are defined variously in connection with features of a stump).

2.2. Technology of the work of the virtual reality and game devices with the use of the electromyographic bracelet

The control of the virtual reality happens as follows: the myoelectric device of reading in the form of a bracelet (or a similar design the device) is put on a forearm or an arm, or other parts of an extremity, as the example, legs, occurs calibration and control of a bracelet, it can be made on the computer, the phone (Figure 4), as also without the computer or the phone.

Figure 4.

The virtual reality and bracelet.

Points of virtual reality consist at least of different types of lenses, the display, the details of the case, the computing system, the accelerometer, the gyroscope, and the sensor of the wireless communication (Bluetooth) (Figure 5).

Figure 5.

Example of the scheme of the device of points of the virtual reality.

Communication between the bracelet and points of virtual reality is been organized by means of any wireless communication, in our case are used by Bluetooth at what by Bluetooth of a bracelet it is ready as conducted, and Bluetooth of points of virtual reality as the master. The myoelectric device for reading, in the form of the bracelet (or the similar design of the device), carries out registration and filtration of the electromyogram (EMG), defines position of the hand, depending on it sends the corresponding command to points of virtual reality, or obtains information on feedback. It is important that the myoelectric device of reading besides the gripper given according to numbers sends to points information from the gyroscope and the accelerometer that allows to define position of the hand in space better. The arrangement of the myoelectric device for reading not only on hands but also on legs, the back, and the neck is possible allowing to define better an arrangement of other parts of the body in space. In addition, points of virtual reality are capable to transfer information of feedback and to start the vibromotor on the myoelectric device for reading that allows to create feedback; also, additional arrangement and other sensors of feedback and sensors is possible.

Management of radio-controlled model of the machine happens as follows: the myoelectric device of reading in the form of a bracelet (or a similar design the device) is put on the forearm or the arm, there is the calibration and setup of the device of the bracelet, it can be made on the computer, the phone (Figure 6), as also without the computer or the phone.

Figure 6.

Example of the scheme of setup of the machine or other executive mechanisms.

The radio-controlled model consists at least of a chip, sensor of the wireless communication (Bluetooth), and engines of motors (Figure 7).

Figure 7.

Scheme of the device of the executive mechanism of the car.

Communication between a bracelet and radio-controlled model the machine is organized by means of any wireless communication, in our case are used by Bluetooth at what by Bluetooth of a bracelet it is ready as the master, and machine model Bluetooth as conducted. After start of both devices, with connection without intermediaries, the device of a wireless communication of a bracelet is connected to the communication device of the executive mechanism; in our case communication requires knowledge by the master of the name and password of the communication device of the executive mechanism. In this case devices are connected through the intermediary device (the computer, phone), and then the intermediary device participates in installation of communication between devices. The myoelectric device of reading carries out registration and filtration of the electromyogram (EMG), defines position of a hand and, depending on him, sends to model the machine the corresponding team. List of tasks: F—the machine begins advance will not receive the next task, B—model the machine begins the movement back will not receive the next task yet, L—model the machine begins the movement on the left so far this task comes, after its termination continues action which executed before receiving commands (advance, back, inaction), R—model the machine begins the movement to the right so far this task comes, after its termination continues action which executed before receiving commands (advance, back, inaction), S—the model stop the machine.

Model of the machine receives the task, and the chip processes it depending on the task and carries out manipulations over motors, to begin rotation, to change the direction of rotation, and to stop. After implementation of the current command, the device is ready to perform the following. Also, from model, the machine can come to the bracelet information of feedback and start the vibromotor on the bracelet or other types of sensors.

Also it is possible a realization chance of the systems of rehabilitation with points of virtual reality, and without them. Namely, when there is a removal of data, for example, on gripper from a forearm or an arm of a hand, and transfer on the computer or phone for performance of certain gripper or three-dimensional motions, movements, on the computer or phone is started the program which obtains information and signals about successful or unsuccessful performance of a task, giving of a signal of feedback on a bracelet is also possible.

2.3. Algorithms of the recognition of EMG: approaches on the work with EMG

In this work we recognize the electromyogram removed from the skin (i.e., it is noninvasive), the recognition purpose—to understand what the gripper has been made by the hand.

Working with data, which has been obtained from muscular activity, it has been shown that for optimum work of algorithms on capture recognition, it is necessary to execute the following main stages of processing:

  • Filtration

  • Preprocessing

  • Then, there is already a submission of data on neural network or an algorithm similar to it

  • Post-data processing and additional training of neural or similar network

Assessment of the efficiency of algorithms is carried out in two main parameters—the accuracy of work of the algorithm and volume of calculations—as the most important for use in real time. Signs for an algorithm of classification have been distinguished from the initial signal; thus, the compactness hypothesis was carried out, where each gripper is the class. To achieve performance of this hypothesis, then practically any qualifier, including the simplest, such as can be suitable for classification: “classification by a minimum of Euclidean distance.” Each certain canal is not of a particular interest since it is not possible to differentiate gesture on one channel. Whether from the channel it was possible to select only information on that reduction of the muscle. Further on what muscles have been reduced and what are not present to distinguish concrete gripper. The algorithm of preprocessing allows, quite precisely defining whether muscles, on this channel (Figure 8) have been reduced.

Figure 8.

Recognition of the signal of EMG.

When algorithms were tested, two main variants were developed; the first one used simple preprocessing of the signal. Moreover, the second neural network hash function as additional preprocessing.

Without the neural network hash function, the results were shown in Figure 9.

Figure 9.

The process without the neural network hash function.

Results of tests without the neural network hash function are shown in Figure 10.

Figure 10.

Results of tests without the neural network hash function.

All gestures were recognized with the probability greater than 70%, of these, two gestures were recognized with a probability of less than 80%, six with recognized with probability <90%, but >80%, eight gestures >90% and two with the probability of a business to 100%.

This method has been tested for real-time amputations. Its accuracy is averaged to about 90%.

Figure 11 shows the algorithm for recognizing gestures with neural network hash function.

Figure 11.

The process with the neural network hash function.

In the first stage, a feature space has been generated from the signal. After that, a hash function has been used based on neural networks. After, the code classifier with the hash function recognizes gestures.

However, in this test, the signals have been recorded beforehand. Real-time mode had been simulated.

This algorithm has been tested on various amputees; the algorithm gives a high accuracy of determining the gesture. Below is the test of the algorithm on a person with shoulder amputation; seven gestures were recognized.

The picture shows the results of the test on seven gestures (Figure 12a), and the picture shows the results of the tests on 11 gestures (Figure 12b).

Figure 12.

(a) Results of the test on seven gestures. (b) Results of the test on 11 gestures.

Neural network hash function consists not from one algorithm of neural net; it is complex of transformations, exactly five levels of different algorithms. On the entrance on the example, there are eight EMG signals, which had been modified with different neural networks and number of neurons. The details cannot be open because it is trade secret of the LLC “Bionic Natali” company and on this base had been created unique product of recognition of EMG signals of the hand.

According to the previous researches described in a source of information [3] for a condition of implementation of requirements of work in real time, the time of recognition of a signal has to occupy no more than 250 ms. For comfortable work of the user productivity or recognition accuracy (percentage of right cases of classification to all considered cases), it has not been lower than 95%, as shown in a source [4]. Most details about recognition can be found in a source of information [5].

Methods of recognition of EMG:

  • For recognition at the first stage from a signal, various signs then the vector of these signs moves on the system of recognition are been taken.

  • Most often as signs, counting bending around from different zones at the moment of time is used. As it is been made in work [6]. In addition, the system of recognition represents a set of rules. But approach at which the value bending around at the moment is chosen has one essential shortcoming. Mix-ups of gripper are possible in an area (Figure 13).

Figure 13.

The bending-around signal EMG.

It is possible to fight against it in several ways: to pass the function which increases (falls down) quicker, than bending around, to use search of the maximum value bending around on an interval, to complicate the qualifier (e.g., to use recurrent or convolutional neural networks) that he considered not only the current values, but also some history or to use counters of operations for each class.

In our work, it has been used as signs of window dispersion. Properties of window dispersion of EMG are considered in work [7]. In addition, contrasting and scaling of value of dispersion have been applied. Then, at reduction of a muscle above which there is a sensor, the value is established in 1, and at relaxation it is in 0 (Figure 14).

Figure 14.

The algorithm of the previous processing.

After the previous processing, the signal arrives on the autoencoder to reduce entrance space of signs. Reduction of dimension of space of signs positively influences quality of the classification, in case of the use of metric qualifiers, because of a so-called “damnation of dimension” [8]. Later, there is a metric qualifier. Further, there is the counter of operations with the comparison block. After distinguished gripper goes to the operated device. The scheme of an algorithm has been submitted in Figure 15.

Figure 15.

Flowchart of the algorithm.

More detailed description is provided in the article [9, 10].

Also, alternatives for algorithms of the recognition exist; details can be found in articles [11, 12, 13, 14, 15, 16].

2.4. Realization of developments with projects of LLC “Bionic Natali” and LLC “Bi-oN EMG”

Despite the fact that huge amount of works has been made, there is still a big area for activity concerning the choice and improvement of an algorithm on recognition of gripper, improvement of mechatronics, and the skin for an artificial hand. Similar work can be compared to art as we will compare the choice of an algorithm, the skin, and the solution of other technical problems with creativity. The current results of the LLC “Bioniс Natali” company in this sphere—it is recognition with probability of 98% on 14 grippers on 8 sensors with amplifiers from a forearm. The concerning removal of data and recognition of capture in disabled people then in practice were difficulties at movements of muscles and pain at a spasm in long muscular tension. In this regard, a decision together with many medical centers to develop a method of restoration of muscles and to create the tool for their training has been made—the electromyographic bracelet from the LLC “Bi-oN EMG” company has been made. An important component is the mathematical analysis of these artifacts and their elimination for the possibility of practical application of bionic artificial limbs based on neural network and other algorithms in practice.

The same methods of control had been analyzed for movement of the knee, the first results just showed amazing implementation, and muscular activity of stump of leg can be used for control of movements of the knee. The control of sole does not need such instruments, because people usually use running artificial limbs for legs. Theoretical and preliminary practical results have shown big prospects in this direction, namely, the use of muscular activity and recognition of movements on the basis of neural network; in the process of completion of works in this sphere, they will be published.

The electromyographic bracelet has found practical application in rehabilitation and game devices as it has been told earlier. And, researches and testing of bionic artificial limbs of hands on disabled people and testing of a bracelet on game devices for the last year have yielded new results which have allowed to draw a number of the main conclusions regarding formation of feedback at users, on the principles of receiving in general feedback on the basis of EMG.

It is important to note that because of these developments, four patents for the invention are created [17, 18].

Advertisement

3. Results of the practical application of developments of the recognition of EMG based on the neural network for disabled people at the management of bionic artificial limbs of hands and for game devices for people without disability

As already it has been noted above, thanks to the created technology researches of innovative feedback based on EMG for people without loss of limbs and also with loss of limbs regarding management of the systems of recognition of EMG based on the neural network have been conducted. Categories of people who participated in the research are the following:

  • Without loss of an extremity (18 people): from 12 to 65 years with different functional adaptabilities to new devices and psychological outlook

  • People with loss of an extremity (16 people): from 22 to 60 years with different psychological views and speed of reaction

Results of the analysis of an experiment have not included people who took only single part in a research as to reveal certain regularities, and there are needs to hold regular testing and checks; single participation is not natural and does not give understanding about feedback which is formed, and also in connection with complex psychological structure of people, the probability of obtaining wrong data is high.

The research objective is to reveal regularities using the control system of recognition of EMG from the shoulder and the forearm based on the neural network for different groups of people.

If to carry out the comparative analysis at people without amputation and with amputation, then results showed that physical training and a training of muscles plays a significant role regarding recognition of an EMG of a signal.

In spite of the fact that the signal in itself at people with an amputation of a hand is much more weak than people without loss have arms, the muscular training sometimes at people with amputation of an amputation allows to receive more accurate signal than a signal at the person of the same age group without loss of the limbs. Regular researches of people with amputation of limbs showed that more often there is a training of muscles and the muscle tone and also a comprehension and “representation” of gripper which are carried out by the person with amputation, and as a result, the subsequent already management of a bionic limb comes back quicker.

If to compare groups of people with the amputation of arms and different age categories, then the research showed that management does not depend on age in any way, but depends on the term of amputation and existence at the disabled person of a bionic/myoelectric prosthesis during this period. People who right after amputation began to use bionic/myoelectric prostheses or had a possibility of a training of muscles, had better discernible signals on frequency, and better coped with a task of management of a bionic prosthesis based on an EMG. There is an assumption that this feature arises because when the person lost a limb, at it the phantom feeling of an arm continues certain time to be held muscle in remembrance (even taking into account that injuries sometimes happen various and a part of muscles cannot remain or undergo other operations), as well as sometimes. If after that are not carried out a series of actions for maintenance of a muscle tone, then eventually there is a sharp atrophy of muscles which is harder and harder for restoring every year.

For example, the girl with amputation of the arm at accident of semiannual prescription, signals are equivalent to signals of the person without loss of a limb, even despite the performed two operations, and arenot comparable with the girl’s signals at all with amputation of an arm of 3-year prescription, they after to which loss of a limb established a cosmetic prosthesis. The girl with amputation of semiannual prescription is capable to carry out 8 of 11 grippers of the hand, while the girl from 3-year prescription is capable to execute only 3 grippers taking into account that each user adapts signals for itself, that is, the neural network with the teacher is used and also taking into account that on all grippers the portable electromyograph fixed signals have been in both cases.

Conclusion: after carrying out operations within rehabilitation and the subsequent use to establish to the recommendation immediately bionic artificial limbs as eventually without trainings and also because of psychology of people “experience of management” the hand is lost, and without trainings of the muscle atrophy. However, restoration is possible; alas, it will demand much more time than in cases of immediate prosthetics by bionic/myoelectric artificial limbs.

In the course of the research pattern at the people using myoelectric prostheses and appreciable difference from the framed system on recognition of an EMG on the basis of neural network from eight sensors and more was taped, namely, at amputation of an arm, for example, management of myoelectric prostheses happens at a muscle tension of a biceps and a triceps at amputation of an arm.

The daily training leads to their appreciable development, but in cases when the same people begin to test an innovative control system on recognition of an EMG based on neural network with the teacher of Bi-oN, excellent recognition of signals regarding the turn of the hand has been provided to them. But performance of other grippers is problematic. The similar story at amputation of a hand is slightly higher than a hand at amputation term more than 3 years: management of signals of muscles to which disabled people got used is accurately traced, but at extension of the list of the gripper and a request to involve other muscles, there are problems with management which are surmountable at a training eventually but at the beginning bring difficulties.

It is important to note that the innovation of developments of Bi-oN has allowed to speak about new types of neural-control interface and feedback on the basis of signal EMG (such division as signals which are sent to the executive mechanism which are executed by the mechanism strongly differ has been allocated):

  • The neural-control interface of the computer or a mobile application (the person)

  • The neural-control interface of the artificial hand (the person)

  • The neural-control interface of other executive mechanism (the person)

Moreover, it is important to note that for the first time when people begin to use neural-control interfaces, there is a fear, and after time there is adaptability.

Fundamental difference of the given neurointerfaces at external observation and according to users in how there is a management and receiving feedback. For example, the neurointerface of the computer-person allows to carry out rehabilitation and has no hidden mechanisms of management; for example, the bracelet is put on an arm, the computer is connected, and there is a calibration of a control system. Further, the user has to carry out the gripper or programs of a task at the movement by an arm. The computer signals about successful or wrong performance of a task. It is important to notice that in this “simplest” mechanism of the neurointerface a very important technique of a training of muscles is hidden; in particular the people who lost limbs or after a stroke when performing these tasks train not only muscular activity but also cerebrally, sometimes framing new neuronic communications which allow to cope with tasks more effectively. Special case, for example, of the most successful use of the neurointerface of the computer-person is used by his disabled person with amputation—at amputation of a brachium compression of “fist” took about 6 seconds; the patient that considerably complicatedly use it for control of other devices therefore the user thought up new approach and a signal on compression, that is it not compression of a fist, but compression of four fingers, except a thumb. Its execution lasted about 1–1.5 s; as a result the user learned to control an imaginary arm by means of new signals which very quickly became current. It is important to notice that feedback on gripper turns out by means of visual signals and the additional vibromotor in a bracelet, which can react at successful performance of a task. Possesses the similar scheme neurointerface with virtual reality which, in difference from augmented reality, transfers the person completely to the virtual world, but it is important to notice that psychologists long use of virtual reality because of yet not studied injuries which this reality can cause is not recommended. Therefore, all methods of rehabilitation take no more than 15–30 min, and even it is less.

Absolutely, a new principle regarding the neurointerface the hand person—it fundamental difference is especially shown at disabled people for whom the hand fastens to a sleeve and allows to control an arm actively. It is important to note that according to disabled people, it is perfect other perceptions and adaptations, namely, unlike the previous mechanism when there is a special ligament of people computer, in this case there is a communication of people hands which frames tactile perception and feelings when performing tasks.

And essentially another from all previous diagrams on neurointerface, is neurointerfaces on control of machines, quadcopters, etc. Their distinctive feature is, first of all, existence of the additional software on reassignment of grippers; for example, movement “turn to the right” is transferred “movement of a hand to the right,” “fist”—advance, “disclosure of a palm”—a stop, etc. That is, the use of system of reassignment leads to new perceptions and appearance of neural communications. This development is for the present researches and finishing, but in the next year shall be complete and receive practical application on bigger number of users.

Videos

There are many videos about research; most of them can be found with the following links:

Advertisement

4. Conclusion

It is worth noticing about essentially new approach in management of limbs if, for example, earlier the signal of muscular activity above a certain level was required for us, then in a case with neural network and special sensors and a payment of management of production of LLC “Bionic Natali” and LLC “Bi-oN EMG,” any signal, even we can “program” very weak signal somehow now; that is, in the absence of muscles or an atrophy of any muscles, it is possible to create a new type of management which is characteristic only of this patient, and also it is important to note convenience of recalibration which the patient can execute itself in house conditions.

On the basis of researches, the important conclusion was drawn: after carrying out operations within rehabilitation and the subsequent use, it is necessary to establish immediately bionic artificial limbs as eventually without trainings and also because of psychology of people “experience of management” an arm is lost, and without trainings of a muscle atrophy. However, restoration is possible; alas, it will demand much more time than in cases of an immediate prosthetic repair by bionic/myoelectric artificial limbs. The current researches also showed requirement of rehabilitation of disabled people before the use of bionic prostheses of arms; as the option is offered the use of an electromyographic bracelet of LLC “Bi-oN EMG.”

Advertisement

Acknowledgments

This research was financially supported by the “Skolkovo Innovation Center” and personally funded by LLC “Avenir” company (LLC “Bionic Natali”).

Advertisement

Notes/thanks/other declarations

Thanks to all members of teams LLC “Bionic Natali” and LLC “Bi-oN EMG” and people with disabilities and finance support from LLC “Avenir” (Boris Zhilin and Olga Masyutina), finance and technological (service of the registration of patents, professor Victor Lempitsky from Skoltech, etc.) support of Innovation center “Skolkovo,” thanks to whom projects are working.

References

  1. 1. Vorotnikov SA, Strunin VS, Vybornov NA. The Biometric Control System of the Artificial Limb
  2. 2. Nikolaev SG. Electromyography: Clinical practical work. Ivanovo. 2013
  3. 3. Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering. 2003;50(7):848-854
  4. 4. Van den Broek EL, Lis’y V, Janssen JH, Westerink JHDM, Schut MH, Tuinenbreijer K. Affective Man–machine Interface: Unveiling Human Emotions through Biosignals. Biomedical Engineering Systems and Technologies: Communications in Computer and Information Science. Berlin, Germany: Springer Verlag; 2010
  5. 5. Haziakhmetov MS. Properties of window dispersion of a myogramma as casual process. Systems and Means Inform. 2014;24(3):110-120
  6. 6. Chernyshev AA, Mustetsov NP. Algorithm upravleniyamnogofunktsionalnym hand artificial limb
  7. 7. Haziakhmetov MSh. Properties of window dispersion of a myogramma as casual process
  8. 8. https://en.wikipedia.org/wiki/Curse_of_dimensionality
  9. 9. Ivaniuk N, Ponimash Z, Karimov V. Art of recognition the electromyographic signals for control of the bionic artificial limb of the hand. In: International Conference of Artificial Intelligence, Medical Engineering, Education, AIMEE 2017: Advances in Artificial Systems for Medicine and Education. 2017. pp. 176-181. https://link.springer.com/chapter/10.1007/978-3-319-67349-3_16
  10. 10. Natallia I, Zahar P, Vladimir K. Control systems of bionic limbs of the new generation. Int Rob Auto J. 2017;3(3):00059. DOI: 10.15406/iratj.2017.03.00059. http://medcraveonline.com/IRATJ/IRATJ-03-00059.pdf
  11. 11. Galushkin AI. Synthesis of multi-layer pattern recognition systems. M.: Energy. 1974
  12. 12. Rumelhart DE, Hinton GE, Williams RJ. Learning Internal Representations by Error Propagation. Parallel Distributed Processing. Vol. 1. Cambridge, MA: MIT Press; 1986. pp. 318-362
  13. 13. Petrov AP. About perceptron opportunities. News of Academy of Sciences of the USSR, the Engineering Cybernetics. 1964;6
  14. 14. Uidrou B, Stirns P. The adaptive signal processing. M.: Radio and Communication. 1989. 440 p
  15. 15. Ayvazyan SA, Bukhshtaber VM, Enyukov IS, Meshalkin LD. Application-oriented statistics: Classification and lowering of dimensionality. M.: Finance and Statistics. 1989
  16. 16. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. Springer; 2001
  17. 17. Патенты: Ivaniuk NM, Karimov VR, Budko RY, Gronskiy PV, Kleyman SM. Patent for the invention of the Russian Federation LLC “Bionic Natali” № 2016149199, 12.12.2016. Method and control system of the intellectual bionic limbs”//Патент России № 2635632.2016 and on the basis of him the international patent of PCT
  18. 18. Karimov VR, Ivaniuk NM, Ponimash ZA, Shepanskiy VE, Petrosian AY. Patent for the invention of the Russian Federation LLC “Bi-oN EMG” № 2017146206, 27.12.2017. “Method and control system of electronic devices by means of the electromyographic system of reading” and on the basis of him the international patent for the invention of РСТ

Written By

Natallia Ivaniuk, Zahar Ponimash, Vladimir Karimov and Valentsin Shepanskiy

Submitted: 31 October 2017 Reviewed: 02 February 2018 Published: 24 October 2018