A Robotics-Based Machine Learning Approach for Fall Detection of People

Abstract

For a person when carrying out household chores or even when walking on the streets, there is a risk of falling. This risk increases throughout the years due to the natural aging process. In this work, a bibliographic review was performed to find related papers who discussed different techniques for fall classification. The aim of this study was to develop two ML models: an SVM and a k-NN model, to classify the fall. An accelerometer, gyroscope, and magnetometer located on the waists of 15 volunteers are the application sensors. The extracted features were the mean, standard deviation, and range for each sensor. The best accuracy obtained was 93.89%, a sensitivity of 85.10%, and a specificity of 96.99%. All results were obtained by simulations, by using the test set separated in the first stage of the implementation. So, a shortcoming is the fact that the ML models were not tested with a hardware implementation. In future works, the models can be embedded into a microcontroller and classify data in real time.

1. Introduction

As the years go by, bodies become weaker and thus give up their physical health. It can lead to new problems and challenges for the elderly because there comes a time when they need to be more cautious, and not everyone can be that way. And it is in this context that falls among the elderly are becoming more and more frequent. Falls among them have more consequences than a scrape on their bodies. People over 60 are gradually becoming more vulnerable to falls [1].

Falls among the elderly happen suddenly and are very frequent. According to Ref. [2], about 30% of people over 65 years old suffer a fall at least once a year, increasing to 50% when they are over 80. Falls are a problem of worldwide interest, which brings consequences to people and governments due to the heavy investment to recover its citizens. Therefore, researchers are always looking for solutions to improve people’s quality of life.

Since 1991 the authors in Ref. [3] began studies to use wearable sensors to solve this problem. Other works in this field were in Refs. [4, 5], which proposed a protocol for evaluating the performance of any developed system.

Usually, those devices are at high end and challenging for a consumer with a low income to acquire because of the costs. The two most popular ways to detect falls are video [6] and measuring signals from an accelerometer placed on the body [7]. There are vast possibilities for integrating these devices with machine learning (ML) techniques to correctly classify data received from video streaming or sensors placed on the body.

2. Falls

“Fall detection involves complex pattern recognition, which tends to vary according to each individual who suffers a fall” [8]. According to Ref. [1], falls can be defined as “an event that results in a person unintentionally stopping their activities on the ground, floor, or a lower level.” Falls can also be defined as “falling to the floor or some other lower level as a consequence of receiving a violent blow, loss of consciousness, paralyzes such as a stroke or a seizure of epilepsy” [9]. Approximate 684,000 fatal falls occur each year, with 80% of these fatalities concentrated in low- and middle-income countries [1].

According to Ref. [9], most falls happen in the sagittal and coronal planes, as shown in Figure 1. These names are related to the human body and its anatomy. It is worth noting that when a fall occurs with the loss of consciousness, as described in Ref. [9], that is when the body suffers more. It is due to the lack of absorption of impact since the body falls directly to the ground. When a fall happens, the person is conscious can absorb the impact by stretching their arms to protect themselves if they fall forwards.

Figure 1.

Sagittal and coronal planes of the human body.

Serious injuries include traumatic brain injury, concussion, hemorrhages, and cuts [6]. In Brazil, the Sistema Unico de Saude (SUS) spends more than R$51 million annually treating various fractures because of falls [6]. According to Ref. [10], approximately one in three adults who live in their homes suffers a fall annually. And of those adults, about half of them will experience falls more frequently. According to Ref. [1], numerous factors can influence a person to suffer a fall, and among the most prominent are age, gender, and health.

2.1 Factors who contribute to falls

According to Ref. [9], the age factor is not enough to describe the risk of a person falling; therefore, a person is more likely to fall depending on several other factors. It is worth noting the risk of an elder suffering a fall is higher due to the inherent aging process. The factors that contribute to the event of a fall can be separated into two categories: intrinsic and extrinsic [6, 9].

Intrinsic factors are those that depend on the person, such as medication use, low muscle mass percentage, dizziness, and lightheadedness [6]. Among these factors, Ref. [9] also includes osteoporosis, Parkinson’s, dementia and cognitive problems, inadequate lifestyle, vision problems, chronic diseases, and previous falls. An inadequate lifestyle is directly linked to a sedentary lifestyle since physical activity helps to strengthen muscles [6].

Extrinsic factors are external to the individual [6]. Among them are slippery floors, stairs, inadequate footwear, crowded places, low light conditions, and damaged sidewalks [1]. Poor condition sidewalks represent a worrying problem in Brazil, based on a study conducted by Ref. [11]. They found that the average score attributed to sidewalks in several cities, on a scale of 1 to 10, is 3.40. A good score for the quality of sidewalks would be 8.0 [11].

2.2 Consequences of falling

There are several consequences because of a fall. Falls as an outcome of accidents are one of the reasons for hospital admissions and the leading cause of death among people over 65 years [9]. Among the types of consequences, Refs. [6, 9] emphasize physical and psychological damage, and in addition, Ref. [9] also mentions financial losses. Serious injuries are related to physical consequences. The most common minor wounds are bruises and scrapes [9]. There are many serious injuries, such as concussions, bleeding, skull trauma, and fractures [6].

According to Ref. [6], the most common consequence among the psychological type is fear of suffering new falls, but still Ref. [9] also mentions the lower quality of life, loss of independence, low self-esteem, and limited abilities. The economic implications are just as important as others because of the medical expenses. Among these expenses are rehabilitation therapies, medical examinations, hospitalizations, and the purchase of medical equipment [9]. Due to such arguments, it is a must to prevent falls. Figure 2 shows an example of a fall registered by the three sensors considered in this work. For every sensor, there are three individual graphs.

Figure 2.

A fall registered by the application sensors.

In Figure 2, one can observe a graph created using the accelerometer, gyroscope, and magnetometer readings while simulating a forward fall. This is a simulation of a fall caused by fainting or syncope forwards. These three sensors are located in the person’s waist.

In this example, the volunteer stands up until the ninth second. When this mark is reached the person falls forwards, simulating a consciousness loss. At this moment, there is an abrupt change in the sensor’s readings, and the accelerometer’s value reached its peak at ±5 g. There was an impact, and towards around the 10th second, the volunteer hit the ground and remained in this position (this scenario did not consider recovery after impact).

2.3 Related works

Bibliographic research was carried out through the Univali Integrated Library System (SIBIUN), which performs a search in the Univali collection, CAPES Portal, EBSCO, Biblioteca A, Saraiva, Vlex, Scielo Livros, Scielo Periodicals, and Open Access Directories. The search strings “Machine Learning” AND “Fall Classification” were used, yielding 184 results. After reading the abstracts, four relevant studies were selected.

In Ref. [12], three sensors collected data from an accelerometer, a gyroscope, and a magnetometer. This group of sensors were placed in five places on the volunteers’ body, such as on their head, chest, waist, wrist, and legs. The authors used six different ML techniques, including k-nearest neighbor (k-NN), support vector machines (SVM), least square method, Bayesian decision making, dynamic time warping, and artificial neural networks. Overall, the work scored optimal results, with an accuracy of 99.91%, a sensitivity of 100%, and a specificity of 99.79% [12]. The best accuracy was achieved by the k-NN algorithm, with 99.1% [12].

In Ref. [13], was carried out a similar work by using the same sensors as previously cited. However, authors placed the sensors only on the waist of the volunteers since the human body’s center of mass is located there. To perform the signal classification were used three stages of a fall. These stages are impact, post-impact, and posture. The proposed solution is based on a threshold comparison to identify each one of the stages. It is worth noting that in Ref. [13], SVM was used to extract thresholds for each phase. With this proposed solution, the result was 100% accuracy, sensitivity, and specificity for the classification [13].

The work in Ref. [6] differs from the related studies. In particular, the authors used an accelerometer and a gyroscope embedded in a smartphone to capture the sensors signals and classify them. A belt was used to secure the smartphone to the volunteer’s waist. Like [12], this study used the idle time. After detecting the inactivity time, data were classified using a decision tree and a threshold classifier and verified the actual orientation of the device. If all verifications are true, a fall is notified. The system in Ref. [6] achieved an accuracy of 93.25%, a sensitivity of 95.45%, and a specificity of 87.65%.

The most recent work is Ref. [14]. The authors also used all three sensors. They created the dataset FallAllD, which is available to the academic community. The volunteers used the set of sensors on three parts of their body: the chest, wrist, and waist. The authors explore four different ML techniques to classify falls: k-NN, SVM, random forest classifier, and convolutional neural network. Although all the three sensors collect data, only the accelerometer readings were used to train the ML models, looking for a simplified operation. The authors found an accuracy of 89.70%, a sensitivity of 95.06%, and a specificity of 95.20% when applying the k-NN technique. The implementation of the SVM technique with a quadratic kernel achieved an accuracy of 85.86%.

In Ref. [15], the authors demonstrate techniques not only to reliably detect a fall but also to automatically classify the type. Fifteen volunteers simulate four different types of falls-left and right lateral, forward trips, and backward slips—while wearing mobile phones. They applied five machine learning classifiers to a large time-series feature set to detect falls. Support vector machines and regularized logistic regression were able to identify a fall with 98% accuracy and classify the type of fall with 99% accuracy.

In Ref. [16], the authors present a comprehensive literature review on various ML-based classifications in fall detection. The authors identify the main problems in threshold-based classification from existing works and find the need for an efficient ML-based classification technique to accurately identify the fall. In addition, the shortcomings associated with the ML-based techniques for future research and other problems, such as data preprocessing and data dimensionality reduction techniques, are investigated. They concluded that ML-based techniques are far superior to threshold-based techniques.

Table 1 shows the comparison between the related works.

Characteristics	[12]	[13]	[8]	[14]	This work
Dataset	Erciyes University	DOFDA	MobiFall, MobiFall2, & own	FallAllD, Sisfall, & UMA-Fall	FallAllD
Number of volunteers	10 men & 7 women	6 men & 2 women	4 youngsters & 4 elders	8 men & 7 women	8 men & 7 women (simulation)
Sensors*	A, G & M	A, G & M	A & G	A, G, M & B	A, G & M
Groups of sensors	6	1	1	3	1
ML algorithms	K-NN, LSM, SVM, BDM, DTW, ANN	Threshold based	Binary tree & threshold	k-NN, SVM, LSTM & other	SVM & k-NN

Table 1.

Comparison between related works and algorithms.

^*

A = accelerometer, G = gyroscope, M = magnetometer, and B = barometer.

3. Development

In this work, the Python programming language was used. Besides the built-in library, we used other embedded resources to manipulate the data samples, that is, to create the ML models and to generate the confusion matrices. In addition, the Pandas’ library was used to manipulate the data. This library is popular among Data Scientists due to its reliability and ease of use. Another functionality of this library is the ability to handle missing samples and to calculate simple statistical characteristics.

The Scikit-learn library was also used in this work. This library allows to create, train, and test the ML models. The Scikit-learn also release access to ML models, and to different training techniques, prediction, and allows to divide the dataset into training and test sets. The confusion matrices are also generated by a function of the Scikit-learn library. The Matplotlib was also used to plot the confusion matrices previously generated. Finally, Pickle allows developers to save and load datasets and ML models.

Datasets available to the academic community were researched. In Refs. [12, 13, 14] were found three datasets. The dataset in ref. [12] has the biggest data samples, however some miss relevant data. On the other hand, the dataset in ref. [13] does not have a pattern in the time domain of sensor readings. In this work, the dataset created in Ref. [14] was used. It was recently created and does not utilize mattresses to cushion the falls, making them more realistic. Figure 3 depicts the block diagram of the proposed system.

Figure 3.

Block diagram of the system.

The information extracted from the dataset contains the sensors readings from an accelerometer, a gyroscope, and a magnetometer. Next, the feature extraction was performed to train and validate the ML model. It is possible to perform the data classification after training the model, which can be done in two categories: Fall or Activity of Daily Living (ADL).

In Ref. [14], developers can capture data from the wrist, waist, and chest. The data captured from the waist was created by 14 volunteers, who used safety equipment to prevent injuries. The authors chose not to use mattresses to cushion the falls to be as realistic as possible. The volunteers were free to choose which ADLs or falls they desired to simulate. All 14 volunteers chose to simulate ADLs and 12 out of 14 volunteers performed simulated falls. Every single scenario was recorded for 20 seconds. During the first 9 seconds, the volunteer had a movement to simulate, when the ninth second was reached the volunteer mimicked a fall, and the person could stay down or recover depending on which type of fall he was simulating.

The authors labeled as ADLs or Falls data samples within the dataset by using numbers as activity IDs. IDs ranging from 1 through 44 are samples representing ADLs. Since we are only considering samples recorded by those sensors located in the waist, ADLs range from 13 through 44, because those activities labeled from 1 through 12 were recorded by sensors located in the volunteer’s wrist. Among those ADLs, one can find activities such as: walking, running, standing up from a chair, and jumping.

Falls were labeled from 100 through 135. Among these falls, you can find different types of falls that normally would occur to people day to day. Volunteers simulated falls slipping, tripping, or losing balance while walking and slipping, and those falls were forwards, backwards, and laterally. They also simulated falls while running, lying in bed, trying to sit down, or standing for a while; these falls were simulated forwards, backwards, and laterally. It is also important to point out that falls with recovery were considered effectively as falls in this work.

It is important mention that those 14 simulating ADLs and those 12 simulating falls had to repeat the scenario several times to obtain the best and most accurate result. They could decide how much time they needed to rest between trials, and also, volunteers could decide the order in which they desired to perform the activities [5]. Repetition becomes a factor, as described in Ref. [5], because the volunteers can get used to the pattern of simulating that activity, resulting in activities performed in an unnatural manner.

With this said, we created a new column to label each sample as ADL or fall, represented by 0 s and 1 s, respectively. For this, we implemented a for loop, in which we compared the value stored in the activity ID column, and if this value was greater or equal to 100, we set the output column to 1, otherwise 0 was attributed.

Since the volunteers performed several times the same activity, the best scenarios were chosen to compose the dataset. Taking this into consideration, the dataset has 1797 samples of simulated falls and ADLs. Three features were extracted from the dataset to train the models: the mean, the standard deviation, and the range. The features were extracted for each one of the three axes of the sensors. The dataset was divided into three parts to perform training, validation, and testing of the models.

It is noteworthy to mention that the dataset needed simple data manipulation before extracting those features. The original dataset published by Ref. [14] is in bytes, so this way authors can adapt the dataset to their sensor’s sensitivity. We considered the same sensitivity for the accelerometer, gyroscope, and magnetometer. The sensitivities were 0.244 mg/LSB, 70 mdps/LSB, and 0.14 mgauss/LSB, respectively. Since the dataset was used as a Pandas dataframe, we multiplied every column by its corresponding sensitivity; after multiplying every data sample, we obtained the sensor’s original readings.

Figure 4 shows part of the Dataframe structure. It has 1798 rows and 7 columns in total. It is important to remark that only the data collected by the sensors located at the waist of the participants were used in this work. Also, the barometer readings were not considered.

Figure 4.

Example of the Dataframe structure.

The feature extraction was performed using the built-in functions in the Pandas’ library. Pandas has a mean, standard deviation, and minimum and maximum functions available and ready to use, so firstly 27 new columns were added to the dataframe to save the features. Eighteen columns were needed since we are considering the three features for every one of the three axes, that is, three columns for acceleration mean in x, y, and z, repeating this to the standard deviation and range of the accelerometer, so having a total of 27 columns.

We transformed the original column of each sensor containing all three axes into three separated columns to represent each of them. Next, we used the functions mentioned before to calculate the features. Since there is no built-in function to calculate the range, we find the maximum value and subtracted the minimum value. After completing these steps, the dataframe has all the characteristics and it is ready to use with the ML model.

Figure 5 shows the Dataframe final state after including the accelerometer, gyroscope, and magnetometer features.

Figure 5.

The modified Dataframe structure.

In Figure 5, it is possible to observe the pure data of the “Acc”, “Gyr,” and “Mag” sensors; however, the models will be trained with the columns that are on the right of those measures. A column called “Fall” identifies whether this event represents a fall or an ADL.

We used 80% of the data (not the 80% of the volunteers) for training, 10% of data for validation, and the residual 10% for final testing. It is important to note that the models were trained using the stratified k-fold cross-validation technique, with k equal to 10, to obtain a good balance among the output classes. Data for both, training and validation, are divided into 10 parts, where k-1 is used for training and k is used for validation. This task is repeated k times to complete the training of the models. Figure 6 illustrates this process.

Figure 6.

k-fold cross-validation. Adapted from Ref. [17].

To perform the data classification, two ML models were created. One model uses the SVM classifier, and the other one uses the k-NN. Both models use all the sensors’ data with their respective characteristics. The models were studied and compared with the results obtained by the authors in the related works.

4. Results

4.1 SVM

In this work, the dataset was divided randomly. By performing the training and validation, it was possible to achieve an accuracy of 95.05% using the SVM model. However, this value cannot be considered the final accuracy because it is necessary to submit the model to a final test. In the final test, we used data which was not previously known by the model. The purpose of this procedure is to classify the unknown data.

The accuracy of the final test was 93.89%, with a sensitivity of 85.10% and a specificity of 96.99%. The accuracy informs how many samples were correctly classified. On the other hand, the sensitivity is the ability to predict the true positives of each available category and lastly, the specificity is the ability to detect the true negatives of each category.

The confusion matrix is shown in Figure 7. This matrix was created from the results retrieved from the final test. It is possible to observe the true negatives, false negatives, false positives, and true positives, where 0 represents ADLs and 1 represents falls.

Figure 7.

Confusion matrix for the SVM model.

It is possible to observe that there were 129 true negatives, 4 false positives, 7 false negatives, and 40 true positives. This is a good result because the model correctly classifies 40 of the 47 falls.

4.2 k-NN

In this technique, we followed the same procedure as described before. After training, the model presented an accuracy of 88.45%. In the final test, with unknown data, it was possible to achieve an accuracy of 87.77%, a sensitivity of 82.98%, and a specificity of 89.47%. In this study, the results of the k-NN model were inferior, when compared with the SVM model, that is, the accuracy was 5.44% lower in relation to the SVM model. The confusion matrix for this model is shown in Figure 8.

Figure 8.

Confusion matrix for the k-NN model.

Compared to the SVM model, the number of false negatives was increased by 1, and the number of false positives increased by 10.

4.3 Analysis

To get a better understanding of the results, it is necessary to make a comparison with the related works. It is worth noting that among the related works there is a discrepancy among the results using the different ML techniques. Likewise, it should be considered that each one of the authors used different features or methods to perform the data categorization (Table 2).

Characteristics	[12]	[13]	[8]	[14]	This work
Sensitivity (%)	99.56	100	95.45	—	85.10
Specificity (%)	99.38	100	87.65	—	96.99
Accuracy (%)	99.48	100	93.25	84.66	93.89

Table 2.

Comparison between the best results achieved in the related works.

The best results can be found in Ref. [12] because the authors in Ref. [13] did not base their solution using ML. The classifier is based on thresholds; however, it is important to note that the extraction of the thresholds was performed using the SVM technique. In Ref. [12], the authors achieved accuracy of 99.1%, and in Ref. [14], the accuracy was 89.70% when applying the k-NN technique. In this chapter, we achieved an accuracy of 87.77%, thus 11.33% below the result of Ref. [12] and 1.93% below the results in Ref. [14]. The results obtained here are comparable to the results in Ref. [14] due to the similarity of accuracy between these studies.

The SVM performed better in this work, so it is possible to make a direct comparison with Ref. [14]. In this chapter, the best accuracy was 93.89% compared with the 85.86% in Ref. [14]. In Ref. [12], the higher accuracy was achieved (99.48%). Different features were used for the accelerometer, magnetometer, and gyroscope sensors, considering that each one of the related works used different features to train the ML technique. In Ref. [14], the authors used three features, obtained from the accelerometer. In this work, we extracted three features, for each one of the three sensors.

Every work has its limitations, and this work is not an exception to that rule. The simple statistical characteristics can represent a limitation of this work. This can be considered as one due to its lack of precision representing the original signal. The original recorded signal was 20 seconds long, as mentioned before, so representing these signals only by using the chosen features can be not accurate enough. This limitation should be taken into consideration if the intent is having a more realistic classifier.

A shortcoming of this work is the fact that the ML models were not tested with a hardware implementation. All results were obtained by simulations, by using the test set separated in the first stage of the implementation. The models can be embedded into a microcontroller and classify data in real time. The outcome of a hardware implementation can yield different results, them being higher or lower in comparison with those obtained by simulation.

5. Conclusions

An ML-based approach to fall problem detection was presented in this work. A literature review made possible to understand what is behind a fall and its consequences on people, as well as to remark the ML techniques explored in the literature to approach this problem. In this study, two models were created using different ML techniques, and training was the same for both. We applied k-fold cross-validation, with training, validation, and testing sets. Both models were trained considering the data obtained from the accelerometer, gyroscope, and magnetometer.

The mean, standard deviation, and range were used as input features for the ML models. The results reached a value that enables comparisons to those in the related studies. The best result was accuracy of 93.89% for the SVM technique. Currently, an embedded system is being developed with an ESP32 microcontroller to communicate with the sensors, embedding the classification algorithm and sending notifications.

This work can be complemented by embedding the ML models and building a physical device to test the models in real time with sensor readings, consequently obtaining more realistic results. To further improve this work, we recommend employing more features, like authors of Ref. [12] did with their work. By applying more characteristics it is possible to have better results, since there is more information regarding the sensor’s readings. Having more information fed to the models is a better approach because they can have a better understanding of what those characteristics are representing; therefore, a better division of possible outputs is achieved.

Acknowledgments

We gratefully acknowledge the support of the Fundação de Amparo à Pesquisa do Estado de Santa Catarina (FAPESC), grant number 2021TR001236; and the National Council for Scientific and Technological Development (CNPq), grant numbers 305835/2021-1, 424937/2021-2.