Smartphone and Portable Media Device: A Novel Pathway toward the Diagnostic Characterization of Human Movement

Abstract

The application of wearable and wireless systems offers the capacity to ameliorate considerable strain on medical resources. In particular the smartphone and portable media device for quantifying human movement characteristics offers the opportunity to evaluate patients in a homebound environment remote from clinical resources and post-processing. Trial data can be easily transmitted as an email attachment with wireless connectivity to the Internet. The utility of the smartphone and portable media device has been demonstrated for quantifying gait, tendon reflex response, movement disorder, and rehabilitation exercise. Further evolution and potential has been demonstrated through the integration of machine learning to provide classification accuracy for differentiating between disparate human movement scenarios. The role of the smartphone and portable media device for quantifying human movement characteristics is further elucidated.

1. Introduction

Wearable and wireless systems, such as smartphones and portable media devices, have been demonstrated as functional wireless accelerometer and gyroscope platforms for quantifying human movement endeavors, such as gait, tendon reflex response, movement disorder, and rehabilitation exercise. Further evolution of the technology capability has been demonstrated with the integration of machine learning for attaining classification accuracy [1–3]. The success of these applications derives from preliminary research, development, testing, and evaluation of wireless accelerometer nodes that are essentially wearable for similar scenarios [4].

Inertial sensors such as accelerometers were originally proposed for the quantification of human movement, however at the time of this perspective they were not sufficiently evolved. The technology evolution of accelerometer systems proceeded in tandem with disparate industries relative to the biomedical, rehabilitation, and health industry [4, 5]. Quantification of rehabilitation status can facilitate the modification of a therapy intervention, especially with the expert clinical acuity of a trained expert, however traditional quantification apparatus, such as gait analysis quantification equipment, is generally restricted to a clinical environment [6, 7]. The role of the accelerometer system steadily progressed with the biomedical, rehabilitation, and health community [8]. Eventually with the development of wireless technology supporting inertial sensors, such as accelerometers and gyroscopes, other techniques, such as tethering, became effectively outmoded [9].

The wireless accelerometer has been successfully demonstrated as a wearable system for the quantification of human movement [4]. Tandem operated wireless accelerometer systems have been successfully applied to the quantified evaluation of hemiplegic gait [10–12]. Other associated endeavors have demonstrated the role of wireless accelerometers for quantifying reflex response and even reflex latency [13–19]. Further testing and evaluation has revealed the utility of wireless accelerometers for quantifying movement disorder tremor, such as for Parkinson’s disease [20, 21]. Further evolution of the capability of wearable and wireless applications is featured through the research, development, testing, and evaluation of smartphones and portable media devices as wireless accelerometer and gyroscope platforms [1–3, 22].

Ever since the origins of the smartphone, one of its well known features is the observation that the screen will shift orientation based on movement and position. This capability is due to its inertial sensor, originally consisting of an accelerometer and now also a gyroscope. With this feature noted LeMoyne and Mastroianni sought to utilize this inertial sensor package to characterize human movement, much like their previous application of wireless accelerometers for similar endeavors [1–3, 22].

2. Smartphone: wireless accelerometer platform for quantifying human movement

Following this requirement definition LeMoyne and Mastroianni successfully acquired of an appropriate software application to record the accelerometer signal data and convey the data package as an email attachment. This capability transformed the smartphone from a telecommunication utility to a wearable and wireless accelerometer sensor system capable of measuring human motion within the context of an assortment of scenarios. Given the compact nature of the smartphone, it can be easily mounted about an assortment of readily identifiable anatomical mounting positions. Preliminary research, development, testing, and evaluation of the smartphone emphasized the role of quantifying gait and Parkinson’s disease [1–3, 23, 24].

During 2010 LeMoyne and Mastroianni applied the smartphone as a wearable and wireless gait analysis device. The smartphone was equipped with an application that enabled it to function as a wireless accelerometer platform. The recorded data package of the acceleration waveform was conveyed by wireless transmission to the Internet as an email attachment [23].

Another major observation of the capabilities of this application was based on the remote nature between the experimental site and post-processing resources. The gait experiment was conducted in the region of Pittsburgh, Pennsylvania. However, the post-processing resources were situated in the greater Los Angeles area. The implications of the research endeavor are that with a suitable software application the smartphone operating as a wireless accelerometer can quantifying human movement characteristics, such as gait, with post-processing resources situated anywhere in the world. Essentially the email resource symbolizes a functional semblance of a cloud computing resource [23].

For the scope of gait analysis the smartphone was secured about the lateral malleolus near the ankle joint by an elastic band. Each gait experiment recorded on the order of 10 s of steady state walking. Temporal and kinematic parameters, such as the stance to stance temporal disparity and stance to stance time averaged acceleration, were acquired in an accurate and consistent manner [23]. Further testing and evaluation of the smartphone as a wireless accelerometer platform for gait analysis were successfully demonstrated using alternative mounting positions, such as the lateral epicondyle of the femur and the lumbar-sacral aspect of the spine [25, 26].

During 2010 LeMoyne and Mastroianni also applied the smartphone as a wearable and wireless accelerometer for the quantified acquisition of Parkinson’s disease tremor. Measuring Parkinson’s disease at the convenience of a patient’s homebound setting is of paramount significance, in order to provide optimal acuity for expert clinical medical therapy intervention. A smartphone could measure Parkinson’s disease hand tremor through mounting to the dorsum of the hand. With the experiment conducted in Pittsburgh, Pennsylvania and post-processing resources situated trans-continentally in greater Los Angeles, the application again demonstrates the ability to remotely situate experimental and post-processing resources. Tremor characteristics were successfully quantified for a person with Parkinson’s disease and contrasted to a non-Parkinson’s subject [24].

The implications of the research, development, test, and evaluation demonstrated by LeMoyne and Mastroianni elucidate the broad power of wearable and wireless systems, such as the smartphone as a wireless accelerometer platform. The smartphone can be easily mounted to effectively any portion of the body that best defined the characteristics of the human movement feature under consideration, such as the near the ankle joint for gait or dorsum of the hand for Parkinson’s disease tremor. From the experimental site, the recorded accelerometer signal data can be conveyed through wireless Internet connectivity to a post-processing resource anywhere in the world. In essence a subject can access the best clinical resources in the world from the convenience of a familiar homebound and autonomous setting. Further testing and evaluation of the smartphone as a wireless accelerometer pertained to the quantification of reflex response.

The reflexes of the lower limb are synergistically interrelated with the function of gait [19]. Therefore with the success of the smartphone as a wireless accelerometer platform for quantifying gait, the patellar tendon reflex response logically should also be a readily quantifiable aspect of human movement. Preliminary testing and evaluation the smartphone as a wireless accelerometer for quantifying reflex response pertained to manual stimulation of the patellar tendon reflex [27]. The accurate and consistent quantification of the patellar tendon reflex can be further facilitated through the application of a potential energy impact pendulum for evoking the tendon reflex response with a prescribed amount of energy that is also targeted to a specified aspect of the patellar tendon [14–19].

Using a remote and effectively rural area as an experimental site, LeMoyne and Mastroianni applied a smartphone as a wireless accelerometer platform in conjunction with a potential energy impact pendulum. The integration of these devices readily acquired a recording of the reflex response acceleration waveform for the patellar tendon. The trial data was conveyed by wireless transmission to the Internet as an email attachment. Remotely situated post-processing resources applied software automation to quickly determine the efficacy of the experimental trial data. The findings advocate that the patellar tendon reflex response can be readily acquired through the integral application of the potential energy impact pendulum with the smartphone as a wireless accelerometer platform in an accurate and consistent manner [28]. Further establishment of the smartphone for quantifying the reflex response was demonstrated though the application of an artificial reflex system [29].

Further investigation of the opportunities for gait quantification emphasized the evaluation of gait for people with transtibial amputation. In consideration of people with transtibial amputation, they require a different mounting technique as opposed to merely applying an elastic band to secure the smartphone. In order to resolve this matter, a 3D printed mounting adapter was applied to secure the smartphone to the transtibial prosthesis. The smartphone conveniently measured the acceleration waveform of the subject’s gait respective of the transtibial prosthesis. Automated software facilitated the post-processing endeavor with gait characteristics acquired in an accurate and reliable manner [30].

3. Smartphone: wireless gyroscope platform for quantifying human movement

With the continuous evolution of the smartphone eventually the gyroscope was integrated into the inertial sensor package. The gyroscope measures the rate of angular rotation, which offers a readily clinically identifiable metric, especially in consideration of the orientation of a particular joint [1–3, 22]. In particular the smartphone as a wireless gyroscope platform can quantify the characteristics of the patellar tendon reflex response [31].

In order to accurately and consistently quantify the reflex response of the patellar tendon through the application of a smartphone as a wireless gyroscope platform, a means of evoking the reflex with a similar level of accuracy and consistency is imperative. The potential energy impact pendulum attached to a reflex hammer was selected, which has been successfully researched, developed, tested, and evaluated, while demonstrating the ability to elicit the tendon reflex at prescribed levels of potential energy with predetermined targeting [14–19]. The integration of the potential energy impact pendulum and smartphone as a wireless gyroscope platform provide considerable opportunity for the accurate and consistent measurement of the reflex response.

Upon the experimentation trial data was conveyed by wireless transmission to the Internet as an email attachment. Each gyroscope signal was post-processed through a software automation program for the acquisition of a pertinent parameter, such as the maximum gyroscope signal of the response. This preliminary investigation of the application of the smartphone as a wireless gyroscope platform in conjunction with a potential energy impact pendulum demonstrated the ability to accurately and reliability quantify the patellar tendon reflex response in terms of the gyroscope signal [31].

Further testing and evaluation of the smartphone as a wireless gyroscope platform investigated the capability to quantify gait. Using a mounting strategy that positioned the smartphone about the trunk, preliminary gyroscope signal data was acquired for gait. The findings implied that the smartphone as a wireless gyroscope platform could also be applied for identifying gait characteristics [32].

Later in the chapter the expanded role of the smartphone as a wireless gyroscope platform is demonstrated for the domain of machine learning classification. An assortment of machine learning algorithms are applied, for which the gyroscope signal provides a suitable feature set. Machine learning using the smartphone as a wireless gyroscope emphasizes the domains of classifying therapy exercise status and differentiating hemiplegic affected and unaffected reflex pairs.

The portable media device represents a similar wireless sensor platform, first with the accelerometer and later with the gyroscope. The portable media device and smartphone are supported by the same software application, such as for the iPod and iPhone. The appropriateness of either device for an experiment quantifying human movement is at the discretion of the research team. For example, the portable media device has a reduced telecommunications footprint, which is restricted to local wireless connectivity for access to the Internet; however the portable media device in general is somewhat lighter and relatively cheaper [1–3]. The following section investigates the capability of the portable media device of quantifying human movement, such as reflex and gait, first from the perspective of a wireless accelerometer platform.

4. Portable media device: wireless accelerometer platform for quantifying  human movement

Preliminary testing and evaluation of the portable media device pertained to the evaluation of gait. The portable media device was mounted proximal to the ankle joint and secured by an elastic band. Connectivity to the Internet was enabled through local wireless connectivity, for which the trial data samples were conveyed as email attachments to a predetermined address. Because of the opportunities facilitated by Internet connectivity though the email address, the experimental site and post-processing resources are trans-continentally situated on either side of the United States of America [33].

Upon testing and evaluation notable advantages of the portable media device are evident. The portable media device is lighter than the smartphone. Because it only relies on local wireless Internet connectivity, there is only a fixed cost for the device as opposed to a continuous marginal cost to sustain a telecommunication footprint. The portable media device demonstrated the ability to acquire gait data in an accurate and consistent manner [33]. Further testing and evaluation of the portable media device as wireless accelerometer gait analysis system about the lower aspect of the trunk successfully identified temporal features of gait [34].

As possession of two portable media devices is relatively more feasible than ownership of two smartphones, the application of tandem mounted portable media devices to quantify the disparity present in hemiplegic gait can be readily accomplished. Both portable media devices functioning as wearable and wireless accelerometers were mounted proximal to the lateral malleolus near the ankle joint for the hemiplegic affected leg and unaffected leg. Again the experimental and post-processing resources were trans-continentally situated with the trial data wirelessly conveyed to the Internet as an email attachment [35].

The post-processing aspect derived the affected leg and unaffected leg temporal disparity of stance to stance, time averaged acceleration of stance to stance, and the affected leg/unaffected leg ratio for stance to stance time averaged acceleration less the offset. These quantified gait parameters demonstrated considerable accuracy, consistency, and reliability. From an inferential statistical perspective the temporal disparity of stance to stance and time averaged acceleration of stance to stance for the affected leg contrasted to the unaffected leg demonstrated statistical significance. Future evolutions to the concept envision automated derivation of parameters and machine learning classification [35].

As previously disclosed gait and reflex of the lower limb, such as the patellar tendon reflex, are neurologically associated [19, 36]. Therefore the portable media device provides a useful means of quantifying the patellar tendon reflex response. In particular the localized wireless connectivity to the Internet is interrelated with the generally indoor and localized environment inherent for obtaining a series of patellar tendon reflex response samples. LeMoyne and Mastroianni demonstrated in 2011 the ability to accurately and consistently characterize the reflex response through manual supramaximal stimulation using a portable media device as a wireless accelerometer platform [37].

Further application of the portable media device as a wireless accelerometer platform amalgamated the utility of the potential energy impact pendulum. The portable media device was mounted proximal to the lateral malleolus of the ankle joint by an elastic sock. Each trial sample was transmitted to the Internet through localized wireless connectivity. Post-processing consolidated the accelerometer signals in three dimensions to a single signal presenting acceleration magnitude. The maximum acceleration of the reflex response was the parameter of interest. The application for quantifying reflexes demonstrated the capacity to characterize the reflex response in a considerably accurate and consistent manner [38].

5. Portable media device: wireless gyroscope platform for quantifying human movement

Further evaluation of the portable media device and its associated software for the capability as a wireless gyroscope platform developed. In particular since the reflex response of the patellar tendon is jointed about the knee, therefore the gyroscope signal revealing the rate of angular rotation provides a highly clinically relevant signal. The trial data of the portable media device functioning as wireless gyroscope platform is wirelessly conveyed to the Internet as an email attachment [31, 39]. Its implications are that the post-processing location, such as in America, can be totally remote from the experimental site, such as remote Tibet.

While on travel LeMoyne and Mastroianni decided to truly test the robust capability of the portable media device as a wireless gyroscope platform for quantifying the patellar tendon reflex response with the experimental site selected as Lhasa, Tibet and the subsequent post-processing in Flagstaff, Arizona in the United States of America. An advantage of the portable media device is the sole need to locally connect to a local wireless Internet zone. The data samples were obtained through supramaximal stimulation of the patellar tendon reflex. The portable media device was mounted to the lateral malleolus of the ankle through a sock. The data was stored at an assigned email site for later post-processing, for which the maximum of the gyroscope signal characterizing the reflex response was acquired with considerable accuracy and consistency [39].

6. Machine learning classification with the smartphone and portable media device as wireless accelerometer and gyroscope platforms

The synergy of devices, such as smartphones and portable media devices, for quantifying human movement and machine learning offers the exciting opportunity of highly objective assessment, automated computer aided diagnosis, and prognostic forecasting. LeMoyne and Mastroianni have recently emphasized the classification of movement features of the hemiplegic affected and unaffected side. Preliminary test and evaluation has demonstrated considerable classification accuracy [1, 2, 22].

These accomplishments have featured the use of Waikato Environment for Knowledge Analysis (WEKA). WEKA is equipped with a considerable array of machine learning algorithms, such as the support vector machine, multilayer perceptron neural network, J-48 decision tree, logistic regression, and K-nearest neighbors. The appropriate machine learning algorithm is selected at the discretion of the research team with consideration to the classification endeavor at hand. The feature set that comprises the data is organized into an Attribute-Relation File Format (ARFF) file. The ARFF file is derived primarily from a series of numeric attributes that appropriately quantify the classes to be distinguished [40–42].

The ability for even a skilled team of clinicians to conclusively decide upon the presence of a hemiplegic reflex pair is a subject of contention [19, 36]. Considerable classification accuracy was attained through the amalgamation of a portable media device as a wireless accelerometer platform, a potential energy impact pendulum, and a machine learning algorithm. Each reflex response sample of the affected hemiplegic reflex response and unaffected reflex response was transmitted through local wireless connectivity as email attachments for post-processing. Software automation consolidated the trial data into a cohesive feature set. The selected machine learning algorithm was the support vector machine. This research demonstrates the considerable promise of the integration of smartphones and portable media devices as wearable and wireless inertial sensor platforms in combination with machine learning algorithms for classification of perceptively distinguishable human movement features [43].

Further investigation regarding the utility of the portable media device as a wireless accelerometer platform extended to the domain of assistive devices for gait, such as a cane. An issue with using a cane involves the concern that the patient is properly using the cane per the advice of a skilled therapist. However, such instruction is only realistically feasible respective of a relatively brief clinical appointment. A portable media device functioning as a wireless accelerometer platform can monitor the usage of a cane while the subject is walking, and the data package can be conveyed by local wireless connectivity to the Internet as an email attachment. Considerable classification accuracy was attained using logistic regression as a machine learning algorithm to distinguish between correct and incorrect cane usage scenarios with the recorded accelerometer signal deriving the feature set [44].

LeMoyne and Mastroianni demonstrated the potential of integrating a portable media device as a wireless gyroscope platform for quantifying the patellar tendon reflex response and machine learning for the classification of a hemiplegic reflex pair. The tendon reflex was elicited through supramaximal stimulation from a manually operated reflex hammer. The portable media device functioning as a wireless gyroscope platform was mounted proximal to the lateral malleolus as demonstrated in Figure 1 [45].

Figure 1.

Portable media device as a wireless gyroscope platform for quantifying the patellar tendon reflex response [45].

In order to consolidate the affected leg and unaffected leg reflex response into a feature set their respective gyroscope signals needed to be considered, such as presented in Figure 2. Three numeric attributes comprehensively characterize the nature of the reflex response:

1. maximum angular rate of rotation;

2. minimum angular rate of rotation;

3. time disparity between maximum and minimum angular rate of rotation [45].

Figure 2.

Gyroscope signals for a hemiplegic reflex pair (affected leg (a) and unaffected leg (b)) for the patellar tendon reflex response [45].

These three numeric attributes represent the reflex response in terms of both kinematic and temporal features. The feature set was evaluated through WEKA using the J48 decision tree. An advantage of the WEKA J48 decision tree is the ability to visualize the decision tree, which may infer the most predominant aspects of the feature set [45].

The J48 decision tree is illustrated in Figure 3. Note that the feature set attribute TimeMaxMinGamma is clearly the dominant attribute for distinguishing between the hemiplegic affected leg patellar tendon reflex response and the unaffected leg patellar tendon reflex response. The research findings demonstrate the capacity of machine learning, such as the J48 decision tree, for attaining considerable classification accuracy that distinguishes between a hemiplegic affected patellar tendon reflex response and correlated unaffected patellar tendon reflex response through the quantification of a portable media device functioning as a wireless gyroscope platform [45].

Figure 3.

J48 decision tree applied for attaining considerable classification accuracy for differentiating between a hemiplegic reflex pair for the patellar tendon reflex response [45].

An extension of the utility of using the portable media device as a wireless gyroscope platform with machine learning was applied to classify the usage of a proposed rehabilitation system. An ankle rehabilitation system intended to promote dorsiflexion was developed through 3D printing. The operation of the ankle rehabilitation system was quantified by mounting a smartphone functioning as wireless gyroscope platform to the foot plate of the ankle rehabilitation system [46].

The ability of the smartphone functioning as wireless gyroscope platform to convey its data sample as an email attachment through Internet connectivity makes the integral application suitable for homebound therapy scenarios. A subject could conduct a therapy session at home, and then the post-processing with machine learning classification could provide a therapist with advanced acuity of subject rehabilitation status. The gyroscope signal was distilled into a feature set for machine learning classification through a support vector machine, for which considerable classification accuracy was achieved [46].

Deep brain stimulation has been demonstrated as an efficacious treatment strategy for people with movement disorders that are intractable with medical therapy intervention. In particular essential tremor can be treated through the application of deep brain stimulation. However an issue is evident with respect to the considerable array of tuning parameters, which can present a daunting task for even an expert clinician. A smartphone as a wireless accelerometer platform presents a promising means for quantifying the efficacy of deep brain stimulation for people with essential tremor [47, 48].

Preliminary research, development, testing, and evaluation of the capability to a smartphone as a wireless accelerometer platform to facilitate the tuning of a deep brain stimulator for ameliorating symptoms of essential tremor pertained to attaining machine learning classification accuracy for differentiating between the deep brain stimulator in ‘On’ and ‘Off’ states. Regarding both of these states a subject with essential tremor was tasked with reaching and grasping a lightweight object with a smartphone mounted to the dorsum of the hand. The trial data of the smartphone functioning as wireless accelerometer platform transmitted the recording as an email attachment with connectivity to the Internet. Subsequent post-processing of the acceleration signals derived a feature set that was applied to a machine learning algorithm. The most appropriate algorithm for the classification task under consideration based on the expertise of the research team was the support vector machine. The support vector machine achieved considerable classification accuracy for differentiating between a person with essential tremor conducting a simple reaching and grasping task with deep brain stimulation in ‘On’ and ‘Off’ status based on the acceleration signal of a smartphone functioning as a wireless accelerometer platform [47].

Reduced arm swing is a subject of concern for people with hemiplegic gait. Objectively quantifying the severity of reduced arm swing for the affected side contrasted to the unaffected side may enable a therapy intervention to ameliorate the impact of reduced arm swing during gait. Given the inherently rotational nature of arm swing during gait, this scenario is particularly relevant to the application of a smartphone functioning as a wireless gyroscope platform [49].

Especially regarding an outdoor environment for walking and evaluating reduced arm swing respective of hemiplegic gait, the smartphone functioning as a wireless gyroscope platform can readily transmit experimental data as an email attachment through its broad wireless connectivity to the Internet. The smartphone can be easily mounted proximal to the wrist joint through an armband as illustrated in Figure 4. Post-processing can be conveniently conducted in a remote location [49].

Figure 4.

Mounting of the smartphone as a wireless gyroscope platform for quantifying reduced arm swing [49].

Figure 5 demonstrates the sample gyroscope signals of reduced arm swing of the affected arm contrasted to the unaffected arm. Based on the consideration of the gyroscope signal data the feature set was composed of aspects, such as descriptive statistics. A multilayer perceptron neural network was selected as the machine learning algorithm as provided in Figure 6. Considerable classification accuracy was attained for differentiating reduced arm swing regarding the hemiplegic affected arm and unaffected arm [49]. Similar results were also achieved with respect to another form of reduced arm swing manifested by Erb’s Palsy [50].

Figure 5.

Gyroscope signals for reduced arm swing regarding the hemiplegic affected arm (a) and unaffected arm (b) [49].

Figure 6.

Multilayer perceptron neural network for classifying reduced arm swing regarding the hemiplegic affected arm and unaffected arm [49].

Further refinement of the machine learning classification of a hemiplegic patellar tendon reflex pair was enabled through the application of the potential energy impact pendulum that provides predetermined amounts of potential energy to evoke the reflex and targeting of the reflex hammer. Upon observation of the obtained gyroscope signals the feature set was composed of three attributes:

1. maximum angular rate of rotation;

2. minimum angular rate of rotation;

3. time disparity between maximum and minimum angular rate of rotation.

The multilayer perceptron neural network was selected as the machine learning algorithm, for which considerable classification accuracy was attained distinguishing between a hemiplegic patellar tendon reflex response and its associated unaffected patellar tendon reflex response [51].

A similar experiment was applied using supramaximal stimulation of the patellar tendon reflex through a manually operated reflex hammer. Figure 7 illustrates the acquired gyroscope signals for the affected leg and unaffected leg regarding the patellar tendon reflex response. Subsequent post-processing derived the multilayer perceptron neural network presented in Figure 8. This research endeavor achieved appreciable classification accuracy for differentiating the hemiplegic (affected and unaffected) patellar tendon reflex pair [52].

Figure 7.

Gyroscope signals for the affected leg (a) and unaffected leg (b) regarding the patellar tendon reflex response [52].

Figure 8.

Multilayer perceptron neural network for classifying the affected leg and unaffected leg regarding the patellar tendon reflex response [52].

Two recent developments regarding the role of smartphones and portable media device functioning as wireless gyroscope platforms for therapy and rehabilitation exercise have been applied to eccentric training using Virtual Proprioception and wobble board therapy. These two applications emphasize the flexibility of these devices, since the smartphone uses its broader telecommunication footprint regarding Virtual Proprioception for eccentric training in a gym setting and the more localized wireless connectivity of the portable media device in a homebound setting. Both experiments involve automated post-processing to develop a feature set for machine learning classification using a multilayer perceptron neural network [53, 54].

Virtual Proprioception for eccentric training is based on the successful application of Virtual Proprioception for real time gait rehabilitation. Virtual Proprioception for real time gait rehabilitation applied biofeedback based on a wireless accelerometer signals from a hemiplegic affected leg and unaffected leg while walking. Both auditory and visual feedback enabled the subject to modify gait in real time to achieve a closer state of parity regarding the acceleration waveforms of both legs. The wireless inertial sensors, such as the accelerometer, provide a virtual alternative to the neurological basis of proprioception [55, 56].

Eccentric strength training has be proposed as an highly effective means of strength training, however the rate of change is significant for the quality of the eccentric training event. Visual real time feedback from the gyroscope signal of a smartphone functioning as wireless gyroscope platform enables Virtual Proprioception for eccentric training. Figure 9 illustrated the mounting of the smartphone through an armband proximal to the wrist joint for a biceps curl using Virtual Proprioception for eccentric training [54].

Figure 9.

Virtual Proprioception for eccentric training demonstrated with a smartphone functioning as a wireless gyroscope platform secured proximal to the wrist through an armband [54].

During the eccentric muscle lengthening phase of a biceps curl the subject while using Virtual Proprioception for eccentric training uses the visual feedback from the smartphone gyroscope signal to ensure that a prescribed threshold is not exceeded. This inertial sensor acuity provides a virtual and quantified representation of neurologically derived proprioception. The control aspect of the experiment consists of an eccentric phase of a biceps curl without Virtual Proprioception. As illustrated in Figure 10 these gyroscope signals display visually notable disparity [54].

Figure 10.

Gyroscope signals for eccentric phase of a biceps curl respective of Virtual Proprioception for eccentric training (a) and without feedback (b) [54].

The experimental trial data was transmitted wirelessly to the Internet as an email attachment. Automated post-processing of the data package developed a feature set based on the gyroscope signal data. Through the application of a multilayer perceptron neural network illustrated in Figure 11 considerable classification accuracy was achieved for distinguishing between scenarios with Virtual Proprioception for eccentric training and eccentric training without feedback through Virtual Proprioception [54].

Figure 11.

Multilayer perceptron neural network applied for classifying between scenarios of eccentric phase for a set of biceps curls respective of Virtual Proprioception for eccentric training and without feedback through Virtual Proprioception [54].

A wobble board is a therapy device that promotes rehabilitation of the ankle foot complex. A portable media device as shown in Figure 12 can be readily mounted to the wobble board to provide advanced acuity as a wireless gyroscope platform. The rotation of the wobble board is notably disparate regarding the hemiplegic affected ankle and unaffected ankle upon observation. The observed disparity can be quantified through the wireless gyroscope platform from the portable media device and conveyed as an email attachment through wireless connectivity to the Internet. Post-processing consolidates the gyroscope signal into a feature set. With a multilayer perceptron neural network considerable classification accuracy was achieved for differentiating between the hemiplegic affected ankle and unaffected ankle [53].

Figure 12.

Wobble board for therapy of the ankle foot complex with a portable media device functioning as a wireless gyroscope platform [53].

7. Future concepts for smartphones and portable media devices, sensor fusion

In the near future the application of sensor fusion will likely be achieved broadly throughout the domain of smartphones and portable media devices as wearable and wireless accelerometer and gyroscope sensors. Sensor fusion involves the tandem operation of multiple inertial sensors, such as an accelerometer and gyroscope. With the combined signal of both accelerometer and gyroscope signal the actual spatial location in terms of displacement, velocity, and acceleration of the sensor mounting position can be determined [57–59]. For example, the spatial representation of foot displacement can be ascertained during gait [57].

Traditional sensor fusion however requires a considerable sampling rate that exceeds the feasible sampling threshold for smartphones and portable media devices as wearable and wireless systems [1–3, 57]. Post-processing resources are generally not portable, and techniques, such as the Kalman filter are required to serve as an orientation filter [57–59]. An orientation filter that satisfies the computational bounds associated with smartphones and portable media devices would be desirable for sensor fusion applications.

Madgwick has proposed a gradient descent algorithm as an orientation filter that is feasible for smartphones and portable media devices for the scope of sensor fusion. The IMU version requires only on the order of roughly 100 computational operations per filter update. The filter is also amenable to reduced sampling rates, such as 10 Hz [58, 59]. These capabilities make the gradient descent algorithm a realistic orientation filter with regards to sensor fusion for smartphones and portable media devices.

Sensor fusion also requires a novel application for smartphones and portable media devices that can simultaneously record the accelerometer and gyroscope signal for the same sample. Cognition Engineering has recently developed a smartphone and portable media device application suitable for acquiring both the accelerometer and gyroscope signal in a simultaneous manner. An example of the Cognition Engineering application has been demonstrated with regards to a person with essential tremor conducting a reach and grasp task with a deep brain stimulator in ‘On’ and ‘Off’ mode [60].

8. Network centric therapy

These trends evidenced through the development of smartphones and portable media devices as wireless accelerometer and gyroscope platforms that are effectively wearable advocate the development of Network Centric Therapy. With the development of Network Centric Therapy a patient and therapist could reside hundreds or thousands of mile remote. Rather than scheduling a traditional clinical appointment, therapy exercises and evaluation could be measured and quantified by systems, such as smartphones and portable media devices as wireless accelerometer and gyroscope platforms, for the quantification of human movement. The patient in a familiar setting of choice could conduct each therapy exercise and evaluation, and the acquired data could be transmitted wirelessly through Internet connectivity as email attachments. The post-processing could apply machine learning for augmented acuity for the therapist to determine critical transition phases of the therapy prescription and optimization of the rehabilitation experience.

9. Conclusion

Smartphones and portable media devices have been demonstrated through progressive research, development, testing, and evaluation as wearable and wireless accelerometer and gyroscope platforms for the quantification of human movement. In particular their utility has been advocated in domains, such as quantifying gait, tendon reflex response, movement disorder, and rehabilitation exercise. Experimental data can be readily transmitted through wireless connectivity to the Internet as an email attachment. Post-processing resources and the experimental location can be remotely situated anywhere in the world. Post-processing the inertial sensor signal data into a feature set enables machine learning classification. Research has demonstrated the capacity of an assortment of machine learning algorithms to achieve considerable classification accuracy for differentiating disparate human movement scenarios, such as contrasting a hemiplegic affected side to its associated unaffected side. Smartphones and portable media devices functioning as wireless accelerometer and gyroscope platforms are envisioned to facilitate the development of Network Centric Therapy, which is predicted to radically advance the therapy and rehabilitation experience.