Virtual Rehabilitation and Training for Postural Balance and Neuromuscular Control

1. Introduction

Virtual Reality (VR) has been used in an increasingly wide range of applications, and technological advances have made it accessible to the everyday consumer. Virtual reality games have become the norm in today’s consumer marketplace. This makes it possible to consider such devices for more productive use of leisure time, such as in exercise and health maintenance. Gaming systems such as the Nintendo® wii-fit™ have already introduced the concept of VR guided balance and fitness training.

Studies have shown that virtual reality has potential for assisting recovery from illnesses such as stroke and vestibular disorders and improvements in cerebral palsy. Custom designed VR systems in the laboratory have been successful with balance training [1]. Active video game systems, like the Sony PlayStation® 2 with the Sony EyeToy®, a camera accessory, have also been used in studies of rehabilitation. Flynn et al. used this system in a case study of a stroke patient and documented the types of training for each game [2]. The Nintendo® wii™ system was designed for in-home training as well as gaming. This system was tested on a 13-year-old patient with cerebral palsy who showed improvements in visual perceptual processing, postural control and functional mobility [3]. Therefore, these gaming systems show promise in rehabilitation, and have the added benefit of entertainment.

Other custom-designed games and virtual environments have also been developed to address specific disabilities and user requirements. Why do these systems work? In order to answer this question, the basics of human postural balance control have to be considered. The balance control system is a complex integration of multiple sensory inputs along with feedback from the motor component. The sensory inputs include visual, vestibular, auditory, tactile, and proprioceptive signals. The motor aspect is the “plant”, driven by neuromuscular control. Figure 1 demonstrates the complex interaction between the various sensory systems in the central processing of incoming signals. Different inputs are assigned weights based on motor learning and experience, i.e. the reliability of the incoming cues.

Figure 1.

Proposed comparator model of postural control: Sensory inputs are compared to each other for consistency, and a matrix of gains [V,C,P] is assigned accordingly. The weighted inputs are combined as S(t) which is the total sensed self-orientation x as a function of time t, with noise η(t) added. This results in a compensatory motor output through body dynamics. Feedback is provided through the senses, but also internally to update a memory or database of previously experienced combinations of sensory inputs. In addition to comparing the sensory inputs to each other, this database is also referenced.

Visual inputs are generally dominant among human sensory inputs for balance control as well as a general source of information, and virtual reality systems have evolved with an emphasis on visual displays. Balance improvement has been demonstrated through both static and dynamic visual stimuli in VR systems. [4] This results in a natural fit for visual-modality systems for both cognitive and neuromuscular control training along with entertainment value. However, other channels of information cannot be discounted, including the vestibular system, which is located in the inner ear and senses head acceleration.

The visual and vestibular systems are very closely linked, for example eye movements respond to vestibular inputs. Since the vestibular system senses head movements, it can be hypothesized that head movements together with visual stimuli should be used in training the balance control system. [5] Cohen et al. showed that purposeful head movement exercises improved balance in vestibular patients. [6] Therefore active games that track player head and body movements could demonstrate potential for balance improvement.

Other inputs include auditory and tactile modalities. Auditory signals provide orientation cues based on sound localization. Tactile cues provide pressure feedback and external inputs regarding contact with surfaces. Proprioceptive inputs are also important, providing us cues about our body position in space, through joint angle feedback signals. These signals are directly relevant to the control of ankles, knees, hips and upper body to maintain balance. Furthermore, the arms can be used for stabilization control and fall prevention.

In this chapter, we will investigate both commercial and custom-designed virtual reality interfaces for balance training and rehabilitation, considering the various inputs that are important for balance control. We can then begin to evaluate the important elements of virtual reality games, so that they can be developed more effectively for rehabilitation applications.

2. Arm movement importance and potential training system

Arm movements have been shown to be very important in balance maintenance and recovery [7]. Both unilateral and bilateral arm movements generate anticipatory and reactive responses in postural leg muscles [8] and hip and trunk muscles [9]. Furthermore, reaching and grasping actions are important in balance recovery and prevention of falls [7].

In a recent study we showed the importance of free arm movements in a series of classical balance and mobility tests in which three out of four tests showed significant improvement with the free use of arm movements [10]:

1. Maximal step length test (length and timing of a single maximally long step)

2. Step test (number of steps up and down onto an elevated surface in one minute)

3. Timed get-up-and-go test (time to get up from a chair, walk to a target, and return to sitting)

4. Walk along an elliptical line

Only test (4) did not show signficant improvement, possibly because there is less change of support motion of the body relative to the task of walking, compared to launching a single large or elevated stepping motion, and certainly compared to rising from a chair. These results show the importance of arm movements in controlling body balance.

Arm movements are affected by stroke and progressive disease, such as Parkinson’s disease (PD). Given the importance of arm movements for balance, such an effect could result in decreased balancing ability or steadiness of stance in these patients. Arm movement training is thus an important part of rehabilitation. We have developed a virtual reality arm movement training system [11], in which different games can be programmed to suit user abilities and interests. This system uses the Microsoft Kinect™ interface, along with pressure sensors attached to the index finger and thumb as an object selection tool. This system shows the actual image of the user on the screen, and we have developed games, which appear overlaid on the user’s image. This allows the user to reach out to the target letters and “select” them by making contact between the thumb and index finger. Initial testing on healthy subjects provided a baseline scores, so that we can now proceed to evaluate patients’ performance.

3. Balance training: Sony PlayStation® 2 with the Sony EyeToy® and the Harmonix AntiGrav™ game [12]

Based on the importance of head and arm movements along with visual inputs in balance training, the Harmonix AntiGrav™ game was selected for evaluation. The purpose of this study was to to characterize balance-related motor learning effects through game performance and to evaluate the effectiveness and specificity of the video game for balance training in healthy subjects. [12]

3.1. Game description

Sony's EyeToy® used video capture technology, in which it imaged the user with a small video camera mounted above the display and used face-recognition and motion-tracking technologies (no body markers were required). [13, 14] The system used two-dimensional user motion in the frontal plane as inputs. Each user was individually calibrated before each session, to account for height differences and range of head motion. During this procedure, the subject’s own image was shown on the screen overlaid with a calibration figure, instructions and controls. The subject had to move to a position such that his/her face was centered on the figure, and then was instructed to wave his/her hand over the “lock” control. In the subsequent screen, the subject was required to bend to the left and right such that the head was aligned with target positions. If this procedure was not properly executed, it became very apparent in the early stages of game-play that the game was not responsive (the avatar movements did not respond to user movements), and the calibration was repeated. A diagram of the system and screenshots is shown in Figure 2.

The user was portrayed in the game as an avatar and the user’s image was not shown after the calibration procedure. The avatar raced on a hoverboard along roadways and rails, over jumps, and when airborne, through rings. The user, standing in front of the display, used head movements to guide the avatar, and was required to jump or duck to avoid obstacles. The user also had to reach with either arm or both to strike targets as the avatar passed them. The arm motions were directed diagonally up or down or to the side, requiring additional head/upper body motions to contact the targets. The game was fast-paced and challenging, and included sound effects and "life or death" consequences. It presented unpredictable situations requiring maintenance of balance and decision making at the same time. As the user gained experience in the game, anticipation of upcoming movements became easier. Upon achieving a pre-set score, the user was advanced to playing both difficulty Level 1 and Level 2 in each training session.

Figure 2.

a) Conceptual schema of the PlayStation^®2, EyeToy^® and hoverboard game (not an actual screenshot) with the user. b) Screenshot of the first step in calibration. The actual image of the user is shown superimposed (off to the side in this picture to highlight the two images) on an outline of a human frame with an opening at the face. c) Screenshot of the game in progress. The avatar seen in the middle of the screen is riding on a “rail”, which constrains the path while presenting targets for arm movements. While elsewhere, head movements steer the avatar, here head movements are required to lean toward targets (if presented on one side) to be struck with rapid arm movements and to duck or jump over obstacles. The lower right corner has an indicator of the user’s head and arm positions in real time. These are also reflected in the avatar actions. In the center, the accumulated points are shown along with the target number of points to beat. The time remaining is shown in the top left corner. The bar on the left side indicates the current position along the racetrack.d) Screenshot of the game in progress. Here the user has successfully struck a pair of targets with both hands, and the user has been given a credit of extra time to complete the race. [12]

3.2. Experimental protocol

The control (no training, n=7) and experimental (video game training, n=7) groups completed the balance tests (as described below) at the beginning and at the end of a three-week period. Both the control group and the experimental group were free to conduct their daily activities as normal, and thus, the control group controlled for the effect of the specific experimental video game training. The experimental group trained 9 times during the three-week period, and the control group received no training. For details, see [12].

Game performance was measured by the number of tokens or targets struck with arm movements and the speed and accuracy in navigating the avatar through the race course using head movements. Additional points could be achieved by doing tricks, executed by arm movements while airborne. The scores were provided after each game, and points were displayed when awarded, which reinforced the immediate feedback provided during game play.

Three balance tests (Balance Board (BB), Tandem Romberg (TR), and One-leg Standing (OL)) were selected to evaluate transfer of the video game training to the real world. These tests evaluated transfer of learning to the real world because they were conducted without the virtual environment and thus tested for changes in balance outside of the game. Many balance-related measures were found in the literature ranging from the Dynamic Gait Index to the Sensory Organization Test to various functional measures and clinical tests. Our tests were selected based on common usage and relevance to overall balance [15-17] rather than factors such as sensory contribution or functional outcomes such as locomotion.

The average scores from two trials for each test were used. To minimize a “ceiling effect” (which occurs if most subjects pass a balance test given for a specified time, e.g. 30 seconds), the time was recorded when balance was lost. This gave a finer measure to detect and quantify improvement.

3.3. Results

3.3.1. Game scores

As expected, the game scores in both levels of difficulty improved over the 9 training sessions. The performance in the game was based on the game scores in both levels of difficulty. All subjects except one achieved the target score for Level 1 in the first session, and were moved to training at both Levels 1 and 2 by the second session. Overall, there was a gradual increase in the total scores for both Levels over the period of the experiment.

3.3.2. Model of motor learning through game scores

The improvement in game scores was consistent with our common understanding of improved performance with practice and with studies of motor learning. [18] The game scores were averaged across subjects and fit to an exponential given in Equations 1 and 2 for Level 1 and Level 2 games respectively [19]:

y(n) = 57,500-41,000e^-0.26n E1

y(n) = 57,500-43,500e^-0.087nE2

where n is the session number and y is the curve fit value. All three variables were initially used to obtain curve fits. It was found that constraining the DC term to be the same for both curves did not affect the quality of the fit, and it represented the average performance plateau after a large number of sessions. These curves are shown together with the averaged data in Figure 3. The curve fits reflected the increase and plateau of motor learning as a function of the session number for the first level (Level 1), and suggested that the subjects had not yet mastered Level 2 because of the absence of the plateau.

Figure 3.

Averaged scores over all 7 training subjects for Level 1 and Level 2 training along with corresponding exponential curve fits (see text). [12]

3.3.3. Balance tests

Performance on balance tests improved over the training period for the training subjects as compared to controls. During pilot trials, it was found that some volunteers were balancing for longer than three minutes on the balance board and two minutes on the Tandem Romberg tests. The BB and TR tests were discontinued after three and two minutes respectively because of the potential for muscle fatigue, which would interfere with the measure of balancing ability. [20] Of the 7 experimental subjects in each group (training and control), three control subjects reached the maximum time in the BB pre-test and one control subject in the TR pre-test.

Figure 4 shows the balancing times in seconds averaged for training and control subjects for each test. Trained subjects showed significant improvement in the BB test, moderate improvement in the TR test, and no significant difference in the OL test, while control subjects did not show significant difference on any of the tests.

3.3.4. Simulator sickness

Virtual Reality games have the potential for simulator sickness, which is related to motion sickness, with many of the same symptoms. Kennedy et al. have established a Simulator Sickness Questionnaire (SSQ) based on work with the US Navy, which is often used in evaluating simulator sickness. [21] It contains 16 items on which the subjects rated their responses (ranging on a 4 point scale; 0 (none), 1 (slight), 2 (moderate), 3 (severe)).

In this study, simulator sickness symptoms were recorded in every training session. Figure 5 shows the incidence of all symptoms across the 7 subjects and 9 training sessions. Most subjects who experienced symptoms reported slight sweating. No severe symptoms were present, and the symptoms subsided over the course of the training sessions. The presence of the symptoms suggested potential for vestibular rehabilitation, and their mildness demonstrated the safety of this system for the general public.

3.4. Discussion

The results showed that game scores improved over the sessions as expected. The average game scores were modeled, and showed an almost linearly increasing component followed by a plateau. Motor learning analysis showed that Level 1 was mastered by the participants while Level 2 was still showing learning. We developed a game difficulty parameter, which could be used in rehabilitation game design to provide appropriate increases in difficulty for continued learning and training.

Pre- and post-test results showed that subjects who played the game had an improvement in balance board and tandem Romberg tests over the 9 sessions. One-legged standing tests did not show improvement, which could be the result of training specificity, since the training was not performed on one leg.

The simulator sickness questionnaire showed the presence of several symptoms, especially in the earliest sessions. These were mild and decreased over the training period. The symptoms indicated that the game had a physiological effect on the participants, and the decrease in symptoms paralleled the motor learning.

Figure 4.

a) Time that the subjects were able to maintain balance on a Balance Board (BB) Pre- and Post-training. The performance for the training subjects (n=7), mean=67 and 142 s, is compared to the performance of the control subjects (n=7) who did not receive training during the 3-week period; mean=97 and 90 s. The trained subjects had a significant difference in balancing times (p<0.005).b) Time that the subjects were able to maintain balance in the Tandem Romberg (TR) test Pre- and Post-training. The performance for the training subjects (n=7), mean=41 and 102 s, is compared to the performance of the control subjects (n=7) who did not receive training during the 3-week period; mean=59 and 64 s. The trained subjects had a significant difference in balancing times (p<0.05).c) Time that the subjects were able to maintain balance in the One-Leg (OL) standing test Pre- and Post-training. The performance for the training subjects (n=7), mean=15 and 24 s, is compared to the performance of the control subjects (n=7) who did not receive training during the 3-week period; mean=25 and 19 s. The differences in balancing times were not significant for either group. [12]

Figure 5.

Simulator Sickness symptom occurrences over 7 subjects and 9 training sessions during a three week period. Some subjects had more than one symptom on a given training day, and some subjects reported no symptoms.

4. Auditory inputs

The auditory system also provides important cues to an individual’s self-orientation in space due to sound-localization capability. This ability is based on the difference in arrival time of a sound to each ear. We have used this concept in a prototype balance training aid, in which both visual and auditory feedback were provided to four healthy subjects. The system was tested with the subjects standing on a balance board. The auditory feedback was provided through stereo headphones worn by the participant. Both the Inter-aural Intensity Difference and the Inter-aural Timing Difference were controlled based on accelerometer signals from the tilt of the balance board. The combined audio-visual feedback resulted in significant improvement in the subjects’ balancing ability [22]. Future studies will test for the relative contribution of each of the two feedback modalities.

5. Tactile inputs

Tactile inputs can be utilized in VR systems with the increasing availability of haptic feedback. Multimodal feedback increases the sense of immersion in the virtual environment, and different sensory inputs can be harnessed for sensory substitution, additional information channels and most importantly, redundancy. Redundancy is a means of bringing important signals to the attention of the user, and alerting the user in emergencies. For this reason, we considered it important to include tactile feedback in the design of an alerting system to inform subjects about potential loss of balance.

We developed a Wireless Vibrotactile Rehabilitation System (WVRS), which provides vibration feedback during body tilt. [23] The system incorporates three vibrating disks on each arm, which are activated when an accelerometer, mounted on the mid-back detects body tilt within certain ranges. Figure 6 shows the design of the system, which includes a wireless link to a computer. The system can record and display the accelerometer signals. The three vibrating disks on each arm are activated as a function of the degree of body tilt and tilt direction.

Figure 6.

System design of a vibrating tactile balance feedback system. a) The main module includes a tri-axial accelerometer, Arduino microprocessor, and ZigBee wireless system, incorporating a Wireless Personal Area Network ID to distinguish the communication with the arm subsystems and the communication with the host computer. b) A system of three vibrating disks was provided on each arm, activated in each arm as shown, based on the amount of body tilt in each direction respectively. The vibration was applied at 200 Hz. [23]

The arms were selected for the vibration transducers to naturally produce a reaction in the appropriate arm, and by eliminating unnecessary processing steps, the arm movement could be rapid enough to help stabilize posture. An example result from one subject is shown in Figure 7.

Figure 7.

The subject was perturbed in the antero-posterior direction, and left and right body tilt were recorded with and without the use of the WVRS system. In both directions, the amount of body tilt was reduced. [23]

Such a system can be used as a VR rehabilitation tool, utilizing multimodal feedback with the visual display and providing progress tracking with the data recording capability. It is also anticipated that in the next design phase, the recording could also be housed in the main module, making the system portable. Such a wearable device could be worn as an everyday balance maintenance aid. This is an example of taking the virtual reality into the real world, or augmented reality.

6. Ankle training

In addition to head/body and arm movements, balance also depends on the quality of control in the joints of the lower limbs. In a balance training program, these joints can be involved in general training, or targetted specifically. When a joint injury occurs, for example in the knee or ankle, then a phased and progressive rehabilitation program is essential.

We have demonstrated ankle VR rehabilitation systems in which ankle movements in up to two degrees of freedom can be used to control a custom-designed VR game. The program was motivated by a sports injury, which can often take a long time to heal without properly designed rest and training. The games tracked the progress of the training and could be used as an inexpensive home-based training tool that motivates exercise adherence. The system had the following objectives [24]:

1. the training has a range of difficulty levels to suit different patient abilities, including isometric contraction when no joint rotation is desired

2. the system has the ability to constrain movement in both magnitude and direction so that specific muscle groups can be trained without the risk of re-injury, and

3. the game provides visual feedback through a game, which provides scores of the ankle control ability related to game performance.

These objectives have been satisfied with the incorporation of either a balance board simulating a 2 degree-of-freedom foot pedal interface [25], or a boot with a frame and locking pins that either prevent motion, or constrain motion to the plantarflexion/dorsiflexion plane [26]. Force sensors and accelerometers measured the muscle activity and ankle motion and provided inputs to the game. Visual interfaces were developed for calibration, as shown in Figure 8. Game scores were tracked and recorded across the training sessions, so that the patient and also the caregiver can monitor progress.

Figure 8.

Windows interface for accelerometer calibration [25]

The important side-benefit of such a visual gaming interface is the motivation to improve scores, and the entertainment value of the game. Sayenko et al. have shown that a custom-designed VR game provides useful feedback and motivation for a spinal cord-injured patient to prevent muscle atrophy through functional electrical stimulation training. [27] The game encouraged adherence to the exercise program. It is important to conduct further studies on patients to develop optimal training protocols.

7. Occupational health and safety: Training for correct lifting posture

Very specific aspects of postural balance can also be isolated and trained through VR. In the field of occupational safety, VR can be a very important training tool and real-time aid in injury prevention. We have developed a lifting technique trainer, in which an avatar shows the correct motion on the screen, based on published positioning guidelines. [28] The trainee lifts a training object wearing a harness instrumented with accelerometers as shown in Figure 9.

Figure 9.

Instrumented harness providing input to the VR training system. Accelerometers attached to the shoulder harness are positioned on the trainee’s shoulder blades, and the belt contains a third accelerometer, which is positioned on the lumbar region of the low back. [29]

The user can view the image of the avatar reflecting actual measured values from his/her upper body during the lifting motion. The discrepancy between the ideal and actual lifting motions is displayed, which guides the trainee to improve his/her lifting technique. Such a VR training aid provides instant visual feedback to allow the user to learn the body proprioceptive feedback signals associated with the correct postural positioning.

8. Conclusions

Different virtual reality training environments were evaluated or developed for improving postural balance and neuromuscular control. Balance training was shown to transfer to the real world from commercial video-game based training, and because balance is related to overall fitness, these training interfaces represent an important means of combining leisure and health maintenance. The results of such training could reduce the likelihood of falls and injury in everyday activities.

The benefits of VR training include the ability to isolate specific sensory inputs and motor outputs. Clearly, VR also provides immediate feedback to the user, which enhances learning. Further, VR provides an automated way to guide and track the progress of the training. Adaptive systems can sense the user’s performance and adjust parameters to provide the appropriate level of difficulty in each training session. Finally, VR provides a relatively safe environment for training compared to many real-world environments.

Acknowledgement

I would like to thank Dr. Ken Norwich, University of Toronto for his insight and wisdom and endless encouragement. I am also grateful to all of my students who have helped me immensely in this work, especially Matija Milosevic and Sumandeep Virk. I also thank the Natural Sciences and Engineering Research Council of Canada for funding the research.