Contributions of an Augmented Reality Musical System for the Stimulation of Motor Skills in Music Therapy Sessions

The music is able to awaken the most varied emotions in humans. According to Robert et al. (1988), listening to music, the ear transforms sound into electrical signals reaching the brain leading to increased production of endorphins. This hormone causes a sensation of wellbeing and relaxes the body, reducing heart rate and blood pressure. Human organism is endowed with a sound system capable of commanding the perception and production of sounds. When there is an imbalance in this system, the sick person feels less motivated and sadder. In this context, music can act as a therapeutic factor bringing back the balance that this person needs (Robert et al. 1988), (Leme, 1999).


Introduction
The music is able to awaken the most varied emotions in humans.According to Robert et al. (1988), listening to music, the ear transforms sound into electrical signals reaching the brain leading to increased production of endorphins.This hormone causes a sensation of wellbeing and relaxes the body, reducing heart rate and blood pressure.Human organism is endowed with a sound system capable of commanding the perception and production of sounds.When there is an imbalance in this system, the sick person feels less motivated and sadder.In this context, music can act as a therapeutic factor bringing back the balance that this person needs (Robert et al. 1988), (Leme, 1999).
Throughout history, music has been used in many ways and for different purposes, whether as a form of expression, communication and interaction, the source of meditation, hobbies, income, either as an aid in the process of prevention, restoration and rehabilitation of health.
According to Streeter (2001), recreational activities and musical composition when they involve the execution of musical instruments, for example, can increase performance in sessions of physical rehabilitation while regular physical exercise become repetitive and tiresome.These activities can arouse the interest of the individual primarily for use of the upper limb and provide stability of body, better range of motor movement thin or thick.During these activities, it is often necessary to use adaptive musical instruments to meet specific physical needs of the patient and help him use (Louro, 2005(Louro, , 2009)), (Lina, 2009).
By adapting instruments, to introduce postures, invest in equipment in the music therapy sector, the therapist contributes directly to improved performance in an area that applies different strategies to motor rehabilitation (manual dexterity, etc.), improved communication and speech, and especially self-esteem.
Many patients have varying degrees of disability, from the lightest to the most serious, requiring specialized teams that contribute to better adaptation of technologies and procedures to the needs of each patient (Nascimento, 2006), (Lina, 2009).Some technological adaptations are made by order to selected music therapists and, therefore, in small quantities for the sector (Nascimento, 2006), (Lina, 2009).Sometimes, an instrumental resource adapter practice serves only the needs of a particular disability, and for others, this same feature can be uncomfortable.Another relevant factor is the cost to manufacture and purchase of adaptive resources.It is also recommended for many patients to continue treatment at home.However, many patients do not have money to buy musical instruments adapted.
On the other hand, the number of computers and Internet access in households in Brazil is growing rapidly (CETIC, 2010).In addition to promoting access to information and communication, digital inclusion, has become an important ally of teaching and learning (Rekimoto;Nagao, 1995), (Thomas et al., 2010), (Lopes et al., 2010), training (Holden;Dyar, 2002), (Golomb, et, al, 2010) and simulation (Hoffman et al., 2003), (Botella, 2010).Thus, various computer technologies, with resources for people with severe physical disability, making it possible to have access to the tasks of daily living.
In health care, computers are used to motivating patients during therapy and to provide quantitative data for monitoring by the therapist (Oliveira et al, 2010).In particular, in the field of music therapy, software and electronics have enabled the "music making" therapeutic and educational for people with severe physical limitations.
Computer vision techniques, for example, enable capture and convert physical gestures of fingers, hands and feet to sound and graphic information (Gorman et al, 2007).Ultrasonic sensors can also be used for the same purpose (Soundbeam, 2011).
The virtual rehabilitation is gaining notoriety in the scientific community for providing the use of virtual environments, developed with virtual and augmented reality technology to rehabilitation.If compared to procedures performed manually, the Virtual Rehabilitation can provide numerous benefits, such as the ability to (Burdea, 2003), (Sveistrup et al., 2003): perform repetitive exercises more attractive with visual and auditory representations that motivate the patient, to obtain objective measures of exercise (speed of limbs, range of motion, hit rates and/or error scores in games, among others), to increase or reduce the complexity of the exercises, store the data collected for remote access, to do household activities assisted or not by the therapist, among others.
With augmented reality is possible create different computer systems for users unable to use conventional device such as keyboard and mouse (Garbin et al., 2006), (Garbin, 2009).This technology provides added virtual elements to the real world user, whose manipulation occurs in a natural way, by hand, without use of conventional devices such as keyboard and mouse adapters and.This augmented reality feature provides access for people with physical disabilities to virtual environments facilitating educational and therapeutic procedures (Thomas et al., 2010) and Therapeutics (Richard et al, 2007), (Assis;Lopes, 2008), (Botella, 2010).There are several ways of building Augmented Reality environments.The most common, affordable is to use a webcam to capture images of the real world, augmented reality software to mix them with virtual elements and a video monitor to display the images mixed (Nischelwitzer, et al, 2008).
However, the high potential of augmented reality for rehabilitation, were not found in the literature, works that demonstrate their effectiveness in music therapy.So far, augmented reality has been used as an attractive instrument to support the process of musical learning, but no major concerns with the interaction of people with disabilities (Zorzal et al., 2005), (Kirner et al, 2006), (Constanza et al, 2003).
Given the benefits provided by this technology in relation to the forms of interaction, we assume that the music therapist procedures when directed to people with disabilities can be facilitated by the use of this technology.In addition, systems designed with augmented reality can be easily accessed at home, which can enhance the treatment, including possibilities for remote monitoring.Another relevant factor is the potential of this technology to improve motivation and satisfaction of patients and positively influence their treatment in rehabilitation.In this research we describe a system that was developed with augmented reality technology to support music therapy and its evaluations.

Characteristics, behavior and benefits of augmented reality
Augmented reality is a technology that allows mixing user real environments with virtual environments.Azuma (1997) defines augmented reality as the overlay of virtual objects, computer generated, in a real environment, generating a mixed environment that can be perceived by some technological device in real time.Figure 1 shows two examples of applications of augmented reality: a sandwich and a butterfly overlayed on the real environment.
Fig. 1.Examples of augmented reality environments (Fischer et al, 2006) The creation of an augmented reality environment requires a computer, a webcam and paper cards (Azuma, 2001).First, a webcam (connected to the computer) captures images of the real environment in search of cards.These cards have printed symbols that are cataloged in the computer software database.Symbols found on the cards are analyzed and interpreted by a software application.Symbols interpretation produces 3D virtual objects that are combined with the user's actual environment.The mixture of two environments is displayed either on a video monitor or on special glasses or a helmet display.
Possibility of transporting virtual objects to the user's physical space (for overlapping) facilitates interaction, which no longer occurs through a single specific component (conventional devices such as mouse and keyboard), but with the whole environment that surrounds the user.Thus, the virtual objects can be manipulated through actions involving tangible and multimodal touch, gestures and voice, making the process easier and more intuitive without using special equipment.Figure 2 shows the basic components of an augmented reality environment.

Fig. 2. Basic components of an augmented reality environment
Ability to carry virtual objects into the real world has fostered new forms of humancomputer interaction, making the process more natural and intuitive.This feature of augmented reality can facilitate the interaction of individuals with severe motor problems.The handling of virtual objects can occur naturally, dragging or touching an object with fingers (Figure 3.a), hands (Figure 3b) or feet (Figure 3c), without necessarily using peripheral or adapters to interact.
Often, computer adaptations are created to attend specific needs of individuals with disabilities.However, depending on the disability specificity, some individuals may require specific resources hindering the therapeutic process, promoting even higher cost for design and use.Augmented reality allows environments to build economically viable systems compared to other environments that use expensive electronic devices such as helmets and electronic gloves.With a computer (for software processing), a webcam (to capture the real environment) and a video monitor (to display mixed environments) a simple augmented reality system can be created.This system has a variety of potential applications to work with individuals with various disabilities (Richard et al, 2007), (Assis and Lopes, 2008).

Fig. 3. Possibilities of interaction with augmented reality cards
The presence of cards in the webcam view field triggers the overlaying of the virtual object associated with the card and the real world captured by the webcam.The manipulation of the card in the real environment also moves the virtual object.If the individual drags the card, the virtual object associated with it moves along.This augmented reality feature facilitates the process of communication of children with learning difficulties or people with physical disabilities in many different difficulty levels, allowing their intellectual and motor development.For physically disabled with a moderate or severe physical capacity degree, the use of conventional musical instruments is restricted (Louro et al, 2005).Interacting with virtual musical instruments in the real world can facilitate interaction in music therapy especially when the therapy takes place in a specialized center for motor rehabilitation.
The next section presents GenVirtual, indications for its use in music therapy sessions and preliminary results of using GenVirtual with patients with motor disabilities.

GenVirtual musical system
We developed an augmented reality musical system (GenVirtual) that allows creation, improvisation and music reproduction such as composition and tunes playing.Also the system enables the user to hear sounds and music, and musical memory games.
GenVirtual adds virtual musical instruments to the real world.Colored three-dimensional cubes represent the instruments.Cubes are able to simulate the sounds of wind, string and percussion instruments.Interaction occurs through GenVirtual cards with musical symbols.Cards replace the keyboard for music composition and are low cost since they can be printed on plain paper in various colors and sizes (Figure 4).

•
Wind musical instrument (flute); • Percussion musical instruments (plate, box, tambourine and triangle); GenVirtual processes the images captured by a webcam to identify the cards on the table.
The webcam is connected to a computer and strategically positioned on top of the table (Figure 4).Symbols on the cards are detected and associated with a sound.To execute a sound, the user just hides a card with his/her hand, without necessarily touching them with www.intechopen.comhis/her fingers (Figure 5).By detecting the occlusion of a card, the system runs the corresponding sound the card represents.The piano sound is associated to the default sound cards, but there is possibility to change the musical instrument sounds played by the cards.Simply add a new card on the table containing the symbol of the desired musical instrument (Figure 5).Thus, the same virtual elements emit different sounds, tones and may take a wind instrument (flute, trombone, trumpet, etc.).or string instruments (violin, guitar, etc).Another possibility is to use cards with symbols corresponding to the percussion instruments, like drums, tambourines, triangles, and symbols that represent sounds of electronic instruments (Figure 6).

Fig. 6. GenVirtual Musical Instruments
GenVirtual allows change images of musical instruments (virtual black cubes) for 3D model of the instrument.Figure 7 shows an image of the violininstrument.

Fig. 7. 3D Musical Instruments of GenVirtual
The main characteristic of this virtual environment is the interaction flexibility.Unlike the keyboard, consisting of fixed keys and sizes, in GenVirtual, the therapist can print cards in different sizes and position them on the table in various ways, according to the motor limitations of each individual.Cards can be arranged on the table, or on the ground, depending on whether the interaction will occur using hands or feet.According to Nascimento (2006), it is important to have a reference of the motor movement so that motor learning occurs.
GenVirtual also has a feature to assist in recreational actives in music.Composite scores were created by melodies that can be read using colored balls.Colors of musical notes of score correspond to the colors of same musical notes GenVirtual. Figure 8 shows one of musical scores available GenVirtual from the book "Song at the piano" featuring the works of Winnie the Pooh Disney Enterprises Company.
Figure 9 shows musical score being projected on the computer screen.Note tha tthe virtual cubes have the same colors of the colored balls from the score.

Indicators to use GenVirtual in music theraphy
GenVirtual indications beyond the scope of rehabilitative therapy, allow household use of the system.With systematic and repetitive use of GenVirtual, individuals are spontaneously motivated.This can be enhanced with memory cognitive rehabilitation and visual stimuli (colors of virtual objects), as well as audio stimulation (sound of virtual objects).Therefore, motor rehabilitation is exercised with repetition of the sequence of motor movements necessary to strike the sounds.
From the analysis of music therapy approaches, Benezon (1998) establishes the four main methods of music therapy: improvisation, re-creation, composition and listening.All these methods can be applied using GenVirtual.For the clinical cases, the re-creation method was used, where the patient performs, reproduces, transforms or interprets a piece of music or musical model as a whole.The objectives of clinical re-creation, according to Benezon (1998) can be: • Develop sensory motor skills.
• Promote rhythmic behavior and adaptation; • Improve care and guidance; • Develop memory; • Promote the identification and empathy with others; • Develop skills of interpretation and communication of ideas and feelings; • Learn to play specific roles in various interpersonal situations.
We can consider that music therapist intervention with an individual with an extremely hypotonic hand, for example, can be processed by GenVirtual in the following aspects: these individuals are not able to keep their fingers extended on the keyboard, or do not have the enough muscular strength to strike piano keys or strum a guitar (Louro et al, 2005).Typically, in these cases, the patient uses adaptations like tips on the hands to play the piano, tambourine fasteners for bimanual hand use, and the help of a music therapist to carry out interventions of music expression, the "music making" (Nascimento, 2006).
With GenVirtual, the performer can use his/her hands closed into a fist to play a musical note, since the cards can be printed in larger proportions.There is no need to adjust the interaction with the virtual environment.To use GenVirtual to play an instrument, the user needs to cover a card without using adaptations.

Evaluation of GenVirtual in music theraphy rehabilitation
Experiments with GenVirtual were performed in music therapy intervention.Five markers formed by symbols and colors of respective musical notes (C, D, E, F and G). Figure 10 shows the music therapist interacting with GenVirtual.
A video monitor was used to visualize the animation of virtual cubes.The webcam was placed on top of the monitor to capture images of cards on the table.As can be seen in Figure 8, the music therapist obstructed cards with her hands without using adapters.

Fig. 10. Evaluation of GenVirtual by music therapist
After testing, the music therapist noted that besides the possibility of using GenVirtual in music therapy, it can bring benefits to individuals with disabilities, since it is a facilitator and motivational tool in the rehabilitation process.As one example: • Arm extension exercises: the goal is to reach a virtual object.With cards' flexibility, specific challenges can be created for each patient according to their motor limitations.
• Wrist flexion exercises: support the forearm on the table and bend wrist up and down "touching" virtual objects to execute the sounds.

•
Repetitive motor training: a sequence of motor movements facilitated by the musical memory game.
• Visual perception exercises: identifying musical elements from the representation of these elements by colourful virtual objects.
• Auditory perception exercises: identify high and low tones (listen to sound) and identify familiar melodies (listen to music); • Reproduction and development of more sophisticated music pieces, experience the "music making" to amplify their cognitive functions like attention, concentration and memory.

Methodology
This research was approved by the responsible Scientific Review Committee: the Committee for Scientific Affairs of the Association of Assistance Children with Disabilities (AACD) under Protocol 089/10.Several experiments were performed in the AACD music therapy field in order to test the impact of GenVirtual when treating two patients with cerebral palsy and three patients with neuromuscular diseases.
The patients' ages were established between five and six years old, with individuals of both sexes, being 01 (one) girl and 04 (four) boys.Patients underwent a trial test with the specific goal of rehabilitation for upper limb by motivation stimulation of musical function through interaction with music.The process began with the implementation of the User Interaction Satisfaction Questionnaire (QUIZ), repeated at the end of the interaction.The objectives of the experiments were: • Assessing motivation, satisfaction and enthusiasm of patient to interact with GenVirtual.

•
Observe the motor effort motivated by using virtual objects that allow music expression.

•
Observe the ability to concentrate.
• Observe family interaction during the intervention.
During the experiments with patients, we used a computer, a sound box, a webcam with a tripod and sixteen cards: seven cards representing seven musical notes (C, D, E, F, G, A and B), three cards representing instruments (piano, violin and guitar), a marker representing a wind instrument (flute) and four markers representing the percussion instruments (cash, plate, drum and tambourine).The experiments were recorded as illustrated in Figure 11.
Initially, the music therapist prepared a linear design with cards on the table.The goal of linear motor planning for these patients was to stimulate the active pendulum motion to the shoulders, push your arm back and forth so as to touch the virtual objects.
After the demonstration of active music therapist's tool operation, the patient spontaneously was motivated and interacted with the system, making the shoulder and arm extension to reach the cards on the table and thus play the virtual objects and enjoy the unique and individual musical composition.The music therapist encouraged the activity by asking the patient to interact with the cards, to switch the left arm with the right, to exercise and active pendulum motion with left and right shoulders.The activity was held in a few moments as an expression of patient musicality (through motor activity).Also, the music therapist used an electronic keyboard to accompany the patient's musical activity.The music therapist process is a sequence of interventions and not simply a single act or therapeutic maneuver alone.A therapist-patient relationship was created, even for those cases where the therapy period was relatively brief.
The type of therapeutic process involved in this clinical trial, where the therapist used the virtual tool, depended on the type of patient and the nature of his/her problem.According to Benezon (1998), there are different types of proceedings, as follows: developmental, educational, interpersonal, artistic, creative and scientific process which can be applied as such individual processes and are not mutually exclusive, but instead overlap, with greater or less emphasis on the process, as presented in these cases.

Results
According to the results collected in the Quiz and to the patient and family feedback, all patients participated actively in the therapy with the strategies suggested by the therapist showing interest in the activities.The musical cards triggered curiosity and demanded concentration and reasoning during the activity.All patients achieved the proposed objectives for the intervention.
GenVirtual fostered the development of therapeutic activities using virtual reality with the computer such as playing percussion instruments with open hands, for example, triangle plate and without the need of a stick, which cannot be used by children with severe upper limb motor problems.As of the Benezon (1998), music therapy analysis approaches, the four main methods of music therapy: improvisation, re-creation, composition and listening, can be applied through using GenVirtual.For the clinical cases presented, the model re-creation where the patient performs, reproduces, transforms or interprets the piece of music or musical model as a whole was the most commonly used method

Concluding remaks
In recent years there has been a trend in software development for the treatment of various motor and cognitive disorders of people with disabilities.In particular, the use of augmented reality technology has been highlighted and enhanced by having an attractive interface generating greater motivation and consequently greater acceptance and participation in therapeutic treatment.
It is important to involve a multidisciplinary team during the inception of a virtual environment for therapeutic interventions.Thus, it is necessary to analyze the characteristics and abilities of patients considering their limitations.Therefore, a team of engineers conducted this research along with AACD therapists.The team especially considered patients treated in the field of music therapy in order to stimulate their musicality functions (through the motor potential), a cognitive characteristic, motivational aspects and individual characteristics.
Results showed that GenVirtual can be useful to include therapeutic interventions for cognitive learning, motor, psychological and social stimulation through musicality.Social programs have disseminated computers even in low-income families.And since the system is based on a conventional computational platform, the prototype can already be used at home.This may provide for family involvement in complementary activities.Therefore, the sequences of operations with GenVirtual may involve stages of maturation or development, gradual learning, development of personal relationships, the performance, composition and spontaneous child improvisation, exploration, experimentation and alternatives selection, as well as ongoing evaluation of the effects of therapy on patient's progress through treatment targets.
Nevertheless, the virtual environments do not replace conventional medical therapies, but the tool will help empower existing treatments.We consider the positive data collected as a result of the patients' satisfaction evaluations and the family feedback is a preliminary result that needs to be extended to a greater number of individuals ('n') to confirm or not the encountered results.