InTechOpen uses cookies to offer you the best online experience. By continuing to use our site, you agree to our Privacy Policy.

Medicine » Mental and Behavioural Disorders and Diseases of the Nervous System » "Cerebral Palsy - Current Steps", book edited by Mintaze Kerem Gunel, ISBN 978-953-51-2649-2, Print ISBN 978-953-51-2648-5, Published: September 21, 2016 under CC BY 3.0 license. © The Author(s).

Chapter 8

Assistive and Adaptive Technology in Cerebral Palsy

By Alejandro Rafael Garcia Ramirez, Cleiton Eduardo Saturno, Mauro José Conte, Jéferson Fernandes da Silva, Mísia Farhat, Fabiana de Melo Giacomini Garcez Garcez, Ana Carolina Savall and Elaine Carmelita Piucco
DOI: 10.5772/64767

Article top


HAAT model. Source: Adapted with permission from [2].
Figure 1. HAAT model. Source: Adapted with permission from [2].
AAC tool desktop interface.
Figure 2. AAC tool desktop interface.
Precedence relationship between symbols and categories.
Figure 3. Precedence relationship between symbols and categories.
Adapted devices.
Figure 4. Adapted devices.
The adapted stapler was a suitable solution.
Figure 5. The adapted stapler was a suitable solution.
Performance of the symbol prediction feature.
Figure 6. Performance of the symbol prediction feature.
Welcome screen.
Figure 7. Welcome screen.
Students profile.
Figure 8. Students profile.
Categories and symbols.
Figure 9. Categories and symbols.
Symbols and the related categories in the Drag and Drop activity.
Figure 10. Symbols and the related categories in the Drag and Drop activity.
Symbols related to the Foods category.
Figure 11. Symbols related to the Foods category.
Hit the Target activity.
Figure 12. Hit the Target activity.
Working with the Foods category.
Figure 13. Working with the Foods category.
PhraseForm software interface.
Figure 14. PhraseForm software interface.
CP volunteer testing the BCI.
Figure 15. CP volunteer testing the BCI.

Assistive and Adaptive Technology in Cerebral Palsy

Alejandro Rafael Garcia Ramirez1, Cleiton Eduardo Saturno1, Mauro José Conte1, Jéferson Fernandes da Silva1, Mísia Farhat2, Fabiana de Melo Giacomini Garcez Garcez3, Ana Carolina Savall2 and Elaine Carmelita Piucco2
Show details


Children who suffer from cerebral palsy (CP) face specific challenges, which arise due to motor dysfunction and communication disorders. In some cases, communication is only possible through eye movements and blink, as well as, low amplitude movements of the fingers and toes. Augmentative and alternative communication (AAC) strategies can be used to promote communication in these complex cases. This chapter discusses our experience developing AAC computer’s solutions for children with motor and communication disorders. Software and hardware approaches are discussed. This chapter describes solutions developed for desktop computers and mobile devices. These solutions act as complements of therapist’s activities, helping disabled people to communicate, and promoting social inclusion.

Keywords: augmentative and alternative communication, cerebral palsy, human-computer interaction

1. Introduction

People with disabilities, such as people who suffer from cerebral palsy (CP), face several challenges in their daily lives. These individuals face specific problems, which arise due to motor dysfunction and communication disorders. These disorders are commonly related to a non-progressive brain damage in early life. CP is also responsible for senses of sight, hearing, speech and language dysfunctions [1].

In addition, cognitive development and communication problems are associated with CP. Like stated in [2], language is affected by brain injury and, therefore, the lack of communication in earlier stages of life can irreversibly impair intellectual ability.

Augmentative and alternative communication (AAC) solutions emerge as solutions to supplement spoken communication or to replace it completely, helping these individuals [3].

AAC solutions can be classified as low-tech or high-tech [2]. The low-tech solutions involve gestures, hand signals and sign language. It also includes the usage of supplementary materials, such as communication boards based on letters, symbols or pictures. It may also be related to picture books, or textured cards using Braille [4].

On the other hand, high-tech solutions involve software and electronic components for standard computers or mobile devices. Dynamic communication displays are examples of high-tech solutions.

However, despite the amount of available technologies, there is not enough guidance available on how to directly collaborate with disabled children and specialists as partners in the design process of assistive technology [5].

This work presents solutions and the methodological aspect of creating, developing and evaluating assistive technologies. These works are based on user centric-design principles [6].

Bibliographic, documentary and experimental research was conducted to achieve our goals. In addition, human-computer solutions designed for severe physical disabilities and lack of speech were studied. The documentary research aimed to analyse the professionals’ feedback, verifying the progress using the proposed technologies. On the other hand, the experimental research aimed to use the developed human-computer interfaces, collecting quantitative and qualitative assessments guiding our future works.

2. Assistive technology

Disability is a complex phenomenon, reflected because of the interaction between the individuals and the society in which it lives. It is the result of a deterrent and it can be physical, cognitive, mental, sensory, emotional, developmental, or some combination of these limitations [2]. Disability might be present at birth or arise during life.

Disability is closely related to sensory limitations and emerges when some barriers (physical, communication and information) constrain the participation of individuals in society.

The term “assistive technology” is relatively new and it is used to identify resources, technology and services that contribute to provide or enhance functional abilities of people with disabilities. It includes a wide range of equipment, services and strategies aiming to mitigate the problems faced by individuals with disabilities [2].

In [2], the authors explore the role of assistive technologies in the lives of people with disabilities. They define a HAAT model that means Human Activity Assistive Technology model. The HAAT model is based on the interaction of four basic components, namely: the activity, the human factor, the assistive technologies and the context in which this interaction occurs (Figure 1).


Figure 1.

HAAT model. Source: Adapted with permission from [2].

The components of the HAAT model play an important role to understand the methodologies to design assistive technologies. First, a person needs to perform a certain activity, for example, to communicate. In addition, that activity happens in a particular context, for example, in school. For that particular context, and taking into account the activity, may exist an assistive technology that could assist the individual, such as AAC solutions.

The combination between the activity and its context will determine which skills are required to fully realize the activity, so guiding the design of assistive technologies. This model allows understanding the role of assistive technologies, guiding the design process.

3. AAC tool desktop solution

In this section, we describe the solution presented in [7]. The AAC tool solution, as it was named, was based on communication boards and iconographic symbols, commonly found in AAC. It was designed to help the speech therapist intervention. Figure 3 shows the software interface. The images shown in this figure are merely illustrative.

This interface works as follows. First, a user selects a desired symbol. Next, it is vocalized and, after that, added to the upper left side of the software interface, Figure 2. The symbols’ library could be customized for each user.


Figure 2.

AAC tool desktop interface.


Figure 3.

Precedence relationship between symbols and categories.

The software adopts strategies to facilitate the symbols choice. Therefore, symbols commonly used are present first, such as people and greetings. Then, the software suggests other symbols according to the previous ones selected. This feature aims to improve communication speed. Suggestions are based on the previous usage of the tool, and it also depends on the settings performed by the therapist or a caregiver.

It is important to notice that the user could autonomously navigate using the automatic scanning feature, selecting the desired symbols. In addition, the automatic scanning time rate can be settled, according to the users’ skills.

The symbols are based on categories, allowing constructing a logical sequence according to the syntax of the user’s language (i.e. person, action and feelings). A possible sequence of symbols is based on the syntactic Portuguese language, as illustrated in Figure 3.

The software uses the “I + WANT + PLAY” structure, because this grammatical construction is commonly employed by specialists in Brazil. However, some other approaches, for example, based on the verb PLAY + I structure could be used instead.

The tool also features predicting sentences based on graph theory [8]. In addition, it considered important guidelines for human-computer interfaces, adapted from WEB content accessibility guidelines, such as in [7].

This way, pictures have text and oral descriptions. This is very important because CP individuals have difficulties keeping attention on what happens on the computer screen. In addition, the tool lets to resize letters, according to the user’s skills, helping to a better understanding of symbols and texts. Border colours and backgrounds are also configurable, according to the user’s needs. This is also very important to facilitate symbols recognition.

The buttons located at the bottom right of the software interface emulate mouse and keyboard functions. For this reason, new hardware interfaces could be added without the need to install specific drivers.

Other issues could be conceived, like a vocabulary with numbers and arithmetic operators for a math class or a specific vocabulary for a chemical class containing the elements of the periodic table, for example.

3.1. Evaluation

Students from Special Education Foundation of Santa Catarina—FCEE participated in the study. The volunteers who participated suffer from choreoathetosis, which is a nervous disorder characterized by involuntary and uncontrollable movements. They have preserved the intellectual ability and act as minds trapped into the body [7].

The research sought to analyse the student’s performance through a dialogue with and without using the AAC tool. First, the system was presented to the students, enabling them to understand how to use the tool. Then the efficiency and satisfaction using the AAC software were studied.

To evaluate the system, the speech therapist prepared a dialogue, talking about things that are part of the child's routine, such as family, leisure, friends, etc. The speech therapist initiated the dialogue using the low-tech technologies available at FCEE, such as communication boards. In a second stage, the therapist performed the same dialogue but using the software, instead. This procedure was repeated several times, changing the dialogues.

It should be highlighted that the students answered what they want, but it is expected that the answer should be closely related to the one previously given by using the physical board.

Concerning to the hardware resources, students at FCEE commonly use the devices showed in Figure 4. Mouse and keyboard devices are commonly used to interact with computers. Besides the usefulness of such suitable devices, they require considerable effort to be actuated and can cause an earlier fatigue.


Figure 4.

Adapted devices.

Figure 5 shows a stapler device, which was adapted to improve the computer access. The adapted stapler was well suited, because of the shape of the child’s hands. It considerably diminishes the spent time to select the symbols on the screen, also reducing the fatigue. This device emulates the click and double clicks functions of the mouse.


Figure 5.

The adapted stapler was a suitable solution.

Symbols selection time rates and errors committed were computed. Typically, students attempt an average rate of 15 symbols selections per minute, when using low-tech communication boards. When using the system, those rates were worse, even though, on several occasions, the students achieved similar result. In addition, sometimes no coherent phrases were constructed, but it was observed that error rates gradually decrease with the usage of the interface [7].

In addition, an evaluation was carried out to demonstrate the symbol prediction feature, which is based on previous symbols selections. The goal was to build a phrase and to repeat it several times. Then the spent time to construct each phrase was verified. The tests used an automatic scanning rate of 1 s. Figure 6 shows that it is possible to decrease the time required to construct new phrases.


Figure 6.

Performance of the symbol prediction feature.

In addition, speech therapists at FCEE performed qualitative assessments. Interviews evaluate issues related to student’s behaviour and performance. Aspects such as simplicity, software interaction, configurability, images quality, screen navigation resources and students’ evolution were evaluated as good, following the recommendations in [9].

The specialists conclude that it is easy to understand the operation and principles of the tool. In addition, according to [7], “they registered that a more efficient interaction with the software will be directly related to the complexity of the student needs and, according to their opinion, this represents the greatest challenge”.

This software is opened to developers and can be accessed at [Accessed: March 26, 2016].

4. AAC mobile solution

After developing the AAC solution for desktops, we think about enhancing the same idea to tablets, applying the solution at Association of Parents and Friends of Exceptional Children (APAE). This section discusses our experience developing the AAC solution for mobile devices.

The problem faced at APAE is that the students with CP also suffer from severe intellectual disability. Therefore, the software AAC tool, developed for desktops, was useless in that context.

For this reason, we redirect our proposal, guided by the HAAT model. Then, a new tool for mobile devices interaction focusing users having intellectual disabilities was designed. The solution concerned about the presentation and organization of content based on accessibility standards [9].

It is worth noting that the development considered accessibility recommendations, according to the W3C Group, July 9, 2009, in particular, the Mobile Web Best Practices (MWBP) [10].

4.1. Interface

The new software guides the work of professionals at APAE. It was designed as an educational strategy, contributing, booth, as a tool to study the intellectual disability and as an AAC strategy.

The app is not by itself decisive to diagnostic a sort of intellectual disability, but it helps in the professional’s decision. It should be used along with other international validated tools and theoretical references founded at Diagnostic and Statistical Manual of Mental Disorders (DSM) [11] and American Association of Intellectual Disability (AAIDI), for example.

Figure 7 shows the initial screen. In order to login, the professional informs its identification and a password. Selecting the “keep connected” option, the specialist may choose to store the password, so that it will no longer be necessary to re-enter it in future accesses.


Figure 7.

Welcome screen.

When accessing the system, the specialist can create and configure the student profile (Figure 8). It is important to notice that the database involves personal data, so it must be treated with all the necessary integrity and security.


Figure 8.

Students profile.

The main activity conceived for this app is to study the student’s cognitive ability. This is done by selecting and grouping symbols from different categories such as clothing, food, animals, transportation, etc., grouping them according to their relationship within each category.

Before this app, the students had to select, by hand, symbols and context cards randomly spread in a round table. This made the task of selecting them difficult for CP users.

Figure 9 shows the default categories and symbols configured for this app. The professionals at APAE could add new categories and symbols. In this screen, the student's name can be heard through the speakers.


Figure 9.

Categories and symbols.

Actions, foods, animals, toys, communication devices, colours, fruits, liquid foods, furniture, vegetables, clothing and means of transportation are the chosen categories (Figure 9). In Figure 8, a link to the student profile and activities was highlighted in black squares.

The About option, located at the bottom in Figure 9, gives information about the student, the APAE professionals and the institutions who collaborate with this research. It is important to remark that all the symbols, feedbacks, texts and sounds can be configured by the professionals.

Figure 10 shows the main activity conceived for this app, named Drag and Drop. The main purpose of this activity is to test the cognitive ability of the student by selecting symbols from different categories such as clothing, food, animals, etc., grouping them according to their relationship within each category.


Figure 10.

Symbols and the related categories in the Drag and Drop activity.

When performing this activity, the student should drag the symbols from different categories, pictured at the top of the screen, dropping them inside the boxes that appear in the lower part of the screen. To this end, the app uses the touch feature, commonly available in mobile devices.

The professionals can configure both the symbols shown at the top of the screen and the symbols that appear at the bottom of the screen. At the end of this activity, the symbols are properly grouped into their respective categories, or not. This test contributes to evaluate intellectual disability.

Figure 11 shows the symbols used in the Foods category. An auditory feedback can be associated to each symbol.


Figure 11.

Symbols related to the Foods category.

Figure 12 shows an activity specially conceived for CP users. This activity was named Hit the Target and aims to analyse the motor skills of the student.


Figure 12.

Hit the Target activity.

The goal of this activity is to measure the time the user spent to select the letter at the upper left of the screen, hitting the same letter at the lower centre of the screen.

To this end it was used the Touch Listener feature in the Android operating system [12]. The Touch Listener feature allows the programmer to create new functions for each move classification, which are then processed whenever a movement event occurs. Among them are the touch itself, identified by the action ACTION_DOWN code and dragging, identified by ACTION_MOVE code.

These data may subsequently be considered in the Drag and Drop activity in order to allow users with motor disorders use the application with autonomy.

Figure 13 shows other kind of activities that can be done using the app. In this case, a speech therapist can work with isolated symbols to develop speech and auditory cognitive abilities, such as in speech therapy sessions, for example. It should be remarked that new symbols and sounds could be included or configured to attend users in different situations.


Figure 13.

Working with the Foods category.

4.2. Evaluation

Nowadays, the app is being evaluated at APAE, using the system usability scale (SUS) questionnaire [13]. The APAE specialists are registering their perception about the student’s performance using the tool. The SUS will highlight the positives and negatives aspects of the system.

According to [13], (at least) ten statements, being evaluated on a scale from 1 to 5, should compose the survey by establishing a balance between positive and negative assertions. “The Software operation is simple” and “The Software often induce errors” are examples of these assertions.

Other assertions are related to the complexity and, confidence using the system, and other usability issues. This questionnaire is under evaluation at APAE.

The app will indicate the degree of physical and cognitive involvement of each child, computing separately the questionnaires and grouping them according to similar cognitive and motor skills.

Preliminary evaluation shows the following contributions: easy handling application, good images contrast, easy calibration and automatic adjust of the touch screen time; useful for AAC; several language stimulation possibilities, including voice recording; and rich in visual stimuli.

Still, we have to improve images resolution, enhance the resources for speech therapy activities and insert new basic functions of the language.

The AAC mobile software is also opened to developers and can be accessed at [Accessed: March 26, 2016].

5. New study

A new study is being performed using a brain-computer interface (BCI) [14]. The development is based on the Emotiv Epoc [15]. In particular, we are using EEG signals [16]. This new user-computer interaction is being integrated with the two software solutions previously described, emulating mouse and keyboard commands.

At this moment, a software pilot solution was designed to test the computer interaction with CP individuals (Figure 14). Our goal is to find metrics, such as success, errors, time rates, number of phrases construction, satisfaction and others parameters that will be identified when performing the next stages of this research.


Figure 14.

PhraseForm software interface.

The PhraseForm software validates the basic headset functions of the Emotiv Epoc SDK (Software Development Kit). The SDK links users and the Emotiv Epoc, processing the electrical signals coming from the headset. The PhraseForm software emulates mouse and keyboard commands, as delete or line break, for example, also selecting characters on the screen in order to form sentences.

The software looks for the best actions to be captured using the EEG signals. Based on this interface, the BCI technique is being validated by CP individuals.

Figure 14 shows the FraseForm user-application interface. Actions such as blink, show teeth, eyebrow, frowning, laugh, neutral, smile, etc., could be configured to interact with the software, selecting or deleting characters. Speech therapist is guiding the process of select the best actions choice for each student.

Nowadays this study is performed at FCEE. Preliminary results promise the access to computer resources with autonomy by CP individuals. Figure 15 shows a CP volunteer using the system.


Figure 15.

CP volunteer testing the BCI.

6. Conclusions

This work discusses our experience developing AAC solutions for students with motor and communication disorders, which is commonly found in cases of cerebral palsy. The students attend the Special Education Foundation of Santa Catarina—FCEE and Association of Parents and Friends of Exceptional Children (APAE) in Brazil.

Our first experience was developed at FCEE. The AAC tool solution features characteristics that are considered relevant to the design of AAC systems. It considered several recommended guidelines to develop human-computer interfaces, adapted from WEB content accessibility.

Using the software, symbols can be selected from a pre-configured vocabulary, inserting them in a designated area. Phrases constructions are based on symbols and their respective categories. This allows mounting a logical sequence according to the syntax of the user’s language. We adopt a sequence based on syntactic Portuguese language. Iconographic symbols convey needs, wishes, desires and ideas.

The study validates symbol suggestion features, demonstrating the efficiency of this approach for assisting sentences construction. There are also presented qualitative assessments from speech therapists.

The second study was developed at APAE, together with the professionals and students who attend this institution. Educators, speech therapists, psychologists and occupational therapists took part in this new study.

The app features an alternative and augmentative communication tool for children having CP, but not restricted to this public. The solution was designed to assist professionals who act in special education assessing the intellectual disability. The app is based on the MWBP accessibility recommendations.

The AAC mobile supports the assessment of students with suspected disabilities. It encompasses various activities/strategies within just one application. It also explores basic functions of language and its categories such as colours, animals and everyday objects, for example, contributing to the speech therapy.

The evaluation process using the system usability scale (SUS) is still under construction, but preliminary results showed its usefulness to study the intellectual disability, which is also useful as an alternative and augmentative communication tool.

This app cannot be considered, by itself, as the only key to diagnose intellectual disability, because there are internationally validated tools to this end. Instead, it configures a new way to interact with children with CP using the technological advances. Nowadays, we are studying new ways of interaction with computers and mobile devices using eye tracking and electroencephalography.

This research deals with a set of accessibility guidelines that benefit researchers and practitioners, giving more evidence about the design of AAC computer-based solutions for people with limited speech or language skills, who are the centre of these solutions. In this work, the HAAT model guides the design of such assistive resources.

Preliminary results in this field promise alternative ways to access computer resources, promoting autonomy, giving more evidence about the design of AAC computer and hardware-based solutions devoted to people having reduced language skills and motor disorders. It is important to remark that the AAC tools we presented are intended to be used in the school, according to the ethics protocol of this research.


The National Council of Scientific and Technological Development (CNPq) support this research under the 458672 process. In addition, the Foundation for Supporting Research and Innovation in Santa Catarina (FAPESC) supported this research under grant 2015TR300. It is also been covered by the Ethics Committee on Human Research.


1 - S. Levitt. O tratamento da paralisia cerebral e do retardo motor (The treatment of cerebral palsy and motor disorders). 1st ed. São Paulo: Manole; 2011.
2 - A. M. Cook, J. M. Polgar. Cook & Hussey’s, Assistive Technologies: Principles and Practice. 3er ed. USA: Mosby Elsevier; 2013. 496 p.
3 - O. C. Ann, L. B. Theng. Biometrics based assistive communication tool for children with special needs. In: 7th International Conference on Information Technology; 12–13 July; Kuching, Sarawak. 2011. p. 1-6. DOI: 10.1109/CITA.2011.5999527
4 - J. Sigafoos, R. W. Schlosser. Dean Sutherland. Augmentative and Alternative Communication [Internet]. 2010. Available from: [Accessed: March 26, 2016]
5 - A. J. Hornof. Designing with children with severe motor impairments. In: ACM CHI Conference on Human Factors in Computing Systems; April 9th; Boston, USA. ACM Press; 2009. p. 2177-62180.
6 - T. Lowdermilk. User-Centered Design: A Developer’s Guide to Building User-Friendly Applications. 1st ed. USA: O’Reilly Media; 2013. 154 p.
7 - C. E. Saturno, M. Farhat, M. J. Conte, E. C. Piucco, A. R. G. Ramirez. An augmentative and alternative communication tool for children and adolescents with cerebral palsy. Behaviour & Information Technology 2015;34(6):632-645. DOI: 10.1080/0144929X.2015.1019567
8 - K. Ruohonen. Graph Theory. [online]. 2013. Available from: [Accessed: May 01, 2014].
9 - W. Albert, T. Tullis. Measuring the User Experience. 2nd ed. USA: Elsevier Inc; 2013. 300 p. DOI: 10.1016/B978-0-12-415781-1.00013-3
10 - J. Rabin, C. Mccathienevile. Mobile Web Best Practices 1.0.W3C [Internet]. 2008. Available from: http:// Www.W3.Org/Tr/Mobile-Bp/ [Accessed: March 26, 2016]
11 - American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders [Internet]. 2014. Available from: [Accessed: March 26, 2016]
12 - J. R. Lewis. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 1995;7(1):57-58. DOI: 10.1080/10447319509526110
13 - N. Smyth. Android Studio Development Essentials. 2015. eBookFrency.
14 - B. Grainmann, B. Allison, G. Pfurtscheller (eds.), Brain–computer Interfaces: a gentle introduction. Brain–computer Interfaces. The Frontiers Collection, DOI 10.1007/978-3-642-02091-9_1, Springer-Verlag Berlin Heidelberg; 2010.
15 - Emotiv Epoc. [March 26, 2016{/Date Accessed].
16 - M. Poulos. On the use of EEG features towards person identification via neural networks. Medical Informatics and the Internet in Medicine 1999;26(1):35-48.