People with autism can experience limitations in their intellectual functions, in their interpersonal development, in their adaptive behaviour and thinking, and in the language they use (verbal and non verbal, in both semantic and pragmatic aspects). In order to improve their functioning and achieve wellbeing on an emotional and physiological level, they need forms of support that are centred on the person, and based on an analysis of their particular needs.
Just like anybody else, they should have the opportunity to take part in family life, in school, in a job, and in their community in general. Sources of support and opportunities to participate must be offered as early as possible and must continue for their entire livesand be adapted to match the evolution of the individual. Professionals, families and the rest of society must be involved in the process.
Technologies can help autistic people and their social environment particularly in the sphere of education and communication. Thanks to the personal adaptation and use of specific technology in their daily lives, we are observing that people with autism can be helped to relate better to others, to learn, and to feel better in themselves.
The use of mobile devices also offers the possibility of using applications that help the user to communicate in different environments and places. These devices also allow families, educators and professionals to intervene in the educational process. Their portability also enables users to perform learning activities any time, anywhere. In addition, the connectivity between mobile devices is very useful in performing communication and learning activities in a group setting, helping the integration of the user in their social environment.
We are working on two projects that take advantage of these possibilities. The first is the Sc@ut Project (Rodriguez et al., 2009) (Sc@ut, 2011), for designing adapted communicators, which runs on computers, Pocket PCs and Nintendo DS. The second is Picaa (Fernandez et al, 2009) (Picaa, 2011), a platform for creating adaptable and collaborative didactic activities via the iPhone, iPod touch and iPad devices.
In this paper we present the main characteristics of these devices and our experiences of their use, highlighting the benefits observed amongst the population using them.
2. Related works
In the context of interactive environments such as learning and teaching, several tools have been developed, for example:
VTech (Vtech, 2011) has commercialized multiple products that combine entertaining electronic formats and engaging content that help children learn. However, these products are not targeted at children with special needs.
JClic (Click, 2011) is an environment that allows the creation of individual activities, but it only runs on desktop computers.
Hot Potatoes (Hot Potatoes, 2011), a suite to create interactive multiple-choice, short-answer, jumbled-sentence, crossword, matching/ordering and gap-fill exercises for the World Wide Web.
In (Schelhowe, 2009) a mobile application is presented, designed for people with special needs who exhibit a mild to severe level of mental disability. It is focused on fostering the learning process directly within the context of use, with a flexible learning speed and a fixed structure. A server is used to feed clients´ mobiles with learning materials that are adapted to the learner profile. A customization and decision engine is used to fulfil this objective. User profiles are provided by the application but they are not configurable. The teacher does not intervene in their creation and they cannot be modified.
Some learning applications have been developed for the iPhone OS system, such as iWriteWords (iwritewords, 2011). This teaches children handwriting while they play an entertaining game. Meanwhile Proloquo2Go (proloquo, 2011) is a product that provides a communication solution for people who have speaking difficulties. These applications are designed for individual use only and they are not configurable.
None of these systems proposes an adaptive approach that takes into account professional directives in an educational context and user-specific needs, and nor do they provide mobility capabilities together with functionalities for cooperative work.
In the following sections we present two of our products that are being used by autistic people for communication and learning.
3. Communication - the Sc@ut project
Language is the main medium for communication and information. It is an instrument to structure thinking, and to regulate the personality and social behaviour. Language allows us to express what is inside. Lack of communication can impoverish a person as it can make them come across as unfriendly, provoking anomalous behaviours that affect their social relationships and their life in general.
Augmentative and alternative communication systems (AAC) (Mirenda, 2003) help people with the above mentioned problems to communicate, and enable them to be trained in social abilities. When making interventions relating to language, the use of total communication (Schlesinger, 1986) is a good choice. Total communication includes the use of images (pictograms, photos, or drawings), sounds (words or sentences), written language and gestures during the communication process. All of these media provide alternative forms of support to people with cognitive problems, such as autistic people, because they help the individual communicate with others (Schaeffer et al, 1980).
In the next section we describe Sc@ut, the tool that we have developed to provide AAC, with total communication. Its main advantages over other systems are that it is adaptable to the user at interface and content levels, and that it can run in mobile devices.
3.1. Sc@ut - description
Sc@ut consists of two applications: the communicator and the generator of communicators.
A communicator is an electronic device for supporting augmentative and alternative communication. The Sc@ut communicator shows a structure of templates with images, similar to a communication board made with paper. Images can be pictograms, photos or drawings, which represent objects that the user can ask for, actions that can be carried out or things that the user wants to say (feelings or opinions, for instance) (Figure 1). When an image is selected, an associated sound, previously stored, is heard. This sound, which can be a word or a sentence, represents reinforcement for the user and allows people who listen to it to know what the user wants. Moreover, our communicator provides links between templates. Visiting a new template is possible from a previous one, when a specific image is selected in it. This navigation between templates allows the construction of structured sentences, or the classification of actions and elements.
An animated pictogram can be also shown when an image is selected. We have used this property to support total communication, presenting an animated character that performs a gesture, showing the text and sound associated (Figure 2). We have drawn 2,200 animated pictograms (aumentativa, 2011) from the gesture language of Benson-Schaeffer. The vocabulary of this language has been designed to be used by people with cognitive impairment as an alternative language with which to communicate.
As Figure 2 shows, Sc@ut provides a version for the NintendoDS portable game console. The arrival of the Nintendo DS game console in the market was presented as an interesting possibility, offering several advantages over other devices:
Feedback: Two screens (one of them touch) allowing more possibilities of interaction.
Multimedia: Being a gaming device, it offers tremendous multimedia capabilities (sound, video, graphics).
Battery life: Approximately 11 hours.
Robustness: A device intended to be used by children (bumps and scratches on the screen).
Motivation: Commercial games can also be used in addition, or as a reward for the child.
Thanks to the new possibilities that this offers it is possible to introduce new functionality in the communicator, taking full advantage of Nintendo DS features, such as new concepts based on visual and auditory stimulation, in addition, of course, to its work as a communicator. Being a two-screen device, the top screen can be used to provide feedback. Thus animations can be shown without losing the context of the action, presenting the cause and the effect of the action together. This helps to identify the sound and image with the concept, and therefore promotes learning.
The second tool provided by Sc@ut is the generator. It is used by educators and families to create and modify communicators, which are adapted to the needs, capabilities, interests, abilities and evolution of users. The generator allows the user profile and the templates for each user to be specified (Figure 3). For example, it offers the possibility of indicating the order in which the images of a template can be selected, and the time during which an image is shown, avoiding the selection of another one. In addition, it allows an agenda and a timetable of actions for each day of the week to be designed, thus helping users to know what can they do and to organize their time.
3.2. Sc@ut - experience of use
We have been working on this project since 2006. Sc@ut has been used under controlled conditions by approximately 100 handicapped people, mainly with autism, from the South of Spain (Andalusia), at educational centres and schools, but also at occupational units for adults. The duration of the interventions varied from six months to one year, depending on the population and teachers.
The overall objective of our studies had been to show the benefits of using Sc@ut amongst people with severe problems of language and communication and of differing ages, for which we propose the following study hypotheses:
H1: The use of Sc@ut by autistic adults and school-age students improves communication, language development and learning.
H2: Improving communication leads to the reduction of disruptive and challenging behaviour.
H3: Sc@ut supports total communication, helping teachers to adapt and customize the content and structure of the communicators.
In addition, we also propose the following objectives:
To give support to teachers and families who use these tools at any time.
To collect user feedback and improve the tool and its usability to make it more flexible and adaptable to the difficulties that people with alternative communication needs may have.
To promote the University's commitment to society.
The intervention process was performed in the following stages:
Definition of hypotheses to be validated in the project.
Presentation of the project to address the participating institutions and professionals.
Training of professionals involved in the systems of augmentative and alternative communication, the use of ICTs, in particular the Sc@ut program. Providing them information on the process to be followed in the study and the materials to be used.
Evaluation and selection of candidate students participating in the intervention. In selecting students we consider their communication needs and their scope for interaction according to their cognitive and motor limitations. Their experience with alternative augmentative communication systems is also taken into account. In addition, we identify the communicative behaviours of users that can be improved with communication support.
Implementation – intervention with Sc@ut as a communication system:
Performing a pre-test amongst all participants.
For each student, choosing the device to use, from among all those available, according to which one will be best suited to his/her needs.
Selecting pictograms, photographs, sounds, videos, scenarios and animations of sign language gestures.
For each student, designing his/her personal communicator: the user profile for the presentation of content and interaction, templates, and the agenda.
Use of the communicator by the student two or three times weekly for approximately forty-five minutes each time. The professionals can make adjustments and modifications to the communicator during the intervention to adapt to the progress and responses from the student.
Involving tutors and family to generalize what they learned in other settings.
The development team visits the users to interview the professionals and families, and observe the use of programs.
Evaluation of results from the pre-test, undertaking the post-test, distributing questionnaires to gather the opinions of the professionals concerned, and video recording in some cases. This allows the validity of the hypotheses to be checked.
Modification of software to fix bugs detected and incorporate participants´ suggestions for improvement and feedback from questionnaires and observations made during the visits of the development team for monitoring.
Creation of a website providing free download of the software. Dissemination via conferences, training courses and journals.
3.3. Sc@ut benefits
All of the autistic people that participated in the study had specific communicative limitations. At best, they only spoke single words, which could only be understood by their parents and close relatives. Not all had this form of expression, making it very difficult to understand them. Some of them used gestures (three or four at most), while others simply looked to the interlocutor in the hope they would guess their needs. Several students presented disruptive behaviour, for instance they appeared to be in despair because they felt misunderstood, and this led them to cry, scream, throw a tantrum and self-harm or, in extreme cases, assault their classmates or adults responsible for their care. They commonly presented a lack of communication, and visible isolation.
We performed the statistical studies in real situations with a degree of quasi-experimental control as we left many variables (staff, facilities, schedules, illnesses, absenteeism, etc.) without control. For the purpose of testing assumptions, we used nonparametric tests, in particular, the Mann-Whitney U test and the Wald-Wolfowitz test. The number of individuals involved in a first study, only for autistic students, was 22. The second study involves 67 students (80% with autism). We used observation questionnaires as the Exam of the existing abilities of communication, the Communication interview and a Scale of evaluation of the communicative pattern in deficient subjects. Observing the values obtained by the tests in the control and experimental groups, in both experiments, we conclude that the degree of significance is high. It implies that intervention is effective, concluding that Sc@ut helps students in several of the aspects suggested by Baumgart (Baumgart, 1990):
Decreased disruptive behaviours: Some students worked specifically to display communication options to prevent such conduct, and others exhibited less disruptive behaviour simply because of feeling better understood and increased personal satisfaction.
Improved oral language through auditory and visual reinforcement and systematic work: For those individuals with opportunities to develop oral language, Sc@ut represents a useful speech tool when phrases or words constructed with the communicator are repeated. Some individuals even learn to anticipate particular components, after confirming that the device tells them what they already know. The verbal label associated with the concepts is useful for helping students to read, being a tool of learning and reinforcement.
Helps solve communication issues: The student can access a wide range of templates to be used in different scenarios to express needs, feelings and ideas. In some individuals, the templates have been used to train in social skills and respond to daily situations.
Reduced level of anxiety, which affects daily living and learning. In particular, the improved communication calms the user, and their level of anxiety caused by misunderstanding of the social environment is brought down to more acceptable levels.
Increasing the impulse to communicate: Once they feel confident that they will be understood, some individuals begin to take the lead on communication and participation, which was unthinkable before the intervention.
Learning and consolidation of gestural vocabulary, increasing the use of gestures for communication: This implies that the student has internalized the gesture as a means of communication, making it independent of any external communication device and promoting spontaneity. We think the use of characters with these signs is more stimulating and encourages users to imitate them. Despite the use of sign language, users must continue with the device to communicate with people who do not understand his/her gestures.
Improved communication between peers and teachers, thus promoting integration and standardization in their social environment.
We have also proved that Sc@ut helps teachers and family, concluding the following:
The generator helps the professional to make a personalized intervention, allowing them to adapt templates, the number and size of pictograms, photographs and images and sounds, animations, etc. It also facilitates the modification of the communicators when the student progresses.
Improved coordination between professionals and family. In many cases, this improvement is thanks to the interest shown by the student, which led to his or her speech therapist communicating with their tutors and parents to share the experience with them.
The familiarity of the supporting devices, and the low cost and free devices of the computer program have also encouraged some families to use Sc@ut in their homes.
Professionals from occupational centres and day-stay units are grateful for the initial training that was given in augmentative and alternative communication and the use of ICTs for education. In general, they are specialists in specific fields (such as ceramics, graphic arts, textiles, etc.) but not in AAC.
With regard to interventions using pictograms with signs, the professionals value their utility in teaching the total communication model to the students.
Table 1 summarises the contributions of Sc@ut.
Sc@ut has been developed for the Linux platform, thanks to a project funded by the Consorcio Fernando de los Ríos, Spain. The usability of the generator tool has also been improved.
4. Learning - Picaa
Learning to read, write, make calculations and solve problems is the basis for living an autonomous life and for being integrated both at work and in daily life.
Education of people with cognitive impairments, including those with autism, must begin as early as possible. People with autism require ongoing intervention that has to be systematic and based on personal motivation. The person has to work on each task in accordance with their skills, bearing in mind that their personal work rhythm and learning style may differ from those of others. Therefore, individualization of the learning process is necessary. However, at the same time, the students must learn to interact with others. Cooperative learning in a classroom setting prepares them to perform daily activities and to integrate in their social environment.
Cooperative learning (Johnson et al, 2000) is a pedagogical method that applies some of the ideas from social constructivism, a psycho-pedagogical paradigm based on the theories of Vigotsky and Piaget. Cooperative learning requires students to work together on a common task. It promotes the personal growth processes, developing the potential of each individual to learn by himself or with others in different situations. Therefore, it is very useful for students with cognitive impairments. The teacher intervenes to create activities and to assign group learning tasks, manages time and resources, and monitors whether the students and groups are working well (Smith, 1996). The main contributions that the cooperative relationship can offer the students with autism are (Barkley, 2005): teachers and classmates can act as role models to imitate; opportunities to do, to say and to feel; team and personal auto-regulation; observing different perspectives from a same subject; constant positive reinforcement; and development of cognitive, social and affective abilities.
The use of technology allows the learning content and user interaction to be adapted, helping the teachers to design personalized learning activities that can be carried out in a group setting. The tactile interaction on mobile devices, which includes learning activities with multimedia content, stimulates the students, gains their attention and reinforces their learning. Picaa, a learning mobile platform for teachers and students which provides an adaptive and cooperative learning tool, works to this particular objective.
4.1. Picaa - description
Picaa integrates, in a single application running on a mobile device, features that allows children and educators to interact with different elements, according to the actions they will carry out and the educational approach.
Its main objective is to facilitate the user’s curricular adaptation by the educator. Learning activities and content must be sufficiently flexible to adapt to the characteristics of each user and integrate the personal data of their inner world, respecting their own particular work rhythm. Activities must be also modified by educators according to user progress or changes in his or her environment. The content, structure and user interface of the applications can be adapted depending on such requirements in order to achieve accessible applications. This is very useful because there are many differences between the interests and learning needs of autistic children, therefore the personalization of learning activities is essential in class.
The Picaa platform allows four kinds of activities to be defined that cover the basic learning tasks. The kinds of activities designed are:
Exploration: sequencing of screens to navigate across a hypermedia system. This allows simple communicators, agendas or stories to be created. Images, sounds and animations are presented. (Figure 4).
Association: assignation of elements to sets. This provides a basis for memory exercises, calculation, discrimination and categorization. (Figure 5.a).
Puzzle: includes pieces with different shapes and numbers. (Figure 5.b).
Sort: consists of items that the student must put in the correct sequence. This activity can be performed with graphics or text. (Figure 5.c).
These activities are aimed at developing the following skills and abilities:
Perception and visual and auditory discrimination.
Acquisition and expansion of vocabulary and understanding of meaning.
Development of memory.
Improvement of phonetics, syntax and pragmatics of language.
Working hand-eye coordination (fine motor).
Examining assumptions, conclusions and interpretations.
Learning cause and effect.
Generalization of concepts.
Managing notions of space/time.
Development of attitudes such as responsibility, cooperation and collaboration.
Development of skills for handling numbers and their applications.
Learning strategies for soling simple problems.
The application is available for three types of devices: the iPhone smartphone, the iPod touch device and the iPad tablet (Figure 6).
Picaa, as with Sc@ut, differentiates between two kinds of use. As we have seen, the students can perform activities, but, the educators can also use it to define user profiles, design the activities and plan a daily agenda (Figure 7). The educator can configure aspects of the activities relative to their content and presentation, for instance: number of components or concepts to be taught, screen composition, screen position (rotation or not), multimedia used to represent the components, difficulty level (goals of the exercise, working out the punctuation), reinforcements and help to the users. Figure 7 shows how to design one of the exercises and the personal agenda.
Additionally, the educator can configure some of the rules to be considered when working in a group context. For instance, the order to be followed when selecting components and the information that the users can see about the interaction performed by other users when they work in a group (number of participants, orderly turns, punctuation, etc.).
4.2. Picaa - experience of use
We began to develop Picaa in 2009. It is available for free, in both English and Spanish, at the online store App Store where it has been downloaded by over 4,000 users worldwide.
To include the educators and students in the development and validation process, we performed a pilot project during a school term. The overall objective of Picaa was to serve as a support tool for people with learning disabilities and poor communication, for which we propose the following hypotheses:
H1. The use of Picaa in school improves communication, and facilitates learning and the development of language.
H2. The integration of the tool on a completely mobile platform system favours adaptation by professionals, families and students.
Some 13 autistic children with ages ranging from 4 to 12 and their teachers participated in the pilot project at several schools from the Andalusia and Murcia regions of Spain. Picaa was mainly used for the following purposes:
Communicative use of exploration activities, as an AAC tool.
Planning and anticipating circumstances through images based on what is going to happen.
Teaching specific topics from curricular areas by designing specific activities.
Working on reading and language comprehension.
Bits of Intelligence.
The intervention process was performed following the same stages as applied in the Sc@ut pilot.
4.3. Picaa benefits
The participation of students and educators in the development process has been very useful because the resulting feedback and suggestions have allowed improvements to be made to the user interface, the tool to be more accessible and usable, and the functionality of the program to be more complete and versatile. Table 2 summarizes the main benefits.
The initial evaluation of teachers is very positive and highlights benefits both for themselves and for their students. We now summarize the main views expressed in the questionnaires that have permitted us to validate the initial hypotheses:
They consider the tool to be simple and easy to use, highlighting the key advantage as being the capacity to both work on the device and make changes more quickly. Thanks to its design and tactile interaction it is intuitive even for people with little knowledge of IT.
The multimedia resources used in activities are integrated into the device itself. Using them is easy because the use of electronic resources means that no scissors, glue, paper, photocopying, recording, etc are required, and that the tool can be reused.
Students have the opportunity of performing activities that were not previously accessible to them, because both the interface and the content can be adapted.
The students can perform the tasks anywhere, thanks to the mobility of the system. This allows their families to intervene in the educative process and facilitates their communication with the school environment.
The use of electronic devices with multimedia content becomes more stimulating and attractive to students than traditional resources and in some cases may even be the only alternative for students with sensory limitations.
Mobile devices can be used to provide customized and integrated support for people with autism. We have designed two applications: Sc@ut for AAC and Picaa for Learning. Both of these can be configured by educators to enable users to follow their own work rhythm, and also integrate with peers, educators and family.
The transfer of our experiences of these applications is easy considering that the software is free and available via the Internet. The devices are inexpensive, readily available and in many cases already present in homes and schools. Our recommendations for transfer are to follow a systematic approach to intervention and to promote widespread use by education professionals and families.
Regarding to the creation of software for people with special needs, we recommend a user-centred design, based on the participation of end users in the specification, design and evaluation of prototypes. In this sense, pilot studies are essential to validate the benefits, detect errors and draw on new suggestions from users to improve the programs and deliver quality products.
As a working group, our assessment is positive and also highlighted the importance of the involvement of the University in applied projects, which may benefit disadvantaged sectors of society.