An Expert System to Support the Design of Human-Computer Interfaces

The concept of human-computer interfaces (HCI) has been undergoing changes over the years. Currently the demand is for user interfaces for ubiquitous computing. In this context, one of the basic requirements is the development of interfaces with high usability that meet different modes of interaction depending on users, environments and tasks to be performed. In this context it was developed an expert system (GuideExpert) to help design humancomputer interfaces. The expert system embeds HCI design knowledge of several authors in this field. As the quantities of recommendations are huge, GuideExpert allows searching the guidelines in a much more friendly and fast manner. It also allows eliciting a series of guidelines for evaluating already implemented interfaces. GuideExpert was evaluated in three Brazilian universities. Due to professors and students engagement, it was possible to correct issues found, both in the implementation and in the guidelines, and to identify the need to develop a more detailed process of HCI requirements elicitation in order for the expert system results become more accurate. The expert system was also used in the development of intelligent adaptive interfaces for a data mining tool, aiming to provide friendly and appropriate user interfaces to the person using the tool. To meet this goal, the interfaces are able to evaluate and change their decisions at runtime. In this context some models of interaction are modeled in order to fit the profile of those who use them. One of them (for novice users) is finalized and is presented in this chapter; the other two are under development.


•
Multimodal Interfaces -these are able to provide lots of "interaction modalities" as well as voice, gestures, and handwriting and synchronize them with multimedia output (Oviatt & Cohen, 2000).These modes are mapped to sensory signals captured by different brain areas.It represents a new perspective enhancing users' productivity and grant greater expressiveness.
• Intelligent user interfaces -these are able to adapt themselves to different users and usage situations.They may also learn with user by providing help and explanations (Ehlert, 2003).According to Ehlert, Intelligent User Interfaces (IUI) use any type of smart technology to achieve the man-machine dialogue.A common feature on both sides is the ability of adaptability.Concerning multi-modal interfaces, it is desirable to be able to move from one form of interaction to another more appropriate if we consider who is using it.By means of an IUI we can improve interface performance and provide more "smartness" while tasks are delegated and the search of solutions is allowed.Adaptability and problem solving are hot topics researched by Artificial Intelligence (Russel & Norvig, 2003), so it is important to incorporate these techniques within this area.

www.intechopen.com
An Expert System to Support the Design of Human-Computer Interfaces 95

Multi-modal interfaces
A multi-modal interactive system is a system that relies on the use of multiple human communication channels.Each different channel for the user is referred to as a modality of interaction.Not all systems are multi-modal, however.Genuine multi-modal systems rely to a greater extent on simultaneous use of multiple communication channels for both input and output (Dix et. al, 1998).Currently, since there is great user diversity, it is rather important to provide different ways of interacting with the machine.A user who has color-blindness, for example, may consider voice interaction something more exciting.In a crowded place the same user may prefer pen interaction instead.Multi-modal interfaces provide different input options and enhance the interaction whether they are used together.Since our daily interaction with the world around us is multi-modal, interaction channels that use more than one sensory channel also provide a richer interactive experience.The use of multiple sensory channels increases the bandwidth of the interaction between human and computer and also makes the interaction look more like a natural human-human interaction (Dix et. al., 1998).We may quote some multimodal applications from systems based on virtual reality to automotive embedded ones.In the 80's there was "Put That There" from Bolt (1980).The work described involves the user commanding simple shapes over a large-screen graphics display surface.Because voice can be augmented with simultaneous pointing, the free usage of pronouns becomes possible, with a corresponding gain in naturalness and economy of expression.Conversely, gesture aided by voice gains precision in its power to reference (Bolt, 1980).Presented by (Cohen et al, 1998), QuickSet was one multi-modal application whose main characteristic was to provide interaction with distributed systems.It used to occur by means of voice or gestures recognition.Image and voice processing were made by software agents used in its architecture (Cohen et al., 1998).Its usage did not stuck to only one field in particular since it was used to perform different tasks as well as military activities simulation and the search for medical information.Concerning the second case, in order to obtain information related to doctors' offices in certain location the user would have to draw the desired area in the map and then the application would retrieve it.Another medical system involving different ways of interaction is the Field Medic Information developed by NCR and Trauma Care Information Management System Consortium (Holzman, 1999).This solution involved electronic patient records that could be updated through spoken responses for synthesized speech.To ensure rapid and accurate interpretation of spoken inputs, the system incorporated a grammar and a restricted vocabulary spontaneously used by doctors to describe medical incidents and patient records (Holzman, 2001).This information is then electronically sent to the hospital for patient arrival preparation.Hardware used for the Field Medic system consists of a small wearable computer and attached headset with microphone and earphones called the Field Medic Assistant (FMA), and a handheld tablet computer called the Field Medic Coordinator (FMC).An example of such flexibility is evident in the Field Medic system as it allows a doctor to alternate between using voice, pen, or both as necessary.This provides the doctor with a hands-free interface whilst he or she cares for the patient and the ability to later switch to a pen and tablet based interface for recording more detailed information at a later time (Robbins, 2004).
In this area of multi-modal interfaces we can highlight systems that incorporate "intelligence" in addition to various modes of interaction.In this class of systems we can cite the following systems: CUBRICON, XTRA, and AIMI.The CUBRICON project (Neal & Shapiro, 1991) developed an intelligent multi-modal interface between a human user and an air mission planning system.The computer displays, which comprised the environment shared between the user and the agent, consisted of one screen containing various windows showing maps, and one screen containing textual forms.User input was in the form of typed text, speech, and one mouse button for pointing.In the CUBRICON architecture, natural language input is acquired via speech recognition and keyboard input.Location coordinates are specified via a conventional mouse pointing device.An input coordinator processes these multiple input streams and combines them into a single stream which is passed on to the multimedia parser and interpreter.Building upon information from the system's knowledge sources, the parser interprets the compound stream and passed the result on to the executor/communicator.The CUBRICON system's knowledge sources are comprised of: Lexicon, Grammar, Discourse Model that dynamically maintains knowledge pertinent to the current dialog, User Model that aids in interpretation based on user goals and Knowledge Base which contains information related to the task space (Robbins, 2004).XTRA (eXpert TRAnslator) is an intelligent interface that combines natural language, graphics, and pointing (Wahlster, 1991).According to the author, XTRA is viewed as an intelligent agent, namely a translator that acts as an intermediary between the user and the expert system.XTRA's task is to translate from the high-bandwidth communication with the user into the narrow input/output channel of the interfaces provided by most of the current expert systems.XTRA provides natural language access to an expert system, which assists the user in filling out a tax form.During the dialog, the relevant page of the tax form is displayed on one window of the screen, so that the user can refer to regions of the form by tactile gestures.The TACTILUS subcomponent of XTRA system uses various other knowledge sources of XTRA (e.g., the semantics of the accompanying verbal description, case frame information, the dialog memory) for the disambiguation of the pointing gesture (Wahlster, 1991).The XTRA system is a multi-modal interface system which accepts and generates NL with accompanying point gestures for input and output, respectively.In contrast to the XTRA system, however, CUBRICON supports a greater number of different types of pointing gestures and does not restrict the user to pointing at form slots alone, but enables the user to point at a variety of objects such as windows, table entries, icons on maps, and geometric points.In added contrast to XTRA, CUBRICON provides for multiple point gestures per NL phrase and multiple point-accompanied phrases per sentence during both user input and system-generated output.CUBRICON also includes graphic gestures (i.e., certain types of simple drawing) as part of its multi-modal language, in addition to pointing gestures.Furthermore, CUBRICON addresses the problem of coordinating NL (speech) and graphic gestures during both input and output (Neal & Shapiro, 1991).AIMI (An Intelligent Multimedia Interface) is aimed to help the user to devise cargo transportation schedules and routes.To fulfil this task the user is provided with maps, tables, charts and text, which are sensitive to further interaction through pointing gestures and other modalities.AIMI uses non-speech audio to convey the speed and duration of processes which are not visible to the user (Burger & Marshall, 1998).The AIMI system utilized design rules which preferred cartographic displays to flat lists to text based on the semantic nature of the query and response.Considerations of query and response included the dimensionality of the answer, if it contained qualitative vs. quantitative information, if it contained cartographic information.For example, a natural language query about airbuses might result in the design of a cartographic presentation, one about planes that have certain qualitative characteristics, a list of ones that have certain quantitative characteristics, a bar chart.AIMI has a focus space segmented by the intentional structure of the discourse (i.e., a model of the domain tasks to be completed).

Intelligent user interfaces
Intelligent user interfaces (IUIs) is a subfield of Human-Computer Interaction.The goal of intelligent user interfaces is to improve human-computer interaction by using smart and new technology.This interaction is not limited to a computer (although we will focus on computers in this chapter), but can also be applied to improve the interface of other computerized machines, for example the television, refrigerator, or mobile phone (Ehlert, 2003).The IUI tries to determine the needs of an individual user and attempts to maximize the efficiency of the communication with the user to create personalized systems, providing help on using new and complex programs, taking over tasks from the user and reduce the information overflow associated with finding information in large databases or complex systems.By filtering out irrelevant information, the interface can reduce the cognitive load on the user.In addition, the IUI can propose new and useful information sources not known to the user (Ehlert, 2003).Intelligent interfaces should assist in tasks, be context sensitive, adapt appropriately (when, where, how) and may:

•
Analyze imprecise, ambiguous, and/or partial multimedia/modal input; • Generate (design, realize) coordinated, cohesive, and coherent multimedia/modal presentations; • Manage the interaction (e.g., training, error recovery, task completion, tailoring interaction styles) by representing, reasoning, and exploiting models of the domain, task, user, media/mode, and context (discourse, environment).As an example of a system that has intelligent interfaces we can cite Integrated Interfaces Systems (Arens et. al., 1998).It uses natural language, graphics, menus, and forms.The system can create maps containing icons with string tags and natural language descriptions attached to them.It can further combine such maps with forms and tables presenting additional related information.In addition, the system is capable of dynamically creating menus for choosing among alternative actions, and more complicated forms for specifying desired information.Information to be displayed can be recognized and classified, and display creation can then be performed based on the categories to which information to be presented belongs.Decisions can be made based on given rules.This approach to developing and operating a user interface allows the interfaces to be more quickly created and more easily modified.The system has rules that enable the creation of different types of integrated multi-modal output displays based on the Navy's current manual practices.The rules for presentation enable the system to generate on demand displays appropriate for given needs.The systems is able to present retrieved information using a combination of output modes -natural language text, maps, tables, menus, and forms.It can also handle input through several modes -menus, forms, and pointing.

www.intechopen.com
Expert Systems for Human, Materials and Automation

98
Both the use of multi-modal interfaces such as intelligent interfaces has shown its wide applicability in various systems.In the following sections, we will present an expert system (GuideExpert) that was used to specify an intelligent interface for a data mining tool.

GuideExpert: An expert system to support the design of human-computer interfaces
T h e i n t e r f a c e s h a v e b e c o m e e a s i e r t o l e a r n a n d d i f f i c u l t t o s p e c i f y .A s a r e s u l t , disagreements related to the implementation of the user interface interaction component become common and are taken to the final stages of development, resulting in a drop in product quality and increase in user dissatisfaction with the system.Research involving human-computer interfaces makes several recommendations for the predesign, design and post-design for the development of a well designed interface (Nielsen, 1993).In the design phase it is of fundamental importance to implement guidelines for interface design, which are, according to (Nielsen, 1993), recommendations for interface design used in heuristic evaluations during the development of an interface.A heuristic evaluation of a HCI is a group of people observing and analyzing the interface in order to identify usability problems and verify the implementation of guidelines in order to solve them.(Shneiderman, 2009) places the guidelines as one of the pillars supporting a successful HCI design, along with usability testing, design tools and good requirements gathering.There are extensive collections dedicated to elicit and propose guidelines for interface design.Two of these collections were put together by (Brown, 1988), with a total of three hundred and two guidelines, and by (Mayhew, 1992), with a total of two hundred eighty eight guidelines.Having too much guidelines to evaluate and apply, one can easily conclude that working with guidelines is not trivial.Working with such a large number of recommendations is the biggest problem faced by the HCI designers.With the aim of helping HCI designers to handle all this knowledge, our team built an expert system to support designers in making decisions related to HCI development.It was designed to suggest and propose guidelines for interface design, as well as perform heuristic evaluations.Three hundred and twenty six guidelines were cataloged, organized and used to build the expert system knowledge base.This work was based on (Nielsen, 1993), (Brown, 1988), (Schneiderman, 1998), (Galitz, 2002), (Cybis et al., 2007).The GuideExpert, as seen in Fig. 1, is comprised of: user interface, the expert system (inference engine and working memory), and the information repositories (knowledge base and database).When the system starts, the expert system module (4) accesses the knowledge base contained in Layer 3 to load knowledge rules and build its working memory.The user interface layer gathers some information with the designer through modules (1) to (3).Gathered information is analyzed by the expert system in order to select appropriate metaguidelines.Finally, as result of this analysis, the system accesses the database at Layer 3 to retrieve guidelines according to meta-guidelines previously selected.The user interface performs three types of analysis with the designer: 1. Users role description -it aims at identifying majority characteristics in the user community such as computer experience (Netto, 2004), personal characteristics (Shneiderman, 2009), domain knowledge (Netto, 2004) and features gathered at requirements phase.The questions the designer has to answer are shown in Fig. 2. 2. Task description -it aims to identify what are the tasks performed by each user role that will interact with the system.For each task are asked what kind of information (alphanumeric, numeric or text) is contained in the HCI, in addition to the graphical interface elements used in its composition.An example of an elicitation screen is given in Fig. 3. 3. User environment description -it verifies the existence of an internationalized system, having extensive documentation and the level of experience of the HCI designer.The question the designer has to answer is shown in Fig. 4.   The expert system inference engine uses the forward chaining strategy to analyze knowledge rules.Through this strategy, the antecedent part of a rule is analyzed and then, in case of a rule that matches the described situation, the consequent part is executed.
To allow the search and selection for the guidelines that best fit a particular design we've established a taxonomy by grouping guidelines according to the characteristics and o b j e c t i v e s t h e y h a v e i n c o m m o n .T h e s e groups are called meta-guidelines.Their nomenclature was defined by the common goal to which each guideline group had.For example, some guidelines suggested how to provide elements for the protection of user data.So, the meta-guideline generated by these guidelines was named "data protection".The grouping of the guidelines resulted in a total of twenty-eight distinct meta-guidelines that can be further expanded in the future.This taxonomy is new in the literature.
To search within this taxonomy, the expert system gathers the user interface requirements list, focusing on descriptions of the role that users have and the tasks they perform, rather than focusing on general aspects of the HCI.This new elicitation does not consider the usability of the system as a whole.It considers task-specific usability.Thus, beginner, intermediate or even experienced IT users need not be faced with considerations that are not suited to their profiles.
The expert system identifies profiles of cognitive styles of the HCI users based on some recommendations found in the literature, mainly by (Shneiderman, 2009) and(Cybis et al., 2007), in order to meet usage expectations in a satisfactory manner.(Cybis et al., 2007), describes general recommendations for three types of user personality profiles.Authors such as Norman Warren cited in (Gleitman et al., 2007), Eysenck cited in (Peck &Whitlow, 1975), andHans Eysenck andSybil Eysenck cited in (Myers, 1999) are being studied in order to determine other personality profiles and user guidelines to elicit interface requirements.
In our ongoing research, we intend to perform experiments that help develop better guidelines, such as the one mentioned by (Shneiderman, 2009): "For extroverts and introverts users, it can be said that the first prefer external stimuli and variety on actions, while the introverts are characterized by cling to familiar patterns and own ideas." The system output is composed by a set of guidelines presented to the designer.It allows the designer to perform heuristic evaluations or to design a new HCI.A set of guidelines is suitable for design inspiration, as a checklist in heuristic evaluation or can serve as a reference for answering specific design questions.Fig. 5 shows an example of some guidelines selected by the expert system.Fig. 5.Some guidelines selected by the expert system.
Besides suggesting guidelines for an HCI under construction as previously described, the system can also be used as a means of providing guidelines for an expert review.In this context, it was developed a module that provides on-demand guidelines to the designer.Through a single interface, the designer selects items or aspects of HCI to be evaluated, as shown in Fig. 6, and GuideExpert selects the corresponding guidelines.GuideExpert was used in the development of an intelligent interface for the KIRA tool.It will be presented in the next sections.

Data Mining teaching tool
Over the years, information amount stored in companies' databases has been growing exponentially.Besides traditional usage, it is possible to extract knowledge from what is stored by means of a process called Data Mining.This knowledge may be used with a wide range of possibilities, which makes the interested to find it the responsible to decide what to do.There are several tools to automate Data Mining and maximize its results; however, they need the user to know the entire process, along with its techniques (Mendes & Vieira, 2009).
In this context, Kira tool (Mendes & Vieira, 2009) has been built.Its purpose is to teach user all the knowledge involved with Data Mining while results are showed.
According to (Mendes & Vieira, 2009) and to (Cazzolato & Vieira, 2009), Kira is efficient in fulfilling its proposed goal; however, its user interface has been built without considering usability, something that positively contributes with increasing user satisfaction regarding a product.
Regarding the current user interface, despite focusing on aiding Data Mining learning, its usability has not been evaluated during the development.In order to verify its effectiveness, user evaluations have been performed to obtain feedback from those who have used it.
The capture of post-use feedback occurred by means of an adapted version of PSSUQ (Post-Study System Usability Questionnaire) (Lewis, 1993).The original questionnaire remained the same in its essence, with few modifications added in order to better understand participants and their opinions regarding the occurred interaction.
In order to accomplish evaluations it was necessary to build usage scenarios.These scenarios refer to ordered descriptions of actions performed by application users.Concerning Kira, a scenario of Data Mining as a whole has been developed with the help of staff working on the area.
The usage scenario has been performed by a mixed public: they all had high levels of expertise with computers; however, their domain experiences were very different.There were those who had not kept contact with Kira and Data Mining, those who had already kept contact with Data Mining but not with Kira, and those who had already kept contact with both, tool and domain.
Those who knew both tool and domain were able to perform the usage scenario without major problems and their interaction time was much lower than the others.The public that knew Data Mining but not Kira was also able to perform the usage scenario without problems; however, their interaction time was higher than those previously described.One of the criticisms had to do with user interface navigation which seemed to be sometimes confusing and not free of errors.
For those who had not kept contact with Kira and Data Mining we can say their interaction time was the highest.Although they had not had domain knowledge, some general concepts were well-known, such as data source, and did not have to be relearned.Their main criticisms related to information excess in interfaces and the lack of information regarding some concepts or even tasks involved.

An adaptive interface for data mining
Once identified problems with Kira current user interface, an adaptive interface was proposed.
The construction of an intelligent user interface is not something trivial, even ad hoc (relying on informal methods and with dubious effectiveness).There is need for tools and techniques that help proper development and production of satisfactory results.Architecture, for example, is a fundamental item to be adopted.Over the years several proposals have been made by different authors, each one with its own characteristics.To use with Kira, a proposal by (Benyon & Murray, 1993) was adapted and used.Overall, there are three components which can be seen in Fig. 7.The domain model is responsible for representing the interface in its form, the context in which it operates and its logical functioning.User interaction aspects which are able to be changed can only be altered whether they are described in this model.Runtime data collected by the dialog record are represented here (Benyon & Murray, 1993).The user model is responsible for representing user regarding their profile, knowledge, and cognitive characteristics.For example, attributes that denote user experience with computers or even its frequency of use may be present.According to (Benyon & Murray, 1993), it inherits all attributes from domain model.
amount of errors made, while the second is the average completion time of the tasks.The third denotes which interface is being used.In the knowledge base there might be the following production rule: IF errors > 15 AND average_completion_time >= 20 THEN interface = novice_interface.Along with this information, inference engine can change interface presented to the user when it makes lots of errors or even takes a long time to perform a task.

Concept maps
Concept maps are graphical tools used with learning or knowledge representation.It consists of related concepts linked through connections in order to represent a domain in particular (Novak & Cañas, 2008).Overall, we may say they are similar to a graph since it has nodes, equivalent to concepts, and edges, equivalent to connections.
The foundation theory of concept maps is called meaningful learning from (Ausubel et. al., 1980).It is correct to say that concept maps must show a familiar content to learners.According to (Ausubel et. al., 1980): "the most important factor influencing learning is what a learner already knows.Find out what he knows and base upon that your teaching."Regarding use of concept maps to teach a knowledge domain, we can say a human being learn more efficiently whether it is presented a more general map instead of one with lots of specific issues (Ausubel et. al., 1980).Despite being simple, concept maps have proven to be a valuable instrument since its use implies attribution of new meanings to concepts and techniques of traditional learning.Regarding Kira's novice user interface, it was developed by means of a concept map representing all concepts and connections fundamental to understand Data Mining process.Due to its similarity with a graph, an adjacency list has been used to represent it with when coding took place.Its logic consists of maintaining a linked list containing all graph nodes, which also store those which they relate to.In order to see them, user only needs to move mouse cursor to a desired concept.
Aiming at reducing complexity regarding presentation of many concepts at the same time (up to 22 depending on the data mining task); we choose to present the map in two parts.What is initially shown is a map which is common to all data mining tasks (Fig. 8).After Kira recognize the task which will be used, map expands itself and presents the rest of process.Regarding concepts amount, twenty-four capable of representing the tasks of association rules and classification were identified.Grouping tasks is not coded yet and its concept map was not developed.The two concept maps were built assisted by specialized staff.Interaction with Kira concept map is not restricted to navigation and obtaining explanations of concepts.Data Mining process requires data to be defined in order to present its results.Therefore, JFrames (adaptive system was coded in Java and so did Kira) internal to the map were developed to perform such function.Each of them performs a specific and well defined task.For example, since it is true that Data Mining needs a data source, there is need to have it inserted.In this context, there is a JFrame triggered by a simple mouse click on concept "Data Source" at the conceptual map of Fig. 8, which allows users to execute this task.
JFrames have also been developed using the recommendations of GuideExpert system as shown in Fig. 9.
The intelligent interface of Kira now fits two types of users: novice through the use of conceptual mappings and monitoring of the user through the stages of data mining; and the old interface of the tool for experienced users.Was elicited in this category that users find it easier if the interface itself controls the dialogue and also allows the use of gestures.
Results of the advantages of adaptive interface for novice users of the KIRA tool are being collected.The incorporation of concept maps to the data mining teaching process and monitoring of the tool has been found positive.

Conclusion
This chapter has presented the state of the art regarding the human-computer interfaces and how they are increasingly focusing on tasks and helping users.This meets the tendency of "ubiquitous" systems and natural way to interact with them.Certainly there is much to research and the help of artificial intelligence area is significant.It was also presented two systems that contribute to the area: the expert system for Human-Computer Interface Design Guidelines (GuideExpert) and an intelligent interface for a data mining tool (KIRA).GuideExpert was used in the development of KIRA user interfaces.Certainly, when finalizing the design of KIRA adaptive interfaces, recommendations on intelligent interfaces can be added to GuideExpert thus providing the acquisition of more specialized knowledge.

Fig. 8
shows the initial map presented to a novice user.Respectively, numbers 1 and 2 from it indicate a concept and a connection.Number 3 indicates an area reserved to aid map navigation.Through it, concept explanations and tips about what should be done are given.