Open access peer-reviewed chapter

Usability Testing Methods and Usability Laboratory Management

Written By

Josef Pavlíček and Petra Pavlíčková

Submitted: 10 June 2022 Reviewed: 23 November 2022 Published: 29 December 2022

DOI: 10.5772/intechopen.109140

From the Edited Volume

Updates on Software Usability

Edited by Laura M. Castro

Chapter metrics overview

158 Chapter Downloads

View Full Metrics

Abstract

Usability testing of software products is of key importance nowadays. In order for usability testing to have the desired effect, the appropriate testing methodology must be chosen. Usability testing can be carried out using qualitative or quantitative methods. A widely used qualitative method today is the heuristic evaluation proposed by Jacob Nielsen. However, there are other testing methods such as cognitive walkthrough or collaborative testing (proposed by Josef Pavlicek and R. Bock). Although heuristic analysis has generally received a lot of attention in the current literature, it is important to put the other methods in the right light. These provide a significantly better view of the user’s passage through the interface under test. The methods better simulate the environment in which the final UI will operate. The user experience (UX) is then significantly better measurable if the participant goes through the test scenario in a cognitive (i.e., mental model-defined) way, rather than by mere heuristic evaluation. A significant milestone is then the cognitive-collaborative passage, where the collaborative element of the evaluators contributes to the evaluation of the solution.

Keywords

  • use case
  • scenario
  • Persona
  • user group
  • usability
  • usability testing
  • usability scenario
  • heuristic evaluation
  • cognitive walkthrough
  • collaborative walkthrough
  • UI Lab
  • collaborative UI Lab
  • testing
  • remote testing
  • qualitative and quantitative usability study

1. Introduction

Creating user-friendly user interfaces is a hot topic at the moment. A quality user interface is easy to use, looks attractive, and gives the user a good user experience. This experience is called user experience, hence the abbreviation UX [1]. In order to design a good user interface, it is necessary to understand the user [1, 2]. The next step is to be able to use technology to best implement the requirements. The other step is to be able to test the user interface. The overall test is called a usability test and is subject to a number of rules. They need to be learned. It should be understood how to learn this knowledge. If these factors of knowledge and skills are combined, results can be expected to go beyond user-friendliness to achieve excellent results. And that is the art of user interface design. This chapter covers the methodology and practices one should adopt, to become a proficient eXperience designer. And just like with mixing cocktails, on the one hand, you need to follow the tried and true procedures (a cookbook) [3, 4], so too is your inventiveness, training, and experience, which you will only achieve with honest practice. The focus of our contribution is to teach readers how to practice interaction design methods.

The design of user interfaces is thematically divided into the following chapters:

User Definition: In this section, authors explain the mental model of the user. Using examples, the authors show the reader how the mental model is affected. They give illustrative examples of how the mental model is affected. They show its influence on the users decision-making. The authors describe how a user interface should behave in order to have a positive influence on the users mental model.

Persona and User Groups: In this section, the authors explain which tools can be used to get a good understanding of user behavior. In the previous section, the mental model of the user was explained. Now, it is necessary to explain how a user with the desired mental model can be found or, rather, how to understand the mental model of a potential user. To do this, the archetypes of Persona user models or Focus Groups users will be used.

Usability: In this section, the authors explain possible methods for testing the user interface. UI testing is a challenging discipline and deserves its own chapter in the book. Therefore, they limit themselves to the most important aspects of user interface testing. They describe heuristic analysis and cognitive walkthrough. They show other testing methods and their advantages.

Advertisement

2. User definition

First, let’s define the user. Who is a user? It is the person who uses our app and is willing to pay for it. And because interaction designers strive to do great work and service, they want to get paid well for it. So, they need to know how the prospective user responds and what their needs are. In other words, you need to know the mental model of the user (Figure 1).

Figure 1.

Who is the user?.

2.1 Mental model

A mental model is an explanation of someone’s thought process about how something works in the real world. It is a representation of the surrounding world, the relationships between its various parts and a person’s intuitive perception about his or her own acts, and their consequences. Mental models can help shape behavior and set an approach to solving problems (similar to a personal algorithm) and doing tasks.

Jay Wright Forrester [4] defined general mental models as: “The image of the world around us, which we carry in our head, is just a model. Nobody in his head imagines all the world, government or country. He has only selected concepts, and relationships between them, and uses those to represent the real system”

For the user-centered design (UCD), it will be useful to leave psychological theory behind and to be more practical.

User mental model from the UCD point of view:

  • Human adaptation to use machine according to gained experience.

  • Shapes human perception of using machine.

  • If anything is broken, user feels wrong and does not want to use the machine.

  • Depends on age and gender (children are more adaptive than adults).

  • Is influenced by geographical conditions.

  • Is influenced by nationality.

    • Perception of symbols (typical example).

    • Swastika (European symbol of war. In Hinduism, the swastika is considered an exceptionally holy and positive symbol and is therefore very often used on all occasions).

    • Crucifix (Christian symbol of human savior, for Romans, a method of capital punishment).

The mental model is best modeled with the help of the so-called Persona; see section Persona.

2.1.1 Manifestation of the mental model

  • It shapes human perception toward understanding and using the user interface.

  • Depends on age and gender (children and young people are logically the most adaptive).

  • Manifested, for example, in the perception of colors and symbols (there is a strong cultural influence here—typically the perception of swastika symbols).

  • If violated, the user feels uncomfortable and stupid and refuses to use the machine (this state is to be avoided).

2.1.2 Emotions and the mental model

The mental model manifests itself in the emotions of the user (Table 1).

Emotions are generally divided into
FearAngerJoySadnessConfidenceDisgustExpectationsSurprise

Table 1.

Human beings’ emotions.

Now, the reader should learn how each emotion affects the user and how it affects their evaluation of the user experience.

In general, interaction design or user-centered design (UCD) distinguishes between low emotional load and high emotional load (Figure 2).

Figure 2.

Mental model schema.

2.1.3 Low emotional load

It is typical in that although it affects us negatively (creates a negative feeling of fear, anger, surprise) it is manageable and, depending on the type of software product, acceptable. A sample of such load is “unnecessary large number of clicks,” placing various controls in unexpected places. This load can be eliminated by learning how to work with the application and is tolerated in applications that are complex and technologically advanced in their foundation: graphic editors, CAD systems, etc. It is unavoidable for complex software products. Computer technological capabilities, and legal and safety regulations can significantly affect them.

This kind of load should not occur with leisure equipment. However, the rule of system complexity also applies here. Take the example of Tetris. The controls should be intuitive. Contrary to a transport aircraft simulator, here the controls will be less intuitive. The controls will therefore impose a certain mental load. However, this will be accepted by the player. After all, controlling an aircraft is challenging

2.1.4 High emotional load

This kind of load must not appear in our solutions. It is a load that brings the user into severe emotional states such as loss of confidence, disgust, and anger. It appears whenever the software performs an operation that is emotionally arousing.

For example:

  • It does not save work in progress.

  • The printout is not formatted correctly, even though the preview is correct.

  • The work is unexpectedly terminated or, on the contrary, the task cannot be completed (such a pattern is, e.g., the former behavior of Windows updates, when trying to shut down the computer, the computer does not continue its shutdown task and updates the computer for another 15 minutes … the user in the meantime urgently needs to catch the train taking him home from work).

  • Computer crash, etc.

2.2 How emotions affect SW evaluation

For the purpose of UX, it is enough to remember that the user load caused by repetitive routines is acceptable to some extent.

Stresses affecting the users feelings (anger caused by the loss of important data and work … e.g., unsaved chapters in a master’s thesis, etc.) is deeply imprinted in the user’s memory. The latter then intuitively discriminates the solution, even if the bug is already fixed in the next version.

2.3 Rules of UI behavior toward the user

2.3.1 Be goal (UseCases) oriented

Remember that every user goal (UseCase) that is implemented in the form of buttons, screens, and other components should be implemented as simply as possible and with a minimum of energy. This is the direction in which the UX rules defined by Jacob Nielsen lead; see the Usability chapter.

2.3.2 The machine should always help

In general, it should be understood that the purpose of a machine or computer program is always to help the human (user). If it stops helping and starts harming, it becomes a hated monster. It is important how the machine alerts the human to a critical condition. The relevant dialog was not meant to be hidden.

2.3.3 Don’t make a fool of me!

From the original “Don’t Make Me Feel Stupid!” It tries to suggest to designers that good software will guide and help the user. It will not confront them with frustrating tasks and ask them inappropriate questions (Figure 3).

Figure 3.

A confused user feels uncomfortable!.

2.3.4 Don’t stress the user with unnecessary questions

Recognize that software is supposed to help the user. It is supposed to anticipate his actions, automatically assuming that the data loss is fatal (i.e., no need to ask whether to actually save). Only when there is a real danger of data corruption by the user’s decision, to warn him.

2.3.5 Inform me clearly about errors

Error messages are essential in UI and are very often underestimated by designers. It is important to remember that the user is not the programmer. He is not even an analyst and did not design the application. What is obvious to us as developers, for example, a connection to the database crashed, is incomprehensible to the average user. Therefore, messages like in Table 2.

Wrong error messages example
The program has performed an invalid operation and will be terminated.
An overflow has occurred at address A3453BCH122AA.
ORA-00933: SQL command not properly ended.

Table 2.

Error message example.

Each message must be translated into the user’s language, and others cannot appear in the UI. Thus, for example:

  • MyTextEdit could not save the last edit. Check that you seem to have enough disk space and repeat the save operation. If the failure persists, save the work to an external disk and contact the administrator.

However, never forget Hick’s law, which says [5, 6, 7, 8]: “The reaction time required for the user to complete his task therefore increases with the number of available options.” Simply put, the more objects in front of the user, the longer it takes to make a selection.

2.3.6 The principle of avoiding surprises

The average computer user likes not to be surprised by anything. Some parts of the user interface should stay in the same place and not change significantly. A typical example is the user menu. If images or icons are used, they should always have the same expression throughout the user’s passage through the program. The user is unnecessarily surprised and confused if the meaning of the icons changes. In that case, he stops liking the program he is using and runs the risk of becoming frustrated with it. Such a case leads to abandoning the program and replacing it with another more intuitive one.

2.3.7 Restrictions on the use of icons

Let’s remember that the icon is closely related to the mental model of the user. Under the pictogram (i.e., icon), different users can easily imagine different meanings. Therefore, the use of icons in the user interface should be minimized. They should be placed where the operation is often repeated. On the other hand, operations that do not occur frequently should rather be supplemented with a textual description, hence the following rule of thumb for navigating the user interface using navigation elements:

  1. Textual description (absolutely clear).

  2. Icon with text description (nice design, unfortunately, duplicates the description and graphic meaning).

  3. Text bubble help icon (ideal for frequently repeated operations, but has less telling power than a text description).

  4. Icon without description and bubble help (used only for chronically known operations inherited from the operating system = minimize, maximize, move, close = window edge operations).

  5. Graphical symbols (buttons, sliders, etc., no longer belong to the primary communication elements and are covered by the Interaction Design dimensions).

In this section, let us cite five basic principles of human-PC interaction called the five dimensions of Interaction Design. The concept of dimensions of interaction design was introduced in Moggridge’s book Designing Interactions [9]. Gillan Crampton Smith [10] wrote that interaction design draws on four existing design languages, 1D, 2D, 3D, 4D. Kevin Silver later proposed a fifth dimension, behavior [11]. Let us quote Josef Pavlicek [3] and Kevin Silver [11]:

Words: This dimension defines interactions: words are the element that users interact with.

Visual representations: Visual representations are the elements of an interface that the user perceives; these may include but are not limited to “typography, diagrams, icons, and other graphics”.

Physical objects or space: This dimension defines the objects or space “with which or within which users interact”.

Time: The time during which the user interacts with the interface. An example of this includes “content that changes over time such as sound, video or animation”.

Behavior: Behavior defines how users respond to the interface. Users may have different reactions in this interface.

2.3.8 The limitations of human memory

To make the use of a computer pleasant and ergonomic for the user, it is necessary not to rely on its memory. In addition, other important features must also be considered. Referring to the book by J. Pavlíček [3], Table 3 can be briefly explained as common rules.

The common ruleSystem behavior
Important informationAlerts, warnings—should not disappear from the screen after a short time!
Data fields should be arrangedIn the order the user expects them, or as is usual on similar sites. Required fields should be in sequence, not in skip order with optional ones.
Make the user’s job easierYou can help, for example, by automatically formatting the phone number “+1-250-0197-2500” instead of “+125001972500.”
Provide help and informationInform about where the user is, or how they got there.
If something is wrong, informInform the user with ongoing feedback. For example, breadcrumb help.

Table 3.

How to help the user.

2.4 System feedback

As you go through the software, you need to communicate to the user what is happening or what has just happened. If the user does not know or is not clear, he or she will most likely stop using the software.

The most basic physical feedback is very important. For example, when a user types “help,” the word “help” should appear on the screen. Physical feedback includes all kinds of feedback, from system sounds to the sound of a mouse click to actually highlighting text if the user clicks on it and hovers the mouse over it, for example.

If there is an operation going on in the program that requires waiting, the user should also be informed of this. For example, an hourglass or a window with the progress of the operation (Progress bar) is established.

You should remember: The user interface should respond to every user action.

If the user moves the mouse, the mouse cursor must move. The guidelines for your chosen operating system (e.g., MacOS Human Interface Guidelines) describe the basic behavior of the user interface. These should be kept in mind by every application developer. They should be followed by the software development. The basic rules include response time (Table 4):

Response timeBehavior
Less than 3 secondsThe UI does not perform any informational activity.
Longer than 3 seconds and shorter than 20 secondsUI displays an hourglass or an icon symbolizing waiting.
Longer than 20 secondsUI displays a status bar (progress bar)

Table 4.

Response time behavior.

Care should be taken: There are several variations of status bars. Usually for an operation where the duration can be calculated (either you know the processing speed or you know the number of operations to be performed), it is advisable to use a percentage = incremental status line. Such a case is, for example, performing a search for a text string in files on disk. If the total number of required operations is known, it is easy to calculate what percentage of the work has already been done by the machine and display this information on the screen. If it is not possible to predict the processing time (or it is not possible to show the user what percentage of the total has already been processed—the processing time is likely to vary according to the performance of the user’s machine), then an infinite status bar is used. This moves from right to left. This signals the user that the operation is still being executed. A typical case might be loading a file of unknown length from the Internet (but it is recommended to avoid this feature if possible).

2.4.1 User attention

Techniques for gaining user attention need to be handled very carefully, especially when designing software. Various flashing banners and pop-ups are usually distracting. The same goes for very rich colors. Users tend to ignore overly significant elements, and the so-called Banner Blindness phenomenon arises. Too strong elements are ignored by users as if they were ads.

It is also advisable not to use more than four different font sizes on one page and a maximum of three different fonts, and it is also not advisable to write only in capital letters. Use text underlining and highlighting only rarely.

NOTE: Underlining the letters in a word is a very useful UI element. The so-called mnemonic aids help to use shortcuts for various operations, see: CTRL + S = Save… Unfortunately, these aids disappear from new operating systems. However, orientation-intensive tools, where the user is expected to adapt deeply to the machine and use abbreviations, still use them.

2.4.2 First-Time experience

This rule (confirmed by J.Pavlíček, R. Bock during the UI Studies) says that there is a very close correlation between how the solution is given to the user (in our case primarily software, but it can also be HW and SW = mobile phone) acts during the first hour. If the user is still not safe in the basic user goals after the first hour of using the solution, then his willingness to use it decreases. This is a very dangerous condition. It affects all software solutions that are not highly specialized for a given activity and are in fact a trend issue: be it Open-Source SW, operating system, and mobile phone programs, after the personal computer operating system.

2.4.3 Houmer Simpson Car issue

The user interface must meet a perfectly limited area of requirements. Respectively, it must be specialized for primary persons. The success of the overall solution is based on the definition of person. This is very important to realize if the people are poorly designed, then the overall UI/UX will also be bad. If properly designed, then the UI must best meet the requirements of the primary person. The requirements of Persona B must allow, but it does not matter that there will be any mental burden on the user. The requirements of a negative C person should not meet, but can only perform as a secondary effect of other functionalities (Figure 4).

Figure 4.

Homer Simpson car issue [12].

It is not possible to satisfy everyone’s requirements equally. It is never possible to satisfy the requirements of all users. This is something to keep in mind. It is a common mistake of beginner designers. This mistake is now in UX slang by “Houmer Simpson’s car.” It satisfies a wide range of average users but always only partially. After all, it is not suitable for “cottages” or “shopping.” The desire to satisfy everyone is very strong, and this mistake manifests itself very often. Just look at the experiments of, for example, car factories with some cars. Certainly, similar diameters can be found in world brands. So, watch out for that.

Advertisement

3. Persona and user groups

In order to understand how to properly design a user interface, it is necessary to model the future user. Then focus the user design specifically on their needs. Of course, this depends closely on the type of product being developed and the means available to present it to the public. If the designer is innovative enough (like Steve Jobs with smartphones, for example), then he does not need a model user. He creates the product and markets it through advertising and technological presentation. However, this is both very risky and very expensive. So if he is developing a more or less traditional software product, then he should follow the established principles (be more careful with innovation). With the help of model users, say whether such a user exists in real life (because he wants to sell the solution). If so, he will use it for the design. Here it is all about Personas.

3.1 Persona

Personas are archetypes that describe the various goals and observed behavior patterns among users. A persona encapsulates critical behavioral data in a way that both designers and stakeholders can understand, remember, and relate to.

Personas use storytelling to engage users’ social and emotional aspects, which helps designers to either visualize the best product behavior or see why the recommended design is successful.

A persona is a fictional character, which is used while designing an application, website, or technology. The persona represents a fictional user of the tested application, which has its own personality, needs, characteristics, specific approaches, and opinions:

  • It is created from the data collected during user interviews or with the use of some other statistical methods.

  • Personas are created by implementing common behavioral patterns of more users into one fictional person.

  • Using personas in creating projects puts a human face to otherwise abstract data and allows the designers to easily understand the needs of the fictional characters.

3.2 Persona variants

The authors suggest to recognizing three types of personas:

Persona A: A is a typical user. (S)he will use all functionalities, and the system MUST BE implemented for him/her. It is necessary to design GUIs for that person’s business goals-derive UseCases from them! (Figure 5)

Figure 5.

Persona A can be young girl.

Persona B: B is an occasional user. The GUI is not primarily implemented for this user. But the GUI should address his UseCases. However, these UseCases can be time- and energy-consuming. Although he/she is not the primary user of the system, we need to provide him/her with some ways to complete his/her business goals (UseCases) (Figure 6).

Figure 6.

Persona B can be a boy.

Persona C: C is a negative persona. It is sometimes called antipersona. This user NEVER uses the system.

In general, different kinds of model users can be created. However, the authors recommend only the following three types. The authors consider the other variants to be redundant. The user interface should be designed for the primary personas! Other personas are users who do not primarily use the system. However, they should be able to work with the system. A negative persona is a user who will NEVER use the system. It is introduced for contrast or because of a different view of UI functionality that should NOT be part of it. It is a view that should be avoided (Figures 7 and 8).

Figure 7.

Persona C never uses the system.

Figure 8.

Persona A example form.

3.3 User group

In some cases, the definition of Personas is somewhat complicated. Or it is not suitable because a large number of model personas would need to be created. In this case, it is appropriate to define the group as Persona. These are typically information site designs. Such sites where there are a large number of user groups. A clear (and painfully tested) example is the university website. Many requirements are mixed here, and it is very difficult to decide who is actually the primary person.

Universities must always address:

  • Students (young people, doctoral and postdoctoral applicants, students of the University of the Third Age).

  • Teachers, professional assistants, other roles such as accountants, PR.

  • Parents and sponsors.

  • State apparatus and bureaucracy.

  • Industry, services, art.

  • Other schools.

Defining a persona can be challenging. Therefore, it is possible to create user groups and label them as people on A, B, and C.

Authors personally do not recommend this method. Our practice is to make it safer to create honest people. Personas can possibly be assigned to groups of the type (teacher, student, and parent).

Advertisement

4. Usability

The term Usability is defined [13] by the international standardization organization ISO (ISO / TR 16982: 2002 standard) as authors quote: [14] ISO/TR 16982:2002 (“Ergonomics of human-system interaction—Usability methods supporting human-centered design”) is an International Standards Organization (ISO) standard that provides information on human-centered usability methods that can be used for design and evaluation. It details the advantages, disadvantages, and other factors relevant to using each usability method. It explains the implications of the stage of the life cycle and the individual project characteristics for the selection of usability methods and provides examples of usability methods in context.

Applicability is also defined by Jakob Nielsen (whose principles are based here) [1, 2]. The usability attributes are visible at the Table 5.

The above attributes are necessary in order to measure user friendliness.

4.1 UseCase and scenarios

4.1.1 UseCase

In software and systems engineering, a use case is a list of actions or event steps typically defining the interactions between a role (known in the Unified Modeling Language (UML) as an actor) and a system to achieve a goal. The actor can be a human or other external system. In systems engineering, use cases are used at a higher level than within software engineering, often representing missions or stakeholder goals. The detailed requirements may then be captured in the Systems Modeling Language (SysML) or as contractual statements.

Use case analysis is an important and valuable requirement analysis technique that has been widely used in modern software engineering since its formal introduction by Ivar Jacobson in 1992 [20]. Use case-driven development is a key characteristic of many process models and frameworks such as ICONIX, the Unified Process (UP), the IBM Rational Unified Process (RUP), and the Oracle Unified Method (OUM). With its inherent iterative, incremental, and evolutionary nature, use case also fits well for agile development.

The creation of UseCases has evolved, and a number of derivations have emerged since the so-called fully dressed UseCases. There is no space in this chapter to deal with the development of UseCases. Therefore, let us skip straight to the method that has worked well for us in designing UseCases.

UseCases are always based on the goals (user requirements) of the software. It goes without saying that one requirement can be implemented by a different number of UseCases. Therefore, the following rule should always be kept in mind:

A UseCase is a users business goal, which he wants to achieve as quickly as possible for the least amount of effort expended.

UseCase should always be written from the users point of view. It is written from the perspective of the users expectations!

It is necessary to distinguish between the user’s expectation and the demand.

The difference between the two words is crucial and should be in the ratio of 80 (expected) : 20 (required) in UCD and UX documents.

This is because UseCases are created (written) at a time when the form of the application is unknown (or mostly unknown). It lacks its controls, which must be derived from UseCases. If a button was required for every desired user action already in the UseCase (e.g., printing a report), then it may be that the controls are designed differently in 1000 UseCases.

There is a risk that each software analyst and designer who creates them may have a different idea of the final implementation of the operation. Consequently, identical operations will differ in implementation. This is another common mistake due to improperly created UseCases.

After all, this is common today. Different parts of the applications have different controls (although they are in accordance with the prescribed Look and Feel of the operating system). Usually, a huge mental burden is put on the programmer in the end, because he then has to implement it.

UseCase is therefore written in terms of user expectations, and it is necessary to avoid defining graphical controls.

For example: the user expects to enter the name of the print report and the number of pages, and sets the printer type. The user confirms the request (it doesnt say how or by what!).

Only if the user is forced to define a control for some important reason (is limited by health-color blindness, for example), then should be used the word demand.

4.1.2 Scenario

The opposite side of UseCases is the scenario. This is always written from the point of view of the system and how the system satisfies the user’s requirements. So in this example, the system will display a print report icon, a text box to enter the number of pages, a combo button to select the printer, and a print button.

The scenario is always precise and describes the controls of the system. It is usually written after UseCase is finished, and there exist prototypes of the screens in the form of wireframes.

4.1.3 UseCase and scenario in UCD

Therefore, the post of writing UseCase and Scenario is generally as follows:

  • First, write UseCase in terms of user expectations. Avoid defining controls. Remember, it’s too early for that.

  • Design a wired screen model for one or more pooled user targets—UseCase.

  • According to expectations and wire models, implement the scenario. This will match the look and feel of the application and describe how the system satisfies the user’s expectations (Figure 9).

Figure 9.

UseCase, WireFrame, Scenario example.

4.2 Usability testing

Usability testing is key to understanding and evaluating the quality of the designed user interface:

  • The aim is to improve or update an application.

    • Should answer the question how and what to update.

  • Measuring how human-made products meet its intended purpose.

  • Observe people (personas) using the product to discover errors and areas of improvement.

  • Verifying that real users can use the system:

    • Understanding how the real users (will) interact with the system.

    • Identification of problems that prevent real users from using the system.

The testing procedure is as follows:

Define the usability test framework:

  • Define which parties participate in the test (client, developer, user, and sample).

  • Define what is the purpose of testing, for whom is it intended, and what hypotheses is confirmed or refuted.

  • Define on which technology the test will be performed:

    • Above the prototype.

    • Above the existing solution.

  • Define where it will be executed:

    • On site.

    • UI lab.

    • Remote.

  • Decide on the testing method:

    • Without users—expert review:

      1. Heuristic evaluation.

      2. Cognitive walkthrough.

      3. Collaborative walkthrough.

    • With users:

      1. User surveys.

      2. Ethnographic observations.

      3. Usability engineering.

      4. Collaborative usability engineering [15, 16].

  • Variants of usability testing methods:

    • Qualitative.

    • Quantitative.

  • Define a story that explains the reason for testing.

  • Define a time frame.

4.2.1 Participants’ definitions

Study participants should be primarily sponsors and stakeholders. Furthermore, creators who learn from their mistakes. Last but not least, application designers, software analysts, and testers should be represented.

4.2.2 Testing purpose

It is important to know the purpose of the test. The test can be done because of a negative “First-Time Experience.” Or, on the other hand, a new system has been created and needs to be tested. It is also possible to test only the design. The design can take the form of paper prototypes, Lo-Fi or Hi-Fi prototypes. The following questions should always be asked as in Table 6.

The Nilesen’s usability attributes
Attribute nameExplanation
Simplicity of learning to use the system:To learn to use a similar system faster.
Recall:To remember the way from one situation to another
Efficiency:To carry out the task quickly and efficiently
Minimum amount of errors:If encountered, inform the users on the cause and an advice how to proceed
Satisfaction of the user:The users is convinced that the task has been successfully achieved

Table 5.

Nielsen usability attributes.

What and how will we test
What is the test goal?What is new in the test?What the test is not?

Table 6.

Test questions.

4.2.3 Testing technology

Various tools can be used for testing. The most common are prototypes. These can be divided into two groups:

Lo-Fi prototype: This prototype requires a relatively short lead time. In the order of hours (paper prototype and pencil sketches) or days (clickable tools in Case, such as FIGMA). It is possible to create a large number of prototype variants. This prototype does not address the final look. Colors and color schemes can be designed as sketches. The prototype does not communicate with the core of the application, and there are no technology links.

Hi-Fi Prototype: This prototype is time- and money-consuming. It simulates the appearance of the future application. It often runs on the final platform. Look and Feel is defined and represents the final look and feel. Very often it is part of the implementation (clickable UI without connection to the core system).

Finished application: The last option is an already finished application. This testing method is very efficient for agile sw development methods. It will be used when the developer can communicate with the client. Testing is done in short iterations. Fixes are fast. Large software products can be developed in this way (if the developer follows the rules of agile development). It is also generally used when fixing a finished application.

4.2.4 Testing environment

The environment in which the testing takes place is equally crucial. In general, they can be divided into environments.

On Site: that is, in the user’s office or wherever the application is to be used. The advantage is that there are natural and therefore realistic conditions. The user uses learned technologies and environments to control the application. Confusions such as “I have a different mouse,” and “The chair is not comfortable” can be avoided.

The disadvantage is that conditions may vary according to different offices. This makes it difficult to repeat tests. It is also very difficult to use the company’s offices for the tests. In an open-space office, other tasks than usability testing are generally performed. The work tasks of non-participating employees can disturb the progress of the test, and even vice versa, the test can be disturbing for non-participants.

This method is suitable for small projects or projects run according to agile methodologies.

UI Lab: The specialized environment for usability testing is called UI Labs. Currently, there are several types of these labs. There is probably no general classification of them at the moment. Therefore, let us divide them here. In authors’ experience, it seems to be an appropriate method to divide UI Labs according to the type of studies we perform. These studies (authors discuss them in more detail below) can be carried out with a focus on:

  • Participants: This is generally heuristic analysis and cognitive walkthrough

  • On collaboration: Cognitive walkthrough and collaborative walkthrough [14, 15, 16]

4.3 Participant-oriented UI Lab

If the UI Lab is participant-oriented, then its architecture is subordinate to that.

The room in which the participant performs the usability test has one or two workstations. This room is generally separated from the observer room by a one-way transparent mirror. This is transparent from the observer room so that they can observe the progress of the usability test. This is where the UI observers are in the front row to record the progress of the test. Next are the programmers and application stakeholders.

This UI Lab is significantly oriented toward the individual (participant) by its architecture. The testing method is then predominantly Jacob Nilsen = Heuristic. Sometimes the cognitive walkthrough method is performed (Figure 10).

Figure 10.

Participant-oriented UI Lab.

However, from our experience [14, 15, 16], there is a criticism of this solution as in Table 7.

AdvantagesDisadvantages
Recorded results are well presented and well processed.The environment is not real. Nowadays, when it is common to connect with colleagues via social networks and work in Open Space offices, it is not common for a user to be alone when working with SW.
It is well known what kind of work the participant is doing.A participant sitting alone (or connected to the test moderator) and aware of his/her observations, probably puts more effort to find a way through the system. In this case, the test is burdened by mental error. Often self-doubt about one’s own abilities will prevent the participant from passing a true assessment of the UI elements. This partially limits the thinking-at-the-voice rule.
Thinking aloud means the mental activity of the participant.There is no real-time testing of processes (i.e., not only UI but also usability of work processes).

Table 7.

UI Lab benefits.

4.4 Collaboration-oriented UI Lab

I created this kind of UI Lab with Rudolf Bock as part of a research project at the Faculty of Management and Economics of the Czech University of Life Sciences [17, 18]. Authors call it Collaborative UI Lab. This kind of lab (covered by a national utility patent – PUV 2016-32993) actually responds to the shortcomings of the original architecture. It allows for extensive real-time testing of processes. At the same time, it allows to perform classical Usability testing. It also allows pair testing or interacting (collaborative) testing. This is the innovative approach promoted by see [14, 15, 16].

The collaborative testing lab contains two essentially semicircular tables, each containing at least three workstations, each station containing a computer, a monitor with a camera to capture the user’s face, headphones and microphone, and a means for audio recording, with the tables positioned so that the center of the imaginary circle of which the table is a part faces the other table, and furthermore, the axes of symmetry of the two tables are parallel and offset from each other (i.e., and not identical), and the workstation monitors are positioned on each table such that the screens are positioned away from the other table. The central workstation of each table is equipped with means for tracking eye movements. The laboratory shall further comprise at least four wide-angle cameras, at least two of which shall be positioned to image the workstations including the monitor screens. In addition, the laboratory shall contain at least two loudspeakers.

In a preferred embodiment, each semicircular table comprises at least five workstations.

The speakers may preferably comprise a fixed or detachable video camera.

The laboratory may preferably be equipped with control means for the wide-angle cameras, means for converting the video signal to a digital signal, and means for recording and editing images transmitted by the cameras or sound from the microphones.

The individual elements may be connected by wires or wirelessly.

According to the presented technical solution, the laboratory allows several test participants to solve one task at the same time. Each of them can perform and be recorded as an individual and/or as a member of a team. This makes it possible to include the influence of social ties in the team in the testing and thus to make the testing closer to reality. Models focusing on team behavior can also be tested in this way. Because participants are tested under exactly the same conditions, the predictive value and statistical evaluation is easier and more relevant.

The semicircular shape of the tables and the placement of the monitors on them ensure that the individual users are not dependent, but the monitors can still be rotated so that 2–3 participants can work together if this is appropriate for the proposed test. The semicircular shape of the tables further creates a sense of unity and coherence of the working group and also ensures equal working conditions for the participants (Figure 11).

Figure 11.

Collaborative UI Lab [18].

The layout of the lab allows for the placement of the test participants so that their faces and their gazes are directed to the space between the tables where the facilitator/testing guide can stand. At the same time, no one has to stand behind the participants, so they do not feel they are looking over their shoulders.

The study facilitator has the option of communicating with study participants individually through headphones or with all of them simultaneously (and identically) through speakers.

In one convenient embodiment, the lab is further equipped with a removable screen placed between the two tables. Screens can advantageously be placed on either side of the screen. The screen allows for competitive tests to be conducted, and brings the possibility of measuring behavior within the competitive environment and its effect on the solution of the task.

Depending on the technical design, the lab is suitable for, for example, conducting usability studies, collaborative usability studies, user experience testing, behavioral research, testing of commercial products and services, media testing, test compositing, and conducting test trials. Unlike previously known usability study labs, it allows parallel testing, reducing costs and increasing the relevance of comparisons between studies (Figure 12).

Figure 12.

Collaborative UI Lab description [18].

A laboratory for collaborative testing [3, 14, 15, 16, 18] characterized in that it comprises two substantially semicircular tables (1), each containing at least three workstations, each station containing a computer, a monitor (2) with a camera for capturing a user’s face, a headset and a microphone, audio recording means, wherein the tables (1) are positioned such that the center of the imaginary circle of which the table (1) is a part faces the other table (1), and further, the axes of symmetry of the two tables (1) are parallel and offset from each other, and the monitors (2) of the workstations are arranged on each table (1) such that the screens are oriented away from the other table, wherein the central workstation of each table is provided with means (3) for tracking eye movements, and wherein the laboratory further comprises at least two speakers and at least four wide-angle cameras (51, 52), at least two of the cameras (52) being arranged to image the workstations including the screens of the monitors (2).

4.5 Remote testing

This is a very modern method of usability testing today. The UI Lab environment is replaced by the user’s workstation. In fact, today’s technology makes it possible to largely replace the classic UI Lab. Technology such as screen recording, voice recording, face capture, eye tracking (Gaze Recorder) is now commonly used. By connecting the workstation to the Internet, using today’s On-Line technologies (Google Meets, MS Teams, Zoom, etc.), the workstation becomes part of a virtual lab. If the fact that the participant is not physically in the same room as the usability study facilitator can be managed or bridged, the test can be conducted. This approach has the following advantages as it is seen in Table 8.

Remote testing benefits
No need to build an expensive UI Lab.
The participant does not have to leave his/her place of work—saves time (which is an important factor). The participant does not have to leave his/her place of work—saves time (which is an important factor).
The participant conducts the test on his own equipment with which he is familiar (no orientation problems = advantages as in the In Situ approach).

Table 8.

Remote testing benefits.

4.5.1 Usability testing methods

Usability testing methods can be divided into two types: without users and with users.

Testing without users (called expert assessment):

  • Heuristic evaluations [1, 2, 19]

  • Cognitive walkthrough [1, 2, 3]

  • Collaborative walkthrough [3, 14, 15, 16, 18]

4.6 Heuristic evaluation

Jacob Nielsen defined 10 general principles of machine-human interaction that a good user interface should support. These are general principles, which is why they are called “Heuristic.” [1, 2, 19].

  1. Visibility of system status

  2. Match between system and the real world

  3. User control and freedom

  4. Consistency and standards

  5. Error prevention

  6. Recognition rather than recall

  7. Flexibility and efficiency of use

  8. Aesthetic and minimalist design

  9. Help users recognize, diagnose, and recover from errors

  10. Help and documentation

Allow us to quote them:

Visibility of system status:

  • A system should not “freeze” and get unresponsive:

    • Show the progress bar.

  • Inform the user what’s going on, give a time estimate if >20 sec.

Match between system and the real world:

  • Keep conventions and metaphors working:

    • Desktop.

  • Icons must behave as expected:

    • Trash can—I can throw objects to a trash can, it can be emptied and also searched in case of accident.

  • Beware of translation errors.

User control and freedom:

  • Users often choose system functions by mistake:

    • Provide “emergency exit” from unwanted states.

  • Undo.

  • Allow canceling of time-consuming operations (rollback).

  • Confirm actions which cannot be undone.

  • Warn before action that cannot be undone.

Consistency and standards:

  • Conventions of the platform should be obeyed.

    • Windows program looks and acts as a Windows program. No other way.

  • Use just system components if possible.

  • System colors and fonts.

  • Use same terms and names:

    • Save vs. Write, etc.

Error prevention:

  • The best error is the one which did not happen.

  • User should not be able to enter an invalid value:

    • Following error message does not help that much.

  • If the input requires some special form, it must be explained (in-line help text, hint):

    • Password strength/requirement.

  • Highlight required fields in forms

  • Confirm dialog where needed.

Recognition rather than recall:

  • Only relevant objects and options are visible.

  • The user is not requested to remember information.

  • Give preview for visual selections (font).

  • Step in a wizard (3 of 7).

  • Position in a tree structure (bread crumb).

Flexibility and efficiency of use

  • Provide keyboard shortcuts (accelerators).

  • Accommodate both novice and power users:

    • Functionality required by power users is mostly not needed at all by standard users.

    • Advanced mode.

  • Keyboard shortcuts/function keys.

  • Macros, templates.

  • Do we really need all that controls?

Aesthetic and minimalist design:

  • Show only relevant information:

    • Each unit of information competes with all other units of information.

  • Less options mean faster action.

  • Avoid useless graphics.

Help users recognize, diagnose, and recover from errors:

  • Best is not to get to the point where an error message is needed.

  • Plain language error messagesno code, no abbreviations, etc.

  • Precise indication and constructive suggestion of a solution.

  • Should educate the user that the error is not repeated.

Help and documentation:

  • System should be usable without any help

  • The help is a must anyway

  • Minimal, easy to search

  • FAQs and examples are better than plain text

  • Contextual help

Nielsen heuristics are designed for expert evaluation. This means that users are represented by experts. On the one hand, this results in fast and high-quality feedback on the usability of the UI. However, it is to keep in mind that this work cannot be done by a mere PC user.

4.7 Heuristic UI evaluation procedure – I/O

Focuses on the product as a whole—Table 9.

InputsOutput
A user interface—working application, a prototype, even a screenshotList of what violates these rules
Description of the target user group(s)
Use-cases/scenarios

Table 9.

Heuristic evaluation steps.

4.8 How to perform heuristic evaluation

Table 10 should be followed when performing the heuristic evaluation.

StepAction
TutorialExplain the application to assessors (experts), provide a scenario if needed.
EvaluationIndividual experts perform the evaluation and record results (which law has been violated and how). For the testing is used Testing scenario:
  • Scenario is a realistic situation, where a person performs a list of tasks using the product being tested, while observers watch and take notes:

    • Thinking aloud.

  • Use the most common or used scenarios.

  • Examples:

    • Create an account in the system.

    • Order a pair of jeans.

      Find information about a particular product.

Make Interview at the end (likes, dislikes, recommendation)“Perfect!”, “I love it!” is not a proper evaluation:
  • Some structure/framework needs to be defined.

  • Metrics need to be defined.

  • So that testers and designers of the UI can discuss the results.

    • Human behavior and actions are complex:

  • Difficult to analyze.

  • Individual difference.

The interview is important to ask (careful order is important):
  • What you liked.

  • What you didn’t like.

  • What are the recommendations.

Priority AssessmentAssess the severity of each problem.
ReportingDiscuss the conclusions with the assessors and design team.

Table 10.

Heuristic evaluation procedure.

Heuristic Evaluation—report example—Table 11.

Problem foundError wiping
Violates:4 Heuristic attribute -Consistency and Standards
Problem statement:Strings “Save” and “Write File” refer to the same function. The users may be confused by this ambiguity.
Priority:Medium (Usability problem, important to be removed).

Table 11.

Report example.

Heuristic evaluation form example

You need to find out, what precisely MDC (MyDreamCompany) can help with:

  1. Go to Google and find the MDC company.

  2. Visit the main page. What does the MDC’s logo tell? (Analyze the Future)

  3. What is the main message in the head banner? (Changing how the World Thinks).

  4. Your boss gives you the task to contact the MDC managing director for your region—Asia:

    1. What is his/her name?

    2. How many years of experience does (s)he have?

    3. What was the his/her position prior to joining IDC?

  5. What is his/her Twitter contact account?

Answers form [3] (Figure 13):

Figure 13.

Answer form for Heuristic evaluation example [3].

Heuristic evaluation form example. Not necessarily all 10 points are included. It depends on the focus of the test (Figure 14).

Figure 14.

Result of Heuristic Evaluation example [3].

4.9 Cognitive walkthrough

According to the Nielsen Norman Group, the Cognitive Walkthroughs is: [1, 2] “a technique used to evaluate the learnability of a system from the perspective of a new user. Unlike user testing, it does not involve users (and, thus, it can be relatively cheap to implement). Like heuristic evaluations, expert reviews, and PURE evaluations, it relies on the expertise of a set of reviewers who, in a highly structured manner, walk through a task and assess the interface from a new user’s point of view.”

Definition by Nielsen Norman Group [1, 2]: “A cognitive walkthrough is a task-based usability-inspection method that involves a crossfunctional team of reviewers walking through each step of a task flow and answering a set of prescribed questions, with the goal of identifying those aspects of the interface that could be challenging to new users.

During the cognitive walkthrough, we ask ourselves the following questions:

  • Will users try to achieve the right result? In other words, do users understand that the action (step) at hand is needed to reach their larger goal?

  • Will users notice that the correct action is available? In other words, is the interactive element that achieves the step visible or easily findable?

  • Will users associate the correct action with the result they’re trying to achieve? Perhaps the right button is visible, but will users understand the label and will they know to engage with it?

  • After the action is performed, will users see that progress is made toward the goal? Based on what occurs after the action is taken, will users know that this action was correct and help them make progress toward their larger goal?¨

Cognitive walkthrough is considered an appropriate method for testing user interfaces. Unlike heuristic evaluation, which looks in depth at general principles, cognitive walkthrough focuses more on the mental model of the user. That is, how the user interface interacts with his mental model. According to the authors’ practice, the cognitive walkthrough more closely resembles reality. Therefore, the authors prefer it as a usability test (Figure 15).

Figure 15.

Cognitive walkthrough evaluation for Zasilkovna.cz example [3].

4.10 Collaborative walkthrough

The collaborative passage is based on collaborative elements.

When author [18] constructed a collaborative usability lab at the Czech University of Life Sciences [17]) name HUBRU [18], also (J. Pavlicek) came up with the idea to implement a usability study based on the model of pair programming. The overall approach to this method is very similar to the cognitive walkthrough. The difference in principle is that a given task is performed by a pair of participants. This method, which authors (Pavlicek, Pavlickova et al. [14, 15, 16]) have been actively using since 2017, brings a number of benefits (Table 12).

The method advantagesCriticism of the method
UI/UX bugs are found faster by a pair of participants than by an individual.Opponents of the method blame the fact of cooperation. Collaboration can supposedly reduce the quality of answers. In other words, participants try less in pairs.
Authors consider this statement to be a hypothesis that has not been confirmed.
If the UI behavior is non-standard, the pair will detect it faster. If the participant is not sure, he/she can ask a colleague. However, this advice (forbidden in classical studies) is a common help when evaluating SW in real-world settings. Therefore, the First Time Experience is better recognized.
By collaborating on tasks, testing can be significantly accelerated.
The overall interview is then carried out in a group where the synergy effect becomes apparent.

Table 12.

The collaborative walkthrough method benefits.

4.11 Testing with users

It can be divided into:

4.11.1 User surveys

Potential users fill in or answer the questions in the questionnaire. The questionnaire has been prepared in advance and contains questions that are not instructional. Typically, the correct question is: “what is the appropriate background color for the XY application.” An incorrect question is: “is green an appropriate background color for XY?” This question is incorrect because it mentally disqualifies green. An even more incorrect question would be a multiple choice question: is it better A) red, B) green, C) blue? Because one intuitively foregrounds the desired answer. Keep in mind the following fact: we DO NOT WANT to force our hypothesis by questioning, but to PROVE its existence. The survey should take no more than 45 min. The advantage of this method is that it can be done remotely. The number of participants should be more than 7 (as opposed to a qualitative usability study).

4.11.2 Ethnographic observations

Ethnographic observations—Table 13

StepWhat to do
Observe users in their environment with no intervention and record and/or take notes.Find out how people use the app etc.
Best if users do not know they are observed.Ethically problematic
Contextual inquiry.
  • Ethnographic approach.

  • Shorter: approx. 2–3 h.

  • Observing an experienced user of the app and taking notes.

  • To identify shortcuts, unexpected, interesting or confused reactions.

  • The participant knows about our presence.

Table 13.

Ethnographic observation.

4.11.3 Usability engineering

Usability engineering—Table 14

StepWhat to do
Observing the users using the system in a simulated environment (usability lab).
Wanted aspects of the real world are simulated.
Users work on predefined, but realistic tasks.
The environment should be as close to the real one as possibleSame system, same monitor(s), etc.

Table 14.

Usability engineering.

4.12 Collaborative usability engineering

Variant to usability engineering. It is again done in pairs or groups.

4.13 Variants of usability testing methods

In general, there are two basic methods of usability testing in terms of obtaining valid data. These are qualitative (generally the more common method suitable for Usability Labs) and quantitative. The latter is suitable for automated testing.

4.14 Qualitative

For the purpose of a qualitative usability study (usually testing with experts), five participants are sufficient (according to the Nielsen Norman Group) [1, 2, 19]. According to Nielsen, five participants will reveal 75% of UI bugs. Increasing the number of participants further does not yield a significant result. In authors’ experience, we prefer to recommend eight participants. The benefit of eight in ideal cases is not in the ideal ratio to five ones. But occasionally, it happens that a participant is delayed or disturbed by a phone call, and therefore, in our practice a number of 8 is safer. Respectively, it does not bring a fundamentally higher burden, and the result is safer (Figure 16).

Figure 16.

How many participants is needed? [19].

4.15 Quantitative

A quantitative study is suitable for automated testing. They are generally generated from automated logs. That is, if the application collects and logs user behavior such as button clicks and user elements. This is quite common today with, for example, operating systems.

4.16 Define a story that explains the reason for testing

Tell the story of why the usability test is important. The story must be interesting and motivating for the participant to take the test.

4.17 Define a time frame

You should remember that the usability test should not take more than 30–45 min. Keep this in mind when planning scenarios. After 45 minutes, the participant will get tired. His/her ability to concentrate decreases, and the results are not relevant.

Advertisement

5. Summary (key results)

The principle of user interface design is a very artistic discipline. There are rules that partially prescribe how designers should proceed and what design patterns to follow. These are Look and Feel for different operating systems. And also, general rules for the placement of graphic elements, as the authors have already shown. On the other hand, there are market forces at work here. It is mandatory for a user interface designer to adhere to the prescribed Look and Feel. This is if he wants to place his solution in the Apple Store, for example. But at the same time, he needs to create a solution that is visually appealing and above all new. Copying the competition is not enough. Exclusivity is what sets it apart from the rest. Our users expect standard behavior from our solutions (the behavior they are used to). But at the same time, they want it to be innovative and attractive. It is called in the slang of interactional designers “sexy design.”

So let us summarize as follows:

  • Collect user requirements for future solutions.

  • Define the user tasks to be achieved.

  • Divide them into atomic user goals (UseCases). Each task consists of a series of steps-user goals.

  • Create personas or focus groups (these allow you to understand the mental model of the users).

  • Based on the archetypal users (personas, etc.) and user tasks and goals, design UseCases. Always describe these from the user’s perspective.

  • Start creating a specification in which you graphically represent the system screens-wireframes—in the form of sketches. These screens will be based on the user tasks. They will consist of individual (cumulative UseCases).

  • Write scenarios for the wireframes. Write these from the system perspective so that they always satisfy the UseCases.

  • Iterate the design process several times (e.g., three times).

  • For wire models, create a Lo-Fi or Hi-Fi prototype.

  • Create test scenarios from user goals.

  • Based on the learned rules, conduct a usability study.

  • Collect all flaws and fix the specification including the prototype.

  • In the second usability test, all the shortcomings of the user interface design should already be eliminated.

Advertisement

Author notes

Figures 1, 3, 5–7 are drawn by Veronika Pavlickova daughter of Petra and Josef Pavlicek—authors of this article. Figures 2, 8–15 are drawn by Petra and Josef Pavlicek—authors of this article.

References

  1. 1. Nielsen J. Usability Engineering (Interactive Technologies). Morgan Kaufmann; 1993. ISBN-10: 0125184069
  2. 2. Nielsen J. 1994. Available from: https://www.nngroup.com/ [Accessed May 31, 2022]
  3. 3. Pavlicek J. The Cookbook for Interaction Design and Human Computer Interaction. 2021. Available from: https://docs.google.com/presentation/d/1nbLjgEX5mS6kl_cRx6CeKuhd-fzz-kyYn_j03vMLkH4/edit?usp=sharing [Accessed May 31, 2022]
  4. 4. Jay W. Counterintuitive behavior of social systems. Theory and Decision. 1971;2:109-140
  5. 5. Hick WE. “On the rate of gain of information” (PDF). Quarterly Journal of Experimental Psychology. 1952;4(1):11-26
  6. 6. Cockburn A, Gutwin C, Greenberg S. A predictive model of menu performance (PDF). In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. San Jose, California; 2007. pp. 627-636
  7. 7. Hyman R. Stimulus information as a determinant of reaction time. Journal of Experimental Psychology. 1953;45(3):188-196
  8. 8. Rosati L. How to design interfaces for choice: Hick-Hyman law and classification for information architecture. In: Slavic A, Salah A, Davies C, editors. Classification and Visualization: Interfaces to Knowledge: Proceedings of the International UDC Seminar. The Hague, The Netherlands; 2013. pp. 125-138
  9. 9. Moggridge B. Designing Interactions. MIT University Press Group Ltd; 2006. ISBN13 (EAN): 9780262134743
  10. 10. Crampton Smith G. Available from: https://www.linkedin.com/in/gillian-crampton-smith-27930b/?originalSubdomain=it [Accessed May 31, 2022]
  11. 11. Silver K. What Puts the Design in Interaction Design. 2007. Available from: https://www.uxmatters.com/mt/archives/2007/07/what-puts-the-design-in-interaction-design.php [Accessed May 31, 2022]
  12. 12. FOX. Available from: https://www.wired.com/2014/07/homer-simpson-car/) [Accessed May 31, 2022]
  13. 13. ISO Standard. Available from: https://en.wikipedia.org/wiki/Usability#ISO/TR_16982:2002_standard [Accessed May 31, 2022]
  14. 14. Pavlicek J, Rod M, Pavlickova P. Usability evaluation of business process modelling Standards–BPMN and BORM Case Study. In: In the International Conference on Advanced Information Systems Engineering. Cham: Springer; 2021. pp. 93-104
  15. 15. Pavlicek J, Hronza R, Pavlickova P, Jelinkova K. The business process models quality metrics. In: Workshop on Enterprise and Organizational Modelling and Simulation. Cham: Springer; 2017. pp. 134-148
  16. 16. Pavlicek J, Pavlickova P. Methods for evaluating the quality of process modelling tools. In: Workshop on Enterprise and Organizational Modelling and Simulation. Cham: Springer; 2018
  17. 17. Czech University of Life Sciences, Faculty of Management and Economics. Software Engineering Department. Available from: https://www.pef.czu.cz [Accessed May 31, 2022]
  18. 18. HUBRU. Available from: https://katedry.czu.cz/en/hubru/home [Accessed May 31, 2022]
  19. 19. Nielsen J. 1994. Available from: https://www.nngroup.com/articles/ten-usability-heuristics/ [Accessed May 31, 2022]
  20. 20. Jacobson I. Use cases—Yesterday, today, and tomorrow. Software and Systems Modeling. 2004:210-220. DOI: 10.1007/s10270-004-0060-3

Written By

Josef Pavlíček and Petra Pavlíčková

Submitted: 10 June 2022 Reviewed: 23 November 2022 Published: 29 December 2022