Introductory Chapter: Enhancing Augmented Reality User Experience (AR-UX) with Design Thinking

1. Introduction

One of most amazing aspects of VR and AR is user experience that can be defined as personal or group feelings about VR and AR applications and products. VR/AR user experience is more than usability, perception or customer adoption patterns, it deals with the emotional interaction with technology as well as physical and virtual environments. Although VR/AR is mostly a visualisation technique, other senses like touch and hearing are increasingly present. One example of how user experience can drive a business model is actually the success of social networks on the Internet. Smartphones too have provided rich user experience with multimedia and VoIP applications. AR has brought richer user experience on smartphones that are accessible through the camera or a VR glass. However the resources limitations on handheld devices have prompted novel methods of computation. Extensive work exists on mobile code offloading towards a server or a cloud to minimise battery depletion. Another method is VM cloning, but despite techniques to overcome hardware and software challenges little is done in view of understanding and improving user experience. This chapter gives a brief apercu of design thinking, a technique that can enhance VR/AR applications user experience. Focus is on AR applications for smartphones.

2. Use cases

“In 2021, the augmented and virtual reality market is expected to reach a market size of 215 billion U.S. dollars.”¹ AR applications are today multidisciplinary. Below are listed some of the areas and AR use cases with specific user experiences. AR can transform one’s home interior into an archaeological site with artefacts 15,000 years back, it is indeed another level of user experience.

Gaming: Gaming experience with AR picked up with the famous PokemonGO. Candy Camera is a simple app that allows to add features to selfies. 3D animation and mixed realities combining physical environments and digital objects provide a special user experience in AR games. Players look for entertainment and sensations that are different from video games with shootings.

Education: AR applications bring new learning and teaching experiences aiming at developing critical thinking and self-learning. In engineering visualisation of artefacts with interaction is another form of instructions. Likewise the study of bones in medicine with an AR app enhances the student perspective.

Healthcare: AR is more and more adopted for patient rehabilitation and for diagnosis. Recent advances advocate that AR will be able to provide surgeons with an environment to prepare themselves prior to a surgery operation. This is an example of kind of user experience in healthcare that is unique to this situation.

Building industry: AR is widely used in building information systems, however AR is used also to safeguard patrimonies in digital format or holograms. History discovery and tourism are typical experiences from such applications.

Navigation: Popular examples are Flight Simulator and Immersion experience. In such applications there is contrast between the physical environment and virtual environment. Immersion generally is fully virtual, whereas navigating in a building or a site is about mixed realities.

VuMark: This is a product code just like the QR code that uses Vuforia technology to display additional information about a product for example when the camera of a smartphone points at the VuMark. It could provide another type of sales experience for consumers. It has been shown that pricing is not always the major determining factor, sales experience makes a difference.

3. AR apps development life cycle

Developing AR apps requires 3D Graphics and Object Design software (Example 3D MAX, a game engine (Unity3D), Qualcomm Vuforia (an AR SDK to track Image Targets), Head Mounted Display or a Smartphone with a high resolution camera). To enhance user experience, there are existing software engineering techniques like co-construction and user involvement. The time consuming activity is about design and production of the graphics objects that amount to more than 80% of a project. Google Libraries for AR objects and AR Kit have facilitated this phase.

4. Design thinking for AR

Design a way of thinking was initially meant for creation and problem solving. It is actually a combination of analytical thinking and intuitive thinking that dates back to the 1950s and 1960s. On its way, design thinking to improve UX has gained momentum. Today’s smartphone resulted from this approach. Figure 1 depicts global view of the main components of design thinking. The process is iterative and non-linear.

Figure 1.

Design thinking framework.

Inspiration: involves an accurate understanding of a problem or an eventual product where AR can add value. It is like envisioning the application even if it appears to be beyond the capacity of current development tools. Inspiration does not come out of the blue, out of the box thinking and observing nature are ways to get inspired.

Empathy: is to look at the needs of the users, not only gathering user requirements but what users would like to experience. It could be higher image resolution, larger display, and quicker response and so on. Ease of use and fulfilling needs are parts of user experience. The psychological aspect of the users towards the AR app should be captured.

Ideation: this is perhaps the most important milestone in enhancing user experience as it deals with challenges and ability to find innovative answers to the findings during the empathy phase but also the opportunities and limitations provide during the prototyping phase. Ideas come with experience or by sharing.

Prototyping: involves implementation and testing of the AR app in an agile manner. Testing is crucial in producing an application that is reliable and answers to the user expectations. The element of surprise or extra excitement is the plus in user experience, a differentiator. A survey conducted on testing of AR apps has revealed that most of the testing fields are technology oriented that are camera settings, native or hybrid app, and so on. There is little information about user experience.

Coming back to the AR development for smartphones, marker-less applications are often more challenging but appreciated compared to marker based applications. Users love the element of surprise and astonishment. But the smartphone is not suitable to plug hardware devices that can add extra sensation apart from video, audio, 3D interaction. Nonetheless, the presence of sensors and access to data from cloud puts forward mobile cloud computing as a serious option for future AR applications with new user experience. Sophisticated scripts and AI code can run on the cloud and render user information and experience in real time on the smartphone. Currently effects are on the graphical objects more than the business rules. Thus an AR developer is first and foremost a graphic designer and a multimedia developer. To further enhance UX, more data analytics is required. On the network side higher bandwidth coupled with advancement in cellular networks, for instance 5G has not yet disclosed the potential for richer AR apps.

5. Conclusion

This chapter highlights user experience component in the development of AR applications. Design thinking is proposed to augment user experience in AR applications for smartphones. It is an iterative method that can add value to the already the hardware and software challenges in AR applications. AR applications may not always be customer or user centric, so it is on a case to case basis that design thinking must be applied. Survey on the prototyping milestone has shown that in the testing phase UX is often negligible although user testing is present compared to technical issues and Empathy is not straightforward. The future of very rich AR-UX lies in mobile cloud computing. The coming chapters in this book deal with more pertinent and complex scientific issues in VR/AR, nevertheless readers will appreciate how far user experience is pervasive.