Facing Digital Dystopias: A Discussion about Responsibility in the Design of Smart Products

Abstract

The paper investigates some critical issues connected to the digitalization of products and systems for the domestic environments involving the collection of personal data. The research focuses on the most innovative solutions, such as those based on AI algorithms for speech recognition, IoTs, wearable devices, cloud computing, and the use of smart phones and devices. These solutions require and imply the collection of personal data and their local or remote processing. The paper provides a design-oriented discussion on the features of smart products with respect to the consequences of design choices on complex dimensions of experience such as sense of self, privacy, and personal identity. The paper aims to set out the terms of a discussion about the most critical factors of services and systems involving personal data, and to create references on the responsibilities of designers acting in multi-disciplinary project teams. The research is based on ethnography at home and on a critical discussion about case studies. The results highlight the importance of considering privacy and control issues in the design of smart solutions and provide some pointers to be used in the development of smart solutions for home.

1. Introduction

The evolution of IoT technologies, of AI algorithms, and of cloud services allows the design of responsive products and environments, and supports the development of new approaches to the design of interactive solutions in every domain of application: human-machine, human-environment, and human-human systems [1].

These innovative solutions provide several valuable advantages for users, in terms of convenient functionalities and improved modalities of interaction between human users and technological systems. Many products and services aim to support users in the monitoring of personal well-being conditions and lifestyles, and to improve the modalities of interaction between users and the environment through the development of new control paradigms. Most of these products rely on the collection of data regarding the users and the contexts, and on their shipment to remote service providers, allowing post processing and final delivery of suitable information or other performances. Cloud data storage services, APIs offered by big companies such as Google and marketplaces such as Mashape, AI agents and other ‘pieces’ of technology, appear today as building blocks that designers can use to create useful interactive solutions for valuable experiences. Indeed, these creative opportunities are without precedents and open a new era for human beings. A new era, which as many other dawns before, carries shadow within as well as the light of new interesting and useful possibilities.

As designers, researchers, and educators in the field of interaction design [2, 3], we consider the recent evolutions of digital technologies as a great opportunity to invent/design valuable products and services. On the other hand, the creation of innovative solutions based on IoTs, on remote processing of data, and on cloud resources provides new challenges for designers. In terms of continuous updating of competences and techniques, the ability to deal with the progressive dematerialization of the object of design from fixed carrier to network and services [4], with a growing complexity of the designed systems also requiring the ability to re-define the role of designers in multi-disciplinary project teams. Furthermore, due to the dimension of the impact of digitalization on private and public aspects of life, we believe in the importance of developing a critical thinking and discussion on ethical issues emerging when dealing with radical innovation. We aim to entice the development of full awareness about the design factors that determine the acceptability and desirability of a product.

In this paper, we focus on the latter of these challenges. We intend to give a contribution to the related discussion by investigating the relationship between the physical and interactive features of the so-called smart products based on cloud computing and on responsive remote processing of data, and the dimensions of personal experience involving the sense of self and the sense of privacy. In our research, we investigate the topic in a wide perspective through a variety of different applications. We focus on products and services designed for domestic environment, the production, and use of personal data from a design perspective. We consider different applications: systems for home automation, devices to centralize the control of domestic appliances, and solutions for monitoring health and personal status.

2. Framing dystopias

The importance and value of data as “the new raw material of the 21st century” is now a common knowledge [5]. Defining Personal Data as “data that relates to a living individual who can be identified from those data, or from those data and other information” [6], we can divide information in to two categories: direct and processed. Direct information is the information extrapolated directly from the data. It is detected as it is and can be used as single value for control and input. Processed information is the value derived from a data processing that crosses more than a value (e.g. heart bit rate and age of the subject).

Indeed, the potentials of personal data collection is now amply recognized as a source of opportunities to generate new services and products, also to produce long-term consequences on the organization of main structural social assets, such as those concerning health care services, energy and resource management, public transportation facilities and so on [7]. On the other hand, dealing with personal data reveals critical issues involving safety, security, privacy, and others [8, 9]. Indeed, through the construction of digital networking of objects, spaces, and people, we are creating the new hybrid digital/physical environments where we will live in the future. Therefore, in the design of smart solutions, we face a double challenge: on one side, we must design safe solutions, protecting users from risks of fraudulent abuse of their data and of conditioning based on social engineering manipulation. On the other hand, we should avoid the risk of producing unwanted consequences on social and individual attitudes, freedom and mental frames through the shaping of interactive processes inadvertently inducing conditioning or limits. The physical world’s way of saying “The tool shapes the function” is already on the digital domain. We receive suggestions by recommender systems according to our preferences [10], we are invited dressing clothes suggested by algorithms [11], we maintain friend’s contacts under the influence of systems analyzing affinities among people, just to mention a few cases. In the current times, we no longer rely on those big social institutions (such as political or religious ones), not as much as we did in the previous centuries [12]. Automatic personal data analysis and usage present, in our opinion, the risk of shaping our life as single individual and communities, in a phenomenon that designers must be aware of.

The management of critical factors requires the joined efforts of three different domains: regulatory laws, responsible technology, and design. Regulatory laws producing principles and norms regulating the right to privacy of citizens and identifying responsibilities. Responsible technology producing protocols and technical solutions for the suitable management of data and of their destiny in the meanders of the cyberspace. Lastly, design, ideating and creating products, services and interfaces enabling the rightful control on the personal data acquired by services.

In this paper, we focus on the last one, and we argue that designers should develop a better awareness of the importance of personal data’s correct handling in the development of products and services, especially for those meant for domestic use, due to the importance of home being (and being perceived as) a private and safe place, as treated later in more detail. Furthermore, we believe that the responsibility toward the correct use of personal data is one of the constraints that should be considered in the design of smart solutions, both to create suitable products and services complying laws and ethical principles, and to produce awareness.

3. Investigating attitudes and feelings through ethnography at home

In springtime 2016, we involved students (almost 100) of a university course in Interaction and Experience Design at the School of Design at Politecnico di Milano, in an ethnographic investigation of domestic environments. This activity produced a high number of cultural probes documenting behaviors, habits, rituals and attitudes at home, concerning people of different age, profession, and social conditions. The investigation has been part of an educational process including lectures on design for experience, and on emotional and cognitive processes involved in actions. It was not directly aimed to the design of a product or service, on purpose intending to demonstrate the suitability of aimless ethnography to provide suggestions for the creation of technology-based interactive solutions out of existing schemes. The observation on field focused on several daily tasks taking place in the domestic environments, such as entering/leaving home, entertainment and leisure, getting ready to go to bed, and so on. The collected documentation reports about feelings, expectations, values of inhabitants, and in the whole, it provides several hints toward designing suitable interactive services and products [13, 14].

With respect to the focus of this paper, the investigation produced some interesting results that can be summarized in the following statements. In our country, most people see home as a shelter and a space of freedom; a repository of most beloved objects and memories; a place to develop the most intimate relationships and where it is possible to develop a personal lifestyle. At home, it is possible to express personal esthetic preferences and values; the formal and functional organization of the house plays an important role in the definition of the sense of self.

The practical organization of objects and activities is tightly related to personal preferences and to those specific models ruling social relationships among the ones sharing the house. Part of observed people appears willing to have a tight control of the functional and esthetic settlement of their private environment; furthermore, they pay attention to their privacy and feel anxiety related to the moments when they leave/enter home.

The survey documents a variety of attitudes and behaviors, but also stresses the relevance of the topics we are focusing on, such as safety, security, and privacy. Privacy is a complex subject, since what we intend with this term varies with each person and each social environment. It also changes with time. Each single individual’s life moment and each social era have different privacy perceptions. It is also tightly connected both with personal values and cultural belongings, as well as with other feelings and attitudes such as personal freedom and sense of decency. Digital technologies make the meaning of privacy evolve, and pose new challenges and questions in terms of civil laws concerning ownership.

In order to design acceptable and desirable home automation solutions, designers should consider reliability of products and services toward privacy needs a primary issue, to be managed through suitable design choices.

4. Personal data and information in everyday objects

We, as human, live and act in the space around us. Smart environments, smart and wearable objects, and IoTs detect precise information about us and about our behavior through sensors’ data. These products have become part of people’s contexts as everyday objects in the private field, at work, and in the different outdoor activities. They represent an evolution of the use of the everyday objects, driving technology, with its power and possibilities, to be imperceptible. Smart and reactive environments are made not only by integrated systems, but also by everyday objects that evolve in nodes of an interconnected network collecting, storing, processing, and spreading information. The use of these objects creates a set of personal information that is useful for various functions of the object itself and for the integration of new functions and services as well.

Although this information is undoubtedly useful for the operation of proactive and intelligent systems, it is necessary to face the effects on user’s sensitivity and awareness concerning the collection and spreading of information about them.

We intend personal identity as the set of stigmas that allow identifying a person through their characteristic elements, peculiarities, and traces in space and time [15, 16]. The use of personal data arrives with multi-level impacts. Concerning privacy and security, the simple use of connected cameras allows the detection of personal identity [17] through facial recognition and tagging [18]. Combining this recognition with a position triangulation (e.g. based on a GPS), it is possible to associate the person to a position in space and time. Concerning the perception of self, most people nowadays are faced with identity doubling. In one hand the physical everyday life, on the other hand the digital identity that include the sum of all the information about an individual that are digitally available [17]. A scenario presenting a further complexity layer, due to the possibility of a single person having multiple digital identities; social media, MMORPGs and other community-based system allow and encourage users in creating profiles which often can grow the features of self-standing personalities. It is the well-known concept of the role mask, extended in the digital field, in which our roles might not be easily traced back to the original human beings [19, 20].

As designers, we embrace the creative potentials connected to several dimensions of existence made possible by digital technologies. On the other hand, we recognize the control of one’s personal information as a natural and universal right of the ones producing them, and dealing with the design of domestic smart appliances, we consider it as a primary issue.

5. The intimate automation

The collection of data through the use of wearable devices such as smart-watches, or through the sensors embedded in most smart phones (accelerometers, audio and video recorders, light sensors, GPS, and so on), allow the processing of personal information on behaviors and contexts, producing valuable and affordable tools to monitor lifestyles and personal health conditions. The sensors integrated in devices as Fitbit for lifestyle, Beast for sport performance, and MedicAlert bracelet for epilepsy monitoring [21–23] allow these devices to constantly detect biological parameters and process data to create value for the users. These systems provide the opportunity to record information to be shared with caregivers, but also to obtain immediate feedbacks thanks to online processing of personal data and the use of suitable algorithm for the modeling and interpretation of activities and contexts.

This approach to the use of personal data on devices and IoT solutions is not strictly related to health monitoring: it is currently widely spreading in many fields of application, such as sports and work performance.

Analyzing the home automation products and services, the personal data stand out as a sort of raw material to be analyzed, processed, and crossed with other data aiming to enable functions and proactivity of the system. The user can perform different actions through the use of automated systems in the intimate context of the home. Actions spanning through the precise control, monitoring, pre-set of single activities or functional scenarios for the home ambient. The products used in the home context are not only related to the automation of home appliances or integrated systems. There are plenty of single devices that perform single or multiple functions for well-being, lifestyle, and personal health of the people living in the house. Despite some of these solutions are made to be functional in everyday life, also out of the home environment, our aim is to analyze them in the intimate context of the home to understand how they are perturbing the intimacy by managing of personal information. Personal data are collected by the devices both using integrated and connected sensors that detect precise values directly and using other devices and appliance to receive second hand information on the users’ habits and behaviors.

6. Personal data as meaningful material: existing solutions

Picking devices and systems examples from market and crowd funding, we aim exploring the current methods and possibilities offered by home automation solutions concerning use of personal information and data. We consider the two main branches of home automation solutions that use personal data while controlling and managing devices and appliances. The first one includes applications that aim to provide and monitor health and well-being. The second one includes applications that aim to simplify and help in everyday and occasional tasks.

The health monitors and well-being providers such as Cubile Health and Sleep Cycle Alarm Clock [24, 25] are focusing on physical data collected directly from integrated sensors. These self-tracking devices can solve problems, identify glitches, or encourage habits [26]. Single values are crossed with other data to process information creating usable value. These applications provide information mainly about the span between the user value and the “normal” range. This normality is decided by designers focusing on the users’ average or provided guidelines. The user is not always aware of what is the normal range and how it is decided. Cubile Health focused on collecting, monitoring, and comparing personal data to ideal standards during the night. While the elders’ or ill people’s data collecting can be useful for a constant monitoring of parameters to be sure they stay between safe ranges, a less motivated monitor may lead to a distorted perception of self in healthy people. The device can tell me that I did not sleep well comparing my data to a non-specified (to the user) standards without consider a wide range of different parameters that go beyond the generic age and gender. Sleep Cycle Alarm Clock app have a very similar data managing procedure returning information about the amount and quality of sleep comparing the user’s data to a regular sleep cycle. However, the feedback graph shows a sleep pattern that is meaningless to users unless they are told how to interpret it. This analysis reveals two main issues related to the personal data collecting, managing, and visualization. Do the standards fit with user’s specific and personal parameters? Are the feedbacks showing their real meaning according to the user’s interpretation skills? Designers must consider that feedbacks from these sleep monitors (as an example) can perturb the opinion users have about themselves and make them feel sleep deprived even if they are not, but must also consider that users might misunderstand the meaning due to a lack of interpretational skills.

The controller devices and appliances aim to simplify users’ task and to optimize time and efforts and they collect data directly (from integrated sensors) or through other objects and appliances. Whether they are single devices or integrated systems, they usually merge and cross different data to create value for the user performing task managing through direct control of appliances and devices or giving useful information and feedback to the final user. Modern integrated systems for home automation can control every connected device (e.g. KNX systems) and let the house act as a semi-autonomous agent acting accordingly to given presets, customized settings or even user’s habits. Home automation integrated system as well as single devices mainly collect data from user’s behaviors through objects using both integrated sensors or simple operation habits (e.g. timing and frequency of switching on/off for a single device) to permit customized objects’ performances. Single devices as the Samsung Family Hub 2.0 [27] smart fridge collect and analyze users’ foods and shopping preferences to manage purchases, storage, and monitoring actions for the users. These data reveal behaviors of the user to perform a task optimization and to give proactivity to the device. Control hubs as Amazon Echo [28] constantly monitor the environment to detect users’ needs and use AI to perform personalized tasks or suggestions accordingly to his ‘perception’. An Experian survey [29] shows that Echo users are likely to place the device in the kitchen. This is probably because it is easily accessible with occupied hands, so the voice recognition used as task manager (smart light and thermostat control, add item to a shopping list, provide information, play music, and so on) is probably the key of the success of its AI Alexa [30]. Amazon Echo uses on-device keyword spotting to detect the wake-up word. When the device detects it, it streams audio to the Cloud, including a fraction of a second of audio before the wake word. The use of voices as a control tool represent a change of paradigm in device control: natural language controls home environmental changes so that the control of the actions is still perceived but no longer tangible. As the provided definition includes the voice recordings in the personal data, with a natural language recognition AI, the user is less aware of the collecting, processing and storing of his personal data. Alexa is an AI made to be perceived as a personified entity and the interaction with her is made to be as similar as possible to a human conversation.

Whether we divided cases in the two branches of use of personal data, some of these solutions actually belong to both, presenting a data usage dual mode. One of these is Moodbox: it can simply react to movement, switching on when somebody walks across its monitored space, but also crosses data from voice analysis and favorite music, weather conditions and time of the day, using artificial intelligence to suggest music that the user will probably like being in a particular mood. All these systems and devices learn from user actions and try to anticipate them, letting users free to use and manage their time in a different and maybe more important way.

While analyzing these cases, an issue arises: is generated info value meaningful enough to absorb the privacy perturbation? The resilience of a technology can be a discriminating factor to its adoption. The real value for the user, being it monetary or service, must compensate not only the effort employed in the use of the device, but also the perception of ‘loss of control’ over their personal data (Figure 1).

Figure 1.

Personal data management and use in the analyzed home automation solutions.

7. Design principles in personal data handling

The reflections above reported lead us to define pointers we wish to discuss within the communities of designers of interactive products and services. The topic is vast and complex, and, due to digital technologies rapid evolution, it requires an open discussion capable to face the continuous ongoing change.

As temporary principles, we assume that, in the creation of interactive digital products and services involving personal data, designers should:

• Provide users with both a full and explicit information about the ways their personal data are employed by the product/service along the whole chain of exchange of information by different stakeholder and a meaningful payback about the data collected, giving them also the capability of properly understanding the content’s value. That should be done considering both their interpretive skills and the possible short and long-term perturbation.

• Provide sufficient protection toward improper or unjustified use of personal data by third parties aiming to fraudulent actions or to exert persuasion through tailored advertising.

• Make people aware of most common social engineering use of personal data.

• Avoid reduction of resilience and robustness of home technical systems involving cloud computing and data processing, and due to the aleatory performances of the internet and to the characteristics of technological devices, such as smart phones, presenting intrinsic risks in terms of fragility and the possibility to be lost/taken.

• Provide usable and accessible means for users to enable/disable the data gathering, and to simplify the control and management of the data concerning the persons and the environment, that are exchanged by the product and system, in any time and any context.

As a last inducement, we suggest to consider as a natural human right, the possibility for the user to ‘negotiate’ the personal data as a matter of design: if data is framed today as a resource and a capital, everyone should be allowed to manage their own as property.

Acknowledgments

This work has been partially funded by TIM S.p.A., Service Innovation Department, Joint Open Lab S-Cube, Milano, Italy.