On the Role of External Representations in Designing for Participatory Sensemaking

Abstract

Public issues demand highly complex collaborations in which different (public, private) stakeholders, each with their own complementary or conflicting interests, expertise and experiences, work toward public good. Typically, collaborative technological applications function to represent people’s ideas and to enable the exchange of representational messages between people. By contrast, we designed [X]Changing Perspectives ([X]CP): an interactive table-system for multi-stakeholder collaboration around public issues. The system aims, not to represent views but rather, to scaffold the emergence of situated meaningful couplings in face-to-face interactions. It helps people to align their visual attention, materialises their input and provokes associations. However, [X]CP does contain representations, such as symbols, tangibles and an interactive visualisation. In reflecting on its design and use, we analyse what these representations do, as seen from the perspective of embodied, participatory sensemaking. We explain how representations are not the foundational building blocks of the system, and how they do not have fixed meanings. Rather, as scaffolds, our representations add a layer of artificial structure that guides the ongoing interactive couplings between people, contributing to participatory sensemaking. Applying this approach to the design of mediating technologies for multi-stakeholder collaborations can open up new ways of interacting and understanding between stakeholders without disrupting their collaboration.

1. Introduction

Public issues are complex as they cross borders of sectors and disciplines. Cross-disciplinary multi-stakeholder collaborations are needed to work on today’s societal challenges [8, 9]. Researchers in the field of human-computer interaction (HCI) are increasingly working on sociotechnical systems for societal issues in the complex context of sociopolitical multi-stakeholder dynamics [6, 16, 21]. Buur and Larssen [3] even call for designers to design ‘new formats of collaboration for large, complex contingents of stakeholders’ ([3], p. 137), and, when facilitating such collaborations, to focus on the role of crossing intentions and conflict.

1.1. Tangible mediation of collaboration

In HCI, a vast body of work [17] is dedicated to mediating collaboration. The work focused on designing interactive systems, such as tangible interfaces and multi-touch tabletop interfaces. Such HCI systems indeed can contribute to collaboration [2], for example, multi-stakeholder brainstorms [1], the creation of narratives [12] or equitable participation [20]. In existing technological mediation (tabletops), tangibles tend to represent predefined meanings or functionality.

1.2. Representation

Within embodied approaches to HCI, and in the existing body of work, a core issue pertains to the role of representation. In tangible- and tabletop-interaction designs, physical objects, visual information on or around objects, as well as interactive behaviour of such objects (e.g., flashing led-light, sounds), are primarily used as re-presentations of digital information. Hereby, the digital information in itself is also a re-presentation: it often represents the insights and ideas generated by the participants: the ‘results’ [10]. Or, in other cases [17], it re-presents the prior knowledge given as ‘input’ to the collaborative process, as it is. Representing this information on a public workspace was expected to help people to associate further on the ideas of others, to combine ideas and knowledge into new ideas, to express one’s own ideas and then communicate them, by means of its external representation, to others, and so on.

Such traditional interactive systems are representational through and through. The users of these systems are understood as cognisers: In the cognivist perspective on sensemaking, representation forms the basis of how insight is created and stored in the minds of individual users [11]. Likewise, representational messages (whether verbal, text or image) are the means by which insights get communicated between users. The task of reaching a shared understanding is regarded as an information processing task, and the system is assumed to have an information processing role: it functions to store, process and represent information to and from the user and to enable the exchange of messages between users.

1.3. Embodiment

We approach the design of interactive tools supporting multi-stakeholder collaboration from an embodied perspective. In our work, we build on embodied cognition theory, which takes an enactive view of cognition [4, 15, 25, 26]. Cognition does not happen solely in our brain, but is an emergent property of our active body as it is interacting with the world. We perceive and make sense of the world by interacting in and with it using our sensorimotor skills in active, ongoing and coupled processes of action and perception [26].

The social- and physical-context in which interaction takes place partakes in embodied processes of sensemaking, as socially situated practice theory investigates [24]. Suchman [24] argues that people’s actions are not pre-planned in their minds, but rather actions are improvised achievements guided by the material and social circumstances: situated action. More specifically, Suchman argues that face-to-face communication and collaboration activities are fundamental for sensemaking. Suchman [24] explains how people inter-subjectively construct knowledge, in the physical world as well as in social situations, and how physical artefacts play a binding role in how people create shared insight together, in action. As every person has different bodies, experiences and skills, interpretations greatly vary amongst different people. Therefore, a rich respectful exchange of perspectives is necessary to reach participatory sensemaking; people influence each other’s individual sensemaking and generate meaning in social interaction [15].

Based on the work of Suchman [24], De Jaegher and Di Paolo [15] and others [13], we regard technological artefacts first and foremost as a collection of publicly available objects that play a coupling role in skilled embodied manipulation and situated social coordination [7, 15, 19, 24].

1.4. Sensemaking

De Jaegher and Di Paolo [15] extend embodied cognition to the social domain: they take an enactive approach to social cognition. They explain that in social encounters meaning is generated in interaction between the actors. De Jaegher and Di Paolo [15] propose participatory sensemaking: ‘the coordination of intentional activity in interaction, whereby individual sensemaking processes are affected and new domains of social sensemaking can be generated that were not available to each individual on her own’ ([15], p. 497).

Joint meaning is generated between actors, in the in-between; it is not generated in each of their heads, as they cannot enter each other’s heads. In the in-between, the interaction process itself becomes autonomous: it can change the actors [15]. In other words, when people interact in a social encounter, they generate meaning that could not have been generated by either person alone and cannot be attributed to either person; a truly new meaning emerges that can change them as persons.

1.5. Design challenge: Sensemaking and the role of representation

Our design challenge was to create a working system that would enable multi-stakeholders to constructively exchange their viewpoints on real-life public issues in their cities in multi-stakeholder consultation sessions. The topics for these sessions, public issues, would be contemporary, but not concrete: they would not be about public spaces, public services or city planning. Instead, the topics would be rather abstract: how should the municipality and citizens be able to make use of publicly available data or what is needed (from municipalities, citizens, housing corporations or SMEs) to support citizen initiatives?

With that question in mind, we embarked on a research-through-design (RtD) process that resulted in the design of [X]Changing Perspectives ([X]CP). It is an interactive system that enables up to 100 multi-stakeholders to discuss and exchange viewpoints on a public issue by (re)positioning tokens with symbols on top, on round high tables fitted that track the tokens’ movements and visualise them on a screen in real-time.

As designers, we are inspired by embodied and participatory sensemaking theory. If the insight and ideas formed within a collaborative setting are not captured by and stored ‘in representations’, then the question rises whether we need any representational artefacts at all, in order to catalyse and sustain a participatory sensemaking process. Earlier work in embodied design [14, 23] shows that in design projects inspired by participatory and embodied sensemaking, representations do come into being in people’s use of such designs. While iteratively exploring our design challenge, our design decisions were informed by the design context compromising theoretical principles for working principles. We found it was helpful to create some representational basic elements within the system, in order to support the sensemaking process and trigger interactions between the participants. Our intention is to address them as sensemakers rather than cognisers. In this chapter, we therefore raise the question: what is the role of representation in participatory sensemaking in collaborations?

We describe in what way the [X]CP system makes use of representations, tangibility and spatiality to stimulate participatory sensemaking in multi-stakeholder consultation settings.

We illustrate examples of participants’ use of representation in our system and we use them to show (1) how the ground for these objects is non-representational in what they do for the participants in terms of participatory sensemaking and (2) how they are nonetheless representations. Through examples in our system, we elucidate what representations can actually do within an embodied, situated conception of participatory sensemaking in multi-stakeholder consultations.

2. Approach

Inspired by the theories outlined in the introduction, we designed [X]Changing Perspectives by taking a research-through-design (RtD) approach [18] . In an iterative design process, we developed low-fi and high-fi prototypes and deployed them in participant explorations in real-life multi-stakeholder settings. We gained insights through the materialising (prototyping) process: that forced us to make decisions: itself, but also through observations from participant explorations.

2.1. Research-through-design process

The concept of [X]CP arose in a cultural exchange of students in Sienna, informed by political history, cultural differences and the contemporary public issues in the city. A team of students and researchers designed Aesthetics of Politics [22], a tool that facilitates debate by writing down arguments on tokens and moving them around a central statement (Figure 1).

Figure 1.

RtD iterations of [X]changing perspectives: chronologically from left to right.

Inspired by the debating tool, the first prototype of [X]CP consisted of a Perspex board, flat writable circles and whiteboard markers. The concept remained similar, but this time, the participants wrote their challenge in the centre, not a statement, and the circles were meant to fulfil the challenge, or important milestones, or preconditions to fulfil it. The circles could be moved by dragging them with the markers, while simultaneously leaving a trace of the movement. In this way, the documentation of the discussion was made active (live) and analogue.

The exploration with users (multi-disciplinary neighbourhood professionals) showed that they used the traces of the pens to refer back to earlier moments in their conversation. Afterwards, however, the traces did not form a meaningful visual to them. The physical circles played a central role in sensemaking, as they invited participants to ask questions and to relate the different aspects (circles) to one another.

However, participants were hesitant to come up with new things to write down on the circles and the relative size of the board and circles did not allow for enough differentiation in positions of the circles.

These insights informed the third iteration of [X]CP, where we redesigned the circles into pillars (fitting better to the hand) with symbols (instead of blank canvasses) on top. Moreover, we scaled up from one Perspex board to 15 Perspex high tables, to be used by up to 100 participants in public consultation sessions. Participants could move the tokens on Perspex high tables, and the movements were tracked and visualised on a screen in real-time.

Participant explorations in nine real-life multi-stakeholder settings showed that the symbols triggered participants to share their primary associations and this started a lively exchange of viewpoints.

In the final iteration of [X]CP, we refined the prototypes and evaluated the system in a participant exploration with five tables.

In what follows, we describe the design’s characteristics in relation to our theoretical frame as well as observations of the usage of the system in a real-life multi-stakeholder consultation session. We conclude with insights on the role of representation in designing for participatory sensemaking.

2.2. Design

Based on our theoretical frame, RtD iterations and earlier work [14], we designed the final version of [X]Changing Perspectives to invite embodied interactions in discussions between stakeholders with the aim of contributing to participatory sensemaking between them.

2.3. [X]changing perspectives system

[X]CP consists of a technological system, a moderation format and a service system. In this chapter, we focus on the usage of the technological system by multi-stakeholder participants, and do not describe the moderation or service around the system.

The technological system consists of 15 discussion tables with integrated camera tracking hardware and visual computing software, see Figure 2. On each table, there are six tokens that are identified by coloured LED light and a symbol on top (as in iteration 3), and are tracked by unique marker patterns on the bottom. The symbols, a bird, Euro sign, a gift box, a wound-up puppet, puzzle pieces and a clock with arrow, were inspired by literature on hurdles in citizen participation [5] but were not inscribed with specific meaning: on the contrary, they were intended to freely associate with.

Figure 2.

Elements of the [X]CP system, f.l.t.r.: real-time visualisation, table with tokens, symbols on tokens, tracking hardware and token hardware.

Participants stand around the tables and discuss a central question, placed physically in the centre of the table. They do so by associating with the symbols on the tokens, and positioning the tokens in a meaningful place on the table, creating a shared landscape of meaning generated on the spot. Intentionally, neither symbols nor tokens or table surface positions have pre-defined meanings or terms of use: the participants at each table generate their own meaningful use of the objects. The symbols can be used to associate content with tokens and the table surface can be used as a scale of importance, where the most important tokens are placed in the middle and others in the periphery, or where the periphery can be used to place pre-conditions for the tokens placed in the inner ring.

While positioning and repositioning, stakeholders exchange different associations and together generate and reshape meanings of the tokens.

The tokens’ (marker) positions are tracked by the tables and represented in real-time on a big projected data visualisation. The visualisation shows a helicopter view of the movements of all tokens at all tables and allows filtering between them, to discover patterns in movements, relative distances, centrality on the table or amount of touches. By showing alternative views of—and relations between—all the table landscapes, the visualisation aims to support a collective reflection between participants of different tables. The visualisation alone does not represent the meaning generated at each table: the meaning forms in-between the participants and as such cannot be captured by the visualisation. Instead, the visualisation is intended to provide a mirror and trigger reflection between table groups.

The role of representational elements in our system is to invite interpretations and associations, rather than to express predefined, instilled meaning.

3. Participant exploration

The [X]CP system was developed in three research through design iterations in which we meticulously tested the technological functionality and evaluated the interaction and usage patterns in participant explorations in nine real-life multi-stakeholder sessions. Implementing insights from each iteration, it was recently prototyped as high-fidelity final design. At the time of writing, we have had the opportunity to implement the final system in one real-life context. In this section, we describe the context, set-up and findings of this first participant exploration with the latest prototype of [X]Changing Perspectives.

3.1. Context

The session was part of a congress about the increasing availability and usage of data for Dutch municipalities. The total of 30 attendees consisted of alderman, civil servants, policy makers, members of the city council and entrepreneurs. The central question was: ‘what is needed in order for the data-driven municipality to work in a good way?’

With the exception of entrepreneurs, all participants worked in municipal institutions. Even though they had different stakes in the discussion, this was an important limitation in participant composition.

3.2. Set-up

The session consisted of five tables and the participants were distributed over the tables so that there were six participants (unacquainted with each other) with different stakeholder roles at each table. The session lasted 50 min: two discussion rounds of 15 min separated by a collective reflection of 10 min. A wrap-up of 5 min concluded the session (Figure 3).

Figure 3.

Partial overview of session set-up.

4. Observations

Substantiated by patterns in observations of earlier sessions, we use examples of the latest participant exploration to illustrate our observations on the role of representations of the [X]CP system in sensemaking processes between multi-stakeholder participants. We describe our observations in three categories: the interactions invited by the representational elements of symbols (1), tokens (2) and visualisation (3).

4.1. Symbols

In this section, we highlight some observations that elucidate the role of the symbols in participatory sensemaking during the use of the [X]CP system.

4.1.1. Symbols trigger primary associations and open inquiry into differences

It was easy for participants to associate with the symbols: they shared their primary associations with the symbols, which was often telling for their viewpoint or background. For example, participant A (entrepreneur) placed the bird-token in the centre for ‘citizens’ autonomy over own data, see Figure 1, and participant B (alderman) reacted ‘oh, it’s funny you said that because I would place it in the centre too, but to me it stands for overview: I think that we [municipality] should monitor the data that we have of the city’. Afterwards, another participant joined in by placing a new token on the table and relating it to the first two interpretations. As the tokens were repositioned, the conversation evolved and their meaning evolved (Figures 4 and 5).

Figure 4.

Participants discuss the firstplaced token.

Figure 5.

Participant C pointing at the ‘last’ token.

In this example, the symbols were used for associating, and at first instance, represented something unique for each of the participants. One striking observation was that symbols functioned as social mediators offering a non-offensive motive to question each other without eliciting a defensive response: indeed, using the symbols as ‘neutral’ objects, people could attend to helped to catalyse an ongoing exchange of associative conversation in which different perspectives, personal experiences, anecdotes and ways of reasoning were shared, something that participants told us does not usually happen in such settings.

4.1.2. Symbols carry dynamic attributed meaning, constantly altered through interactions

After the placement of a first token, the other participants joined in and shared their associations, sometimes adding other tokens to the table. The different associations with symbols were the beginning of a participatory sensemaking process wherein different meanings and relations were discussed and changed on-the-fly. For example, the symbols turned out to be used as on-the-fly generated representations of values, bottlenecks or goals. Meanings changed while interacting physically with other tokens (repositioning) and other symbols (pointing, orienting other agents attention [13]) to compare their meanings. In doing so, the participants generated new meaning together.

4.2. Tokens

In this section, we highlight some observations that elucidate the role of the symbols and token in participatory sensemaking during the use of the [X]CP system.

4.2.1. Tokens allow for intuitive expressions

For example, on one of the tables, five of the six tokens were positioned and one was left on the side. When the moderator announced that there was only 1 min left, the untouched token gained an interesting role. Participant A said to her table: ‘OK we should add the puzzle piece!’. ‘Why then?’ asked participant B. Participant A, humorously: ‘Because we do not want to leave it alone and exclude it, it would be sad… and-’. All laughed and then participant C stepped in: ‘actually, for the puzzle piece, you know that when you put tech-guys together (…)’ and he enriches their landscape with a new relevant meaning, that was only possible because the neglect of the token was physically visible—its physical distance to the other tokens bothered participant A. In other words, the physicality of the token invited to share a feeling, a line of thought that may have not been shared otherwise.

The physical presence of the tokens changes the way of interacting with each other. Intuitive expressions come to the fore, verbally, when moving them physically. Body language seems to be amplified, as the tokens afford different interactions that could communicate something to the other participations. Three examples of such communications were: (1) gesturing around tokens to indicate their preciousness or (2) tapping on the tokens to highlight their importance or to communicate that they should be related to the current conversation topic or (3) ticking or drumming around the token on the table surface to communicate one’s interest to speak next (Figures 6 and 7).

Figure 6.

Pointing and repositioning a token to relate it to the others.

Figure 7.

The visualisation is used to reflect across tables.

4.2.2. Tokens lead to relations between discussed elements (multidimensional image)

The physicality of the tokens also means that they are physically positioned ‘in space’, on the table surface. Naturally, after symbols were given meaning, the connections between tokens were discussed: where should it be placed, closest to which other token, or how does one relate to the other?

The meanings were not limited to definitions of symbols; instead, they were narratives of arguments, anecdotes and interests that were brought to the table by all participants. The eclectic or even conflicting input was not brought to a ‘safe middle way’ or consensus; instead, the input was tied together as a story, supported by the physical token positions in space, the invisible traces on the table that the visualisation made visible through the digital representation of movements.

This was evident during the collective reflections, in which participants explained their landscape as a holistic story, of which the separate elements or symbol meanings could not be attributed to any participant alone anymore; they had emerged in the interactions between the participants. Moreover, the symbols functioned not only as external placeholders/representations of one (shared) meaning, but were continuously altered through ongoing interactions and in relation to other tokens.

4.3. Visualisation

In this section, we highlight some observations that elucidate the role of the visualisation in participatory sensemaking during the use of the [X]CP system.

4.3.1. Visualisation invites taking a new perspective

By showing the same view of all tables, the visualisation (Figure 5) enables the participants to relate their landscape to that of others. Initially in the collective reflection phase, participants were excited to see what the ‘technology’ would show them. Soon, however, they realised that without participant’s explanations, the visualisation had no meaning at all. Together, the moderator and participants could discover patterns in movements of specific tokens but what could those movements mean? The moderator invited several tables to explain their landscapes, to give meaning to the visual representation on the screen. Participants were very curious to hear the stories and generated meanings of the other tables’ landscapes. Moreover, they reacted to the explanations when a statement was made that connected to their discussion by giving a shout-out to share their views on it.

5. Reflections

Our observations of interactions with the [X]Changing Perspectives system shed light on several roles that the representations played in participatory sensemaking processes. Perhaps, somewhat contrary to our initial focus designing for the non-representational aspects of interaction, reflecting on the design case brings forward that representations have an important role to play in participatory sensemaking. However, observations show that representing information is not the primary function of the table—what we see is that representation forms an added ‘scaffolding’ layer that enhances the capacity of people to engage in a face-to-face, situated process of participatory sensemaking (see [11, 21] for related views). The representations we used mostly function to provide:

1. a layer of playfulness that breaks the ice;

2. a layer of associations that structures interactions while leaving open interpretations: ‘social-embodied scaffolds’; and

3. (tactile as well as digital) visualisations of conflicting interests that make them discussable.

As our design was the vehicle that allowed us to observe the interactions, we are able to move beyond the description our intentions (as we did in the beginning of this chapter) toward pointing to the characteristics of the representations that supported the sensemaking processes in our design case. Our main reflection is that the [X]Changing Perspectives system provided a scaffolding structure for sensemaking, and to allow this we needed a careful balance between ‘structuring’ representations with open interpretations (point 2 in the list above). For example, the symbols triggered primary responses (structuring the interactions) from participants, disarming them, taking people out of their ‘labelled’ role and engaging them as whole person, with experiences, emotions and creativity next to expertise. The symbols, however, were not pre-defined (open interpretations): they were asked to be interpreted by the participants. The same applies to the table surface: it provides structure in the sense that it frames a circular area, and it defines the proximity of participants standing around it, but it does not provide a structure for positioning tokens. At the same time, it does imply a structure due to the central question placement.

Reflecting on those examples, we regard the value of representations for participatory sensemaking processes to be in the balance of providing representational elements such as structure, while at the same time leaving open what they stand for and how they could be used.

The role of representations in collaborative sensemaking is especially interesting in the context of multi-stakeholder collaborations and consultations regarding public issues. Namely, in this context, the topics are often highly abstract, formal, and relate to different disciplinary expertise as well as corporate interests and different emotional or otherwise engaged interests. Structuring the dynamics between these interests in relation to an abstract topic is a complex task. The [X]Changing Perspectives system demonstrated that representations as social-embodied scaffolds can make (open-up) embodied, intuitive and personal interactions (leading to participatory sensemaking, to the shared generation of new understanding of the topic) approachable without resulting in discomfort, conflict or abstract meta-discussions.

Acknowledgments

We thank the participants and organisers of the sessions. We convey our thanks to RI.SE Interactive for the development of the software of the [X]CP system. We thank students Bastiaan van Hout and Tom van Rooij for their assistance in prototype development and camera work. We also thank our partners Necker van Naem and the municipality of Eindhoven, and particularly, we express our gratitude to the municipality of Eindhoven for their funding support.