Unveiling the Expressivity of Complexity: Drifting in Design Research

Abstract

Design research is regarded to be a mode of inquiry particularly suited to engage with complex topics. In our work, we are interested in unpacking the complexity at the heart of an embodied aesthetic experience. In this article, through our digital and physical artefacts and a methodological reflection, we illustrate an ongoing design research project that a multi-disciplinary team of interaction designers, professional dancers, software developers, artists and 3D modelling experts are carrying out to develop insights on how to understand this complexity and how to use such insights as inspiration for interaction design-related projects. By embracing combinations of design, new technologies and simple visualisation tools, the project investigates the complex and hidden expressivity embedded in the skills of dancers in a programmatic design research approach. This investigation leads to insights on different levels. Firstly, cycles of formulation, realisation and reflection on design programs express parts of this complexity and this lets new research interests emerge. Secondly, as a body of work, reflecting on these cycles exposes how our “drifting” within this programmatic approach has started to unveil the complexities inherent in our research program. In this article we aim at contributing to the growing understanding of what designerly ways of knowing might be and how a practice aimed at expanding and contributing such knowledge unfolds.

1. Introduction

Our approach is based on constructive design research [1]: research based on design action that builds things, with some form of reflection or evaluation on that action that generates knowledge. In previous work [2, 3], we employed this approach to design and build prototypes aimed at articulating how certain qualities may elicit an engaging aesthetic experience in interaction. In the project presented in this publication, we employ a similar designerly approach in a different way. Instead of developing and implementing our understanding of qualities in designing for an aesthetic experience, our intention here could be considered the opposite. Here, we aim to use our skills, interests and technologies to design from an experience—defined by the whole of its aesthetic qualities—in order to gain access to its complexity by expressing and amplifying the qualities we find in it.

The qualities of the dancers’ experience are revealed in this process through cycles of reflection-on-action, where the action is constituted by different ways of making elements of the dance explicit and the reflection happen as a joint dialogue between designers and dancers. Such qualities are subsequently explored through several individual design research projects, in which we transpose them into other instances, where design becomes a tool to better understand them. These design projects follow several different avenues of exploration, with different design teams, within different contexts, and they exist in varying states of completion. We ask the reader to bear with us, since we will explain in detail and with clear examples, what we just mentioned in a generalised way.

We report on intermediate results of the project here because we believe they are valuable to the Interaction Design Research community on two levels: firstly, the various avenues of exploration that have emerged form a palette of different opportunities in which an embodied perspective on the aesthetic experience can become relevant in other design projects. In this sense, what has emerged from these different individual projects so far is not a knowledge contribution per se. It rather constitutes emergent design programs that are being further developed in parallel. This palette of opportunities is illustrated in the fourth section of this paper.

Our focus, in this publication, is thus less directed towards the knowledge of aesthetic or movement qualities that these individual projects have generated (or are generating) as isolated studies. Our main intention is to present a wider view, to frame and place these projects from a methodological perspective: as a body of work that explores the complexity of experience in dance through design. As a body of work, this brings about new design programs. The overarching project, in its current state, offers an opportunity to be used as a research vehicle to expose and reflect on our programmatic design research approach, that is, investigating the complexity of experiences. The work provides different perspectives on this complexity when framed as a programmatic design research approach. In reflecting on the program/experiment dialectics [4] that guided this research project, we aim to contribute to the growing understanding of what designerly ways of knowing [5, 6] might be. Or, more particularly, what ways of drifting there might be in design research [7, 8] and how a practice aimed at expanding and contributing such knowledge unfolds.

2. Related work and scope

Building on an embodied perspective [9] the importance of movement for interaction design is clear and the community has produced a variety of different approaches, methods and tools to develop this importance. In particular, many approaches and methods have been developed to use dance, as a way of moving that is expressive and has aesthetic qualities, in embodied interaction design research (e.g. [10–13]). Instances of work more directly related to our research program here are projects by Silang Maranan et al. [14] and Alaoui et al. [15] that also aimed at visualising and understanding the qualities of movement in dance using interactive technologies. Other related works that capture, translate and transpose qualities from movement into visualisations or materialisations, include both artistic (e.g. [16–18]) and academic (e.g. [19]) projects. Similar to this work, our intention in this research program is to use our design skills to create prototypes that emphasise and amplify the qualities hidden within the complexity of dance.

Similar to our other design research work, this program builds on theoretical foundations that include ecological perception [20], phenomenology [21] and embodied cognition [22]. These foundations share a notion of embodiment: we experience and make sense of the world subjectively, through our bodies, by acting in the world. Meaning is released in dialogue and so our experiences become ungraspable, ephemeral and dynamic. A phenomenological and embodied perspective on our experiences with the world thus stresses the inherent complexity of such experiences. In this publication we aim at exposing how our drifting has started to unveil the complexities inherent in this research program.

3. Methodological framing

Since Frayling [23] coined the term Research through Design, many scholarly efforts have been made to articulate methodologies, strategies and approaches as to how this form of research can generate knowledge. Koskinen et al. [1] aimed at specifying such ideas further, proposing the notion of constructive design research, where the construction of artefacts or interventions takes centre stage as the source of new knowledge.

A particular approach to articulating how knowledge may be generated within such a research endeavour is programmatic design research [24]. The program in this case is a provisional knowledge regime, a statement that acts as a lens through which to view the research interest. This statement proposes handles on how and where to start designing. An experiment then can be a designed artefact, intervention or even proposal, that brings (parts of) the program to expression. In a dialectic process, the researcher repositions herself between program and experiment: both program and experiment mutually influence and sharpen one another [4].

3.1. Research cycles

When we cast our own research work in a programmatic approach, we can make a distinction between the research program and the design program. The research program formulates an intention and area for an overarching research interest. Design programs are particular instances of that research program more directly related to the design experiments within a project. The research program for the projects presented in this publication is to use the forming language of design to amplify and express the (hidden) qualities and intentionality embedded in the skills of dancers. The research program is grounded in theory and practice: it builds on a embodied and phenomenological understanding of experience and points to a design space: expressing the qualities in dance. This design space is always in flux, as it is impossible to fully grasp. Yet, it can be seen as an encapsulation of complexity on a certain level of abstraction (see Figure 1).

Figure 1.

The design space as existing between increasing levels of abstraction and complexity: as a metaphor for the potential of possible designs, it is infinitely complex, as the subject of a research investigation it is limited by the particular design domain that is of interest.

From the research program, we can formulate a design program: the starting point of a design research project (see Figure 2). The design program provides a particular perspective on the design space providing handles on where and how to start designing. The design program is realised through design experiments (see Figure 2). The design experiment is engaging directly with the complexities of the design space. They extend outwards, towards abstraction, as they gain value in relation to the design program through reflections. Individual reflections exist on a certain level of abstraction but stay connected to particular experiments. Reflecting on these results in relation to the design and research program, allows for the formulation of a new program. Cycles of formulation, realisation and reflection on design programs express parts of the complexity of the research topic and suggest where further expression can be found [25].

Figure 2.

The three stages of a cycle: formulating a design program (left), realising the program through individual design experiments and reflecting on individual results (middle) and reflecting on the results and program (right) to gain a new perspective.

In this publication, we present and reflect on what we have learned about our research program using this approach. Our focus here is not on describing in detail the lessons learned from each individual cycle, nor on presenting a highly precise theoretical model describing all the complexities inherent to design research practice. Rather, our focus is on describing how each cycle influences the following cycles, thereby articulating how the body of work as a whole engages with the overarching research program in a complementary way. It is therefore important to note that the three-stage model of a cycle, introduced earlier in this section and presented in Figure 2, is intended merely to illustrate the dynamics of a constructive design research process. The position, size and form of the elements in the model should thus not be considered as formally accurate individually, but rather as relative to one another. For example, the design program (Figure 2, left) is a perspective on the design space and thus exists on the outside, looking in. However, the shape of this perspective (angle, width) in the illustration is arbitrary: the design space is an infinite domain of possible designs and the limits of a design program are inherently unknown before realisations are created. Similarly, it is difficult to accurately draw the shape and position of the new perspective one gains from reflecting on a cycle in the third stage (Figure 2, right). How this new perspective relates to the infinite possibilities of the design space, or the potential of a design program, is impossible to describe accurately. What we do know, is something about how this new perspective exists in relation to the graphical elements that describe stages 1 and 2. First, its position in relation to the design space is more abstract than that of the original design program, because it is a reflection on this program. Second, it casts a wider perspective on the design space as new insights of the possibilities inherent to this design space have emerged. And finally, that it is formed by, and thus overlaps with, the design program and experiment generated in stages 1 and 2.

3.2. Drift and stabilisation

This process is characterised by drift [4, 24]: as understanding of the research interest develop, one’s perspective often shifts from its original position to a new one. It is what such drift exposes that we aim to address in this publication. In particular the work we present exemplifies a kind of mechanics—between research program and its series of heterogenous design explorations—that reveals while leaving complexity intact.

Several other scholarly efforts [7, 8] have been made that attempt to describe what this drifting is, how it may be controlled or steered and what its relation is to knowledge production. To describe drift in a design research process if fundamentally difficult, because it is often used as a general term to describe the many elements that form the ephemeral activity of designing. This includes the tacit and explicit knowledge that steer decisions [6, 26], intuition and skill of the designer [27], the personal agendas present in multi-disciplinary design teams [4], serendipity or even more banal issues such available resources. Nonetheless, we agree with others (see e.g. [8, 28]) that it is important for design researchers to attempt and articulate the process of drifting, in order to better understand how design as an activity functions in generating knowledge.

What is easier to describe than the process of drifting and the factors and forces that influence it, are the moments of stabilisation: points where we are not synthesising different concerns through designing, but take a step back to relate what we have done to our research interest. Although these points are also certainly not accurate: they are to a certain extent post-rationalisations of a dynamic process. They serve as anchor points: moments where we state an intention (in the formulation of a design program) and moments of reflection (on realisations of that program). In describing and exposing our considerations, we can describe vectors that connect these perspectives on the design space and that allow us to sketch what happens as we drift between these perspectives (see Figure 3).

Figure 3.

By articulating start and end stages of different cycles that explore the same topic, we can sketch how we drift from one perspective to another, and how these perspectives together express the complexity of the research topic.

On the following pages, we first introduce the initial design project that formed the start of our investigation through describing the formulation, realisation and reflection of its design program. The pages thereafter sketch different avenues of exploration, that is, new design programs that have emerged from reflections on previous design programs. We conclude with a methodological reflection on how the continuous repositioning between program and experiment in this approach is a valuable way to give expression to complexities inherent in the research interest.

4. Design research cycles

The following pages briefly discuss several of the different avenues of exploration, framed within the programmatic approach through their presentation: as full cycles of formulating, realising and reflecting on a design program. These three stages of each cycle are illustrated on the left-hand side of each subsection image. These cycles are intended to illustrate how knowledge generated in one cycle, influences the actions and reflections in following cycles. The reflections on the design program and its realisation highlight insights that steer further drift after a moment of stabilisation in the completion of the design program. Salient points of these reflections form the basis for the formulation of new design program(s).

4.1. MoCap tango performance

A performance with world class tango dancers Murat Erdemsel and Sigrid Van Tilbeurgh took place during the Midnight Light Tango Festival in Umeå, Sweden in June 2015 (see Figure 4). For more information on this project, please refer to [29].

Figure 4.

Still image from the MoCap tango performance with the dancers in the middle, and a real time visualisation of their movements projected behind them. Photo courtesy of Murat Erdemsel.

Design program: use novel technologies to create the possibility for an audience to appreciate dimensions of the dance that are normally unperceivable for the audience, but clearly present for the dancers.

Realisation: the dancers’ movements were tracked through a motion capturing system and custom designed wearables. The movements were visualised in real time through a projection behind the dancers. By building on the embodied experience of the dancers, we visualised particular aspects of the dance, that is, accelerations and traces of movement through space. The visualisations emphasise elements of the dynamics and makes them linger in time, to enhance their perceivability.

Reflection: the overall sophistication of the visualisation was not at the same level of the dancers’ performance. However the design case has operationalised the research program, by creating a common vocabulary (both verbally and visually) between dancers and designers, enabling a mutual understanding of aesthetic qualities that we aimed at exposing. Another reflection relates to the method of visualisation: the projection reduced an embodied experience to a two-dimensional representation of parts of that experience, leading to interest as to how the movements might acquire a physical, embodied presence in its representation that resonates more strongly with its original (see Section 4.2). Shared reflections with the dancers revealed another quality: in discussing their bodily, felt experience in relation to the visualisation, the dancers were able to show how glimpses of their intentionality, the play of forces between partners to indicate future movement that is normally hidden for an audience, were visible in the visualisation (see Section 4.3).

4.2. Materialising movement qualities

The enormous amounts of data gathered in the MoCap tango performance provided the basis for a wide area of explorations with the intention of exploring ways to materialise the qualities of movement embedded in the dance.

Design program: find an expression of the captured data that is as rich, beautiful, dense as the dance.

Realisation (see Figure 5): data from dancing tango dancers were gathered with a motion capture system, allowing specific steps and figures to be isolated. Both 3D models and animations of the data in those intervals were generated. Renderings were done both by using particle systems and ray-tracing techniques and we investigated the visual effect obtained through assigning different kinds of materials. Sculptures were realised with FDM techniques and visualisations of the data were projected onto fabric sculptures to examine ways in which the visualisations could acquire a physical presence in the same space as the dancers.

Figure 5.

Experiments in materialising movement qualities with data captured during the MoCap tango performance. Clockwise from top left: animation experiments, rendering experiments, 3D printing movements, and materialising projections. Bottom left photograph courtesy of Murat Erdemsel.

Reflection: there are qualities about the aesthetics of the material that is being produced along this project, that is simply, irresistibly, sensually beautiful and is demanding us to keep on exploring: the grace, sensuality, viscerality of the visual material that we realised so far; the combination between live dance and visualisation or the materialisation of dance (yes, we can touch a dance now!); all these elements, make this design research project irresistible and push us to go further and dig deeper. Leaps from one medium, one materiality to another, starts unveiling hidden elements, even to the dancers. At that point, we started noticing ephemeral leads: in materialising movement qualities, different models and prototypes provided new perspectives on the dance. The dancers themselves were able to see and discuss things they were unable to perceive before. This brought us to elaborate, later on, educational material, meant for the dancers, in their pedagogical work (see Section 3.4). The physical materialisation of the movement, especially in forms of 3D printed sculptures, showed how the dance may inspire spatial explorations, further explained in Section 3.5.

4.3. Intentionality

Together with the same tango dancers that we have collaborated for the performance design program, we started working on how to expose the dialogue among the two dancers, by digging into how the initiative that is taken by one dancer, is communicated to the other. Together with the dancers, we have chosen to isolate the element of transmitting pressure on the floor, as one way to visualise how the intention of undertaking a specific step can be studied, exposed and made more apparent (see Figure 6).

Figure 6.

Exploring intentionality in tango: pressure sensors embedded in the shoes of the dancers are used to gather data and visualise pressure, force and points of contact. Photos courtesy of Murat Erdemsel.

Design program: to explore ways in which the intentionality, the hidden forces with which partners sense each others’ movements before they happen, may be expressed and become perceivable.

Realisation: we initially defined ‘pressure transmitted on the floor’ as the element to visualise, in order to further explore ways to transmit intentions in the dance dialogue; we did some photoshop work to simulate how we could visualise such pressure (see Figure 6). We then recorded the pressure, by implementing a pressure sensor under the dancers’ shoe soles and gathered data while motion capturing movements. The pressure sensor data is currently being integrated with the motion capture data, in order to add the dimension of intentionality to the visualisations.

Reflection: the first trials in visualising downward pressure from the feet of the dancers are promising, in the sense that it shows activity. However, as our investigation is a design-oriented one, and not an analytical inquiry into objective, measurable elements of tango, this also highlights a difficulty: in itself, the sensor data in this project is meaningless, unless integrated with other elements of the dance, like the whole movements. Shared reflections with the dancers, in which they explain their intentions and felt experience in relation to the data, are one way in which these missing dimensions might be exposed. Moreover, reflecting on this design program also made clearer that our real interest regarding intentionality is in the direct dialogue between dancers: for example, the physical dialogue at points of contact between the dancers’ bodies that elicit or guide successive movements. This latter reflection is the driving force between a new design program, that explicitly aims to use the laid by this project, into interaction design: produce material to inspire a reflection intentionality in intelligent products and systems, in relation to how such products and systems can detect and respond to human cues and intentions.

4.4. Educational material

The first and second design programs and their realisations informed us about the possibilities of creating visualisations to be used for educational purposes, in teaching Argentine tango. The suggestion came from the dancers: through the materialisations, renderings and animations of the captured data, they were able to compare and articulate their felt experience and reflect on it. In some cases, their felt experience was completely different from what happened in reality (e.g. feeling like make a circular movement pattern, but in reality making a distinctly different shape, see Figure 7).

Figure 7.

Renderings of the data can be used to provide a new perspective on the movements (in time and space), or to isolate particular body parts. In the above still image, only the feet of the man and woman dancing are shown (green and pink, respectively) from a bird’s eye view. This visualisation showed a professional tango dancer that the patterns of movement he thinks he makes are in fact very different from what he does in reality.

Design program: develop material that supports the teaching of dance, exposing elements of complexity that are difficult to verbally explain to others. Such tacit elements are, for instance, the physical dialogue between dancers, complex movements that require coordination of several body parts or better visualisation of movements’ characteristics, such as place and space.

Realisation: in tight collaboration with the dancers, we have developed a visual language that has specific aesthetic characteristics, such as how long the trace of movements lingers, how acceleration and speed are represented (the thicker the line, the faster the movement), how the position of the feet is represented (light dots), or what interval of dance is visualised (see Figure 6). We have elaborated a series of visualisations that show separately movements of feet, movements of hips and movements of shoulders.

Reflection: this design program constitutes a very practical application of the first, more abstract and artistic case, that is, the MoCap tango performance. In proposing a specific representation of the movement (with defined aesthetic qualities), we make a decision concerning the exposure of certain movement’s features. The material that has been produced so far, is currently used for educational purposes and will produce a feedback by its impact on students, which will inform the next iterations. From a design perspective, this design program also highlights an interesting tension: to isolate certain elements of the movement reduces its complexity to a more handleable form, however, at the same time, this process reveals elements that were obscured by its complexity.

4.5. Tools for spatial design

The design program aimed at materialising movement qualities (Section 3.2) drew attention to the sculptural and spatial qualities of the dance. This led to a new design program, in which we explored specifically how the technologies that we were using to understand dance, might become functional as ways to directly design spaces. Within a research project called +Plus on sustainable production methods for residential buildings, we carried out a series of explorations with the contemporary dance collective Nomodaco. The movement sessions had specific assignments for the dancers, who were sketching with their bodies in a virtual reality space, using the HTC Vive technology, a motion capture system, and a mixture of custom and commercially available softwares.

Design program: inspire new ways and create new (digital) tools that support the design for spaces, by using one’s body movement as a means of sketching.

Realisation: the explorations were designed in order for the dancer to express the interactive qualities of an everyday activity (making and drinking tea), using different obstructions, such as being physically connected with each other with a rigid joint or with a flexible joint. The movement was traced with brushes connected to their hands in a virtual reality space and later, with sensors on all of their body, tracked by a motion capture system. The sketches were visualised and materialised using different methods of representation.

Reflection: each representation highlights different elements of the session and caters for a different purpose. The most relevant are: firstly, sculpturally annotating movements to give expression to specific interactive qualities that can be abstracted and turned into spacial qualities to inspire the design of a space. Secondly, by moving carving out space from a solid volume, to directly shape a space and be able to experience the qualities of that space to inspire further design steps. Further iterations will be carried out on the basis of this first exploration: in particular to explore how the tools and techniques used in this workshop, may be used by architects, as experts in the design of spaces (Figure 8).

Figure 8.

Designing spaces from movement, annotating movements using motion capture (left) and carving spaces with dance virtual reality (right).

5. Perpetual perspectives

To conclude this article we reflect on our cycles of formulating, realising and reflecting on design program, to expose how the drifting this process causes allows for expressions of the complex research topic. Reflecting on a design program and its realisation, affords a shift in perspective on the design space (see Figure 9). We drift from one position to a new one: a vantage point where our point of view on the design space is guided by reflections on the work and the knowledge it generated. This new perspective highlights new opportunities within the design space: it can show us something that is interesting to pursue further from the particular realisation in this design program. For example, the ephemeral intentionality that is part of the embodied dialogue between dancers: glimpses of it were visible in the first visualisation experiments for the performance. This suggested we could further expose them and leading to the design program explained in Section 3.3. This reflection at the end of a cycle might also highlight or emphasis qualities we have neglected to address in the current design. For example, the richness that is lost in creating a two-dimensional representation of a three-dimensional movement (leading to the design program explained in Section 3.2).

Figure 9.

The conclusion of the first cycle (left) and the conclusion of the fifth cycle (right).

We find the distinction between the research program as a whole, and the individual design programs that substantiate parts of this whole, useful. The research program is concerned with expressing the (hidden) qualities embedded in the experience of dancers. It is difficult for individual design programs, and the design experiments they bring forward, to address of all the complexity that constitutes this experience. However, in casting the design programs as a certain systematic way to address new elements found in this complexity, and to build upon earlier work in order to expose it, aids us in viewing separate design projects as part of a bigger whole (see Figure 9). To develop new cycles based on earlier cycles, balances what has been discovered, with what is missing and latent. This way in which past experiments and design programs stay relevant for subsequent cycles, and thus for the research program as a whole is crucial. Design experiments typically focus on parts of the complexity of the design space and express certain parts of this complexity. To unveil and understand this complexity, these individual expressions as need to be considered as a whole, mutually influential and relevant, to respect and embrace the complexity.

Acknowledgments

We would like to thank all those involved in the various MoCap tango related design programs: tango dancers Murat Erdemsel and Sigrid van Tilbeurgh, Carolina Backman and Tove Skeidsvoll of dance collective Nomodaco, as well as Ronald Helgers, Olov Långström, Nigel Papworth, Thom Persson, Fredrik Nilbrink, Nicole Sampanidou, and Willem Zwagers of RISE Interactive.