Perspective Chapter: A New Space of Storytelling

Abstract

Following decades of inventing new display and sensing technologies, augmented, virtual, and mixed reality (XR) devices, platforms, and communities provide a new space for storytellers to tell their desired stories. While we are in the process of developing new conventions for fully immersive storytelling, we must first define the properties and functionalities of the new digital medium. This chapter is based on my research in the field of virtual reality (VR). I would like to share some insights I have gained through exploring a wide range of VR works from various countries.

1. Introduction

Imagine yourself flying through a dark room, a room that feels so enormous as it fills with words, drawings, and stories. You are led by a voice or music but try hard not to get lost. You end up seeing a tree, with all its leaves being letters that are falling into dust. You find yourself wondering if this is a dream or something so common in life: a feeling but end up returning to your reality. What I just described is a famous virtual reality experience called Chalkroom [1] created by Laurie Anderson and Hsin-Chien Huang. After decades of innovation in creating display and sensing technologies, new forms of digital media, especially virtual reality, have provided a space that radically changes our perception of stories or storytelling.

“Story” is a fascinating word, as it suggests magic itself. It is a magical analog that attracts and entertains people. It has a structure and a background with a number of characters involved. It is created and told by a storyteller who has the skills to reassemble all sorts of materials in a way that both pleases and challenges readers. It may fit into certain types, like fiction or nonfiction, comedy or tragedy, etc. Considering that we live in an era of consuming TV shows, Movies, or short video clips from social media on a daily basis, we can certainly say that story is nothing but a sequence of media objects. If reusing some ancient concepts, a story can be defined by its beginning, middle, and end. If you happen to be interested in Artificial Intelligence (AI) technology while reading this chapter, I will highly recommend you generate your own stories through AI by using the prompt of the three ancient concepts just mentioned above. Considering the capacity of AI technology to create stories, it is crucial to acknowledge that we humans might not be the only storytellers in the future. While defining AI as a storyteller seems obnoxious to some, as it is opposed to human writers’ authority and their talents for telling stories, I find it quite inevitable since it invites a discussion about the core concept of stories, creating meanings and hence resonating with people. Can AI understand meanings or human emotions? As AI is not within the range of our discussion and I am not an expert in it, I am going to leave this question and discussion to you, my dear readers.

2. Storytelling and digital storytelling

What is storytelling? Storytelling is the act of telling a story. It is the sharing of experiences, ideas, and emotions through words, images, and sounds. Storytelling has many basic elements, such as characters, plot, setting, theme, point of view, style, and tone. Characters can be people, animals, or even objects. They are the ones who play a role in the story and move the story forward. We are most familiar with human characters and often define two main types of them: protagonist and antagonist. In literature, a plot is most commonly defined as the series of events that make up a story. Without a plot, a story becomes a collection of unrelated elements whose meaning most readers are unable to read. The setting is the time and place in which the story takes place. Storytellers may also use the settings to create symbolic or metaphorical meaning. A theme is the main idea that the narrator wants to convey or the universal meaning that the narrator seeks to explore. A point of view is the point of view from which the story is told. It can be first person, second person, or third person. Style and tone are the way an author writes a story, including word choice, syntax, and imagery. It can convey moods, emotions, and atmospheres. These elements work together to create a cohesive narrative. Narrative refers to the way a story is told. It is the structure and composition of the story. In storytelling literature, there are several commonly used narrative structures such as Chronological structure, Flashback structure, Nonlinear structure, and Open-ended structure. When telling our stories, we intertwine the various elements of meaning, emphasizing or relativizing them in a never-ending process [2]. In this process, we wish to create something that is inspiring and memorable. By creating a sense of anticipation and engagement, a well-crafted narrative ensures that the audience truly cares about what happened to the characters. Hence, storytelling can be used to educate people about how to build relationships. According to Roland Barthes, “narrative begins with the very history of humanity” [3]. It is an ancient practice that has been used since the dawn of civilization to share stories and preserve culture and history. As a cultural practice and life form, storytelling is deeply embedded in the history of different societies. It becomes a constructional tool to bridge the gap between self-perception and social surroundings.

What is digital storytelling? This can be a harder question. Combining these two words, the word digital can suggest any digital device or medium that uses some degree of digital technology. The term emphasizes the use of various platforms and technologies, such as interactive video, mobile apps, websites, virtual reality, and augmented reality. Almost 60 years have passed since the first digital electronic computing device—ENIAC—was released in 1964, and now we have a vast and ever-growing toolbox of digital devices. And old technology like the camera, which was invented in 1816 now has also taken advantage of digital technologies to display real-time image capturing. If it is difficult to concisely define the word “digital devices” or “media that use digital technologies” in a short sentence, we arrive at the fact that we are living in the most innovative period of human history in terms of the development of a “digital world.” With the unstoppable force of inventing new digital technologies and media, the word “digital storytelling” has an ever-increasing societal value when fusing two domains together. In the new digital world, we are witnessing a new future with many possibilities, from more engaging content created by incorporating various multimedia elements to user-centered interactive experiences. Digital storytelling captures a wider range of audiences for different purposes. This allows for a more nonlinear story structure, as it can contain multiple plots, different points of view, and nonlinear events. This gives the audience more freedom to explore the story and experience the story in their own way. In addition to technology, digital storytelling has been used for creative and artistic expression. Digital storytelling allows YouTubers, vloggers, and all kinds of social media influencers to create stories that reflect their unique perspectives, experiences, and identities. Expressing your identity through personal stories is becoming the new norm of our time. Most importantly, by sharing personal stories, we can gain a deeper understanding of ourselves or others. We will dive deeper into the functions and purposes of digital storytelling in the next few sections, taking virtual reality (VR) as the targeted digital technology.

But first, let us identify the properties of these two domains. We, humans, cannot easily “plug in” to the digital world but are naturally connected with story and storytelling. Hence, the digital world is desolate and cold while the story field is approachable and warm. As long as we can extract a storytelling act or some analog content from the digital world, it is sufficient to recreate meanings intrinsically. From a different perspective, in a digital world, it is not only a story that can create meanings but elaborated data patterns are also likely to generate meanings. With meanings created, and narratives formed, a series of influences can be observed through its receiving audiences. Ever since the huge success of television shows and broadcasting, dozens of academic research studies have been conducted and put into place for suggesting the negative impact of television exposure on young children. However, as time goes by, we do not throw out television or stop shows but, instead, realize the capacities of new generations to adopt and adapt to new technologies. It will not be truthful to suggest there are no negative impacts found from any new digital storytelling content, but I want to highlight the possibilities that new technologies can transcend the cognitive development of a generation. So far, we have dealt with some basic ideas about stories and digital storytelling. In the next section, more elaboration will be given to describe the uniqueness of VR storytelling.

3. The space of your presence

First of all, the core concept of VR is explored as both a technology and an emerging medium. Virtual reality technology is a computer-generated simulation of a three-dimensional image or environment through special electronic equipment, such as a helmet with a screen inside. It tracks the user’s head movements and provides a realistic, 360-degree visual experience. The medium of virtual reality technology is rapidly transforming. It is not enough to define this medium by only using a technical definition. In one of my previous research papers, I explored five individual building blocks [4] for creating immersive VR storytelling experiences. Simply put, I believe that the best-designed VR experiences, which will release this medium’s full potential require (1) innovative visual effects, (2) professional sound design, (3) self-embodied and user-centered spatial interface, (4) complex or creative narrative structure, and (5) a robust but unique interactive mechanism. Among all these five important elements, I would like to reexamine the third one: spatial interface at a different level. How to define space? What are the roles of audiences in the virtual space? Answering these questions will help us better understand how they might affect the formation and characteristics of the narrative subject.

Space, the matter of the boundless three-dimensional span, is a framework in which events take place and objects have relative positions. Artists view the space as more than a conceptual framework, but as an individual intimate area that can be shared, embraced, explored, and cocreated. Chinese philosopher Laozi believed that space was a concept of emptiness, a lack of form, and an absence of anything that could be perceived or experienced. He viewed space as a metaphysical reality in which all things exist and interact with one another yet remains part of the same unified whole.

He wrote in the book Tao Te Ching:

“Thirty spokes join together in the hub. It is because of what is not there that the cart is useful. Clay is formed into a vessel. It is because of its emptiness that the vessel is useful. Cut doors and windows to make a room. It is because of its emptiness that the room is useful. Therefore, what is present is used for profit. But it is in the absence that there is usefulness [5].”

From a conceptual framework to an intimate creator place, then to a metaphysical reality, combining all these three ideas, we might be able to peek into the ultimate truth of VR as a spatial medium. In other words, it is the illusional space that is different from the audience’s current physical environment that enhances the audience’s feeling of immersion. Yet we have not asked the question about how exactly the illusional space works. Why is it important to the audiences?

Typically, when reading a novel, a reader does not need to enter a location or scene where a character lives or is acting. Whether the setting is a twenty-first century study room in Louisiana or the picturesque British countryside of the eighteenth century, the reader will be able to picture the story’s universe based on the written descriptions and their own experiences. The reader will merely be the observer in the stories and has no control over how the plot unfolds. This is the same as listening to your friends telling their’ personal life stories but with a small difference: you may interact with a friend by asking questions, and hence you can switch from being a passive listener to an active audience. Even if you still have no control over the general plot, you may have the ability to interrupt the conversations and affect how the story is told, because now your friends will be recalling additional details to address your questions. It is difficult to determine which type of narrative is more appealing, being a passive reader or an engaged listener. Because you might identify strongly with a book but not so much with your friend’s humorous anecdotes, or vice versa. It is crucial to understand that while the levels of engagement are different in the two scenarios mentioned above, the audience in neither case has access to see, hear, or feel the stories’ settings.

Movie watching is another example. Audiences are passively taking in the story as well, but they will be able to visually explore all the designed scenes on the 2D projector screen in front of them. We are aware that music and sound can be important components to entertain our bodies and change how we feel about things because we frequently observe people tapping their fingers or feet while listening to music. As a result, movies with sound-image synchronization can offer the audience more immersive content. But even after including two more components, the audience still has no control over the plots or the outcomes of the characters in this kind of storytelling.

Let us now examine storytelling in virtual reality. The high-end VR headset will provide you with six degrees of freedom, also known as 6 DOF. The system can track the change in angle and distance between you and the 3D objects, as three rotational axes and three directional axes are all being tracked simultaneously. In other words, the viewer is free to walk around the virtual environment and view nearby items from any perspective. For the developers of VR experiences, realistic-looking, vivid 3D assets can be easily found on the Unity Asset Store or the Sketchfab website. In addition, the photogrammetry technique has been tirelessly developed by researchers, technicians, and artists. It is now a simple method for integrating real-world content into VR experiences. As of the time this paper is written, users of the most recent Apple products, such as the iPhone 12 Pro, 12 Pro Max, or iPad Pro 2020+, could create scans of huge objects or areas in the real world using the top-rated LiDAR sensors. By bringing these scans into the virtual space, one can easily reconstruct a “real” world. No wonder why the scholar Mel Slater stated in his paper that “The strength of the experiences created using VR technology is that they feel especially ‘real,’ thanks to the illusions arising in the virtual environments; the place illusion of ‘being there’ (in the location depicted by the VR application), and the plausibility illusion [6].”

Besides the virtual environment and objects, spatial audio increases the immersion of the VR experience by bringing an additional layer of realism to the virtual environment. Spatial audio is a type of audio that is designed to create the illusion of sound coming from different directions and distances in a three-dimensional space. There are numerous methods for producing spatial audio in VR, such as binaural audio, which uses two microphones to record sound in a manner that simulates how our ears perceive sound in the actual world. Other methods include wave field synthesis, which uses a huge number of speakers to simulate the sound of a certain location, and ambisonics, which employ a number of microphones to record sound from all directions. In all forms of interactive storytelling, spatial audio is particularly effective at creating a sense of presence and immersion. Put simply, users feel they are present in VR; they are dropped right into a scene as if they were part of the story [7].

As another way to enhance the immersion of VR experiences, olfactometers (odor machines) are also explored by some developers. These are devices with use of scent-emitting chemicals to produce specific smells. However, it is important to note that olfactory displays are still in the early stages of development and are not widely available.

Audiences can even interact with the setting or plot moments with the aid of controllers or haptic gloves. Some controllers, like the Oculus Go controller, have very clear limitations and would not monitor the user’s actual body motions (production stopped in 2020). So, some interactive VR experiences are hard to play with this product. For example, Vader Immortal, a Star Wars-themed VR series that puts you in the role of a Force-sensitive smuggler who encounters Darth Vader on the planet Mustafar. You can only play this game on Oculus Quest and Rift systems. Meanwhile, there are several options of haptic gloves available on the market right now, including the HaptX Gloves, Captogloves, SensoryX VR Free Gloves, and Hi5 VR Gloves.

Overall, highly realistic and immersive VR experiences can be achieved through a combination of visual, auditory, and smell stimuli, as well as tactile sensation, which allows users to touch and manipulate objects within the virtual world. In addition, if it is a storytelling piece, then the first-person perspective is extremely important. In first-person interactive storytelling, the player can control the character from whose point of view they are experiencing and make decisions that influence the plot’s progression. As a result, the player feels more directly connected to the events of the story, allowing for a more immersive and personal experience.

Hence, I would like to utilize a particular VR storytelling experience to highlight the difference between passively consuming stories and engaging with them from the first-person perspective. Is Anna Ok? is a two-part combined narrative of one story, told by a twin Anna and Lauren, about the traumatic experiences that resulted from an accident. The audiences can engage in the story by choosing Anna’s or Lauren’s side of the story in their preferred order. Following a series of baby pictures and brief anecdotes about their childhood, you will begin your actual journey by taking a photo frame of the two of them from either Anna’s or Lauran’s hands. It is a very intimate moment as if you are stepping into someone’s life story. In the following dark scene, listening to Anna’s or Lauren’s description of that night when the tragedy happened, you can tell their emotions become tangible through the tone of their voices. The transition from the third-person perspective to the first-person view is so smooth that you do not need any further explanation to realize that you are no longer an audience but have become Anna or Lauren in the following scenes.

As Lauren, when you visit Anna in the hospital or rehabilitation center, you can see her struggle through her facial expressions. You start learning about the interaction mechanism by picking up your phone or playing Lego with Anna. After bringing her home, you notice Anna’s personality shifts and mood swings when she throws away her clothes. You must also digest your own feelings of horror by returning to the night when you witnessed Anna’s accident, the flashback of Anna’s torn jacket surrounded by giant shards of glass in front of you. In Anna’s side of the story, her narrative about losing her sense of time drags you with her in that time period to also experience memory loss and brain injury. You surely will notice that the TV show is incomprehensible because there are no actual human figures but color blocks; you will pick up objects for the purpose of rehabilitation only to have them vanish from your grasp. Every small interaction you make, such as picking up things, will trigger snippets of narrative. You will go for a walk in the hill scene after picking up a coffee cup with a tiny hill scene inside. Inside the bedroom, by holding a glowing lantern with a pocket-sized Christmas fair inside, you will be transported to the end scene of Christmas night, an open market. As Anna, you will then experience the overwhelming feeling through the fast-moving motion-blur human figures around you and their intermittent dialogs.

All of these magical moments for you to follow along with their daily lives, feelings about the situations, and scene transitions between story plots create a very unique storytelling structure. It is so one-of-a-kind that it can only be experienced through the medium of virtual reality, by interacting with the story. And, at the end of the two stories, by holding Lauren’s or Anna’s hands, you will have another major disembodiment experience. As you see Anna and Lauren walking away from you at the end of the story, you no longer use their bodies or their identities. In both stories, the very last voiceover is about how you, Lauren, will accept Anna’s personality change and how you, Anna, believe Lauren is the only person who truly understands yourself. At the moment, you will receive a huge emotional impact from this VR storytelling piece as the third person who is left inside the story while Lauren and Anna faded into the darkness. The work is extremely powerful because participants always feel a strong connection to Lauren and Anna. Participants are also deeply touched by their unwavering love for one another. It is true evidence of arousing empathy and embodiment through VR experience.

Is Anna Ok? is a masterpiece of VR experience and an innovative work of first-person interactive storytelling. The feeling of presence in this experience is clearly influenced by a number of factors, including the design of the virtual environment, the actual physical space, and the participants’ own expectations and experiences. I saw this piece at the 2018 Sheffield Doc Festival, and many attendees told me that they were immediately drawn into this story when they interacted with the traffic accident scene. Shifting red and blue lights have strong reflections on shards of glass, allowing you to touch and swipe away glass while seeing Anna’s favorite jacket torn apart in front of you. But you cannot get closer. The soundtrack you heard is very intense and clearly resembles the sound of an ambulance. And all of the sudden, all the glass was dropped from the air. The next thing you know is that you are in the hospital lying in bed as Anna or standing next to Anna’s hospital bed as Lauren.

The scene is highly impactful as participants will associate the abstract elements with their social expectations and life experiences. Even most people have never been in a car accident. We know from all of the TV shows and movies that the car’s windows will be shattered during the accident. Just like we expect to picture an ambulance at the accident scene. The true genius of this brief plot is that the metaphoric scene can be easily related to an emotional experience in a traffic accident. The participants are informed by voice narration and immediately realize that the torn jacket is a symbol of performing surgery with Anna to save her life. The limits of body movements convey a sense of desperation. In this plot, the ambiguity of time and space contributes to an atmosphere in which participants experience visceral feelings of loss and confusion. Synesthesia is a non-pathological sensory condition where specific sensory inputs or concepts automatically trigger additional (often visual) sensations, which can be more or less complex [8]. All of the elements and visual representations are meticulously designed to enhance the scene’s impact and trigger synesthetic sensations.

The physical installation of the piece was well-designed to match the virtual experience. You are instructed to sit on the bench for Anna’s story or stand on the other side for Lauren’s story. When your physical body’s position is matching with the VR experience, where Anna mostly sits in the rehabilitation, hill park, and bedroom scene while Lauren mostly stands up in those scenes. Embodied cognition is a psychological theory that suggests that our understanding of the world is not just a product of our brains. Just like what Donghee Shin suggested: “Embodied cognition acknowledges that the mind and body are agents working together to make meaning of VR experiences [7].” Research has shown that a strong sense of presence can have a significant impact on the user’s emotional and cognitive responses to the VR experience, leading to increased engagement and immersion. Participants are truly involved in this experience by interacting with vivid story scenes and a distinct storyline, as well as feeling immersed by unique visual and sound designs.

Besides the feeling of presence, I have every reason to believe that most participants experience a mental state called “flow” when interacting with the small objects in the scene. One of the missions for Anna(you) to adapt to normal life is to learn how to put a plate on a table. Lauren will encourage you to do so by saying “You just want to put it on the table, Anna.” It is a simple mission but a strong statement emphasizing the fact that you are still suffering from brain injury. Learning how to master a VR controller like using your own hand with this mission creates a feeling of learning things like a kid. Flow is related to presence; it is a state of profound enjoyment and concentration experienced during activities or performances [9]. Here, the participants are fully engaged in the narrative and gameplay. They will feel a sense of control over the story and their actions within it, and the narrative and gameplay are well-balanced, with neither element dominating. And inside the bedroom scene, participants can choose what objects they want to pick up: a shoe, a small bike toy, or a teddy bear. A flow state can be enhanced by using techniques such as branching narratives, player choice, and dynamic gameplay that adapts to the player’s actions and decisions. By creating engaging interactive gameplay, designers, and storytellers help participants achieve a flow state and become fully immersed in the story.

As a new form of digital storytelling, VR storytelling experiences provide an empty canvas with a unique framework for storytellers, artists, and designers to create deeply immersive stories. An amazing VR story like Is Anna Ok? is an illusional space for anyone to engage, embrace, be emotionally attached to, and have fun with. As participants, we actively shape the experience by selecting the order of two parts of the story or the objects with which we interact. The feeling of presence, the state of flow, the synesthetic sensations, and the emotions from the participants fill in the space of emptiness and eventually bring the metaphysical reality to life. Last but not least, a good story can bring us together and make us feel deep empathy for others. We create a real entity from learning and understanding in this process, something truly useful and meaningful.

4. Recreating historical and cultural space

One of the key functions of storytelling is to preserve culture and history. Historical sites, such as landmarks, buildings, public places, and structures, can be preserved and maintained as a way to honor and remember the past. Virtual reality (VR) is one of the ideal mediums for representing places that no longer exist or that went through a radical transformation from the past [10]. It is also a perfect medium to tell historical stories for educational purposes. If a personal story can be told by “reliving the memory” as individuals articulate their past memories to convey the story to the listener. The relationship between public memory and storytelling is a little bit more complicated. In general, public memory refers to the collective memory of a society or community. They are closely related as storytelling shapes and transmits public memory, and how stories are told can influence how public memory is formed and shared.

The representation of cultural heritage in virtual reality transcends space and time: (1) users can visit remote places that are not accessible to regular users, or places that are too far; (2) users can visit places the way they were at a different time in history through the digital reconstruction of a historical building [11]. For example, The Mogao Caves, also known as the Thousand Buddha Caves, are a UNESCO World Heritage Site located in the Gobi Desert in northwest China. The caves contain a large number of Buddhist paintings, sculptures, and texts dating back over 1000 years. As a popular tourist destination in China, the Mogao Caves attract millions of visitors each year. The Mogao Caves, however, are now under significant threat from climate change due to rising temperatures and increasing desertification in the region. Furthermore, the large number of visitors has raised concerns about the potential impact on the caves and their contents. In order to protect the caves, a number of measures have been put in place to limit the number of visitors allowed inside the caves at any given time. Besides, some caves have time limits for visitors, in order to give more people, the opportunity to see the caves while still protecting them from overcrowding.

Looking back at history, as a form of prehistoric art, cave paintings were created by ancient humans in cave systems around the world. Cave paintings have played an important role in storytelling and the transmission of cultural knowledge, and the Mogao Caves have been a major site of Buddhist pilgrimage for centuries. These caves are also an important source of information about the cultural, artistic, and religious history of China, as they contain a wealth of historical artifacts and manuscripts that provide insight into the lives of people in China during the fourth to the fourteenth centuries. In 2018, NTU imLab present a virtual tour of Dunhuang Mogao Caves (神遊敦煌-莫高窟第61窟 [12]). You can use teleportation to explore the caves and appreciate the spatial characteristics of the entire space. The VR experience recreates the stories behind the murals by connecting a series of murals for you to examine closely. It also restores some statues and murals that have already vanished or been damaged. Because flashlights should be avoided in actual caves, you can only enjoy some vivid details of art using a virtual flashlight in this VR experience. There is no need to be concerned about travel costs or visiting restrictions because virtual reality technology transforms this ancient art space into spatial content that Chinese society’s public memory can easily perceive, experience, and interact with.

Taking a look at the world today, the COVID-19 pandemic has resulted in unprecedented travel restrictions all over the world. Border closures, travel bans, quarantine, and test requirements, for example, have varied greatly from country to country. Overall, many countries have taken drastic measures to limit the virus’s spread. These measures have had a significant impact on the global economy and have disrupted tourism industries around the world. Virtual Reality experiences, particularly immersive experiences or applications related to historical sites and cultural heritage, address the pandemic challenges. It creates a precious window through which people who are confined inside can look out, travel around, and live their lives.

Even today, more and more researchers believe that VR storytelling should be classified as interactive digital storytelling (IDS) because it is an experience where the user actively tailors the story. Still, there are many projects using 360° videos or 360° videos mixing other interactive elements for showing cultural heritages or communicating historical events. According to VR Nonfiction: A Mediography project: “Virtual Reality nonfiction is an emergent and a rapidly evolving new medium for filmmaking that draws from—and builds upon—traditional forms of nonfiction, interactive media, gaming, and immersive theater [13].” Thanks to this VR Nonfiction Mediography database, we can easily access 603 VR nonfiction titles released between 2012 and 2018.

There are several pieces in the database that I want to explore here. First, the VR short The Last Goodbye uses CGI, billboarded stereo video, and photogrammetry techniques to share an emotionally impactful historical event with us. The audience will embark on an intimate journey around the Majdanek concentration camp, accompanied by Holocaust survivor Pinchas Gutter. In May 1943, around 20,000 Jews were deported to the camp, and Gutter’s twin sister Sabina was one of them. Sabina was sent to the gas chamber as soon as she arrived. Gutter never got the chance to say goodbye to her. The audience will walk with Gutter inside the railway car, gas chamber, shower room, and barracks of Majdanek. Through the process of sharing his story as an 11-year-old child, Gutter achieves a level of relief and emotional healing that his virtual presence within the experience amplified audiences’ own sense of emotional and social presence, which contributed to it being such a profoundly moving VR experience. This is the experience that teaches us what we should learn from history and the consequences of losing faith in humanity.

Second, Apollo 11 VR is a 5-minute multisensory VR experience. It tells the story of humanity’s greatest journey, Apollo 11, to the moon. As an audience, you can experience this historic event of that day as if you were there in 1969. Before embarking on this thrilling virtual journey to the moon, you will first view John F. Kennedy’s famous speech on the projector. Then, before boarding the rocket, you enter an elevator with Neil Armstrong. After exiting the elevator, you take a seat in front of a control panel with small windows on the rocket by your side. You can sense the nervousness and excitement of being an actual astronaut by hearing the countdown and rocket launch. A floating pen in front of you will soon indicate that gravity has officially shifted, and you have entered space. The view from the windows is spectacular, with a distant view of the moon. After landing, you will see Neil Armstrong take his first steps on the moon and place the American flag on the moon’s surface. The visual representation can easily fool viewers into thinking they are experiencing space travel. The audience cannot interact with the scene via controllers and must instead sit back and watch the story unfold. But even without any fun interactions, the whole experience is still very engaging. There is a YouTube video showing the tears of joy when a father experiences this piece of work with help from his son [14]. The father was deeply moved as he said he always wanted to be an astronaut since he was a kid. This VR storytelling piece made his dream come true. Even if it is not a perfect VR experience, this piece allows you to relive the events of 1969 and gain an insider’s perspective on an event that is etched in the collective memory of Americans. Furthermore, it is an excellent piece of educational content for children who are interested in space travel.

The last one I want to mention is a VR app called Anne Frank House VR. During WWII, German-born Jewish girl Anne Frank and seven other people lived in the annex of the building at Prinsengracht 263 for over two years. Anne kept a diary in which she wrote about her life in hiding under Nazi persecution. This Secret Annex was established as the Anne Frank House on May 3rd, 1957, with the help of Otto Frank, Anne’s father. The VR app provides a rare glimpse inside the Secret Annex. The audience will be able to tour the furnished rooms, where all furniture was meticulously built in accordance with the actual history from 1942 to 1944. Because the Secret Annex is empty in the actual museum, this app adds a practical and meaningful layer to our reality. After the people who lived there to hide from the Nazis were arrested, Anne’s father requested that the rooms of the secret annex be intentionally left empty. Most people interpret Anne Frank’s existence through her diary because of how well-known her story is throughout the world. Although Anne Frank is a master storyteller, it still can be challenging for readers to visualize the actual chamber where she lived. Fortunately, the breathtaking details from this app, such as the portraits on the wall, the books in the shell, or the light coming through the windows, really help you feel the mood of their living place as you are connected to their daily experiences. By clicking the quotation marks on the walls or touching the gloves on the objects, you will get snippets from her diary. You can also pick up a few items to observe as it activates voice-overs from the diary. When you touch the door, it will fade to black. Then, all you have to do is move to the predetermined location for the next room to appear. The entire experience is intuitive and engaging, mirroring audiences’ previous interactions with other digital interfaces. During the tour, you are presented with more details and information about this meticulously reconstructed space. You are given the opportunity to explore the space at your own pace and learn about topics of interest to you. Overall, it’s a very powerful app that allows more people to experience an important piece of history, as well as a valuable emotional experience that serves the greater purpose of education.

VR technology has the potential to create immersive and interactive historical and cultural spaces that allow users to experience and learn about the past in an engaging and memorable way. VR can create historical and cultural spaces by accurately and authentically recreating historical events or locations. Nonfiction stories in virtual reality transport viewers to different locations and times, allowing them to experience events as if they were physically present. Individuals can learn about and connect with their town’s or society’s history and culture by participating in storytelling within this spatial medium.

5. The social space

In contrast to spatial presence (the “sense of being there”), social presence describes the “sense of being together” [15]. In the world of virtual reality, “social space” can refer to how people use technology or platforms to connect with one another and create a sense of community or shared experience. AltspaceVR, for example, is a social VR platform acquired and developed by Microsoft. Users can create or join events, participate in group activities, or simply hang out and chat with friends using their own customized 3D avatars. There are several features and activities available at Altspace VR, such as virtual reality games, karaoke, trivia nights, yoga lessons, and more. It is compatible with a variety of VR headsets and can also be used on desktop computers. It might be worth discussing how this desktop VR application can use the affordance of a spatial medium within a 2D screen to make the experience easier and more accessible to people who do not have a VR headset. This discussion, however, will not be directly related to the topic of this paper. I will therefore leave it to other scholars or authors. Other virtual reality social platforms with similar features to Altspace VR include VR Chat, Rec Room, and Bigscreen. I am excited to see what kinds of storytelling activities are taking place within these virtual reality social platforms.

I have participated in several small presentations, conference events, and meetups in AltspaceVR since the pandemic. The social urge to interact or even work with people in the same environment was met. You will get the opportunity to socialize with people from all over the world inside the predetermined theater or conference room environment. People brought their personal stories into the room. In the year 2019, for example, a video clip began trending (俩老外在中国服务器里疯狂飙中文!甚至还互相教学 [16]) on the Chinese famous video-sharing website Bilibili. At the time of writing this paper, the video clip has received over 11 million views. Three users are hanging out in VR Chat and talking about learning Mandarin in the video clip. The user who used the Robotic avatar is Chinese, the user who used the big sausage avatar is American, and the user who used the small black cat avatar is from Iraq. The “sausage man” joked about being French before revealing that he is actually an American. After becoming acquainted, they began making various Chinese jokes as a language-learning exercise. The sausage man even sang a Chinese song. It’s a funny song about counting from one to seven and wondering where my friend is. “I am here, your friend is here,” the black cat added. They also shared some dirty jokes. The sausage man expressed his desire for finding a girlfriend in real life. Soon after, the video was posted online and quickly went viral. When they met again in the space, their personal stories came to light. The black cat fell silent for a while after learning that they had become famous in China. Just when everyone assumed he had gone offline, he began telling his personal stories about being unpopular in real life. He did not have any friends at school, and he felt like this was the only time in his life that he could get so much attention. At that moment, everyone felt for him. You would never expect a person you have only met twice to make such a sincere statement about his loneliness if you were not in the VR space.

Virtual togetherness is influenced by the sense of being in the virtual world and the communication between the users in the virtual world [17]. Because of the embodied experience and the sense of community, as well as the fact that users will remain anonymous for the majority of the VR social experience, it is not difficult to notice that users can be opened up quickly and remain more open than in real life. Narratives play an important role in shaping people’s memories, knowledge, and beliefs [18]. And the overall narrative for VR social space is about creating a community that benefits from communication without geographical barriers. In this way, social experiences on VR platforms form a new kind of storytelling, one that should be understood by artists and developers. To better understand this kind of storytelling, we should pay attention to how users are shaping the experience. Just like what Marcin Wieczerzycki pointed out “For the co-creation narrative and collective storytelling, despite dedicating a lot of attention to the impact that this experience exercises on consumers, consumer influence on the narrative-based goods themselves has been largely overlooked [19].”

During the process, a fundamental belief between users can be formed within the social scene: trust. It is not surprising that the sausage man would disguise his true identity at the start of their three-person conversation, given that he was only communicating with the other two in his sausage avatar. It is also not surprising that he quickly realized that there was no need to keep his disguise on and that he should reveal his true identity. What made this kind of transformation and attitude shift possible? It can be extremely difficult to analyze. I am sure he was fully aware that he might never meet the other two users in the same place again because meetups are usually random, especially if they did not change contact information later. There is a lot more going on inside this type of social space, and anyone can expect to hear a good story if you are truly committed to having genuine conversations with others. Loneliness in your real-life circle can be chased away, and your emotional needs can be met. The fundamental property of the VR social scene as a spatial medium is void, a void that represents the gap between real life and virtual life. Storytelling within the VR space acts as a means for closing the gap and creating new personal memories. However, when users serve as characters but are not genuine storytellers within the scene, role-playing becomes an option; by pretending to be a character, you can gain popularity or attention. This will undoubtedly put humanity to the test. Fortunately, a good VR social space can provide users with the ability to switch between realities, as well as a sense of safety, physical privacy, shared experience, and a sense of belonging [20]. All of this will contribute to a socially acceptable experience.

In another scenario, the “sense of being together” fulfills deeper human emotional needs. MBC, a South Korean broadcaster, aired a VR documentary: “Meeting You [21]” that is about reuniting a mother with her deceased daughter 2 years ago. This piece helped the family to cope with the grief of death. Meeting You tells the story of a family who lost their 7-year-old daughter, Nayeon, due to illness. Jang Ji-sung, the mother, was able to touch, play, and talk with her daughter again after using a Vive headset and trackers with Noitom’s Hi5 virtual reality gloves. As the mother and daughter exchange “I miss you,” Nayeon’s father, brother, and sister look on in tears, sharing in the grieving process [22]. The little girl picked a flower for her mom and then told her mother that she is not sick anymore. After reading the letter she wrote for her mother, the young girl told her mother how much she loves her, and her mother also expressed her feelings. “I still have things to do, but when I’m done, Nayeon, I’ll be with you,” she said. They expressed their undying love for one another. The little girl then said her goodbyes and fell asleep peacefully next to her mother. Finally, she transformed into a beautiful butterfly and flew away. Even though this work raises some ethical concerns, it is without a doubt the most heartbreaking piece of VR storytelling I have ever seen. Because death is frequently unpredictable and abrupt, people who have lost loved ones do not always have the opportunity to say their final goodbyes inside the hospital. Personally, I hoped that this work would eventually assist the family, particularly Nayeon’s mother, in overcoming her desperate feelings of loss. Although virtual reality is not a replacement for professional therapy or support, it may assist people in feeling more connected to the world around them and finding ways to move forward after a difficult experience. People may find it easier to return to a healthy life with inner peace after releasing negative and overwhelming emotions.

6. Conclusion

In recent years, virtual reality systems and associated devices have become more affordable and accessible. VR is very likely to be widely used in a variety of contexts for a variety of purposes in the near future. For example, VR can be used for a range of therapeutic and educational purposes, such as treating phobias or helping people learn new skills. By employing high-end VR headsets, spatial audio, odor machines, and haptic gloves, a well-designed VR storytelling experience can create an immersive and realistic experience, thus creating empathy and changing perception. As a spatial medium, it has great potential for digital storytelling. It is crucial to recognize its potential for creating new narrative structures or even forms of storytelling, regardless of how we define the audience’s function as a participant, viewer, user, or player within the medium. It expands the territory of digital storytelling into a corresponding philosophical bound where Laozi has portrayed a unique concept and scenario. The empty virtual space brings all kinds of creative designers, developers, artists, and many others into the same universe. Here, all creators can equally contribute to a new storytelling convention. As an illusional space, virtual reality storytelling may help you better understand who you are and how you think. It prompts you to consider how you can experience your existence physically or emotionally. As a fantastic tool for transporting people to experience the past, the future, or wherever you want to go, VR storytelling entertains, educates, and touches you. From a historical perspective, it is most powerful for cultural practices and the creation of a new form of life. In addition, virtual togetherness provides a new approach to some existing social issues, as many are feeling lonely and isolated to some extent due to the pandemic, remote work, or technology. Through a shared experience or within a cocreated community, virtual reality storytelling allows you to express your opinions and share your personal stories. It opens channels for people to cocreate their lives and embrace who they are, especially their inner feelings. It also creates unique moments that are deeply moving through some seemingly impossible cases. It is a magical space that may remain unknown until you, my dear readers, come in for exploration.

Conflict of interest

The authors declare no conflict of interest.

Notes/thanks/other declarations

Special thanks to Weijiang Gloria Lin and John Joseph for proofreading.