Introduced into theaters in the 1860s, Pepper’s Ghost startled theatergoers with an effect that allowed live people or objects to materialize into the scene. The illusion of a ghost is an actor located forward of and below the stage floor. The glass illustrates the reflection of the offstage “ghost,” while the leftmost “ghost” simulates what the audiences see. Modern versions of this effect consist of a completely new way of projecting video to create the illusion of life-size, full-color, moving images but projected as 2D images into a set. The mind of the audience creates the 3D illusion. This technology enables a new line of communication, which is called “holographic telepresence” that delivers a life-sized holographic experience in real time, enabling to connect more effectively and make an impact on audiences. The technology reduces expenses and saves on time travel. This project identified the parameters for correct setup of holographic telepresence, so that future users will be able to replicate and use with ease. The project used action research, which provides fast and effective solutions. The results demonstrated that it was possible that by defining the parameters and a guide for setting up a holographic telepresence.
- remote communication
- holographic telepresence
- holographic projection
Communication over time has been evolving and improving so that people could have a much simpler and easier access toward the information that they need. Even before the emergence of technology, communication has been at the forefront of relationship building and business development. Ever since smoke signals, carrier pigeons, telegraph to the modern computer, and smartphones, the thing that is common among these technologies is the way it has changed how the human being communicates.
Newer advancements like texting and messaging apps have spurred even more efficiency within workplace communication. We have come a long way since the days of written letters and memos. Even email has become a secondary form of communication in the workplace as chat platforms are taking over. Advancements in communication continue to stimulate efficiency in every workplace. These advancements continue to improve and even to displace other efforts such is the case of written letters to email, which has become a secondary form of communication. This is where the concept of “telepresence” enters, and it consists of a combination of technologies that seek to represent a person that is in a distant location as if it was there. For this, it is necessary that the user can use his senses and obtain stimuli from the remote place. It communicates the position, movements, actions, and voice, and in some cases, there can be interaction with documents and other objects. So the information that is being interchanged is wide and rich in terms of type or kind of medium. Although in telepresence systems, it is important for the images to have a real-world scale. In telepresence systems, the goal is to make the user lose the notion that has some intermediate devices with a lot of technology and that can act in a natural way.
Over 30 years ago, MIT professor and artificial intelligence pioneer Marvin Minsky laid out an ambitious plan calling for the development of advanced teleoperated robotic systems that would usher in a “remote-controlled economy.” He wrote about it in the science and science fiction magazine Omni in 1980. In his essay, Dr. Minsky envisioned a “remote-controlled economy.” He coined the term “telepresence” to describe these systems, which in his futuristic vision would transform work, manufacturing, energy production, medicine, and many other facets of modern life . He considered that the biggest challenge in developing telepresence is to achieve that sense of “being there.” According to the company, Digital Video Enterprises, telepresence refers to technologies that allow a user to appear to be present, feel like they are present, or have some effect in a space the person does not physically inhabit. Telepresence can include video teleconferencing tools, where a picture and audio stream is conveyed to a remote location. It is a multidisciplinary combination of communication that integrates engineering, psychology, and the television broadcasting .
2. Pepper’s ghost effect and its evolution
Since 1863, John Pepper and Henry Dircks developed the “Peppers Ghost Effect,” by showing a play of Charles Dickens’s The Haunted Man that basically involves a stage that is specially arranged into two rooms, one that people can see into or the stage as a whole and a second that is hidden to the side, the “blue room” (Figure 1). A plastic foil is a polymeric mirror with a specific formulation, thickness, and oriented in a way to maximize holographic effect. This film is angled so that whatever it will reflect can be hidden from the audience in a secret room. The hidden room is an entirely black mirror-image of the stage where the actual “ghosts” are placed. When it is time to make the ghosts appear in front of the audience, the hidden figures are lit and their reflection appears in the glass. The figures in the mirror-image room will be arranged so that their reflection corresponds with where they should appear on stage. For example, if you wanted to make a ghost appear at a table, the room visible to the audience would already have a table and a chair in it. However, in the mirror-image room, a figure would be sitting on a black chair, or similar prop, positioned so that the reflection lines up with the table and chair in the main room. When a light is turned on the figure, it will appear as though a ghost is sitting at the table visible to the audience .
Holograms have always been a subject of fascination by humans because it always seems as technology that comes from the future. Although there are already 3D holograms that give a tactile sensation, until now there is no technology that can produce a full body holographic projection of a person in any place. Currently, there is a new communication technology in development that will allow people to interact inside a control-simulated environment, even if they are thousands of kilometers apart .
Holography can create an accurate visual simulation, with total parallax: a replica of the real object made of light, which has the real object’s visual properties but is immaterial, intangible. Holographic images appear to be three dimensional and with volume and depth which can be seen with the naked eye .
Since the early 1980s, there have been experiments in projecting holographic dynamic images and transmitting them remotely. The usual cinema and television images are built on the viewpoint decided by the filmmaker: the scene is created and presented through the filmmaker’s eyes and perspective. The few spectators of the first 47-second monochromatic holographic movie, made in 1976 by the Russian scientist Victor Kumar, reported that they could see a young woman holding a bouquet of flowers .
3. What is holographic telepresence?
“It is a system that projects full-motion, realistic, and 3D images in real-time. A holographic telepresence system captures images of real, remote people and/or surrounding objects and compresses and transmits the images and sound over a broadband network.” Once transmitted, it decompresses the images and finally projects them. It also includes real-time audio communication that further enhances the realism experience. In some cases, it could truly rival with the physical presence of a user .
Holographic telepresence is the next step of communication that consist a full-motion, 3D video conferencing system that can project distant people and objects in a room, with live feed audio and video communication, ranging from remote participation in meetings and conferences to virtual on-stage appearances at concerts .
In other words, “holographic telepresence it’s the combination of one or more telepresence technologies with a holographic projection as the main medium of communication between users” . A holoprojector will use holographic technology to project large-scale, high-resolution images onto a variety of different surfaces, at different focal distances, from a relatively small-scale projection device . There are multiple intents and approaches in achieving a realistic holographic projection. Developers of the holographic display are working on a technology that will be used for teleconferencing. Thanks to its relatively low Internet bandwidth and computer processing requirements, a conversation between users that are being projected and transmitted would need the same bandwidth of a modern 2D video call .
4. Defining the problem
Standard video communication systems can be difficult to set up, challenging to use, and frequently unsatisfying in quality. Globalization has increased the need for remote telepresence systems that allow for remote collaboration among geographically dispersed colleagues and partners. Increasingly, business discussions must include not just multiple people and multiple work teams but multiple locations. Many of today’s telepresence, video communication, and collaboration tools provide an enormous productivity boost; that is why they are being put as the next medium of communication, because what is sought is to break the time and distance barrier.
While these video conferencing systems can be conducted at any time of the day, replace many in-person business trips, increase productivity by eliminating many barriers by helping decisions to be made faster, they still lack personal interaction, and in some cases, meetings require a personal touch to be successful. Video conferencing can be less personal than meeting face to face, and it can be possible to miss vital body language when you are struggling with a pixelated image or stuttering video. Setting these kinds of video conferencing in an office can be a bit expensive for small-sized companies. Simple features can fit into the budget, but if advanced features are required, then a substantial amount of expenditure must be done.
Increasing the capability of a team of people, such as approaching a complex situation, gaining comprehension, and finding solutions; wherever they are in the world.
This research seeks to define a setup and identify and set up the parameters for remote holographic telepresence communication, and it is justified by flashy, pricey room systems which have been one of the most common deployments of videoconferencing technology in the workplace. These kinds of setups are still a facet of the executive conference room that use the latest video and audio systems that the market has to offer which leads to a high-end budget .
Achieving the humanization of the virtual remote contact, stimulating teamwork and academic collaboration will fundamentally change the form in which people will communicate in the future. Being a communication system, it is logical that the whole process will be direct, obtaining an interaction and communication channel to be as natural as face-to-face communication holograms . Holographic telepresence combines technologies that already exist with a special care in the environment in which it takes place. The position of the cameras is fundamental so that the people that are receiving the transmission can appreciate real-world proportions and give continuity where it is being projected.
Holographic telepresence can revolutionize the way we think about and experience modern communication systems. In fact, it has the potential to change diverse types of communication systems. This type of technology can reduce the time, money, and effort otherwise wasted for traveling for business meetings or people that give conference or lectures. It may facilitate distance education like never before by connecting geographically remote classrooms, illustrating learning processes, and homogenizing the education level of schools and professors .
So why it is not more popular?
According to Arjona , most of the technical problems that impede a greater profusion of videoconference or remote telepresence technologies have been fixed long ago:
Image quality: 2 Mbps is more than sufficient for obtaining a similar quality as of a normal television set. The new standard H.265 is still in a development stage, and it will support ultra HD images as far as 8 Mega pixels.
Latency: the new processors are more capable of supporting the demanding requirements that video codecs require, which minimizes the time between capture, coding, transmission, and decoding of images making possible a more fluid line of communication.
Flexibility: It is now possible to establish a videoconference via the Internet, perform video calls from almost any mobile device.
Cost: the cost of webcams, laptops, projectors, and mobiles has decreased considerably.
Users are still showing signs of resistance to the use of these systems. Some of the main motives are:
Speed connection: remote telepresence requires establishing different channels of communication for audio, video, and data, and in some cases, it requires a dedicated bandwidth; all of these require a vast amount of time.
Localization: the user needs to be sited in a special room that is not always at their disposal.
Naturalness: the camera is usually located on top of a screen or in a monitor, and when the user speaks to another person, they are looking in any other place but not in the other user’s line of sight.
5. Defining the problem
The purpose of this work is to propose a setup and a method that can potentially improve and help so that others may easily carry out their own holographic telepresence communications with ease. As shown in Figure 2, this method will present the knowledge, basic requirements, and step-by-step guide, so that the users can understand and may save time in the installation and setup. This method is not intended to replace other forms of communication, but on the contrary, it is a proposal for enhancing the remote telepresence experience, so that its use can be further divulged and be as natural as other mediums of communication as a telephone, instant messaging, social networks, etc.
The motivation of this research is to improve the way holographic transmissions are taking place so that new users will be drawn to it for being practical and easy to use. Because sometimes setting these kinds of systems can be demanding, time-consuming and can demotivate and turn away potential users (Figures 3–5).
The main objective is to establish the parameters for the correct setting of a remote holographic telepresence system as a medium of communication and support. With the purpose of demonstrating, it is possible to improve and humanize long-distance communication and interaction. During the investigation, it is expected to accelerate the setting process of a holographic projection system for future presentations, by taking measurements of the distance of the projector depending on its luminosity, room illumination, room space, dress code, Internet bandwidth, etc.
This technology will increase the reach, impact, and remembrance between work colleagues and students. It will be the next evolutionary step of videoconference systems that are currently limited to a television screen, changing the way we communicate and travel around the world.
During the development of the project, the questions that we are looking to respond to are:
What are the constant challenges that are present while setting a remote holographic telepresence system? What are the basic requirements and their classification that is needed for a live transmission? Is it possible to realize live transmissions in small clustered spaces? Is it possible to improve the communication by improving the visual perception of the image by correcting the parameters of the holographic telepresence?
Action research refers to a wide variety of evaluative, investigative, and analytical research methods designed to diagnose problems or weaknesses—whether organizational, academic, or instructional—and help educators develop practical solutions to address them quickly and efficiently. Action research may also be applied to programs or educational techniques that are not necessarily experiencing any problems but that educators simply want to learn more about and improve .
Action research is a form of collective introspective inquiry undertaken by participants in social situations with the objective of improving the rationality and justice of their social practices or education, as well as the comprehension of these practices and the situations that they take place in.
It is a form of research that binds the experimental focus of social science with social action programs that respond to social principal problems. Because of social problems that emerge from the usual, action research starts the questioning of the phenomena from the usual, traveling systematically, as far as philosophical. Through action research what it is intended to treat in a simultaneous way are knowledge and social changes, in a manner that theory and practice unite.
The process of action research consists of:
Unsatisfaction with the current state of things (observe)
Identifying the problem area (observe)
Identifying a specific problem to be solved by action (think)
Formulation of multiple hypotheses (think)
Selecting a hypothesis (act)
Executing the action to prove the hypothesis (act)
Evaluation of the effects of the action
Practitioners who engage in action research inevitably find it to be an empowering experience. Action research has this positive effect for many reasons. Obviously, the most important is that action research is always relevant to the participants. Relevance is guaranteed because the focus of each research project is determined by the researchers, who are also the primary consumers of the findings. Therefore, the proposed research methodology for this project will be of a qualitative nature, because it will allow the use of different sources of information and will produce descriptive data (peoples own words, written or spoken, and observable behavior). Its objective is the description of qualities of a phenomenon, since it seeks a deep understanding about the research topic. This methodology is of inductive character; it helps to understand the context and the people under a holistic perspective, that is, they are not reduced to variables, if not considered. It studies people in the context of their past and in the situations that they are in. Qualitative investigation is flexible as to how to conduct studies, it follows oriented guidelines, and its methods are at the service of the researcher, which is not dependent on a single procedure or technique. When using the qualitative methodology, one can obtain rich and profound information of which can draw inferences from data. It is conceived that qualitative methods are the first level of approach to reality, so that later a second level is possible to take on a more rigorous and profound methodology .
The qualitative methods to be used are interviews, action research, and observation. In some cases, surveys will be used as a quantitative method, with the sole purpose to enrich the investigation.
As a procedure of data recollection that allows to obtain information about a phenomena or event as it occurs. In some investigation processes where subjects are needed that cannot provide verbal information, observation is used as a data gathering method.
It is a technique in which a person solicits information from another one or from a group, to obtain data from a specific problem. It is believed to provide a deeper understanding of the experiment.
The quantitative methods to be used are:
It is a brief interview or discussion with individuals about a specific topic. It is a term often used to mean collect information. In this case, the survey is a list of questions aimed at extracting specific data from a specific group of people. The survey will be applied by email; a closed-ended questionnaire will be sent to the participants of the experiment.
7. I challenge
What happens when you put two minds in the same physical room? The exchange of ideas, talent, knowledge, and creativity. That is what holographic and telepresence technologies are going to enable. The level of engagement and interaction will become more human and meaningful.
In the search of making the telepresence experience the best possible, the Tecnológico de Monterrey research group has arrived at holographic projection. In this context of new technological resources, in the digital world of constant transformations, where this research group is currently working on, they developed an educational innovation that proposes holographic projections in real time, in which the professor can be seen and heard by the students, through a holographic projection that includes sound and voice in two ways.
Telepresence through holographic projection has been used in recent years as a manner of delivering conferences in international conferences. But toward giving official lectures in college level, there is only one record of initiatives on an experimental level. The telepresence with the holographic projection applied in a college course will allow the professor to, without limits as far as distance, weather, time difference, etc., give his class on time and in a form that while he is not physically there in the classroom. This technology enables cost saving because it will not be necessary to travel to other cities just to give a class, conference, or meeting. The use of the holographic projection is not only merely academic, it can be versatile and multipurpose in universities, for example, a directive that is out of town can attend a meeting through holographic projection .
In the University of Tecnológico de Monterrey, professor Eduardo Luévano from campus Zacatecas has been doing research and working on making the long-distance education process more efficient. His research is centered on the telepresence concept. He and his research team were searching to improve the telepresence sensation given by the professor, so they proposed to integrate a complement to long-distance education, which is holographic projection (Figures 6 and 7). They believe that integrating this technology with videoconference and telepresence robot can assemble a technological package that will allow supplying, but never to replace, the temporary physical absence of the teacher in the classroom .
An initiative called “Reto i” (i challenge) is a collaboration network that was launched to a group of universities across all over Latin America that used multiple telepresence technologies ranging from traditional videoconference systems as Skype, telepresence robots, to a holographic display system. This was done to demonstrate that with the use of these technologies a professor or instructor can offer support or advice for multidisciplinary groups.
The designed instruments used were surveys, photographic recollection, and field notes. Some of the main results of the instruments were analyzed to prove the project impact:
87% perceived the holographic projection as the social presence of their professor.
86% of the students were satisfied with the project.
88% of the students felt comfortable with the “Professor Avatar.”
93% would recommend this model to other students.
97% would participate again in telepresence projects.
Therefore, the engagement of the students is highly positive and productive.
The 1-week i challenge required students to construct a sustainable electric generator using recycled material found in dumpster and recycling centers that were useful to solve the necessity of a local community in poverty. Facing a real problem in a community promoted between the students a social commitment that allowed them to relate what they learned in a classroom to practice, work collaboratively, and to develop decision-making, communication, and leadership skills offer sponsorship for the project  .
Basic metrics to evaluate the students work in the 1-week i challenge were:
|Energy generation (Watts-hour)||40%|
|Low cost (up to 100 USD)||40%|
|Social impact (people benefited)||20%|
The student survey applied by the end of the project showed the following results:
|100%||Considered that telepresence contributed to learning improvement.|
|87%||Considered that the activity goal was met.|
|97%||Considered that new skills were developed.|
|98%||considered that the i challenge helped to get them involved in their social, economic, and environmental reality.|
The result of the project has five generators were constructed at full completion of the task requirements. By the end of i challenge, every institution donated the generator to a local vulnerable zone.
The scalability and potential of the project have motivated corporations as Samsung, BlackBoard, and Prezi to offer sponsorship for the project .
7.1 Setting up the remote holographic telepresence system
The key factors that define the correct setup for this means of communication are listed and described and divided into three categories: structure/components, software, and ambient considerations.
The essential structure and components for the parameters that are considered in this study are:
Transparent glass or acrylic. It is necessary to construct a projection screen in each site viewing the holograph. The screen is a pane of tempered glass or transparent acrylic, preferable 2 m high and at least 1.3 m wide. This transparent pane should be held upright in a sturdy manner and should have nothing immediately behind it.
Holographic film. This is a polarized semitransparent film sticker that adheres to a transparent glass or acrylic screen. The film has crystalline nanostructures that retain light emitted by the projector; this produces a holographic effect when the film is adhered to a glass or acrylic pane.
Computer. A computer is necessary for live transmission or for reproduction of recorded videos. The computer can be a desktop or laptop. A computer with more RAM memory and a faster processor will render greater image stability and quality.
Internet connection. For live transmission, it is critical to have a good internet connection, preferably 10 Mbps or greater. Connecting by a cable (Ethernet) is recommended over Wi-Fi because a cable generally provides a faster and more stable connection. If possible, we recommend designating an exclusive internet channel for transmission, and if this is not possible, it is recommended that participants in the session turn off the internet access of their mobile devices so that it will leave open bandwidth in the area.
Projector. Another key component is the projector. Here are some considerations:
Use a projector of at least 3500 lumens.
Long throw (standard) projectors emit uniform luminosity. When projecting an entire person in life-sized scale, the entire body is correctly illuminated. The restriction of long throw projectors is that they require a greater distance between the projector and the screen to achieve a life-size scale.
Short throw projectors have the advantage of reducing the distance necessary between the projector and screen to achieve life-size scale. The disadvantage of these machines is that they emit nonuniform luminosity. For example, when projecting an entire human at a life-size scale, only the upper half is well illuminated, while the legs fade out of sight. We recommend short throw projectors for half-body images. One example would be a person sitting at a desk and visible from the surface of the desk upward.
The projector is placed behind the holographic screen shining toward the audience, but at a slight angle so that its light does not hit the audience directly in the eyes. The holograph will appear uniformly visible to the audience.
Webcams and cameras. The quality of the holographic image depends on the webcam used by the presenter. For this reason, the recommendation of external HD webcams rather than those cameras integrated into laptops. Also an external webcam that is connected via the USB port can be more easily manipulated, and its position can be adjusted, so that the audience and the speaker can see either part with a better view.
Audio equipment. A small audio system (desktop speakers, Bluetooth speakers, surround sound, etc.) is necessary for transmitting voice and sound clearly to the entire audience. The dimensions of your audio system should correspond to the dimensions of your classroom or auditorium.
Black background for transmission and recording. The background can be a paper cyclorama or a fabric with no sheen, such as muslin. The black background is necessary during recording because it disappears during projection and leaves only the body of the professor in the holograph.
Lighting equipment. Professional lighting equipment is not necessary. It is necessary to fully illuminate the presenter, especially above the waist. Artificial light is preferable to natural light because it is controllable. Further, excessive light should be avoided because this causes shadows behind the professor or causes the backdrop to glow behind the professor. Lamps with dimmers can be useful for controlling the level of illumination. The ideal color temperature should be between 3000 and 3500 K .
When you light the background of the screen with artificial lighting with dimmers, it makes the hologram illusion even more believable.
The critical and essential software parameters that are considered in this study are:
Videoconference software. For this exercise, the videoconference that was used was Skype desktop app.
Strong daylight. This issue is pertinent in any situations where projectors are being used as a source of video. Despite manufacturers having made great strides in the last couple of years with the brightness of their projectors, there will never be anything available that will be able to compete with natural daylight. In circumstances of strong natural daylight, the options are either to create a controlled light environment or make a stage with ceiling and walls.
Space. It is important to remember and to consider the transmission and the reception area. This is a key factor because it determines the conditions in which everything else is going to be set up.
For the transmission area, it is recommended that there should be sufficient space for at least an average adult to be standing in front of the black background so that the camera can capture the subject for a full body transmission. The recommended distance between the camera and the speaker should be between 1.5 and 2 m for a full body presentation.
For the reception area, it is recommended that there should be sufficient space for the audience. Here the factors depend on how many spectators are going to be, the size of the room or auditorium, etc. In the case that the projector being used is a long throw projector, it needs to be behind the holographic screen from a faraway distance, so that the image that is being projected can be adjusted to human scale. With short throw projectors, the distance from the holographic screen can be dramatically reduced, but using these projectors may reduce quality. For this purpose, it is deeply recommended set the reception of the transmission to test prior to a conference or lecture and to leave some marks of the position of everything.
Dress code. We recommend the presenter wear light colors because dark colors can be confused with the black background and disappear in the projection. Do not wear black. Avoid wearing elaborate patterns as these may project with a poor resolution depending on the quality of internet and other equipment.
In Figure 7, it is illustrated how the proposed remote holographic telepresence reception should be setup according to the requirements mentioned before. In this case, it will be used a long throw projector that is recommended to be set at 5 m behind the holographic screen, so that it will be a lot easier to adjust the image to a full human scale body (as illustrated in the figure). For the position of the webcam, the ideal position should be at the eye level of the spectators, but if that is not possible, another recommendation will be to put it on a location that gives it a broad field of view of the target audience, so that the speaker may be able to see who he is addressing.
In Figure 8, it is illustrated how the proposed remote holographic telepresence transmission should be setup according to the requirements mentioned before. In the figure, it is shown that the user needs to be in front of the black background. The position of the webcam needs to be adjusted so that the only thing visible that will be transmitted will be the black background and the user and nothing else. Depending on the space where the transmission will be taking place and the type of lighting, it may be necessary to have a couple of fill lights, so that the user may appear well lit.
It was verified that by identifying the parameters for a remote holographic telepresence transmission, challenges and complications could be identified; also it was possible to streamline and facilitate the assembly process, since this series of instructions served and could be replicated easily in another place without major problems. By correcting the parameters, it was possible to change drastically the quality of the image both recorded and live transmission.
As for the transmission, it was possible to do it in a reduced space of 0.5–2 m of distance of the camera, although only the projection left in half body. But for the reception of the holographic projection due that the work was made with a long throw projector the most that the space could be reduced space sacrificing human scale was 4 m of distance between the projector and the holographic screen. Tests with short throw projectors were not made due to the fact that the equipment constantly failed and overheated which was useless to test on.
3D holographic projection technology clearly has a big future ahead. As this audiovisual display continues to get high profile credibility, we are likely to see more companies advertising their products or marketing their business in this way. Holographic telepresence can revolutionize the way we think about and experience modern communication systems. In fact, it has the potential to change diverse types of communication systems.
Holographic projectors will be able to render sharp projected images from relatively small projection devices (e.g. cell phones) because they do not require high intensity, high-temperature light sources. Researchers at different industries and schools are working toward applied science that could make real-time holographic projections in everyday-used devices .
I wish to thank Dr. Eduardo González Mendívil for opening the doors to the research group and providing constant support in this journey and the members of the Thesis Committee, M.Sc. Pablo Guillermo Ramírez Flores, and Dra. Norma Patricia Salinas Martínez, for having shared projects together helping to develop my skills in each of these.
To my team members and colleagues M.Sc. Gabriel Pantoja García, Engineer Héctor Eduardo Ramírez, professor Luis Eduardo Luévano Belmonte, with whom I collaborated in different virtual and augmented reality and holographic telepresence projects. As well all the people involved in Virtual and Augmented Reality in Education (VARE 2015) conference and in the RWTH-Tecnológico de Monterrey holographic telepresence transmission.