Holographic Pepper’s Ghost: Upright Virtual-Image Screen Realized by Holographic Mirror

A holographic mirror is a reflection-type holographic optical element that works as an off-axis mirror. It realizes an upright see-through screen serving as a virtual-image display and virtual camera. Such screen enables to realize virtual-image-based attractive applications like Pepper’s ghost only with a thin optical system. This chapter describes the concept of a holographic-mirror-based virtual-image display and virtual camera, an experimental method for exposing the holographic mirror based on holographic printing, methods for dispersion compensation, and experimental results for the proposed virtual-image display and camera.


Introduction
The fusion of optical images and real objects has been an interesting topic in the field of optics and information technology. A famous example is Pepper's ghost [1], which was invented over 100 years ago. In Pepper's ghost, a virtual image is displayed on real objects by using a slanted half mirror, which can realize surprising visual experiences like optical illusions. The perception of cyber-physical fusion using virtual images mainly relies on the imperceptibility of the frame of the display, which is caused by an axial displacement between the image plane and the screen plane. Recently, such technology has been revisited in the context of augmented reality (AR). For example, virtual imaging has been used in various applications from head-mounted displays (HMDs) [2] to public theaters [3], where digital images are displayed as overlapping on real objects.
To realize the virtual-image-based applications only with a thin optical system, we have proposed a new optical system that integrates an HOE-based mirror referred as holographic mirror, dispersion-compensation optics, and a digital projector [21]. We also showed that a similar optical design can be applied to the realization of a virtual camera, by using a virtualization method of a camera device based on off-axis image capturing [21]. In this chapter, we describe background on the virtual-image-based applications in Section 2, a method for exposing a holographic mirror in Section 3, the concept and verification of the proposed virtualimage display in Section 4, and the concept and verification of the proposed virtual camera in Section 5. More detailed background information for the work described here is given in [21].

Holographic Pepper's ghost
Pepper's ghost is an illusion technique exploiting virtual images. Since the virtual image is formed outside the frame of a display, it is perceived as if it was appearing on the air. This feature is useful for realizing the unconventional visual systems based on cyber-physical fusion, which is recently referred as AR technology. This kind of visual applications can provide attractive and surprising user experiences such as the ultra-realistic telepresence system.
The classical realization of an optical system for the Pepper's ghost is based on the use of a slanted half mirror, like that in Figure 1(a). It is simple to realize this arrangement of the optical system; however, the optical system will likely be bulky due to the tilted alignment. If the screen for a virtual-image display could be implemented in an upright alignment like Figure 1(b), it could be integrated with flat walls, doors, windows, and existing 2-D screens. Such usage might be interesting because it allows ordinary environments to be converted into screens for virtualimage display. For instance, an ordinary wall can serve as a screen for a virtual-imagebased video-communication system [22]. Figure 2 presents the concept on such system realized by the holographic Pepper's ghost which is presented in this chapter. In the figure, a person is talking with a virtual image of another person on a real chair behind an upright window with achieving the line of sight. Exploiting the feature of the holographic mirror, a display for virtual-image formation can be realized by an upright thin screen unlike the conventional Pepper's ghost with a slanted half mirror.
An HOE can be used for realizing such an upright virtual-image screen. Since an HOE is a kind of hologram, flexible optical functions can be implemented on a thin  flat film by means of wavefront recording and reconstruction. For example, it is possible to realize a holographic mirror which works as an off-axis mirror by Bragg diffraction. The holographic mirror can be used for an upright virtual-image screen as mentioned above and shown in Figure 1(b). One problem in applying a holographic mirror to a virtual-image screen is the chromatic dispersion caused by diffraction, which results in spatial blurring of the virtual image. This problem can simply be solved by using a laser light source; however, especially when presenting a large, deep virtual image, safety and speckle become problems. Insertion of a band-pass filter is another possible solution [16]; however, this reduces the light-use efficiency.

Exposure of a holographic mirror using a hologram printer
A holographic mirror can simply be implemented by exposing a photosensitive material using two coherent parallel beams. Figure 3(a) shows the experimental setup used in our experiment. In the setup, a diode-pumped solid-state (DPSS) laser (Samba 100 mW manufactured by Cobolt, 532 nm) was used as a light source. A half-wave plate (HWP) and a polarization beam splitter (PBS) were placed in front of the laser to split the beam with a controlled intensity ratio. In addition, an acousto-optic modulator (AOM) was inserted to function as an electrical shutter. The two beams were delivered by polarization-maintaining single-mode optical fiber (pmSMF) to regions close to the photosensitive material. We used a photopolymer (Bayfol HX200 manufactured by Covestro) as the photosensitive material. The photopolymer was exposed by the interference fringes. We adopted the scanning-based exposure method called holographic printing [23][24][25][26][27][28][29] to realize spatially uniform diffraction efficiency of the holographic mirror. The photopolymer was mounted on a two-axes-motorized stage for scanning.
The angle between the two beams incident on the photopolymer was set to 135 ° . The duration of each exposure was 10 ms, and the diameter of each beam on the photopolymer was 1.1 mm. The total number of scans was 175 × 175 . The spatial interval of each scan was 0.8 mm, and the time interval between scans was 2 s. The overall size of the holographic mirror was 14 × 14 cm. The appearance of a virtual image of a white paper including a star symbol formed with an exposed holographic mirror is shown in Figure 3(b). Thanks to the holographic printing, the brightness of the image was spatially uniform. The diffraction efficiency measured based on ISO 17901-1 was 72.4% at 526.4 mm. More detailed information is given in [21].

Virtual-image display using a holographic mirror and dispersion-compensation optics
Since the holographic mirror is a volume hologram, the diffracted light should disperse chromatically, and this chromatic dispersion results in spatial blurring of the virtual image. The size of the blur caused by dispersion can be modeled as follows: where b is the size of the blur, z is the depth of the virtual image from the holographic mirror, ∆ λ is the range of transmissive wavelengths through the holographic mirror, λ is the central wavelength of the propagating light, and θ is the diffraction angle of the holographic mirror. For a more generalized model, see [21]. As indicated by Eq. (1), the size of the blur linearly scales with the depth of the virtual image, the spectral width of the light, and the diffraction angle of the holographic mirror.
To suppress the blur, dispersion compensation is necessary. Figure 4 illustrates the concept in the case of an integrated optical system with a holographic mirror, projection optics, and blur-compensation optics. The key idea is the replacement of a real display with an intentionally dispersed image, which contributes to dispersion compensation of the holographic mirror. In the system, a projector projects an image on a diffuser via a diffractive optical element (DOE). In such an optical system, a dispersed image appears on the diffuser screen. If the direction and the amount of dispersion are correctly designed for dispersion compensation, the spatial blur of the virtual image can be compensated. As a result, an observer can see a sharp virtual image through the holographic mirror. As a related method, dispersion compensation using two identical HOEs was proposed [30,31]. Compared with the conventional method, the advantages of the proposed DOE-based method are superior light-use efficiency and a practical level of blur suppression [21]. As mentioned also in the Introduction section, another related method is to limit the spectral width by using a laser light source or a bandpass filter. Compared with this method, our method has merits from the perspective of safety and brightness.
We verified the proposed method using the setup in Figure 5. We placed a reflection-type DOE (VIS Holographic Grating manufactured by Edmund optics) having 1200 grooves per millimeter in a 50 mm square in front of a projector (EH-TW5200 by EPSON) with an internal metal halide lamp. We also placed an A4-sized diffuser screen and a holographic mirror described in the previous section. We set the distance between the diffuser and the holographic mirror to 600 mm, and that between the DOE and the diffuser to 600 mm. The design conditions for the system parameters are presented in [21]. Figure 6(a) shows the images projected on the diffuser without and with a DOE. The images without the DOE were generated by replacing the DOE with a mirror. As shown in the figure, the image with the DOE was chromatically    dispersed along the direction of diffraction (in this case, vertical direction). The images were severely blurred in direct observation; however, these are the expected results. Figure 6(b) shows the virtual images generated without and with a DOE, captured by a camera placed at the observer's position. The results without the DOE indicate that the virtual images were blurred along the dispersion direction. In contrast, the virtual images with the DOE were successfully resolved even along the dispersion direction. Using a resolution chart, the vertical resolution was improved from 0.12 to 0.42 cycle/mm. Figure 7 shows the depth of the virtual images from multiple observations while changing the observer's position. As indicated in the figure, motion parallax was confirmed experimentally with a blur-compensated virtual image. In addition, since the camera focused on the virtual mirror, the holographic-mirror screen was defocused. These observations show the displacement of the axial position of the holographic mirror working as a screen and the displayed virtual image.

Virtual camera using holographic mirror and dispersion-compensation optics
The virtual-image display having the geometry in Figure 1(b) can also work inversely by replacing the projector with a camera, which can realize a virtual camera [21]. A virtual camera is a camera in which a virtual image of a real camera captures subjects, which allows off-axis image capturing. One benefit of off-axis image capturing is that frontal shooting of the subject can be accomplished by a camera placed at an invisible position. Figure 8 illustrates the concept. In the figure, an observer in front of a holographic mirror is captured from the front as if an invisible camera were placed behind the mirror, where a real camera device is placed at an off-axis position. By integrating the virtual camera with the virtual-image display, a virtual-image screen that can also capture frontal images can be realized. Such a screen can be applied to, e.g., a virtual-image-based video-communication system that achieves line of sight image capturing [22].
To make use of the virtual camera with an upright holographic mirror, dispersion compensation is needed, as with the virtual display described above. In principle, the same optical system as that used for the virtual-image display can be adopted for the virtual camera; however, the insertion of a diffuser is not suitable for image capturing because the light intensity is severely reduced, and the intensity of environmental light sources (e.g., sunlight) cannot be controlled in general. To deal with this problem, we propose an alternative optical design without a diffuser for dispersion compensation. Figure 9 shows the concept of the proposed optical system. Compared with the display application in Figure 4, the diffuser and the projector are replaced with a convex lens and a camera, respectively. In this configuration, the light source is environmental illumination such as sunlight or room light. A convex lens is used for converting the spectrally diverging dispersed light into converging light. As a result of inserting the lens, a real image of the subject is optically formed between the lens and the DOE. A camera captures the formed real image via the DOE. Since the light is not diffused in the optical system, the light-use efficiency is superior to that of the diffuser-based display system, but on the other hand, the acceptable positions of the camera for capturing the image are restricted. Figure 10 shows the setup used for experimental verification. The holographic mirror and the DOE are as same as those described in the previous section. The diameter and focal length of the lens were 100 and 300 mm, respectively. A color CCD camera (Flea3 manufactured by FLIR) was used for image capturing. The   distance between the holographic mirror and the lens was 590 mm, and that between the lens and a DOE was 680 mm. Design conditions for the system parameters are given in [21]. Figure 11 shows the images experimentally captured by the proposed virtual camera system. Without a DOE, the chromatic dispersion of the holographic mirror degraded the vertical spatial resolution of the captured image. In contrast, with the DOE, the resolution degradation was successfully restored. By visual assessment of the images of a resolution chart, the vertical spatial resolution was improved from 0.80 to 1.46 cycle/mm. The result with a doll indicates the possibility of the proposed optical system for visual video-communication systems for human users.

Conclusion
In this chapter, we introduced a technology on the holographic Pepper's ghost based on a virtual-image display and a virtual camera using a holographic mirror and blur-compensation optics. The holographic mirror works as an off-axis mirror, which can be used for an upright screen of the virtual-image display and the virtual camera. To make use of the holographic mirror in imaging systems, compensation © 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. of chromatic dispersion is necessary for preventing resolution degradation. We proposed two optical systems that integrate DOE-based dispersion-compensation optics, imaging devices, and a holographic mirror. In the systems, the chromatic dispersion of the holographic mirror was compensated optically. We experimentally verified the realization of the concepts on the virtual-image display and the virtual camera, and the effectiveness of the dispersion compensation.
The proposed systems can be applied to upright, thin, see-through screens for virtual-image displays and virtual cameras. The system can be used for, e.g., virtual-image-based interactive displays and video-communication systems where the screen can be integrated with environmental objects like flat walls and screen panels.