3D Capture and 3D Contents Generation for Holographic Imaging

Abstract

The intrinsic properties of holograms make 3D holographic imaging the best candidate for a 3D display. The holographic display is an autostereoscopic display which provides highly realistic images with unique perspective for an arbitrary number of viewers, motion parallax both vertically and horizontally, and focusing at different depths. The 3D content generation for this display is carried out by means of digital holography. Digital holography implements the classic holographic principle as a two‐step process of wavefront capture in the form of a 2D interference pattern and wavefront reconstruction by applying numerically or optically a reference wave. The chapter follows the two main tendencies in forming the 3D holographic content—direct feeding of optically recorded digital holograms to a holographic display and computer generation of interference fringes from directional, depth and colour information about the 3D objects. The focus is set on important issues that comprise encoding of 3D information for holographic imaging starting from conversion of optically captured holographic data to the display data format, going through different approaches for forming the content for computer generation of holograms from coherently or incoherently captured 3D data and finishing with methods for the accelerated computing of these holograms.

1. 3D capture and 3D content generation by digital holography

Three‐dimensional (3D) displays are the next generation displays. The claim for 3D imaging is indisputable in mass television, game industry, medical imaging, computer‐aided design, automated robotic systems, air traffic control, education and cultural heritage dissemination. The ultimate goal of 3D visual communication is 3D capture of a real‐life scene that is followed by creating its scaled exact optical duplicate at a remote site instantaneously or at a later time moment [1]. Tracking the development of 3D imaging devices from Wheatstone stereoscope designed in 1830 to modern full HD 3D displays with glasses (Figure 1) reveals some memorable periods of booming public interest to this area, e.g. the 3D theatres boom observed in 1950 as a counterpoint to the increasing popularity and commercialization of the television or the last years reaction to the ‘Avatar’ movie. Multiple parallax 3D display technology has evolved to full‐parallax displays for a naked eye observation as integral imaging displays [2] and super‐multi‐view displays [3]. However, since invention of holography by Dennis Gabor in 1948 [4] and the first holographic 3D demonstration by Emmett Leith and Juris Upatnieks in 1964 [5], the consumer public has high expectations for a truly holographic display in view that holographic imaging is the best candidate for 3D displays.

Figure 1.

History of 3D displays with some of the major events depicted as a diagram with years along the horizontal axis and intensity of the public interest along the vertical axis.

Holography is the only imaging technique which can provide all depth cues. High quality of 3D imaging from analogue white‐light viewable holograms is well known [6]. They provide a wide viewing angle due to the very fine grain size of the holographic emulsions. Realistic images can be viewed by an arbitrary number of viewers with unique perspectives. Motion parallax both vertically and horizontally and tilting of the head are possible. The viewer is capable of focusing at different depths. There are no convergence‐accommodation conflict and discontinuities between different views as for multiple parallax displays. Holographic imaging allows for building autostereoscopic displays. To realize a holographic display, the 3D scene description should be encoded as two‐dimensional (2D) holographic data. The 3D content generation is carried out by means of digital holography [7] based on the classical holographic principle. According to it, holography is a two-step process enabling storing and reconstruction of a wavefront diffracted by 3D objects. The hologram records as a 2D intensity pattern the interference of this wavefront with a mutually coherent reference wave as depicted schematically in Figure 2. The wave field from the object is characterized by a complex amplitude, $O\left(x,y\right)=a_O\left(x,y\right)\exp \left[j{\varphi }_O\left(x,y\right)\right]$, where $a_O\left(x,y\right)$ and ${\varphi }_O\left(x,y\right)$ are the amplitude and phase of the object beam, respectively. Interference of $O\left(x,y\right)$ and the mutually coherent reference beam $R\left(x,y\right)=a_R\left(x,y\right)\exp \left[j{\varphi }_R\left(x,y\right)\right]$ results in four terms superimposed in the hologram plane (x, y):

$\begin{array}{c}{I}_H\left(x,y\right)={\left|\mathit{O}\left(x,y\right)+\mathit{R}\left(x,y\right)\right|}^2\\ =\mathit{O}\left(x,y\right){\mathit{O}}^{*}\left(x,y\right)+\mathit{R}\left(x,y\right){\mathit{R}}^{*}\left(x,y\right)+\mathit{O}\left(x,y\right){\mathit{R}}^{*}\left(x,y\right)+\mathit{R}\left(x,y\right){\mathit{O}}^{*}\left(x,y\right)\end{array}$E1

Figure 2.

Schematic representation of holographic recording.

where the asterisk denotes a complex conjugation. The sum of intensities of the object and reference beams gives the zero‐order term. The last two terms are the +1 and –1 diffraction orders and contain the object wavefront information. The object field $\mathit{O}{\mathit{R}}^{*}\mathit{R}=\mathit{O}$ or ${\mathit{O}}^{*}\mathit{R}{\mathit{R}}^{*}={\mathit{O}}^{*}$ brings into focus the virtual or the real image when $I_H\left(x,y\right)$ is multiplied by $\mathit{R}\left(x,y\right)$ or its conjugate. These twin images are separated for the off‐axis geometry where the object and reference beams subtend an angle $\theta$, and overlap in inline geometry at $\theta =0$.

Digital holography grew from a purely academic idea into a powerful tool after the recent progress in computers, digital photo‐sensors (CCDs or CMOS sensors) and spatial light modulators (SLMs). The capability of digital holography for digital analysis and synthesis of a light field forms two mutually related branches. The branch dedicated to analysis comprises methods for optical recording of holograms using digital photo‐sensors. The holograms are sampled and digitized by the photo‐sensor, stored in the computer and numerically reconstructed using different approaches to describe diffraction of light from the hologram and free space propagation to the plane of the reconstructed image [8]. Thus, capture of both amplitude and phase becomes possible enabling numerical focusing at a variable depth and observation of transparent micro‐objects without labelling [9]. The holographic data are in the form of a real‐valued 2D matrix of recorded intensity according to Eq. (1). In this branch, different techniques have emerged for the two decades of existence of digital holography as (i) digital holographic microscopy with a plane wave or a point‐source illumination [10, 11]; (ii) optical diffraction tomography with multi‐directional phase‐shifting holographic capture [12]; (iii) infrared holography in the long wavelength region for capture of large objects [13]; (iv) determination of sizes and locations and tracking of particles in a 3D volume [14]. Feature recognition based on digital holography has been proposed [15]. A lot of efforts were dedicated to instrumental or software solutions of the twin image problem [16].

The branch dedicated to synthesis of a light field comprises methods for computer generation of holograms [17] which are fed to some kind of SLMs for optical reconstruction of the images they encode. Computer‐generated holograms (CGHs) are used for holographic displays [18], holographic projection [19] and diffractive optical elements [20]. In principle, the CGHs provide the only means to generate light fields for virtual 3D objects. A CGH is a real‐valued 2D matrix of amplitude or phase data; it may have also binary representation.

Both branches are closely related to the task of direct transfer of optically captured digital holograms to a holographic display. To realize the chain 3D capture—data transfer—holographic display digital holography requires coherent light and 2D optical sensors and SLMs with high resolution and big apertures. The 4D imaging, when the time coordinate, is added to the data further aggravates the task of building a holographic display because the latter needs much higher resolution and much more information capacity than other types of 3D displays. Generally speaking, there are two ways for 3D content generation for holographic displays: (i) conversion of optically captured holographic data; (ii) computer generation of holograms. Furthermore, we discuss these two main tendencies—direct feeding of optically recorded digital holograms to a holographic display and computer generation of interference fringes from directional, depth and colour information about the 3D objects—on the basis of our experience in forming the holographic content.

2. 3D content generation from holographic data

The ultimate goal of digital holography is to build a system for 3D scene capture, transmission of captured data and 3D optical display. Although based on clear theoretical grounds given by Eq. (1), it is hard to fulfil this task because of limitations encountered at digital implementation of the holographic principle due to the discrete nature of photo‐sensors and display devices, their small size and low spatial resolution. The modern devices are characterized with a pixel periods from 1 to 20 μm and active areas from 1 up to 2–3 cm². 3D content generation from optically captured digital holograms should include three steps: (i) multi‐view capture by a set of cameras or by sequential recording from different perspectives (Figure 3); (ii) conversion of the captured data to a display data format; (iii) feeding the data to a display from many SLMs to enlarge the viewing angle (Figure 3).

Figure 3.

Schematic diagram of multi‐view holographic capture by many digital photo‐sensors and multi‐view optical reconstruction after mapping the 3D contents to a display set of SLMs.

A key problem of digital holographic capture and imaging is the very small value of the maximum angle between the object and reference beams which satisfies sampling requirement for the spatial frequency at current low spatial resolution of electrically addressable devices. In theory, the photo‐sensor must resolve the fringe pattern formed by interference of the waves scattered from all object points with the reference wave. The holographic display should support some space‐bandwidth product with regard to limitations of the human visual system. The maximum angle, ${\theta }_{\max }$, between the reference and the object beams that satisfies the Whittaker‐Shannon sampling requirement for a wavelength ${\lambda }_{c,d},$ and a pixel period ${\Delta }_{c,d}$, where ‘c’ and ‘d’ are attributed to capture and display devices, is found from

$\sin \left(\frac{{\theta }_{\max }}{2}\right)=\frac{{\lambda }_{c,d}}{4{\mathrm{\Delta }}_{c,d}}$E2

The limitation set by Eq. (2) means capture of small objects at a large distance from the camera and a small viewing angle at optical reconstruction. If the object lateral size D is much greater than the sensor size, the minimum distance between the object and the photo‐sensor is about $z_{\min }=D{\Delta }_c/{\lambda }_c$. Usage of coherent light seriously restricts the viewing angle of the holographic display and the size of the reconstructed image [21]. A planar configuration of many SLMs allows for visualization of larger objects [22] but the problem with the small viewing angle remains. Enlarging the viewing angle for pixelated SLMs by using higher diffraction orders and spatial filtering is proposed in Ref. [23]. Under coherent illumination, a circular arrangement of SLMs puts less severe requirements to the space‐bandwidth product of the display and supports full‐parallax binocular vision at an increased viewing angle. Different circular configurations have been recently proposed [24–26].

Effective operation of the holographic imaging system requires maintaining the consistent flow of data through capture, transmission and display blocks. So the other problem of 3D content generation is non‐trivial mapping of the data from 3D holographic capture with non‐overlapping camera apertures to arbitrary configuration of display devices (Figure 3). In the general case, the wavelength, the pixel period and the pixel number differ at capture and display sides, i.e. ${\lambda }_c\ne {\lambda }_d,{\Delta }_c\ne {\Delta }_d,N_c\ne N_d$. This alters the reconstruction distance and the lateral and longitudinal dimensions of the reconstructed volume [27]. Another difficulty arises from the requirement the set of digital holograms captured for multiple views of the 3D object to be consistent with the display configuration built from many SLMs. Although both the amplitude and the phase can be retrieved from the captured holograms, the type of the SLM entails encoding the holographic data only as amplitude or as phase information. To illustrate the non‐triviality of 3D content generation from optically recorded digital holograms, we consider two characteristic examples from our experience with data mapping. The detailed description of the capture and display systems is given in Refs. [25, 28]. Here, we focus only on data transfer from the holograms to SLMs.

In the first example, the capture parameters substantially differed from the parameters on the display side. The mapping was done for a circular holographic display under visible light illumination when the input data were extracted from a set of holograms recorded at 10.6 μm [29]. The interest in capturing holograms in the long wavelength infrared region is due to the shorter recording distance, larger viewing angle and less stringent requirements to stability of the system. The object was a bronze reproduction of the Benvenuto Cellini Perseus sculpture with a height of 33 cm [28]—a large object for digital holography. Nine off‐axis digital holograms were captured by rotating the object with an angular step of 3° using an ASi (amorphous silicon) thermal camera with $N_c=n_{xc}\times n_{yc}=640\times 480$ pixels and ${\Delta }_c=25\text{μm}$. The object beam interfered with a spherical reference wave given in paraxial approximation in the plane of the photo‐sensor as $R_c\left(x_c,y_c;r_c\right)=\exp \left[-j\pi \left(x_c^2+y_c^2\right)/\left({\lambda }_c{r}_c\right)\right]$; the radius $r_c=z_o/2$ was equal to the half of the distance $z_o=$ 0.88 m between the object and the photo‐sensor. The nine phase‐only SLMs in the display set‐up were characterized by $N_{xd}\times N_{yd}=$ 1920 x 1080 pixels, a pixel period ${\Delta }_d=$8 μm and phase modulation from 0 to 2$\pi$; the illuminating wavelength was ${\lambda }_d=$ 0.532 μm. The SLMs, arranged in a circular configuration, were illuminated with a single astigmatic expanding wave by means of a cone mirror whose apex was at a distance $D_s$ from the point light source positioned on the line of the cone mirror axis [25]:

$W\left(x_d,y_d\right)=\exp \left(-j\frac{2\pi }{{\lambda }_d}\frac{x_d^2}{D_h}\right)\exp \left[-j\frac{\pi }{{\lambda }_d}\frac{{\left(y+h_{SLM}/2\right)}^2}{D_h+D_s}\right]$E3

where $D_h$ is the distance from the cone mirror axis to the SLM centres and $h_{SLM}$ is the SLM height. The reconstructed images were combined above the cone mirror by a slight tilt of the SLMs at a distance of 35 cm from each SLM. A linear stretching of images with a coefficient, $m={\mathrm{\Delta }}_d/{\mathrm{\Delta }}_c$, occurs. A reference wave at a different wavelength, ${\lambda }_d$, and a different radius, $r_{rec}$, yields a new reconstruction distance $z_i$ [30]:

$\frac{1}{z_i}=\frac{1}{r_{rec}}\pm \frac{\mu }{m^2}\left(\frac{1}{z_o}-\frac{1}{r_c}\right)$E4

and the reconstructed image undergoes longitudinal and lateral magnifications:

$M_{long}=\frac{d{z}_i}{d{z}_o}={\left(\frac{z_i}{z_o}\right)}^2\frac{\mu }{m^2},M_{lat}=\frac{\mu }{m}\frac{z_i}{z_o}$E5

where $\mu ={\lambda }_c/{\lambda }_d$. Generation of 3D contents for each SLM was based on Eq. (1) and included: (i) retrieval of the phase ${\varphi }_O\left(x,y\right)$ of the object field from the captured holograms; (ii) compensation for the non‐plane wave illumination of the SLMs; (iii) adjustment of the reconstruction position to the mandatory distance of 35 cm. The phase ${\varphi }_O\left(x,y\right)$ was retrieved by filtering in the spatial frequency domain to extract only the real image in Eq. (1) and to suppress the zero‐order term and by multiplying the filter output with the numerical reference wave ${\mathit{R}}_c^{*}\left(x_d,y_d,r_{rec}\right)$ taken with a new radius, $r_{rec}=z_i/2=z_o{m}^2/\mu$ in the display coordinates $x_d=l{\Delta }_d,y_d=n{\Delta }_d;l=1,\mathrm{2..}{N}_{xd},n=1,\mathrm{2..}{N}_{yd}$. The amplitude was discarded in the object field and it became $H\left(x_d,y_d\right)=\exp \left[j{\varphi }_O\left(x_d,y_d\right)\right]$. To compensate for the non‐symmetrical illumination, the phase of $H\left(x_d,y_d\right)W^{*}\left(x_d,y_d\right)$ was fed to each SLM. The holograms were placed at the centres of the SLMs as depicted in Figure 4. Shining $W\left(x_d,y_d\right)$ on the SLMs with $HW$ represented as a phase creates reconstruction at the plane wave illumination when $r_{rec}\to \infty$ in Eq. (4). The reconstruction distance in this case is $z_i=$ 1.78 m. The reconstruction is stretched longitudinally and squeezed laterally at $M_{\text{long}}=$ 2.04 and $M_{\text{lat}}=$0.32. A digital converging lens, $L_1\left(x_d,y_d\right)=\exp \left\{j\left(2\pi /{\lambda }_d\right)\left(x_d^2+y_d^2\right)/{\rho }_1\right\}$ with a focal distance of ${\rho }_1=$ 43.5 cm was introduced to adjust the reconstruction distance to 35 cm. The image was separated from the strong non‐diffracted beam caused by the pixelated nature of the SLMs by multiplying the array with the holographic data with a tilted plane wave, $P\left(y_d\right)=\exp \left(j2\pi y_d\sin {\theta }_t/{\lambda }_d\right)$, where ${\theta }_t=2^{\circ }$. The phase of $W^{*}\left(x_d,y_d\right)$ was attached to the pixels outside the hologram plus the phase of the lens $L_2\left(x_d,y_d\right)=\exp \left\{j\left(2\pi /{\lambda }_d\right)\left(x_d^2+y_d^2\right)/{\rho }_2\right\}$ with ${\rho }_2=$ 35 cm to gather the light reflected from these pixels below the reconstructed image. The arrangement of the wave fields on the surface of each SLM is depicted in Figure 4 (actually, the phases of these fields were fed to each SLM). The processing allowed for combining the images created by all SLMs into a single reconstruction which could be viewed smoothly within an increased viewing angle of 24°. The video with the reconstruction can be found in reference [29]. The most remarkable fact of this data mapping from far infrared capture to a circular display was that we achieved more or less same longitudinal and lateral magnifications $M_{\text{long}}=$ 0.078 and $M_{\text{lat}}=$0.062 of the reconstruction volume.

Figure 4.

Schematic diagram of the chain holographic capture—data transfer—holographic display for far‐infrared capture of a large object [29] and visible light visualization [25].

The second example of data mapping is related to visualization of transparent objects by a holographic display with phase encoding of the input data. The object beam $O\left(x,y\right)$ in this case was provided by simulation of a noiseless diffraction tomography experiment in which transmission holograms of weakly refracting transparent object with a size of 25 μm were recorded by a phase‐shifting technique [31]. The object had refractive index variation from 1 to 1.004 but due to its small size it gave rise to a strong diffraction. The capture parameters were: ${\lambda }_c=$ 0.68 μm, ${\Delta }_c=$ 2.4043 μm, $N_c=200\times 200$, $z_o=$ 68 μm [31]; the display parameters were as above. Direct optical reconstruction from the captured phase‐only data failed. Usage of the full complex amplitude $O\left(x,y\right)$ provided numerical reconstruction as a concise 3D shape resembling closely the 3D refractive index distribution within the object (Figure 5). Omission of the amplitude, $a_O\left(x,y\right)$, destroyed entirely this 3D shape. The observed severe distortions showed the necessity for phase modification in the hologram plane. This was done iteratively by applying the Gerchberg‐Saxton algorithm at known correct complex amplitude at the reconstruction plane. Quality of the numerical reconstruction from the modified phase was satisfactory. Thus the problem with optical reconstruction was solved. Introduction of a digital magnifying lens at the SLM plane enlarged the reconstructed object to 6 mm [31]. This gives about 240 times magnification in comparison with its original size.

Figure 5.

Numerical reconstructions of a virtual transparent object given in the left section of the figure as a 3D distribution of the refractive index n_O (green colour, n_O =1.001; red, n_O = 1.002; yellow, n_O =1.003; blue, n_O = 1.004 [31]).

3. Computer generation of 3D contents for holographic imaging

3.1. Methods for computer generation of holographic fringe patterns

A CGH is a fringe pattern that diffracts light into a wavefront with desired amplitude and phase distributions and seems to be the most appropriate choice for 3D content generation. This wavefront can be created both for real and virtual objects. The goal in developing the CGHs input data for a 3D holographic display is to have real‐time generation of large‐scale wide‐viewing angle full‐parallax colour holograms which provide photorealistic reconstruction that can be viewed with both eyes. These CGHs must support a motion parallax and the coupled occlusion effect expressed in the visible surface change according to the viewer position. For the purpose, a CGH must have a very large number of samples displayed on a device with high spatial resolution. So the most important requirements to CGH synthesis are computational efficiency and holographic image quality. The CGH computation involves digital representation of the 3D object which includes not only its geometrical shape but also texture and lighting conditions, simulation of light propagation from the object to the CGH plane, and encoding of the fringe pattern formed by the interference of the object light wavefront with a reference beam in the display data format.

There are two basic frameworks for CGH generation depending on the mathematical models of 3D target objects: (i) point cloud algorithms and (ii) polygon‐based algorithms. In the point‐cloud method [32], the 3D object is a collection of P self‐luminous point sources. The method traces the ray from a source ‘p' with spatial coordinates $\left(x_p,y_p,z_p\right)$ to the point $\left(\xi ,\eta \right)$ on the hologram plane at $z=0$ and is sometimes referred to as a ray‐tracing approach; the distance between both points is $r_p={\left[{\left(\xi -x_p\right)}^2+{\left(\eta -y_p\right)}^2+z_p^2\right]}^{1/2}$ (Figure 6). Each point source emits a spherical wave with an amplitude, $a_p$, and an initial phase, ${\phi }_p$. The amplitude and phase distributions in the point cloud can be controlled individually. The fringe patterns for all object points are added up at the hologram plane to obtain the CGH. The method is highly flexible due to ability to represent surfaces with arbitrary shapes and textures, but it is very time consuming.

Figure 6.

CGH synthesis from a point cloud model.

Polygon‐based representation is a wave‐oriented approach [33–35]. The object is a collection of P planar segments of a polygonal shape (Figure 7). Each polygon is a tilted surface source of a light field calculated by propagation of its angular spectrum of plane wave decomposition [27] using a fast Fourier transform (FFT). An angular‐dependent rotational transformation in the spectral domain is applied to find the spectrum in a plane parallel to the hologram in the global coordinate system from the spectrum in the tilted plane of the local coordinate system $(x_p,y_p,z_p),p=1,\mathrm{2...}P$ for each polygon [36, 37]. The z‐axis in the local coordinate system is along the vector to the polygon surface. The object field is found after FFT of the final angular spectrum which is a sum of the transformed angular spectra of the polygon fields in the global coordinate system. Computation of a polygon field is slower than that of a spherical wave emitted by a point light source, but the number of polygons is much smaller than that of point sources and the total computation time is shorter compared to the point cloud approach. The traditional polygon‐based method evolved to analytical implementation when the angular spectrum of a triangle of arbitrary size, shape, orientation and location in space is analytically calculated from the known spectrum of a reference triangle [38–40]. The analytical method eliminates the need to apply FFT for each polygon.

Figure 7.

CGH synthesis from a polygon‐based model.

The CGH synthesis of real objects can be carried out by 3D capture based on holographic means or structured light methods under coherent or incoherent illumination [41]. The output from, e.g. profilometric/tomographic reconstruction can be converted into a point cloud which allows for CGHs synthesis. The substantial advantage is the option to adapt the captured data to any holographic display. Incoherent capture of multiple projection images to generate holographic data has a lot of advantages such as incoherent illumination, no need of interferometric equipment and display of large objects. The concept was advanced 40 years ago in [42] by generating a holographic stereogram (HS). High quality of large format HSs as a ray‐based display, especially those printed by HS printers, are well known [43]. The input data for HS imaging are composed from colour and directional information. This causes a decrease of resolution for deep scenes and blurring. Introduction of a ray‐sampling plane close to the object and computation of the light wavefront from this plane to the hologram is proposed in Ref. [44]. Synthesis of a full‐parallax colour CGH from multiple 2D projection images captured by a camera scanned along a 2D grid is proposed in Refs. [45, 46]. The approach given in Ref. [46] relies on calculation of the 3D Fourier spectrum and was further improved [47] by developing a parabolic sampling of the spectrum for data extraction and needing only 1D camera scanning. Methods in which directional information from projection images is combined with a depth map are under development [48–50].

Over the last decade, the efforts were focused on improving image quality by different rendering techniques and on accelerating the CGH computation. The holographic data—amplitude and phase—should encode occlusions, materials and roughness of the object surface, reflections, surface glossiness and transparency. It is difficult to create an occlusion effect using 3D object representation as a collection of primitives—points or polygons—due to the independent contribution of all primitives to the light field. To decrease the computational cost of occlusion synthesis, a silhouette mask approach has been proposed in the polygon‐based computation [35]. The mask produced by the orthographic projection of the foreground objects blocks the wavefront of light coming from background objects. The method allows for synthesis of a very large CGH [35] but is prone to errors at oblique incidence. Computation is accelerated if occlusion is included in a light‐ray rendering process from multiple 2D projection images during the synthesis of a CGH as an HS [51]. As the method suffers from decrease of angular resolution in deep scenes, accuracy is improved by processing occlusion in the light‐ray domain along with sampling the angular information from the projection images. This is done in a virtual ray‐sampling plane [52, 53]. Furthermore, the sampled data are converted by Fourier transform to the object beam complex amplitude. Considering occlusion as geometric shadowing [54], effective CGH synthesis can be carried out by casting from each sample at the hologram plane, a bundle of rays at uniform angular separation within the diffraction angle given in Eq. (2). Such approaches are described in Refs. [54–56] with representing the 3D objects as composed from planar segments parallel to the hologram plane [55] or performing a lateral shear of the 3D scene to use the z‐buffer of the graphic processing unit (GPU) for the rays with the same direction to accelerate computation [54]. Occlusion, texture and illumination issues can be handled by computer graphics techniques. Their effective use is possible when the ray casting is applied by spatially dividing the CGH into a set of sub‐holograms and building different sets of points or polygons for them [57–59].

At specular reflection, the viewer is able to see only part of the object while the diffuse reflection sends light rays in all directions. Both types of reflections must be encoded in a CGH by adopting different reflection models to represent texture of the objects [60–62]. In the CGH synthesis, the luminance is encoded in the amplitude while reflectance is incorporated as a phase term. The task of representing reflection becomes rather complicated at non‐plane wave illumination or in the case of background illumination [60]. A perfect diffuse reflection is achieved by adding a uniformly distributed random phase. Unfortunately, this causes speckle noise at reconstruction [63]. A variety of methods have been proposed for fast synthesis of CGHs as look‐up table methods with pre‐computed fringes [63, 64], recurrence relation methods instead of directly calculating the optical path [65], introduction of wavefront recording plane [66], HS methods and many others. Hardware solutions as special‐purpose computers like ‘Holographic ReconstructioN (HORN)’ [67] or GPU computing [32, 68–70] are very effective for fast calculation because the pixels on a CGH can be calculated independently.

3.2. Phase‐added holographic stereogram as a fast computation approach

Effective acceleration of computation is achieved in coherent stereogram (CS) algorithms when the CGH is partitioned into segments and the directional data for each segment are sampled (Figure 8). A similar idea has been advanced in the diffraction‐specific fringe computation by Lucente [71] with partitioning the hologram into holographic elements called hogels and each hogel having a linear superposition of weighted basic fringes corresponding to the points in a point cloud. Each segment in the CS emits a bundle of light rays that form a wavefront as a set of patches of mismatched plane waves due to lack of depth information. This drawback was overcome by adding a distance‐related phase [72]. The phase‐added stereogram (PAS) is computationally effective if implemented by FFT. To clarify this point, we depict schematically the PAS computation with FFT in Figure 9.

Figure 8.

Synthesis of a CGH as a coherent stereogram with partitioning the hologram plane into square segments and sampling the directional information.

Figure 9.

Schematic representation of the synthesis of a CGH within a segment.

In CS and PAS algorithms, the hologram is partitioned into $M\times N$ equal square segments with $S\times S$ pixels. The object is described by a point cloud with P points. The segment size, ${\Delta }_{d}S\times {\Delta }_{d}S$, where ${\Delta }_d$ is the pixel period at the hologram plane, is chosen small enough to approximate the spherical wave from a point as a plane wave given by a 2D complex harmonic function within the segment. This approximation means that the contribution from a point source is constant across the segment and is determined with respect only to its central pixel. In this way, the input data and computation time are substantially reduced; for the segment $(m,n),m=\mathrm{1..}M,n=\mathrm{1..}N$ the contribution from the point ‘p' comprises spatial frequencies $(u_{mn}^p,v_{mn}^p)$ of the plane wave at a wavelength ${\lambda }_d$, the distance between the point ‘p' and the central point, $r_{mn}^p$, the initial phase of the sinusoid, ${\Phi }_{mn}^p$. The spatial frequencies are determined by the illuminating angles, $\left({\Theta }_{mn}^p,{\Omega }_{mn}^p\right)$, of the ray coming from the point ‘p' to the central point of the segment $\left(m,n\right)$ and angles ${\theta }_{R\xi }$ and ${\theta }_{R\eta }$ of the plane reference wave with respect to $\xi$ and $\eta$ axes at the hologram plane as follows: $u_{mn}^p=\left(\sin {\Theta }_{mn}^p-\sin {\theta }_{R\xi }\right)/{\lambda }_d$, $v_{mn}^p=\left(\sin {\Omega }_{mn}^p-\sin {\theta }_{R\eta }\right)/{\lambda }_d$. The phases ${\Phi }_{mn}^p,m=\mathrm{1..}M,n=\mathrm{1..}N$ ensure matching of the wavefronts of the plane waves diffracted from all segments and may contain the initial phase ${\phi }_p$ and also the distance‐related phase $2\pi r_{mn}^p/{\lambda }_d$. For all object points, the fringe pattern across the segment is approximated as a superposition of 2D complex sinusoids. Computation of this pattern is carried out by placing the amplitudes of the sinusoids to the corresponding frequency locations in the spatial frequency domain and by applying an inverse Fourier transform to the spectrum. FFT implementation is the second step for acceleration of CGH computation (Figure 9). The FFT step moves the spatial frequencies to the nearest allowed values in the discrete frequency domain. The complex amplitudes remain the same. The two‐step procedure is repeated for each segment to compute the CGH.

The PAS approximation should yield a wavefront close to the wavefront provided by the Rayleigh‐Sommerfeld diffraction model that treats the propagating light from a point as a spherical wave. The complex amplitude in the reference model is given by:

$O_O^{RS}(\xi ,\eta )={\displaystyle \sum _{p=1}^P\frac{A_p}{r_p}}\exp \left(j\frac{2\pi }{{\lambda }_d}{r}_p\right),A_p=a_p\exp (j{\phi }_p)$E6

We applied PAS computation to generate digital input contents for a wavefront printer developed by us for printing a white‐light viewable full‐parallax reflection hologram [73, 74]. The printed hologram was recorded as a 2D array of elemental holograms. The CGH for each elemental hologram was fed to amplitude SLM with 1920 × 1080 pixels. The object beam encoded in the CGH was extracted by spatial filtering and demagnified using a telecentric lens system. Unlike the HS printers [43], the wavefront printer uses full holographic data. That is why the synthesis of a large number of elemental holograms, e.g. 100 × 100, takes a very long time. This requires a fast computation method that provides quality of imaging close to the reference model. We solved this task by developing a fast PAS (FPAS) method [75] as a further elaboration of the already existing PAS methods. Usage of the FFT is crucial for fast PAS implementation, but this may affect negatively the quality of imaging due to spatial frequencies mapping to a predetermined coarse set of discrete values. The sampling step, ${1/S\Delta }_d$, in the frequency domain could not be made small due to necessity to approximate the reference model. Thus, the fringe pattern generated by the PAS with FFT inaccurately steers the diffracted light. The improvements developed to compensate the error caused by the frequency mapping are based on the two possible ways for steering control—phase compensation and finer sampling of the spectrum attached to each segment. The functional form of the developed approximations is shown in Table 1 which gives the fringe pattern at a single spatial frequency in the segment $\left(m,n\right)$; $\left({\xi }_{mn}^c,{\eta }_{mn}^c\right)$ is the central point of the segment and the following notation is introduced for the complex sinusoid:

$F\left(u_{mn}^p,v_{mn}^p\right)=\left(A_p/r_{mn}^p\right)\exp \left\{j2\pi \left[u_{mn}^p\left(\xi -{\xi }_{mn}^c\right)+v_{mn}^p\left(\eta -{\eta }_{mn}^c\right)\right]\right\}$E7

Fringe pattern of the method
CS: $F(u_{mn}^p,v_{mn}^p)$
PAS (no FFT): $F\left(u_{mn}^p,v_{mn}^p\right)\exp \left(jk{r}_{mn}^p\right)$
PAS (FFT): $F\left({\widehat{u}}_{mn}^p,{\widehat{v}}_{mn}^p\right)\exp \left(jk{r}_{mn}^p\right)$
CPAS:$F\left({\widehat{u}}_{mn}^p,{\widehat{v}}_{mn}^p\right)\exp \left(jk{r}_{mn}^p\right)\times \exp \{j2\pi [\left({\widehat{u}}_{mn}^p-u_{mn}^p\right)\left({\xi }_{mn}^c-x_p\right)+\left({\widehat{v}}_{mn}^p-v_{mn}^p\right)\left({\eta }_{mn}^c-y_p\right)]\}$
APAS: $F\left({{\widehat{u}}^{\prime }}_{mn}^p,{{\widehat{v}}^{\prime }}_{mn}^p\right)\exp \left(jk{r}_{mn}^p\right)$
ACPAS: $F\left({{\widehat{u}}^{\prime }}_{mn}^p,{{\widehat{v}}^{\prime }}_{mn}^p\right)\exp \left(jk{r}_{mn}^p\right)\times \exp \{j2\pi [\left({{\widehat{u}}^{\prime }}_{mn}^p-u_{mn}^p\right)\left({\xi }_{mn}^c-x_p\right)+\left({{\widehat{v}}^{\prime }}_{mn}^p-v_{mn}^p\right)\left({\eta }_{mn}^c-y_p\right)]\}$
FPAS: $F\left({{\widehat{u}}^{\prime }}_{mn}^p,{{\widehat{v}}^{\prime }}_{mn}^p\right)\exp \left(jk{r}_{mn}^p\right)\times \exp \left\{j2\pi \left[u_{mn}^p\left({\xi }_{mn}^c-x_p\right)+v_{mn}^p\left({\eta }_{mn}^c-y_p\right)\right]\right\}$
Spatial frequencies: ${\widehat{u}}_{mn}^p=\frac{l_u}{S{\Delta }_d},{\widehat{v}}_{mn}^p=\frac{l_{\upsilon }}{S{\Delta }_d};-\frac{S}{2}\le l_u,l_{\upsilon }\le \frac{S}{2}$; ${{\widehat{u}}^{\prime }}_{mn}^p=\frac{l_u}{S^{\prime }{\Delta }_d},{{\widehat{v}}^{\prime }}_{mn}^p=\frac{l_v}{S^{\prime }{\Delta }_d};-\frac{S^{\prime }}{2}\le l_u,l_{\upsilon }\le \frac{S^{\prime }}{2},S^{\prime }<S$

Table 1.

Single frequency fringe pattern in the segment.

The first improvement CPAS (compensated PAS) [76] performed some steering correction by adding the phase, which includes the difference between the spatial frequencies in the continuous and the discrete domains. The CPAS provided a better reconstructed image than the PAS with FFT at almost the same calculation time. Finer sampling was proposed in the accurate PAS (APAS) [77] by computing the FFT in an area which exceeds the segment and by properly truncating the larger‐size IFFT output. Phase compensation and directional error reduction by finer sampling were merged into a single step in the algorithm ACPAS [57] which yielded quality of reconstruction very close to the reference model. The best results are provided by the FPAS algorithm which is characterized with better phase compensation than the previous methods. This was confirmed by quality assessment with conventional image‐based objective metrics as intensity distribution and peak signal‐to‐noise ratio [75] for reconstruction of a single point and also by good quality of reconstruction from white‐light viewable colour holograms (Figure 10) printed by our wavefront printing system [73] on an extra‐fine grain silver‐halide emulsion Ultimate08 [78]. The CGH computed by the FPAS algorithm for each elemental hologram was displayed on an amplitude type SLM. The demagnified pixel interval was 0.42 μm at the plane of the hologram and gives a diffraction angle of ± 39.3°. For uniform illumination of the CGH on the SLM without decreasing too much the laser beam intensity we used only 852 × 852 pixels in the SLM to project CGHs. Thus, the size of the elemental hologram became equal to 0.38 mm by 0.38 mm. The printed holograms are shown in Figure 10; their size is 5 cm × 5 cm and 9 cm × 9 cm. The smaller hologram consisted of 131 × 131 elemental holograms. The segment size for calculating the CGH fed to a given elemental hologram was 32 × 32 pixels while the FFT computation area was 128 × 128 pixels, and each elemental hologram comprised more than 700 segments.

Figure 10.

Photographs of reconstruction from printed holograms: (a)–(c): different views of a church model; (d): 9 cm × 9 cm printed hologram of a bunch of flowers.

4. Conclusion

Holographic imaging is a 3D imaging with all depth cues and inherent vision comfort for the viewer. That is why the last decade was marked by rapid development of methods of 3D capture and 3D content generation for holographic display, holographic projection and holographic printing. In the chapter, we considered implementation of the holographic imaging by digital means when the input data are in the form of a 2D real‐valued matrix, which should encode the light wavefront coming from the 3D scene. This wavefront can be extracted from optically recorded holograms or synthesized numerically using various 3D scene descriptions. Holographic recording by digital photo‐sensors or computer generation of holograms for pixelated SLMs imposes severe limitations on the space‐bandwidth product of the capture/display system. We discussed two cases of data mapping from holographic capture to holographic display to show that holographic data transfer from optically recorded digital holograms to the data format of a given display is not a trivial task due to inevitable distortions introduced as a result of different capture and display parameters. Representing 3D contents as computer‐generated holograms seems more flexible and promising way to create input data for holographic displays. The main requirements are to improve the quality of imaging and computational efficiency. We presented an algorithm for fast computation of holograms and showed the good quality of imaging it provided in holographic printing of white‐light viewable reflection holograms.

Acknowledgments

This work was supported by the Ministry of Science, ICT and Future Planning (Cross-Ministry Giga KOREA Project) and Bulgarian National Science Fund, project H 08/12 “Holographic imaging, beam shaping and speckle metrology with computer generated holograms”.