Analytical Method for Reflection and Refraction

Abstract

In computer graphics, ray tracing is very simple and powerful method to present physical phenomena especially light-related things such as reflection and refraction since it traces the ray from the eye to the light source; however, we cannot understand how the result image is generated. Then, this chapter describes the mechanism of reflection and refraction. It is very time-consuming to render the target object considering reflection and refraction. If the object distorted by reflection and refraction is previously obtained, it is very fast to generate the result image since all we have to do is to render the distorted object without considering reflection and refraction. In the proposed method, firstly, a virtual object, which is constructed with vertices translated from original ones by considering reflection and refraction, is generated. Then, the image with reflection and refraction is generated by rendering the virtual object. In the analysis, total reflection and attenuation of light power are also considered. At last, the proposed method is applied to two types of transparent objects: cubed glass and cylindrical glass, and the comparison between the simulation results and the real photos is performed to demonstrate that the generated images are the same as the real ones.

1. Introduction

Computer graphics is a very powerful tool to visualize natural phenomena such as thunder with lighting, avalanche in blizzard, reflection on a glass window, and so on. Among them, one of the most challenging issues is to visualize optical-related phenomena such as reflection and refraction. Although ray tracing is the most powerful method, it takes huge amount of time to trace the ray that has emitted from a light source and reaches the eye through each pixel of an image. The method searches the trace by dividing it to two phenomena: reflection and refraction. It generates the image with a recursive process so that the image can be generated with a simple algorithm. However, the image generated on a glass by reflection and refraction is so complicated that we cannot understand which part of the image is generated by reflection and/or refraction.

Then, this chapter describes an analytical method that draws an image generated on a geometrically defined object such as parallel glass, cubed glass, or cylindrical glass by reflection and/or refraction. The method analyzes the trace of the ray in a transparent object and generates a virtual object, which is constructed with vertices translated from original ones by considering reflection and refraction. In addition, the mechanism of total reflection and the attenuation of light power are also investigated. Thereafter, each image with reflection and/or refraction is generated by rendering the virtual object without considering reflection and refraction. Then, the final image is generated by combining every image drawn separately with the virtual objects.

Moreover, two types of applications with the proposed method are presented. One is to generate the image with reflection and refraction on a cubed glass, where there are six facets that have different number of reflection and refraction, and also total reflection happens on some facets depending on the eye position. The other is to represent a refractive image on a cylindrical glass, where four refractions happen when the light passes in the interior of the cylindrical glass.

At last, in order to demonstrate that the images generated with the proposed method are realistic, the simulation images are compared with real photos. One is the image that has reflection and refraction on a cubed glass, and the other is one that has refraction on a cylindrical glass.

2. Related works

In generation of realistic images with computer graphics, Whitted invented ray tracing [1, 2], which is the most simple and powerful method; however, it requires a lot of time to trace the ray for every pixel of the image. Then, Heckbert et al. proposed beam tracing [3] and Amanatides suggested cone tracing [4]. These methods are faster than ray tracing because they trace the ray not for each pixel but for the bundle of it by utilizing spatial coherence. On the other hand, Blinn et al. had invented texture mapping [5] to generate realistic images with a simple and fast method. Then, Pauline et al. used texture mapping to represent waves [6], and Hakura et al. proposed parameterized environment mapping [7], which is an extended method of texture mapping and reproduces local reflection from pre-rendered viewpoints. Texture mapping can create realistic images in real time since some pre-generated texture images are used in the rendering; however, multiple images must be prepared especially when the viewpoint changes. Then, G’enevaux et al. suggested another method to obtain realistic and interactive effect of refraction [8].

On the other hand, Ofek et al. had proposed a method to generate reflective images by using virtual objects that are previously transformed from the original ones [9]. Conversely, Iwasaki et al. used texture mapping to render caustics on water surfaces [10]. Wang et al. also utilized environment mapping and interpolated the map in real time by using depth buffer [11].

Nowadays, graphics processing unit (GPU) is utilized for general purpose calculation that is used to be performed by central processing unit (CPU). Then, Wyman used GPU and performed an interactive image-space approach to represent refractive images on transparent objects [12]. Hu et al. also utilized GPU power and proposed an interactive computing scheme for rendering of reflection, refraction, and caustics [13]. In addition, Oliveira et al. utilized programmable GPU for image-space technique that calculates the intersection between the ray and objects efficiently [14]. Furthermore, Rodgman et al. proposed another rendering method for refraction in volume graphics [15], and Rousiers et al. also proposed another method for representing refraction through a transparent object that has rough surfaces [16]. On the contrary, Chen et al. suggested a depth acquisition method from refractive images, which method can measure the depth from the eye if the cause of refraction is already known [17]. If we use this method, we can estimate the original shape of objects deformed by refraction.

Generating realistic images with reflection and refraction requires huge amount of time so that most of research use texture mapping with the help of GPU power. Although these techniques produce realistic images with reflection and refraction very fast, we do not understand which part of the image is generated by reflection and/or refraction. On the contrary, the virtual object that Ofek et al. [9] proposed has vertices translated from the original ones by considering reflection. Then, if we can understand how the original object is transformed by reflection and refraction, the image can be generated very fast. Therefore, Mukai et al. investigated how the vertices of an object are translated from the original ones by refraction in a cylindrical glass [18] and also represented the complex image on a cubed glass by combining each image that has simple reflection and/or refraction [19]. Therefore, this chapter describes how an object is transformed by reflection and refraction and how the vertices constructing the object are translated from the original positions. First, the analysis of reflection and refraction is described, and a virtual object, which is transformed from the original one by reflection and refraction, is generated. Then, the image with reflection and refraction is generated by combining each image rendered with the virtual object. At last, the proposed method is used for two types of applications: the image with reflection and refraction generated on a cubed glass and the image with refraction generated on a cylindrical glass. In addition, the result shows that the simulation images are the same as the real ones by comparison between the generated images and the real photos.

3. Virtual object

3.1 Reflection

This section describes how to generate a virtual object, which has vertices translated from original ones by considering reflection. Figure 1 illustrates the reflection on a parallel glass. In the figure, $E\left(0,0\right)$, $\mathrm{\theta }$, and $\mathrm{M}\left(x_m,y_m\right)$ are the eye position that is the origin of the Cartesian coordinate system, incident angle of light on the front plane of the parallel glass, and one vertex constructing the object shown in red, respectively. The dotted red line shows the trace of the ray that has emitted from a vertex $\mathrm{M}\left(x_m,y_m\right)$ and enters the eye position $E\left(0,0\right)$. The front plane of the parallel glass is placed on y = p. If $\mathrm{V}\left(x_v,y_v\right)$ is obtained, we can generate the image by just rendering the virtual object including $\mathrm{V}\left(x_v,y_v\right)$ instead of the original one including $\mathrm{M}\left(x_m,y_m\right)$. The virtual vertex $\mathrm{V}\left(x_v,y_v\right)$ translated from the original vertex $\mathrm{M}\left(x_m,y_m\right)$ is calculated as follows:

$$\mathrm{V}\left(x_v,y_v\right)=\mathrm{V}\left(x_m,y_m+2T\right)=\mathrm{V}\left(x_m,y_m+2p-y_m\right)\left(\because y_m\le p\right)$$E1

Figure 1.

Virtual vertex due to reflection on a parallel glass.

3.2 Refraction

Figure 2 shows the refraction through a parallel glass, which has the thickness of H. In the figure, $n_1$ and $n_2$ are the refractive indices of the air and the glass, respectively. The incidence angle of the ray that outputs from the eye is $\mathrm{\theta }$, and the refraction angle is $\mathrm{\varphi }$. If $\mathrm{V}\left(x_v,y_v\right)$ is obtained, we can generate the image by just rendering the virtual object with $\mathrm{V}\left(x_v,y_v\right)$ instead of the original one with $\mathrm{M}\left(x_m,y_m\right)$. Then, the virtual vertex $\mathrm{V}\left(x_v,y_v\right)$ is calculated as the following:

$$L_y=\text{Htan}\left(\varphi \right)\mathit{tan}\left\{\left(\pi /2\right)-\theta \right\}$$E2

$$\therefore L_y=\mathrm{H}\left\{1-\mathit{tan}\left(\varphi \right)\mathit{tan}\left[\left(\pi /2\right)-\theta \right]\right\}=\mathrm{H}\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\}$$E3

Figure 2.

Virtual vertex due to refraction through a parallel glass.

However, $\mathrm{\theta }$ is the incidence angle for the vertex $\mathrm{M}\left(x_m,y_m\right)$ constructing the original object, and the angle $\mathrm{\theta }$ depends on the positions of $\mathrm{E}\left(0,0\right)$ and $\mathrm{M}\left(x_m,y_m\right)$. On the other hand, we can obtain the following equations from △ EOM in Figure 2 and Snell’s law:

$$\mathit{tan}\left(\theta \right)=x_m/\left(y_m-L_y\right)$$E4

$$n_{1}sin\left(\theta \right)=n_{2}sin\left(\varphi \right)$$E5

By substituting Eqs. (4) and (5) to Eq. (3), the following equation is obtained:

$$x_m=\left(y_m-H\right)\mathit{tan}\left(\theta \right)+\text{Htan}\left\{{\mathit{sin}}^{-1}\left[\left(n_1/n_2\right)\mathit{sin}\left(\theta \right)\right]\right\}$$E6

Finally, by solving Eq. (6) with an iterative method, the incidence angle $\mathrm{\theta }$ is calculated. With the calculated incident angle $\mathrm{\theta }$, $\mathrm{\varphi }$ is also calculated with Eq. (5). Then, the vertex of the virtual object is calculated as follows:

$$\mathrm{V}\left(x_v,y_v\right)=\mathrm{V}\left(x_m,y_m-L\right)$$E7

3.3 Inclination of a plane

The above description is true for the case that the reflective or the refractive plane is parallel to X axis of the coordinate system. However, the rotation of the coordinate system is required for the case that the plane is inclined against X axis. Figure 3 shows the process of the vertex translation with the coordinate system rotation. In Figure 3(a), the plane is inclined against X axis by the angle $\mathrm{\omega }$. By the rotation, the plane becomes parallel to X axis, and the translation of the vertex A in Figure 3(b) is performed with the above process (the translated vertex is indicated as B in the figure). Then, the final vertex is obtained by the inverse rotation of the coordinate system.

Figure 3.

Vertex translation with the coordinate system rotation. (a) Rotation (b) Movement and (c) Inverse rotation.

3.4 Combination of reflection and refraction

Figure 4 shows the case where both reflection and refraction happen on a parallel glass. $\mathrm{M}\left(x_m,y_m\right)$ is a vertex that constructs an original object, and D is the thickness of the parallel glass. The ray that has emitted from $\mathrm{M}\left(x_m,y_m\right)$ is refracted on the front plane of the glass and also reflected on the back plane. Then, the ray reaches the eye after it is refracted again on the front plane. If we can calculate the temporary virtual vertex for the reflection on the back plane, we can consider this phenomenon, which has two refractions and one reflection, as another phenomenon that has two refractions and no reflection with another parallel glass which thickness is 2D instead of D. In the figure, this temporary vertex is shown as $M_1\left(x_m,y_1\right)$. In addition, if we can calculate the virtual vertex, we can consider the phenomenon as another phenomenon with no reflection and no refraction. This virtual vertex is shown as $\mathrm{V}\left(x_v,y_v\right)$ in the figure. If we can obtain the virtual vertex $\mathrm{V}\left(x_v,y_v\right)$, we can draw the image without considering both reflection and refraction. The final virtual vertex $\mathrm{V}\left(x_v,y_v\right)$ is calculated as follows:

$$M_1\left(x_m,y_1\right)=M_1\left(x_m,y_m+2\left\{T+D\right\}\right)=M_1\left(x_m,y_m+2\left\{\left|y_m-p\right|+D\right\}\right)$$

$$=M_1\left(x_m,2p-y_m+2D\right)$$E8

$$2D-L_y=2\mathit{\text{Dtan}}\left(\varphi \right)\mathit{tan}\left\{\left(\pi /2\right)-\theta \right\}=2D\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)$$E9

$$\therefore L_y=2D\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\}$$E10

Figure 4.

Reflection and refraction on a parallel glass.

We can also obtain the following equation from Eq. (4) by replacing $y_m$ with $y_1$:

$$\mathit{tan}\left(\theta \right)=x_m/\left(y_1-L_y\right)$$E11

By substituting Eqs. (5) and (11) for Eq.(10), the following equation is obtained:

$$x_m=\left(y_1-2D\right)\mathit{tan}\left(\theta \right)+2\mathit{\text{Dtan}}\left\{{\mathit{sin}}^{-1}\left[\left(n_1/n_2\right)\mathit{sin}\left(\theta \right)\right]\right\}$$E12

From Eq. (8), $y_1=2p-y_m+2D$. Then, Eq. (12) can be solved with an iterative method because $\mathrm{\theta }$ varies depending on the positions of the eye and each object. If $\theta$ is decided, we can calculate the virtual vertex $V\left(x_v,y_v\right)$ as follows:

$$V\left(x_v,y_v\right)=V\left(x_m,2p-y_m+2D-L_y\right)=V\left(x_m,2p-y_m+2D\left\{\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\}\right)$$E13

The same calculation can be applied to the general phenomenon with $\mathrm{n}=2\mathrm{k}-1\left(\mathrm{k}\ge 1\right)$ repetitive reflections inside the parallel glass and two refractions on the front plane:

$$L_y=2\mathit{kD}\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\}$$E14

$$x_m=\left\{y_1-2\mathit{kD}\right\}\mathit{tan}\left(\theta \right)+2\mathit{\text{kDtan}}\left\{{\mathit{sin}}^{-1}\left[\left(n_1/n_2\right)\mathit{sin}\left(\theta \right)\right]\right\}$$E15

$$V\left(x_v,y_v\right)=V\left(x_m,2p-y_m+2\mathit{kD}\left\{\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\}\right)$$E16

Figure 5 shows the relation between the number of repetitive reflections inside the parallel glass and the thickness of the virtual glass. The original thickness of the glass is D. When there is $1=2\mathrm{k}-1\left(\mathrm{k}=1\right)$ reflection inside the glass, we can consider the phenomenon as another phenomenon that has no reflection inside the glass and two refractions on the side planes with the parallel glass, which thickness is $2\mathrm{D}=2\mathrm{kD}\left(\mathrm{k}=1\right)$. When the number of the reflection is $3=2\mathrm{k}-1\left(\mathrm{k}=2\right)$ or $5=2\mathrm{k}-1\left(\mathrm{k}=3\right)$, the thickness of the glass becomes $4\mathrm{D}=2\mathrm{kD}\left(\mathrm{k}=2\right)$4D or $6\mathrm{D}=2\mathrm{kD}\left(\mathrm{k}=3\right)$, respectively.

Figure 5.

Relation between the number of reflection and the thickness of the glass.

3.5 Total reflection

When light enters the side plane of the glass, the incident angle changes. If the incident angle on the back plane of the glass is over the critical angle, total reflection happens. That is, there is no refraction and all light energy is reflected on the back plane. Even in this case, we can consider the phenomenon, which has one total reflection on the back plane and two refractions on the both side planes, as another phenomenon that has only two refractions on the side planes and no reflection on the back plane. In addition, we can generate the image by rendering the virtual object that has virtual vertices. Figure 6 illustrates this situation. In the figure, it is supposed that the back plane of the parallel glass is placed on $\mathrm{y}=\mathrm{q}$, and the incident and refractive angles on the side planes are $\mathrm{\theta }$ and $\mathrm{\varphi }$, respectively. Then, the difference of the reflective angle $\mathrm{\psi }$ of the back plane and the critical angle ${\psi }_c$, which is defined as ${\mathit{sin}}^{-1}\left(n_1/n_2\right),$ can be calculated as follows by using Eq. (5):

$${\mathit{sin}}^2\left(\psi \right)-{\mathit{sin}}^2\left({\psi }_c\right)={\mathit{sin}}^2\left\{\left(\frac{\pi }{2}\right)-\varphi \right\}-{\left(\frac{n_1}{n_2}\right)}^2={\mathit{cos}}^2\left(\varphi \right)-{\left(\frac{n_1}{n_2}\right)}^2$$

$$=1-{\mathit{sin}}^2\left(\varphi \right)-{\left(\frac{n_1}{n_2}\right)}^2=1-{\left(\frac{n_1}{n_2}\right)}^2{\mathit{sin}}^2\left(\theta \right)-{\left(\frac{n_1}{n_2}\right)}^2=1-{\left(\frac{n_1}{n_2}\right)}^2\left\{1+{\mathit{sin}}^2\left(\theta \right)\right\}$$E17

Figure 6.

Total reflection.

Here, $n_1$ and $n_2$ are the refractive indices of the air and the glass, respectively. $n_1$ is 1.0 and $n_1$ is in [1.43, 2.14] if we use the normal glass. Then, Eq. (17) is greater than 0.02 and always plus number. That means total reflection always happens on the back plane when light enters the side of the glass. Even in this case, we can consider the phenomenon, which has one total reflection on the back plane and two refractions on the both side planes of the parallel glass, as another phenomenon that has only two refractions on the side planes and no total reflection by using a virtual vertex shown as $M_1\left(x_m,y_1\right)$ in the figure. It is also supposed that the position of the back plane is $\mathrm{y}=\mathrm{q}$.

From the figure, the following equations are obtained:

$$M_1\left(x_m,y_1\right)=M_1\left(x_m,2S+y_m\right)=M_1\left(x_m,2\left|y_m-q\right|+y_m\right)=M\left(x_m,2q-y_m\right)$$E18

$$\left(W-L_x\right)\mathit{tan}\left(\theta \right)=\mathit{\text{Wtan}}\left(\varphi \right)$$E19

$$\therefore L_x=W\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\}$$E20

$$\mathit{tan}\left(\theta \right)=y_1/\left(x_m-L_x\right)\left(\because x_m\ge L_x\right)$$E21

With Eqs. (5), (20), and (21), the following equation is derived:

$$2q-y_m=\left(x_m-W\right)\mathit{tan}\left(\theta \right)+\mathit{\text{Wtan}}\left\{{\mathit{sin}}^{-1}\left[\left(n_1/n_2\right)\mathit{sin}\left(\theta \right)\right]\right\}$$E22

Then, the incident angle $\theta$ is calculated with an iterative method, and $\varphi$ is also calculated with Eq. (5). Finally, the virtual vertex is obtained as the following:

$$V\left(x_v,y_v\right)=V\left(x_m-L_x,2q-y_m\right)=V\left(x_m-W\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\},2q-y_m\right)$$E23

Figure 7 shows the case that has two total reflections. In this case, we can consider the phenomenon, which has two total reflections on the front and back planes, and two refractions on the both sides, as another phenomenon that has one total reflection on the back plane and two refractions on the both sides by using a virtual vertex $M_1\left(x_m,y_1\right)$. In addition, we can consider the phenomenon as another phenomenon that has no total reflection and two refractions on the both sides by using another virtual vertex $M_2\left(x_m,y_2\right)$. Finally, we can consider the phenomenon as another one that has no total reflection and no refraction by using $\mathrm{V}\left(x_v,y_v\right)$.

Figure 7.

Two total reflections.

$M_2\left(x_m,y_2\right)$ and $\mathrm{V}\left(x_v,y_v\right)$ are calculated as follows by using Eq. (20):

$$M_2\left(x_m,y_2\right)=M_2\left(x_m,y_m+2D\right)$$E24

$$V\left(x_v,y_v\right)=V\left(x_m-L_x,y_m+2D\right)=V\left(x_m-W\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\},y_m+2D\right)$$E25

Figure 8 shows another case that has three total reflections. Even in this case, we can consider the phenomenon as another one that has no total reflection and two refractions on the both sides by using a virtual vertex $M_3\left(x_m,y_3\right)$. Then, we can generate the image without considering any reflection and refraction by using another virtual vertex $\mathrm{V}\left(x_v,y_v\right)$.

Figure 8.

Three total reflections.

$M_3\left(x_m,y_3\right)$ and $\mathrm{V}\left(x_v,y_v\right)$ are calculated as follows by using Eq. (20):

$$M_3\left(x_m,y_3\right)=M_3\left(x_m,y_m+2\left\{S+D\right\}\right)=M_3\left(x_m,y_m+2\left\{\left|y_m-q\right|+D\right\}\right)$$

$$=M_3\left(x_m,2q-y_m+2D\right)$$E26

$$V\left(x_v,y_v\right)=V\left(x_m-L_x,2q-y_m+2D\right)$$

$$=V\left(x_m-W\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\},2q-y_m+2D\right)$$E27

The same calculation can be applied to the case that has n total reflections inside the glass. In fact, $V\left(x_n,y_n\right)$ is classified to two types depending on the number of total reflection as the following:

$$V\left(x_v,y_v\right)=$$

$$\left\{\begin{array}{cc}V\left(x_m-W\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\},2q-y_m+\left(n-1\right)D\right)& \left(n=2k-1,k\ge 1\right)\\ V\left(x_m-W\left\{1-\mathit{tan}\left(\varphi \right)/\mathit{tan}\left(\theta \right)\right\},y_m+\mathit{nD}\right)& \left(n=2k,k\ge 1\right)\end{array}\right.$$E28

3.6 Attenuation

Figure 9 shows the case that has multiple reflections and refractions in the parallel glass, where light energy attenuates gradually every time reflection or refraction happens.

Figure 9.

Multiple reflections and refractions.

Fresnel equations mention that the reflection rate changes depending on the incident angle; however, the following approximated equation is usually used for the reflection rate calculation in computer graphics field:

$$F_r\left(\theta \right)=F_0+\left(1-F_0\right){\left(1-\mathit{cos}\left(\theta \right)\right)}^5$$E29

$$F_0=\left\{\left(n_1-n_2\right)/\left(n_1+n_2\right)\right\}$$E30

where $\mathrm{\theta }$ is the incident angle and $n_1$ and $n_2$ are the refractive indices of the air and the glass, respectively. In addition, $F_0$ is the reflection rate in the case that light enters the plane perpendicularly. If the total energy of light is 1, the summation of the first reflective light energy ($R_1$) and the first transmitted light energy ($T_1$) should be 1. Then, the reflective and the refractive energies after n reflections and refractions can be calculated as follows if there is no energy loss:

$$R_1=F_r\left(\theta \right)$$E31

$$T_1=1-R_1=1-F_r\left(\theta \right)$$E32

$$R_2=T_1{F}_r\left(\varphi \right)=\left\{1-F_r\left(\theta \right)\right\}{F}_r\left(\varphi \right)$$E33

$$T_2=T_1\left\{1-F_r\left(\varphi \right)\right\}=\left\{1-F_r\left(\theta \right)\right\}\left\{1-F_r\left(\varphi \right)\right\}$$E34

$$R_3=R_2{F}_r\left(\varphi \right)=\left\{1-F_r\left(\theta \right)\right\}{F}_r{\left(\varphi \right)}^2$$E35

$$T_3=R_2\left\{1-F_r\left(\varphi \right)\right\}=\left\{1-F_r\left(\theta \right)\right\}{F}_r\left(\varphi \right)\left\{1-F_r\left(\varphi \right)\right\}$$E36

$$R_n=\left\{1-F_r\left(\theta \right)\right\}{F}_r{\left(\varphi \right)}^{n-1}\left(n\ge 2\right)$$E37

$$T_n=\left\{1-F_r\left(\theta \right)\right\}{F}_r{\left(\varphi \right)}^{n-2}\left\{1-F_r\left(\varphi \right)\right\}\left(n\ge 2\right)$$E38

In the rendering, the reflection and the refraction rates are used as an alpha value for alpha blending, when multiple images are combined together.

4. Cubed glass

4.1 Analysis of reflection and refraction

Figure 10 shows an image generated on a cubed glass, which image has some reflections and refractions. We can estimate that there is a string of “GRAPHICS” under the cubed glass; however, we do not know how the image is generated and which part has the effect of reflection, refraction, or the combination. We can see that the top plane is divided into four regions and some images are flipped horizontally, vertically, or on both directions. On the other hand, Figure 11 illustrates the ray path in the cubed glass. In Figures 10 and 11, eight points indicated as A to H are the vertices constructing the cubed glass. In Figure 10, four numbers indicated as (1) to (4) show the divided regions on the top plane, and in Figure 11, the same four numbers show four rays that pass the same regions as those in Figure 10. In addition, Figure 11 shows that the numbers outside the parenthesis are near to the eye point, while the numbers inside the parenthesis are far from the eye. Each divided region in Figure 10 has the effect of reflection and/or refraction as the following:

Figure 10.

Reflective and refractive image on a cubed glass.

Figure 11.

Ray path in a cubed glass. (a) YZ cross section and (b) XY cross section.

1. Two refractions on the bottom and the top planes.

2. Two refractions on the bottom and the top planes and one total reflection on the side of ABFE.

3. Two refractions on the bottom and the top planes and one total reflection on the side of BDHF.

4. Two refractions on the bottom and the top planes and two reflections on two sides of ABFE and BDHF.

For generation of the image drawn on the top plane of the cubed glass, we need to decide the boundary between regions. In Figure 10, the image is flipped on the boundary line, and the flip is caused by total reflection inside the cubed glass. For example, the region (2) has one total reflection on the side of ABFE in addition to two refractions on the bottom and the top planes. Then, the boundary line on the top plane can be decided by the ray that passes the bottom edge line (EF) of the cubed glass. In addition, the boundary line between the regions (1) and (2) or the regions (3) and (4) is parallel to Z axis, and the boundary line between the regions (1) and (3) or the regions (2) and (4) is parallel to X axis. Therefore, the searching algorithm is as follows for the boundary line that divides the regions (1) and (3) or the regions (2) and (4). Figure 12 shows the process of the algorithm:

Figure 12.

Boundary line search algorithm. (a) Process of 1-5 (b) In case of AQ<AE and (c) In case of AQ>AE.

<Boundary line search algorithm>

1. Set A as A′ and C as C′.

2. Set the initial point P as the middle point of A′C′.

3. Calculate the refractive light with the incident and the refractive angles ($\mathrm{\theta }$ and $\mathrm{\varphi }$).

4. Calculate the intersection point of the refractive light and the line of AE, and set the point as Q.

5. If Q nearly equals to E, the line that passes P and is parallel to X axis is the boundary. Stop here.

6. If the length of AQ is shorter than that of AE, set the middle point of C′P as the next point of P, and set the original P as A′. Otherwise, set the middle point of A′P as the next point of P and set the original P as C′.

4.2 Simulation with a cubed glass

Figure 13 shows the simulation result with the proposed method for the case of Figure 10 and the comparison with the real photo. The top plane is divided into four regions, and the images on some regions are flipped due to total reflection on the side planes.

Figure 13.

Simulation result and the real photo for a string of “GRAPHICS.” (a) Simulation result and (b) Real photograph.

Figure 14 shows another simulation result and the real photo for an eraser that has texture on the surfaces. In this case, the front plane is divided into four regions, and some images on the regions are flipped due to total reflection. We can see that the two images are very similar.

Figure 14.

Simulation result and the real photo for an eraser with texture. (a) Simulation result and (b) Real photograph.

Figure 15 shows the process to generate Figure 14(a). Figure 15(c) is the image without reflection and refraction because the part is directly seen from the eye position and the ray does not pass the cubed glass. All images except for (c) have two refractions on the back and the front planes. The image (b) has only two refractions on the back and the front planes of the cubed glass. It does not have any total reflection. On the other hand, the image (a) has one total reflection on the side plane so that the string on the eraser is generated by flipping the image (b) horizontally. The image (e) also has only one total reflection on the bottom plane so that the image is generated by flipping the image (b) vertically. In addition, the image (d) has two total reflections on the side and the bottom so that the image is generated by flipping the image (b) horizontally and vertically. The image (f) is the combined image. Then, we can understand which part is generated by reflection and/or refraction by using the proposed method.

Figure 15.

Divided images and the combined image generated on one plane. (a) Reflection on side (b) No reflection (c) Image outside the class (d) Total reflections on side and bottom (e) Total reflection on bottom and (d) Combined image.

In Figures 13 and 14, the simulation result has an image only on one plane: top or front. However, the method can be applied to multiple planes. Figure 16 shows the application, where the images are generated on three planes.

Figure 16.

Simulation result and the real photo for an image generated on three planes. (a) Simulation result and (b) Real photograph.

In addition, Figure 17 shows the process to generate Figure 16(a). In the figure, all parts have an image seen directly from the eye position, so that it has no reflection and refraction. The image (c) has no total reflection and refraction. It has only the direct reflection of the eraser on the top plane; however, it is difficult to identify it because the image is not so clear, which is the same as that of the real photo (Figure 16(b)). Both (a) and (b) have only two refractions; however, the refractive planes are different. The refractive planes are parallel since there is no total reflection. The one is the plane that has the image, and the other is the plane that is parallel to it. On the other hand, both (e) and (f) have one total reflection on the bottom plane so that the string on the eraser is vertically flipped. The images (d) and (e) are generated by flipping the images (a) and (b) vertically, respectively. Finally, (f) is the combined image. The process of the image generation helps us to understand which part is generated by reflection and/or refraction.

Figure 17.

Divided images and the combined image generated on three planes. (a) No reflection on left front (b) No refraction on right front (c) Direct reflection on top (d) Total reflection on left bottom (e) Total reflection on right bottom and (f) Combined image.

5. Cylindrical glass

5.1 Movement of virtual vertex

This section describes how to generate the image for a cylindrical glass. In the inside of a cylindrical glass, there are four refractions at the most, and the position of a virtual vertex moves dynamically. Figure 18 shows the movement of a virtual vertex by each refraction through a cylindrical glass. In the figure, the thickness of the cylindrical glass is W, and Q is the original vertex. Vertex $\mathrm{Q}$ is directly seen from point D; however, the ray is refracted at point D so that the vertex cannot be directly seen from point C. If the vertex moves from Q to $Q_D$, it is directly seen from point C. The same movement should be done for the rest. If vertex $Q_D$ moves to $Q_C$, it can be directly seen from point B, and if vertex $Q_C$ moves to $Q_B$, then it can also be directly seen from point A. Finally, if vertex $Q_B$ moves to $Q_A$, then it can be directly seen from the eye position.

Figure 18.

Movement of a virtual vertex.

Then, each virtual vertex can be calculated as follows. The virtual Vertex $Q_D$ is calculated as the intersection between the line that passes C and D and another line that passes point $\mathrm{Q}$ and is parallel to the line OD. The remains are calculated with the same method. Vertex $Q_c$ is calculated as the intersection between the line that passes B and C and another line that passes point $Q_D$ and is parallel to the line OC. The vertex $Q_B$ is calculated as the intersection between the line that passes A and B and another line that passes point $Q_c$ and is parallel to the line OB. Finally, virtual vertex $Q^{\prime }=Q_A$ is solved as the intersection between the line that passes A and E and another line that passes point $Q_B$ and is parallel to the line OA.

If the final virtual vertex ${\mathrm{Q}}^{\prime }$ is obtained, we can generate the refractive image just by rendering the objects constructed with virtual vertices without considering any refraction.

5.2 Simulation with a cylindrical glass

Figure 19 shows the simulation image generated with the proposed method and the real photo. The eraser is distorted inside the cylindrical glass due to four refractions. The simulation result is very similar to the real photo, and the image can be generated in real time. That is, the simulation image changes in real time according to the movement of the real eraser.

Figure 19.

Simulation result and the real photo for a cylindrical glass. (a) Simulation result and (b) Real photograph.

6. Conclusion and future work

In this chapter, a new method to generate reflective and refractive images has been proposed. The method generates virtual objects that have virtual vertices translated from the original ones by considering reflection and refraction. By rendering the virtual objects, we can generate the reflective and/or refractive images without considering reflection and refraction. For the parallel glass, the calculation equations for the virtual vertex have been derived. In addition, total reflection and attenuation of light energy have been considered, and the attenuation is used as an alpha value to blend some images in alpha blending. On the other hand, the position of a virtual vertex moves dynamically for a cylindrical glass, and the position can be calculated as the intersection of two lines. Finally, the proposed method has been applied to two types of glass: cubed glass and cylindrical glass. On the both cases, the simulation results have been very similar to the real photos, and the image can be generated in real time.

The proposed method helps us to understand which part of the image is generated by refraction, reflection, or total reflection. In addition, we can understand on which plane the reflection happens or which pair of planes causes the refraction. However, by comparing the simulation results with the real photos, we see that the image quality is different, especially the transparency. Then, in order to improve the image quality, we have to try another technique such as blending the simulation result with environment map.