3D Modeling and Computer Graphics in Virtual Reality

Abstract

In the era of digital information technologies, 3D modeling and computer graphics techniques not only apply to the development of virtual models for computer simulation, artificial intelligence (AI), big data analytics, etc., but also they can be applied in many different applications in virtual reality (VR). However, the computer graphics effect and visual realism are usually the trade-offs with the real-time and realistic interaction in VR. In this book chapter, we would like to review the general flow of the VR program development process, and the recent 3D modeling and texture painting techniques used in VR. On the other hand, we would introduce some of the key 3D modeling and computer graphics techniques that can be applied in VR in order to enhance the speed of interaction. The key techniques including smoothing techniques and mesh editing modifiers are not only useful for the designers to learn the 3D modeling process, but it also helps to create less complex mesh models maintaining good visual effects. The techniques are particularly important in the development of 3D models to satisfy the demanding computation requirement of real-time interaction in VR program.

1. Introduction

In the past few decades, virtual reality (VR) has been widely used in many different areas including entertainment, education and training, manufacturing, medical and rehabilitation. The compound annual growth rate for VR revenue is expected to grow more than fifty percent from 2018 to 2023. It is expected that education and training is one of the leading sectors in the coming 5 years [1]. VR not only provides immersive stereoscopic visualization of virtual environments and sound effects, but participants can also interact with virtual objects and environment with haptic feedback. No matter what kind of application to be applied by the VR, the visualization effect and computer graphics are critical to enhance the engagement of participants and thus increases the education and training effectiveness [2]. Nevertheless, increasing the visual realism in VR is not an easy task because it is not only due to artist’s sense of the design engineers but also due to the drawback between the realistic VR environment and the demanding computation requirement of real-time interaction in VR.

3D modeling and computer graphics techniques have been developed for several decades [3]. Due to the era of digital information technologies, 3D modeling and computer graphics techniques drive the explosive growth and becoming crucially important in the recent years. The techniques not only apply to the development of virtual models for computer simulation, virtual reality (VR), augmented reality (AR), mixed reality (MR), etc., but also it can be applied to many various application such as artificial intelligence (AI), big data analytics, etc. [4]. Despite VR technologies have been developed for many years, the development of computer hardware and the 5th generation (5G) mobile network bloomed the 5Vs of the data flow including volume, velocity, value, veracity and variety [5]. As a consequence, the computation requirements and the flow of big data in VR is very demanding not only due to the need for real-time interaction, wireless connection, data inter-exchange, but also due to the greater expectations in computer graphical effects, realistic 3D models and infectant of virtual environments.

We would like to organize this book chapter as following sections. In Section 2, we aim to review the major software in 3D modeling and rendering in computer graphics. We will present the key computer modeling, computer graphics and VR programming software and tools. The techniques in computer modeling and graphics are particularly important for real-time and realistic interaction in VR. Therefore, in Section 3, we will describe some of the key modeling techniques used in VR. These techniques include shading and mesh editing modifiers. We will compare the difference of these techniques and their visual effects.

2. Modeling and texture painting tools

The development of VR models is divided into several key procedures. The VR models are used to create the virtual scenes used in the VR program. Figure 1 shows the flow chart of the VR program development. In general, the development process is developed into three major steps including modeling, texture painting and VR programming. The virtual models are firstly modeled using 3D modelling tools to create the object 3D geometries. After completion of the 3D modeling process, the models are rendered using computer graphics techniques including materials painting, texture mapping, etc. This process can be done directly on the 3D modeling software. Then, the 3D models including the corresponding graphical UV texture maps have to be imported into the game engine for the development of VR computer program. Alternatively, the texture painting and rendering processes can be performed by separate professional software. Then, the 3D models including the texture maps are used as the input of the game engines. The 3D models and texture files can be exported into various file formats depending on the compatibility between the software. Some of the commonly used file formats of 3D models are FBX, OBJ, STL, etc. FBX (Filmbox) is a proprietary file format (.fbx) developed by Autodesk and is used to provide interoperability between digital content creation applications. FBX is commonly used as the part of game wares and is recommended in the development of VR program [6].

Figure 1.

The flow chart of the VR program development. The development process includes modeling, texture mapping and VR programming.

Nowadays, there exists number of 3D modeling tools such as ZBrush, Blender, SketchUp, AutoCAD, SolidWorks, 3Ds Max, Maya, Rhino3D, CATIA, etc. Table 1 summarized and compared the major differences of these 3D modeling tools. Most of the commonly used modeling tools are professional and used for industrial application. These tools not only used in the computer-aided design (CAD), but also provides some advanced features such as computer-aided engineering (CAE) for performing analysis [7], additional manufacturing (AM) and 3D printing [8].

Modeling software	Ease of use	Category	Support formats
SketchUp	Beginner	CAD	dwg, dxf, 3ds, dae, dem, def, ifc, kmz, stl
Blender	Intermediate	CAID	3ds, dae, fbx, dxf, obj, x, lwo, svg, ply, stl, vrml, vrml97, x3d
ZBrush	Professional	CAID	dxf, goz, ma, obj, stl, vrml, x3d
AutoCAD	Professional	CAD	dwg, dxf, pdf
SolidWorks	Professional	CAD	3dxml, 3 dm, 3ds, 3mf, amf, dwg, dxf, idf, ifc, obj, pdf, sldprt, stp, stl, vrml
3Ds Max	Professional	CAID	stl, 3ds, ai, abc, ase, asm, catproduct, catpart, dem, dwg, dxf, dwf, flt, iges, ipt, jt, nx, obj, prj, prt, rvt, sat, skp, sldprt, sldasm, stp, vrml, w3d xml
Maya	Professional	CAID	dxf, fbx, obj, stl
Rhino3D	Professional	CAID	3 dm, 3ds, cd, dae, dgn, dwg, emf, fbx, gf, gdf, gts, igs, kmz, lwo, rws, obj, off, ply, pm, sat, scn, skp, slc, sldprt, stp, stl, x3dv, xaml, vda, vrml, x_t, x, xgl, zpr
CATIA	Professional	CAD	3dxml, catpart, igs, pdf, stp, stl, vrml

Table 1.

Comparison of major modeling software [11].

Traditionally, CAD tools are used to translate the CAD file into VR format directly by a downstream process [9]. However, the CAD tools usually provides a complex and highly detailed CAD data, common in engineering design and other industries, which makes it translates into excessively large VR models. This makes the models difficult to maintain the speed of computation in an acceptable level. In this case, models optimization need to be implemented to allow real time interaction by reducing the complexity of the models which makes the modeling process complicated.

Computer-aided industrial design (CAID) tools not only provide 3D modeling features, but it is also used in various industries like 3D printing, animation, gaming, architecture, and industrial design for digital production. The CAID tools provide designers with improved freedom of creativity compared to typical CAD tools [10]. The 3D model can be saved in a format that can be read for AM to speed up the prototyping process, so that designers can has more time to focus on the design processes. CAID also provides a larger flexibility for sketching, design and modeling for designers, thus particularly suitable to create flexible models to meet the extensive demand of visual realism in VR nowadays.

Table 1 summarized the major CAD and CAID modeling software in the market. AutoCAD, SolidWorks, CATIA are the major CAD software for engineering design developed for many years. The software is designed professionally not only for performing engineering design, but also provides a number of features for engineering analysis and simulation. 3Ds Max, ZBrush, Maya are the CAID software widely used in various professional design application. It provides a larger freedom for designers to perform freeform and digital sculpting, and allows the model files to be exported into AM formats for rapid prototyping. Blender was initially released in 1994 and was developed by the Blender Foundation. Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, video editing and 2D animation pipeline [12]. It provides various modeling functions for VR and are easier to be handled by most of designers and engineers.

Besides the 3D modeling, texture painting is an essential step to enhance the visual effects and increase the realism of virtual environments. Most of the 3D modeling software such as Blender, 3Ds Max, ZBrush provides the texture painting features and pipelines for 3D rendering, which are sufficient to most of the VR production. Other professional 3D texture painting software include Substance Painter, Mari, Armor Paint, Quixel Mixer, etc. The software is professional and some of them are even used in movie production. However, it may require more professional skills and experience to handle the software. Table 2 shows the major 3D texture painting software for creating 3D models textures in VR.

Texture painting software	Ease of use
Blender	Intermediate
Armor Paint	Professional
ZBrush	Professional
Quixel Mixer	Professional
Substance Painter	Production
Foundry Mari	Production

Table 2.

Major 3D texture painting software.

3. Modeling and computer graphics in virtual reality

In order to create 3D models in VR for real-time interaction, one approach is to perform optimization to reduce the complexity by minimizing the mesh size of the models. However, a significant drawback of this approach is that the visual realism of the models may be affected. Therefore, in this section, we will describe some essential modeling and computer graphics techniques that can be applied to create 3D models in VR. These techniques not only able to reduce the mesh size of the models, but also keep the visual realism effectively without the need of additional modeling procedures.

There are some fundamental techniques we need to understand in order to make good models. Shading is one of the key techniques in 3D modeling. There are several approaches to perform mesh shading including flat-shading and smooth-shading. As seen in Figure 2, most of the models are represented by polygons and truly curved objects are often approximated by polygon meshes [13]. When rendering the models, you may notice that these polygons appear as a series of small, flat faces (Figure 2a). In order to create a desirable effect, traditionally edge split and subdivision surface can be applied to smooth the model (Figure 2b). However, this will increase the number of faces and vertices of the models thus its complexity therefore is not desired in VR applications. The easiest way is to generate visually smooth model is to apply the auto smooth shading filter to quickly and easily changes the way the shading. The mesh shading does not actually modify model geometry, it simply changes the way of shading by calculating across the surfaces, giving the illusion of a smooth surface (Figure 3).

Figure 2.

Example of 3D mesh model rendered: (a) flat and (b) smoothed using subdivision surface.

Figure 3.

Same 3D mesh model applied smooth shading.

The shading approaches can create the mesh non-destructively by calculating the faces normal. Alternatively, or in some cases, mesh editing tools such as bevel, subdivision, loop cut, etc. may need to be applied at the edge to create better visual effects. Figure 4 shows the 3D models applied the bevel modifiers with 20 segments, 2 segments and 6 loop cuts (from left to right). The visual effects looks similar, but the mesh size increases significantly with the number of bevel segments. The bevel modifiers with 2 segments can create similar effect with 6 loop cuts, but the modeling process is easier. Although 2 bevels and 6 loop cuts are more effect, but usually we would prefer the 3D models to look nice and smooth. The loop cuts will create a sharper edge, therefore bevel segments are more preferred in VR modeling.

Figure 4.

Example of using bevel modifiers with 20 segments, 2 segments and 6 loop cuts (from left to right).

The corresponding models are rendered using the real-time render engine Eevee in Blender. Figure 5 shows the visual effects of the models represented by red, green and blue color (from left to right). The visual effects can be previewed quickly in the modeling software. The red (left hand side) model can create a very smooth visual effect, which looks similar to the green (middle) one. However, the blue (right hand side) model shows sharp edges clearly which may not be desired visually. Therefore, modeling with bevel modifiers with a few segments will be preferred. Sometimes, it can also be applied together with the smooth-shading to create a better visual realism of 3D models in VR.

Figure 5.

The rendering effect of the models applied bevel modifiers with 20 segments, 2 segments and 6 loop cuts (from left to right).

4. Conclusions

In this book chapter, we have reviewed the recent exiting 3D modeling and texture painting software packages and the difficulties in handling the software. The key techniques used in the creation of 3D models for VR are also described. The techniques including the shading and mesh editing modifiers not only help reducing the mesh size of the 3D models but also maintaining the visual realism of the models. It is particularly important to meet the demanding computation requirement of real-time interaction in VR program. Results have also shown that bevel modifiers with a few segments can enhance the visual effects compare with the loop cut modifier. However, this feature will change the mesh size of the model. The smooth shading modifiers not only maintain the complexity of the models but also enhanced the visual realism significantly. The mesh editing and shading modifiers can also be applied based on the requirement of the models in VR program.

Acknowledgments

The author(s) received financial support from the Hong Kong Polytechnic University, the Hong Kong Special Administrative Region, China, for the research, authorship, and/or publication of this chapter.

Conflict of interest

The authors declare no conflict of interest.

Declarations of gratitude

We would also like to extend our thanks to the Department of Industrial and Systems Engineering, the Hong Kong Polytechnic University, for the support in our VR projects.