1.1. Evolution of technology: Vision and goals
Like many technologies, virtual reality began as a dream and a vision. For example, the desire to fly had to wait for technology to progress before becoming a reality. Though the story of Icarus and Deadalus might have inspired a Leonardo to draw a bird-like flying machine, centuries past before science and technology set the stage for flight beyond mere kites and gliders. A high powered, light weight gas engine was one of many innovations required for heavier than air aircrafts to become a reality in the early 20th century. Over the course of a century, a series of innovations and inventions emerged that that were critical to the development of modern aircrafts. Although in principal today’s aircrafts share much with their earlier predecessors, they have capability that far exceeds those early machines. Why it is now possible to fly at 30,000 ft in comfort, at speeds of over 500 mph, can best be understood as the culmination of a convergence in technological development over the last century.
The proposed replacement for the F16 fighter, the F35, was only possible with improvements in material technology. Being composed of carbon fiber material makes this jet a third lighter than its predecessor. With computer numeric control (CNC) the F35 is built to tolerances not previously achieved for carbon fiber aircraft. The engine built by Pratt &Whitney, today’s most powerful engine, produces over 50,000 lbs at weight to power ratios much greater than could have been achieved even just a few years ago. Avionics and sensors give “situational awareness” and provide the F35 a level of virtual intelligence and awareness critical to mission success (Keijsper, 2007).
The piloting of these aircraft is assisted by the onboard computer capability. To assist the pilot in making critical decisions, a helmet mounted display gives information on targets and the aircraft’s control systems. Today’s modern jet aircrafts bear little similarity to the early predecessors made of wood canvas and wire and yet, both fly. The technology of flight has advanced beyond the dreams even of science fiction writers of just a century ago. Though the goals of flight have remained the same, the form of their technological solution has evolved in ways that could never have been imagined.
Virtual reality, in its development parallels that of flight. The simulators used in flight training during the Second World War share much in principal with those made today for military and passenger jets, even if they look like inventions out of retro-science fiction movies. Built in the day when mechanical lineages and analogue electrical gauges provided feedback in modern airplanes, these flight simulators could train air force pilots to fly in conditions of total fog or darkness. The desire to improve on the safety record of early pilots was the incentive for the Edwin Link to create simulators built from the same technology used to control pipe organs of the day. Yet even these crude devices incorporated much of what characterizes flight simulators today. With the link trainer, the student pilot was placed inside an enclosed cockpit mounted on a 360 pivot. Once inside the cockpit, the pilot would practice flying blind. With working controls and instruments, the pilot could practice flying in an immersive environment complete with feedback. The stick of the link-trainer worked like an actual airplane and allowed the trainer to turn and bank. Mimicking a real aircraft, artificial horizons and altimeter gauges provided the pilot with the feedback need to fly blind. Pilots would progress through a simulated mission with their route traced on a large table with a pen mounted on a small motorized carriage. Using radio headsets, communication between the trainer and the pilot simulated the actual sound between ground control and pilot. Advanced models featured the full instrumentation of the modern fighters of the day. With over 10,000 of these units built during the Second World War, they can still be found in many air museums in North America and Europe (Link Flight Trainer, 2000).
The link simulator provides two important lessons in the history of technology. First, virtual reality and the desire to have an immersive training environment preceded the computer revolution that began in the 1960’s. Today’s simulators, though similar in functionality and purpose, share little in the underlying technology used to accomplish their goal. Today’s multimillion dollar simulators reproduce the view, sound and motion experience in a real jet cockpit. Built on top of a six degrees of freedom motion platform, the entire cockpit of a jet can bank, yawl and pitch as it flies under simulated conditions. When first developed, the computing requirements for these simulators were at the cutting edge of computer technology. Advanced graphic engines, parallel processing, high resolution graphic displays, motion controlled platforms, and a geographic database of the world’s topographical features are all critical milestones in the history of computing and have contributed to the design of modern flight simulators (George, 2000)..
Second, VR as a simulation of reality has been instrumental in the advancement of the technology it simulates, in this case flight. The ability to fly an advanced fighter jet or one of the new generations of fly-by-wire passenger jets was dependent on the same technology used to create advanced simulators. For each advanced jet that flies today there is a simulator that prepares pilots for the actual experience of flight. Companies like Boeing, CAE and Lockheed Martin operate advanced simulators which utilize motion platforms, high resolution graphics systems and databases containing all the world’s land features and airports at high resolution. In these simulators which mimic almost every aspect of flying in the cockpit, pilots can prepare themselves for such experiences as flying into bad weather or responding to a mechanical or electrical systems failure. The safety of an entire industry now depends on these advanced simulators and their capability to train pilots on how best to prepare for these extraordinary events.
Millions of dollars have been invested in creating simulators for the military that have proven indispensible for training pilots for all types of aircrafts. Simulated worlds are also valuable tools in many fields for exploration, testing and training. But, beyond flight simulators, advanced applications are rarely found in training and education. For example, there are a few advanced auto simulators used to in research on passenger and driver safety in US and Europe (NADS, 2012; Schwartz, 2003). However, given their great expense, it is doubtful they will be used to train a young teenage driver when the alternative, a practice permit and a shopping mall parking lot on a Sunday morning, is a simpler and less costly first introduction to the driving experience.
1.2. Simulators and the demand for high end graphics
When computing technology was first used in the development of simulators, the requirement for realistic simulation pushed the envelope of computing for the period. In the 1990’s when simulators emerged as important training tools, displaying geographic detail in real time demanded high-end graphics mainframes. In part, SGI’s early success resulted from the creation of the ONYX Infinite Reality engine, capable of rendering multiple views in high detail. Built for high-end rendering, an Onyx could contain up to 20, four 150 mhs processors. The Reality Engine2 Graphics system in 1998 (George, 2000; SGI) was capable of rendering up to 2 million mesh polygons and 320 M textured pixels per second, descriptions of these computers, which are still impressive by today’s standards (NVIDIA, 2012), were critical to rendering the multiple views needed for each of the cockpit windows in a flight simulator. Pilots could view in details of cities on their flight path, and the landing and taxing to gates of any major international airport. When these virtual reality simulators were placed on a six degrees of freedom motion platform pilots had the experience of flying without ever leaving the ground. Today’s consumers can have a taste of this experience by purchasing PC game programs like Flight Simulator (Microsoft, 2012) that put you in the cockpit of a Boeing 747 or WWII fighter. With multiple screen display, a PC with a gamer’s video card and a dedicated yoke lets the average consumer achieve what the creators of the Link simulator could only have dreamed of half a century ago. Though inexpensive hardware has had a critical role in the history of VR, the demands for specific applications for work and play may be the force in promoting the diffusion of VR in the future.
1.3. Building the market for VR
Every technology benefits from an application that creates a growing demand. Without growth in markets, products remain in a niche supported by a few high-end users, e.g. the flight simulator. To drive down unit costs, products must appeal to a growing audience. Like the first computers built after the war, the demand for these costly goliaths served only a very small number of corporate and military users. For VR to grow beyond the use in the military and commercial aviation, a new group of potential users would have to be found. This would require the development of hardware and software solutions for medicine, architecture, urban planning and entertainment. Unlike flight simulators designed for a single task, the approach was to accommodate a range of applications in a multipurpose VR facility. Ultimately, this strategy could expand the number of potential users. CAVE’s with multiple screens displaying content in 3D would seem to have offered a technological solution that would satisfy a range of users. With government research support, many universities and national labs established virtual reality centers, which were to offer engineers, design professionals, urban planners and medical researchers with a much needed facility for advanced visualization.
2. Applications of VR
2.1. The design of cities
In the 1990’s, universities had sufficient funding to acquire sophisticated computing power. For example, UCLA obtained an SGI ONYX. In this facility Los Angeles city planners were given a first opportunity to visualize urban form in an immersive environment. Rather than crowding around a computer monitor to view data, specialized projection screens enabled planners and government officials to experience and evaluate development proposals within a life size 3D virtual world. The work of Jepson and Liggett at UCLA offered a glimpse into a future that promised public participation into the urban design process (Hamit, 1998; Jepson & Friedman, 1998; Liggett and Jepson, 1993). In their simulator, it was possible to see the impact of "what if questions" while driving down the city streets of Los Angeles.
VR had promising beginnings with several cities taking up the challenge of using VR as a tool in urban planning. Several universities would be in the vangard of this movement, following in the footpaths of Liggett and Jepson, including the University of Toronto and the New School for Social Science Research in New York. Creating VR environments with detailed virtual cities required the time and resources of CAD modelers and programmers (Drettakis, G., Roussou, M., Reche, A. & Tsingos, N., 2007; Batty, M., Dodge, D., Simon. D. & Smith, A., 1998; Hamit, 1998). Since the 1990’s many North American, European and Asian governments have created CAD models of their cities, but they are often used to produce animations, rather than used to enhance the planning process (Mahoney 1994; Mahoney 1997; Littelhales 1991). Animations are important as part of marketing campaigns to promote, for example, a new train line, landmark commercial development or public space. Without a driving interest by the professional planner to use VR in urban design and planning, its application over the last two decades has been limited to the exceptional case. Even today with GIS to easily show a city’s buildings in 3D, models of a city are largely inaccessible to planners who often lack training and access to their corporate GIS.
Though planning has embraced the charette, open house and web-based survey, the discipline has yet to grab hold of design in real time. In part, this is a problem of logistics and cost. Finding facilities adequate to hold even half a dozen individuals is difficult. For those wishing to display 3D worlds in stereo, the price of glasses, special projectors or displays makes the technology out of reach for most city governments (Howard & Gaborit, 2007). Finally, there is a cultural dimension of planning practice which limits the adoption of VR. Planning is still largely done in a 2D world. Zoning maps and plot plans are easily stored, visualized and analyzed in 2D. Even for planners who have received their education during the last decade, an introductory course in CAD or GIS may not have been required. The older generations of planners, now in more senior positions, are even less likely to be knowledgeable GIS and CAD users ( Mobach, 2008; Wahlstrom, M, Aittala, M. Kotilainen, H. Yli-Karhu, T. Porkka, J. & Nyka°nen, E. 2010; Zuh, 2009).
Potentially, the impact of land use change on future development could be better understood using simulation tools of the kind used by transportation planners in designing and maintaining a road system for a city. Yet, most planning departments rarely use such modeling approaches. In contrast, the game world since 1985 has had a simulation tool, SIM City, which allows anyone with a PC to manage a city’s budget and understand the impact of land use planning on the future development of the city (Simcity, 2012). Interestingly, a land use planning tool designed for professional city planners has yet to be become the norm in urban planning practice. Without the vision for what simulation can do for planning, serious tools have yet to appear in practice. Without the commitment to a vision of what potentially could be accomplished through the application of computer technology, these tools will await future development. Furthermore, land use policy can be implemented and enforced without the benefit of an advanced information system. In fact, VR and other advanced visualization technology may be counter productive to the planning process. Visualization of proposed development, if not carefully introduced to the political electorate, may incite adverse reactions from the public and create more work for the planners and their staff (Al-Douri 2010, Forester 1989; Mobach, 2008). Advanced modeling and simulation for this reason may not always be seen as a beneficial by practicing planners.
2.2. The rebirth of physical urban models
Recent innovation in 3D printing may actually reinforce the use of a physical model over that of the virtual world, when it comes to visualizing the future urban form of cities. With lower costs associated with 3D printing, it is now possible to create plastic models directly from CAD models. In the past, the expense of creating a model of a city with all of its buildings was not a simple task. Scale models of an entire city required teams of artists and cartographers to complete. Unlike paper maps and drawings, you would also need a large space for storage and examination. Brest, Cherbourg and Embrun are examples of a few cities for which scale models were commissioned by Louis XIV and constructed under the direction of Sébastien Le Prestre de Vauban (Marshall of France, b1663-d 1707) Known for his publications on siege and fortification, Vauban supervised the creation these models as important tools in the preparation of military defenses (de Vauban, 1968) Ultimately, more than 140 of these models were constructed for the King of France(Lichfield). In the 20th century, there are numerous examples of these types of large scale models. The most impressive include those representing Daniel Burnham’s Plan commissioned for the Chicago Foundation in 1909, a model of Rome commissioned by Mussolini, the model of Los Angeles built under the WPA’s in the 1930’s and a model of Moscow completed in 1977(Itty Bitty Cities, Urbanist).
In the 1980’s under Donald Appleyard, Director of the Environmental Urban Lab, University of California, Berkeley, College of Environmental Design, a full scale model of San Francisco was constructed(Environmental Simulation Laboratory). Prior to the use of computer modeling, this physical model served an important role in assessing the impact of new development on the immediate surroundings. By employing a video camera mounted with a model scope fixed to a moving gantry, it was possible to drive along roads and view the existing city and proposed developments. These models lit by an electric lamp were also capable of simulating the sun and shade at various times of the year. By examining alternative plans for development within this physical model it was possible to have a tool for facilitating public review of urban projects.
With the recent introduction of inexpensive 3D printing, it is now possible to create 3D models of entire cities. Like models of the past, such models of Tokyo, Toronto, and New York offer a bird’s eye view of a large area (Itty Bitty Cities, Urbanist). Though the virtual version would offer greater flexibility in data retrieval, viewing alternative design concepts, understanding impact of zoning, and having physical representations are considered highly desirable by government officials, planners and architects. Like Lego Land and doll houses there is a strange attraction to such miniature worlds that is difficult to comprehend on a rational level.
2.3. Architecture and the art of image making
Advancements in BIM (Building Information Modeling) and CAD has given the architecture profession new tools for creating buildings. BIM can offer designers a completely interactive design space. Buildings can be designed from the ground up in a virtual environment. Using a host of simulation tools it is now possible to work collaboratively with the client and other consultants on the design of a building. Though it would appear to offer advantages over more traditional tools, it is yet to be an approach universally accepted by architects within their culture of design (Levy 1997; Novitski 1998). Since the Renaissance, it was through the art of drawing that architects distinguished themselves from the building trades. Even today architects differentiate themselves from engineers and urban planners by their artistic abilities and talents. Beginning in the 15th, architects of the stature of Michelangelo, Sebastiano Serlio Palladio and Bramante employed the plan, elevation and section to create their designs(Kostoff 1977; Million 1997; Palladio, 1965). Knowledge of these drawing conventions, first mentioned in the oldest surviving architectural treatise by Vitruvius, Il Quattro Libri (1st Cent AD) are still considered essential skills for architecture students today (Kostoff 1977; Vitruvius 1960). One important aspect of orthographic constructions (plan, section, elevation) is the ability to take direct measurement from the scaled drawings. Borrowing from these established drawing conventions, CAD applications reflect the architect’s preference for working in plan and elevation for determining design solutions. Visualizing the design in perspective occurs after the architect has created his concept in plan.
Perspective as a tool emerged during the Renaissance with the inventive work of Filippo Brunelleschi (1377-1456). Borrowing from the theory of optics and the mathematics of the period, Brunelleschi should be credited in creating the first augmented reality device. Using a set of mirrors, Brunelleschi was able to position a perspective drawing of the Baptistery of Florence onto a view of it in the Piazza Santa Maria, thus demonstrating that the new science of perspective drawing could simulate reality. The actual device consisted of two mirrors. The first of the two mirrors would be positioned in front of your eye. A small hole at the centre would allow you to view a second mirror, which was placed showing a perspective view of the Baptistery. When standing in front of the Baptistery in the exact same location from which the perspective was constructed (as a mirror image) the observer would see the image of the perspective reconstruction of the baptistery superimposed over its actual location. Varying the distance between the two mirrors would change the size of the perspective image relative to the actual surroundings. By removing the mirror furthest from the eye, the observer could compare the actual image with the perspective construction. Offered as proof that perspective was a tool for presenting how an architect’s design would look when constructed, perspective would become a device for presenting both the real and the imagined (Collier, 1981; Edgerton, 1974). In the 16th and 17th century, perspective would become particular useful in the design of stage backdrops for fantastic architectural scenes used in opera and the theatre. Giovanni Maria Galli-Bibiena is perhaps one of the most important artist of this period whose work would grace the opera houses of Europe and would later be published as engravings by Christopher Dall'Acqua, and JA Ambrose Orio Pfeffel in 1731 (Pigozzi 1992).
Though a perspective drawing is an important visualization tool for architects, it is mostly relegated to the role of a presentation graphic. Often created for the client’s benefit, perspective drawings are not the working tools of architects. Instead, it is the plan and elevation that serves the architect during the conceptualization and construction process. In published works by Andrea Palladio and Inigo Jones, their skillful use of plan and elevation is still studied by today’s architects (Kostoff, 1977; Million 1997; Palladio, 1965). Later this approach to design would be adopted by the Ecole des Beaux Arts in Paris. With an emphasis on acquiring a high level of proficiency in creating high quality images in ink and color washes, the school’s graduates would dominate the teaching ranks of architecture in the US and Europe during the period in the 20th century prior to emergence of modernism. Even with the emergence of the Bauhaus in the 1920’s, programs of architecture in the US or Europe would still demand expert draftsmanship from their graduates (Kostoff, 1977, Levy 1980).
Once a design is approved by the client, architects create the working drawings needed for the bidding and construction phase. In the past, working drawings were drawn in pencil or ink on velum to create the needed blueprints. The process of creating “working drawings” was a significant part of practice, consuming many hours of draftsmen’s time. Photos of architects’ offices from the last century often show junior architects working on drafting tables producing the drawings, which today would be printed on wide carriage plotters. In practice, changes and additions are part of the design process. With paper drawings, even a simple change, like the replacement of one window style with another could require hours of redrawing. Beginning with the introduction of CAD (computer aided design) in the 1980’s, architects were relieved of the burden of having to make endless changes to paper drawings. CAD offered advantages over the traditional drawing methods used by architects since the Renaissance, but for many architects, the art of drawing distinguished them from engineers and technicians (Kostoff, 1977; Levy, 1997). Even when the interactive age of design seemed imminent, CAD was never widely adopted. Today, the culture of architectural design has yet to fully embrace the use of advanced CAD tools in design. Within an architectural firm, decisions rest with the senior partners; often, they are uncomfortable with the new CAD technology. For this reason, these new tools are the used primarily by technicians to create construction documents. Virtual reality design, a hopeful prospect in 1990’s has yet to be fully developed or implemented.
With the advancement of BIM in recent years, architects have new tools for design and construction management. With BIM, an extension of CAD tools from the 1980’s, it is possible to create integrated design solutions from concept to finished drawings. The design process begins with the development of a massing solution that responds to the urban context of zoning, adjacency of other buildings, and topography. Once a massing solution is produced, architects can move to the next phase of the interior space plan, which must respond to the needs of the program. At this stage, design becomes a multi-dimensional problem. BIM can provide both the senior and junior architect with a testing environment. BIM tools encompass the full range of design activities including the structural frame, HVAC system, and the electrical and mechanical systems. It can even analyze the flow of pedestrians responding to an emergency evacuation. Potential conflicts can be resolved early in the design process, rather than after the project begins, when changes and additions would need to be made to working drawings. In a virtual design environment, the production of documents for bidding is a matter of freezing the design solution. Unlike the past when drawings were inked on vellum, these virtual buildings can now be sent electronically to the general contractor responsible for the actual construction of the building. Using software like Autodesk’s Navistar, it is now possible to plan for every aspect of the construction process. This includes the critical placement of cranes that accommodate the lifting of all materials to higher floors in the case of commercial buildings (Autodesk, 2012).
2.4. Accommodating the design process: The work environment and architecture
Design requires the participation and involvement of a team of professionals. For a large scale project, sharing and working with a large number of documents, drawings and CAD models requires a versatile and flexible work space. Traditionally, at the early stages in the development of a design, critical decisions are reviewed by sitting around a boardroom table with documents strewn on the table’s surface. With drawings pinned to the walls, architects with notebooks and pens in hands, sketch, take notes, and exchange ideas about possible design solutions. The history of VR is one marked by the need for specially designed hardware and rooms needed to view a virtual world. This includes some of the earliest hardware, Sensorama in 1956, the work of Ivan Sutherland in 1960’s, followed by more advanced hardware used for flight simulators and CAVE installations in the 1980’s and 1990’s. Hardware solutions required for interactive viewing has always been expensive and cumbersome to use in group settings. Imagine conducting a group design session with HMD’s or wearing shutter glasses in a CAVE environment (Benko, Ishak & Feiner 2003, 2004; Coltekin 2003; Sutherland, 1965; Zhu, 2009). The cultural context of design always needs to be considered if VR technology is to gain greater acceptance among architects. If VR is to be used in the design, then it needs to support the culture of design. Visiting a dark room or isolating individual users with helmets and gloves will probably be never acceptable for architects.
Internet-based solutions for distributing work among collaborators may have had a greater impact on architecture design than VR. Today, a process of distributing work is becoming the norm in architectural practice. At the commencement of a project, a senior designer meets with the client to discuss objectives and concerns. Drawings, sketches or simple massing models created in a program like Sketchup may be used to establish the constraints, opportunities and context of the project. Commonly, many senior designers without education in CAD, provide their sketches or even models to staff for later conversion into CAD models. Frank Gehry, the architect of the Walt Disney Concert Hall and Guggenheim Museums in Bilbao and the Experience Music Project in Seatle is often cited as an architect who works from models which are later converted into CAD format by his staff architects. Once in CAD form, models can be shared between client, partners in other offices and consulting engineers (Kolarevic, 2003). For larger international firms distributing these tasks over several offices is the trend. By passing a project at the end of the day to associates in another part of the world at the beginning their day, a project can be completed more quickly by using a full 24 hour working day. This approach has two significant advantages. First, it allows firms to shorten the time required to complete a single design cycle. More important, firms can take advantage of lower wage scales abroad. Over the last decade architectural offices have employed designers, drafters, and CAD technicians in China and India, where the wages are significantly lower. By engaging firms that specialize in architectural design and production, a dramatic reduction is possible in the cost of working drawings, specifications and computer models used to produce high quality animations and renderings (Bharat, 2010; Pressman, 2007).
VR is making progress in changing the approach to design for interior design by major house ware retailers. IKEA now offers on-line a design tool that gives the prospective buyer an opportunity to layout a complete kitchen. The user can begin with the floor dimensions of their kitchen area and then add cabinets and appliances. Working in 3D they can alter styles for the cabinet, wood finishes, materials for the kitchen tops and the choice of appliances. By examining these designs in 3D, the client has an opportunity to create a virtual world of their future kitchen. Once completed, an order list and price sheet is generated for the store to complete the sales transaction with the client. Already, IKEA is experimenting with home design and will complete a major housing project in Europe in 2012. Perhaps the future of interactive design lies with manufacturers of homes where complete environments are delivered to the job site for quick assembly. (IKEA, 2012).
2.5. Archaeology and VR
Virtual Rome, developed under the direction of Bill Jepson at UCLA, was one of the first projects to take advantage of the rendering capability of the SGI ONYX reality engine. Using technology first developed for the film industry a computer model of the entire ancient city of Rome could be rendered in real time. A major attraction at conferences like SIGGRAPH and AEC (Architects, Engineering and Construction), this model of Rome was displayed on large panoramic screens in 3D. With the support of GOOGLE in recent years, the Virtual Rome Project can now be viewed in Google Earth, though the visual impact is much less on the small screen. The success of the Virtual Rome Project has fostered the creation of other virtual historic models including those of Jerusalem and Pompeii. More than models, these worlds can support virtual tours complete with guides that provide historical background and information. When viewed in 3D environments, on panoramic displays or in Geodes, like the one found in Paris, researchers and the public have a new venue for viewing and studying these ancient sites (Fore and Silotti, 1998; Frischer; 2004 Firscher, 2005; Rome Reborn; 2012; Ancient Rome, 2012)
With the appearance of long and short range scanners, archaeologist have been capturing data on historic sites and building. Once captured, it is possible to use this data to reconstruct these sites in their entirety by reproducing missing elements. One of the more notable projects involves the reconstruction of the Parthenon. Under the direction of Paul Debevec of UC Irvine, a team was assembled to scan both the Parthenon in Athens and plaster copies of the Elgin Marbles, which are preserved in Bern Switzerland. By adding the missing sculptures found in the pediment to the virtual model, it was possible to see for the first time in almost 200 years the Parthenon complete with all of its sculptural detail. Animations and images from this model were later used to promote the Olympics in Athens 2008. The ability to recreate an environment free of smog may be an additional benefit of viewing these models in virtual space (Addison 2000; Addison, 2001; Eakin, 2001; Levoy, 2000; Tchou, et al, 2004; Stumpfel, et al, 2003).
Visualization technology has been instrumental in the advancement of medical science, beginning in the Renaissance with the printing of Vesalius, De Humani Corporis Fabrica Librie Septem in 1543 (Saunders, 1973; Vesalius, 1998). Though theory perpetuated by the Galen still persisted long after his death in 199 AD, the publication of this treatise would eventually transform medical science. The illustrations contained in this tome were based completely on human dissection, revealing all the organs and skeletal structure of the human body and transforming the study of medicine and anatomy. With the advancement of imaging and computing, this approach has been extended. It is now possible to transform an individual’s MRI data into a virtual model of the individual patient, opening new doors to medical diagnosis. Using advanced 3D imaging, the “Lindsay Virtual Human” allows a student of medicine to examine the human anatomy at any scale: organs, tissues and cells. Furthermore, unlike the printed page, this virtual human is completely interactive. With the ability to simulate physiological processes, the virtual human can be used to help medical students understand life processes in real time. Viewed in stereo displays or mobile touch devices, “Lindsay Virtual Human” provides access to a virtual living being ( Lindsay, 2012; Von Mammen, et al, 2010).
In the use of virtual models in surgery, a major challenge has been to develop haptic peripherals that allow the surgeon to have needed feedback to perform delicate operations. Without sensitive feedback from surgical instruments, the response of actual tissues and organs to incisions made by surgical tools would not be experienced realistically in the virtual world. In many areas of surgery, including removal of brain tumors, the virtual and the real have merged to create an approach for performing challenging surgical operations. In the 1980’s during early days of robotic surgery, only pre-operative images were used to guide the surgeon. Robots compatible with MRI’s in 1990’s were developed that provided the surgeon with images reflective of the patient’s condition throughout the surgery. Over the last two decades improvements to neurosurgical robots have included better imaging technology that can distinguish soft tissues, a robot with a full 6 degrees of freedom, and greater precision in the actual surgical instrumentation. Filtering out a surgeon’s hand tremor has made for a much higher level of precision in these delicate procedures. Though robotic surgery is still far too expensive for general use, its continued development shows promise. With improvement in artificial intelligence, kinesthetic feedback and user interface, neurosurgery will see more robots assisting surgeons in the operating room (Greer, 2006; Howe & Matsuoka, 1999; Sutherland, 2006).
VR has had the greatest universal impact on society in the merging of play and computing (Johnson 1999; Shaffer et al, 2004). Even in the early days of computing when all computing was done on mainframes, a first space war game created by Steve Russell at MIT in 1962 allowed the user to control a spaceship in a world where gravitational forces shaped the strategy for destroying adversaries. Later this game would be released as an arcade game in 1971 and became one of the first games to employ vector graphics. Interestingly, this early arcade game can still be purchased in its arcade form by game officinatos (Space Wars, 2012). With each improvement in graphics and computing power, games were able to attain a higher level of realism. Better shadows, real water, particles, photorealistic lighting, and more life-like characters provided gamers with experiences that mirrored those found in the real world. Today, games can simulate every aspect of life, real or imagined, on PC’s or game consoles. Driving games, flight simulators, fantasy, role playing and war games are a few of the genres that have spawned from an industry that competes with the movie industry in size and value. With each new release, higher levels of graphics and realism are anticipated by gamers. It is now possible to simulate photo realistic lighting and architectural details as a game player drives through European cities while competing in the Grande Prix. In simulations of military combat, series like “Call of Duty” have re-created the war theatre for many of the famous engagements in Europe during the Second World War II (Call of Duty, 2012). More recently Activision, the developer of “Call of Duty’, has turned to military engagements in the Middle East and wars placed in the future. Within these game environments, groups of combatants are able to play against each others (MMOG). Success in these MMOG’s requires both time and dedication. The addictiveness and interest in war games has not been lost on the US Military( Johnson, 2004; Johnson, 2010; Stone, 2002). With America’s Army 3, developers have created a game that that allows participants to experience life in the military in a massive on-line experience. In this world, you can fire weapons and participate in elite combat unites ( US Army, 2012). Perhaps the military’s most successful recruiting tool, America’s Army Game, this virtual world allows you to assume roles and responsibilities of battle field soldiers and to train for a variety of missions.
Virtual worlds are not limited to recent historical events. In the Assassin’s Creed series, the opportunity is given to engage enemies in worlds that mix historical fact and fantasy. In Assassin’s Creed series the gamer is placed in 15th Florence and Venice. Strangely set in the 21st century, the central character Desmond Miles, having escaped from Abstergo Industries, is forced to relive past memories in a virtual past. Rendered in high detail, this world would provide a class in art and architectural history with a virtual classroom to view some of the greatest achievements of the Renaissance. Assassin’s Creed with its open world play environment established a new level of visual accuracy in detail for a virtual world even if there is an occasional mixing of content from different historical periods.(Assassin’s Creed, 2012) Unfortunately, time is of the essence in this game. The mission to avenge the murder of father and brothers leaves little time to gaze upon architectural wonders of the past.
Gamers who demand a high level of emersion in their visual filed of view can now purchase 3D TV’s and 3D monitors that offer the quality of a 3D movie experience at a price only slightly higher than that of the average display. With movie theaters and production companies capitalizing on the 3D movie experience, this feature can now be added to most game experiences. Many games today are 3D ready and only require a video card that supports 3D, as well as either a 3D TV or a computer screen. By purchasing peripherals developed to support a specific game genre, a completely immersive experience is possible, whether flying a plane, driving a car or fighting off the enemy.
The Wii in 2006 introduced a new level of interaction and immersion to gamers. No longer did the user need to rely on a game controller that required hours of practice to master. Instead, the Wii controller uses motion sensing, a few buttons and gesture to control the virtual world. Using the Wiimote, a tennis rackets or a sword can be simulated and with purchased attachments, the action of a driving wheel and other devices can be imitated (Nintendo, 2012). Since the introduction of the Wii by Nintendo, Microsoft and Playstation have introduced game player controllers. Furthermore, the Kinect by Microsoft, employs real time mapping of the human form and gives gamers a sense of freedom by eliminating the need for any game controller (Microsoft, 2012). Even the youngest of game players quickly learn how to ski jump, play tennis or bowl in these virtual games. After several decades of experimentation, virtual reality has a universal audience of devoted gamers.
2.8. Military applications
The demonstrated value of flight simulators in training would spawn other applications of VR to the field of military training. Tank simulators are an example where VR would provide a valuable training environment. Inside an enclosed replica of an actual tank, gunners and drivers can maneuver and fire. Because tanks have a limited field of vision, simulating their view does not require high resolution graphic displays and expensive six degrees of freedom motion control platforms. Like a MMOG, tanks can be networked together. With coordination from central command, tank groups can practice field maneuvers. Today, the ability to link together training environments consisting of planes, tanks, and ships provides the military with opportunities to train and test strategies that require the coordination from central command of numerous combatants in the field, the air and on the sea (Johnson, 2004; Johnson, 2010; Stone, 2002).
Coordinated efforts of actual infantry on the ground in a true simulation of actual battles are the ultimate VR challenge. One solution which has been used for decades is to build mock villages for soldiers to practice their engagements. Still used today to prepare soldiers for conflicts in the Middle East, these towns and villages are inhabited by actors speaking the language of the countries where the troops are to be stationed. More recently, a completely virtual experience has been created using the most advanced motion capture and immersive technology. VIRTSIM, designed and developed as a joint venture between Ratheton Corporation and Motion Reality, Marietta, GA, is similar to the holodeck featured in Star trek from TV and film. In a space as large as a basketball court, a dozen soldiers wearing high resolution wireless 3D glasses can engage simulated combatants in a completely virtual world. Being able to view each member of the platoon as they engage virtual combatants provides the trainers with a unique perspective. In addition, it is now possible reconstruct actual troop engagements based on pervious battles logs from GPS transponders. To simulate battlefield conditions, each soldier wears electrodes that respond to virtual bullets and bomb blasts (Economist, 2012; Virtism 2012). Though perhaps the most expensive interactive video game environment ever created, these virtual training environments were outside the limits of computing and visualization until recently.
Not all simulated military training require the holodeck. In developing training environment to teach members of the US military Arabic and Farsi, an inexpensive PC based solution was deployed. In the Tactical Language Training System, students are introduced to Arabic, Farsi and Levantine languages and culture through participation in a virtual world. Like a video game, this role playing is used extensively. In these worlds, students have an opportunity to be immersed in a game space where they interact with animated characters in settings based on urban, rural and village life found in Iraq. An interactive story environment engages the learner; animated characters provide feedback on both pronunciation and dialogue. Speech recognition technology, which focuses on the most likely responses, gives the student feedback on appropriate responses with native speakers. This approach has shown promise even with students having limited prior experience in foreign language instruction (Johnson et al. 2004, Johnson 2010).
3. Augmented reality
From the early days of Virtual Reality and AR (augmented reality), researchers shared any of the same issues. Both required knowing where the user is relative to the scene in view. In AR, it is critical to superposition computer generated content accurately on the objects and architecture in the real world. In the early history of AR, a portable computer capable of generating even simple wire frames of models in 3D was no small feat. Then, projecting the content on to glasses worn by the user was an additional challenge. Cumbersome HMD (glasses) were expensive, heavy and difficult to wear for long periods of time. Making this a mobile solution would be almost impossible given the weight and size of laptops, batteries, video cameras and glasses. Not surprising, the early days of AR transformed the user into a borg-like image from Star trek. Finally, there was the need to access a database of places and associated attributes. The potential to store some information on a PC or PDA about a building or city existed, but a solution that would work in any locale would need to access the Internet. In the days before Wi-Fi and cell towers, access to the internet was not assured (Benko, Ishak & Feiner 2003, 2004; Coltekin 2003, Sutherland, 1965).
Within the last decade, AR has finally become a reality with the miniaturization of portable and wearable technology. The diffusion of inexpensive smart phones and tablets provide a platform capable of supporting AR based applications. With a video camera, GPS and compass, a smart phone can access a database of content and superimpose directly into the view of the touch screen. With the introduction of applications like Google Goggles and Layars, developers can use the power of an image search database. Though AR applications are still in their infancy, opportunities to apply this technology will be greatly expanded with the diffusion of tablet computers. With the potential for displaying a larger image in view, tablets with two video cameras, a powerful processor and access to the Internet will make AR applications exciting for a range of uses including, tourism, architecture, engineering, medicine, and education. Today, a foreign tourist can take a picture of a restaurant sign and gain access to the menu in his or her own language. It would also be possible to provide the specials of the day and the local critic’s reviews all translated in real time. Similarly, AR could provide a tourist with a guided tour through an historic neighborhood and learn about the people and events that happened in the past. Filters could be added to confine the information to recent history or perhaps, to provide the architectural history of significant buildings in the area (Benko, Ishak & Feiner 2003, 2004; Bimber 2005; Gutiérrez, Vexo & Thalmann 2008a, 2008b).
One limitation of GPS is that it works only in open space. Furthermore, GPS will not provide precise superposition of content on the scene. Another strategy is to rely on object recognition. By capturing views and matching them against a library of known objects, it is possible to determine what the user is looking at and to provide the appropriate content overlayed or tagged to the object. However, even when image search is not feasible, it is possible today to use the camera in a smartphone to capture a QR Code marker, which links to web-based content. A perfect solution for educational institutions or museums, individuals with either a smart phone or a tablet can access text, images, video and 3D objects (Schneider, 2010).
If carrying and pointing devices at buildings and signs feels unnatural for some users, in the future, it may be possible to have a hands free AR environment by wearing contact lenses or light weight glasses. Research is already promising in this area. Companies like NOKIA can project web content onto lightweight stylish tinted glasses, complete with wireless earbuds and a haptic wrist controller. Ultimately, it will be the users who will decide if this approach is acceptable (Nokia, 2012).
In the 1980’s, VR was only capable of producing a scene composed of wire frame images. Even finding a screen that could show graphics was beyond the reach of most researchers. In 1990’s the development by SGI of high resolution real time graphics made simulators possible and met a critical need for the military and civil aviation. Over the last two decades, VR has been used to train pilots in simulators and provided surgeons with real time models of the human body. Since the beginning, building a market for VR has been a challenge. Other than commercial games, the audience for VR has been fairly limited. In creating applications for a narrow group of users, the cost of development has restricted its diffusion. The cost of application development includes building models, designing and animating characters, coding behaviors and responses and building the GUI interfaces. Even a modest virtual world requires considerable financial resources. If there are few users to bear the true cost of development, then the application’s price will restrict it to a limited audience. If the use of virtual environments for training and education in the workplace, home or school is to have a promising future, reducing development costs is key.
With better scripting tools it may be possible to reduce the costs and time of developing a VR application. Software like FlowVR, now offers programmers an integrated solution. With FlowVR, it is possible to build worlds that merge the on-line worlds created in Second life with Layar Augmented reality (FlowVR, 2012). By using Microsoft Xbox Kinect and Emotiv (thought controlled headset), inexpensive hardware can be used to view and control virtual worlds for training and education. For those who only require a simple navigation through a virtual world, Unity, an inexpensive game engine offers a cost effective solution for architects, planners and archaeologists to visit virtual worlds on a PC.
In the history of technology it is often difficult to predict the impact inventions will have on society. When computers were being used to solve business problems after WWII, the transformative power they would have on society would have been hard to imagine. Though the exact path VR has followed may not have been predicted, its development as a force owes much to those who had the vision and belief in its potential use to train, educate, and entertain. Hardware and software will always be a limiting factor in the development of VR. Similarly, an acceptable immersive solution for visualizing content will also be part of this equation. The need to wear cumbersome head mounted displays or visit a remote site to view a design in an CAVE has certainly limited the diffusion of VR. In creating VR solutions, the technology must accommodate the professional culture of the user. For architects sitting around a boardroom table or a surgeon in an operation room, the technology must do better in the context of professional practice. Perhaps the promise is portable personal devices like tablets and smart phones. The capability of these devices increases with each new model. With a projected world wide use of over two billion by 2015, this platform offers the greatest hope for the future of both AR and VR (Parks Associates, 2011). Everyone has one, and we carry them everywhere. Developers will finally have a market sufficiently deep to create applications that serve every market and profession. Perhaps VR is not dead, but merely ported over to a more universal accessible platform.