SYNTHESIS: A Platform of Platforms for Integrated Management, Curation, and Visualization of Digital Cultural Experiences through VR and AR Technologies

Abstract

SYNTHESIS is a platform of platforms (PoP) for the management, curation, and creation of digital cultural experiences related to cultural heritage material (artifacts), and their visualization through mobile applications and virtual and augmented reality (VR/AR) technologies. The digital cultural experiences may relate to specific cultural heritage monuments or exhibits of museum collections and may reside either on site or remotely. The platform provides a comprehensive supervisory tool that allows curators to select digital artifacts from different databases around the topic of interest, associate them through narratives that translate them into a sequence of semantic correlations that can be visualized using VR/AR technologies. The SYNTHESIS of platforms contributes to the creation of a new condition for the promotion of cultural heritage sites, museums, and exhibits, and facilitates the management and interaction of the user-visitor with them. The platform contributes to the creation of an innovative system for the management and curation of large volume of digital material related to an exhibit and consequently its promotion by unifying the stages of: (a) production of digital content; (b) connection of additional digital multimedia material related to the digital content; (c) curation of the digital content and transformation to a digital experience in accordance to a desired narrative; and (d) visualization of the digital experience in-situ or remotely with the use of mobile apps and VR/AR technologies.

1. Introduction

The SYNTHESIS PoP has emerged from the aggregation of three digital platforms that have been developed at the Integrated Systems Laboratory (ISL) over a period of seven years in the context of DICE (Digital Immersive Cultural Environments), a concept and architecture of a comprehensive ecosystem for the promotion of digital cultural experiences introduced by the author in 2014 and detailed in his publication, alongside with his research team at the Media Art Innovation (MAI) Unit of ISL, in 2016 [1]. The three building blocks of SYNTHESIS, namely the platforms SYNTELESIS/iGuide, AFIGISSI (Narration), and wayGoo, have been described as stand-alone platforms in the references [2, 3, 4, 5, 6, 7, 8, 9] (in Greek), and in [10, 11, 12].

SYNTHESIS PoP aspires to create a unified digital environment for the coherent creation, management, editing, and visualization of digital material related to cultural heritage artifacts (monuments, museums, museum exhibits, art exhibitions), and the development of personalized interactive narrative experiences using mobile and VR/AR technologies for visitors to places of cultural interest.

The SYNTHESIS PoP aims to offer tools for managing a large volume of digital material, supporting various forms of digital content, such as text, images, video, audio, 3D objects, BIM objects, Unity scenes, etc., and covering a wide range of available material. These building blocks, combined with recorded stories and information, will be interconnected to compose digital exhibits that will be converted into digital experiences, either in-situ through mobile and AR technologies, or remotely through VR technologies.

SYNTHESIS will contribute to the creation of a new condition for the promotion of cultural heritage sites or museum exhibits and facilitate the management and interaction of the user/visitor with them, and the overall user experience. The PoP will results in an innovative environment for the integrated management of a large volume of digital material, the creation of narrative experiences related to digital exhibits for visualization in physical and virtual spaces, and consequently the promotion of cultural heritage by supporting: (a) the production of digital content; (b) the linking of digital content to additional multimedia digital material related to said content, such as relevant data relating to the process used for creating digital content, relevant digital and digitized audiovisual material, recorded stories and information relevant to the context of the subject matter and narrative used as the basis of creation; (c) the designing and curating a narrative flow for a deeper understanding of the content associated with the subject of an exhibition of digital material, both virtual reality and physical space, by visitors and end users; and (d) the transformation of the curation outcome into experiences for in-situ and/or distance visualization capable of mobilizing both tourist and social interest in the importance of the cultural heritage as a holistic educational-educational experience and the need for its preservation and promotion, Figure 1.

Figure 1.

Curation of digital objects, creation of digital experiences through narration, and visualization with virtual and augmented reality technologies and mobile apps [1, 2, 4].

In the context of a digital experience (be it at a natural exhibition or museum, an archeological space, a virtual space, or a hybrid environment), it is possible to connect the digital exhibits semantically and thematically digital exhibits to a physical or virtual exhibition, museum, or archeological site, through the use of graphs and graph databases (Figure 2). In this process, the content reporting platform wayGoo [10] is used to visualize the georeferenced narrative in an easily to understand way in a physical, virtual, or hybrid reality space.

Figure 2.

Using graphs and graph databases for seamless digital content interconnection and digital narrative creation.

The easy understanding of the content by the end user in the context of a narrative is achieved through the transformation of the narrative into an immersive interactive experience capable of being consumed by common mobile tablets and smartphones using AR technologies in the exhibition/museum space/archeological site, Figure 3a, as well as through more integrated solutions such as VR headsets and/or projection imaging technologies for remote experiences in virtual space, Figure 3b. A narrative can be created by artists and people in the field of culture, using an accessible and friendly graphic environment. Storytellers will not be required to possess advanced and specialized knowledge of the underlying technologies used to implement it.

Figure 3.

(a) In-situ experience through AR. (b) Remote experience through VR technologies.

SYNTHESIS is accompanied by two applications available for use by the public (Figure 4).

1. An augmented reality application for mobile devices that offers possibilities of interactive exploration of an exhibition and consumption of all available digital material associated with it as an immersive storytelling experience during the physical presence of the user in situ.

2. A virtual reality application that allows the interactive exploration of an exhibit remotely and the consumption of rich 3-dimensional models and interactive scenarios with realistic graphics directly from a VR-enabled device.

Figure 4.

VR/AR applications of the SYNTHESIS PoP to be used by the general public for immersion experiences.

The SYNTHESIS PoP also offers advanced tools based on artificial intelligence (AI) and natural language processing (NLP) techniques to assist the discovery of contextual correlations between digital content, digitized assets, and digital artifacts and exhibits [13, 14]. By analyzing the structural elements of the exhibits alongside with all known connections among them, the system identifies possible undiscovered correlations and provides a multimodal and user-friendly representation and visualization of the results. AI and NLP tool and technologies are also used to implement a system of personalization and personalization of the experience in accordance to the user preferences. The personalization system recommends relevant exhibits and narratives based on user interests, in order to maximize the in-depth understanding of the content of the narrative associated with the curated experience in accordance with the curation and visualization approach of the digital content curator.

Finally, using blockchain technology, it is possible to implement smart contracts to control the use of digital content by its creator, and to track the ways of its utilization. This will safeguard the integrity of the digital content and protect the copyright and financial rights of its creator, Figure 5.

Figure 5.

Schematic representation of the SYNTHESIS of platforms with blockchain functionality.

In summary, the main goal of the SYNTHESIS PoP is to contribute to the promotion of cultural heritage items and places of cultural interest to a global audience by visualizing the results of recording and research with VR/AR technologies and the possibility of interactive and personalized experience of the relevant content. situ and/or remotely. It also aims to highlight the work of artists and people in the field of Culture, and to promote culture to the public through tools to create innovative and engaging experiential experiences using cutting-edge technologies. It will offer visualization of an exhibition through personalized interactive narrative experiences of virtual and augmented reality (VR/AR) and the georeferencing of its individual exhibits in places of cultural interest (art exhibitions, museums), allowing the creation of educational and educational experiences, the mobilization of tourism and social interest, and the preservation, promotion, and curation of its exhibits. In Figure 6, we give an example of using the wayGoo platform [9] to visualize an interactive experience of digital representations of cultural treasures from the Minoan palace of Phaistos in Crete created with the help of the SYNTELESIS platform by the Integrated Systems Laboratory with the use of digital immersion technologies, such as content geo-reporting (wayGoo), AR/VR, and 2D/3D animation.

Figure 6.

Example of using the wayGoo platform to visualize an interactive experience at the palace of Phaistos in Crete, Greece.

Additional objectives of the SYNTHESIS PoP include:

1. The further promotion and editing of cultural heritage items through the visualization of the results of recording and research and the creation of complex digital objects.

2. The promotion of the work of artists and people in the field of Culture, and the promotion of areas of cultural interest, and the promotion of culture at large to the public, through the provision of tools for creating innovative and attractive mixed-reality experiences using cutting-edge technologies.

3. The transfer of know-how on and the experience from research and development of state-of-the-art products from research centers and professional researchers to start-ups through dissemination of knowledge and networking, to assist them generate profit, develop human resource skills, and attract new and supporting extroversion and competitiveness.

4. The creation of new jobs in the creative industry and the provision of intellectual property and rights, including financial, for creators through smart contracts and integration of blockchain technology in the SYNTHESIS PoP ecosystem [1].

The SYNTHESIS PoP seeks to strengthen the activity in the field of Culture, Tourism and Creative (CTC) Industries, in terms of the production of digital content (entertainment, educational, cultural, commercial or other interest), and the organization of big scale activities and events. The PoP and user applications are expected to be marketed either as separate products or as a single system. The architecture and modular system of the platform, as described by its main components (wayGoo, NARRATION, iGuide VR/AR component), allows future adaptation to content and requirements of different organizations. The modularity of the SYNTHESIS PoP allows to accommodate different markets and different players in the CTC Industries in the SYNTHESIS ecosystem in a way that is expected to lead to maximizing market penetration and benefits for the constituencies of the ecosystem by enabling the efficient management of digital content, the facilitation of the conversion of digital content into experiences, and the protection of IPRs and profit flows through blockchain and smart contracts.

2. SYNTHESIS and the DICE¹ ecosystem

The adoption of the SYNTHESIS PoP is expected to lead to the following benefits in each one of the five (A, B, C D, and E) sectors of the DICE Ecosystem, Figure 7.

1. For businesses

   • Facility of specialized users in the design and editing of large-scale interactive experiences, through a narration tool that allow easy and creative workflow.

   • Creation of a perfect product, developing with scientific methodology and dynamic commercial presence.

   • Strengthening the technological base, gaining a technological lead in the field of production and supervision of interactive producers, which sets the specifications for the expansion of the developing platform.

   • Enhancing innovation and competitiveness in the field of artistic curation. Internationally, there is no system that provides a complete solution for large-scale interactive events.

   • Opening to new national and international markets. Penetration in the market through the possibility of highlighting and reusing places of cultural interest: Museums, Art Galleries, Archeological sites, preserved Industrial sites etc.

2. For cultural institutions

   • Strengthening cultural tourism by increasing the tourists’ interest in visiting cultural sites as a result of upgrading their on-site experience, but also through a strategy of diffusion of the results of the implemented activities.

   • Strengthening the networking of cultural organizations through the creation of business networks with the cultural and creative industry for the joint promotion of the innovative products of the project.

3. For the Creative Industry

   • Improving the productive potential of digital technology and digital content companies.

   • Approach of new markets in the field of culture and tourism, inaccessible until today, through their activity in designing interactive audiovisual content.

4. For research centers

   • Utilization of research for the development of market-oriented solutions, through the conversion of the research product into a commercial product. Added value that results through innovative interventions and interdisciplinary cooperation, in order to create a final product that promotes Greek tourism and culture.

   • Distribution of research results to the general public, through the commercial product but also through presentations at conferences.

5. For the Economy and Society

   • Improvement of the user experience and enrichment of the means of receiving the artistic/creative result, through his active participation in the interaction in natural and virtual space.

   • Promoting cultural spaces through the implementation of technologically and artistically perfect cultural events and enhancing the interest of the general public for them.

   • Promoting the collaboration of artists with audiovisual events.

Figure 7.

The digital immersive cultural environment (DICE) ecosystem [1].

The SYNTHESIS PoP is expected to act as a catalyst in the articulation of a digital creativity ecosystem for enabling the creation of DICE (Digital Immersive Cultural Environment) [1], Figure 7. The incorporation of smart contracts and blockchain technology in SYNTHESIS is expected to stimulate the process of crowdsourcing digital artifacts, narratives, and experiences alongside with their visualization, from professionals, as well as qualified non-professionals, in the fields of art, culture and creativity. The embedded protection of the Intellectual Property Rights (IPRs) of the digital creator build in SYNTHESIS, and the use of blockchain-enabled Smart Contracts to speed up the contracting and outsourcing process, track revenue streams form the use or sales of digital content, and protect the agreed upon revenue sharing with the creators of the digital content, narrative and experience, are expected to spearhead the process of digital creativity across the entire digital creativity ecosystem as shown in Figure 8.

Figure 8.

Enabling digital creativity ecosystem for the creation of digital immersive cultural environments (DICE) [1].

3. Methodology of flexible design and development of technological solutions

The SYNTHESIS PoP aims to create a coherent platform for the creation of advanced digital assets, the curation of digital objects and their semantic and thematic interconnection through the creation of narratives, and the creation of digital experiences in virtual and real space through the georeferencing of edited digital content and related storytelling, with a range of virtual and augmented reality technologies using tablets, smartphones, virtual reality glasses, and projection imaging technologies.

The SYNTHESIS PoP is based on the coherent integration of existing platforms, services, algorithms, and applications that have been developed in the Integrated Systems Laboratory (ISL) and their effectiveness has successfully been evaluated. The timely and effective accomplishment of SYNTHESIS objectives is largely ensured through the validation of the existing know-how and the possibilities of adopting it in developing solutions for other use cases, similar ot completely new. Technological solutions already developed will be adapted to new requirements and tools already available will be leveraged, adapted as required, and integrated in a coherent SYNTHESIS PoP.

The three platforms SYNTHESIS PoP is made up of, namely wayGoo, SYNTELESIS and AFIGISSI (Narration), are briefly describe in the following. All three platforms have been developed in the Integrated Information Systems (ISL) Laboratory of the Institute of Informatics & Telecommunications at NCSR “Demokritos” through a series of funded projects that are described in the publications [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] and acknowledged at the end of the Chapter. The three platforms are:

1. wayGoo - wayGoo is a platform for Geolocating and Managing indoor and outdoor spaces and content with multidimensional indoor and outdoor Navigation and Guidance. Its main components are a Geographic Information System, a back-end server, front-end applications and a web-based Content Management System (CMS). It constitutes a fully integrated 2D/3D space and content management system that creates a repository that consists of a database, content components and administrative data. wayGoo can connect to any third party database and event management data-source. The platform is secure as the data is only available through a Restful web service using https security protocol in conjunction with an API key used for authentication. To enhance users experience, wayGoo makes the content available by extracting components out of the repository and constructing targeted applications. The wayGoo platform supports geo-referencing of indoor and outdoor information and use of metadata. It also allows the use of existing information such as maps and databases. The platform enables planning through integration of content that is connected either spatially, temporally or contextually, and provides immediate access to all spatial data through interfaces and interactive 2D and 3D representations. wayGoo constitutes a mean to document and preserve assets through computerized techniques and provides a system that enhances the protection of your space, people and guests when combined with wayGoo notification and alert system. It constitutes a strong marketing tool providing staff and visitors with an immersive tool for navigation in indoor spaces and allowing users to organize their agenda and to discover events through the wayGoo event scheduler and recommendation system [10]. Furthermore, the wayGoo platform can be used in Security applications and event management, e.g., CBRNE and fire incidents, man-made and natural disasters, pandemics, etc., to document and geolocate information and sensor data (offline and real time) on one end, and offer navigation capabilities in indoor and outdoor spaces. Furthermore, the wayGoo platform can be used for the creation of immersive environments and experiences in conjunction with VR/AR (Virtual & Augmented Reality) technologies [9, 10].

2. SYNTELESIS – A platform designed to provide a combined web application with a mobile application and a unity desktop application to support immersive digital experiences with digital cultural assets in cultural heritage sites or in virtual counterparts. The platform features an extended exhibit object model, geo-referencing of places and exhibits with for digital physical support, and a variety of mobile and AR capabilities to support an immersive experience in-situ or remotely. A case study of an immersive digital experience of the archeological site of Phaistos, Crete, has been designed and implemented with the use of SYNTELESIS [1, 2, 7].

3. AFIGISSI (NARRATION): An integrated system for the management and editing of digital content and the production of personalized individual and collaborative narratives considers the objectives of the Faro Convention [15]. AFIGISSI aims at the development of a platform for the support, management, and promotion of digital content for infrastructures of cultural and tourist interest through the creation of personalized individual and collaborative narratives [7]. Digital collections offer the opportunity to examine objects not only as individual cultural elements but as an integral part of the cultural heritage as defined by the European Union [16].

Access to the narratives is possible either during the physical visit to the infrastructure during their regular operation and in extraordinary, periodic or temporary events or remotely. Traditional narrative practices address the development of narrative by placing objects on a linear axis. Instead, the work highlights cultural content with new narrative structures, operating with less strictly prescribed scenarios. In fact, according to new media theorist L. Manovich [17], the database is generally the way to tell a story today. The creation of narratives is achieved in an automated and semi-automated way, by reversing the time axes and with a variety of thematic grouping options, making a theoretically infinite number of combinations [18], in order to respond to modern perceptions of object multiplicity and the possibility of multiple interpretations. Personalization is done both for individual users and for groups of users who work together to create a narrative. According to the current level of collaborative platform technologies, there are possibilities to connect multiple users for the purpose of a common task [19]. The system allows specialized and non-specialized users to produce stories of various sizes based on the content available and share them with the public. The aim is a holistic approach to storytelling that will lead to mature technology and an innovative tool aimed at the market for (a) the design and implementation of digital collection management software platforms for cultural operators, and (b) web applications, mobile and VR/AR applications for organizations of digital exhibitions and creation of narratives from a registered collection of museums and other cultural exhibits.

In the SYNTHESIS PoP, the three building-block platforms are redesigned, expanded, and integrated into a single controllable and coherent digital creativity environment with the addition of blockchain technology for the protection of IPRs and the introduction of smart contracts for the facilitation of signing contracts for the creation of digital assets, content, context, and experiences, and the verifiable of equitable sharing of revenue streams across the entire creativity chain involved in the creation and visualization of digital experiences, Figure 9.

Figure 9.

Schematic representation of the main structural elements of the SYNTHESIS platform.

4. SYNTHESIS PoP architecture

Figure 8 depicts the Implementation Architecture of the SYNTHESIS PoP. In the architecture, there are three distinct layers: The Data layer, the Business Logic layer, and the Presentation layer. Each layer is made up of various components as indicated in the infographic of Figure 10.

1. The Data layer supports:

   1. (interfaces with) third party databases;

   2. innate to SYNTHESIS databases, such as mongoDB, Neo4j, PostgreSQL, …;

   3. a repository of digital assets; and

   4. a blockchain module for the protection of IPRs and the support of Smart Contracts.

2. The Business Logic layer supports:

   1. an Application Programming Interface (API) that allows users and end users of the PoP to interface with SYNTHESIS through either

      1. the Web Portal for the production, curation, management of digital content and the creation of experiences, or

      2. the Visualization Applications (mobile and VR/AR);

   2. A Back-end Support System (docker) for dockerizing content;

   3. an AI (Artificial Intelligence) component that consists of two modules:

      1. a Content Management System based on Natural Language Processing (NLP) to support contextual and semantic correlations and AI algorithms and applications, and

      2. a Personalization and Recommendation system alongside a variety of statistical, fuzzy and AI based processing algorithms, to support user-tailored digital experiences of the curated digital material and narratives.

3. The Presentation layer supports:

   1. a User Interface (UI) in the form of a Web Portal that allows to:

      1. Insert Digital Content in SYNTHESIS, and

      2. Manage Digital Content in PoP;

   2. User Experience (UX) through End User Applications that include:

      1. mobile apps for both Android and iOS smartphones, and

      2. VR/AR technologies for a variety of portable platforms and devices.

Figure 10.

Implementation architecture of the SYNTHESIS platform of platforms.

Moreover, the SYNTHESIS PoP architecture and the accompanying applications support the following:

• Enrichment of support and management of geo-reported data related to natural areas, together with services for the consumption of this information with GIS/Tile services.

• Improving the determination of geographical location indoors to facilitate indoor navigation with the use of existing know-how from the implementation of a Bluetooth Beacons infrastructure developed in the context of the European funded projects FlySEC, http://fly-sec.eu/ [20] and TRESSPASS https://www.tresspass.eu/The-project [21].

• End users can consume the digital content of the platform through a mobile application. The application can be used: (a) either during the physical presence of the user in an exhibition, where is guided in the tour of the exhibition, taking into account user priorities, or (b) remotely, where the user is guided to explore the exhibition and delve into the digital content that the platform provides. Using GPS technologies and/or Bluetooth beacons, the application can locate the user physical location, suggest relevant exhibits and narratives, and assist in exploring and navigating the area. Users can locate exhibits and places of interest through an interactive map, from a list through a narrative, by scanning their own QR code, or through a search with free text which is interpreted lexically and semantically by the back-end AI module. Information about the exhibit is then presented, accompanied by its correlations with other exhibits, narratives, dates, and natural sites. Finally, it is possible to view 2D/3D models directly from the device, as well as play accompanying multimedia material in the form of images, videos, panoramic images, and sounds [7, 8, 9].

• Improving the process of creating narratives and their representation using graphs, as well as the general management of digital content and repositories [2, 4, 5].

• Design and development of Blockchain infrastructure for the use of smart contracts. To preserve the integrity of each element and to track the ways of its utilization, SYNTHESIS utilizes blockchain technology. This technology forms the basis for creating an unchanging event log. This file through the security properties of Blockchain will provide a high degree of protection of data integrity as well as their continuous availability. Within the SYNTHESIS PoP, the blockchain will record the interfaces between nodes of the graph as well as the changes in these interfaces. In addition, it will store metadata and metadata related to digital data as well as record which users of the “SYNTHESIS” platform utilize each component. Additionally, blockchain will record information about derivative projects that utilized a digital component as well as project location data in both physical and virtual exhibitions. The above functions make it possible to use the smart contracts executed on the blockchain. Smart contracts will allow creators to control which users (creators or curators) will be able to use the digital data they have created and for what purpose. Finally, smart contracts enable any interaction (e.g., data entry) with the blockchain.

• SYNTHESIS allows the establishment of correlations between different narrative elements (recommendation process) enriched with audiovisual material if required [13], and offers personalization services and personalized functions [10]. Based on the categorization of the narratives and the corresponding evaluations by the users of the application, machine learning and automatic classification techniques are applied to identify similarities between the narratives and their individual structural elements. These similarities are multi-layered, reflecting similarities that are based on the preferences of the users but also based on the content, and are determined conceptually and lexically, after processing natural language from any descriptions/tags/categories that accompany each narrative and each structural element. In addition, where possible, from each element, depending on its format (audio file, image, video, text), additional metadata/features are extracted using deep machine learning techniques. The additional metadata/features represent each object based on its components (e.g., color choices and image contrasts, rhythm/style of music, etc.). Through correlations with the respective user ratings, similarities are calculated in greater depth allowing the discovery of new correlations and features that determine user preferences. The result is a multidisciplinary graph that represents the narratives and objects, and the similarities between them at all the levels of similarity considered [11].

• With the aim of visualizing different types of experiences, the SYNTHESIS PoP supports the visualization of digital content of various types and formats (2D/3D) through a virtual reality application for mobile devices and VR headsets. Content can come from 3D modeling tools, e.g., Building Information Management (BIM) objects, or gaming machines, e.g., interactive experiences in 3D environments, (https://vimeo.com/isldemokritos). Each type of content offers different possibilities and therefore it is necessary to support all types of 3D content. The visualization of rich 3D experiences is realized with a user-centric approach [17, 18, 19] and through a (separate) application implemented in the Unity game machine and provides the possibilities of 3D visualization and navigation. The wayGoo application opens the Unity application and loads the corresponding experience by downloading the corresponding Unity asset bundle from the platform. Communication between the two applications can be implemented with mobile URL schemes that allow the use of parameters when opening applications [12].

5. SYNTHESIS PoP use in different use cases for pilot tests

The three platforms SYNTHESIS is based on, namely iGuide/SYNTELESIS, Narration/AFIGISSI, and wayGoo have been tested in different use cases, environments, and applications as pilot tests. Below we make reference to three such use-case scenarios by summarizing the use case environment, the scenario used and the targeted audience and objectives.

The developed solutions in the three use case pilot test scenarios aimed at analyzing the of SYNTHESIS and the three platforms within it, in terms of the efficiency and effectiveness of creating immersive experiences in the context of DICE. The three carefully selected use cases demonstrated the ability of SYNTHESIS, and the three platforms is made up of, to create:

1. iGuide Knossos VR: An immersive 3D VR tour guide for the palace of Knossos [22], a blueprint paradigm for touring and experiencing archeological sites that are partially or totally ruined, with the help of mobile and VR/AR technologies, either in situ or remotely [2, 3, 4], Figure 11.

   In March 1900, on the Greek Island of Crete, Arthur John Evans, a British archeologist, unearthed what he called the Minoan Civilization; a colorful world that seemed forgotten by time. A blend between reality and imagination, the reconstruction of the Palace of Knossos has become an iconic symbol of the past, triggering endless debates and providing a rich canvas for creative reconstructions.

   iGuide Knossos is a HD/3D VR reconstruction of the Knossos Palace that merges Evans’ vision of the site with contemporary archeological evidence. It is meant to be used as a VR edutainment environment, providing a fertile ground for future development of educational, entertainment and action games such as Fruzzle, Knosseau and Knossos Labyrinth (thumbnails provided), which are using the Minoan civilization and myths as a foundation for their action play.

1. wayGoo in-situ AR/VR tour of the archeological sites Knossos and Phaistos, a blueprint paradigm for in-situ touring and experiencing archeological sites, that are partially or totally ruined, with the help of mobile and VR/AR technologies, that allow in-situ visitors to experience rich digital multimedia content (text, audio, video, including 360^o, 2D/3D models and animations, games, etc.) and associate it with relevant locations and PoI’s² in the physical site through wayGoo’s geolocative and automatic navigation capabilities, Figure 12.

1. AFIGISSI (NARRATION) The AFIGISSI platform has been used in a number of pilot project to demonstrate its functionalities in curating digital cultural assets into experience that can be visualized using different technologies. We present two use cases next. One from the Teriade art museum of contemporary paintings in Lesvos, Greece [24]; and the other a curated audiovisual experience based on an imaginary narrative leveraging 3D assets from iGuide Knossos VR and visualized using projection mapping technology [25].

Figure 11.

iGuide Knossos is a HD/3D VR reconstruction of the Knossos palace.

Figure 12.

Floor plans and drawings: Levi, D., 1976. Festos e la Civilta Minoica I, Rome: Ediz.dell’Ateneo. 3D modeling and application of materials [23].

5.1 Digital curation of the Teriade museum exhibits using the AFIGISSI platform

The AFIGISSI (NARRATION) platform is developed to support, manage and promote digital content of cultural and tourist infrastructures (tourist sites, museums, archeological sites, exhibition spaces, etc.) through the creation of personalized individual and collaborative narratives. Access to narratives is possible either during a physical visit of an infrastructure during its regular operation (such as permanent exhibitions) and/or its temporary events (such as periodic exhibitions) or remotely. The project reveals cultural content by implementing new narrative structures, less strictly defined scenarios, automated and semi-automated modes, by overturning time sequences and by offering a variety of filtering and grouping options to respond to modern perceptions of versatile objects and their multiple interpretations. The platform allows specialized and non- specialized users to produce narratives based on the available content and to share them with targeted audiences. The curation of the (digitized) exhibits of the Teriade museum has been used as a pilot use case for demonstrating the ability of the AFIGISSI platform to store, manage, curate digital content in accordance to a narrative, and transform the curated outcome into an experience that can be visualized with smartphones and VR/AR technologies, Figure 13a.

Figure 13.

a. Digital curation example using the AFIGISSI platform: Conceptual depiction of the transitioning: (a) from museum exhibits to digital assets, (b) from digital assets to digital exhibits, (c) from digital exhibits to digital narratives; and (d) from digital narratives to exhibits and their visualization with smartphones and VR/AR as in Figure 3a and Figure 3b, respectively.

5.2 From i-guide Knossos to the journey of Ariadne

An audiovisual projection mapping installation inspired by the world of Knossos and its digital interpretation and constitutes a blueprint for the creation of an immersive digital experience (Ariadne’s Journey) from 3D digital assets and animations created within iGuide Knossos for the reconstruction of the ancient Minoan Palace of Knossos in Crete, Greece, an original narrative that addresses the process of knowledge discovery through excavations of historic monuments, and history itself, by a young contemporary girl, and the visualization of the outcome of this curatorial effort with VR technology and projection mapping [5, 25]. The narrative, artistic and spiritual field of Ariadne’s journey explores the construction of memory and the meeting points of the historical and mythological constructions that experience the unexpected journey of a young modern heroine. Through esthetic narrative, the work aims to create an experience for listeners of all ages, translating, or perhaps more accurately, the interpretation of CH images and historical documentation into a living esthetic vocabulary, Figure 14 [23].

Figure 14.

Ariadne’s Journey concerns an audiovisual projection mapping installation, inspired by the people of Knossos. Ariadne leaves the city behind to discover a fantasy world, created from familiar and strange images, where the myth, the archaeological hoe and the dreamy mood are intertwined in a colorful fantasy puzzle. Ancient ruins are transformed, frescoes come to life [23].

6. Conclusions

In this chapter an extended description of the SYNTHESIS Platform of Platforms (PoP) has been given. The architecture and the three platforms SYNTHESIS is based on are described in detail. SYNTHESIS results from the integration of the three platforms iGuide, Narration (Afigissi), and wayGoo, in a coherent environment that allows the seamless creation of augmented digital assets, their management and curation in the context of a narrative, and the creation and visualization of digital experiences with mobile applications and VR/AR technologies. SYNTHESIS is the outcome of an over a decade research and development at the Integrated Research Laboratory at NCSR Demokritos towards the development and implementation of the Digital Immersive Cultural Environment (DICE) concept. Examples of three different digital immersive experiences that have been developed as DICE applications to demonstrate the capabilities of SYNTHESIS as PoP for creating digital experiences, have been given. SYNTHESIS supports blockchain and smart contract technologies that are used to protect IPR’s and enable smart contracts to safeguard the use of digital content and encourage the creation of new digital content, its curation and distribution by protecting the IPR’s of the creators and honoring contractual obligations and revenue streams agreed upon between creators of digital experiences and distributors of these experiences,

Acknowledgments

The research described in this paper has been supported by the following research contracts:

AF3- Advanced Forest Fire Fighting: under the Programme FP7 with Contract No. 607276, duration 36 months (1/5/2014- 30/4/2017) and the Funding Organization is the European Union.

SYNAISTHISI: Intelligent data collection and processing platform for energy efficient applications, Contract No. 2012 KRHPIS, Hellenic General Secretariat of Research & Technology, Duration: 32 months (02/04/2013 – 12/31/2015), Co- financed by Greece and the European Union- European Regional Development Fund (ERDF), under the O.P. “Competitiveness and Entrepreneurship” (OPCE II) and of R.O.P Attica, R.O.P Macedonia- Thrace.

SYNTELESIS: Innovative Technologies and Applications Based on Internet of Things and Cloud Computing National projects, November 1, 2017 - April 30, 2020.

AFIGISSI (NARRATION): Integrated system for management and curation of digital content and production of personalized and collaborative narratives, National project, European Union- European Regional Development Fund, Contract No. Τ1ΕΔΚ-03584, June 4, 2018 - December 3, 2021.

COSMOS: Cultural Osmosis - Mythology & Art, National project, European Union- European Regional Development Fund, Contract No. Τ1ΕΔΚ-04283, July 9, 2018 - August 31, 2021.

TMI Builder: Tailor Made Itinerary Builder, National project, European Union- European Regional Development Fund, Contract No. Τ1ΕΔΚ-03580, July 9, 2018 - August 31, 2021.

FLYSEC: Optimizing time-to-FLY and enhancing airport SECurity,” Programme: Horizon 2020, European Union Grant Agreement No. 653879, Duration: 01/05/2015 - 31/07/2018, http://www.fly-sec.eu.

TRESSPASS: Robust Risk Based Screening and Alert System for Travelers and luggage, Grant Agreement No. 787120, Call: H2020-SEC-2016-2017-2, https://www.tresspass.eu/The-project.