Open access peer-reviewed chapter

Supporting New Ways of Interaction in Cultural Environments

By Ricardo Tesoriero, Jose A. Gallud, Maria D. Lozano and Victor M. R. Penichet

Published: December 1st 2009

DOI: 10.5772/7740

Downloaded: 1256

1. Introduction

Cultural environment guides are evolving. Some years ago, these guides were audio devices where visitors had to introduce a code to get information about an art piece. Nowadays these devices are being replaced by PDAs or SmartPhones where multimedia information (audio, video, photos and audio text) is acquired from a server that is part of the cultural environment management system. Thus, through this architecture the information is easily updated on the server providing visitors with latest information about the expositions. Besides, mobile devices provide visitors with new possibilities of interaction through different communication technologies such as Bluetooth, Wi-Fi, IrDA, RFID and so on that can be applied to provide location awareness.

Some of the reasons this type of systems are difficult to develop are: the distributed nature of the system considering the development of both the client and the server applications. As this is an this information based application, the internationalization process may not be trivial because information is not limited to just to text (multimedia information should be localized, too). Location aware information should be attached to art pieces, rooms, showcases and so on in order to gather information from the system. Although localization in outdoor environments is almost solved with GPS technology; there is not a definitive solution for indoor environments, even more, combination of technologies should be taken into account to succeed. Finally, the system integration with legacy software is one of the most important issues to take into account because these systems are mainly intended to exposition designers, so the implementation of the system should only affect them and not the whole personnel.

The starting point of every project is the definition of a conceptual model that should be able to represent the information contained in this type of scenarios. As a consequence, a conceptual model for Cultural environments is exposed, where information is clearly divided into two well defined domains: the environmental and the catalogue information ones.

While catalogue information contains technical information that is not relevant to cultural environment visitors, but very important to museum stuff (i.e. arrival and maintenance dates, owner, and so on); the environmental information provides cultural environment exposition designers with the ability to adapt this information to visitors. Besides, environmental information provides the system with the ability to indentify objects and use this information to provide visitors with context aware information.

Once the conceptual model is defined, the software system architecture is defined by five components: the Client (visitor device), the Server (information repository), Space Administrator (the ex location designer interface to locate art pieces), the External Database (containing legacy information) and the Synchronizer (that keeps information repository up to date with external database data).

The software architecture supports the location aware system that provides visitors with new ways of interaction according not only to the user position but to the user gestures. This is possible thanks to the multi-technology position support that offers the system. Thus, different ways of interaction according to space granularity and user attention are provided to visitors.

2. Related Work

Studies performed by (Barber, C. et al., 2001) revealed that PDAs are the most suitable device to be used for augmenting museums and art galleries. The study includes prototypes of a guide using HMD (Head Mounted Displays), PDA and Tablet PCs and is focused on efficiency, effectiveness and overall preference. Results showed that the most effective device was the PDA. The first place on efficiency, conducted by an ANOVA, was shared by a PDA and a HMD. And, in the overall selection of devices, the first place was for the PDA.

There are several projects that have tried to use wearable computers in museums. One of the most important references in this subject is the work developed by (Ciavarella, C. & Paternò, F., 2003); (Ciavarella, C. & Paternò, F., 2004) in the Marble Museum of Carrara (Italy). This project is currently running at the museum and it offers a PDA to the visitors with all the information pre-loaded in memory cards. The PDA is able to detect when a visitor is entering in a new room using infrared devices located at the entrance of each room.

Other Museums have already developed projects based on handled devices (Steele B., 2002), as the Field Museum in Chicago [1] -, Herbert F. Johnson Museum of Art of New York [1] - or Kew Gardens outside London [1] -. Most of them use handled devices as a useful tool in the inventory process. All these projects are prototypes, and ordinary visitors do not have access to this information by PDA.

Exhibitions are ideal scenarios for applying augmented reality or mixed reality. There are many HCI groups working in this area. In (Ciolfi, L & Bannon, L., 2002) an interactive museum exhibit is designed using mobile devices or the work performed by (Schiele, B. et al., 2001) where a wearable computer is developed as an alternative to the traditional guides.

A system that is currently being exploited, and turns this vision into reality on art museums is exposed in (Gallud J. et al., 2005) at The Cutlery Museum in Albacete, known as MCA in Spain.

This work has been evaluated by final users. These user evaluations were performed according to the CIF (Common Industry Format for Usability Reports) standard defined by the ISO/IEC DTR 9126-4 (Tesoriero, R. et al., 2007).

According to the evaluation results we have improved both the system model and architecture (Lozano, M., et al., 2007) and the user interface (Gallud, J., et al., 2007).

The final version of the system including all features we describe in this book chapter were published in (Tesoriero, R. et al., 2008).

3. A Conceptual Model for Cultural Environments

One of the first tasks related to the design of mobile software for museums is the definition of the conceptual model.

We think a museum entity is conceptually composed by, at least, two types of information, the Catalogue information and the Environmental information.

Catalogue information is related to museum registry and it is stored according to defined structures and procedures that museums should follow to accomplish international standards (Carretero et al., 1996) (ICOM-CIDOC, 1995). Usually, this information is available in electronic format and is described in technical language. On the other hand, environmental information represents information surrounding a piece instead of the piece itself.

3.1. Catalogue Information

Although environmental information is provided to visitors in general; technical information may be extremely useful for users that are related or simply require more information about pieces in the museum.

The person in charge of managing the art exhibition uses the Space Manager application to define spaces which will host art objects. The concrete art object is managed by means of a legacy software. This software is supposed to be in the museum before introducing the mobile solution based on PDA.

The existence of legacy software introduces the need of a Synchronizer that will maintain the coherence between the external database and the internal one.

The Catalog component is responsible of managing the technical information for an art object with the information provided by the legacy database.

The Contents Provider provides the Client with the information to be shown and all the information needed to keep user the state.

Figure 1 shows the class diagram modelling the entities we have in our system. As we can see, the key entity is the piece. This model has been defined according to national and international recommendations (Carretero et al., 1996) (ICOM-CIDOC, 1995) allowing us to easily deploy the system in different museums.

Figure 1.

Catalogue Model.

3.2. Environmental Information

An art object or piece being exposed is wrapped by extra information that depends on the exposition environment; for example the physical place of a piece. Thus, a central item of our model is the Space. A piece must be exposed into a Space and this piece has a Space associated to itself.

Pieces are usually exposed within a container, for instance a show. A show may represent a showcase or a frame, in the case of a painting, or anything that is able to contain a piece (or a set of them). A Space has a graphical representation, an associated resource and an identifier.

An overview of the conceptual model is depicted in Figure 2.

An interesting thing to point out is the fact that the Identifier entity allows the system to be isolated from the specific technology used to locate the object in the real world (RFID, WI-FI, etc). Besides, an object may be identified by one or more Identifiers. Thus it is possible to use different position and location technologies at the same time to improve the system precision. For instance, you can use RFID to identify a showcase and a barcode to identify a piece into the identified showcase.

Although a museum may have a media repository associated to its catalogue, extra information about pieces should be provided in order to present or adapt the information to visitors.

Media and content exposed to visitors may differ from the museum technical information. This information should be related to museum pieces. Pieces are not the only spaces that may have related information. Often, spaces provide contextual information about the pieces they contain.

Figure 2.

Environmental Information model.

The information described on the previous paragraph is represented by resources. Art objects may be associated to different media resources, for instance; images, audios, videos or texts. Media should be also customized in different languages.

A museum is a cultural environment which may be physically organized in one or more buildings. A building is divided into floors and a floor is divided into rooms. In a room we can find terminals. A terminal represents a device that can be placed into a room. It can be a show or a panel.

Museums usually have information about catalogued pieces. These pieces are exhibited in shows. However, there is some information that is not catalogued; this information is provided to visitors through panels.

Sometimes large shows are divided into regions to organize pieces in groups in order to improve information understanding. Each region is represented by a section that groups pieces that are related in some way. In consequence, pieces may be contextualized according to some defined criteria, providing a context for each group of pieces.

Pieces are physically represented by a physical representation (PieceRepresentation) that relates the physical place of the piece to the piece information itself.

The only linking point between catalogue information and environmental information is established by PieceRepresentation and Piece. Thus, we decouple the piece physical representation (PieceRepresentation) from the piece technical information (Piece). PieceRepresentation acts as an Adapter (Gamma et al., 1993) between a concrete piece and the place it takes in the real world, providing additional information about the environment. This characteristic provides us with the ability to adapt the model to any kind of pieces.

The GraphicRepresentation is the graphical representation of a space. Each space is represented in two ways: Internally and Externally.

The Internal representation is used to show the space itself while the external representation represents the space from the container space point of view.

GraphicRepresentation is a way to decouple the space graphical representation.

4. Software Architecture

In this section we will focus on the definition of the architecture that is based on these elements: positioning, client, server, DB Server and space manager (see Figure 3 on the left). They lead to the definition subsystems and components shown on Figure 3 (on the left) and discussed in the following subsections.

4.1. The Client System

The positioning subsystem is responsible for giving the PDA a specific location according to a reference system. The client is composed by two components: the client GUI and the positioning system.

The Positioning System allows the system to provide users with an unknown experience when they are visiting an art museum. From a technological point of view, we consider the most relevant techniques to solve the automatic positioning system, and up to now, we have not closed the topic with a definitive decision. We have considered using infrared, RFID, WI-FI and mixed approaches.

The variety of positioning systems forced us to define the system separately from the hardware employed. Figure 3 shows how the client is able to interact with the environment in order to know its position in the real building. The client program running on the PDA can receive information from the environment in different ways: infrared sensors, RFID tags, Bluetooth or WI-FI devices or any other system available now or to appear in the future. Then, it sends this information to the server and receives the requested information back from the server.

4.2. The External Database System (External DB)

This database contains legacy information that is loaded through legacy systems (Enterprise Information Systems). This point is really important to take into account when you are deploying the system in existing museums with previous software applications.

Personnel in charge of loading information (mainly catalogue information) are not usually the same in charge of designing the cultural environment exposition; thus, user profiles are not the same.

Figure 3.

Software Architecture.

As the system is designed for exposition designers and visitors instead of cataloguing personnel, the idea of rebuilding the whole information system, for both designers and catalogue personnel, is not welcomed by cataloguing personnel because they have to learn how to interact with a new system without getting any benefit.

The alternative is the integration of the system into an existing architecture that is the responsibility of the Synchronizer, as shown in figure 3.

4.3. The Database Synchronization System (Synchronizer)

The automatic database synchronization subsystem is responsible for maintaining the coherence between both the internal and external databases. The internal database refers to the database that may be previously installed in the museum containing piece catalogue information and eventually a media repository. The external database refers to our specific database to support the positioning subsystem and other requirements.

4.4. The Space Manager

The person in charge of managing the art exhibition uses the Space Manager program to define spaces which will host art objects. The concrete art object is managed by means of legacy software. This software is supposed to reuse the information the museum had before the introduction of the new mobile solution based on PDA.

To cope with information reuse, the system integrates with any legacy software introducing the need of a Synchronizer. The Synchronizer keeps the coherence between the External and Internal databases.

4.5. The Application Server

The Catalogue component is responsible for accessing information stored in the internal database. Relationships among art object information that is not related to environment is solved by this component. The Space administrator relates the museum catalogue information to the environment information. This information is usually structured and organized by the SpaceManager.

The Contents Provider provides the client with the information to be showed. This information is client independent because it is represented in XML format and it can be easily read by most devices. All the information translation and communication is managed by this component too.

5. Location aware information

The positioning element is responsible for giving the PDA a specific location according to a coordinate system allowing visitors to experience location aware information gathering.

From the visitors’ point of view, this new way of interaction has the following advantages against traditional interaction methods:

  1. No specific interaction is explicitly required to access information that can be deduced from the user position;

  2. The information provided to users may be filtered according to user interests avoiding information overload (profiling).

  3. From the museum exhibitor point of view, this system may provide information about visitor’s interests and their personal opinion piece interest.

5.1. Granularity Levels

We have conceptually modelled physical spaces as a hierarchy of virtual spaces in order to consider all approaches and even the mixed ones. A space is composed by spaces; consequently a tree of spaces is created. The root node represents the Cultural Space and the leaves represent Pieces.

As mentioned before, a Cultural Space is composed by Buildings, and each Building is composed by floors, floors contain rooms and rooms may contain Panels, Showcases, Frames, etc. Panels contain information that is not directly related to a piece, a Frame contains a painting and showcases are composed by sections. A section groups a set of pieces that are related among them. This kind of spaces allows expositors to manage big sets of pieces according to the PDA screen size. Finally, PieceRepresentation entities are contained in sections. They represent the physical information about concrete pieces from the Catalogue.

Based on this hierarchy, we faced two problems in order to get location aware information from users. The first one is the user position within indoor spaces. The second one is how to provide the user with a way to identify objects that are not identified by his / her position, for instance a piece. The user position may identify a showcase or a frame; however, it is not possible to identify a piece within a showcase using the same data.

Thus, two granularity levels have been defined: coarse grained and fine grained spaces. Coarse grained spaces are spaces that may be implicitly identified by the user position. On the other hand, fine grained spaces need an explicit action performed by the user to be identified.

Coarse grained spaces may be identified using positioning systems. For example, GPS is a good solution for outdoor location; however it is proved that it is not a good one for indoor location. There are lots of alternatives to achieve indoor position of users, Wi-Fi, IrDA, Active RFID, etc. Each alternative has advantages and disadvantages and provide better or worst solutions according to a specific situation.

On the other hand, fine grained spaces may be identified by other mechanisms, such us barcodes, IrDA, passive RFID, graphical marks, and so on. Figure 4 shows this situation.

As the model supports multiple ways of identifying spaces, it is possible to use different position and location technologies at the same time to improve the system precision. For instance, we could use active RFID tags to identify a showcase and barcodes to identify pieces into showcases.

Figure 4.

Granularity Levels.

5.2. Attention Levels

The concept of Granularity allows the definition of a system that may be used with different levels of attention. Coarse grained spaces will remain on the user periphery and will not take his/her attention until necessary. Based on Calm Technology (Weisser M. & Brown, J. S., 1997) characteristics, the system is able to warn user, without taking his/her whole attention, using a beep for instance; when he/she is near to a coarse grained space of interest. Thus, users can easily perform other activities as talking while they are visiting the museum. If the user is interested in the information the system is warning about, he/she can focus on it. On the other hand, as users can explicitly perform an action, for instance by a gesture, to retrieve information in fine grained spaces.

It is important to note that although some pieces, as sculptures and paintings, can be treated as coarse grained spaces; usually fine grained spaces are accessed through coarse grained spaces.

As a result of this Analysis, three levels of attention have been defined (low, medium and high) according to the degree of attention is required by the user to interact with the application.

The lower level of attention allows us to perceive the periphery without requiring attention from the user. So the user can perform other activities without being interrupted; but, he or she can be aware of other interesting events when they occur. For instance, if we are visiting a gallery and we are talking while walking across a corridor. We may come across an important masterpiece without noticing and we would miss it. However, if we were notified on time, we would not have missed it. To implement the solution, we used active RFID tags to perform this task. When the PDA is near to an interesting point, the user is notified about it and related information is retrieved.

The medium level requires more attention from the user because he or she has to perform an explicit action to retrieve information about a piece or space. For instance, if we are watching a showcase and we want to get some information about a specific piece, then we can put the device near the piece label and retrieve extra information about it. This technique uses passive RFID tags that are detected by the reader at 5 - 8 cm from the tag.

The higher level requires even more attention than its predecessors, and the user should point to an object or label in order to retrieve information about it. Interaction is not as natural as the other alternatives, but it provides a good method when information is not easily reachable because of the distance. Suppose that we would like to retrieve information about an artefact that hangs from the roof. The user may point to the object and retrieve information. To implement this alternative, IR technology is used and the user has to point the mobile device to the object to get the information.

5.3. The Location-Based Positioning and Suggestion Systems

The positioning system used in this project is a centralized relative position system. It means that each device can get its position relative to an object based on server information.

To determine a device position, an identifier is read by the device and is sent to the server. The server looks for the identifier posted by the client and retrieves the associated space. The required space information is retrieved and sent back to the client. This information contains the space associated to the object identifier among other useful information.

As we have mentioned before, a space may be identified by several identifiers, so many positioning systems can be used simultaneously.

Barcodes, passive RFID, active RFID, IrDA, etc., can generate an identifier, so coarse and fine grained spaces can be referenced by the system in order to get the user location.

The positioning system is closely related to maps, thus, in order to add a space into the system using the Space Manager Tool, internal and external representations should be defined.

The internal representation is defined as an image. This image defines the map of the space; it contains information about image resolution and physical size of the space. This data can be used to define the physical / virtual size relation (centimetres by pixel) to define distances between objects. Distances will be used by the suggestion system to get information about near objects. As the PDA screen resolution is not high, long rooms with lots of terminals on them can be difficult to see. A solution to this problem may be a virtual room; a physical room may be divided into several virtual rooms. The representation of virtual rooms may be represented by images of a lower resolution keeping the physical / virtual size relation.

On the other hand, the external representation is defined by a polygon into the internal representation of the parent space, that is to say the space that contains the space from which we want to define the external representation.

The Space Manager Tool also manages the information related to the space description, associated resources and identifiers, and internationalization issues.

The system prototype uses an active RFID reader (shown at the bottom right of Figure 5) to read active RFID tags (shown at top right of Figure 5) for coarse grained space identification. On the other hand, it uses a passive RFID reader (at the bottom left of Figure 5) to read passive RFID tags (at the top left of Figure 5) for fine grained space identification.

Figure 5.

Active RFID Technology.

Figure 6.

Suggested routes based on distances.

Although both systems can be used simultaneously, they cannot work together because of hardware restrictions (only one compact flash port is available on PDAs commercially available up to now).

Finally, a suggestion system based on the user location is provided by the system (see Figure 6). Thus, the system is able to point out in the map of the client pieces that may be interesting for the user to visit based on the physical / virtual relationship established in the space definition.

6. Internationalization

Cultural environments are usually visited by people from all around the world and supporting as many languages and cultures as possible, is a must.

We mainly face two problems:

  1. How to choose a language?

  2. How to organize information and resources in different languages and cultures, while reusing information of a legacy catalogue database?

The first problem is the language selection. This is a very common problem regarding internationalized applications. It is a “base case”like problem, how to interact with something that I cannot understand or communicate?

In this case, we propose a gesture based interface. Graphical language is a universal language. Thus, in order to configure the application the user goes to a language selection panel and brings the device near a flag (that defines the user language and culture) and the device is automatically configured. Figure 7 shows the language selection panel.

Figure 7.

Language selection panel.

The second problem refers to the organization of the information. There are many issues to take into account when organizing related data that is represented in many languages.

Our approach is based on separating textual from media information.

Textual information as a piece description and names are stored in a collection of databases. Each database contains information related to a culture, thus in order to access this information related to a specific language or culture, the right database should be accessed (Figure 8 on the left).

Media information cannot be stored in the same way as textual information due to the following reasons. Media information is usually specific to each particular language (i.e., audio files). However, there are some resources (photos, original audios or some video clips) that are common to all languages.

The replication of resources is not an option because of space and maintenance reasons. Thus, resources are hierarchically organized according to language and culture. For instance, a picture or photo that is common to all languages, and subsequently, to all cultures is placed on the root of the hierarchy. On the other hand, a resource common to all cultures belonging to a language is placed on the first level of the tree, in a language node, being common to all cultures of this specific language. Finally, there are resources that are specific to a culture (for instance, audios); so they are placed on the lower level of the hierarchy (right hand side of Figure 8).

Figure 8.

Internationalization scheme.

The implementation is based on resource naming conventions. Prefixes for resource file names are added according to the language and culture. Resources on the root starts with R_, resources for all cultures in a language LANG starts with LANG_ and finally resources for culture CUL in a language LANG starts with LANG_CUL_. For example, a resource for a specific culture, Spanish for Spain is named ES_ES_xx.

A search order is established in order to have default resources for missing files. If a resource for a specific culture (LANG_CUL_) is not found, then a search for resource language is performed on the parent node (on files LANG_xx) and if it is not found then the default resource (on root, R_xx) is retrieved.

7. Conclusions

This chapter presents a novel approach to improve and enrich the users experience when visiting any kind of cultural environments such as museums or art galleries.

The use of audio guides is widely used in these environments to assist visitors along their visit. In this context, we define a system architecture which includes the use of mobile devices to provide visitors with new possibilities of interaction through different communication technologies such as Bluetooth, Wi-Fi, IrDA, RFID and so on that can be applied to implement location systems.

This type of systems are difficult to develop due to its distributed nature considering the client and server applications involved in the development and the communication mechanisms needed for gathering the correct information and offering it accurately to the user.

Besides, as this is an information based application, the internationalization process is not trivial as the information managed is not limited just to text, but multimedia information is also very important. Location aware information need to be attached to art pieces, rooms, showcases and so on in order to gather information from the system. Although localization in outdoor environments is almost solved with GPS technology; there is not a definitive solution for indoor environments, even more, a combination of technologies have be considered. Another important issue to take into account in the development of this kind of interactive systems is the integration with legacy software which stores important information.

The starting point of this approach is the definition of a conceptual model for cultural environments that represents the information contained in this type of scenarios. The information is divided into two well defined domains: the environmental and the catalogue information ones.

Once the conceptual model is defined, the software system architecture is described defining five components: the Client (visitor’s device -PDA-), the Server (information repository), Space Administrator (the exposition designer interface to locate art pieces), the External Database (containing legacy information) and the Synchronizer (that keeps the information repository up to date with the external database data).

The software architecture supports the location aware system that provides visitors with new ways of interaction according not only to the user position but to the user gestures. This is possible thanks to the multi-technology position support that offers the system. Thus, different ways of interaction according to space granularity and user attention are provided to visitors, improving considerably their experience along their visit.


  • Field Museum in Chicago,
  • Herbert F. Johnson Museum of Art of New York,
  • Kew Gardens outside London,

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Ricardo Tesoriero, Jose A. Gallud, Maria D. Lozano and Victor M. R. Penichet (December 1st 2009). Supporting New Ways of Interaction in Cultural Environments, Human-Computer Interaction, Inaki Maurtua, IntechOpen, DOI: 10.5772/7740. Available from:

chapter statistics

1256total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

Related Content

This Book

Next chapter

Establishing the Hallmarks of a Convincing Chatbot-Human Dialogue

By Jurek Kirakowski, Patrick O’Donnell and Anthony Yiu

Related Book

First chapter

Automated Generation of User Interfaces – A Comparison of Models and Future Prospects

By Helmut Horacek, Roman Popp and David Raneburger

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us