Application of the ergonomics methods to the technologies.
The production paradigm of the European Industry sector has changed in the last few years. If large companies have already adopted
Currently, around 230 000 European Manufacturing enterprises with 20 or more employees provide 34 million people with work. (Manufuture SRA, 2006). New technologies appear the most appropriate approach for reinforcing European SME manufacturing ground. As described in the FuTMoN report (uture of Manufacturing In Europe 2015-2020 – the challenge for sustainable development): “Not only does RTD
With most influencing factors in a constant, and even turbulent state of flux, the next step is to progress towards what can be described as the ‘virtual factory’ of the future. This will require a European platform for digital manufacturing engineering, having the capability to create, maintain and use a dynamic system of networks, in which all the actors contribute and add value to the manufacturing chain, without the constraints of physical co-location or rigid partnerships. New products are required to fill markets gaps and customers’ expectations, and the new production systems are designed accordingly as a collaborative action with the suppliers. The requirements are detected with an immediate communication between producers and clients supported by new technologies. A company has normally delocalised production sites in different countries and communication is essential for maintaining the corporate line coherence and the production plan. New technologies, such as Virtual Reality (VR) and Augmented Reality (AR), permit an immediate communication among different entities with a higher participation grade.
The research in DiFac aims to develop various environments to support SMEs working on a product in the design phase, or creating a new production line without physically moving the machinery, or again training workers in a safe and secure situation.
In recognition of their importance to the success of Virtual Environments (VEs), the framework is based on three main pillars: presence, ergonomic and collaboration. They are composed of both tools and methodologies. This is the base on which the tools for supporting the manufacturing activities are developed.
In order to demonstrate the potential of the structure, three validation scenarios have been implemented as pilot cases. Two of these are directly derived from project industrial partners, PRIMA INDUSTRIE and Pantelis Pashalidis & Sons (PPS), and are focused on product development and training. A third scenario is developed following the requirements of a highly committed Romanian industry named “Compa”. It works in the automotive sector and applies DiFac results for Factory Constructor. Moreover a European industrial group has participated in the project with a validation role. Their involvement was critical in developing a successful DiFac solution. The DiFac toolset supports collaboration among different sites and users during design, prototyping and manufacturing through an interactive environment to support SMEs delivering better quality products and services. The use of such a technology will definitely lead directly to reinforcing competitiveness in the European industry and solving societal problems since it will provide many benefits, as follows.
The DiFac European consortium and project is just part of a wider project with Korea and Switzerland for creating a new effective Digital Factory Environment, through the associated Intelligent Manufacturing System (IMS) project.
This chapter presents the innovation behind the three pillars on which the environments are based, and the different technical solutions that constitute the DiFac integrated scenario.
2. A Digital Factory sustained by three pillars
A Digital Factory is a comprehensive approach of networking of digital models, methods, and tools - including modelling, simulation and 3D/Virtual Reality visualization- integrated by a continuous data management (DiFac brochure, 2007). The DiFac framework identified three activity areas to address- training, product development and factory design/evaluation.. The objectives of these activities were defined by the industrial partners of the project including the tasks of the workers, the designers, and the production department. In collaboration with the industrial and development team of the consortium, the characteristics of the three theoretical pillars of the framework were defined in terms of Ergonomics, Presence and Collaboration.
Therefore the basic framework of DiFac (and the main innovation) was to provide understanding of these three main pillar components, with supporting tools and methods, to help the development of specific components from the three main activities in the product/process life cycle.
DiFac main results are the following:
The following paragraphs contain a definition of the investigated field and the application of the methodologies to the DiFac environments.
Taking proper account of people’s needs and capabilities in design, implementation and operation - being human-centered and addressing the human factors - is the province of Ergonomics. As a discipline it is concerned with the theory and practice of learning about human characteristics and capabilities, then using that understanding to improve people’s interaction with the hardware, software and people with which they interact, and with the environments in which they do so (Wilson 2005). Poor ergonomics (for example failure to account for the users’ end needs; making systems excessively complicated; providing interfaces which do not support the user) has been cited as the reason for failure in many Information and Communications Technology (ICT) projects (Wellington 2006; Beynon-Davies 1999; UK National Audit Office 1999 and UK Public Accounts Committee 2000). In all cases, the abilities, needs and limitations of the people working within these systems or with the equipment have not been understood and accounted for. Conversely, successful products or work systems typically demonstrate that the needs of their users have been addressed during conception, design, implementation and operation (Eason 1997; Wastell and Cooper 1996;).
Within DiFac, the focus throughout the development of the technologies has been on the human factors: that is, ensuring the solutions were usable and satisfying to use. However, consideration was also made of the ergonomics of the factories, products and training, for which DiFac aims to improve the productivity and safety of people at work. For this, the DiFac technologies provide the opportunity for a proactive approach to ergonomics - identifying and resolving ergonomics issues at an early stage in the design process, when changes are less costly to implement, rather than responding to complaints of ill-health in the workplace. This has been achieved by enabling workers to evaluate digital representations of workplaces, thus benefiting from their tacit knowledge of working practices and from the opportunities afforded by visualisation technologies (Chaffin 2001; Laughery 2005 and Lawson et al. 2006).
Concerning the development of the technologies, one of the main research aims was to investigate the suitability of existing ergonomics methods for application to the digital factory. The DiFac project presented many issues typical of any development programme, including: restricted access to the anticipated end-users; limited resources for user-testing and inconsistent availability of prototypes for evaluation work. However, the DiFac project presented some unique challenges affecting the ergonomics evaluation process, including: an emphasis on distributed collaborative working, with the development partners and end-users in several countries with different time-zones, first-languages, cultures and operational stereotypes; a necessity to conduct evaluations remotely due to the geographical dispersion; and an extension beyond typical software and web interfaces to include, for example, virtual and augmented environments.
The general approach to development was a user-centred participatory design and evaluation process. Involving end-users can improve the solution through the benefit of their unique insight into working activities. Moreover, they are the people who will ultimately evaluate the success of the technology. This approach has been proven to lead to quality improvements, and a reduction in throughput time and costs. It not only ensures that the technology will match the end-users’ needs, but also increases their acceptance of the final solution through an increased sense of ownership. The essential elements of this approach include direct involvement of the end-users, a focus on developing the system to match their capabilities, and an iterative development approach (Vink et al., 2005). End users were involved at every possible stage; when time or availability prevented their involvement, their needs and capabilities were considered inviolable.
Hereafter follows a description of the specific methods used to evaluate the various iterations of DiFac technologies. They were selected based on the priority for end-user involvement, in addition to the specific challenges of DiFac mentioned above.
The application of the various methods was dependent upon the state of the prototype, as well as the geographical location of the Ergonomist and end-users. An overview is provided in Table 1. In the early stages, development partners generated videos of proposed solutions, based on pre-existing prototypes. Industrial end-users were asked to watch this, and then complete the user requirements questionnaire. This process was conducted remotely, with the users downloading the video and completing the evaluation from their sites, then emailing their completed questions back to the Ergonomist for analysis. Thereafter, prototype systems became available in which users were able to explore and attempt certain relevant tasks before completing SUS as well as the user requirements questionnaire. Again, this process could be conducted remotely. If interim developments were released they would be subject to an expert review using the UNott heuristics checklist. As more detailed working prototypes became available, co-located evaluation sessions were held in which the DiFac end-users and external companies and associations gathered in one location to evaluate the technologies. The participants would receive a demonstration, and then be invited to attempt tasks. They were then asked to complete SUS and the user requirements questionnaire.
|Technology stage of development||User-requirements questionnaire||SUS||Heuristics checklist|
|Early video clips showing proposed solutions||Y (distributed evaluation)||Y|
|First prototypes||Y (distributed evaluation)||Y (distributed evaluation)||Y|
|Higher fidelity prototypes||Y (co-located evaluation)||Y (co-located evaluation)||Y|
Note: Distributed refers to geographical separation of the participants from the evaluator; with co-located the evaluation was conducted with both in the same room
The overall approach can be considered a success, given the positive response of the end-users and EU reviewers towards the final stages of the DiFac project. The following specific points were noted from the application of the chosen ergonomics methods:
Asking end-users to evaluate video clips of proposed solutions enabled them to provide feedback at a very early stage of the development process. This helped clarify the user-requirements, and also identify any mismatches in the proposed solutions before any development work had occurred; hence changes were easier to manage than had they arisen later in the development process.
Conducting distributed evaluations was particularly useful for reducing travel costs. It also minimised the time SME employees were removed from their work places. Most of the distributed evaluations lasted considerably less than one hour.
The user requirements questionnaire and SUS were useful for identifying significant usability issues and mismatches between system performance and user requirements. However, it was necessary to supplement this process with the UNott heuristic checklist, which enabled a more detailed evaluation of the interface. Through the heuristics checklist the Ergonomist made recommendations for aspects such as navigation, icon design, labelling, window behaviour and movement in the VE; the other methods would only indicate a general problem area and would not generate recommendations of sufficient clarity for a developer to easily respond.
The UNott heuristic checklist was also useful for tracking changes with development iterations – it was not feasible to ask the user-partners for their feedback at every stage of evaluation, due to the demands of their jobs.
Despite the aforementioned benefits of the distributed evaluations, the co-located evaluation was invaluable for the Ergonomist to witness first hand subtleties such as body language, facial expressions, and utterances by the industrial end-users when trying out the technologies. It provided the opportunity for all the participants to ask questions so that direct feedback could be provided and also more detail could be pursued. It was also useful for the developers to witness this rich feedback, rather than summarised feedback from the other methods.
The methods chosen were appropriate for the ergonomics development of technologies in the digital factory. The combination of methods was necessary to ensure that all elements were captured, from general user requirements to detailed user-interface design. Distributed evaluations enabled end-user input at an early stage in the development, and reduced travel costs, but the process also benefited from co-located evaluations when prototypes became available.
The “natural responses of human and environment to each other” (Sheridan 1996) can be considered a definition of the Presence concept, and the most suitable summary of the sense of Presence is the sensation of “being there” (Ijsselsteijn & Riva, 2003) without physically being in a precise physical place. In a Virtual Environment the sensation of being inside the environment depends on different factors. Within DiFac, the factors were analysed according to the activities to be developed: training, product development and factory design.
Inside the perceptual features important are the graphic vividness, the interactivity and control of the entire environment and its elements. Since the sense of Presence has a strong personal aspect, there are many individual factors: the imagination and suspension of disbelief; identification; personal motivations and goals; and emotional state during the precise moment in which the customer uses the environment. The content of the environment and the story are very important for involvement with the environment, and linked to this, are the social and cultural aspects (Riva, 2008). On this basis, we defined the requirements for different project environments. However, the innovation of DiFac is the measurement methodology. Presence can be measured using many modalities that can be divided into two main areas: subjective and objective. The first evaluates the personal experience in utilizing the environment with a questionnaire, the second calculates an objective amount of data while the user experiments with the environment (e.g. heart beating, blood pressure…)
The innovation of the project is the use of a psychological theory called “Flow” by Csikszentmihalyi (1996). Flow is a subjective state that people report when they are completely involved in a task to the point of forgetting time, fatigue, and everything else but the activity itself. "You're right in the work, you lose your sense of time, you're completely enraptured, you're completely caught up in what you are doing [...] when you are working on something and you are working well, you have the feeling that there's no other way of saying what you're saying" (Csikszentmihalyi, 1996). The intense experiential involvement of Flow is responsible for three subjective characteristics commonly reported: the merging of action and awareness, a sense of control and an altered sense of time.
The state of Flow can be experienced in different fields and doing various activities, but it has always these characteristics:
flow tends to occur when the activity one engages in contains a clear set of goals
flow results from a balance between perceived challenges and perceived skills
when perceived challenges and skills are well matched attention is completely absorbed
Having these features in a VE should ensure the success of the work to be done in the synthetic environment (Riva, 2008).
DiFac linked Presence and Optimal Experience detected in “classical” Presence factors to determine the sense of being in the environment, and the Flow state to establish the quality of the experience and consequently understand if the use of the virtual environment was near to the Optimal Experience. During the project time framework a new questionnaire was written by merging questions from the Witmer & Singer Presence Questionnaire and the Flow Questionnaire applying to the Virtual Reality a methodology already consolidated for identifying experience in real world. Furthermore the questionnaire checks the balance between skills and challenges, two main components for reaching the optimal state and the disorientation level.
The questionnaire was scientifically validated as a PhD study in three steps. This first step, was to evaluate a questionnaire initially composed of 135 questions, with a sample of 400 participants (200 male – 200 female) between 18 and 35 years old. Each participant was tested immediately after a set of experiences with various levels of Presence and Flow:
reading a narrative description of a tennis match
watching a television programme of a tennis match
playing a tennis game using a static system console
playing a tennis game using a dynamic system console: Nintendo Wii ™.
At the end of each trial the participant filled in the questionnaire. The collected data was analysed with SPSS software. It revealed the structural relation among the constructs, and reduced the number of questions to 40. The 40 item questionnaire, now called
During the third and final study, a sample of 90 participants (45 male – 45 female) was divided into three groups: 30 for each. The VE representing an imaginary city was presented to each group in three different versions: low/media quality and interaction; middle quality and interaction; and high/media quality and interaction. Each participant navigated around the environment for around 15 minutes. The objective of the third study was to verify at different levels of the manipulate variable (media-interaction) the different scores of Presence and Flow using the Flow for Presence questionnaire. Further test wass the use of a portable mini ECG for measuring some objective data. This last study phase is still on going.
4.2. Example Application: the Factory Constructor Case
The Factory Constructor (FC) is a VE that allows the re-planning of an existing shop floor or the design of a new one. The results were collected in Italy with ten users and in Korea with a further 20. The users answered the questionnaire after 15 minutes trial.
Since the Likert scale used was from 1 to 5, where 1 is the lowest grade and 5 is the highest, 3 is the border between the negative and positive score.
For keeping a high level of interest, skills and challenges should be balanced. Skills of the FC appear a little low because the subjects were people with no professional need to use the VE and therefore they were expected to have a higher level of difficulty as they did not have the basic skills required for the activities. The technician in a real situation faced with a real production line, real machinery, real people and the economic implications of their decisions, can have a high stress factor and carrying out these activities in a VE could reduce this.
The Disorientation level changes depending on the type of VE. Whenever the environment is immersive, the disorientation level can depend on the hardware (helmet, gloves, glasses…) and personal vestibule system sensitivity. The disorientation value for the FC is 1,32. The low score is mostly because the test was made on PC (both desk- and lap-top PC) and not using any immersive hardware. Finally the Coping is strictly linked to the ability to manage unforeseen situations. This value indicates the subject’s ability to find the solution in front of unexpected situations or problems. This can be useful for evaluating the interface and the level of the friendly use of it. The Coping rate was 3,39, indicating that the instruction section was easy and generally intuitive.
4.3. Conclusions and further development
Collaboration has been defined as “the process of shared creation: two or more individuals with complementary skills interacting to create a shared understanding that none had previously possessed or could have come to on their own” (Schrage, 1990). Through collaborative working practices, a shared meaning can be created about a process, product or event. Collaboration can be co-located, when individuals work together in the same physical space and are able to communicate directly, or remote where individuals are geographically distributed and hence face-to-face meetings may not be possible. A further distinction is one of time: synchronous collaboration is when information is exchanged immediately (e.g. face to face meeting, instant messaging); asynchronous involves collaborative working but at different times (e.g. email, voicemail). Computer mediated communication tools can be used to support coordination and collaboration between geographically dispersed members of a team to coordinate their individual efforts and inputs (Saikayasit, 2008).
An aim of DiFac was to support the future manufacturing plant in a distributed, collaborative working environment, in recognition of the importance of this type of work in today’s business climate (DiFac consortium, 2006). To achieve this, seamless collaboration is required amongst workers, machines, suppliers and customers, regardless of their physical location. The tools were required to support distributed design and manufacture, information sharing, decision making and project management, all tasks which require effective collaboration. The project provided an ideal opportunity to test a new questionnaire developed at the University of Nottingham for evaluating the collaboration aspects of new technologies.
The collaboration questionnaire applied to DiFac was developed at the University of Nottingham in recognition of the lack of existing, validated methods for evaluating collaborative features on emergent technologies (Saikayasit, 2008). It was developed for evaluating the collaborative features provided by tools, and for evaluating the extent of the collaboration these features support. The questionnaire was created such that it could be applied to any collaborative system, yet provide a score on the same scale. It was initially developed through literature and focus groupswith Human Factors experts who were asked to list 10 of the following:
key functions which collaborative technologies should support
factors which may result in effective collaboration
factors which may result in ineffective collaboration
They were then asked to rank the factors from most important to least important. The results of this study were used to formulate an 11-item questionnaire, which can be used to evaluate collaboration. An extract from the questionnaire is shown in Figure 3.
The questionnaire was administered in a user-trial of the DiFac technologies. Initially, key features of each technology were demonstrated by a DiFac representative. Thereafter, the participants were given time to explore the systems before being asked to attempt a list of tasks which were typical of the anticipated end-use of the given technology. These included collaborative tasks, where appropriate. Thereafter, the users were asked to complete the collaboration questionnaire for each of the tools.
The results focus on the utility of the collaboration questionnaire for evaluating new technologies, rather than an evaluation of the collaborative features and support for these offered by the tools.
Firstly, the administration of the questionnaire was simple. It was printed onto 1 A4 sheet, and took less than 5 minutes to complete per technology. The 5-point scale was easy to understand, and complete.
Analysis of the ratings was also straightforward. Median ratings were used for each question. A graph of these can be used to rapidly identify problem areas. For example, Figure 4 shows the results from an early evaluation of the iPortal. It is clear that, in particular, questions 6 and 7 (relating to awareness of others’ actions and indicating a location on a shared file) require improvement.
Unfortunately, one potential issue arises as a result of the simplicity of the questionnaire. The evaluation criteria (and questions) are the same for all technologies. However, the type of collaboration required for an expert and novice emergency training simulator is different to that for a product design tool. In the DiFac application, this led to some confusion as some of the tools received a poor score for one or more aspects of collaboration, but in reality the Human Factors specialists recognised that these were not necessary for the type of collaboration required by the tool. Therefore, the effects of this problem can be mitigated through expert analysis of the results.
A problem experienced as part of the evaluation, but not strictly related to the questionnaire, was that quite often the collaboration features did not work on the prototype technologies when tested away from the development partners’ sites. This was partly due to different operating systems, and as a result it was difficult to explore fully the collaborative nature of the system..
The collaboration questionnaire (Saikayasit, 2008) was a useful tool in the development of the DiFac technologies. It was quick to administer, easy for participants to understand, yet enabled simple identification of the aspects of collaboration receiving a poor score. It was found to be improved through expert input, in particular if an aspect of collaboration which was poorly rated was not necessary for the application. Future research could enhance this aspect of the tool, for example by clarifying the types of collaborative tasks for which the questionnaire is most suitable.
6. DiFac Integrated Scenario
The following section presents a concrete scenario to illustrate the use of the DiFac toolset. It concerns the design and production of a new customised laser cutting machine.
The following hypothetic companies are the participants of the Integrated Scenario, although they mirror the real enterprise partners of the DiFac consortium:
SECONDA company which is a laser machine producer (in the consortium PRIMA INDUSTRIE is an Italian producer of 2D and 3D laser cutting machines)
PACOM company, a customer interested in purchasing a cutting laser cell (COMPA is a Romanian automotive industry)
A new IT company providing the DiFac Solution. This is composed of the R&D DiFac partners, who worked on the different solutions, headed by ROPARDO, who developed DiFac iPortal.
Here follows the story of the scenario. PACOM orders a new laser cell from SECONDA. The order is composed of a PLATINO laser cutting machine which has to be customized in order to meet special customer’s needs.
The SECONDA general manger, taking advantage of collaborative, presence and ergonomics features provided by the DiFac solution, creates a new internal project whose main goal is to design and assemble the new laser cell named “
When the design phase starts, two designers in different locations work on the design of the laser machinery in a collaborative way in real time. They develop two different possibilities, which are presented to the General Manager. To help decide which design to progress, the designer starts an evaluation session using the DiFac decision support system. This includes evaluation criteria including for example, weight, benefits and costs. The team members (General Manager, Assembly Engineer, Designer and Factory Planner) rate the design against these criteria, to help support the decision-making process. When the selected design is sent to the production sector, there are implications for the production process due to the modified components. To better assess these implications, the process is analysed with Factory Constructor. The new shop floor can be visualized in an intuitive and user-friendly way thanks to the Presence evaluation done during its development. The workers can verify the effective ergonomics of their own workstation and give back some suggestions to the Factory Planner. The proposed new layout passes through a web-based simulation tool that detects a bottle-neck in the production line. This is used to identify a solution, implemented by the Factory Planner.
Since the shop floor has been re-planned, it is very important that workers are retrained to manage emergency situations. This is achieved through a simulation environment, in which workers can learn emergency response procedures.
The virtual prototype design tool allows the project manager to present the new machine to the customer who can make some changes in collaboration with the project designer. When the machine has been settled at PACOM, the DiFac system provides the maintenance procedures. The customized maintenance procedures, modified for the personalized Platino machine, can be uploaded on the iPortal to provide an efficient remote maintenance service. A technician is available for specific remote maintenance services using the Augmented Reality techniques.
The integrated scenario has been written as a test bed for DiFac solutions. It demonstrates the following aspects:
The DiFac solution supports group work in an immersive and interactive way, for concurrent product design, factory design and optimisation as well as worker training in dangerous moments or for machine maintenance.
The various partners’ results are integrated into a comprehensive solution: changes in the data handled by a component are reflected in other components and data can be exchanged between the software tools where it is needed.
The outcomes of the 3 pillars (i.e. Presence, Collaboration and Ergonomics) have increased the tools’ functionalities, composing the DiFac solution, from the key pillars of collaboration, presence and ergonomics
DiFac solution is modular and scalable
The DiFac project has concentrated its attention on the realisation of the “digital factory” tools taking into consideration human aspects. This paper has presented the three main pillars above which the framework is built: ergonomics, presence and collaboration are considered to be the foundation for human oriented production systems. In building up the environments for training, product development or production design the three pillars were analysed and specific methodologies and guidelines developed.
The first application of the presented methodologies gave a positive score in the evaluation of the environments. The validation process was also conducted in order to collect external industrial’s evaluations.
The complete solution will be soon available trough the DiFac iPortal, where the users can use the project pillars as integrated part of the Software solutions for better customization of their results.
Beynon-Davies P. 1999Human error and information systems failure: The case of the London ambulance service computer-aided dispatch system project. Interacting with Computers, 11(6), 699 720.
Brooke,1996. SUS: A Quick and Dirty Usability Scale. In: Usability Evaluation in Industry, Jordan, P.W., Thomas, B., Weerdmeester, B.A, McClelland, I.L. (Eds), 189 194, Taylor and Francis, 978-0-74840-460-5London
Chaffin D. B. 2001Digital human modelling for vehicle and workplace design. Society for Automotive Engineers, 978-0-76800-687-2Warrendale, PA
Chryssolouris G. Pappas M. Karabatsou V. Mavrikios D. Alexopoulos K. 978-1-84628-801-2, Li W.D., Ong S.K., Nee A.Y.C. and McMahon C. (Eds.), Springer Series in Advanced Manufacturing, Springer-Verlag, London (April 2007
Consoni Florenzano. Souza M. Sacco M. Vieira Porto A. J. 2006Virtual manufacturing as a way for the Factory of the Future. Special issue on E-Manufacturing and web-based technology for intelligent manufacturing and networked enterprise interoperability, Journal of Intelligent Manufacturing, 17/6, Springer Verlag, 0956-5515.
Csikszentmihalyi M. 1996Creativity: Flow and the psychology of discovery and invention. New York: Harper Collins
Csikszentmihalyi M. 1977Boredom and Anxiety, second printing. San Francisco Jossey-Bass
Csikszentmihalyi M. Le Fevre J. 1989Optimal Experience in Work and Leisure, Journal of Personality and Social Psychology, 56 (5), 815-822.
DiFac consortium 2007DiFac project brochure,
DiFac consortium 2006Annex I- "Description of Work". IST 5 035079.
Eason K. D. 1997Understanding the organizational ramifications of implementing information technology systems. In: Handbook of Human-Computer Interaction, M. Helander, T. Landauer, P. Prabhu (Eds), 1475 1495(2nd ed), Elsevier, 0-44481-876-6
EC 2006Manufuture: Strategic Research Agenda, report of the high-level group,
EC 2006Future of Manufacturing In Europe 2015-2020- the challenge for sustainable development
Hoffman D. L. Kalsbeek W. D. Novak T. P. 1996Internet and Web Use in the United States: Baselines for Commercial Development, Special Section on Internet in the Home, Communications of the ACM, 39 (December), 36 46.
Ijsselsteijn W. Riva G. 2003Being There: The experience of presence in mediated environments, Being There: Concepts, effects and measurement of user presence in synthetic environments, G. Riva, F. Davide, W.A IJsselsteijn (Eds.) Ios Press, Amsterdam, The Netherlands
Laughery R. 2005Simulation and modelling for analysing human performance. In: Evaluation of Human Work, Wilson, J. R. and Corlett, N. (Eds.), Taylor and Francis, 219 238, 978-0-41526-757-1London.
Lawson G. D’cruz M. Wilson J. R. Smith S. Bourguignon D. Pentenrieder K. Pavlopoulos C. 2006D3 Ergonomic requirements for human safety and productivity. DiFac IST 5 035079. Nottingham: The University of Nottingham.
Mancini F. Viganò G. Liao Z. Sacco M. Boër C. R. 2004The Virtual Factory: A Semi-immersive Interactive 3D Environment. SCSC 2004, Summer Computer Simulation Conference, San Jose, CA, USA.
Pappas M. Karabatsou V. Mavrikios D. Chryssolouris G. 2006"Development of a web-based collaboration platform for manufacturing product and process design evaluation using virtual reality techniques", International Journal of Computer Integrated Manufacturing, ( 19 8Dec. 2006), 805 814.
Riva G. 2008Psicologia dei nuovi media, il Mulino ed.,
Saikayasit R. 2008Supporting distributed collaboration. Internal report, University of Nottingham, Nottingham.
Schrage M. 1990Shared Minds: The New Technologies of Collaboration, Random House, 978-0-39456-587-3New York
Sheridan T. B. 1996Further Musings on the Psychophysics of Presence. Presence: Teleoperators and Virtual Environments 5 2 241 246
Stanton N. Salmon P. Walker G. Baber C. Jenkins D. 2005Human Factors Methods: A Practical Guide for Engineering and Design. Ashagate Publishing Ltd., 978-0-75464-661-7Hampshire, England
Tromp J. G. Nichols S. 2003VIEW-IT: A VR/CAD Inspection Tool for use in Industry. Proceedings of 10th International Conference on Human-Computer Interaction, Crete, Greece, June 2003, Lawrence Erlbaum, London.
UK National Audit Office, 1999The Passport Delays of Summer 1999. Report by the Comptroller and Auditor General, 27th October. London: The Stationery Office.
UK Public Accounts Committee, 2000Improving the Delivery of Government IT Projects. House of Commons, London.
Vink P. Nichols S. Davies R. C. 2005Participatory Ergonomics and Comfort. In: Comfort and Design, P.Vink (Ed), 41 54, CRC Press, 0-84932-830-6
Wastell D. G. Cooper C. L. 1996Stress and technological innovation: A comparative study of design practices and implementation strategies. European Journal of Work and Organizational Psychology, 5, 377 397.
Wellington S. B. 2006Poor requirements-defintion equals ICT failure. Computerworld, 9 November, 2006
Wilson J. R. 2005Methods in the understanding of human factors, In: Evaluation of Human Work, Wilson, J. R. and Corlett, N. (Eds.), Taylor and Francis, 1 31, 978-0-41526-757-1London.
Witmer B. G. Singer M. J. 1998Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments, 7 225 240