Measurement Instruments for Ergonomics Surveys - Methodological Guidelines

Ergonomic surveys are very important tools to evaluate and identify problems in workplaces such as industries, hospitals, and laboratories. Strategies to tackle the ergonomic issues can be proposed based upon the results of the surveys. Therefore, the surveys should be carefully prepared to obtain information in a clear and reliable way. Usually, ergonomic surveys rely upon measurement instruments (questionnaires) that are applied to workers on the workplace to collect the necessary information.


Introduction
Ergonomic surveys are very important tools to evaluate and identify problems in workplaces such as industries, hospitals, and laboratories. Strategies to tackle the ergonomic issues can be proposed based upon the results of the surveys. Therefore, the surveys should be carefully prepared to obtain information in a clear and reliable way. Usually, ergonomic surveys rely upon measurement instruments (questionnaires) that are applied to workers on the workplace to collect the necessary information.
In this chapter, we present a description of methodological guidelines used to prepare a new questionnaire or to adapt an already developed one.
The first step in developing a questionnaire is to clearly define the questions (construct) you want to answer with the ergonomic survey (Snyder et al., 2007). Based upon those questions, careful searching for questionnaires that have already been used to similar cases should be done. Having found questionnaires that measure exactly what you want, further analysis should be carried out about the questionnaire language and the sample which it was applied.
With the growth of the number of questionnaires developed for a specific culture, their use in other countries, cultures, and languages has become an important tool with the crosscultural adaptation process (Beaton et al., 2002). Minor changes in the original questionnaire can be done to better adapt it to your purposes.
So, how to decide if it is better to use an existing questionnaire or to create a new one?
There are some advantages in using existing questionnaires: time saving in developing a questionnaire based upon steps suggested in literature; possible comparisons with previous studies involving the same questionnaire; psychometrical properties analysis in different situations; and no necessity to develop the administration and analysis processes.
Sometimes it is necessary to change some specific terms in existing questionnaires to fulfill all the requirements of the intended construct. In those cases, a content validity process should be carried out to check whether the proposed changes are misunderstood (Wynd et al., 2003).
On the other hand, when no questionnaires are found to measure the intended construct, new questionnaires can be developed. In those cases, there are steps recommended by the scientific community that guide the development of the questionnaire, such as items selection, domains development, and evaluation of the psychometric properties (Lynn, 1986;Streiner & Norman, 1995;Polit & Hungler, 1995;Turner et al., 2007;Snyder et al., 2007). In general, developing a new questionnaire is a long, laborious process. Therefore, a new questionnaire should be developed only if there are no other questionnaires for the same construct.

The importance of a cross-cultural adaptation process of a questionnaire if the original one was developed to be used in another language/country
Having decided to use an existing questionnaire developed in another language, it is important to carry out a cross-cultural adaptation. This adaptation allows one to apply the questionnaire for a different culture and/or tongue, and to compare results among different countries.
The term cross-cultural adaptation has been used to indicate the process that takes into account the two languages (original and adapted) and the cultural adaptation during the development of a new questionnaire to be used in different context (Beaton et al., 2002).
The cross-cultural adaptation process should follow established rules because the adaptation of a questionnaire to be used in another country, culture, or tongue needs a method to keep equivalence between the original and the adapted questionnaires (Beaton et al., 2002). The questionnaire items should be well translated and be culturally adapted to keep the validity of the instrument (Beaton et al., 2000). Guillemin (1995) have pointed out that measuring in different locations in equivalent ways is prerequisite in order to compare results from different cultures.
Before proceeding with the cross-cultural adaptation process it is necessary to request authorization for the authors of the original questionnaire regarding the use and adaptation of their instrument. The process is then completed after the following steps have been fulfilled: a. Translation: this is the first step where two independent translations are recommended of the original language to the target one used in the current survey. The translations should be done by bilingual translators where the mother tongue should be the target tongue. Only one of the translators should present previous experience about the theme of the survey and may be informed about the aspects to be investigated by the survey. b. Synthesis: The second step is when the two translators and the principal investigator (or a third translator) analyze and compare differences between the translations in order to synthesize the results and obtain a single, definitive version of the adapted questionnaire (Beaton et al., 2000). c. Back-translation: the synthesized version should be translated back to the original language by two translators that have not participated on the first step. Their mother tongue should be the one of the original questionnaire and should not be informed about concepts to be explored within the instrument. These translators do the translations independently, without previous knowledge of the original questionnaire. d. Content evaluation: After the first three steps, an expert committee is organized to evaluate the content of the questionnaire. This committee is composed by bilingual www.intechopen.com professionals with large experience on the topics covered by the questionnaire. The professionals receive the translations, the synthesis, the back-translations, and instructions about how to carry out the evaluation of the questionnaire content. After a detailed analysis, the professionals produce a pre-final version of the adapted questionnaire. e. Pre-test: With the pre-final version of the questionnaire, a pre-testing is carried out in a sample of typically 40 subjects (Beaton et al., 2000). Each of the subjects fill the questionnaire and is interviewed about the understanding of the items, words, and easiness of the filling the questionnaire. During this step, the subjects can point out difficulties and suggest modifications to improve the instrument. If the suggested changes are significant and extensive, another analysis of the expert committee is necessary. At the end of this step a final version of the adapted questionnaire is obtained.
Researchers are following these steps when performing a cross-cultural adaptation process (Vigatto et al., 2007;Gallasch et al., 2007;Toledo et al., 2008; and it is possible to verify that they used carefully methods in order to conduct the process in a reliable way.
It is important to note that, often, one can find in the literature questionnaires that measure the construct to be evaluated with good psychometric properties. After permission of the original authors of the questionnaire, it is possible to use it without making modifications if the recommendations presented in the instrument are followed.
However, one must be careful when using an instrument ever built. When it was created, were the psychometric properties evaluated with the same population you intend to study?
If the answer to this question is "yes", you can use the questionnaire with greater tranquility, but you must verify whether the cultural context and the situation are similar to yours.
If the answer is "no", you should evaluate the psychometric properties of this questionnaire to the other population. This probably can occur when you choose to use a questionnaire to assess a construct in a generic form, i.e., when it is not designed to a specific population. An example of this situation is the study conducted by Shimabukuro et al. (2011), which aimed to adapt a generic questionnaire that evaluates the workers' perception regarding job factors that can contribute to musculoskeletal symptoms to physical therapists. The authors made some changes in the questionnaire's content and evaluated the psychometric properties with the specific population.
And why is it important to check these properties again? It is simple. Applying a questionnaire to a population different from that involved in the study during its development process, one can find different results (better or worse) than the original. Therefore, such assessment can demonstrate if the questionnaire is also reliable and valid for the other population.

A description of all steps for developing a new measuring instrument
When a new questionnaire is necessary, researchers should follow standard and systematic methods that aim to improve the quality of measuring instruments (Haynes et al., 1995;Keszei et al., 2010;Pittman & Bakas, 2010).
The following steps are suggested: definition of the conceptual structure; definition of the target population and the objectives of the instrument; development of the domains and selection of the items; organization of the instrument; evaluation of the content validity and pre-test; and finally the evaluation of the psychometric properties.
a. Definition of the conceptual structure: This step aims to help an initial development of the items and domains. Some methods can be used in this stage such as literature search, interviews with specialists in the field and/or with subjects of the target population, focus groups, other questionnaires analysis, and meetings with a referee committe (Benson & Clark, 1982;Berk, 1990;Turner et al., 2007). b. Definition of the target population and the objectives of the instrument: It is important to characterize the target population in order to justify the relevance of a specific questionnaire (Turner et al., 2007). It is also fundamental to establish a link between the concepts involved and the development of the questionnaire (Fagarasanu & Kumar, 2002). c. Development of the domains and selection of the items: The domains to be investigated with the questionnaire are listed based on the relevance of the proposed survey (Snyder et al., 2007). The selection of the items of the questionnaire can be obtained through literature search and interviews with subjects of the target population and specialists in the field (Streiner & Norman, 2002;Turner et al., 2007). The literature search should be carried out in databases, looking for related constructs and questionnaires in order to determine reference constructs. The interviews with the target population aim to determine individual perceptions about the involved aspects and provide important preliminary data during the development of the questionnaire. The interviews with specialists allow to verify the content to be explored with the questionnaire. d. Organization of the instrument: At this step, the items are organized in their respective domains and a final form for the questionnaire is prepared which includes title, instructions, and response scale. The response scale type and scores are determined based upon the easiness for understanding and answering by the subjects, and evaluating by the researchers (Turner et al., 2007). e. Evaluation of the content validity: This is an essential step in the development of a new questionnaire. It allows associating abstract concepts with measurable and observable quantities (Kirshner & Guyatt, 1985). Details of this step will be provided in section 8 of this chapter. f. Pre-test: The pretest should be applied in a sample of the population in order to verify the understanding of the new questionnaire. After the administration of the questionnaire, the investigator should interview each subject individually and ask him/her about the understanding of words and items as well as about the procedures of filling in their answers. Modifications can be made according to the suggestions of these subjects. When the changes are significant, it is important to be evaluated and approved again by the expert committee that carried out the content validity. After this phase, the measuring instrument is completed and its psychometric properties can be studied. g. Evaluation of the psychometric properties: The evaluation of the psychometric properties of a new questionnaire is one of the most important steps because it allows verifying the validity and reliability of the instrument t o b e u s e d i n o t h e r r e s e a r c h a n d / o r ergonomics practices. When we create a questionnaire, we intend to disclose it to the scientific community. If the questionnaire shows good psychometric properties, it can be widely used by other researchers. Therefore, its use can be widespread whether it is well constructed and evaluated.
The techniques to verify the psychometric properties will be explained in sections 8 and 9 of this chapter.
It can be noted that recent studies involving the development of new questionnaires are following these steps (Farias et al., 2008;Buysse et al., 2010;Bergman et al., 2011;Giesler et al., 2011;Marant et al., 2011;Young et al., 2011). These studies showed the steps of literature review on the topic being discussed and literature review on other scales that could be used for the same purpose. Furthermore, some researchers consulted experts with experience on the area of interest during the selection of domains and items (Farias et al., 2008;Bergman et al., 2011); others conducted focus groups and semi-structured interviews to obtain relevant information for the generation of items (Buysse et al., 2010;Young et al., 2011); and others have conducted interviews with a sample of the target population in order to obtain important suggestions during the developing of the conceptual model of the questionnaire (Giesler et al., 2011;Marant et al., 2011).

Content validity -How to do and how to evaluate this validity using qualitative and quantitative methods?
There are controversies about the terminology and the concept of content validity (Sireci, 1998;Haynes et al., 1995). For some authors, content validity is associated in determining in which fraction the selected items represent appropriately the important aspects of the concept to be evaluated (Contandriopoulos et al., 1999). It aims to verify the extension of the items that determine the same content (Rubio et al., 2003). For other authors, the content validity is an answer for the following question: Are the items of the questionnaire representative among all the questions that can be formulated about the topic in analysis? (Polit & Hungler, 1995).
Another way to define the content validity is the process to evaluate the degree of relevance and representativeness of each element of the questionnaire with respect to a specific construct (Haynes et al., 1995). The elements of the questionnaire include the items, instructions, and format of the answers because all of them can influence the data collection.
For some authors the content validity comprises only the evaluation by an expert committee (Dempsey & Dempsey, 1996;Fitzner, 2007). However, the content validity has been described as judgment process composed by two distinct parts: (i) the development of the questionnaire, and (ii) its evaluation by an expert committee (Lynn, 1986;Polit & Beck, 2006).
The number and qualification of the judges of the committee is controversial. Lynn (1986) suggests a number between 5 and 10 whereas Haynes et al. (1995) suggest a number between 6 and 20 with groups of at least 3 individuals of each field. Other aspects such as the characteristics of the questionnaire, formation, qualification and availability of the judges can be taken into account (Lynn, 1986;Grant & Davis, 1997).
Different criteria can be used to select the group of specialists such as clinical experience, research and publication on the field, expertise on the involved conceptual structure, and methodological knowledge about development of questionnaires and scales (Berk, 1990; www.intechopen.com Grant & Davis, 1997). It is also suggested the participation of lay persons related to the target population of the questionnaire (Tilden et al., 1990;Rubio et al., 2003).
In cases involving cross-cultural adaptation, a multidisciplinary committee is suggested (Hutchinson et al., 1996). In this case, the committee would be formed by bilingual specialists that know the concepts and measures involved (Guillemin et al., 1993).
The evaluation by the judges can involve both quantitative and qualitative procedures (Tilden et al., 1990;Burns & Grove, 1997;Hyrkäs et al., 2003). The process begins with an invitation of the judges that receive instructions and a specific questionnaire for the evaluation (Grant & Davis, 1997). A letter of invitation should explain the reason because the specialist was chosen, the relevance of the involved concepts, and overall explanation of the questionnaire (Lynn, 1986;Grant & Davis, 1997), including the aim of the survey, the scales used, and the adopted score (Davis, 1992;Rubio et al., 2003). The letter can also include conceptual and theoretical foundations from the questionnaire (Davis, 1992) and information about the target population.
If lay persons will compose the committee, a description of the educational level of the members can be specified in the letter (Rubio et al., 2003).
Initially, the judges should analyze the coverage of the questionnaire, i.e., if each domain has been covered by the selected set of items (Tilden et al., 1990). In this stage, the committee can include or remove items of the questionnaire (Rubio et al., 2003). Then, a detailed analysis of the items is performed individually. The committee should evaluate the clarity on the writing of each item to guarantee that each item is not misunderstood (Grant & Davis, 1997). The committee also should analyze if the number of items are adequate and relevant to reach the aims of the survey (Grant & Davis, 1997;McGilton, 2003). Suggestions by the judges to improve specific items can be done at this stage (Tilden et al., 1990;Rubio et al., 2003).
The dynamics of the evaluation process by the judges can occur either individually by each judge followed by a group discussion or interactively through interviews and discussions about the controversial points (Grant & Davis, 1997).
To quantify the level of agreement among the specialists during the evaluation of the content validity, different methods can be used: a. Percent agreement score: The agreement between the specialists (in percentage) is quantified by the ratio of the number of specialists that agree with each other and the total number of specialists (Tilden et al., 1990;Hulley et al., 2003). This is the simplest method to determine the level of agreement (Topf, 1986) and has been used on the initial determination of the items (Tilden et al., 1990;Grant & Davis, 1997). The simplicity in the calculation is an advantage of this method. However, some limitations forbid the use of this method in all cases (Topf, 1986). This methods should be used considering an agreement of 90% among the specialists (Topf, 1986;Polit & Beck, 2006). b. Content validity index: This method quantifies the proportion of judges that agree about some specific aspect of the questionnaire and its items. It is used commonly on the health field (Wynd & Schaefer, 2002;Hyrkäs et al., 2003;McGilton, 2003).
The method allows analyzing each item individually and also the questionnaire as a whole through the use of a Likert-like scale with score from 1 to 4. The numbers express the level of changes/understanding the judge had about the item. For example, the following definitions can be applied: (i) 1 = not representative, 2 = needs major revision to become representative, 3 = needs minor revision to become representative, 4 = representative (Lynn, 1986;Rubio et al., 2003), or (ii) 1 = not clear, 2 = unclear without item revision, 3 = clear but needs minor modifications, 4 = very clear (Hyrkäs et al., 2003;Wynd et al., 2003;DeVon et al., 2007).
The content validity index for each item of the questionnaire is then calculated by the ratio of the number of answer with scores "3" and "4" and the total number of answers (Grant & Davis, 1997;Wynd et al., 2003). Items with score "1" and "2" should be revised or even removed.
To evaluate the questionnaire as a whole, different ways can be used. For instance, Polit and Beck (2006) presented three ways: (i) use of the average of the proportions of the items considered by the specialists; (ii) use of the sum of all indexes calculated separately divided by total number of items analyzed; and (iii) use of the ratio of the total number of items considered as relevant by the specialists and the total number of items.
It is also important to define acceptable agreement rate. Some authors consider the number of specialists on the evaluation of the individual items. When the number of specialists is less than 5, all should agree (rate equal to 1) for an item to be considered as relevant. For a number of 6 or more specialists, the rate should not be less than 0.78 (Lynn, 1986;Polit & Beck, 2006). Some authors suggest a minimal rate of 0.80 to check the validity of new instruments (Davis, 1992;Grant & Davis, 1997) however the recommended rates should be larger than 0.90 (Polit & Beck, 2006).
c. Kappa coefficient: The kappa coefficient is the ratio of the proportion of the number of specialists that agreed and the maximum proportion that the specialist could agree (Hulley et al., 2003, Siegel & Castellan, 2006. It is useful when the data are divided in categories and represented nominally (Siegel & Castellan, 2006). The values of kappa are in the range of -1 (no agreement) to 1 (total agreement) (Hulley et al., 2003).

The reliability assessment: Importance and procedures to evaluate it in a new questionnaire
Reliability is the ability to consistently reproduce a result in time and space, or using different observers (Contandriopoulos, 1999). It indicates aspects about the questionnaire coherence, precision, stability, equivalence, and homogeneity (Lobiondo & Haber, 2001).
It can be evaluated by three different methods: the stability (test-retest), the homogeneity, and the equivalence (inter-observer).
The stability aims to analyze the consistency of the instrument when repeating the measures using a test-retest design (Polit & Hungler, 1995). When you decide to use this method, the situation that is being measured must be in the same conditions in both test, and some differences between tests must be due to random errors (Burns & Grove, 1997).
The homogeneity or internal consistency can be evaluated to verify whether all items of a questionnaire are related to different aspects of the same construct (Streiner & Norman, 1995). Using this method, you can verify if the questions of the instrument measure the same concept (Lobiondo & Haber, 2001).
The equivalence reliability is an inter-observer measure and it allows verifying whether the administration of a specific instrument by two different persons will provide the same results. www.intechopen.com

Validity concepts: Types, importance, and procedures to evaluate the psychometric properties
Validity is an important psychometric property used to evaluate the quality of an instrument (Polit & Hungler, 1995). It is related to the fact that a questionnaire should really measure what is intended to, i.e., the validity can show if the questionnaire represents the concept that it is trying to measure (Lobiondo & Haber, 2001).
Content validity was already defined in section 8 and it aims to analyze whether the questionnaire items are relevant to measure the proposed content.
Criterion validity is used when there is a "gold standard" questionnaire to compare with your questionnaire. This method indicates whether the results obtained with the target questionnaire corresponds to the results obtained with another observation/instrument that measures the same content of interest (Guillemin, 1995).
Construct validity is one of the most important characteristics of an instrument because it evaluates how much the instrument measures the construct of interest. It involves the generation of a hypothetical model to describe the constructs to be assessed and to determine their relationships (Fayers & Machin, 2000).
This type of validity covers a variety of techniques aimed, therefore, to assess whether the theoretical construct appears to be an appropriate model and whether the measuring instrument corresponds to the construct.
Factor analysis is one of the most important and powerful methods to establish the construct validity (Fayers & Machin, 2000). This type of analysis allows us to establish whether there is strong correlation between variables within the same group, but weak correlations between variables from outside the group (Fayers & Machin, 2000).
The factor analysis can be exploratory when you are developing a new questionnaire and there is no prior knowledge of the structure to be used, i.e., it creates a structure for the instrument (Fayers & Machin, 2000). It can also be confirmatory when the goal is to test whether the correlations correspond to the predefined structure of the questionnaire, confirming the number of items previously developed as well grouping the items into factors or domains (Fayers & Machin, 2000).
There is also the construct validity that uses the known-group technique, which consists of looking for different results when applying a questionnaire to groups with contrasting characteristics (Polit & Hungler, 1995;Dempsey & Dempsey, 2000).
The convergent validity is also another technique to verify the construct validity. It consists in showing that a dimension of the new instrument correlates with other dimensions of questionnaires theoretically related (Fayers & Machin, 2000). In contrast, the divergent validity assesses the questionnaire domains correlating them to other domains of questionnaires which content should not be related to the investigation.
Depending upon the type of the questionnaire, we should choose different techniques to evaluate the reliability and validity. This choice should be based on the availability of key technical aspects for each type of technique. For example, if you are developing a new measuring instrument which construct was not measured by any other questionnaire, probably you will not be possible to perform criterion validity.

The questionnaire application methods and the procedures to decide the better way to assess a population
During the development of a new questionnaire, the researcher has to think about how to apply it. The type of application method can influence which questions can be asked and in what format (Streiner & Norman, 2002). It is possible to choose one of these four types of methods: face-to-face interviews, self-administration, over the telephone, and by mail.
a. Face-to-face interviews: This method is used when the author decides to interview each subject individually. The researcher must recruit each subject, explain the importance of research and how to proceed, and, from the consent of the subject, perform the questions and record the answers of the subjects. This method has the advantage of a greater participation of the subjects because the researcher has the opportunity to personally explain the importance of his/her study. In addition, the researcher can clarify doubts during the administration of the questionnaire when the subject demonstrates any difficult on answering it.
It is important to consider another aspect of this method. If the researcher has any link with the research site or any of the subjects who participate in the study, it is recommended that the researcher do not conduct the interviews. In order to minimize any interference in the responses of the subjects, the researcher must instruct another person to apply the survey. This person must be able to answer any questions presented by the subjects. b. Self-administration: This technique can be chosen when the researcher has sufficient knowledge whether the subjects are able to answer the questionnaire by themselves. Therefore, one should consider the educational level of the population studied and whether the terms used in the questionnaire will be understood by the subjects. You can apply this technique in two ways. In both the researcher can explain the importance of the survey in person and give instructions on how to complete the survey. Then, the researcher can choose to leave the questionnaire with the subject and set a date and time to collect it. Or the researcher can ask the subject to answer the questionnaire in his/her presence. The disadvantage of this method is that the researcher can not clarify any doubt of the subjects, even if the researcher is present. The advantage is that there is less bias to answer, i.e., less interference from the researcher in the subject's response. c. Over the telephone: This method can be an interesting alternative when there is difficulty in performing a presence interview. There are some advantages such as reduction of blank answers, clarification of doubts, and recruitment of a larger number of subjects for participation in the research. However, there may be difficulty in obtaining the informed consent of subjects for study participation and some people may suspect the intention of the researcher as they do not see him/her personally. In addition, the questionnaire applied over the telephone can be useful when the questionnaire has only open-ended questions and when it is not too long, as most people do not appreciate to stay long time on the telephone. d. By mail: This technique can be the cheapest one and it allows the recruitment of a large number of subjects for participation in research. You can also send along with the questionnaire a formal request for written consent of the subjects. In addition to these documents, a letter explaining the importance of research and how the subject should respond to the questionnaire should also be included. However, the most important disadvantage of this method is the highest number of denied participation in the www.intechopen.com research. It is almost impossible to recover the questionnaire whether the subject, even with reminders sent by the researcher, does not return the instrument. Another disadvantage is the number of blank or invalid answers, because the subject can try to answer the questionnaire in a sequence different from what the researcher would like and this may influence the responses.

Summary
This chapter provides useful information for researchers interested in evaluating surveys on ergonomics. The instrument used to obtain data -questionnaires -should be carefully chosen based on the target population, constructs intended to be measured, existence of similar questionnaires, methods of administration, and psychometric properties. For questionnaires previously developed for a different language and/or culture, the chapter also presents the steps to a cross-cultural adaptation. If a new questionnaire is really necessary, which is decided after a careful analysis, the procedures to develop it are also explained. Finally, in order to show that the questionnaire is suitable for the target population and whether it measures what is intended to, the types of evaluation of the psychometric properties -reliability and validity -are described.