Open Access book publisher

and in specialized areas within epidemiology. Taken illustrate the contrast between the attempt to extend the limits of applicability of epidemiological research, and the "regular" scientific activity in this field or an applied epidemiology. Epidemiologists with different levels of expertise and will be able to find informative and inspiring readings among the of


Systematic review/meta-analysis vs. traditional reviews
The aims and scope of systematic reviews are clearly distinct from those of meta-analyses, and these methods can be used independently from each other, although both provide tools useful in the process of summarising information. These terms, however, are often used as synonyms, reflecting the fact that meta-analyses are seldom accomplished out of the context of a systematic review, but the opposite is not true.
Systematic review may be defined as "the application of strategies that limit bias in the assembly, critical appraisal, and synthesis of all relevant studies on a specific topic" [7], while meta-analysis is "a statistical analysis of the results from separate studies, examining sources of differences in results among studies, and leading to a quantitative summary of the results if the results are judged sufficiently similar to support such synthesis" [7]. To perform a meta-analysis we compute a simple weighted average of the results from individual studies, although some techniques involve statistical procedures with a degree of technical complexity much higher than the suggested by weighted averaging [8][9][10]. The weights vary with the method selected for meta-analysis, but the studies with more precise estimates are always assigned higher weights [3].
Meta-analysis may be a valuable resource to summarize the evidence gathered in a systematic review, which is expected to yield an unbiased sample of the evidence available on a topic, and the quantitative synthesis therefore provides a summary estimate usually interpreted as the state of the art on that specific subject. However, in the context of a literature review, the results of meta-analyses conducted over biased samples of the evidence, which are more likely when a systematic approach is not adopted, are meaningless [3]. A large proportion of the published systematic reviews do not include a meta-analysis, either because the statistical synthesis of the results is not advisable, when studies are not considered similar enough or their results are heterogeneous, or possible, when the needed information is not available or is provided in different formats across the studies. A small number of studies use meta-analysis to summarize results not obtained from systematic reviews, which reflects the fact that these statistical techniques may be used in any set of individual studies considered "combinable", even if the units of analysis are not obtained from a literature review (e.g. meta-analyses of results from different centres of multicentre studies [11]).
In a sample of published systematic reviews/meta-analyses presented in Figure 1, less than 5% of the reports used meta-analysis on results that were not obtained from a systematic review and approximately one-third were systematic reviews that used meta-analysis for quantitative synthesis of the results; the remaining systematic reviews relied on other methods for data synthesis.
The systematic reviews, with or without meta-analysis, are expected to address a clearly defined research question and to present the methodological options and procedures with the necessary detail to make the review fully replicable by others, in addition to providing a thorough description and discussion of the characteristics of the studies and their results [12][13][14]. Although there is an element of subjectivity in the definition of the criteria to determine the eligibility of the studies/results for the review and when drawing the conclusions, the whole process is transparent, as long as all decisions are specified clearly, and it is possible to estimate the extent to which systematic reviews yield biased conclusions www.intechopen.com [3]. In contrast, it is often impossible to judge whether traditional or narrative reviews are trustworthy, as the definition of the objectives is often ambiguous and the materials and methods opaque [15].
It should also be kept in mind that although the rigorous framework of a systematic review is expected to contribute to limit some of the biases that may affect literature searches, by itself it does not ensure the quality of the review, or the validity of its conclusions. The final result is dependent on the quality of the original studies that are evaluated, and the systematic review cannot be seen as a tool to improve the quality of primary sources that are methodologically flawed. A relatively large proportion of systematic reviews has been reported to have suboptimal quality and caution is needed in their interpretation, as for any other type of epidemiological study [16][17][18][19][20].
Despite the potential for improvement in the conduct and reporting of systematic reviews and meta-analysis, these remain the best option when the aim is a quantitative synthesis, or a transparent and reproducible qualitative assessment of the evidence. The latter is the most likely outcome when focusing on complex interventions/exposures or heterogeneous reports. There is also room for narrative reviews or essays addressing broader questions, providing essential information on relevant concepts or theory, or discussing key studies in detail, which may contribute to place the evidence into context and identify new directions for research. However, these should not be confused with unbiased systematic reviews regarding their appropriateness and potential for driving evidence-based decisions [21][22]. *Estimated from the analysis of abstracts (and full paper whenever necessary to obtain the required information) from the 50 systematic reviews (classified as such when at least the search strategy was described) or meta-analyses (classified as such when summary estimates were computed) published closest to the end of the first semester 2007, identified through a Pubmed search using the following expression: "meta-analysis" [Text Word] OR "meta-analysis" [Publication type] OR "systematic review"[Text Word] AND English [Language].

General structure and procedures for conducting a systematic review
The methodology that have gradually been developed for systematic reviews are expected to limit the potential biases to which literature reviews are prone to. A protocol for the review, with the detailed description of the methods to be used, should be prepared in advance and, ideally, maintained unchanged until the study is finished. The detailed description of the whole process allows the assessment of the quality of the reviews and validity of their conclusions. Figure 2 provides an overview of the essential steps of a systematic review, and identifies the main determinants of its quality.

Definition of a research question
The whole review builds upon the research question being targeted, which largely determines the impact of the conclusions and how smooth the review process will be. We may identify two major determinants of accomplishing this step successfully. On the one hand, the setting up of a relevant research question depends primarily on how well the researchers know the topic under study, from the biological, clinical and epidemiological standpoints, as applicable. Thus, having in the research team people with a deep understanding of the issue at hand is highly advisable. On the other hand, the precision of the objectives defined for the review will influence the amount of work required and the external validity of the conclusions. Research questions with a broad scope are probably more appealing to a general audience and also more likely to generate conclusions that www.intechopen.com apply to different contexts or settings. However, these are also more likely to result in a quantity of information unmanageable in a reasonable time if no substantial resources are available. For example, the largest ever report on how diet, physical activity and body fatness affect the risk of different cancers was published by the World Cancer Research Fund/American Institute for Cancer Research (WCRF/AICR) [23]. It included evidence from over 7000 papers, and involved research teams from nine different institutions and over 200 people worldwide [23]. When the available resources are more modest, which is usually the case, a much smaller number of exposures and outcomes need to be addressed (e.g. fruit/vegetables and gastric cancer, instead of diet and cancer) for the review to be feasible.
The number of systematic reviews and meta-analyses being published has increased substantially in the last years ( Figure 3) and it is not surprising that many interesting research questions have been reviewed before by other authors. However, a new systematic review on a topic that has been previously addressed with these methods is not necessarily redundant, and frequently is necessary. For example, one of the k e y p r i n c i p l e s o f t h e C o c h r a n e C o l l a b o r a t i o n ' s w o r k i s " k e e p i n g u p t o d a t e , b y a commitment to ensure that Cochrane Reviews are maintained through identification and incorporation of new evidence" [24]. The WCRF global network is also committed to the updating of the evidence of the 1997 and 2007 Expert Reports on food, nutrition, physical activity and cancer [25]. Several strategies, techniques or statistical methods have been developed to support different aspects of the updating of systematic reviews/metaanalyses [26][27][28][29].
The findings of the previous systematic reviews also need to be taken into account for finetuning of the objectives and/or a more efficient design of a new review on the same topic; the identification of systematic reviews/meta-analyses conducted before should probably be the first step of any new review.

Identification and selection of original studies
Ideally, systematic reviews would be based on the assessment of all the evidence available on a given topic. Although this may be possible, when the eligibility criteria restrict the search to a small number of investigations or when reviewing clinical trials, which may be identified in registries of this type of studies, it is virtually impossible otherwise, and an unbiased sample of the evidence is usually the aim to be targeted.
The search strategy should be as comprehensive as necessary to minimize bias. The decision on the number and type of the data sources to be included is influenced by the researchers understanding of the topic under study and the available time and resources.
The main sources of data for reviews are presented in Figure 4; the publications from journals indexed in electronic databases such as MEDLINE are easily available, while unpublished material may only be obtained from the authors and therefore its retrieval tends to be a much more difficult and time-consuming task.
Electronic databases are the source of the largest number of articles included in any systematic review on health-related topics, which reflects both the easy access to these www.intechopen.com resources, and their wide coverage. However, the inclusion of data sources with different characteristics, namely those that include unpublished results or publications with a more limited circulation may be essential to overcome selection bias, as the probability of a study being published or the place where it is published may depend on the nature of the results and their statistical significance (publication and related biases [30]).  www.intechopen.com Regardless of the sources of data selected, a decision has to be made on the eligibility of studies written in different languages; the impact of language restrictions in the comprehensiveness of the search and the potential for selection bias depends primarily on the subject of the review.
On the one hand, language restrictions may lead to the exclusion of a large proportion of the available studies when the outcomes or the exposures being studied have a geographical distribution that makes likely the publication of a large number of articles in a language other than English. For example, when addressing a topic related with the Chagas disease, which is found mainly in Latin America, it is important to consider articles written in Portuguese and Spanish [31], while for a review on the treatment with hyperbaric oxygen for neonatal hypoxic-ischemic encephalopathy, which is frequent in China and used much less often in Western countries [32], articles written in Chinese should be included. Resources such as LILACS (http://lilacs.bvsalud.org/), which indexes specifically scientific and technical Latin American and Caribbean literature, and Chinese bibliographic databases [33], respectively, are likely to allow the identification of a large proportion of studies for reviews on the previous topics.
On the other hand, studies not published in English, predominantly in journals with a more limited circulation, are more likely to have non-statistically significant results or "negative" findings, and therefore language restrictions may contribute to biased samples of studies to be reviewed.
When conducting searches over several electronic database, it should be taken into account that each of them may have different search fields and key-words for indexation of the articles, which requires that the search expressions are adjusted to the specificities of each source. Therefore, a detailed description of the search expression used in each database is essential for the systematic review to be replicable by others. Unfortunately, that does not seem to be the rule in many published reviews.
The indexation of the articles in the electronic databases is known to be imperfect, and a hand-search may be used toincrease sensitivity. This strategy, however, is time consuming and its use is restricted to searches on specific topics that are addressed in a relatively small number of journals.
Citation searching is usually one of the components of any search strategy, namely through the identification of the articles cited by those included in the systematic review ("backward citation tracking"). It is also possible to identify reports that cited specific studies included in the systematic review ("forward citation tracking"), which may be useful when we may expect that a specific article is important enough to be cited by a large proportion of those that we aim to identify. This requires the use of resources that include citation databases, only available by subscription, such as the Web of Knowledge or Scopus. Citation searching may also be useful when defining the search strategy, as it may be used as an independent source of references that provides valuable information to estimate the completeness of the main database searches, to improve the search expressions and the overall search strategy.
The glossary of the Cochrane collaboration [34] refers to "grey literature" as "the kind of material that is not published in easily accessible journals or databases" and it is expected to include "things like conference proceedings that include the abstracts of the research presented at conferences, unpublished theses, and so on". A large number of internet resources may be used to locate the so called grey literature [35][36]. However, each of them has a different scope and relatively limited coverage, in addition to specific modes of functioning, which results in the need of using several of these resources to answer a specific research question.
It should be taken into account that the different sources of data may yield quite heterogeneous results regarding the quality of the investigations and the detail of the reporting. For example, when the investigations are reported solely in the form of abstract, often reflect preliminary analyses that will be improved in subsequent publications based on the same datasets and it is also less likely that the methods and results are described with the necessary detail [37]. A large number of reports that may be classified as "grey literature" are not peer reviewed, which may translate into a larger heterogeneity of the studies identified in these sources as well as a lower average quality of the studies and their reporting.
The ideal search strategy should maximize sensitivity, as this is likely to be necessary to avoid selection bias. However, a high sensitivity comes with a high number of non-eligible references that need to be read. Figure 5 establishes a parallelism between the yielding of a search strategy and the accuracy of a diagnostic test, to depict the relation between the sensitivity of a search strategy and the number of reports needed to read for a systematic review. If one assumes that the gold-standard is an optimized search of all available resources, a search strategy with a high sensitivity [a/(a+c)], i.e., that misses a small www.intechopen.com proportion of all eligible reports (c), is likely to have a low positive predictive value [a/(a+b)], i.e., from all the studies identified only a small proportion is eligible for the review, which corresponds to a low precision and a high "number needed to read" [(a+b)/a] [38]. Comprehensive searches of multiple sources are usually necessary to ensure that the systematic review is based on an unbiased sample of the available evidence. However, the resources available to conduct a systematic review are limited, and it is always necessary to find a compromise between a high sensitivity and a low "number needed to read". The setting-up of the search strategy for a systematic review finds a parallelism in the definition of the sampling methods for cohort, case-control or cross-sectional studies. In both cases, although the options for sampling and recruitment of the participants/studies may not be the ideal due to logistic constraints, no compromises are acceptable below a certain threshold of the study validity. However, the reasoning for the definition of the sample size in the epidemiological research in general does not apply when conducting systematic reviews/meta-analyses. In the former, the impossibility of meeting the sample size estimated for the study will probably result in the decision of not doing it or in methodological rearrangements to make it worthy. In systematic reviews, the number of eligible studies is usually relatively small and the search strategy is designed to minimize bias instead of aiming a specific number of reports. On the one hand, the assessment of a small sample of studies in a systematic review does not compromise its potential to provide a valid summary of the best available evidence. On the other hand, the assessment of a high proportion of all the eligible studies does not correspond necessarily to an unbiased sample, as the studies missed may be substantially different from those included in the review. This reasoning finds a parallelism with the interpretation of the participation rates in an epidemiological study, as even a high participation may correspond to a differential participation.
The screening of the reference lists obtained from different sources should be based in clear and sound criteria defined a priori, and both training of the people involved in the search and the independent assessment of the references by more than one researcher may contribute to reliable results in this phase of the review.
Arbitrarily defined eligibility criteria may compromise the validity of the reviews and it is not surprising that decisions driven by the results end up in meaningless or biased conclusions. It may be more appropriate to have broad inclusion criteria and to conduct stratified analyses than to restrict the analysis to a highly selected group of studies, which may result in missing important information.
The independent assessment of the references lists is a resource consuming task, and it has not been demonstrated that it is absolutely necessary to ensure reliability. We described the performances of inexperienced and experienced reviewers, in a three-step approach ( Figure  6) to the screening of the reference lists, based on criteria defined in advance, and showed that the those with no previous experience may achieve good results when trained to adopt conservative selection procedures, despite consuming more time than the experienced [39].

STEP 1 STEP 2 STEP 3
Decision on: eligibility

Based on: title and abstract
Decision on: eligibility Based on: full text (reports not excluded in STEP 1)

Decision on: availability of data in the format required for data synthesis
Based on: full text (reports not excluded in STEP 2) same set of (conservative) criteria Fig. 6. Three-step approach to the screening of bibliographic references in the context of a systematic review.
In this approach to the screening of bibliographic references, the first and second steps are based in the same set of criteria for the exclusion of the studies that the reviewer can be absolutely sure that are not eligible for the review, and differ only in the amount of information that is available for the reviewer to decide. In step 1 the decisions are based only in the title and/or abstract (although it may be more appropriate not to decide on exclusions based only on the title), while in step 2 the reviewer has the full report available for analysis and a definite decision can be made. Training of the reviewers involved in this task is required for the adoption of a conservative that ensures that only the studies that clearly are not eligible are excluded without a thorough assessment of the full report.
Step 3 is also based in the full reports, and involves the assessment of the availability of data in the appropriate format for data synthesis. Although experienced reviewers may conduct steps 2 and 3 simultaneously, it may be easier to standardize the procedures and avoid errors if these are conducted separately.

www.intechopen.com
Documenting all the decisions taken across these steps is essential, since guidelines for the reporting of systematic reviews require information on the studies excluded, according to the reasons underlying the decision, usually in a flow-chart [14].

Data extraction
From each study included in the systematic review it is necessary to collect information for the assessment of the study quality (methodological aspects that are essential to interpret the results and/or to understand the heterogeneity of the results across studies), as well as the effect measures to be summarized and corresponding precision estimates, or the information needed to compute them.
The overall quality of a systematic review/meta-analysis depends on the quality of the studies being reviewed. The synthesis of biased or confounded effect estimates yields equally invalid conclusions ("garbage in, garbage out"), and therefore the assessment of the quality of the original studies is an important component of any systematic review. Several instruments have been developed to produce summary scores of the characteristics of the studies that may influence the validity of the results, but the assessment of the impact of the relevant methodological aspects individually is the most appropriate way of dealing with the information on the quality of the primary sources [40][41]. Likewise, the strategies of analysis that weight the results according to their quality [42] should also be dismissed.
A large inter-observer variation in data extraction and consequent decision on the studies to include in the review may be observed, due to different choices and errors [43]. Many reports provide several results potentially eligible for extraction and in different forms, requiring accurate decisions on those to be selected, and frequently is necessary to express all the extracted data in the same format, which may easily originate conversion errors.
For example, in a replication of a previous meta-analysis on the relation between dietary calcium intake and blood pressure [44], the reassessment of the original studies showed "that data from one study had been inappropriately extracted and converted, leading to an understatement of the calcium-blood pressure relation by a factor of about 30" and "raised questions about the extraction and conversion of data from several other studies and about the statistical methods used" [45]. A study on data extraction errors assessed 27 metaanalyses that used standardized mean differences and showed that a high proportion had errors. The authors concluded that "although the statistical process is ostensibly simple, data extraction is particularly liable to errors that can negate or even reverse the findings of the study" [46]. Another investigation [43] addressed the inter-observer variation in the extraction of continuous and numerical rating scale data from trial reports for use in metaanalyses and compared experienced methodologists with PhD students. The agreement was somewhat higher among the former, but "disagreements were common and often larger than the effect of commonly used treatments" [43].
Data extraction is a demanding task and a great deal of effort is needed to ensure the validity and reliability of this procedure. On the one hand, it should be conducted following a previously defined protocol, to limit the potential for different judgements to result in different choices about the data to extract. Although there is some margin for adjustments taking into account unexpected observations, a proper understanding of the topic under study together with experience in the conduct of systematic reviews and meta-analyses should allow the definition of a protocol that requires only minor changes throughout data extraction. Most of the variability in the methodologies and reporting of data can be anticipated and taken into account in the protocol. On the other hand, the conduct of data extraction by experienced researchers is expected to contribute to minimize errors, and the independent data extraction by more than one researcher, together with consensus meetings, will contribute for the identification and correction of errors before summarizing the evidence.

Data synthesis
The book based on the seminal meta-analysis conducted by Glass et al [47] starts with a reference to the mathematician David Hilbert that once said "one can measure the importance of a scientific work by the number of earlier publications rendered superfluous by it" [48]. The extent to which systematic reviews provide relevant answers to their objectives depends on the accomplishment of a sound synthesis of the results, in addition to the detailed description (results and methodological characteristics) of each original study being reviewed. Figure 7 depicts a framework for deciding on the strategy for data synthesis, and the corresponding general aims of the systematic review.
Meta-analysis should be conducted only when the individual studies are homogeneous regarding their methodological characteristics, to the extent necessary for the weighted average of the results from the individual studies to be meaningful, which tends to occur more frequently among experimental studies than in those with observational designs. However, the fulfillment of this condition is not sufficient, and the homogeneity of the effect measures is also required. There are several methods available to identify and quantify heterogeneity [3,[49][50], but this will not be addressed in this chapter. Under the above mentioned circumstances, meta-analysis may be a valuable option to summarize the evidence, contributing to overcome the statistical power limitations of the individual investigations.
The improvement in the statistical power may be illustrated by the meta-analysis represented in the logo of the Cochrane Collaboration [51]. It shows a meta-analysis of randomized controlled trials (RCT) assessing the effect of a "short, inexpensive course of a corticosteroid given to women about to give birth too early". The first RCT was reported in 1972, and after one decade there was strong evidence that corticosteroids reduce the risk of babies dying from the complications of immaturity. However, because no systematic review of these trials had been published at that time, for several years a large number of people did not benefit from this effective treatment. Another good example is a meta-analysis on the cardiovascular adverse effects of rofecoxib [52], in which the authors conclude that the drug should have been withdrawn several years earlier. This shows the importance of using meta-analysis to obtain more precise estimates of an effect that is estimated with a very low precision in each of the individual studies because the outcome is too rare, which frequently occurs when dealing with adverse drug reactions. In both examples, the meta-analysis contributes for a more efficient use of the resources, for research and health-care.
When the participants' characteristics, study designs, exposures/interventions, or measurement of outcomes differ meaningfully across a set of studies, or when the results differ beyond the expected due solely to the play of chance, the combined estimates are likely to be meaningless, and an analytical rather than a synthetic approach is required [4].  This strategy of analysis may be exemplified with two systematic reviews that addressed the relation between fruit and vegetables consumption and the occurrence of gastric cancer in cohort studies [53], and the relation between Helicobacter pylori infection and gastric cardia cancer [54], respectively. In the former, the heterogeneity of the results was explained mainly by the differences between studies in the outcome being addressed (incidence or mortality) and in the duration of the follow-up. In the latter, the heterogeneity was largely explained by the characteristics of the populations studied, with stronger association between infection and cancer being observed in the studies conducted in settings with a risk of gastric cancer. In both cases, analysis of subgroups of studies with different methodological characteristics and metaregression were used to understand the heterogeneity.
The methods adopted throughout the whole process should aim the reduction of bias, but this may be accomplished to different extents in different systematic reviews, and the readers www.intechopen.com should be able to assess this, as in any other research design. Even when the number of studies is small and heterogeneous, and neither more precise summary estimates nor an important contribution to the understanding of heterogeneity are possible, the thorough description of the materials and methods allow its replication by others. If no other reason persists for opting for a resource and time consuming systematic review, its transparency should suffice.

Conclusion
There is a large consensus regarding the importance of having sound and transparent syntheses of the literature for health care providers, policy makers and researchers to be able to integrate the unmanageable amounts of biomedical information that is constantly being produced.
The objectives of systematic reviews/meta-analyses may be primarily synthetic, when aiming more precise average estimates, or analytic, when concerned with understanding the different results observed across studies, even if some overlap between these two pathways may occur, depending on the homogeneity of the original sources of data. These approaches may be placed on the top of each one of the two hierarchies of study designs proposed by Vandenbroucke [55], corresponding, respectively, to the confirmation of hypotheses when the a priori probability is high and to "discovery and explanation".
Understanding the place of systematic reviews/meta-analyses in modern epidemiology, and the determinants of the option between predominantly synthetic or analytic approaches for data synthesis, are crucial for a proper utilization of these resources.