Open access peer-reviewed chapter

Evolving Roles of Spontaneous Reporting Systems to Assess and Monitor Drug Safety

Written By

Emanuel Raschi, Ugo Moretti, Francesco Salvo, Antoine Pariente, Ippazio Cosimo Antonazzo, Fabrizio De Ponti and Elisabetta Poluzzi

Reviewed: 06 July 2018 Published: 05 November 2018

DOI: 10.5772/intechopen.79986

From the Edited Volume


Edited by Charmy S. Kothari, Manan Shah and Rajvi Manthan Patel

Chapter metrics overview

1,649 Chapter Downloads

View Full Metrics


This chapter aims to describe current and emerging roles of spontaneous reporting systems (SRSs) for assessing and monitoring drug safety. Moreover, it offers a perspective on the near future, which entails the so-called era of Big Data, keeping in mind both regulator and researcher viewpoints. After a panorama on key data sources and analyses of post-marketing data of adverse drug reactions, a critical appraisal of methodological issues and debated future applications of SRSs will be presented, including the exploitation and challenges in evidence integration (i.e., merging and combining heterogeneous sources of data into a unique indicator of risk) and patient’s reporting via social media. Finally, a call for a responsible use of these studies is offered, with a proposal on a set of minimum requirements to assess the quality of disproportionality analysis in terms of study conception, performing and reporting.


  • pharmacovigilance
  • signal
  • spontaneous reporting system
  • disproportionality analysis

1. Introduction

Prescription of a medication is based on a balance between expected benefits, already investigated before marketing authorization, and possible risks (i.e., adverse effects), which become fully apparent only as time goes by after marketing authorization. Premarketing development, in fact, provides evidence on efficacy of drugs in ideal clinical setting of use (i.e., clinical trials); only the most frequent side effects are recognized in this step. The use of drugs in the real-world circumstances will show the actual risk-benefit profile.

The World Health Organization (WHO) previously defined pharmacovigilance (PhV) as “the science and activities relating to the detection, assessment, understanding and prevention of adverse effects or any other possible drug-related problems” [1], a definition that, in the recent past, was regarded as being synonymous with post-marketing surveillance for adverse drug reactions (ADRs).

After the adoption in 2012 of the new pharmacovigilance legislation (Regulation (EU) No 1235/2010 and Directive 2010/84/EU) [2, 3] approved by the European Parliament and European Council in December 2010, PhV embraces the whole risk-benefit assessment, thus dealing with multiple types of evidence emerging along the life cycle of drugs for continuous reassessment of the place in therapy of each medicine, both in clinical and in regulatory terms.

Many sources of data and relevant methods of analysis are used in PhV: from disproportionality analyses (DAs) in spontaneous reporting systems (SRSs) to analytical studies (cohort or case-control designs). These traditional approaches are now integrated by innovative strategies (e.g., social media mining and case-population studies) in the fourth-generation PhV [4].

In this chapter, current and emerging roles of DAs in SRSs will be critically discussed, keeping in mind both regulator and researcher viewpoints. A panorama on key data sources (and their proper selection) will be described, followed by a critical appraisal of methodological issues and debated future applications, including exploitation and challenges in evidence integration (i.e., merging and combining heterogeneous sources of data into a unique indicator of risk) and patient’s reporting via social media. All these issues are based on key publications of the authors and on the latest advances published in the literature (MEDLINE, as of May 1, 2018). Finally, a call for a responsible use of these studies is offered, with a proposal (authors’ personal ideas) on a set of minimum requirements to assess the quality of DAs in terms of study conception, performing and reporting.


2. Post-marketing data sources

Not only notification of suspected adverse drug events is mandatory for health professionals, but also other subjects can report events to the relevant regulatory authorities. According to ICH-E2 guidelines (International Conference on Harmonization,, each National Drug Agency maintains its specific SRS to collect all notifications and routinely use data-mining algorithms (DMAs) to process data, with the aim of identifying possible signals of unknown drug-effect associations. These DMAs identify drug-reaction pairs occurring with a significant disproportion in comparison with all other pairs, through the method of case-non case approach. Reactions are usually recorded according to the “MedDRA” classification (medical dictionary for regulatory activities), which allows to select cases at different hierarchical levels (from SOC—system organ class to PT—preferred term;

Clinical pharmacology knowledge is requested to design and interpret results from DMAs and to decide if further examination is needed (either within the same source of data or by other types of data) or specific bias affects the validity of the findings. Other healthcare data sources are available for PhV to corroborate results of SRS data mining, despite developed for other reasons. As a general classification, they can be pooled into two main groups: electronic medical records (EMRs) and claim databases.

Electronic medical records (EMRs) aim to assist physicians in daily clinical practice (including appropriate prescription) by collecting sociodemographic and clinical information (diagnoses, risk factors, treatments, and outcomes). Primary care is the most frequent setting to develop these kinds of databases such as Clinical Practice Research Datalink (CPRD; formerly General Practice Research Database—GPRD, in UK); Health Search (by the Italian College of General Practitioners); The Health Improvement Network (THIN), in UK; and Interdisciplinary Processing of Clinical Information (IPCI), in the Netherlands. The high quantity of data makes them valuable sources to address clinical pharmacology questions, including new effects of drugs (especially on primary endpoints, to confirm premarketing evidence) and assessment of appropriate drug use (closer to the main purpose of the registries).

Claim databases were mainly created for administrative purposes, and together with hospital databases provide valuable sources to address PhV questions: data provided (e.g., diagnoses of hospital admissions, reimbursed prescriptions of drugs and diagnostic procedures in ambulatory care) are generally used for reimbursement and other economic issues, and, as a secondary aim, they represent an important source of information for epidemiological questions (taking into account that nonreimbursed intervention is usually not recorded, information on lifestyle and actual exposure to medicines is lacking.


3. Main spontaneous reporting systems

Each National Drug Agency collects its own reports in a dedicated spontaneous reporting database, and some international SRSs gather reports originating both by systematic flows from national databases and by direct submission of the reporter. Each source has specific characteristics and limitations to be considered when planning a drug safety analysis (e.g., completeness of data and options for database interrogation); however, collecting information from all these accessible sources is the mainstay in PhV.

Table 1 shows an overview of main international PhV databases, which cover a very large population and heterogeneous patterns of drug use and ADR reporting attitudes. Public access to SRSs is becoming a standard, as addressed in Section 8.2.

FAERS WHO—VigiBase EudraVigilance Australian Database of Adverse Event Notifications (DAEN) Canada Vigilance Adverse Reaction Online Database Japanese Adverse Drug Event Report database (JADER)
Access Full data access (download) since 2004a Web-based interface (VigiLize™, VigiFlow™, VigiMine, applications for full data accessc) Web-based interface (different access policies for full data accessd) Full data access (download) Full data access (download)
Timeframe 1969–present 1968–present 2001–present 1971–present 1965–present 2004–present
Products covered All drugs and biologicsb All drugs and biologics, including vaccines All drugs and biologics authorized in the European Union All medicines, including vaccines, used in Australiae All drugs, biologics, vaccines, and natural health products licensed in Canadaf All drugs and biologics, including vaccines used in Japan
Source of reports Healthcare professionals, drug companies, patients/consumers National and regional pharmacovigilance centers (which may receive reports from patients, healthcare professionals, or drug companies) National competent authorities and marketing authorization holders (currently, no direct reporting from patients and healthcare professionals) Healthcare professionals, consumers, and market authorization holders Healthcare professionals, consumers, and market authorization holders Healthcare professionals, consumers, and market authorization holders
Current number of reports available >12 million (as of April 2015), more than 1,000,000 per year (2012–2014) >10 million (as of 2016) >1 million received in 2013 Unknown (no public statistics provided) Unknown (no public statistics provided) ~500,000 (as of 2017)
Origin of submitted reports USA and serious/unexpected reports from EU, Japan, and other extra-US countries Worldwide (107 official members and 33 associate members), but majority from EU and the US EU Australia Canada Japan
Coding system for event MedDRA MedDRA MedDRA MedDRA MedDRA MedDRA
Search strategy through “free text” in the narratives No (a Freedom of Information Act can be requested to the FDA) No No No No No

Table 1.

Overview of major international spontaneous reporting systems that can be searched via online systems.

Different web-based tools are provided; see Böhm et al. [14]. Recently, the FDA has launched the FAERS Public Dashboard, a highly interactive web-based tool that allows to query FAERS data in a user-friendly fashion (

Devices, vaccines, and other products are not included, as they are specifically recorded in ad hoc databases: MAUDE—Manufacturer and User Facility Device Experience (ttps://, VAERS—Vaccines Adverse Event Reporting System (, and CAERS—Center for Food Safety and Applied Nutrition Adverse Event Reporting System (

Freely available for all members in the WHO Program for International Drug Monitoring.

Specific access policies are described depending on stakeholder groups. For details, see the following link: On November 22, 2017, a new and improved version of EudraVigilance was launched. The new system has enhanced features for the reporting and analysis of suspected adverse reactions to support a better safety monitoring of medicines and a more efficient reporting process for stakeholders.

Medical devices are not included, as they are specifically recorded in the ad hoc Database of Adverse Event Notifications—medical devices (

Data on human blood and blood components have only been included since September 1, 2015; data on vaccines used for immunization have only been included since January 1, 2011; the majority of vaccine reports are submitted to the Canadian Adverse Events Following Immunization Surveillance System (CAEFISS).

Modified and updated from [32, 113]. ADR: adverse drug reaction; MedDRA: Medical Dictionary for Regulatory Activities.


4. The appropriate choice of data source according to the research question

The identification of the most appropriate source of data is a key step to properly address the research question, considering strength and limitations of the different approaches (Table 2). For instance, SRSs represent the best source of data to investigate the so-called designated medical events (DMEs), usually rare with strong drug-attributable component (e.g., Torsades de Pointes and Stevens-Johnson Syndrome) [5, 6]. Conversely, possible role of drugs in events with high background incidence (e.g., myocardial infarction) can be better investigated by healthcare databases (EMRs and claim databases) [7, 8]. No matter of the type of ADR, a typical time sequence to detect safety profile of drugs considers data mining of SRSs as the first step of the analysis, followed by investigation through healthcare databases to confirm or refuse statistically significant associations.

Strengths Weaknesses
Disproportionality approach It can be conducted rapidly, and it is easy to implement.
It can be conducted on spontaneous reporting systems and healthcare databases.
Good performance (accuracy in discriminating false from true positives) when major confounders and biases are accounted for.
Highly suitable for rare events with high drug-attributable risk (e.g., TdP and DILI).
Does not provide risk estimates.
Loss of information due to aggregated data.
Unable to handle numerous confounders.
Sensitive to protopathic and indication biases.
Less suitable for events with high background incidence (e.g., myocardial infarction).
Traditional pharmacoepidemiological designs It provides risk estimates (cohort and case-control design).
It allows controlling for confounders if matching and nesting are performed (case-control design).
Robust to confounders that are stable over time (case crossover, self-controlled cohort, and self-controlled case series).
Highly suitable for events with high background incidence (e.g., myocardial infarction).
It needs very large dataset to have enough power to detect signals in case of rare events (cohort and case-control design).
Less suitable for rare events with high drug-attributable risk (e.g., TdP and DILI).
Prescription sequence symmetry analysis (PSSA) Rapid and easy to be performed (it only requires patient identifier, medication code, and medication dispensed date).
Graphical output can be generated to help data visualization and interpretation.
Highly specific and moderate sensitivity.
It can control for time-constant confounders.
It does not provide risk estimates (it complements disproportionality approach).
Prescribing trends are affected by external factors (adjustment is required).
Inappropriate identification of new use (exclusion/censoring of switchers is required).
Time-variant confounders.
Sensitive to inverse causality, protopathic, and indication biases.
Systematic review with meta-analysis It can provide risk estimates (especially if RCT is the primary source).
It does not require additional data collection.
It can be conducted rapidly.
It can highlight gaps in research.
Validity depends on the scientific rigor of the methods, quality, and type of primary source (RCT or observational studies).
Meta-analysis of nonrandomized studies (observational) is currently not standardized.

Table 2.

Overview of study designs to assess safety of medicines.

Modified from [9]. DILI: drug-induced liver injury; RCT: randomized controlled trial; TdP: torsade de pointes.

From data cleaning (a mere data managing step, see later) to statistical analyses, all steps of data management are considered tasks to address questions on ADRs. Usually, each source of data requires specific data-mining approaches (e.g., disproportion calculation for SRSs and multiple regression analysis for EMRs), but emergent strategies to better exploit the more accessible sources are now appearing in the literature (e.g., self-controlled time series and prescription sequence symmetry analysis—PSSA) [9]. In fact, data mining could virtually provide as many associations as possible between drug and effect, but without consensus among experts on the methodological steps and confirmation of pathophysiological pathways, the association can easily conduct to interpret errors.


5. Current applications of disproportionality analyses and case-by-case assessment

5.1 The regulator’s view

Traditionally, regulatory decision-making has relied on detection of safety signals through spontaneous reports. Today, things are changing for several reasons, including increased awareness of prescribers on the importance of PhV and the emerging role of different health professionals and patients.

A modern model involves signal detection, signal validation (i.e., signal should represent a novel causal relationship between a drug and an event), signal prioritization (evaluation of clinical impact of the safety issue), and some other steps to drive the decision-making, also on the basis of data on how drugs are used in a population and how their utilization can be influenced. Drug consumption is also now frequently analyzed by regulators to evaluate the actual impact of risk minimization strategies in a specific settings, such as the risk of progressive multifocal leukoencephalopathy with multiple sclerosis therapies [10].

Regulatory agencies routinely perform analyses of SRSs to detect disproportionality signals, especially for new drugs. Although the Food and Drug Administration (FDA) and the European Medicine Agency (EMA) have different frameworks, they are promoting rigorous scientific information exchange for optimal post-approval drug safety monitoring [11]. Both agencies publicly posted the list of signals emerging from internal analyses, with the aim to promote transparency and stimulate research while avoiding alarm. Usually, many of these signals remain (fortunately) unnoticed by clinicians, and only a minority of them result in measures affecting clinical practice, such as ketoacidosis with sodium-glucose cotransporter-2 inhibitors, which in turn prompted the FDA to revise relevant labels.

Also for old drugs, the importance of spontaneous reports should not be overlooked, especially because the amount of time of a drug on the market (drug age) is correlated with the number of signals detected [12]. The recent case of tiocolchicoside, restricted in recommended dose and treatment duration by the EMA, is noteworthy: after withdrawal of tetrazepam, the use of alternatives (including tiocolchicoside) and relevant spontaneous reporting increased, which made evident specific safety concerns [13].

In the past, regulatory actions on a given safety issue did not support clinical practice. The case of haloperidol and the risk of torsade de pointes (TdP) is a typical example: an ECG before administration was indeed recommended in some circumstances before administering the medicine. However, it was not duly taken into account that a psychotic crisis does not usually allow appropriate ECG measurement, and this results in the inability to use injectable haloperidol in the emergency setting. The clinical consequence was a loss of this therapeutic option and its substitution with alternatives, which are not necessarily better.

5.2 The researcher’s view

Disproportionality analyses (DAs) are attracting considerable interest in the medical literature for several reasons:

  1. there is increasing availability of publicly accessible SRSs and open-access tools to independently analyze international databases [14]; the various web-based resources mainly differ in terms of data transparency, possibility to customize searches and analyses (e.g., correction for confounders);

  2. DAs are inexpensive and relatively quick and easy to perform, at least by frequentist methods such as reporting odds ratio (ROR) and proportional reporting ratio (PRR); these methods can be applied systematically to analyze a given pharmacological class or specific DMEs such as TdP [15];

  3. they are likely to be published in a high ranking journal, especially when sophisticated analyses are presented, claiming to correct for multiple confounders [16], and a strong signal emerges. This aspect raises ethical issues: on one hand, the researcher may be more prone toward an alarming interpretation of the findings to increase the impact of the publication. On the other hand, when broadly looking at the published literature in the past 5 years, only a minority of industry-sponsored studies provided “negative findings,” that is, the lack of statistically significant DAs [17, 18].

This “uncontrolled” scenario has generated what someone coined “apophenia,” that is, the perception of meaningful patterns and causal connections among random data [19], or the so-called pharmacovigilance syndrome, that is, the incorrect use of spontaneous adverse event reports to infer that a drug causes an adverse reaction, what the incidence or prevalence of such events may be, and whether one drug has lower or higher risk than another [20]. This in turn increases the complexity in the risk-benefit assessment [21] and may generate false alarm among clinicians [22].

It must be emphasized that statistical techniques, usually referred to as quantitative analyses [23], cannot be used as a standalone approach to assess a drug-related risk because no risk quantification can be offered: they should be viewed in conjunction with a qualitative analysis of individual reports, whenever feasible, and other pieces of evidence (e.g., observational studies). In other words, they cannot replace a proper clinical judgment in the individual patient.

In the recent past, a debate arose on the proper use of DAs and the benefit of their publication [24, 25]. However, no actions have been taken so far. The key applications of DAs are summarized as follows:

  1. Signal detection (including specific events or the overall safety profile). This is the main goal of DAs, especially for medicines with unpredictable pharmacokinetics-pharmacodynamics such as biologicals [26], or recently marketed drugs with still undefined safety profile. This is also justified for rare adverse events that may escape detection in premarketing clinical trials (e.g., TdP, liver injury) or in case an imbalance (not reaching statistical significance) emerged from clinical data, as happened for pioglitazone and bladder cancer [27]. The choice of comparator group is pivotal in signal detection, especially in terms of clinical implications. For instance, a novel antidiabetic drug should be compared with other antidiabetic drugs through the so-called analysis by therapeutic area (i.e., comparing the reporting of a given drug with other agents belonging to the same therapeutic class), in order to identify patients that are likely to share the common risk factors, mitigate the confounding by indication bias, and investigate the potential intraclass variations of risk [28, 29, 30, 31, 32]. As a matter of fact, a suspected risk for a drug can be interpreted by a clinical point of view only if compared to the same risk of therapeutic alternatives, especially for severe disorders (e.g., diabetes) because patient cannot be left without treatment.

  2. Test/verify/confirm a pharmacological hypothesis. This can be illustrated by a number of examples in the recent past, including the relationship between hERG blockade and occurrence of TdP in humans [33]; the risk of diabetes by antipsychotics, which was more frequently associated with agents blocking simultaneously histamine H1 and serotonin 5-HT2C receptors [34]; the association between different receptor occupancy and antipsychotic-induced movement disorders [35], and the link between dopamine receptor agonist drugs and specific impulse control disorders [36].

  3. Address/verify methodological issues. This aspect is receiving an increasing attention because it may strongly impact on final results. Before planning the analysis, it is important to verify all potential biases affecting the drug(s) or event(s) under investigation and prespecify strategies to handle with these confounders (see Section 7) [37, 38, 39, 40, 41, 42, 43].

  4. Investigate the likelihood of drug-drug interactions. A few pilot initiatives proposed theoretical strategies as well as relevant automated methods to detect signals resulting from drug-drug interactions (DDIs) in PhV databases [44, 45, 46, 47, 48]. Various approaches can be used to highlight adverse drug interactions: (a) reported suspicion of interactions as noted by the reporter in a case narrative, (b) assignment of the two drugs as interacting (c) drug-drug interaction reported as adverse event, and (d) increased co-reporting for the drug pair when disproportionality is applied [49]. There is also interest in using SRSs to investigate whether a given drug-drug combination moderates the frequency of an adverse event [50, 51].

A recent systematic review highlighted that only a minority of studies aimed at confirming or supporting previous regulatory decisions on a given safety aspect [52], thus strengthening the aforementioned concept that DAs do not usually support, on their own, regulatory actions but must be integrated with other data sources.

Apart from DAs, the value of case-by-case assessment should not be disregarded. In fact, the individual evaluation of reports performed by pharmacovigilance experts with medical background has multiple aims: (a) it may per se be used for signal detection of rare ADRs, such as in the case of DMEs by detecting potential drug-event combinations even earlier than DAs [53] and (b) it may confirm or refuse disproportionality signals, by strengthening/reducing causality assessment or by identifying duplicates by automated strategy (through the use of narratives). The key challenging aspect of case-by-case analysis is represented by causality assessment, that is, the process of differential diagnoses to prove actual causal relationship: exclusion of alternative causes, biological and temporal plausibility, evidence of dechallenge and rechallenge (usually unintentional) should be verified. The complexity of causality assessment stems from the fact that it needs to be viewed from the context of the patient treated rather than the drug product [54]. Although several approaches are available to assess causality, no single method is universally accepted and there is no gold standard [55]. The choice of the most suitable approach may also depend on the event under investigation; for instance, ALDEN is a specific algorithmic score validated for assessment of drug causality in Stevens-Johnson syndrome/toxic epidermal necrolysis [56], whereas Roussel Uclaf Causality Assessment Method (RUCAM) was implemented for drug-induced liver injury [57].

As a conclusive remark, it should be recognized that most researchers are from academia, and in fact, their additional role is university teaching. In the last few years, experts of medical teaching have strengthened the importance of PhV in the core curriculum of undergraduate students of healthcare courses (i.e., medicine, pharmacy, dentistry, nursing, etc.). WHO and the most active national PhV centers are committed to better define knowledge, skills, and attitudes that students should acquire in order to have an active role in pharmacovigilance [58].


6. Potential future applications: evidence integration and risk estimates

Integration of heterogeneous data (literature including mass media, clinical trials, observational studies, spontaneous reporting data analysis, case reports, and preclinical data) is currently in the research domain at the preliminary level, with the degree of confidence and reliance on a given source as key unresolved issues. An attempt to achieve a risk score on the pro-arrhythmic potential of drugs was undertaken within the ARITMO project [59], where a Dempster-Shafer model was used to combine evidence from heterogeneous and independent sources using expert judgment [60]. The only published experience on data integration in pharmacovigilance comes from the (useful) interplay between SRSs and healthcare databases to increase the accuracy of signal detection [61, 62].

In the following section, the issue of evidence integration for research purposes will be addressed in the context of systematic reviews, which are increasingly being used as they can make researchers and readers aware about what is known, how it is known, how evidence varies across studies, and thus about what is not already known [63].

Issues of data quality and inherent limitations cause remarkable impact in spontaneous reporting studies in which more sources of variability (e.g., missing data) and biases affecting the results could be identified (competition or notoriety bias). Nevertheless, so far, no specific tools or techniques have been developed to select, compare, or pool together data from DAs. This could be due to a relative paucity of this kind of analysis in the medical literature.

Disproportionality is used to detect “signals of disproportionate reporting” (SDRs) that, once detected, are usually investigated through other, and more precisely, study designs. It is thus rare to have additional DAs regarding the same outcome related to the same drug or drug class and that used a comparable tool for signal detection (frequentist vs. Bayesian approaches). Nevertheless, at least theoretically, techniques and statistical basis used to perform meta-analysis could also be used to analyze results from disproportionality, at least to evaluate consistency of signal across different databases. A consistent signal found in two databases could be probably prioritized in comparison with inconsistent ones. Notably, raw data cannot be pooled because of the existence of an unquantified degree of redundancy (i.e., duplicates across databases), but results can be combined to reach a single “pharmacovigilance score” [59].

It is well known that results of DA cannot be considered as measures of risk: the number of cases in a spontaneous reporting database does correspond to neither the number of cases that happened under the drug nor to that of cases induced by the drug, and the number of exposed people is not measured. From this point of view, including results of disproportionality in a meta-analysis could be considered inappropriate, although identification of heterogeneity in reporting may be of interest [64]. In the absence of any clear guideline, disproportionality studies could be searched and included in (qualitative) systematic reviews, but their results must be kept separated from pooled risk estimates of (quantitative) meta-analyses [65]. A recent experience by a French team on safety of drugs acting on the nitric oxide pathway in pulmonary hypertension considers together results from a DA of VigiBase and from a meta-analysis of clinical trials and concludes that the safety profiles of riociguat and phosphodiesterase inhibitors were different, thus providing a rationale for safe prescribing [66]. This approach, as the integration of spontaneous reporting analysis in meta or teleoanalysis [67], is still a research question.

Preliminary findings raise the hypothesis that, provided that all technical and clinical aspects are addressed, the performance of DAs is remarkable [7] and may approach the relative risks of analytical studies, thus providing an initial indication of the likely clinical importance of an adverse event [68].


7. Methodological aspects

7.1 Current concepts in study design

Once the research question has been identified, the researcher must keep in mind the various limitations and biases affecting SRSs to reduce the likelihood of detecting spurious signals. Moreover, clinical, pharmacological, and statistical considerations are needed to select the most appropriate dataset, definition of cases, exposure, and covariables for stratification/adjustment.

Although the discussion on performance, accuracy, and reliability of different approaches to perform DAs was fascinating a decade ago, at present there is still no recognized gold standard methodology, and the key factor that may influence results is represented by the threshold defined for the number of cases [69, 70]. DAs in spontaneous reporting databases test whether an ADR is reported more frequently than expected; they allow identifying the so-called SDRs [23, 71]. These SDRs must be differentiated from safety signals because the existence of a SDR is not sufficient to constitute a safety signal (it does not always result in one, in fact), and a safety signal does not always imply a corresponding SDR [72].

As previously described, the various SRSs differ in terms of accessibility, catchment area, drug codification, and other technical issues. For instance, two key steps must be managed when analyzing the publicly available version of FAERS: drug mapping and removal of duplicates. These aspects have been extensively covered in the previous book chapter, and the reader should refer to this publication for details [73]. The FDA is continuously working to develop a probabilistic record-linkage algorithm combining structured and unstructured data (narratives) to improve the detection rate and accordingly reduce the occurrence of false positive signals [74].

7.2 Bias and strategies for their minimization

Before considering a potential causal relationship for a given identified SDR, main biases that affect signal detection from spontaneous reporting must be eliminated or at least mitigated. Notably, even after accounting for major bias, clinical association cannot be inferred from SRSs, and channeling bias (selective prescription of newer drugs to patients with more severe disease [75]) is unlikely to be fully accounted by statistical adjustments. They are described later together with practical examples and relevant minimization strategies as shown in Table 3.

Bias Example Underlying reason Minimization strategy
Indication bias Angiotensin Converting Enzyme (ACE) inhibitors showing signal of hypoglycemia. These agents are largely used in diabetic patients. Sensitivity analysis including only nondiabetic patients (i.e., using antidiabetic agents).
Drug competition bias Anticoagulants when analyzing drug-induced bleeding. Anticoagulants are expected to cause bleeding as toxic effect of their drug class. Analysis by excluding reports with anticoagulants.
Event competition bias Extrapyramidal syndrome (ES) when analyzing first-generation antipsychotics (FGA). ES is a typical ADR in FGA-treated patients. Analysis by excluding ES to detect new safety signal for FGA.
Notoriety bias Rhabdomyolysis occurrence with statins after regulatory warnings. After that alert, the number of events arose. Studying signal before the alert.
Dilution bias Suicide ideation related to new antidepressant. A warning issued for a whole pharmacological class has stronger impact for newer drugs because the new ADR is diluted by other ADRs for older drugs. Taking into account the time of drug approval and investigate different sources of dilution (e.g., warnings, publications, etc.).

Table 3.

Major biases in disproportionality analyses and strategies for their minimization.

Modified from [114].

Overall, we can identify: (A) indication bias when a drug is found to be associated with a given event for the sole reason that it is indicated in patients with comorbidities that increase the risk of that event; (B) competition bias also called “masking effect” when an event/drug more frequently reported for a given drug/event can “mask” identification of other possible ADRs/drugs [42, 76, 77, 78, 79, 80, 81, 82];(C) notoriety bias when media attention (e.g., regulatory warning and milestone publication) causes over-reporting of peculiar ADR for specific drugs [37, 38]; and (D) dilution bias when a whole drug class is influenced by media attention for an event, older drugs with a larger numbers of reports are less likely to generate safety signal than newer drugs (with less reports) [83].

The Weber effect is an additional factor that may influence the reporting of given drugs, although it cannot be formally considered as a source of bias [84]. It was originally described as a higher reporting especially during the first 2 years after marketing approval, thus suggesting novelty per se as a risk factor for notification, although modern adverse event reporting systems seem less affected by this bias [43].


8. Unsettled issues

8.1 Patient reporting: current status

The 2012 PhV legislation forced national competent authorities and marketing authorization holders to record and report cases of suspected adverse reactions reported by patients [3]. This, in turn, caused legislation remarkable increase of the total number of patient reports (+113%) after 3 years, with the Netherlands, the UK, Germany, France, and Italy accounting for 75% of all patient reports [85]. The relevance of patient reports is heterogeneous, and a recent survey on 141 countries worldwide showed that in one-fourth of them, patients were not allowed to report. Conversely, countries receiving the highest percentage of patient reports in 2014 were the USA (64%) and Canada (30%).

More than 70 countries had fewer than 50 reports from patients [86]. The quality and the value of patient reports in the context of signal detection were evaluated in many published studies [87, 88, 89, 90, 91]. The value of the reports as a signal is directly dependent on the amount of clinically relevant information, in addition to the fact that an ADR report requires a thorough examination of the potential drug-event association. Most of the published studies comparing information reported by patients and healthcare professionals focused on the completeness of information [86, 92].

Patient reports give detailed descriptions of suspected ADRs, attribute reactions to specific medicines, and provide information useful for assessing causality. Patient reports often have richer narratives than those of healthcare professionals, including detailed information about the impact of the suspected ADR on the patient’s life [91].

Many studies, mainly from the UK and the Netherlands, showed that patient reports allow for the identification of new ADRs and lead to the strengthening of signal detection activities [90, 93, 94].

In summary, patient’s reporting offers a different perspective in drug safety assessment and may potentially contribute in signal detection. However, it is important to further investigate its actual role in drug safety assessment; in fact, the large number of reports without clear causal relationship (recently called “precautionary report”) may alter adverse event profile by masking safety signals or, conversely, creating spurious associations [95].

8.2 Ethical and transparency issues

The relevance of patient reporting highlights the need of public access to spontaneous reporting data, and many countries now provide public access to SRSs, with the possibility to have summary presentations for reactions associated to each single drug in the database or a case listing of limited information for each single case report. Both EMA and WHO Uppsala Monitoring Centre (UMC) developed web tools to access a limited set of spontaneous reporting data in their database, EudraVigilance ( and VigiBase (

The EMA policy includes the possibility for academia or nonprofit organization to ask for a greater access to data as aggregated data outputs or line listings based on core data elements ( However, it has been commented that the EMA’s approach to transparency over PhV data is too timid. The public access of PhV data is even more restricted for vaccines, mainly due to the potential negative impact of this public access to the vaccination campaigns. The reporting of serious adverse events not causally related to the vaccination could lead to a misrepresentation of vaccine risks that could be used by antivaccine movement. To our knowledge, very few European countries (e.g., Italy and the Netherlands) give public access to spontaneous data related to vaccines.

A different approach to transparency is followed by UMC and FDA. In VigiBase, custom search service provided by UMC is performed upon request. Any stakeholders can use the custom search services to request a limited set of data for specific studies or projects for a fee.

The best level of transparency is observed for FDA data. Data for both drugs (FAERS) and vaccine (VAERS) can be obtained using web-based search tools that return structured and/or unstructured data. Moreover, the entire database is quarterly downloadable in comma-separated value (CSV) or other formats. This access needs technical skills to properly process the relational database files and any unstructured fields. However, it gives the possibility to any users to analyze FDA spontaneous reporting data even applying DAs [5]. Since June 2014, the FDA developed an innovative platform called openFDA ( to facilitate access and use of big important FDA public datasets by developers, researchers, and the public through harmonization of data across disparate FDA datasets provided via application programming interfaces (APIs) [96]. Recently, the FDA has also launched the FAERS Public Dashboard, a highly interactive web-based tool that will allow to query FAERS data in a user-friendly fashion ( These different approaches to public access spontaneous reporting data lead to a bizarre situation because the reports included in EudraVigilance, VigiBase, and FAERS are largely overlapped, and it could be possible to have different information for the same report.

8.3 Social media: opportunities and challenges

An area of emerging interest for research is represented by the use of information provided by patients in social media on personal experiences when using a given drug. At present, it is under investigation whether or not (and how) social media data mining can contribute to signal detection [94, 95].

A recent review summarizes prevalence, frequency, and comparative value of information on adverse events of healthcare interventions from user comments and videos in social media. The study assessed over 174 social media sites, with discussion forums (71%) being the most popular. The overall prevalence of adverse event reports in social media varied from 0.2 to 8% of posts. Moreover, there was general agreement on overall concordance between adverse events mentioned in social media and those already documented in other sources (such as drug labels and published trials) [97].

The web-recognizing adverse drug reaction (Web-RADR) project, leaded by EMA and funded within the innovative medicines innovation (IMI), aims to recommend policies, frameworks, tools, and methodologies in the use of social media and mobile technology to improve drug safety [98]. Specific objectives are as follows: (a) to develop the specific mobile application prototypes to support adverse drug reaction reporting and the provision of drug safety information to application users and (b) to assess the usefulness of social media data for PhV and more specifically in signal detection activities.

The theoretical advantages of social media in the context of signal detection rely on potential earlier identification of rare and serious drug-related problems, in comparison with conventional SRSs, considering the opportunity to share information as fast as possible and the large number of active users in the social media. It has been reported that patient reports of suspected adverse reactions, particularly for specific reactions, can precede those of healthcare professionals [99]. One study of social media posts containing discussions of adverse drug events (“Proto-AEs”) found that there were nearly three times as many Proto-AEs found in Twitter data than reported to the FDA by consumers, with rank correlation between them at the distribution of reactions at MedDRA SOC level [100].

Another important value from social media analyses comes from extracting qualitative insights into the actual discussions made by patients around a drug and an adverse event. This can be of great value for addressing issues related to the patient experience around an ADR and its impact on the quality of life [101]. Moreover, mining data from social media gives us a greater chance of capturing ADRs that a patient would not necessarily complain about to their doctor or nurse and can also help assessment of the risk perceptions of patients.

Key challenge is represented by the identification of drugs and ADRs in the text strings through a particular type of machine learning called natural language processing (NLP). From the perspective of PhV and NLP specifically, user posts on social media contain colloquial language and also misspellings. Especially when using lexicon-based approaches, these present problems as the accuracy of direct matches decreases. Colloquial and informal language is more difficult to parse, and thus, recent research tasks have focused on developing NLP tools specifically for data from social media [102, 103]. The balance between sensitivity and specificity of these tools in identifying ADRs is a key issue because a high number of false positives could heavily impact the efficacy of signal detection activities.

Another key element is the quality of the information on adverse events reported in the social media, which was analyzed only by a few works. A study where Internet narratives posted by patients were evaluated showed that the informativeness level was very incomplete and makes their assessment and use for PhV purpose difficult [104].

Concerning the potential of social media analyzes for earlier signal detection, contrasting data are published [105, 106].

Social media data mining uses information for PhV purposes, which were not primarily shared by the patient for this purpose. This raises a number of ethical questions, especially about identification of individuals by utilizing additional information, such as the geocode location on posting, username, and other potentially personally identifiable information [107], which are still unresolved. How would patient using social media react when approached for additional information by organizations that collect PhV data? Since this is a new area, ethically sound policy guidance needs to be developed.

A different approach in the use of Internet data for signal detection is the use of anonymized logs of web searchers [108]. In a recent study, a web-based search query method called “query log reaction score” was developed to detect whether adverse events associated with certain drugs could be found from search engine query data. The web query methods have moderate sensitivity (80%) in detecting signals in web query data compared with reference signal detection algorithms, but many false positives were generated, and this method had low specificity [109].


9. Future perspectives

The continuous increasing number of spontaneous reports and the increasing quality in their systematic archiving and accessing comply scientific community to improve methods of analysis and ways to interpret them for regulatory, clinical, and research purposes.

A specific debated issue on the current role of data-mining procedures of SRSs regards the possibility to directly compare drugs within the same therapeutic class [110]. We are in favor of this approach and strongly encourage further research regarding the use of SRSs, under stringently defined conditions, to compare adverse event rates for drugs [111]. To this aim, all the following criteria must be fulfilled:

  1. Same therapeutic indication(s). The effect of the underlying disease may be reduced by restricting DAs to drugs within the same therapeutic area [29, 30].

  2. Similar market penetration and utilization. Drug consumption/prescription should be considered in order to: (i) complement DAs by highlighting possible risk differences through reporting rates (especially for vaccines and DMEs) [112]; (ii) weigh the drug risk at the population level (and assess the public health impact of ADRs); and (iii) prioritize safety signals emerging from traditional DAs [113].

  3. Similar time on the market. This aspect should be carefully considered in the analyses to avoid the temporal or time-point bias, especially when comparing first- versus second-generation drugs. Standardization of the time on the market using the same fixed-length post-approval time-frame has been proposed [110].

  4. Data distortions are unlikely to occur or apply in a similar manner across the drugs under investigation. Stratification (for age and sex) or adjustment should always be considered to minimize the presence of known confounders. Moreover, the existence of specific biases should be verified and accounted for.

An emerging application of SRSs, in the era of Big Data, is represented by their integration with other heterogeneous sources of healthcare data (e.g., the availability of prescription-data, hospital admission and discharge, population-based, disease-based, death registries, social media, and literature) to support proactive PhV in the risk-benefit assessment, as performed in the ARITMO projects through the Dempster-Shafer approach [59].

Finally, the question arises as to whether all disproportionality studies should be published in scientific journals. Supporters of scientific transparency and full release of datasets via Open Science would undoubtedly call for public availability of study results, including negative findings. A proposal was recently formulated [114].

This controversy on the quality of DAs raises the concern on how best assess it and reach consensus on a “set of minimum requirements to assess the quality of DAs in terms of study conception, performing and reporting.” Provisional criteria have been recently proposed (from the experience of antidiabetic drugs) [114], but further discussion is warranted:

  • Clear title. Avoid the general terms such as “pharmacovigilance analysis.” Prefer the following terms: “disproportionality analysis,” “analysis of spontaneous reporting system,” and “analysis of spontaneous reports.”

  • Scientifically sound study conception. The scientific rationale must be clearly indicated and fall within one of these aforementioned categories (DAs are particularly suited for DMEs). Regulatory approach (i.e., identification of a potential signal during routine monitoring of spontaneous reporting systems) and commissioned analysis for regulatory purposes should not be formally eligible for publication in a journal, unless an added value emerges (e.g., the analysis is extended to the entire pharmacological class).

  • Transparent study design. The unit of analysis should be described. Case(s) and exposure (reference group) definition should be specifically defined. The search strategy must be stated, and a clear description behind the choice is warranted. Key confounders to be accounted for must be a priori identified. Strategies to handle these biases must be indicated, including stratified or adjusted analyses. Notoriety must be carefully assessed: a structured literature evaluation is recommended, instead of a mere check to summary of product characteristics.

  • Balanced discussion and conclusion. Prefer the term “disproportionality signal” and “signal of disproportionate reporting,” and avoid the terms such as “alarm signal,” “signal of risk,” “increased risk,” “association,” “incidence.” Compare the results with those emerging from similar studies (emerged from the structure literature evaluation). Limitations should be provided in a dedicated section, avoid a mere listing of known biases affecting spontaneous reporting system. Avoid the specific recommendations (decision-making approach) to support drug prescription or selection of drugs claimed to be safer.

From a technical standpoint, good signal detection practices have been published by the Innovative Medicines Initiative Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium (PROTECT) project, which have formulated 39 recommendations for those working in the PhV community [115].

A final issue regards the timeliness of publishing DAs when keeping with signal detection. For instance, the analysis by Elashoff et al. [16] on pancreatitis reports with incretin-based drugs, apart from methodological flaws and data misinterpretation causing unjustified alarm, was also untimely, considering that observational studies had already been carried out. Conversely, liver injury with direct-acting oral anticoagulants (DOACs) was studies because of limited predictivity of premarketing phases in detecting clinical signals of liver toxicity and previous concern with ximelagatran: the disproportionality signal raised for rivaroxaban in FAERS [116] was tested by the recent US population-based studies, which found lower hospitalization rates for liver injury with DOAC initiators than patients starting warfarin, with rivaroxaban and dabigatran associated with the highest and lowest risk [117, 118], although confounders are likely to exist [119, 120]. This case underscores the value of performing well-conducted DAs and the importance of directing subsequent analytical research to confirm or refute the drug-related hypothesis.

All these unsettled issues witness the need and the importance of implementing research to finally clarify the role of DAs in clinical practice.


10. Concluding remarks

Regulators and especially clinicians are appreciating the importance and the role of DAs to monitor and assess the safety profile of marketed drugs. All “actors” dealing with SRSs must always be aware of the so-called seduction bias and self-deception bias (i.e., over-reliance on mathematical models and the subconscious confidence in expecting a given output from results), thus be reminded of inherent limitations that, at present, do not allow to assess actual risk in clinical practice, mainly because of the lack of certainty in the occurrence of adverse events and the lack of exposure data [121].

From a research perspective, there is an urgent need to raise the bar, aiming to increase the accuracy and reproducibility (in one word the quality) of this kind of study. From one side, there is a room for improvement in several aspects of the analysis of SRSs, including relevant implications and their appropriate use such as the aspect of “no findings” (i.e., findings of nondisproportional results), which has not received sufficient attention so far. Moreover, different research teams are implementing sophisticated methods to account for confounders in signal detection, so that DAs may approach relative risk. In the meantime, we propose to include disproportionality studies in (qualitative) systematic reviews keeping results separated from pooled risk estimates of (quantitative) meta-analyses [63].

In conclusion, SRSs represent an invaluable source to monitor and assess the safety of medications, including drugs, vaccines, and healthcare products.

We call for a responsible use and publication of DAs, which should be regulated through a consensus approach among experts; this would finally establish the use and transferability of DAs in clinical practice.


  1. 1. Edwards IR, Aronson JK. Adverse drug reactions: Definitions, diagnosis, and management. Lancet. 2000;356:1255-1259
  2. 2. European Parliament. Regulation (EU) 1235/2010—2010 pharmacovigilance legislation. 2010. Available from:
  3. 3. European Council (2010) Directive 2010/84/EU—2010 pharmacovigilance legislation. Available:
  4. 4. Laporte JR. Fifty years of pharmacovigilance—Medicines safety and public health. Pharmacoepidemiology and Drug Safety. 2016;25:725-732
  5. 5. Poluzzi E, Raschi E, Moretti U, De Ponti F. Drug-induced torsades de pointes: Data mining of the public version of the FDA adverse event reporting system (AERS). Pharmacoepidemiology and Drug Safety. 2009;18:512-518
  6. 6. Poluzzi E, Raschi E, Motola D, Moretti U, De Ponti F. Antimicrobials and the risk of torsades de pointes: The contribution from data mining of the US FDA adverse event reporting system. Drug Safety. 2010;33:303-314
  7. 7. Harpaz R, DuMouchel W, LePendu P, Bauer-Mehren A, Ryan P, Shah NH. Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system. Clinical Pharmacology and Therapeutics. 2013;93:539-546
  8. 8. Coloma PM, Trifiro G, Patadia V, Sturkenboom M. Postmarketing safety surveillance: Where does signal detection using electronic healthcare records fit into the big picture? Drug Safety. 2013;36:183-197
  9. 9. Arnaud M, Begaud B, Thurin N, Moore N, Pariente A, Salvo F. Methods for safety signal detection in healthcare databases: A literature review. Expert Opinion on Drug Safety. 2017;16:721-732
  10. 10. Anton R, Haas M, Arlett P, Weise M, Balabanov P, Mazzaglia G, et al. Drug-induced progressive multifocal leukoencephalopathy in multiple sclerosis: European regulators’ perspective. Clinical Pharmacology and Therapeutics. 2017;102:283-289
  11. 11. Dal Pan GJ, Arlett PR. The US Food and Drug Administration-European Medicines Agency collaboration in pharmacovigilance: Common objectives and common challenges. Drug Safety. 2015;38:13-15
  12. 12. Pacurariu AC, Coloma PM, van HA, Genov G, Sturkenboom MC, Straus SM. A description of signals during the first 18 months of the EMA pharmacovigilance risk assessment committee. Drug Safety. 2014;37:1059-1066
  13. 13. Pageot C, Bezin J, Smith A, Arnaud M, Salvo F, Haramburu F, et al. Impact of medicine withdrawal on reporting of adverse events involving therapeutic alternatives: A study from the French Spontaneous Reporting Database. Drug Safety. 2017;40:1099-1107
  14. 14. Bohm R, von HL, Herdegen T, Klein HJ, Bruhn O, Petri H, et al. OpenVigil FDA—Inspection of U.S. American adverse drug events pharmacovigilance data and novel clinical applications. PLoS One. 2016;11:e0157753
  15. 15. Sakaeda T, Tamon A, Kadoyama K, Okuno Y. Data mining of the public version of the FDA adverse event reporting system. International Journal of Medical Sciences. 2013;10:796-803
  16. 16. Elashoff M, Matveyenko AV, Gier B, Elashoff R, Butler PC. Pancreatitis, pancreatic, and thyroid cancer with glucagon-like peptide-1-based therapies. Gastroenterology. 2011;141:150-156
  17. 17. Hauben M, Hung EY. Revisiting the reported signal of acute pancreatitis with rasburicase: An object lesson in pharmacovigilance. Therapeutic Advances in Drug Safety. 2016;7:94-101
  18. 18. Hauben M, Hung EY, Hanretta KC, Bangalore S, Snow V. Safety of perflutren ultrasound contrast agents: A disproportionality analysis of the US FAERS database. Drug Safety. 2015;38:1127-1139
  19. 19. Gagne JJ. Finding meaningful patterns in adverse drug event reports. JAMA Internal Medicine. 2014;174:1934-1935
  20. 20. Greenblatt DJ. The pharmacovigilance syndrome. Journal of Clinical Psychopharmacology. 2015;35:361-363
  21. 21. Onakpoya IJ, Heneghan CJ, Aronson JK. Post-marketing withdrawal of 462 medicinal products because of adverse drug reactions: A systematic review of the world literature. BMC Medicine. 2016;14:10
  22. 22. Moore N, Blin P, Gulmez SE. New oral anticoagulants (NOAC) and liver injury. Journal of Hepatology. 2014;61:198-199
  23. 23. Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiology and Drug Safety. 2009;18:427-436
  24. 24. de Boer A. When to publish measures of disproportionality derived from spontaneous reporting databases? British Journal of Clinical Pharmacology. 2011;72:909-911
  25. 25. Montastruc JL, Sommet A, Bagheri H, Lapeyre-Mestre M. Benefits and strengths of the disproportionality analysis for identification of adverse drug reactions in a pharmacovigilance database. British Journal of Clinical Pharmacology. 2011;72:905-908
  26. 26. Giezen TJ, Mantel-Teeuwisse AK, Meyboom RH, Straus SM, Leufkens HG, Egberts TC. Mapping the safety profile of biologicals: A disproportionality analysis using the WHO adverse drug reaction database, VigiBase. Drug Safety. 2010;33:865-878
  27. 27. Piccinni C, Motola D, Marchesini G, Poluzzi E. Assessing the association of pioglitazone use and bladder cancer through drug adverse event reporting. Diabetes Care. 2011;34:1369-1371
  28. 28. Poluzzi E, Raschi E, Koci A, Moretti U, Spina E, Behr ER, et al. Antipsychotics and torsadogenic risk: Signals emerging from the US FDA adverse event reporting system database. Drug Safety. 2013;36:467-479
  29. 29. Salvo F, Raschi E, Moretti U, Chiarolanza A, Fourrier-Reglat A, Moore N, et al. Pharmacological prioritisation of signals of disproportionate reporting: Proposal of an algorithm and pilot evaluation. European Journal of Clinical Pharmacology. 2014;70:617-625
  30. 30. Grundmark B, Holmberg L, Garmo H, Zethelius B. Reducing the noise in signal detection of adverse drug reactions by standardizing the background: A pilot study on analyses of proportional reporting ratios-by-therapeutic area. European Journal of Clinical Pharmacology. 2014;70:627-635
  31. 31. Raschi E, Poluzzi E, Koci A, Antonazzo IC, Marchesini G, De Ponti F. Dipeptidyl peptidase-4 inhibitors and heart failure: Analysis of spontaneous reports submitted to the FDA adverse event reporting system. Nutrition, Metabolism, and Cardiovascular Diseases. 2016;26:380-386
  32. 32. Raschi E, Parisotto M, Forcesi E, La Placa M, Marchesini G, De Ponti F, et al. Adverse events with sodium-glucose co-transporter-2 inhibitors: A global analysis of international spontaneous reporting systems. Nutrition, Metabolism, and Cardiovascular Diseases. 2017;27:1098-1107
  33. 33. De Bruin ML, Pettersson M, Meyboom RH, Hoes AW, Leufkens HG. Anti-HERG activity and the risk of drug-induced arrhythmias and sudden death. European Heart Journal. 2005;26:590-597
  34. 34. Montastruc F, Palmaro A, Bagheri H, Schmitt L, Montastruc JL, Lapeyre-Mestre M. Role of serotonin 5-HT2C and histamine H1 receptors in antipsychotic-induced diabetes: A pharmacoepidemiological-pharmacodynamic study in VigiBase. European Neuropsychopharmacology. 2015;25:1556-1565
  35. 35. Nguyen TT, Pariente A, Montastruc JL, Lapeyre-Mestre M, Rousseau V, Rascol O, et al. An original pharmacoepidemiological-pharmacodynamic method: Application to antipsychotic-induced movement disorders. British Journal of Clinical Pharmacology. 2017;83:612-622
  36. 36. Moore TJ, Glenmullen J, Mattison DR. Reports of pathological gambling, hypersexuality, and compulsive shopping associated with dopamine receptor agonist drugs. JAMA Internal Medicine. 2014;174:1930-1933
  37. 37. Pariente A, Gregoire F, Fourrier-Reglat A, Haramburu F, Moore N. Impact of safety alerts on measures of disproportionality in spontaneous reporting databases: The notoriety bias. Drug Safety. 2007;30:891-898
  38. 38. Raschi E, Piccinni C, Poluzzi E, Marchesini G, De Ponti F. The association of pancreatitis with antidiabetic drug use: Gaining insight through the FDA pharmacovigilance database. Acta Diabetologica. 2013;50:569-577
  39. 39. Hoffman KB, Demakas AR, Dimbil M, Tatonetti NP, Erdman CB. Stimulated reporting: The impact of US food and drug administration-issued alerts on the adverse event reporting system (FAERS). Drug Safety. 2014;37:971-980
  40. 40. Maignen F, Hauben M, Hung E, Van HL, Dogne JM. Assessing the extent and impact of the masking effect of disproportionality analyses on two spontaneous reporting systems databases. Pharmacoepidemiology and Drug Safety. 2014;23:195-207
  41. 41. Maignen F, Hauben M, Hung E, Holle LV, Dogne JM. A conceptual approach to the masking effect of measures of disproportionality. Pharmacoepidemiology and Drug Safety. 2014;23:208-217
  42. 42. Arnaud M, Salvo F, Ahmed I, Robinson P, Moore N, Begaud B, et al. A method for the minimization of competition bias in signal detection from spontaneous reporting databases. Drug Safety. 2016;39:251-260
  43. 43. Hoffman KB, Dimbil M, Erdman CB, Tatonetti NP, Overstreet BM. The Weber effect and the United States Food and Drug Administration’s adverse event reporting system (FAERS): Analysis of sixty-two drugs approved from 2006 to 2010. Drug Safety. 2014;37:283-294
  44. 44. Van Puijenbroek EP, Egberts AC, Heerdink ER, Leufkens HG. Detecting drug-drug interactions using a database for spontaneous adverse drug reactions: An example with diuretics and non-steroidal anti-inflammatory drugs. European Journal of Clinical Pharmacology. 2000;56:733-738
  45. 45. Thakrar BT, Grundschober SB, Doessegger L. Detecting signals of drug-drug interactions in a spontaneous reports database. British Journal of Clinical Pharmacology. 2007;64:489-495
  46. 46. Leone R, Magro L, Moretti U, Cutroneo P, Moschini M, Motola D, et al. Identifying adverse drug reactions associated with drug-drug interactions: Data mining of a spontaneous reporting database in Italy. Drug Safety. 2010;33:667-675
  47. 47. Strandell J, Wahlin S. Pharmacodynamic and pharmacokinetic drug interactions reported to VigiBase, the WHO global individual case safety report database. European Journal of Clinical Pharmacology. 2011;67:633-641
  48. 48. Labat V, Arnaud M, Miremont-Salame G, Salvo F, Begaud B, Pariente A. Risk of myopathy associated with DPP-4 inhibitors in combination with statins: A disproportionality analysis using data from the WHO and French Spontaneous Reporting Databases. Diabetes Care. 2017;40:e27-e29
  49. 49. Strandell J, Caster O, Bate A, Noren N, Edwards IR. Reporting patterns indicative of adverse drug interactions: A systematic evaluation in VigiBase. Drug Safety. 2011;34:253-266
  50. 50. Fadini GP, Bonora BM, Mayur S, Rigato M, Avogaro A. Dipeptidyl peptidase-4 inhibitors moderate the risk of genitourinary tract infections associated with sodium-glucose co-transporter-2 inhibitors. Diabetes, Obesity & Metabolism. 2018;20:740-744
  51. 51. Fadini GP, Sarangdhar M, Avogaro A. Pharmacovigilance evaluation of the association between DPP-4 inhibitors and heart failure: Stimulated reporting and moderation by drug interactions. Diabetes Therapy. 2018;9:851-861
  52. 52. Dias P, Penedones A, Alves C, Ribeiro CF, Marques FB. The role of disproportionality analysis of pharmacovigilance databases in safety regulatory actions: A systematic review. Current Drug Safety. 2015;10:234-250
  53. 53. Hochberg AM, Hauben M. Time-to-signal comparison for drug safety data-mining algorithms vs. traditional signaling criteria. Clinical Pharmacology and Therapeutics. 2009;85:600-606
  54. 54. Ralph EI. Causality assessment in pharmacovigilance: Still a challenge. Drug Safety. 2017;40:365-372
  55. 55. Agbabiaka TB, Savovic J, Ernst E. Methods for causality assessment of adverse drug reactions: A systematic review. Drug Safety. 2008;31:21-37
  56. 56. Sassolas B, Haddad C, Mockenhaupt M, Dunant A, Liss Y, Bork K, et al. ALDEN, an algorithm for assessment of drug causality in Stevens-Johnson syndrome and toxic epidermal necrolysis: Comparison with case-control analysis. Clinical Pharmacology and Therapeutics. 2010;88:60-68
  57. 57. Garcia-Cortes M, Stephens C, Lucena MI, Fernandez-Castaner A, Andrade RJ. Causality assessment methods in drug induced liver injury: Strengths and weaknesses. Journal of Hepatology. 2011;55:683-691
  58. 58. van Eekeren R, Rolfes L, Koster AS, Magro L, Parthasarathi G, Al Ramimmy H, et al. What future healthcare professionals need to know about pharmacovigilance: Introduction of the WHO PV core curriculum for university teaching with focus on clinical aspects. Drug Safety. 2018. DOI: 10.1007/s40264-018-0681-z
  59. 59. Final Report Summary-ARITMO (Arrhythmogenic potential of Drugs)—Project ID: 241679—Funded under FP7-HEALTH. 2017. Available from:
  60. 60. Park SJ, Ogunseitan OA, Lejano RP. Dempster-Shafer theory applied to regulatory decision process for selecting safer alternatives to toxic chemicals in consumer products. Integrated Environmental Assessment and Management. 2014;10:12-21
  61. 61. Li Y, Ryan PB, Wei Y, Friedman C. A method to combine signals from spontaneous reporting systems and observational healthcare data to detect adverse drug reactions. Drug Safety. 2015;38:895-908
  62. 62. Pacurariu AC, Straus SM, Trifiro G, Schuemie MJ, Gini R, Herings R, et al. Useful interplay between spontaneous ADR reports and electronic healthcare records in signal detection. Drug Safety. 2015;38:1201-1210
  63. 63. Gough G, Oliver S, Thomas J. An Introduction to Systematic Reviews. 2012. Available from: (First: 1-288)
  64. 64. Cheng YJ, Nie XY, Chen XM, Lin XX, Tang K, Zeng WT, et al. The role of macrolide antibiotics in increasing cardiovascular risk. Journal of the American College of Cardiology. 2015;66:2173-2184
  65. 65. Raschi E, Salvo F, Poluzzi E, De Ponti F. Safety meta-analysis: A call for appropriate use of disproportionality measures from spontaneous reporting systems. Journal of the American College of Cardiology. 2016;67:2193
  66. 66. Khouri C, Lepelley M, Roustit M, Montastruc F, Humbert M, Cracowski JL. Comparative safety of drugs targeting the nitric oxide pathway in pulmonary hypertension: A mixed approach combining a meta-analysis of clinical trials and a disproportionality analysis from the World Health Organization Pharmacovigilance Database. Chest. 2017;S0012-3692(17):33265-33268
  67. 67. Wald NJ, Morris JK. Teleoanalysis: Combining data from different types of study. British Medical Journal. 2003;327:616-618
  68. 68. Macia-Martinez MA, de Abajo FJ, Roberts G, Slattery J, Thakrar B, Wisniewski AF. An empirical approach to explore the relationship between measures of disproportionate reporting and relative risks from analytical studies. Drug Safety. 2016;39:29-43
  69. 69. Slattery J, Alvarez Y, Hidalgo A. Choosing thresholds for statistical signal detection with the proportional reporting ratio. Drug Safety. 2013;36:687-692
  70. 70. Candore G, Juhlin K, Manlik K, Thakrar B, Quarcoo N, Seabroke S, et al. Comparison of statistical signal detection methods within and across spontaneous reporting databases. Drug Safety. 2015;38:577-587
  71. 71. Van Puijenbroek EP, Bate A, Leufkens HG, Lindquist M, Orre R, Egberts AC. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiology and Drug Safety. 2002;11:3-10
  72. 72. Hauben M, Reich L, Gerrits CM, Younus M. Illusions of objectivity and a recommendation for reporting data mining results. European Journal of Clinical Pharmacology. 2007;63:517-521
  73. 73. Poluzzi E, Raschi E, Piccinni C, De Ponti F. Data mining techniques in pharmacovigilance: Analysis of the publicly accessible FDA adverse event reporting system (AERS). In: Karahoca A, editor. Data Mining Applications in Engineering and Medicine. Croatia: InTech; 2012. pp. 265-302
  74. 74. Kreimeyer K, Menschik D, Winiecki S, Paul W, Barash F, Woo EJ, et al. Using probabilistic record linkage of structured and unstructured data to identify duplicate cases in spontaneous adverse event reporting systems. Drug Safety. 2017;40:571-582
  75. 75. Petri H, Urquhart J. Channeling bias in the interpretation of drug effects. Statistics in Medicine. 1991;10:577-581
  76. 76. Gould AL. Practical pharmacovigilance analysis strategies. Pharmacoepidemiology and Drug Safety. 2003;12:559-574
  77. 77. Almenoff J, Tonning JM, Gould AL, Szarfman A, Hauben M, Ouellet-Hellstrom R, et al. Perspectives on the use of data mining in pharmaco-vigilance. Drug Safety. 2005;28:981-1007
  78. 78. Wang HW, Hochberg AM, Pearson RK, Hauben M. An experimental investigation of masking in the US FDA adverse event reporting system database. Drug Safety. 2010;33:1117-1133
  79. 79. Pariente A, Didailler M, Avillach P, Miremont-Salame G, Fourrier-Reglat A, Haramburu F, et al. A potential competition bias in the detection of safety signals from spontaneous reporting databases. Pharmacoepidemiology and Drug Safety. 2010;19:1166-1171
  80. 80. Pariente A, Avillach P, Salvo F, Thiessard F, Miremont-Salame G, Fourrier-Reglat A, et al. Effect of competition bias in safety signal generation: Analysis of a research database of spontaneous reports in France. Drug Safety. 2012;35:855-864
  81. 81. Juhlin K, Ye X, Star K, Noren GN. Outlier removal to uncover patterns in adverse drug reaction surveillance—A simple unmasking strategy. Pharmacoepidemiology and Drug Safety. 2013;22:1119-1129
  82. 82. Salvo F, Leborgne F, Thiessard F, Moore N, Begaud B, Pariente A. A potential event-competition bias in safety signal detection: Results from a spontaneous reporting research database in France. Drug Safety. 2013;36:565-572
  83. 83. Pariente A, Daveluy A, Laribiere-Benard A, Miremont-Salame G, Begaud B, Moore N. Effect of date of drug marketing on disproportionality measures in pharmacovigilance: The example of suicide with SSRIs using data from the UK MHRA. Drug Safety. 2009;32:441-447
  84. 84. Hartnell NR, Wilson JP. Replication of the Weber effect using postmarketing adverse event reports voluntarily submitted to the United States Food and Drug Administration. Pharmacotherapy. 2004;24:743-749
  85. 85. Banovac M, Candore G, Slattery J, Houyez F, Haerry D, Genov G, et al. Patient reporting in the EU: Analysis of Eudravigilance Data. Drug Safety. 2017;40:629-645
  86. 86. Matos C, Harmark L, van HF. Patient reporting of adverse drug reactions: An international survey of national competent authorities’ views and needs. Drug Safety. 2016;39:1105-1116
  87. 87. Inacio P, Cavaco A, Airaksinen M. The value of patient reporting to the pharmacovigilance system: A systematic review. British Journal of Clinical Pharmacology. 2017;83:227-246
  88. 88. Avery AJ, Anderson C, Bond CM, Fortnum H, Gifford A, Hannaford PC, et al. Evaluation of patient reporting of adverse drug reactions to the UK ‘Yellow Card Scheme’: Literature review, descriptive and qualitative analyses, and questionnaire surveys. Health Technology Assessment. 2011;15:1-234
  89. 89. Inch J, Watson MC, nakwe-Umeh S. Patient versus healthcare professional spontaneous adverse drug reaction reporting: A systematic review. Drug Safety. 2012;35:807-818
  90. 90. van HF, de WS, Harmark L. The contribution of direct patient reported ADRs to drug safety signals in the Netherlands from 2010 to 2015. Pharmacoepidemiology and Drug Safety. 2017;26:977-983
  91. 91. Rolfes L, van HF, van der LL, Taxis K, van PE. The quality of clinical information in adverse drug reaction reports by patients and healthcare professionals: A retrospective comparative analysis. Drug Safety. 2017;40:607-614
  92. 92. Rolfes L, van HF, Wilkes S, van GK, van PE. Adverse drug reaction reports of patients and healthcare professionals-differences in reported information. Pharmacoepidemiology and Drug Safety. 2015;24:152-158
  93. 93. Hazell L, Cornelius V, Hannaford P, Shakir S, Avery AJ. How do patients contribute to signal detection?: A retrospective analysis of spontaneous reporting of adverse drug reactions in the UK’s Yellow Card Scheme. Drug Safety. 2013;36:199-206
  94. 94. Watson S, Chandler RE, Taavola H, Harmark L, Grundmark B, Zekarias A, et al. Safety concerns reported by patients identified in a collaborative signal detection workshop using VigiBase: Results and reflections from Lareb and Uppsala Monitoring Centre. Drug Safety. 2017;41:203-212
  95. 95. Klein K, Scholl JH, De Bruin ML, Van Puijenbroek EP, Leufkens HG, Stolk P. When more is less—An exploratory study of the precautionary reporting bias and its impact on safety signal detection. Clinical Pharmacology and Therapeutics. 2017;103:296-303
  96. 96. Kass-Hout TA, Xu Z, Mohebbi M, Nelsen H, Baker A, Levine J, et al. OpenFDA: An innovative platform providing access to a wealth of FDA’s publicly available data. Journal of the American Medical Informatics Association. 2016;23:596-600
  97. 97. Golder S, Norman G, Loke YK. Systematic review on the prevalence, frequency and comparative value of adverse events data in social media. British Journal of Clinical Pharmacology. 2015;80:878-888
  98. 98. Ghosh R, Lewis D. Aims and approaches of Web-RADR: A consortium ensuring reliable ADR reporting via mobile devices and new insights from social media. Expert Opinion on Drug Safety. 2015;14:1845-1853
  99. 99. Egberts TC, Smulders M, de Koning FH, Meyboom RH, Leufkens HG. Can adverse drug reactions be detected earlier? A comparison of reports by patients and professionals. British Medical Journal. 1996;313:530-531
  100. 100. Freifeld CC, Brownstein JS, Menone CM, Bao W, Filice R, Kass-Hout T, et al. Digital drug safety surveillance: Monitoring pharmaceutical products in twitter. Drug Safety. 2014;37:343-350
  101. 101. Abou TM, Rossard C, Cantaloube L, Bouscaren N, Roche G, Pochard L, et al. Analysis of patients’ narratives posted on social media websites on benfluorex’s (Mediator(R)) withdrawal in France. Journal of Clinical Pharmacy and Therapeutics. 2014;39:53-55
  102. 102. Nikfarjam A, Sarker A, O’Connor K, Ginn R, Gonzalez G. Pharmacovigilance from social media: Mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. Journal of the American Medical Informatics Association. 2015;22:671-681
  103. 103. Demner-Fushman D, Elhadad N. Aspiring to unintended consequences of natural language processing: A review of recent developments in clinical and consumer-generated text processing. Yearbook of Medical Informatics. 2016:224-233
  104. 104. Kheloufi F, Default A, Blin O, Micallef J. Investigating patient narratives posted on Internet and their informativeness level for pharmacovigilance purpose: The example of comments about statins. Thérapie. 2017;72:483-490
  105. 105. Pierce CE, Bouri K, Pamer C, Proestel S, Rodriguez HW, Van LH, et al. Evaluation of facebook and twitter monitoring to detect safety signals for medical products: An analysis of recent FDA safety alerts. Drug Safety. 2017;40:317-331
  106. 106. Duh MS, Cremieux P, Audenrode MV, Vekeman F, Karner P, Zhang H, et al. Can social media data lead to earlier detection of drug-related adverse events? Pharmacoepidemiology and Drug Safety. 2016;25:1425-1433
  107. 107. Sloane R, Osanlou O, Lewis D, Bollegala D, Maskell S, Pirmohamed M. Social media and pharmacovigilance: A review of the opportunities and challenges. British Journal of Clinical Pharmacology. 2015;80:910-920
  108. 108. White RW, Wang S, Pant A, Harpaz R, Shukla P, Sun W, et al. Early identification of adverse drug reactions from search log data. Journal of Biomedical Informatics. 2016;59:42-48
  109. 109. Colilla S, Tov EY, Zhang L, Kurzinger ML, Tcherny-Lessenot S, Penfornis C, et al. Validation of new signal detection methods for web query log data compared to signal detection algorithms used with FAERS. Drug Safety. 2017;40:399-408
  110. 110. Michel C, Scosyrev E, Petrin M, Schmouder R. Can disproportionality analysis of post-marketing case reports be used for comparison of drug safety profiles? Clinical Drug Investigation. 2017;37:415-422
  111. 111. Hochberg AM, Pearson RK, O’Hara DJ, Reisinger SJ. Drug-versus-drug adverse event rate comparisons: A pilot study based on data from the US FDA adverse event reporting system. Drug Safety. 2009;32:137-146
  112. 112. Svendsen K, Halvorsen KH, Vorren S, Samdal H, Garcia B. Adverse drug reaction reporting: How can drug consumption information add to analyses using spontaneous reports? European Journal of Clinical Pharmacology. 2018;74:497-504
  113. 113. Raschi E, De Ponti F. Drug utilization research and pharmacovigilance. In: Elsevier M, Wettermek B, Almarsdóttir AB, Andersen M, Benko R, Bennie M, Eriksson I, Godman B, Krska J, Poluzzi E, Taxis K, Vlahovic-Palcevski V, Stichele RV, editors. Drug Utilization Research: Methods and Applications. Chichester, UK: John Wiley & Sons; 2016
  114. 114. Raschi E, Poluzzi E, Salvo F, Pariente A, De Ponti F, Marchesini G, et al. Pharmacovigilance of sodium-glucose co-transporter-2 inhibitors: What a clinician should know on disproportionality analysis of spontaneous reporting systems. Nutrition, Metabolism, and Cardiovascular Diseases. 2018;28:533-542
  115. 115. Wisniewski AF, Bate A, Bousquet C, Brueckner A, Candore G, Juhlin K, et al. Good signal detection practices: Evidence from IMI PROTECT. Drug Safety. 2016;39:469-490
  116. 116. Raschi E, Poluzzi E, Koci A, Salvo F, Pariente A, Biselli M, et al. Liver injury with novel oral anticoagulants: Assessing post-marketing reports in the US Food and Drug Administration adverse event reporting system. British Journal of Clinical Pharmacology. 2015;80:285-293
  117. 117. Alonso A, MacLehose RF, Chen LY, Bengtson LG, Chamberlain AM, Norby FL, et al. Prospective study of oral anticoagulants and risk of liver injury in patients with atrial fibrillation. Heart. 2017;103:834-839
  118. 118. Douros A, Azoulay L, Yin H, Suissa S, Renoux C. Non-vitamin K antagonist oral anticoagulants and risk of serious liver injury. Journal of the American College of Cardiology. 2018;71:1105-1113
  119. 119. Alonso A, MacLehose RF, Chen LY, Bengtson LGS, Chamberlain AM, Norby FL, et al. Oral anticoagulants and liver injury: The threat of uncontrolled confounding. Heart. 2018;104:84
  120. 120. Raschi E, De Ponti F. Liver injury with direct-acting anticoagulants: Has the fog cleared? Heart. 2017;103:2010
  121. 121. Hauben M, Patadia V, Gerrits C, Walsh L, Reich L. Data mining in pharmacovigilance: The need for a balanced perspective. Drug Safety. 2005;28:835-842

Written By

Emanuel Raschi, Ugo Moretti, Francesco Salvo, Antoine Pariente, Ippazio Cosimo Antonazzo, Fabrizio De Ponti and Elisabetta Poluzzi

Reviewed: 06 July 2018 Published: 05 November 2018