OpenVigil Cave-at document

Version 1.0 (2011-11-27,

The practical usage of FDA FOI AERS data (and in principal pharmacovigilance data of other sources, too) is limited by the following methodical problems and shortcomings of the spontaneous reporting system:

consequences for data analyses
under-reporting In several jurisdictations health care professionals are not legally bound to file adverse events. Overlooking adverse events or non-reporting because of heavy workload often lead to under-reporting which is estimated to range between 1:10 to 1:100. Absolute numbers of cases might range 10-100 times higher. Cases for rarely used drugs might be missing.
The act of reporting and the choice of a primary and secondary suspected drug causing an event is dependent how important and plausible this issue appears to a physicians or patient. Different overview of literature could skew the number of reports per drug or per adverse event.
More extensivly used drugs have higher total numbers of records.
Queries should not always rely on the item that specifies which drugs are suspected to cause the reaction (DRUG.ROLE_COD in the AERS database). This is particulary important for signal detection which aims to discover relations hereto unknown.
missing denominator
The total amount of use for a drug (e.g., defined daily dose (DDD) or total number of applications) is not gathered in traditional pharmacovigilance data. Any normalizations or relations to the real world (e.g., odds ratio, risks) are complicated by this.
A rough estimate of drug usage and therefore substitute of DDD might be the total number of reports which include this drug.
wrong data
Reporters might accidentally use wrong formfields for items or mix up cases.
Sanitizing the data might catch some of these cases.
missing data
Reporters might not have all necessary data available or they cannot afford the time of entering all avilable data due to their workload.
Some important data (e.g., magnesium level on Torsades de pointes onset) are not gathered in traditional pharmacovigilance data.
These records with missing data can be filtered out or some kind of extrapolation might be applied.
duplicate or multiplicate records
A report might be reported by the sponsor of a trial, the affected participant and his general practitioner.
Reports might be sent to a domestic and a foreign database and consequently be reported in duplicate to larger multi-national databases (e.g., VigiBase).
Checks for records with different case numbers but the same age, sex, date of onset of adverse event and other database items can catch these kind of multipicates.
Besides formal inconsistency (e.g., the gas xenon cannot be applied by intravenous route), the mechanism needs to be explained (e.g., some licensed lipid lowering drugs (e.g., orlistat) are connected to pharmacologically unexplainable adverse reactions (i.e., influenza)).
Health care professionals should browse the list of case reports and manually correct inplausibilities and inconsistencies. This implies that any statistical findings cannot be used in legal proceedings.

Furthermore, FDA FOI AERS data is not completely sanitized and does not follow a single naming scheme for drugs (like WHO-DD or XEVMPD). It does, however, use MedDRA to code the adverse event. Be sure to manually sanitize the patient age coding and dosages if you rely on these data for an anlysis!