decrease in benefit, or both, for either the general population or a definable subgroup. Second is how time-urgent is the need for a regulatory response, based on the nature of the safety signal. The third involves how large the change in risks or benefits must be, on both relative and absolute scales, to justify a regulatory response. Fourth is what the other causes of a given adverse event (or failure of benefit) might be, and how strongly they are predictive. Fifth is the quality of data likely to be gathered as part of any given design on drug exposure, outcomes, confounders and other relevant patient, disease or contextual characteristics. Sixth is a judgment of how study design, conduct or context is likely to affect the transportability of the study results. Seventh is what the logistical requirements of a design will be, including data access, cost and feasibility. Finally, there are considerations of ethical burden, consent, confidentiality, and study oversight. These factors can lead to the choice of either a single design type or a combination of studies with counterbalancing strengths and weaknesses.

With the above considerations in mind, the committee made some general observations about the strengths and weaknesses of specific designs. The RCT is considered the gold standard for studies of a drug’s benefits because of the ability of randomization to control for potential biases and confounders, both known and unknown. Although the committee agrees that a well-conducted, high-quality RCT has many theoretical advantages over other study designs, it also recognizes that what can be achieved in practice in assessing safety endpoints can fall short of the ideal. Noncompliance, cross-over and dropout, limitations in study size or duration, failure of the study population or procedures to adequately represent circumstances in the general population of users, and the realization that safety endpoints are sometimes unforeseeable and cannot always be specified in advance can decrease the advantages of RCTs over observational studies for evaluating the risks posed by approved drugs. In many cases, the latter may provide estimates closer to the actual risks in the target population if one considers the combination of bias, precision, and transportability of results.

In addition, because RCTs alter a patient’s clinical experience, they may entail more ethical complications than observational studies. (That said, as part of the consent process, the information patients receive about benefits and risks of study treatment options, as well as alternative treatments, may be more complete than a typical health provider supplies.) Other disadvantages of RCTs are the cost and time required to conduct such studies; the duration of studies is particularly problematic when an urgent public health question needs to be answered. An advantage of an RCT, however, is the ability to ascertain moderate relative risk elevations of common outcomes with confidence. A small relative risk (for example, RR <1.5) increase in a common outcome (for example, MI) may represent a very large absolute increase in risk with great public health importance. The adequacy of confounding control in many observational designs may not be sufficient to estimate such risk elevations with high confidence. Additionally, RCTs have the potential ability to assess both benefit and risk in the same group

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement