National Academies Press: OpenBook

Ethical and Scientific Issues in Studying the Safety of Approved Drugs (2012)

Chapter: Appendix D: Decision Conferencing and Multicriteria Decision Analysis

« Previous: Appendix C: Open Session Agendas
Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×

D

Decision Conferencing and Multicriteria Decision Analysis





Benefit–risk assessment characterizes information regarding estimates of benefits, estimates of risks, and the severity and comparability among health endpoints associated with benefits and risks. Much has been published about methods for quantitative benefit–risk assessment (Coplan et al., 2011; Guo et al., 2010). Such assessments are widely used in decision-making contexts, particularly with regard to environmental regulation, and the mode of their particular application varies (NRC, 1994, 2009). The committee does not wish to prescribe a particular method for conducting benefit–risk assessment, particularly inasmuch as formal quantitative approaches are likely to be used only in select circumstances when disagreements about potential regulatory actions arise. Instead, the committee highlights in this appendix two decision tools that incorporate key considerations of benefit–risk assessment relevant to regulatory decision-making: decision conferencing, which is a social process intended to engage all the relevant stakeholders to provide scientific judgments at key points in the decision-making process, and multicriteria decision analysis (MCDA), which is a technical model for making decisions that have multiple objectives (Walker et al., 2005). Decision conferencing and MCDA have been used in other contexts and are offered here as examples of potentially useful approaches that integrate analytic and deliberative processes for in-depth evaluation and informed assessments of the benefit–risk balance of approved drugs (Phillips, 2006; Walker et al., 2005). Both processes, when used in a transparent way that documents the inputs into the process, can help to identify the underlying sources of scientific disagreements that are discussed in Chapter 3.

Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×

DECISION CONFERENCING

Decision conferencing is a tool that brings key experts and stakeholders together to generate a shared understanding of a challenge, create a sense of common purpose, and gain commitment to a way forward (Phillips, 2006) to aid evaluation and assessment of the benefits and risks associated with a drug (Walker et al., 2005). Decision conferencing has four basic elements: attendance by key stakeholders (for example, regulators; methodologists who have expertise in design, conduct, and analysis of observational studies and clinical trials; decision scientists; physicians who have relevant clinical expertise; patients; and the public); impartial facilitation to guide discussions of estimates of benefit and risk, degree of uncertainty, and values and preferences for health endpoints; on-the-spot modeling with continuous display of the developing model; and an interactive and iterative group process (Phillips, 2006). When quantitative benefit–risk assessments are needed in situations in which disagreements about appropriate regulatory action arise, a neutral facilitator can guide the group through the stages of discussing the issues, developing models for evaluating the issues, and eliciting assumptions about the quality of evidence regarding benefits and risks and about underlying ethical values and preferences for health endpoints without contributing to the content of discussions. Although quantitative estimates resulting from benefit–risk assessment may appear to provide objective information about optimal regulatory decisions, assumptions used in the model are often based on individual judgments about the quality of evidence related to benefits and risks, as discussed in Chapter 3, or based on individual values or preferences for different health endpoints. Using a process like decision conferencing helps to frame the issues and identify the relevant data and evidentiary gaps to guide later data-gathering and thereby improves the efficiency and transparency of the benefit–risk assessment process (Phillips, 2006).

The decision-conferencing process could be integrated into FDA’s current processes and requirements under the Federal Advisory Committee Act,1 and it is similar to initiatives that FDA currently has planned. For example, FDA is working with stakeholders, using faculty of the George Washington University as facilitators, to develop guidance material for approving obesity drugs (McCaughan, 2011). Decision conferencing in benefit–risk assessment has four advantages:

•  It facilitates and focuses thinking about a complex challenge.

•  It helps to establish common purpose among participants and commitment to move forward.

•  It promotes transparency and a shared understanding of how stakeholders define benefits, risks, degree of uncertainty, ethical values regarding outcomes, and potential regulatory actions for managing risk.

image

1Federal Advisory Committee Act, PL 920-463, 86 Stat. 770 (1972).

Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×

•  Because decision conferencing can generate both the outcome and a description of the process that leads to the outcome, it can improve stakeholders’ and the public’s understanding of a regulatory decision.

MULTICRITERIA DECISION ANALYSIS

MCDA is a set of methods designed to bring together evaluations of options on different criteria into one overall decision (CHMP, 2008). There are many variants of MCDA, some of which may be adapted to consider the uncertainty of the decision-maker (Linkov and Seager, 2011), and MCDA can be used to provide either a qualitative or a quantitative assessment. The basic methods of MCDA are scoring and weighting (CHMP, 2008). Scoring involves the process of assigning numerical values to options according to particular criteria. Weighting ensures the comparability of the numerical values assigned to all criteria, which allows comparison of different health states with a single metric (Linkov and Seager, 2011). The weights assigned to scores reflect the relative importance of the underlying criteria for the benefit–risk assessment outcome (Walker et al., 2005).

MCDA uses four steps (Linkov and Seager, 2011):

1.  Defining the problem and the decision context.

2.  Identifying stakeholders, decision-makers, assessment criteria (for example, health outcomes of interest), and the relative importance of different health outcomes.

3.  Defining and assessing management alternatives whereby the effects of different regulatory decisions on each criterion, or health outcome, are assessed.

4.  Allowing for variability in weighting of different criteria and accounting for the stochastic nature of data through the use of probabilistic sensitivity analysis to provide a rank order of different alternatives for distinct stakeholder groups.

Those four steps help to ensure that all participants understand and are in agreement about the need for a regulatory decision, the criteria by which benefit–risk balance is judged, the evidence and its uncertainties, the values and preferences of different stakeholders, and the consequences of different regulatory decisions. The outputs or information synthesis of the benefit–risk assessment stage of the framework should include model inputs and model outputs (Linkov and Seager, 2011). The model inputs would include estimates of benefits, estimates of risks, the degree of uncertainty, and preferences for health outcomes based on ethical values. The model outputs may be characterized quantitatively, for example, the benefits of a drug outweigh the risks 85 percent of the time; or qualitatively, for example, there is clear and convincing evidence that the benefits of a drug outweigh its risks. The synthesis should also discuss any uncertainty

Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×

analyses that were conducted and the process by which the benefit–risk assessment was performed—that is, a description of the “decision conferencing” or other process that FDA used to seek and include stakeholder input as necessary. Useful information includes statements of who provided the inputs and who moderated the process. Both the process and the outcome should be documented as a way to set the stage for understanding the regulatory decision. The outcome of the assessment process, whether quantitative or qualitative, becomes the evidentiary basis of the regulatory decision-making that is at the heart of the next stage—benefit–risk management.

LIMITATIONS OF USING MODELING APPROACHES FOR BENEFIT AND RISK ASSESSMENTS

MCDA is a useful tool for benefit–risk assessment, but it has its limitations, both in its own right and as an aid to regulatory decision-making. Using a quantitative MCDA approach to benefit–risk assessment forces participants to be explicit about how they evaluate existing evidence regarding the effectiveness of a drug and its associated harms and about the degree of uncertainty regarding benefits and risks. Such a quantitative assessment, however, can obscure underlying interpretations of scientific findings. The interpretations should be explicitly described, and decision conferencing can mitigate some concerns by describing differences in underlying assumptions. In addition, quantification of intangible factors, such as a patient’s preferences that might be based on various degrees of dread for different diseases, may be difficult in MCDA models although relevant for the decision-making process.

Preferences regarding the relative importance and severity of health states associated with the disease or the treatment may also vary widely among stakeholders and could potentially be obscured by using modeling approaches. However, decision conferencing can serve as a useful tool for conducting MCDA by explicitly describing stakeholders’ values and preferences and how they may affect regulatory decisions.

Even with the aid of social and technical decision tools, benefit–risk assessment is unavoidably limited by the quality and quantity of available evidence. Uncertainty analysis may identify the key information needed to support a particular regulatory action, but such data may not be available in the time needed to address the public health issue at hand. The available data may lead to multiple interpretations and contribute to decision-making gridlock. That delay could compound limitations of the resources and expertise available in FDA to conduct benefit–risk assessments and result in a backlog. Such risk-assessment backlogs have occurred in the Environmental Protection Agency (NRC, 2009). If tools like decision conferencing and MCDA are used to enhance transparency, the rationale for regulatory decisions can at least be understood by stakeholders even if disagreements about optimal regulatory action remain. The transparency

Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×

allows stakeholders to have a shared understanding of differences in scientific assessment of preferences regarding health outcomes, which can help to determine whether additional data are needed and whether those data will meet thresholds for influencing future regulatory action. Decision conferencing and MCDA, however, may not be appropriate for every situation, and benefit–risk assessments should be scalable to the severity and scope of the particular public health concern at issue, the level of controversy surrounding it, and FDA resource constraints.

REFERENCES

CHMP (Committee for Medical Products for Human Use). 2008. Reflection paper on benefit-risk assessment methods in the context of the evaluation of marketing authorisation applications of medicinal products for human use. London, UK: European Medicines Agency.

Coplan, P. M., R. A. Noel, B. S. Levitan, J. Ferguson, and F. Mussen. 2011. Development of a framework for enhancing the transparency, reproducibility and communication of the benefit-risk balance of medicines. Clinical Pharmacology & Therapeutics 89(2):312-315.

Guo, J. J., S. Pandey, J. Doyle, B. Bian, Y. Lis, and D. W. Raisch. 2010. A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. Value Health 13(5):657-666.

Linkov, I., and T. P. Seager. 2011. Coupling multi-criteria decision analysis, life-cycle assessment, and risk assessment for emerging threats. Environmental Science and Technology 45(12):5068-5074.

McCaughan, M. 2011. FDA rethinking weight loss standards: A test case for patient-focused drug development. “The Pink Sheet” Daily, June 6, 2011.

NRC (National Research Council). 1994. Science and judgment in risk assessment. Washington, DC: National Academy Press.

NRC. 2009. Science and decisions: Advancing risk assessment. Washington, DC: The National Academies Press.

Phillips, L. D. 2006. Decision conferencing. Operational research working papers, LSEOR 06.85. London, UK: Operational Research Group, Department of Management, London School of Economics and Political Science.

Walker, S., L. Phillips, and M. Cone. 2005 (unpublished). Benefit-risk assessment model for medicines: Developing a structured approach to decision making.

Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×

This page intentionally left blank.

Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×
Page 255
Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×
Page 256
Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×
Page 257
Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×
Page 258
Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×
Page 259
Suggested Citation:"Appendix D: Decision Conferencing and Multicriteria Decision Analysis." Institute of Medicine. 2012. Ethical and Scientific Issues in Studying the Safety of Approved Drugs. Washington, DC: The National Academies Press. doi: 10.17226/13219.
×
Page 260
Next: Appendix E: Benefit and Risk Assessment and Management Plan Document Template »
Ethical and Scientific Issues in Studying the Safety of Approved Drugs Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $44.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

An estimated 48 percent of the population takes at least one prescription drug in a given month. Drugs provide great benefits to society by saving or improving lives. Many drugs are also associated with side effects or adverse events, some serious and some discovered only after the drug is on the market. The discovery of new adverse events in the postmarketing setting is part of the normal natural history of approved drugs, and timely identification and warning about drug risks are central to the mission of the Food and Drug Administration (FDA). Not all risks associated with a drug are known at the time of approval, because safety data are collected from studies that involve a relatively small number of human subjects during a relatively short period.

Written in response to a request by the FDA, Ethical and Scientific Issues in Studying the Safety of Approved Drugs discusses ethical and informed consent issues in conducting studies in the postmarketing setting. It evaluates the strengths and weaknesses of various approaches to generate evidence about safety questions, and makes recommendations for appropriate followup studies and randomized clinical trials. The book provides guidance to the FDA on how it should factor in different kinds of evidence in its regulatory decisions.

Ethical and Scientific Issues in Studying the Safety of Approved Drugs will be of interest to the pharmaceutical industry, patient advocates, researchers, and consumer groups.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!