3
Risk Assessment Definition and Goals

This chapter addresses the definition of risk assessment proposed by the Office of Management and Budget (OMB). The definition is important because it determines which agency analyses are subject to the standards set forth in the bulletin. As discussed here, the committee finds that some departures from long-standing concepts could create confusion and controversy. The chapter also reviews the goals set forth in the bulletin. The goals are generally constructive but raise questions about the emphasis on efficiency rather than scientific quality.

The committee notes that the bulletin does not define risk, which lies at the core of “risk assessment.” Risk can be defined as a hazard, a probability, a consequence, or a combination of probability and severity of consequences. Although the bulletin hints at taking both probability and severity into account, it appears to treat risk primarily as the probability of adverse effect, which is an incomplete conceptualization of risk.

DEFINITION OF RISK ASSESSMENT

Section I of the bulletin defines risk assessment as “a scientific and/or technical document that assembles and synthesizes scientific information to determine whether a potential hazard exists and/or the extent of possible risk to human health, safety or the environment” (OMB 2006a, p. 23). The supplementary information explains that “for the purposes of this Bulletin, this definition applies to documents that could be used for risk assessment purposes, such as an exposure or hazard assess-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget 3 Risk Assessment Definition and Goals This chapter addresses the definition of risk assessment proposed by the Office of Management and Budget (OMB). The definition is important because it determines which agency analyses are subject to the standards set forth in the bulletin. As discussed here, the committee finds that some departures from long-standing concepts could create confusion and controversy. The chapter also reviews the goals set forth in the bulletin. The goals are generally constructive but raise questions about the emphasis on efficiency rather than scientific quality. The committee notes that the bulletin does not define risk, which lies at the core of “risk assessment.” Risk can be defined as a hazard, a probability, a consequence, or a combination of probability and severity of consequences. Although the bulletin hints at taking both probability and severity into account, it appears to treat risk primarily as the probability of adverse effect, which is an incomplete conceptualization of risk. DEFINITION OF RISK ASSESSMENT Section I of the bulletin defines risk assessment as “a scientific and/or technical document that assembles and synthesizes scientific information to determine whether a potential hazard exists and/or the extent of possible risk to human health, safety or the environment” (OMB 2006a, p. 23). The supplementary information explains that “for the purposes of this Bulletin, this definition applies to documents that could be used for risk assessment purposes, such as an exposure or hazard assess-

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget ment that might not constitute a complete risk assessment as defined by the National Research Council [NRC 1983]. This definition includes documents that evaluate baseline risk as well as risk mitigation activities” (OMB 2006a, p. 8). It is important to note that the bulletin’s definition of risk assessment is closely tied to which documents need to comply with the standards of the bulletin. That is, the applicability of the bulletin is intrinsically related to the definition of risk assessment because anything defined as a risk assessment will need to comply with the standards as indicated in Section II of the bulletin (“Applicability”), which states that “to the extent appropriate, all agency risk assessments available to the public shall comply with the standards of this Bulletin” (OMB 2006a, p. 23). A recurring theme in comments received by OMB on the bulletin from organizations, associations, and individuals concerned the definition of risk assessment. Of the 78 public comments submitted to OMB (OMB 2006b), 50 (64%) discussed the definition of risk assessment. Most of those comments mentioned that the proposed definition is too broad and may create confusion and other problems. Several agencies responding to the committee’s questions also pointed to potential confusion and the need for further clarification.1 The definition of risk assessment in the bulletin is extremely broad. Specifically, OMB defines risk assessment as a document. That characterization conflicts with standard risk assessment definitions. Risk assessment is a process from which documents can result. To define risk assessment as a document is problematic. It can capture many “documents” that are not risk assessment. More important, OMB defines risk assessment in such a way that its individual components, such as hazard assessment and exposure assessment, are inappropriately classified as “risk assessment.” Expanding the definition of risk assessment in such a way has a number of disadvantages: Hazard and exposure assessments are components of a risk assessment but do not in themselves constitute a risk assessment. A hazard assessment—which describes and assesses the nature of a hazard—and an exposure assessment—which estimates the expected intensity, frequency, and duration of an exposure—clearly are different from a risk 1 See Appendix E, pp. CPSC-2 to -3, DOE-8, HHS-A, HHS-7 to -12, DOL-4 to -5, DOT-6, and NASA-7.

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget assessment, which incorporates these components with hazard characterization or dose-response assessment to determine the likelihood and severity of an adverse effect or event given specified conditions. Equating risk assessment with components of risk assessment creates confusion by referring to different types of analyses with the same name. In addition, including hazard and exposure assessments would require application of the requirements of the bulletin to an extremely large number of documents, adding substantial time and resource burdens to the agencies (see “Costs” in Chapter 6 for further discussion of this issue). The committee emphasizes that although the technical requirements indicated in the proposed bulletin should not necessarily be applied to each component, the goals of higher quality and transparency should be met by all components of risk assessment. Previous NRC documents and other relevant documents (NRC 1983, 1989, 1993, 1994, 1996) use definitions of risk assessment that clearly differentiate risk assessment from its components. Similarly, the glossary of the 1997 Presidential/Congressional Commission on Risk Assessment and Risk Management (PCCRARM 1997) and the glossary of the Society for Risk Analysis (SRA 2003) include definitions of risk assessment that differentiate risk assessment from its components or “steps.” Uniform general guidelines may not be able to be issued for exposure assessment. Authors of the 1983 NRC report Risk Assessment in the Federal Government: Managing the Process concluded that “exposure guidelines, in contrast with guidelines for other risk assessment steps, are not now readily amenable to uniform application in various agencies,” and “the agencies have rather narrowly defined interests regarding exposure” (NRC 1983, p. 81). Several requirements of Sections IV and V of the bulletin are aimed at risk assessments and cannot be applied to exposure or hazard assessment or other components of risk assessment (for example, evaluation of risk reduction alternatives). Because it is not clear how those standards of the bulletin could be applied to hazard or exposure assessments, it also is not clear how the agencies could issue certificates of compliance for those documents. Some of the documents listed as examples of influential risk assessments in the supplementary information collect and summarize information from a variety of sources and studies and provide it in a format that is useful to both health professionals and the public. Many of the documents contain hazard identification, dose-response assessments, or

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget both but do not include exposure assessments or risk characterizations. Subjecting those documents to the requirements for risk assessment detailed in the bulletin could greatly delay release of important health information to the public. It is unclear whether the broad definition pertains to many safety guidelines that are now issued without going through a detailed risk assessment process. That could lead to delays in putting out important guidelines, warnings, and alerts. Examples include guidelines for healthcare workers on the handling of hazardous biologic materials, such as body fluids from HIV patients; guidelines for respirator fit testing; National Institutes of Health guidelines for research practices, particularly in relation to new therapies and technologies (for example, those on the use of recombinant-DNA products); and health information alerts or warnings, which may result from reports of adverse effects of a therapy or medication. At the public meeting for this committee, the Food and Drug Administration (FDA) representative warned that OMB’s definition would include most FDA safety alerts and that the risk assessment standards could delay the issuance of safety alerts regarding the adverse effects of drugs, medical devices, or foods. It also is not clear whether epidemiologic or toxicologic research used in risk assessments to identify factors that affect human health would now be classified as risk assessment and thus be subject to the standards in the bulletin. OMB appears to redefine risk assessment to include some aspects of risk mitigation, such as analysis of risk reduction measures to inform risk management decision-making. The bulletin and the supplementary information approach this point in different ways, creating the potential for inconsistent interpretation and implementation of the standards. Specifically, the bulletin refers to risk mitigation only in relation to regulatory analyses (see Section IV[7]), where this reference is appropriate, and not in the definition of risk assessment (see Section I), where such a reference would be a sharp departure from the long-established conceptual distinction between risk assessment and risk management. However, the supplementary information specifies that the definition of risk assessment “includes documents that evaluate baseline risk as well as risk mitigation activities” (OMB 2006a, p. 8)—an auxiliary definition that highlights the departure from the conceptual distinction. If the definition from the supplementary information is incorporated into the risk assessment definition, the bulletin would conflict with the 1983 NRC recom-

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget mendation, reinforced in numerous reports, to “take steps to establish and maintain a clear conceptual distinction between assessment of risk and consideration of risk management alternatives” (NRC 1983, p. 7). In making that recommendation, the 1983 NRC committee noted that experience shows that difficulties can arise from not having a clear distinction between those closely related, but different, aspects of setting regulatory standards. For example, if nonrisk factors, such as the expected economic or political consequences of proposed regulatory action, were seen to affect either the interpretation of scientific information or the choice of default options, the credibility of the assessment inside and outside an agency could be compromised, and this might reduce the legitimacy of the risk management decision itself. Since the publication of the 1983 NRC report, there has been some debate as to how much one can separate risk management from risk assessment. Nevertheless, the 1994 NRC report Science and Judgment in Risk Assessment stated that “protecting the integrity of the risk assessment, while building more productive linkages to make risk assessment more accurate and relevant to risk management, will be essential as the agency [EPA] proceeds to regulate the residual risks of hazardous air pollutants” (NRC 1994, p. 260). Furthermore, the 1996 NRC report Understanding Risk stated that “what is needed for successful characterization of risk must be considered at the very beginning of the process and must to a great extent drive risk analysis. If a risk characterization is to fulfill its purpose, it must (1) be decision driven, (2) recognize all significant concerns, (3) reflect both analysis and deliberation, with appropriate input from the interested and affected parties, and (4) be appropriate to the decision” (NRC 1996, p. 16). Thus, the committee believes that risk assessors and risk managers should talk with each other; that is, a “conceptual distinction” does not mean establishing a wall between risk assessors and risk managers. Indeed, they should have constant interaction. However, the dialogue should not bias or otherwise color the risk assessment conducted, and the activities should remain distinct; that is, risk assessors should not be performing risk management activities. GOALS The bulletin and the supplementary information lay out five goals, also called “aspirational goals” (see Table 3-1). The goals can be seen as having to do with both the efficiency and the quality of a risk assessment.

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget TABLE 3-1 Goals for Risk Assessment as Stated in Bulletin and Clarified in Supplementary Information Goal Bulletin Description Supplementary Description 1 “The objectives of an assessment shall be a product of an iterative dialogue between the assessor(s) and the agency decision maker(s)” (OMB 2006a, p. 23) Goal related to problem formulation: “there will be many choices regarding the objectives, scope and content of the assessment, and an iterative dialogue will help ensure that the risk assessment serves its intended purpose and is developed in a cost-effective manner” (OMB 2006a, p. 10) 2 “The scope and content of the risk assessment shall be determined based on the objectives of the assessment and best professional judgment, considering the benefits and costs of acquiring additional information before undertaking the assessment” (OMB 2006a, p. 23). Goal related to completeness: “there is often a tension between…completeness in the scientific sense and…a well-defined scope that limits the inquiry to a set of practical, tractable and relevant questions.” “The scope…should reflect a balance between the desire for scientific completeness and the need to provide relevant information to decision makers” (OMB 2006a, p. 10). 3 “The type of risk assessment prepared shall be responsive to the nature of the potential hazard, the available data and the decision needs” (OMB 2006a, p. 23). Goal related to effort expended: “level of effort should be commensurate with the importance of the risk assessment…nature of the potential hazard, the available data, and the decision needs” (OMB 2006a, p. 11). 4 “The level of effort put into the risk assessment shall be commensurate with the importance of the risk assessment” (OMB 2006a, p. 23). Goal related to resources expended: “take into account the importance of the risk assessment in gauging the resources, including time and money, required to meet the requirements of this Bulletin” (OMB 2006a, p. 11). 5 “The agency shall follow appropriate procedures for peer review and public participation in the process of preparing the risk assessment” (OMB 2006a, p. 23). Goal related to peer review and public participation: “when a draft assessment is made publicly available for comment or peer review, the agency is required to clarify that the report does not represent the official view of the federal government.” “Public comments play an important role in helping to inform agency deliberations…when people are engaged early in the process, the public typically has an easier time concurring with government documents and decisions which may affect them” (OMB 2006a, p. 11).

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget All federal-agency risk assessments are subject to OMB’s Information Quality Guidelines (67 Fed. Reg. 8452 [2002]), which require utility, objectivity, and integrity. As a first approximation, goals 1, 3, and 5 focus on quality, and goals 2 and 4 on efficiency. Objectivity and integrity are addressed by the five goals to the extent that peer review and public participation contribute to these attributes. Goal related to problem formulation (1). This is principally the goal of good communication between the risk assessor and the agency decision-maker or client. Although the emphasis is on an iterative discussion in the bulletin, the supplementary information adds a cost-effectiveness component. Goal related to completeness (2). This is principally the goal of balancing the completeness of a risk assessment in providing relevant information to the agency decision-maker with the decision-maker’s immediate needs. The goal calls for a cost-benefit balancing of scientific completeness with practical usefulness in making decisions in keeping with OMB’s Information Quality Guidelines (67 Fed. Reg. 8452 [2002]). Having the scope and content linked to the assessment seems logical, and one would hope that this recommendation is already implicit in most risk assessments. The supplementary information raises a number issues about satisfying the goal. For example, the supplementary information refers to a well-defined scope as one that “limits the inquiry to a set of practical, tractable and relevant questions” (OMB 2006a, p. 10). However, how should the properties of practical, tractable, and relevant be established? In addition, the supplementary information indicates that “the scope of an assessment should reflect a balance between the desire for scientific completeness and the need to provide relevant information to decision makers” (OMB 2006a, p. 10). One might expect that decision-makers would want nothing less than scientifically relevant information. What constitutes scientifically complete information might be a contentious issue. A risk assessment might be conducted on a new class of hazards or a new engineered system before extensive data are available. Then, the question would be, How does the magnitude of the uncertainty affect the policy decision? Goal related to effort expended (3). In what may be only an oversight, this goal differs somewhat between the bulletin and the supplementary information. The goal according to the bulletin addresses the type of risk assessment performed, whereas the goal according to the supplementary information addresses effort and resources. These are not

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget contradictory, but different. The former seems to be what was intended. But the meaning of “the type of risk assessment prepared” is not self-evident and is not clarified in the bulletin or the supplementary information. Goals related to resources expended (4). This goal is a corollary of the aforementioned goal related to completeness. This goal says that the time and money invested in the risk assessment should be commensurate with the use to which the results are to be put, that is, the “importance of the risk assessment.” That is redundant in light of goal 3 in the supplementary information; goal 3 might be better represented by its description in the bulletin than by that in the supplementary information. Goal related to peer review and public participation (5). This goal involves principally adequate review of the product of the risk assessment. Although the bulletin suggests peer review and public participation in the “process of preparing the risk assessment,” the supplementary information emphasizes the product. Taken as a whole, the five goals say, in essence, that a risk assessment should be tailored to the narrow need for which it is undertaken; balanced in scope, time, and cost with the importance of the issue; and peer-reviewed and subject to public participation. To the extent that current practice is inadequate in coordinating the focus and scope of a risk assessment with the objectives of the agency decision-maker, and to the extent that the outcomes of a risk assessment are inadequately reviewed and not subject to public comment, goals 1 and 5 are beneficial in promoting higher-quality risk assessments. Whether those conditions exist is a separate question. A risk assessment usually involves incomplete data, scientific uncertainty, and the need for expert judgment. The pressure to narrow the scope becomes a pressure to give inadequate attention to those complications. Thus, the goals may lead to less expensive and quicker risk assessments, but they do not necessarily lead to higher-quality risk assessments. The dominating theme of the bulletin and the supplementary information is improving the quality of risk assessments undertaken by federal agencies, but the stated goals do not all support this theme. The goals stated in the bulletin and the supplementary information emphasize efficiency in the conduct of risk assessment activities more than quality.

OCR for page 26
Scientific Review of the Proposed Risk Assessment Bulletin from the Office of Management and Budget REFERENCES NRC (National Research Council). 1983. Risk Assessment in the Federal Government: Managing the Process. Washington DC: National Academy Press. NRC (National Research Council). 1989. Improving Risk Communication. Washington DC: National Academy Press. NRC (National Research Council). 1993. Issues in Risk Assessment, Volumes I, II and III. Washington DC: National Academy Press. NRC (National Research Council). 1994. Science and Judgment in Risk Assessment. Washington DC: National Academy Press. NRC (National Research Council). 1996. Understanding Risk: Informing Decisions in a Democratic Society. Washington DC: National Academy Press. OMB (U.S. Office of Management and Budget). 2006a. Proposed Risk Assessment Bulletin. Released January 9, 2006. Washington, DC: Office of Management and Budget, Executive Office of the President [online]. Available: http://www.whitehouse.gov/omb/ inforeg/proposed_risk_assessment_bulletin_010906.pdf [accessed Oct. 11, 2006]. OMB (Office of Management and Budget). 2006b. Comments on Proposed Risk Assessment Bulletin. Office of Management and Budget, Washington, DC [online]. Available: http://www.whitehouse.gov/omb/inforeg/comments_rab/list_rab2006.html [accessed Oct. 13, 2006]. PCCRARM (Presidential/Congressional Commission on Risk Assessment and Risk Management). 1997. Glossary. Pp. 153-157 in Risk Assessment and Risk Management in Regulatory Decision-Making, Vol. 2. Washington, DC: U.S. Government Printing Office [online]. Available: http://www.riskworld.com/Nreports/1997/risk-rpt/volume2/pdf/v2epa.PDF [accessed Oct. 3, 2006]. SRA (Society for Risk Analysis). 2003. Risk Analysis Glossary [online]. Available: http://www.sra.org/resources_glossary_p-r.php [accessed Oct. 13, 2006].