Evaluating Research Efficiency in the U.S. Environmental Protection Agency (2008) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 3-10
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 3... ...
oversaw the appointment of the Committee on Evaluating the Efficiency of Research and Development Programs at the U.S. Environmental Protection Agency and charged it to answer the following questions: • What efficiency measures are currently used for EPA R&D programs and other federally funded R&D programs?
|
From page 4... ...
In its discussion the committee uses the terms inputs, outputs, and outcomes as defined by OMB, except as modified and discussed below: • Inputs are agency resources -- such as funding, facilities, and human capital -- that support research. • Outputs are activities or accomplishments delivered by research programs, such as research findings, papers published, exposure methods developed and validated, and research facilities built or upgraded.
|
From page 5... ...
Many of the process-efficiency metrics proposed by agencies other than EPA have been accepted by OMB, but several similar metrics proposed by EPA have not been accepted. Metrics that typically have been proposed or used by the federal agencies address only a small piece of a research program, and none attempts a comprehensive program evaluation.
|
From page 6... ...
• Ultimate outcomes usually depend on actions taken by others. The PART guidance urges agencies to develop outcome-based efficiency metrics,3 even though the PART questions do not specifically refer to such metrics and no agencies have been able to develop them for research programs.
|
From page 7... ...
Earned Value Management (EVM) is a quantitative tool that can track aspects of research programs against milestones.6 Moreover, to facilitate the evaluation process, the committee recommends modifying OMB's framework of results to include the category of intermediate outcomes, as distinguished from ultimate outcomes.
|
From page 8... ...
Expert review provides an independent forum for evaluation of research and complements the efforts of program managers in reviewing research activities and judging them against multi-year plans and anticipated outcomes. The expert-review panel can use intermediate outcomes to focus on key steps in the progress of any research program and to fill gaps in the spectrum of research results between outputs and ultimate outcomes.
|
From page 9... ...
Although the examiners cannot be expected to bring scientific expertise to their discussions with program managers, they should bring an understanding of the research process as it is performed in the context of federal agencies. OMB decisions about whether to accept or reject metrics for evaluating the efficiency of research programs have been inconsistent.
|
From page 10... ...
The primary goal of research is knowledge, and the development of new knowledge depends on so many conditions that its efficiency must be evaluated in the context of quality, relevance, and effectiveness in addressing current priorities and anticipating future R&D questions. The criterion of relevance and timely application of the outputs from R&D in ORD and in certain program offices to the regulatory process is particularly important at an agency like EPA.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.