Evaluating Research Efficiency in the U.S. Environmental Protection Agency (2008) / Chapter Skim
Currently Skimming:
Appendix B: Evaluating the Efficiency of Research and Development Programs at the Environmental Protection Agency: Workshop Summary
Appendix B: Evaluating the Efficiency of Research and Development Programs at the Environmental Protection Agency: Workshop Summary
Pages 75-85
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 75... ...
PRESENTATION BY OFFICE OF MANAGEMENT AND BUDGET STAFF The rationale for using PART is that taxpayers deserve to have their money spent wisely to create the maximal benefit. OMB developed PART in 2002 because the reporting process associated with GPRA was losing momen 75
|
From page 76... ...
" A way to measure efficiency is required for a "yes" response. Question 4.3 asks, "Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?
|
From page 77... ...
. The PART Appeals Board ruled that EPA could use publications on condition that the WQRP develop an "outcome-oriented efficiency measure." That agreement helped EPA achieve a "green" rating in March 2007 on the PMA.
|
From page 78... ...
EPA considered other measures, such as research vs overhead and citations per dollar invested. For ecologic research, it tried a "version of EVM," comparing projected costs for a long-term goal with actual costs in the context of scheduled output for the goal.
|
From page 79... ...
As one example, the Energy Efficiency and Renewable Energy program uses an overhead metric but only for operational and construction programs, not for R&D. The program also experimented with using the relationship between the corporate program-management line and the total program R&D budget but found it to be "ungameable." For R&D, DOE uses the "alternative efficiency measure" of peer review for all portfolios every 2nd or 3rd year.
|
From page 80... ...
PRESENTATION BY NATIONAL INSTITUTES OF HEALTH STAFF NIH created an Office of Portfolio Analysis and Strategic Initiatives to examine systemic assessments and practice the "science of science management." Two general points emerged: the difficulty of using a business-model approach to measure efficiency in science and NIH success in using PART on research and research-support activities. Some 99% of the NIH portfolio had been "PART-ed"; 95% of programs were rated effective, and the other 5% were rated moderately effective.
|
From page 81... ...
In business, risk is usually undesirable; in research, whether in the private sector or the public sector, high-risk projects are strongly associated with breakthrough innovative outcomes. Nor does the business model capture the null hypothesis, which states that a negative result gives valuable information.
|
From page 82... ...
Eight NIOSH programs are being studied by other ad hoc committees of the National Research Council for relevance, impact, and emerging issues. NIOSH research results can be divided into outputs, intermediate outcomes, and outcomes: • Outputs include peer-reviewed publications, NIOSH publications, communications to regulatory agencies or Congress, research methods, control technologies and patents, and training and information products.
|
From page 83... ...
. PRESENTATION BY IBM STAFF The company uses efficiency measures for some kinds of activities, such as • Return on investment in the summer internship program and graduate fellowship program: What percentage of the recipients return as regular IBM research employees?
|
From page 84... ...
The resulting metric is used to evaluate the overall value of R&D programs and the current budget focus. Although the reported numbers constitute a lagging indicator, the company tries to encourage business case development and projects expected financial impact on current and future projects whenever possible.
|
From page 85... ...
Appendix B 85 • For program review: - Aggregate R&D expenditures by laboratory group or identifiable programs and publish value capture or success rate for each annually.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.