Evaluating Research Efficiency in the U.S. Environmental Protection Agency (2008) / Chapter Skim
Currently Skimming:
5 Findings, Principles, and Recommendations
5 Findings, Principles, and Recommendations
Pages 58-68
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 58... ...
In the planning, execution, and review of research programs, efficiency should normally be subordinate to the criteria of relevance, quality, and effectiveness for reasons explained in Chapter 3. However, all federal programs should use efficient spending practices, and the committee suggests which aspects of efficiency can be measured in research programs and how that might best be done.
|
From page 59... ...
EPA and its Office of Research and Development (ORD) have a sound strategic planning architecture that provides a multi-year basis for the annual assessment of progress and milestones for evaluating research programs, including their efficiency.
|
From page 60... ...
This committee's recommendations are designed to further that aim. More broadly, the committee agrees that the research programs of federal agencies should be evaluated regularly, as are other programs of the federal government.
|
From page 61... ...
This integrated view is a reasonable starting point for the evaluation of research programs. Principle 3 The process efficiency of research should not be evaluated using outcome-based metrics.
|
From page 62... ...
The totality of those activities might be called portfolio management, a more familiar term that suggests linking research activities with strategic and multi-year plans. Sound portfolio management is the surest route to desired outcomes.
|
From page 63... ...
. A consistent theme of the present report is that for many research programs there can be no "outcome metrics"; that is true especially for core research, as discussed in Chapter 3.
|
From page 64... ...
. Research programs, especially programs of core or basic research, are unlikely to be able to respond "yes" to that question, because research managers cannot set baselines and targets for investigations whose outcomes are unknown.
|
From page 65... ...
.5 Evaluation of research should begin not with efficiency but with the criteria of relevance, quality, and effectiveness and should secondarily address efficiency only after these criteria have been reviewed. Recommendation 1 To comply with PART, EPA and other agencies should only apply quantitative efficiency metrics to measure the process efficiency of research programs.
|
From page 66... ...
should be revised to make it explicit that quantitative efficiency metrics should be applied only to process efficiency. Recommendation 2 EPA and other agencies should use expert-review panels to evaluate the investment efficiency of research programs.
|
From page 67... ...
Recommendation 3 The efficiency of research programs at EPA should be evaluated according to the same overall standards used at other agencies. EPA has failed to identify a means of evaluating the efficiency of its research programs that complies with PART to the satisfaction of OMB.
|
From page 68... ...
Because the framework of PART is virtually the same for all agencies and because the principles of scientific inquiry are virtually the same in all disciplines, the implementation of PART should be both consistent and equitable in all federal research programs. It should be noted that actual consistency is unlikely to be achieved in the vast and varied universe of government R&D programs, which fund extramural basic research, mission-driven intramural labs, basic research labs, construction projects, facilities operations, prototype development, and many other operations.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.