Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 128
Evaluating Research Efficiency in the U.S. Environmental Protection Agency Appendix I PART Guidance on Efficiency Measures1 DESCRIPTION OF EFFICIENCY MEASURES FOR PART Efficiency Measures While outcome measures provide valuable insight into program achievement, more of an outcome can be achieved with the same resources if an effective program increases its efficiency. The President’s Management Agenda (PMA) Budget and Performance Integration (BPI) Initiative encourages agencies to develop efficiency measures. Sound efficiency measures capture skillfulness in executing programs, implementing activities, and achieving results, while avoiding wasted resources, effort, time, and/or money. Simply put, efficiency is the ratio of the outcome or output to the input of any program. Because they relate to costs, efficiency measures are likely to be annual measures. Outcome efficiency measures: The best efficiency measures capture improvements in program outcomes for a given level of resource use. Outcome efficiency measures are generally considered the best type of efficiency measure for assessing the program overall. For example, a program that has an outcome goal of “reduced energy consumption” may have an efficiency measure that shows the value of energy saved in relation to program costs. Output efficiency measures: It may be difficult to express efficiency measures in terms of outcomes. In such cases, acceptable efficiency measures could focus on how to produce a given output level with fewer resources. However, this approach should not shift incentives toward quick, low-quality methods that could hurt program effectiveness and desired outcomes. 1 OMB 2006.
OCR for page 129
Evaluating Research Efficiency in the U.S. Environmental Protection Agency Meaningful efficiency measures consider the benefit to the customer and serve as indicators of how well the program performs. For example, reducing processing time means little if error rates increase. A balanced approach is required to enhance the performance of both variables in pursuit of excellence to customers. In these instances, one measure (e.g., increase in customer satisfaction) may be used in conjunction with another complementary measure (e.g., reduction in processing time). In all cases, efficiency measures must be useful, relevant to program purpose, and help improve program performance. An efficiency measure for a Federal program tracks the ratio of total outputs or outcomes to total inputs (Federal plus non-Federal). Leveraging program resources can be a rational policy decision, as it leads to risk or cost sharing; however, it is not an acceptable efficiency measure, because the leveraging ratio of non-Federal to Federal dollars represents only inputs. Although increasing the amount leveraging in a program may stretch Federal program dollars, this does not measure improvements in the management of total program resources, systems, or outcomes. 3.4: Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution? Purpose: To determine whether the program has effective management procedures and measures in place to ensure the most efficient use of each dollar spent on program execution. Elements of Yes: A Yes answer needs to clearly explain and provide evidence of each of the following [see Box I-1]: The program has regular procedures in place to achieve efficiencies and cost effectiveness. The program has at least one efficiency measure with baseline and targets. BOX I-1 Measures and PARTWeb To receive a Yes answer, the program must include at least one efficiency measure, baseline data/estimates, and targets in the Measures screen in PARTWeb. Only measures that meet the standards for a Yes should be entered in PARTWeb. Please ensure that the proper characterization of measures is selected in PARTWeb (that is “efficiency”). Make sure to indicate the term of the measure in PARTWeb too (that is, long-term, annual, or long-term/annual).
OCR for page 130
Evaluating Research Efficiency in the U.S. Environmental Protection Agency There are several ways to demonstrate that a program has established procedures for so improving efficiency. For example, a program that regularly uses competitive sourcing to determine the best value for the taxpayer, invests in IT with clear goals of improving efficiency, etc., could receive a Yes. A de-layered management structure that empowers front line managers and that has undergone competitive sourcing (if necessary) would also contribute to a Yes answer. For mandatory programs, a Yes could require the program to seek policies (e.g., through review of proposals from States) that would reduce unit costs. Also consider if, where possible, there is cross-program and inter-agency coordination on IT issues to avoid redundancies. The program is not required to employ all these strategies to earn a Yes. Rather, it should demonstrate that efforts improving efficiency are an established, regular part of program management. An efficiency measure can be the per-unit cost of outcomes or outputs, a timing target, and other indicator of efficient and productive processes germane to the program. Efficiency measures are likely to be annual measures since they relate to cost. The answer to this question should describe how measures are used to evaluate the program’s success if achieving efficiency and cost effectiveness improvements. Elements of No: A No must be given if the agency and OMB have not reached agreement on efficiency measures that meet PART guidance. Not Applicable: Not Applicable is not an option for this question. For more detailed discussion on defining acceptable efficiency measures please see the section called “4. Select Performance Measure” of this document or visit OMB’s PART website.2 Evidence/Data: Evidence can include efficiency measures, competitive-sourcing plans, IT improvement plans designed to produce tangible productivity and efficiency gains, or IT business cases that document how particular projects improve efficiency. 4.3: Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year? Purpose: To determine whether management practices have resulted in efficiency gains over the past year. Elements of Yes: A Yes answer needs to clearly explain and provide evidence of each of the following [see Box I-2]: The program demonstrated improved efficiency or cost effectiveness over the prior year. When possible, the explanation should include specific information about the program’s annual savings over the prior year as well as what the program did to achieve the savings. 2 http://www.omb.gov/part/.
OCR for page 131
Evaluating Research Efficiency in the U.S. Environmental Protection Agency BOX I-2 Question Linkages If a program received a No in Question 3.4, the program must receive a No answer to this question. Efficiency improvements should generally be measured in terms of dollars or time. For example, programs that complete an A-76 competition—an indicator of cost-efficient processes—would contribute to a Yes answer, provided that the competition resulted in savings. Not Applicable: Not Applicable is not an option for this question. Evidence/Data: Evidence can include meeting performance targets to reduce per unit costs or time, meeting production and schedule targets; or meeting other targets that result in tangible productivity or efficiency gains. Efficiency measures may also be considered in Questions 4.1 and 4.2. REFERENCES OMB (Office of Management and Budget). 2006. Program Assessment Rating Tool Guidance 2006-02. Office of Management and Budget. March 2006 [online]. Available: http://www.whitehouse.gov/omb/part/fy2006/2006_guidance_final.pdf [accessed Dec. 17, 2007].
OCR for page 132
Evaluating Research Efficiency in the U.S. Environmental Protection Agency This page intentionally left blank.