Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 86
Evaluating Research Efficiency in the U.S. Environmental Protection Agency
Appendix C
Program Assessment Rating Tool (PART) Questions1
OFFICE OF MANAGEMENT AND BUDGET
Section I.
Program Purpose and Design
1.1: Is the program purpose clear?
1.2: Does the program address a specific and existing problem, interest, or need?
1.3: Is the program designed so that it is not redundant or duplicative of any other Federal, State, local or private effort?
1.4: Is the program design free of major flaws that would limit the program’s effectiveness or efficiency?
1.5: Is the program design effectively targeted so that resources will address the program’s purpose directly and will reach intended beneficiaries?
Section II.
Strategic Planning
2.1: Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?
2.2: Does the program have ambitious targets and timeframes for its long-term measures?
1
OMB (Office of Management and Budget). 2007. Program Assessment Rating Tool Guidance No. 2007-02. Washington, DC.
OCR for page 87
Evaluating Research Efficiency in the U.S. Environmental Protection Agency
2.3: Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program’s long-term goals?
2.4: Does the program have baselines and ambitious targets for its annual measures?
2.5: Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?
2.6: Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?
2.7: Are budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program’s budget?
2.8: Has the program taken meaningful steps to correct its strategic planning deficiencies?
Specific Strategic Planning Questions by Program Type
2.RG1: Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals? (Regulatory)
2.CA1: Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity? (Capital Assets and Service Acquisition)
2.RD1: If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals? (R&D)
2.RD2: Does the program use a prioritization process to guide budget requests and funding decisions? (R&D)
Section III.
Program Management
3.1: Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?
3.2: Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?
3.3: Are funds (Federal and partners’) obligated in a timely manner, spent for the intended purpose, and accurately reported?
OCR for page 88
Evaluating Research Efficiency in the U.S. Environmental Protection Agency
3.4: Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?
3.5: Does the program collaborate and coordinate effectively with related programs?
3.6: Does the program use strong financial management practices?
3.7: Has the program taken meaningful steps to address its management deficiencies?
Specific Program Management Questions by Program Type
3.CO1: Are grants awarded based on a clear competitive process that includes a qualified assessment of merit? (Competitive Grants)
3.CO2: Does the program have oversight practices that provide sufficient knowledge of grantee activities? (Competitive Grants)
3.CO3: Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner? (Competitive Grants)
3.BF1: Does the program have oversight practices that provide sufficient knowledge of grantee activities? (Block/Formula Grant)
3.BF2: Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner? (Block/Formula Grant)
3.RG1: Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations? (Regulatory)
3.RG2: Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines? (Regulatory)
3.RG3: Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals? (Regulatory)
3.RG4: Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity? (Regulatory)
3.CA1: Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals? (Capital Assets and Service Acquisition)
3.CR1: Is the program managed on an ongoing basis to assure credit quality remains sound, collections and disbursements are timely, and reporting requirements are fulfilled? (Credit)
OCR for page 89
Evaluating Research Efficiency in the U.S. Environmental Protection Agency
3.CR2: Do the program’s credit models adequately provide reliable, consistent, accurate and transparent estimates of costs and the risk to the Government? (Credit)
3.RD1: For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality? (R&D)
Section IV.
Program Results/Accountability
4.1: Has the program demonstrated adequate progress in achieving its long-term performance goals?
4.2: Does the program (including program partners) achieve its annual performance goals?
4.3: Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?
4.4: Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?
4.5: Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?
Specific Results Questions by Program Type
4.RG1: Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits? (Regulatory)
4.CA1: Were program goals achieved within budgeted costs and established schedules? (Capital Assets and Service Acquisition)
