|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Summary of Major Conclusions
and Recommendations
Since World War II, the financing of higher education has changed
through the expansion of the role of need-based student aid. Scholarships
for students who were both meritorious and needy have always been avail-
able. The introduction of need-tested aid in the l950s, however, and the
major federal support and regulation of student aid distribution that fol-
lowed have resulted in what is today a complex aid system.
Postsecondary students can receive financial aid from the federal gov-
ernment, states, postsecondary institutions, other private organizations, or a
combination of these sources. Direct aid to students is dominated, however,
by federal aid programs (grants, loans, and work-study programs covered
by Title IV of the Higher Education Act of 1965~. In the academic year
1989-90, total available student aid equaled $27.3 billion, of which gener-
ally available federal aid (Title IV programs only) equaled $19.0 billion.
Of the students enrolled full-time in postsecondary schools in 1989-90, 57
percent received aid from at least one source, and federal funding went to
44 percent of the students enrolled. The large expenditures and the wide-
spread use of student aid create great uncertainty concerning the adequacy
of the system for delivering federal student financial aid.
Especially troublesome for the U.S. Department of Education are the
recurring questions about the accuracy with which the share of educational
expenses to be borne by the family is determined and the effectiveness of
the systems of accountability mandated by federal regulations. To address
these questions, the U.S. Department of Education requested that the Na-
tional Academy of Sciences convene a panel to study quality control in the
OCR for page 2
2
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
Title IV student financial aid programs. In particular, the panel was asked
to study (1) the quality control practices employed by the Department of
Education to measure the accuracy of awards provided to students and (2)
the methods used by program managers, based on this information, to re-
duce errors. The panel's report goes beyond a simple assessment of quality
control, however. It also considers possible use of this information to pre-
vent errors through appropriate corrective actions.
This summary presents the panel's major conclusions and recommenda-
tions. Details of the analysis and other suggestions for improvement are
provided in subsequent chapters.
"ERROR" IN STUDENT FINANCIAL AID PROGRAMS
Types of Error
A study of the accuracy of awards made during the 1988-89 academic
year found that, of the approximately $15.4 billion in aid distributed, almost
11 percent was awarded in error (Price Waterhouse, 1990~. Although inac-
curacies are possible in this and other estimates of program error, there is
little doubt that the errors, whatever their exact total, are considerable. At
each step of the delivery process (applying, determining eligibility, calculat-
ing awards, disbursing awards, and monitoring educational progress) errors
can be made. Efforts have been focused on detecting errors in examined
files and correcting them, but too little has been done to prevent error-
changing the procedures that allowed the errors to occur.
Although reducing payment error (defined below) is an important as-
pect of quality particularly in programs that disburse public funds it is
only one of many dimensions of quality. Its pervasiveness, however, makes
an understanding of payment error prerequisite to an understanding of "quality
control" as practiced by the Department of Education.
In principle, two kinds of payment error occur in student financial aid
programs: (1) errors of overpayment and (2) errors of underpayment. Over-
payments can be subdivided into (1) excess payments to eligible recipients
and (2) all payments to ineligibles. In parallel fashion, underpayments
comprise (1) insufficient payments to eligibles and (2) the lack of payments
to those mistakenly classified as ineligible.
Errors can be further classified as substantive or technical. Errors in
the information used to determine eligibility for student financial aid as
well as errors made in the calculation of the award based on that informa-
tion are called substantive errors. Technical errors occur when a legally
required document is missing from a student's file. Depending on other
factors, inclusion of the document may or may not have rendered the stu-
dent eligible. The lack of such a document makes the student categorically
OCR for page 3
SUMMARY
3
-
ineligible for financial aid. Examples of technical errors include failure to
document satisfactory educational progress, failure to indicate registration
for Selective Service, failure to include a statement of educational purpose,
and failure to include a financial aid transcript.
A third distinction deals with the originating source of the error whether
it is the institution, data processing contractor, or the student. The three
sources of error are the primary focus of quality control efforts.
Student Error
Student error may occur because of failure to use correct data, inaccu-
racies in forecasting data, and/or complexities of the application and in-
structions. For errors that are not made deliberately, several "student er-
rors" could not be corrected even with additional care on the part of the
applicant. For example, some problems arise because, to apply for aid from
some states and institutions, the application must be completed before the
applicant completes his or her federal income tax form. Thus, information
provided on dependency status, taxes, and adjusted gross income is gener-
ally an estimate of the data that will be reported in the tax return. Similarly,
household size and the number of dependents in postsecondary school are
necessarily projections for the following academic year; as circumstances
change from current plans, this information will turn out to have been "in
error." The panel recommends simplifying the application procedure to
include fewer items subject to error.
Recommendation 5-1: For applicants who have filed a tax return in
either of the prior two tax years, the information used to complete the
application and determine the award should be based on the most re-
cently filed of these income tax returns. When the earlier tax year is
used, updated information should be required as soon as a new tax
return is filed. The updated information should not be used to change
the award before the next term. Further, "household size" and "num-
ber in postsecondary school" should be based on the situation as of the
application date.
Institutional Error
Institutional errors occur when institutions inaccurately process appli-
cations or do not adhere to established guidelines. The panel found that
technical errors, such as a missing statement of educational purpose or
financial aid transcript, have a large dollar impact on measured error, since
their absence renders the entire award erroneous. Such cases may or may
not otherwise be eligible for financial aid. The General Accounting Office
OCR for page 4
4
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
(GAO) requested from the Department of Education that some student files
be analyzed for substantive errors only (technical errors due to missing
documentation were to be ignored). The analysis found that the estimated
dollar value of errors dropped by about a third, and the percentage of cases
with "institutional error" dropped by half (General Accounting Office, 1985~.
The panel believes that there is a need to refocus the effort used to detect
technical errors.
Recommendation 5-2: The Department of Education should determine
which, if any, of the administrative requirements that can lead to "tech-
nical error" are useful surrogates for variables that deal directly with
program goals but are difficult to measure. Administrative require-
ments that are useful in this sense should be retained, but others should
be eliminated.
Failure to Estimate Error Leading to Nonawards
According to the Department of Education's data for the 1982-83 aca-
demic year, 42 percent of Title IV aid recipients received overawards and
21 percent received underawards. Typically, overaward has been estimated
as much larger than underaward whenever such a classification is made.
There is, however, a serious defect in all the major studies of error in
student financial aid programs. The various surveys analyze only cases of
awards; there is no coverage of students to whom financial aid was totally
denied. It is thus not surprising that the studies find more overpayment
error than underpayment error. On a more basic level, a major concern of
the panel is an underemphasis by the department on measuring progress
toward the program goals of access and choice in postsecondary education.
Potential recipients mistakenly denied financial aid despite actual eligibility
surely have restricted access and choice.
Recommendation 5-5: The Department of Education should improve
estimates of "error" by including estimates of the coverage of student
financial aid programs, that is, ascertaining the frequency with which
eligible applicants are mistakenly deniedfinancial aid and the underaward
amounts associated therewith.
CURRENT QUALITY CONTROL PROCEDURES
The Department of Education's current quality control process is de-
scribed as focusing on prevention, inspection, and oversight. Prevention
activities include training provided by the department to data entry contrac-
tors, financial aid administrators, and other institutional officers, lenders,
accrediting agencies, state scholarship and guaranty agencies, and others in
OCR for page 5
SUMMARY
s
the student financial aid community. Other activities that are carried out in
an effort to prevent award error entail editing and verifying student data.
Inspection consists of after-the-fact monitoring activities audits and
program reviews. Such activities are often developed in the belief that they
help to ensure compliance because the possibility of penalties and/or sanc-
tions acts as a deterrent.
Oversight consists of periodic studies of various program areas within
the Department of Education. The studies are conducted by the department's
Office of Inspector General or the GAO and focus on review of procedures
for monitoring compliance with various requirements and the management
of those activities. The panel found information in the various studies to be
very useful in its deliberations.
Edits and \7erification of Applicant Data
Applicant-supplied data undergo several levels of editing and review.
First, edits are performed by data entry contractors. During the edits, ques-
tionable data are identified and "highlighted" on the Student Aid Report
(SARI that is mailed to the student. The student is asked to review and
correct all data and, if corrected, return the SAR for reprocessing.
Verification of applicant-reported data items by the institution is the
Department of Education's primary tool in its efforts to control applicant
error. Verification activities place a major responsibility and burden upon
the schools and, as a result, are very controversial. Each school is required
to verify key elements of the applicant's record for all records flagged by
the department's selection methodology (in recent years, up to 30 percent of
an institution's student aid applications, but the 1992 reauthorization of the
Higher Education Act allows the Secretary of Education to mandate up to
100 percent verification). The department describes the current verification
system as an attempt to strike a balance between error reduction and the
burden imposed on institutions. Efforts to move toward 100 percent verifi-
cation of applicant data have been tempered by institutional lobbying to
stress the excessive burden that institutions must endure to correct errors
not of their making.
Applicant records are selected for verification through a complex se-
quence of statistical procedures intended to target the most error-prone ap-
plicants. The verification system has been studied and modified over the
years, but an extensive review by the panel yielded no evidence that major
advances have been made in responding to the major criticisms of the sys-
tem. The cost-benefit of the system remains a question.
Although some cases of fraud or abuse undoubtedly will arise in a
program of this size, previous studies have not uncovered any widespread
problems of fraud by students. The panel recognizes that the Department of
OCR for page 6
6
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
Education believes verification is necessary to fulfill its congressional man-
date, yet the panel concludes that the verification process as currently used
is unfairly burdensome for the overwhelming majority of well-intentioned
people working within the system. In using any other technique, the em-
phasis should be to move from the current focus on catching mistakes to
also providing information for future preventive actions.
Recommendation 4-1: A research effort should be initiated to assist the
Department of Education's contractor in developing better criteria for
selecting records to be verified and, more important, identifying oppor-
tunities for earlier removal of errors through instructions, form im-
provements, and SAR highlighting strategies. Current andfuture changes
to the verification selection methodology should be carefully documented.
Audits and Program Reviews
The Department of Education has the responsibility to ensure that the
approximately 8,000 participating postsecondary institutions administer stu-
dent financial aid programs in compliance with federal law and regulations.
The department executes this responsibility through two major retrospective
activities: (1) requiring schools to contract for compliance audits by an
independent auditor and (2) conducting program reviews through its re
gional offices.
The Title IV student financial assistance audit and program review pro-
cesses are designed to determine the accuracy with which an institution
administers its student financial aid programs. Specifically, the Title IV
audit objectives are to measure the reliability of an institution's financial
data, the adequacy of its internal control systems, and its compliance with
program regulations. The objectives of a program review of an institution
are to determine compliance with program regulations and to evaluate fi-
nancial and administrative capabilities. The panel examined the operations
of these retrospective activities as they relate to the problem of error in the
delivery of student financial aid.
The rules and regulations governing student eligibility for financial aid
are complex. The panel questioned whether and, if so, to what extent the
complexities themselves are significant sources of error. The current moni-
toring and compliance activities and data bases do not address this issue.
They are designed to assess an institution's absolute performance with re-
spect to the accurate administration of student financial aid programs, to
count the occurrences of error, and to impose sanctions on the institution
based on error.
The audit and program review processes used by the Department of
Education to enforce quality standards are marked by considerable duplica
OCR for page 7
SUMMARY
lion, ineffectiveness, and wasted effort. The independent audit of schools
checks for internal controls and compliance with program regulations in a
way that essentially duplicates parts of the program review process. Such
checks are more effectively performed by knowledgeable reviewers and
should be fully incorporated within the program review process.
Inspection is a deterrent to poor work only when it is possible to do a
better job. To maximize an institution's potential, the audit and review
processes must be better integrated and redesigned so as to have a sharper
focus on addressing meaningful measures of quality and quality improve-
ment rather than the current concept of measuring compliance with an unre-
alistic "zero defect" standard. To expend resources where risk is greatest,
for example, data from past audits and reviews should be used to improve
the methods for selecting institutions and the methods for sampling records
at those institutions.
Audit and review activities are necessary if the Department of Educa-
tion is to fulfill its responsibility to ensure that program participation is
limited to those institutions that are willing and able to operate responsibly,
in accordance with program goals and expectations. Since factors such as
changing economic conditions and student financial aid personnel affect an
institution's ability to maintain desired levels of quality, "problem" institu-
tions will appear sporadically and must be dealt with promptly and effi-
ciently. Audits provide reasonable promptness, especially now that they are
an annual requirement. Program reviews are relatively infrequent, however,
because they must be performed under departmental budgetary and person-
nel ceilings. Yet, the review process has the greater potential to be proactive
and to provide the useful instruction and technical assistance that promote
quality and help to build a sense of partnership between the institutions and
the department.
Further, the match between the capabilities of the inspector and the
inspection function warrants careful attention. More information is needed
about the reliability of independent auditor findings. While the audit func-
tion can provide timely information, program review is the only system of
institutional quality checks that is entirely in the control of the Department
of Education. The panel believes that program reviews have the greatest
potential for reliability and objectivity. The panel also believes that the
department needs to revamp the audit and program review systems in order
to support the gate-keeping function, provide technical assistance to institu-
tions, and provide useful data to policymakers.
Recommendation 4-3: The Department of Education should redesign
the current system of program reviews and independent audits. Pro
gram reviews should focus on compliance as part of an overall quality
improvement program. Checks on institutional compliance and internal
OCR for page 8
8
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
controls should be performed only in program reviews and audits should
focus only on financial attestation.
The Department of Education should also develop, test, and imple-
ment methods to systematize and standardize the program review pro-
cess. The department should not interpret the 1992 reauthorization of
the Higher Education Act as a requirement to review every school ac-
cording to a fixed schedule. Risk-based statistical methods should be
used to identify problem schools for more frequent reviews, and other
schools should be selected randomly at a nominal rate that would fulfill
the necessary '~gate-keeping" functions. The department must improve
the evaluative feedback and technical assistance provided to institu-
tions during reviews. At the same time, the review should be used to
accumulate data that provide the department with a continuous over-
view of error rates, compliance levels, and other information of signifi-
cance for management in making policy.
THE APPLICANT'S VIEW
As the ultimate customer of financial aid services and as the provider of
the information that puts the overwhelming inspection processes into mo-
tion, the applicant's or potential applicant's experience must weigh heavily
in decisions concerning process improvement. The panel focused on two
areas of importance related to the applicant's need for quality: early aware-
ness and the application process.
Early Awareness
Very few ongoing data sets are available from which to obtain infoIma-
tion about students who have not applied for financial aid. The Department
of Education maintains administrative records on only those students apply-
ing for financial aid, and studies of error have captured data on aid recipi-
ents only. Better use of the National Postsecondary Student Aid Study and
coordination with the National Center for Education Statistics could help.
Recommendation 5-7: The panel commends the Department of Educa-
tion for its recent efforts to improve early awareness of federal finan-
cial aid programs, such as providing financial aid software to high
schools. The panel recommends increasing those efforts. Studies of the
effectiveness of the efforts and users' reactions to the timeliness, accu-
racy, usefulness, and clarity of outreach and counseling services are
needed.
OCR for page 9
SUMMARY
9
Recommendation 5-8: The Department of Education should gather data
on the reasons for nonutilization of student financial aid by potential
recipients. Consideration should be given to ways of estimating the
number of potentially qualified applicants who are discouraged for one
reason or another from applying for financial aid in the first place.
Further, the department should consider ways of estimating the number
of potential students who do not even attempt to enter a postsecondary
school because of their ignorance of available financial aid. Also, data
on the knowledge of aid should be collected from students and their
families before the student finishes secondary school. Such data could
be obtained in a variety of ways, including ongoing national surveys.
The resulting information will be important in devising a program for
reaching those who are eligible but do not apply.
The Application Process
Students who wish to apply for federal student financial aid can make
application in one of three ways: complete the Application for Student
Financial Aid (AFSA) produced and distributed by the Department of Edu-
cation; complete an application form produced by one of the multiple data
entry contractors who process applications; or enter application data through
the Electronic Data Exchange application process. Panel members were
concerned that the current design of these application "forms" could cause
confusion among applicants and financial aid officers, lead to inadvertent
errors in financial aid decisions, and deter some students from applying at
all. Such up-front errors could then lead the department to perceive a need
for more inspection. Testimony before the panel, as well as the panel's own
deliberations, suggest that the application forms for student financial aid
might be difficult to complete, especially for the typical lower income ap-
plicant, and that the design of the for might complicate the work of stu-
dent financial aid officers who advise students about financial aid, check
the accuracy of applicants' responses on applications forms, and attempt to
correct erroneous data. As a result, the panel undertook a study of the
federally produced AFSA and the Electronic Data Exchange process. Based
on that analysis, the panel makes two recommendations:
Recommendation 5-9: The complexity of the forms, instructions, and
information booklets leads to excessive burden for applicants and is a
cause of error. Thus, the Department of Education should consult ex-
perts in form and question development, such as those found at cogni-
tive research laboratories, to aid in its efforts to improve the applica-
tion materials.
OCR for page 10
10
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
Recommendation 5-10: The Department of Education should continue
to improve and expand the availability of its electronic data exchanges,
making sure to address the issue of appropriate balance between the
potential for improved data and additional burdens that might be placed
on the schools.
CONCERNS OF POSTSECONDARY INSTITUTIONS
Postsecondary institutions are increasingly burdened by the administra-
tive complexities associated with federal student aid programs. From an
original promise to administer the eligibility requirements centrally, the gov-
ernment has moved to an increasingly complex financial statement form,
which must be verified by school personnel. The addition of information
unrelated to need, such as Selective Service status, is collected through this
form, which has grown from two pages to four.
The Department of Education requires institutions to collect signed statements
regarding nonuse of illegal drugs and educational purpose, but the certifica-
tion is not of direct use to the institution because it neither prevents drug
use nor ensures that the student has a valid educational purpose. Under the
current system, institutions bear a considerable amount of the burden in
obtaining this type of certification. In addition, hard copies of applications
and related "underwriting" materials must be physically retained by the
institution for long periods of time, which creates the need for expensive
warehousing in financial aid offices. There is, however, no reason why the
depository cannot be changed so that the department can take on an equi-
table amount of responsibility.
Congress has made some progress in initiating changes along these
lines. The 1992 reauthorization of the Higher Education Act calls for in-
creased utilization of verification mechanisms in which students' reported
information can be verified through record matching, by using either an
automated or other system. These provisions include requiring that data
base matches with the Selective Service be made and that the Social Secu-
rity Number of all aid recipients be verified. With this advancement toward
electronic data transfer of student financial aid information, the burden im-
posed by the collection of student certifications could be greatly reduced.
Recommendation 5-11: The Department of Education should increase
its efforts to remove unnecessary burden from students and institutions
by further development of automated data matches whenever possible.
A VISION FOR A TOTAL QUALITY APPROACH
Many industrial and service sector organizations have recognized that a
system based on prevention of errors rather than inspection is necessary to
OCR for page 11
SUMMARY 1 1
achieve quality improvement and efficient operation. Such systems utilize
a total quality approach. A total quality approach requires a commitment
by the highest levels of management to lead quality improvement efforts
rather than merely assigning responsibility for quality assurance to an orga-
nizational unit. It also requires a focus on the "customer" and having
suppliers and customers work together to define and solve problems. It
requires process simplification to reduce waste, remove redundant efforts,
and eliminate time-consuming and non-value-adding steps. It requires data
collection and analysis that are designed for fact-based decision making and
problem solving.
These strategies are as valid in the Department of Education and other
governmental agencies as they are in private industry. A total quality ap-
proach to administering student financial aid would seek not only to im-
prove action taken at each step of the process, but also seek to continuously
improve important outcomes expected of the entire process, such as accu-
racy of awards and accessibility of aid for eligible students. Additionally, a
total quality approach seeks improvements that may eliminate mistakes be-
fore they are made and that strives to lower costs, provide faster processing,
and reduce red tape. Such an approach brings greater value to all the
customers and stakeholders the Department of Education, Congress,
postsecondary institutions, students, parents, and taxpayers.
A NEW STRATEGY FOR THE
DEPARTMENT OF EDUCATION
The Department of Education's old strategies for ensuring quality are
characterized by a centralization of authority, reliance on after-the-fact in-
spections to bring about quality, and burdensome sets of frequently changed
regulations that focus on compliance with process requirements. Under the
old strategies, the student financial aid delivery system is burdened with a
multiplicity of process checkpoints that add to the cost of running the sys-
tem for schools and for the government. They also contribute to a tendency
for the relationship between customers and suppliers in the system to be
perceived as confrontational. The deficiencies of these strategies have led
some within the department to seek new strategies.
One of the difficulties in developing new strategies to improve quality
is conceptual. In the absence of a statutory definition of quality, regulatory
efforts fall into the trap of having no margin for error. As reported to the
panel, the Department of Education has been operating with a "zero error"
standard that is unattainable in practice and therefore not effective in pro-
moting quality improvements. Departmental staff reported to the panel on
efforts to develop new strategies that are characterized by delegation of
authority and responsibility, the "ownership" of performance measures by
OCR for page 12
2
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
those who are performing the functions of the system, and a deregulation of
the aid delivery process that empowers an institution to determine how best
to accomplish program objectives. The new strategies, as found in the
Institutional Quality Control (IQC) Pilot Project, focus accountability on
results rather than process and emphasize at participating institutions the
continuous collection of data leading to quality improvement. Foregoing
the usual imposition of external quotas, these strategies are intended to
form proactive partnerships rather than confrontational relationships and to
focus on service improvements for the end customers. At the same time,
they are intended to meet the need to assess overall system performance by
measuring quality on an ongoing basis. With this experiment, the Depart-
ment of Education hopes to reduce the burden on educational institutions,
encourage the development of innovative management approaches, improve
service to students, and reduce error in the delivery of Title IV financial
assistance.
Support of the IQC Project
Participation in the IQC Pilot Project is voluntary and assumes that the
commitment to ensure data quality already exists in participating schools.
The Department of Education has many positive activities associated with
efforts to support IQC institutions, including training programs for new and
continuing institutions, providing quality control software, recognition and
awards processes, publishing a newsletter, state and regional meetings, and
providing technical assistance from departmental and contracted personnel.
The IQC staff within the department is continually updating its support
mechanisms to be able to offer more help to the institutions.
The panel examined the IQC project at great length and gave much
consideration to its future. The panel had many questions about the effec-
tiveness of the project, but too little objective data to answer most of them.
The IQC project offers a chance for the Department of Education to be
more proactive, but it may not be making the best use of the chance.
Possibly, the IQC process is overwhelming. Perhaps the institutions
need more advice on the quantitative aspects of the process. Many institu-
tions will have someone outside the institution's aid office who is fairly
knowledgeable about such quantitative methods and could learn more about
the aid process as he or she goes along. Other institutions may have to ask
for outside help. Statistical expertise also needs to be developed at the
Department of Education to generate the appropriate statistical information
from the IQC institutions and share the aggregate results with everyone.
Recommendation 8-3: Institutions in the IQC pilot program should
have more at-hand statistical support, and institutions volunteering to
OCR for page 13
SUMMARY
13
participate in IQC activities should be required to show access to sta-
tistical expertise in sampling and data analysis as a condition of being
in the program. To some extent this can be accomplished by improving
the training of internal staff. Adding training material on basic statisti-
cal skills to the training program could help, but more effective strate-
gies might include subsidizing locally obtained coursework for finan-
cial aid office staff or developing cooperative agreements with local
quantitative departments to provide ongoing assistance. The Depart-
ment of Education should assist the IQC institutions in these efforts by
helping develop the cooperative ties and providing monetary support.
Recommendation 8-4: The IQC program should be retained and given
the support of top management. However, when the common-cause
regulatory burdens imposed by audits and reviews are eliminated (for
details see Chapters 4 and 5), the incentives to join the pilot project
will change. Thus, the following improvements should be considered:
· Incentives for institutions to join the IQC should be reexamined
with the aim of encouraging schools with known administrative difficul-
ties to seek help from Department of Education staff.
· The department's measure of success in the IQC program should
be reexamined with the aim of going beyond measuring retention in the
IQC program to identifying ways to improve processes.
· More use should be made of IQC data. For example, the Depart-
ment of Education should inform all institutions about the progress of
the IQC program and suggest use of measurements and practices found
to be successful in addressing specific concerns about quality.
FURTHER INITIATIVES FOR
IMPROVEMENT IN THE SYSTEM
In its review of the issues related to quality in the delivery of student
financial aid, the panel has endorsed a customer-oriented focus that is con-
sistent with a mission of supporting the educational needs of the student
applicant while making the most efficient use possible of taxpayer dollars.
A customer service philosophy must embrace all of the internal customers
of the system: the students, the postsecondary educational institutions, the
lenders, guaranty agencies, the various relevant units within the Department
of Education, and the Congress. The 1992 reauthorization of the Higher
Education Act, which occurred as the panel was engaged in its delibera-
tions, is a step in the right direction. One of its main goals is to simplify
student aid programs. As a result of the passage of the bill, some of the
concerns already expressed in this report concerning the complexity of the
OCR for page 14
4
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
application and its effect on student error have been reduced or even elimi-
nated.
Part H of the reauthorization bill focuses on the strengthening of the
process by which institutions of higher education are allowed to participate
in federal student aid programs through state licensure, accreditation, and
federal eligibility, certification, and program review processes. The panel
hopes that enactment of this section will lead to the development of perfor-
mance-based statutes that avoid affixing common-cause solutions to spe-
cial-case events and that aim at achieving measurable and meaningful out-
comes rather than additional layers of bureaucracy. Examples of meaningful
outcomes would include lower default rates and higher graduation rates,
employment rates, and customer satisfaction.
Toward a New Applicant Processing System
The panel strongly believes that there is a need to change the current
institutional verification process, which is fundamentally redundant and in-
efficient. The panel suggests that the Department of Education use the
knowledge it has at hand in much of the contract research that has ad-
dressed verification. Those reports have indicated that (1) large errors re-
main even after verification, (2) the cause of much of "student error" lies in
the complicated application process, and (3) data items that must be fore-
cast (e.g., estimated income, household size, and number in college) are the
main contributors to student error. The reports have produced little evi-
dence that the institutions themselves can develop procedures that will fur-
ther reduce student error very much. True corrective actions lie mainly in
the hands of the Department of Education and the Congress.
The Department of Education must move from a system that focuses on
detecting errors after awards have been made to a system that prevents error
to the extent possible. Some error could be removed by making the applica-
tion materials more user friendly, but additional action is needed. A combi-
nation of changes to some data requirements and changes in the activities
associated with the SAR and verification could result in large reductions in
errors. For example, the panel believes that the department should develop
a front-end match of applicant data with Internal Revenue Service (IRS)
data tapes. Such matches are being performed at several agencies and are
reportedly very successful. (The panel recognizes that the department will
likely need the help of Congress to pass legislation directing IRS to allow
these matches and to make some changes in data requirements.) The de-
partment must take actions that affect the system in order to reduce "student
error" appreciably and not impose further requirements on the institution in
the verification and compliance-review procedures.
OCR for page 15
SUMMARY
15
Recommendation 9-1: The Department of Education and the Congress
should work together to effectuate changes in data requirements that
should simplify award determination and increase accuracy. Develop-
ment of a system that requires applicant data to be matched with IRS
data should receive top priority.
Centralized Verification
The panel believes that a system based on centralized verification wills
in the long run, simplify the verification process and increase accuracy.
Institutional burden will be removed if the signed statements of educational
purpose, Selective Service compliance, and the like, are required as part of
the initial application. This change would also remove considerable burden
from the institution and from the Department of Education's audit and re-
view activities. Similarly, data base matching with other agencies as di-
rected by Congress in Title IV legislation should be used to the fullest
extent possible. Federal concerns internal to the department, such as loan
repayment problems and prior aid received, as measured by the financial aid
transcript, should be identified from the department's own records. Here
too, Congress has instructed the department through Title IV legislation to
improve its internal data bases.
The remaining major sources of institutional error (cost of attendance
computation, enrollment monitoring, special situation exception determina-
tions, bachelor's degree determination, and aid packaging) are associated
with information and actions most easily found at the school. With the
reduction of other unnecessary activities, the Department of Education and
the schools could concentrate on developing effective methods to reduce
errors in these activities. Freed of trivial inspection activities, the institu-
tions and the department should have more time to focus on important
issues of quality, such as effectiveness of educational programs, institu-
tional integrity, and the value of the service purchased by the student and
taxpayer.
With data base matches and up-front certifications, many of the under
lying causes of student error would be removed. Thus, the verification
strategy should change. Rather than flagging the record for institutional
action during verification, the SAR could request corrections or clarifying
information when there are conflicts with data base information. These
changes could be processed centrally, which would result in an initial award
determination that would be considerably more accurate than is produced
under the current procedures. It is possible that a small group of applica-
tions might require assignment to institutions for further examination to
resolve special situations that might otherwise delay the determination of
award.
OCR for page 16
16
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
While up-front data processing costs might increase under this pro-
posal, the Department of Education should consider total cost and service
improvement in making decisions on implementation. Repeated processing
of SARs would likely decrease. Financial aid administrators would be freed
of burden, allowing them to spend more time on cases requiring special
attention. Awards might be determined sooner, a service improvement for
applicants. Consistency of verification and documentation would also be
better controlled. Finally, award error would be controlled on all applica-
tions, not just those selected for verification, as is now the case.
Recommendation 9-2: The Department of Education should begin the
development of a front-end and centralized verification system in which
the schools' verification burden is drastically reduced. Under a new
system, the department would identify exceptional cases, far fewer than
the current 30 percent, for institutional review activities.
Organizing Information About the System
The Department of Education also asked the panel to comment on the
organization of available information about award error and the usefulness
of such data bases for making program decisions. In a similar investigation,
the General Accounting Office (1985) found the department's approach to
issues regarding the quality of awards to be unsystematic. That study found
that decisions that might affect improvement were hindered by a lack of
goals and analyses that were hurried due to operational demands. The study
cited problems in coordinating offices within the department that needed to
work together and labeled the resultant reactive decision making "remedial"
and "not preventive."
Recommendation 5-6: The Department of Education should not rou-
tinely embark on surveys of the type that have been used in the past to
estimate total error levels in student financial aid programs. The re-
sources are likely to be better spent on continuous monitorin~using
data from audit and review activities, for example~nd other approaches
to quality improvement. Special studies should be done, but only when
the study objectives are clearly linked to a policy evaluation or process
improvement plan and steps can be taken to eliminate the methodologi-
cal defects of past surveys.
The panel, after discussions with Department of Education personnel
and review of earlier studies, also found no particularly well-organized use
of the data that focused on improving the quality of award decisions. It is
evident that operational demands continue to limit proactive thinking. At-
tempts to bring the parts of the department together to work on cross
OCR for page 17
SUMMARY
17
organizational issues are being made, but comments to the panel indicate
that there is no strong management direction in these areas.
Recommendation 6-1: The panel believes that a totally integrated ap-
plicant data base is essential for problem identification, the develop-
ment of solutions, and monitoring progress important activities if true
quality improvements are to take place. The data base would compile
information from all the Title IV aid programs and across years to
provide, for each applicant, a complete history of participation in the
financial aid system, thus facilitating a broad range of cross-sectional
and longitudinal statistical analyses. The Department of Education
should allocate sufficient resources and personnel to accomplish this
task without further delay.
Top Management Leadership
Despite the genuine movement toward quality improvement that the
1992 reauthorization will bring about, there is much more in that direction
to be accomplished. For example, a major detriment to sustaining the de-
sired focus on quality is a lack of stability in leadership positions that
influence student financial aid programs. Changing leadership may cause
inconsistency in long-term vision. Frequent turnover in the position of
Assistant Secretary for Postsecondary Education, a major reorganization of
the Office of Postsecondary Education, and a number of shifts in line man-
agement assignments are examples of the shifting leadership at the Depart-
ment of Education. While, in themselves, these changes may not be bad,
and some might be necessary, the panel sensed that the long-term vision for
quality in student financial aid programs often suffered as a result of fre-
quent turnover.
The panel found that while the Department of Education required schools
to invest heavily in quality inspection activities, it allocated limited re-
sources for its own quality improvement activities. Further, the panel noted
either inadequacies in the information technology available to the Depart-
ment of Education or the lack of use of available information by the depart-
ment in almost all the areas studied in this review.
Recommendation 9-3: To signal top management's commitment to a
Total Quality approach at the Department of Education, the department
should initiate a Total Quality training program that starts with the
leadership group and rapidly includes everyone in the department. In
this way the fundamentals, vocabulary, and principles of Total Quality
will be integrated into the department. Corporations have found this to
be a critical first step in implementing quality improvement. Such a
training effort should include leadership's role in the commitment to
OCR for page 18
8
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
change, process-improvement and problem-solving methods, Total Quality
tools, focus on customer satisfaction and its measurement, teamwork,
and quality planningldeployment.
Recommendation 9-4: In its review of survey activities, verification
procedures, and management information systems, the panel was disap-
pointed to find a paucity of statistically trained personnel available to
the student financial aid programs to analyze data and to interact with
contractors. The Department of Education should develop a much greater
in-house statistical capability to manage contracts that demand high
levels of statistical expertise, and the data developed by contractors
should be thoroughly documented and made available for in-house analysis.
The department should also strengthen the analytic capabilities of its
entire work force, including those who will not be expected to attain the
level of statistical "expert" but yet should be skilled in handling and
interpreting data. Specifically, the department should perform a needs
analysis of statistical and computer literacy. Then it should develop
training programs to improve abilities and purchase the hardware re-
quired to carry out the necessary analytic tasks.
The panel recognizes that technical improvement cannot begin without
some initial funding targeted toward efforts such as pilot projects, data base
improvement, and hiring staff with appropriate skills to help in these ef-
forts. Thus, the panel requests that Congress take action to help make this
possible.
Recommendation 9-5: Congress should ensure that there is adequate
funding and staffing to develop the quantitative information needed to
manage and review the student financial aid programs effectively.
The panel found an absence of performance measures linked to long-
term planning. Setting long-term plans that transcend leadership changes is
important. To make sound decisions on necessary changes to the programs,
a new top management team must gain experience and familiarity with the
system and its processes. A thorough understanding of a system as complex
as the student financial aid programs takes time.
Recommendation 9-6: To ensure the development of and commitment to
long-term planning, the politically appointed position in charge of the
student financial aid programs (the assistant secretary for postsecondary
education) should have a fixed (commonly five-year) term.
Representative terms from entire chapter:
student financial
