|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 117
6
Financial Aid Data Systems
The panel has discussed the extensive body of information available to
student financial aid program managers from special studies and the edit
and verification processes. The Department of Education also asked the
panel to comment on the organization of information about award error and
the usefulness of such data bases for making program decisions. In a simi-
lar investigation, the General Accounting Office (1985) found the department's
approach to issues regarding the quality of award determination to be un-
systematic. That study found that decisions that might affect improvement
were hindered by a lack of goals and analyses that were hurried because of
operational demands. The study cited problems in coordinating the activi-
ties of offices within the department that needed to work together and la-
beled the resultant reactive decision making "remedial" and "not preven-
tive."
The panel, after discussions with Department of Education personnel
and review of earlier studies, also found no particularly well-organized use
of the data that focused on improving quality of award decisions. It is
evident that operational demands continue to limit proactive thinking. While
attempts are being made to bring the parts of the department together to
work on cross-organizational issues, comments by departmental staff to the
panel indicate that there is no strong management awareness of the data
resulting from the quality control activities described in Chapter 4, nor is
there direction in cross-functional efforts.
Yet, there exist formally organized data sets related to some student
financial aid programs. In this chapter, the panel reviews the data system
117
OCR for page 118
118
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
for entering applicant data-the Central Processing System and that used
with the loan programs. Department of Education personnel reported to the
panel that there is no applicant data base for Campus-Based programs-
they know how many get aid, how much money goes out, and forgiven loan
amounts, among other things, but not who receives it or items useful for
quality improvement analyses.
THE MANAGEMENT INFORMATION SYSTEM
The Management Information System (MIS), the main tool used by the
Department of Education to monitor and control the Title IV application
processing system, draws data from virtually all aspects of the Central Pro
cessing System (CPS). The MIS is part of a coordinated set of management
controls, called Project Control Information Services; the other parts are
quality control, configuration management, and process control. (The CPS
and the MIS are operated for the Department of Education by a contractor.)
The MIS enables the Department of Education to monitor data, (e.g.,
number of applicants, eligibility rates, and turnaround times) generated dur-
ing production processes. Massive amounts of data are condensed into
coherent tabulations and presented in four groups, which are aimed at vari-
ous management levels (National Computer Systems, 1990-91a and 1990-
91b). The four groups of data are as follows:
· highly summarized information to give top management a snapshot
of key CPS operational and program areas;
· operating management data (e.g., summary information for monitor-
ing funding level and detailed contract, quality control, and project infor-
mation) to allow monitoring of day-to-day operations and the performance
of data entry contractors;
· detailed statistics for monitoring the effect of applicant characteris-
tics and behavior on program costs, compliance with regulations, the effect
of system changes on applicant population characteristics, and quality con-
trol by contractors; and
· detailed information for monitoring the receipt, handling, and mail-
ing of applicant documents.
The contractor submits printouts of these routine MIS reports to the
department at appropriate intervals, such as weekly or biweekly, during the
award cycle. Some data from previous cycles are included to permit com-
parisons. A few process-monitoring tools, such as control charts, are in-
cluded in the reports on an ongoing basis, but their use seems limited,
identifying occasional shocks to the system from causes such as peaks in
work flow of applications. Several staff members have used the MIS to
draw samples for additional inspection in areas exhibiting some unusual
characteristics in the periodic MIS reports. Special reports are produced on
OCR for page 119
FINANCIAL AID DATA SYSTEMS
119
request, usually through a task order to a contractor. At the end of the
award year, the contractor prepares a final report.
The MIS reports are also available to Department of Education staff on-
line. The information can be
· viewed on terminal and workstation screens,
· printed via menu selection, and
· downloaded in ASCII format and translated to other formats (e.g., a
worksheet or data base file), manipulated, analyzed, or graphed.
Some of the department's information needs can only be met through
use of the entire applicant data base. However, many of the tabulations do
not require an exact count; an approximation with a sufficiently small confi-
dence interval provides adequate information. Such tables are produced
from a Sample Data Base of approximately 50,000 cases drawn several
times during each processing cycle.
The department also has information needs that require statistical analysis
and programming involving numerous data transformations and complex
statistical procedures. The Sample Data Base can be used in such analyses
with a high degree of precision and at a reasonable cost. Access to the
Sample Data Base is provided through the on-line terminals and personal
computer workstations at the department, which enables staff to run tabula-
tions and perform statistical analyses. The Statistical Analysis System is
used as the major data base management and reporting tool for the Sample
Data Base.
The panel commends the Department of Education and the CPS con-
tractor for what appears to be a thorough development of an MIS data base
that presents operational data targeted appropriately to the functions of vari-
ous management levels. Improvements have also been made in recent years
in the presentation of the data. The panel is concerned, however, that the
data base is underutilized. While appropriate data seem to be captured with
the system (almost all the data are captured), and the system recognizes the
need to target data to the differing needs at various levels of management,
the reports produced tend to be dollar and timing oriented and not focused
on improvement of the system by identifying potential causes of error in
application and award processes. There are seemingly unbounded opportu-
nities to mine the data set to find ideas for process experimentation and the
potential to improve by finding and removing true causes of error.
Use of any data base requires some degree of expertise with computers
and knowledge of statistical methodology. Use of a sample data base and a
high-level statistical package as a tool for access and analysis raises the
level of expertise needed. More fundamental, use requires hardware, the
lack of which the panel heard mentioned by Department of Education staff.
Further, the utility of the Sample Data Base for special projects is not fully
appreciated. The panel, when discussing this data base with departmental
OCR for page 120
120
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
personnel often found a lack of appreciation for the usefulness of such a
data base. There was a tendency for departmental personnel to be con-
cerned that 50,000 records in the sample data base were too few of the over
seven million applications, without appreciating the degree of precision that
such a large sample can have for most purposes.
It appears that much of the analysis of MIS data is done through con-
tracts. While contracting for such analyses in itself may not be bad, a data-
driven organization should encourage the development of expertise within
its own staff. Several contractors commented to the panel that the depart-
ment does not always encourage their innovative suggestions and that some
lack of communication within the department hindered their efforts to pro-
vide effective analysis. One contractor did comment that there has been a
recent move toward encouraging its input, and that its contract now requires
that it allocate time to consider improvements. When technical expertise,
either statistical or computer, is obtained from outside sources, the depart-
ment should consider the technical advisors as partners in the process and
encourage unsolicited suggestions for creative analyses.
The MIS has the potential to provide appropriate information to the
appropriate levels of management. While the current implementation has
noticeably improved over the past few years, users of the system seem
hampered by a lack of equipment, methodological knowledge, and time.
Similar problems were observed in other areas of the department's financial
aid programs. (The panel makes a recommendation addressing these prob-
lems in Chapter 9.)
STUDENT LOAN DATA
The panel reviewed several studies of student loan programs that com-
mented on the data systems needed to monitor the loan programs, and it
spoke with Department of Education personnel about those systems. In one
study, the General Accounting Office (199Ob:3) describes the loan data sys-
tem this way:
As part of its monitoring of the program, the Department each year collects
Stafford loan information from guaranty agencies and consolidates this
information into the Stafford loan data base, commonly referred to as the
tape dump. This data base is maintained by a contractor for approximately
$900,000 a year and is primarily for the internal use of the Department.
The contractor edits the data base's information to check whether guaranty
agencies are reporting complete data. For fiscal year 1988, the Department
required guaranty agencies to report 95-percent-complete data for 10 data
fields the Department considered critical, such as students' social security
numbers and loan amounts guaranteed. However, these edit checks can
only determine completeness, not accuracy.
OCR for page 121
FINANCIAL AID DATA SYSTEMS
121
The department primarily uses information in the loan data base for
routine trend analysis (e.g., the number of students obtaining loans). These
analyses can be found in the Annual Guaranteed Student Loan Data [Book,
which is prepared by a contractor. Departmental staff reported to the panel
that at least some individuals within the department were dissatisfied with
the quality of the data, lack of some data items of interest, and the lack of
analysis with the existing data.
PROBLEMS WITH THE DATA SYSTEM
The General Accounting Office (1990bJ and various officials at the
Department of Education (e.g., Thompson, l991b; Winkler, 1991) have written
detailed critiques of several of the problems in the operation of guaranteed
student loan (GSL) programs. Briefly, the findings indicate the following:
· Financial management capabilities and maintenance of accounting
records are poor.
· Computer systems are inadequate. Data elements in the systems are
either old, unreliable, or not collected at all. The systems are not easily
adaptable and require major reprogramming to provide information when
policy, regulatory, or financial analyses raise complicated questions.
· Data and analyses are inadequate to (a) monitor guaranty agencies
and lenders, (b) predict more accurately default and interest costs, (c) im-
prove collections, (d) forecast volume, (e) outline borrower characteristics,
(f) analyze proposed legislative and regulatory changes, and (g) evaluate
program effectiveness.
· Guaranty agencies are not allowed access to the loan data base, a
procedure that would help prevent making a loan to someone who is in
default or in excess of loan limits, and thus ineligible.
To remedy these problems, the General Accounting Office (199Ob) rec-
ommended that the Secretary of Education:
· direct guaranty agencies to ensure that their student loan data are as
accurate and complete as practical, with special emphasis given to data on
defaulted borrowers, and provide updated information to the department.
· report the ability of ineligible borrowers to obtain loans as a material
internal control weakness under the Federal Managers' Financial Integrity
Act.
· analyze the data the department now has and provide to guaranty
agencies on a periodic basis the data needed to prevent defaulters from
obtaining new loans.
· develop a more comprehensive annual report that gives guaranty
agencies the data they need to more efficiently detect loan limit abuses, and
require guaranty agencies to use the report.
OCR for page 122
22
QUALITY IN STUDENT FINANCIAL AID PROGRAMS
A PROPOSED SOLUTION
The 1986 amendments to the Higher Education Act authorized the De-
partment of Education to develop a national student loan data system (NSLDS),
a computer system that would make national student loan data accessible to
guaranty agencies. The purposes of the system include improving data on
The system
default rates, reducing defaults, and reducing overpayments.
could be used to assist lenders and guaranty agencies in guarding against
borrower abuse. The General Accounting Office (199Ob:3) notes, however,
that "because the 1986 amendments contained a provision that the Depart-
ment could not require guaranty agencies to use the system before guaran-
teeing new loans, the system was not developed. The 1989 Budget Recon-
ciliation Act, however, allows the Department to require guaranty agencies
to use such a system before approving new loans. The Department plans to
develop the new system by 1993; the guaranty agencies will have to use this
system before guaranteeing new loans.
was awarded in early 1993. Winkler (1991) notes that data content, appli-
cations, and linkages with other GSL-related systems must be defined for an
NSLDS, and the prospective cost of the system must be scrutinized care-
fully.
Thompson (199lb) suggests that the Department of Education should
expedite the development of the NSLDS in order to protect the integrity of
the program more effectively. He notes that the system is crucial in provid-
ing departmental officials with part of the information they need to prevent
student-borrowers from abuses, such as exceeding statutory loan limits and
receiving additional loans when they are already in default.
The need for an NSLDS, as included in the 1986 amendments to the
Higher Education Act and as reemphasized in the 1992 reauthorization of
the act, will be even greater in the future. While the direct lending demon
~ " A contract to develop the system
stration project authorized in the act would affect the loan distribution sys-
tem, there will still be difficulties with students' address changes that have
to be kept up-to-date as they leave school, enter the work force, reenter
schools, and/or change schools. Generally, young people at this point in
life are difficult to track. In addition, the vast increases in amounts autho-
rized under the Parent Loans for Undergraduate Students program will re-
quire very careful attention if default rates are to be kept in check. Parents
who borrow $25,000 or more and are not able to pay it back may go into
default as much as a year before the current tracking system could identify
that default and deny another $25,000 loan.
Recommendation 6-1: The panel believes that a totally integrated ap-
plicant data base is essential for problem identification, the develop-
ment of solutions, and monitoring progress important activities if true
quality improvements are to take place. The data base would compile
OCR for page 123
FINANCIAL AID DATA SYSTEMS
123
information from all the Title IV aid programs and across years to
provide, for each applicant, a complete history of participation in the
financial aid system, thus facilitating a broad range of cross-sectional
and longitudinal statistical analyses. The Department of Education
should allocate sufficient resources and personnel to accomplish this
task without further delay.
OCR for page 124
Representative terms from entire chapter:
financial aid
