Efforts to Model Workload and Resource Requirements
This appendix reviews major efforts to model workload and resource requirements for federal immigration enforcement and similar criminal justice processes.
CHANGES IN THE WORKLOADS OF IMMIGRATION COURTS
In fiscal 2007, staff of the U.S. Department of Justice’s (DOJ’s) Office of Planning, Analysis, and Technology (OPAT) worked with a statistician to complete an analysis of immigration court workload. OPAT used data from the Executive Office for Immigration Review (EOIR) and the U.S. Department of Homeland Security (DHS).
The analysis (U.S. Department of Justice, 2008) indicated that EOIR has limited tools available for predicting its future workload. The bulk of the workload comes from “notice to appear” issued by DHS, and complete information on the number, issuing agency, and place of issuance is not available to EOIR in time for the predictions. Even if these data could be obtained, the relatively short time lag for 80 percent of the cases of less than 3 months between the issuance of a notice and intake by EOIR is not enough to provide for meaningful advance planning or budgeting.
A more useful indicator of EOIR’s potential workload would be the trend in apprehensions of non-Mexicans by the Border Patrol. Most non-Mexicans cannot be returned directly to their native countries, and they are likely to appear before immigration courts. Attempted unlawful entries by non-Mexicans respond to a variety of causal factors, but for
most countries, abrupt increases or decreases in the level of those apprehended and issued notices are unusual and can often be traced to specific events. During the study period, EOIR’s case intake tracked the number of apprehensions of non-Mexicans along the southern border with a time lag of several months. Again, this time lag does not permit long-range planning. EOIR’s Mexican and non-Mexican caseloads are significantly different. Mexicans who appear before EOIR generally do so because they have records of previous immigration violations or criminal charges are being brought against them. They are likely to be detained, and their cases reach EOIR faster than those of others. They are somewhat less likely to file for relief from removal than non-Mexicans, and even less likely to file for asylum, which leads to swifter resolution of their cases. Finally, the Mexican caseload that reaches EOIR has been growing since fiscal 2004, while the trend for the other major nationalities was down for fiscal 2006 and 2007. If EOIR is able in the future to obtain data on apprehensions from the Border Patrol by month, nationality, and location in a time-sensitive manner, it would be beneficial for short-term workload planning.
EFFECTS OF HIRING INVESTIGATORS ON THE WORKLOAD OF NONINVESTIGATIVE SYSTEM COMPONENTS
In 2005, the House Appropriations Committee expressed concern that the budget request submitted by DOJ, whose highest priority was the prevention of terrorism, did not fully support the budgetary needs of the criminal justice components. DOJ was directed to submit a report “describing how the hiring of an investigator impacts the workload of the U.S. Attorneys, the U.S. Marshals Service, the Office of the Federal Detention Trustee, and the Federal Prison System.” DOJ contracted with BearingPoint, Inc., which built a prototype workflow model (based on readily available data) to test the feasibility of the concept that mathematical relationships can be established and determine what areas should be pursued to build a functional model (U.S. Department of Justice, 2005). The prototype model illustrates the effects of hiring agents in the front end of the criminal justice system on the workloads of downstream agencies, such as the U.S. Marshals Service (USMS), the Office of the U.S. Attorneys (USAO), and the Bureau of Prisons (BOP). It uses data on the resources that were historically required to process the number of criminals received—explicitly assuming that the historical trends in these ratios will continue into the future with little fluctuation.
The model consists of inputs (agents added to the Federal Bureau of Investigation [FBI]; the Bureau of Alcohol, Tobacco, Firearms, and Explosives [ATF]; and the Drug Enforcement Agency [DEA]), which will create
the following outputs: number of U.S. attorneys; number of U.S. marshals; number of correctional officers; number of criminals arrested; number of arrestees detained; number of defendants prosecuted; and number of defendants sentenced to prison.
The first test of the model determined how many data points had predictive value for each component of the model. In this analysis, BearingPoint used trends observed from 1999 to 2001 (with current initiatives and trends being more heavily weighted) and predicted a value for 2002 on the basis of these trends. To evaluate the quality of the reliability of the predicted values, BearingPoint calculated the standard deviation of data for 1999-2002. (The higher the standard deviation, the more difficult it is to produce accurate predictive values in the future.) The prototype model produced 19 of 32 data points within the standard deviation, or approximately 60 percent.
Limitations of the model include the following:
• The assumption that historical case procedures used by DOJ components and historical trends in types of criminal activity will continue into the future with little fluctuation may not be realistic. Account should be taken of changes in underlying trends in criminal and law enforcement priorities and changing levels of productivities over time.
• The model is based on comparisons of total personnel to total outputs, rather than focusing on marginal, or year-to-year, increases in criminal processing due to the addition of investigative agents.
• Each district can focus on specific crimes and thus have statistics that are different from the national average. A district/regional approach would be needed, at least for some of the larger districts whose statistics differ substantially from the national level. However, understanding the historical workloads associated with these district statistics would require going directly to the agencies and gathering this information on a district level.
PROJECTING FEDERAL DETENTION POPULATIONS
Projecting future detention trends and estimating budgetary resource requirements for the criminal detention program has historically been a difficult task, at both macro and micro levels.
At the macro level, impediments to accurately projecting the detention population include the dynamic nature of the federal criminal justice process; on-going changes in federal criminal law and policy; changes in federal law enforcement priorities; and events external to the criminal justice process, such as unforeseen events that might cause mass ille-
gal migration to the United States. At the micro-level, these macro-level impediments translate to volatility in (1) the number of federal arrests and bookings reported to the USMS, (2) prosecutorial priorities and declination criteria, (3) offender or offense characteristics necessitating pretrial detention, and (4) case processing time that results from overburdened criminal justice resources. Accordingly, projecting the impact of systemic or short-term events or initiatives that will affect arrests and bookings is the greatest challenge in projecting the detention population.
The Office of the Federal Detention Trustee (OFDT) documented the challenge of doing such projections almost 10 years ago, and their basic approach for projecting the detention population is still used (see Scalia, 2004). The primary source of data for the OFDT detention population projection model is the USMS Prisoner Tracking System (PTS). OFDT receives extracts of PTS that include individual records of each prisoner processed by the USMS.
Time-series models lie at the heart of the population projection. These atheoretical models are based on the assumption that historic trends—and the factors that influenced those trends—are useful predictors of future events and that the observed relationships will continue into the near future. The time-series analysis produces weights that are used in a micro- simulation model that generates future booking replicates.
Recognizing that simple time-series models may not produce reliable results in an environment in which the underlying trend of a series can be substantially affected by exogenous factors, OFDT incorporated law enforcement and U.S. attorney staffing data into its process for estimating future detention. The staffing model has been described (by those familiar with it) as useful for incremental changes, but not for levels; it has also been characterized as informative but not definitive. The staffing model uses aggregate staffing data for the U.S. Customs and Border Patrol (CBP) and the U.S. Immigration and Customs Enforcement (ICE). In the context of “modeling the past,” the staffing model also contains indicator variables for things such as changes in administration.
At the tail end, OFDT tries to makes adjustments for policy initiatives and changes (i.e., they are not built into the model itself and do not necessarily have “data support”). With regard to the validity of predictions, the model does best when the policy environment is relatively stable. Time in detention, which is another model component, tends to be more stable and predictable than how many people come into the system. However, in the period immediately following the implementation of Operation Streamline, the length of detention fell in a way that was not foreseen by the existing model (although those predictions have since stabilized).
With regard to regional projections in the staffing model, OFDT can link staffing data to specific duty stations. The OFDT model does account
for district- and regional-level variations in law enforcement and prosecutorial productivity. Statistically, the accuracy of projections depends on the size of the base population, the variability of the data series trends, and the length of the forecast interval. One method for evaluating the validity of the projection methodology and the resulting projections is to monitor the individual components of future detention populations and identify which component is the primary source of the observed error.
The reliability of the OFDT model is evaluated on a monthly basis by using simple time-series methods to re-calibrate the original projections with real-time population statistics.
ESTIMATING WORKLOADS FOR THE FEDERAL CRIMINAL JUSTICE SYSTEM
The U.S. General Accounting Office (GAO) developed a model designed to provide Congress and federal agencies with estimates of the potential effect that budgetary changes for part of the federal criminal justice system may have on the system as a whole (U.S. General Accounting Office, 1991). The work was undertaken after GAO evaluated the existing criminal justice models and determined that they did not meet the needs mandated by Congress: they were either designed to address only a single part of the system or required data not routinely available at the federal level.
The model developed by GAO is based on ordinary least squares regression analysis with a zero intercept and no lag times. It assumes that historic trends are useful predictors of future events and that the historic relationships observed will continue into the near future. The accuracy of the model’s estimates of future workload may be limited by a significant change from the past budget and workload trends on which the model relies.
Limitations of the model include the following:
• General crime categories were used to make the estimates reliable (since specific crime types account for such a small portion of the total). The use of broad crime categories is a drawback if the user wants to estimate the impact of changes in resources for a particular crime type that has been combined with others to form a generic classification.
• The model provides only national estimates, which obscures differences among individual judicial districts.
• The model can only provide reliable estimates of the impact of resource changes within reasonable limits. For example, if
resources were increased by 50 percent in a single year, the estimates produced by the model would be unreliable.
• In order to provide useful results over time, the model will require annual updating of the mathematical formula on which it is based. This is necessary to reflect changes in the criminal justice system that may affect the relationships between resources and outputs.
PROJECTING SPACE NEEDS IN JUVENILE DETENTION AND CORRECTIONAL FACILITIES
The Office of Juvenile Justice and Delinquency Prevention (OJJDP) projects juvenile commitment populations by using a mathematical flow model (Butts and Adams, 2001). The model requires explicit assumptions about the case processing factors that might influence the size of confinement populations. The complexity of juvenile justice decision making virtually guarantees that detention and corrections populations will not closely follow arrest trends in the Violent Crime Index.
Analysts can produce more useful projections when they include juvenile court processing data in projection models, and projection models are more useful if they can account for changing patterns in court processing. Projection models are also likely to perform better when they include more than a single source of information and when they analyze more than a single point in the juvenile justice process.
The value of different projection scenarios is limited by the lack of more detailed data. For example, the models used in this analysis divided the population into only four categories of offenders, and projections would be more useful if offenses could be divided into additional categories.
SIMULATING THE IMPACT OF SENTENCING GUIDELINES ON PRISON POPULATIONS
In response to a congressional mandate that the U.S. Sentencing Commission evaluate the impact of its sentencing guidelines on the future prison population, the Bureau of Prisons adopted a simulation model (Gaes et al., 1993), FEDSIM, in 1987 as its primary source for projecting future inmate populations. The model overestimated (with a fairly high margin of error) the percentages of cases receiving straight probation. The explanations for this inaccuracy have to do with changes made to the guidelines after the initial modeling efforts. The model also greatly overestimated the number of split sentences; this may have had something to do with the modelers’ lack of prior experience with federal guideline
sentencing that would have informed them about judges’ behavior. However, the 3-, 4-, and 5-year projections for the federal prison population (the primary goal of the modeling effort) were quite accurate.
The overestimations reflect the fact that a simulation task is complicated when people affected by modifications in the system behave differently than they did prior to the changes. (This problem can be approached as an exercise as sensitivity analysis, which refers to the degree to which the outputs of a model are affected by changes in assumptions about the model’s inputs and its parameters.) The greatest error occurred in projecting future conviction rates trends for some of the offense categories. The reason the model was relatively accurate, despite the errors in conviction trends, was that the structural change in sentencing was so dramatic that it dwarfed the impact of changes associated with conviction trends. However, as time served stabilizes, it will become more important to accurately predict future conviction trends. It will also be important to separate out projections for certain groups of prisoners who have distinct causes for changes in admission rates and length of stay than the typical federal inmate.
This page intentionally left blank.