Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 105
Strengthening the National Institute of Justice 4 Research Operations and Staffing Resources The history and broad outline of the research portfolio of the National Institute of Justice (NIJ) described in this report reflect frequent changes in leadership, an increased oversight role for the Office of Justice Programs (OJP), several upswings and downturns in funding, and a shift from support for a predominantly social science research program to one in which the majority of funding is expended on technology-related activities. In an effort to better understand these changes and their ramifications, the committee examined NIJ’s research operations and staffing resources. In this chapter, we first describe how NIJ plans its agenda, develops solicitations, conducts peer review, awards grants, and reviews its research reports. We examined these processes in light of three overarching issues: (1) the transparency of the processes, (2) the relationship between the research and practitioner communities and NIJ staff, and (3) the relationship between NIJ and OJP. When documentation is available, we describe how the processes changed over time. We then describe NIJ’s staffing resources and analyze staff turnover and other staffing characteristics that have affected how NIJ conducts its business. Taken together, this analysis provides an important context for understanding and assessing the current program and its evolution into the research organization of today.
OCR for page 106
Strengthening the National Institute of Justice RESEARCH OPERATIONS Consistency, openness, efficiency, and fairness are characteristics associated with good research and development (R&D) project management (National Advisory Committee on Criminal Justice Standards and Goals, 1976). They are also useful as a yardstick against which one can measure the quality of a research agency’s operations and determine how well they contribute to fulfilling its mission. For our analysis of research operations, the committee reviewed written OJP guidelines and policies (Office of Justice Programs, 2007c) and also relied heavily on interviews with former and current staff as to how these processes worked in reality and over time. Anyone familiar with bureaucracies knows that often there is a divide between what an official policy or process proposes and how things are actually done. In this report, we have found it necessary to distinguish the two. We have also tried to describe the differences between the way the Office of Research and Evaluation (ORE) and the Office of Science and Technology (OST) handle various functions. As described earlier, until the late 1990s, the social science research activities dominated the science and technology ones. Consequently, the background information on operations primarily describes how social science research activities are handled. After the mid-1990s, when the OST program greatly expanded, these processes were either adopted by OST or were modified to reflect its own statutory requirements and programmatic needs. We have tried to indicate the nature of these changes and when and how they occurred. Planning Past Planning Activities Traditionally, planning at NIJ has been a bottom-up process, with staff playing a major role in determining research priorities. Although there had been some attempt to use an advisory board (see Chapter 2) to solicit input from a broader audience and build support for research activities, there is no documentation that the NIJ Advisory Committee in the 1970s or the NIJ Advisory Board in the 1980s contributed to a formal research plan. Rather than rely on an overall advisory board to advise it on long-range planning and annual priorities or on an advisory infrastructure for specific program areas, NIJ has convened topical meetings and workshops to solicit input from practitioners and researchers1 on its social science research ac- 1 Throughout our report, we use the terms “researcher” and “practitioner” to refer broadly to two groups of people who generally differ from each other in their training, expertise, and
OCR for page 107
Strengthening the National Institute of Justice tivities. One example of meetings aimed exclusively at practitioner groups is the 21st-century working groups convened during 1991-1993. Three separate groups of law enforcement, courts, and corrections practitioners provided informal guidance about the research agenda. The law enforcement group consisted of newly appointed and progressive police chiefs who met three or four times a year. The courts and corrections groups were convened less frequently but also assisted in identifying critical research needs.2 Through the years, NIJ staff have also participated heavily in conferences and meetings held by national criminal justice practitioner organizations, such as the International Association of Chiefs of Police and the American Correctional Association; research organizations, such as the American Society of Criminology and the Justice Research and Statistics Association; and meetings convened by other federal agencies, such as the Centers for Disease Control and Prevention. NIJ relies heavily on these meetings to exchange information and to generate interest in its activities.3 Until the mid-1990s, planning was conducted and coordinated by a director of planning who headed the Analysis, Planning and Management staff and reported directly to the NIJ director. The staff prepared budget requests for the Office of Management and Budget (OMB) and the U.S. Department of Justice, developed budgets, tracked spending, handled personnel matters, and oversaw an institute-wide planning process. The director of planning was responsible for communicating with outside groups, soliciting their input, and working closely with NIJ management and staff to develop an annual program plan. Strategic Planning NIJ undertook development of a long-range strategic plan in the early 1990s (National Institute of Justice, 1993). Six long-range goals were identified: (1) reduce violent crimes and their consequences; (2) reduce drug-related crimes; (3) reduce the consequences of crimes for individuals, households, organizations, and communities; (4) develop household, school, business, workplace, and community crime prevention programs; (5) improve the effectiveness of the criminal justice and service systems in responding to offenses, offending, and victimization; and (6) develop and evaluate information for criminal justice responses to changing and emerg- occupation. The term “researcher” is inclusive of all people whose primary training, expertise, and occupation involve research activities regardless of where they work. 2 E-mail from Craig Uchida, former director ORE (August 20, 2009). 3 Memorandum to David W. Hagy, director of NIJ, from Thomas Feucht, assistant director of ORE, on practitioner consultations, August 2, 2007.
OCR for page 108
Strengthening the National Institute of Justice ing crime patterns and for use of new technologies. The idea was to build a body of cumulative research—both basic and applied—and to develop practitioner-oriented activities around these as well. The result of this planning was the development of a substantial number of solicitations that were issued in fiscal year (FY) 1994. A matrix was developed that could be used to monitor progress in conducting the research over a 10-year period. With the change in leadership in 1994, the plan was relegated to an open solicitation and its use was abandoned.4 In FY 2002, NIJ developed a 3-5 year strategic plan.5 This plan, referenced in NIJ’s submission to OMB’s assessment process in FY 2005, identifies 7 broad strategic goals and 10 program areas. These same goals appear in NIJ’s 2002 Annual Report to Congress and represent a change from mission statements that appear in its 1999-2001 annual reports. After 2002, no similar statement of mission or strategic goals appears in NIJ’s annual report to Congress. More recently, NIJ took steps to develop a strategic plan and to obtain some feedback on its research priorities, in requesting this study from the National Research Council (NRC). The lack of a structured strategic planning process makes NIJ even more susceptible to having its priorities set by others.6 Sometimes these priorities are supportive of NIJ’s mission, but sometimes they are questionable. On the science and technology side, for example, one program that was developed at the request of the attorney general was research on new technologies to detect concealed weapons at a distance. This line of research had never received much support from practitioners, but NIJ staff agreed that it would be useful and subsequently requested and received additional monies from the attorney general’s office. A solicitation was issued and, according to NIJ staff, high-quality proposals were received. In a similar way, the school safety technology program became a priority because of congressional and departmental interest and continues to be supported today. But this priority did not engender the same kind of support from senior managers, who questioned whether the investment in new technology was an appropriate response to the spike in school homicides. 4 E-mail from Carol Petrie, former acting director of NIJ, August 7, 2009. 5 In response to the committee’s request for this plan, we received a memorandum dated August 15, 2002 prepared by Sarah Hart, then director of NIJ, to Deborah Daniels, assistant attorney general. According to the memo, NIJ’s initial attempt to develop a strategic plan was insufficient, and the revised plan was being submitted as part of the document. No other documentation regarding a strategic plan was uncovered by the committee. 6 With the exception of the planning efforts by OST, there is little documentation regarding the priority-setting process across NIJ. The information on concealed weapons and the school safety program that follows was taken from staff interviews.
OCR for page 109
Strengthening the National Institute of Justice Annual Research Plans Intermittently during the 1980s and 1990s, an annual NIJ research prospectus or research plan was published. It contained general descriptions of “strategic challenges” and focused almost exclusively on social science research activities, with increasing mention in the 1990s of other activities, such as crime mapping, DNA analysis, and standards development (National Institute of Justice, 1987, 1991, 1992, 1996, 1997, 1998, 1999). The documents also contained information on targeted or “directed” solicitations—for example, solicitations linked to 1994 Crime Act initiatives. After 2000, this kind of annual document signaling NIJ priorities to the research community was discontinued, until 2009, when NIJ released High Priority Criminal Justice Technology Needs (National Institute of Justice, 2009a), a document that describes how NIJ sets its research agenda. Unlike its predecessors, it focuses exclusively on the programs of OST. Although no research prospectus for NIJ itself was produced during the period 2001-2008, the Violence and Victimization Research Division of ORE prepared a detailed divisional strategic plan in 2005. Covering FY 2006 “and beyond,” the plan built on an earlier planning document written in 2001. It reflected the deliberations from a two-day planning meeting as well as ideas from an earlier Violence Against Women research agenda. The strategic plan contained descriptions and justifications for more than a dozen future solicitation topics. Currently, agenda setting and planning is decentralized, and ORE and OST present their plans separately to the director. Consultation on program priorities between the director and the senior management staff is conducted on an ad hoc basis. Although several people have had responsibility for planning in the director’s office since 2000, the current Planning, Budget, Management, and Administration Division does not appear to have major responsibility or oversight for institute-wide programmatic planning. Its responsibilities appear to be exclusively confined to providing financial oversight and monitoring activities, such as responding to audits and other financial inquiries, overseeing the packaging of grant proposals, and managing contracts. Office of Research and Evaluation Planning and agenda setting is handled differently by ORE and OST. ORE uses an ad hoc process that varies from year to year and from division to division. It relies heavily on staff expertise and consultation, primarily with practitioner groups, and policy makers through participation at professional meetings. In a listing of consultations with practitioners and policy makers for 2006-2007 prepared by ORE staff, approximately a dozen
OCR for page 110
Strengthening the National Institute of Justice meetings were held for the purpose of planning research activities. Also listed were NIJ staff participation in research advisory committee meetings of the International Association of Chiefs of Police, and other meetings involving groups of related programs, such as the Project Safe Neighborhoods, the Multi-jurisdictional Task Force Cluster, and the Serious Violent Offender Re-entry Initiative. There were also meetings to plan research workshops (e.g., on elder abuse, sexual violence). As mentioned earlier, ORE also sponsors periodic workshops and meetings on particular topics as the need arises. According to NIJ, it has held approximately two dozen meetings or workshops (it is unclear what distinguishes one from the other) since 2000, an average of about two a year. NIJ relies heavily on these “strategic planning” meetings to articulate new or emerging knowledge needs; to assess and evaluate a body of recently matured research findings; or to describe options and priorities for further research in a given area.7 The list of various meetings covering 2001-2008 is available online, along with many of the meeting agendas, participant lists, and summaries.8 Of the 27 meetings listed, 12 focused on sexual violence, victimization, and violence theory; 4 were on policing; 4 were related to forensic science, including 2 on DNA evidence; 2 were on drugs and crime; 2 were on terrorism; and single meetings were held on pretrial research, hate crime, and human trafficking. With one or two exceptions, most of the meetings involved a mix of researchers and practitioners, with the number of practitioners dominating a majority of the meetings. In general, the meeting materials reflect a broad overview of the subject, with a general listing of issues or questions needing research. With few exceptions, there appeared to be little emphasis on the science but more on what information practitioners need to do their jobs better. No further information on these meetings exists, and with few exceptions it was not possible for the committee to determine the relationship between research ideas presented at these meetings and any action taken by NIJ in response to them. There are no standing advisory committees for ORE as a whole or for any of its divisions. Occasionally NIJ-wide working groups have been formed, but these are strictly composed of NIJ and other OJP staff with an interest in the particular area. In 2007, according to ORE, there were at least eight of these working groups, as well as other internal NIJ working groups that participate in an activity known collectively as the Innovative Assessment Program. These internal NIJ staff groups meet for the purpose 7 Memorandum to David W. Hagy, director of NIJ, from Thomas Feucht, assistant director of ORE, on practitioner consultations, August 2, 2007. 8 A list of NIJ’s research meetings and workshops with links to available background materials is available online at http://www.ojp.usdoj.gov/nij/events/research-meetings.htm [accessed October 15, 2009].
OCR for page 111
Strengthening the National Institute of Justice of identifying, verifying, and assessing innovative criminal justice programs and policies.9 Once research topics are identified by the ORE divisions, they wind their way up to the director, following a series of meetings between staff and the ORE director. The process is not a formal one (i.e., no written guidelines or requirements). Similar meetings are held between the NIJ director and the OST director. The NIJ director then makes decisions on what particular research and programs will help accomplish the missions, with some consultation with the office directors and to some extent from senior management of OJP and the offices of the attorney general. The interviews with current NIJ staff reveal considerable differences in their knowledge of the steps in the planning process as well as the quality of the process itself. The overriding sense of these interviews was that the process was a haphazard one. Some described the process as “intermittent,” with no focus on long-range efforts. Generally, there was a sense that “hot topics” rule, and some areas are completely neglected due to scarce resources. Some staff attributed the ad hoc nature of the process to the fact that senior managers are not provided with budget numbers around which to plan. Others expressed the view that, with such limited resources, there was no incentive to plan. Ongoing consultation with the broader research community outside more formal planned meetings or workshops appears to be very limited and dependent on individual staff initiative. According to the staff interviews, some staff contact researchers directly to obtain feedback on particular issues and to keep abreast, and other staff depend on meetings with “experts,” presentations at professional association meetings, and contact with relevant Justice Department offices to provide opportunities to engage the research community. Office of Science and Technology In contrast to ORE, OST has had a history of working with a large and complex set of advisory and working group entities. It has relied on these groups to provide feedback on its overall program, to identify practitioner needs, and to coordinate with others conducting technology R&D (National Institute of Justice, 2009a).10 Currently, OST relies on the Law 9 Memorandum to David W. Hagy, director of NIJ, from Thomas Feucht, assistant director of ORE, on practitioner consultations, August 2, 2007. 10 From 1994 to 2003, Vice-Admiral Al Burkhalter, Jr. USN (Ret.), chaired an executive panel of members with science backgrounds that advised OST and provided detailed reviews of particular programs.
OCR for page 112
Strengthening the National Institute of Justice BOX 4-1 Technical Working Groups (TWGs) Biometrics Body Armor Communications Technologies Community Corrections DNA Forensics Electronic Crime Explosives General Forensics Geospatial Technologies Information-Led Policing (Institutional) Corrections Less-Lethal Technologies Modeling and Simulation Personnel Protection Pursuit Management School Safety Sensors and Surveillance Enforcement and Corrections Technology Advisory Council (LECTAC),11 a group of approximately 40 senior-level representatives from federal, state, and local criminal justice agencies; labor organizations; and international criminal justice organizations. They meet once a year to review field needs developed from various technical working groups (TWGs) and make a list of the top-10 R&D priorities. Through this list of priorities, NIJ uses practitioner-based input to help shape the activities of its S&T portfolio. Although the number of active TWGs has fluctuated, in 2008, there were 17 TWGs developing high-priority needs and reporting them to LECTAC (Law Enforcement and Corrections Technology Advisory Council, 2008). See Box 4-1 for the core technology portfolios represented by these TWGs. 11 A law enforcement advisory body, LECTAC has its roots dating back nearly 30 years before the establishment of the current TWG process (Law Enforcement and Corrections Technology Advisory Council, 2008). The Technology Assessment Program Advisory Council (TAPAC), established in the mid-1980s and consisting of more than 80 senior federal, state, and local law enforcement officials from the United States and Canada, preceded LECTAC (National Institute of Justice, 1994a). TAPAC was preceded by the National Advisory Committee for Law Enforcement Equipment and Technology, established in 1977 under NILECJ. According to David Boyd, former director of OST, LECTAC played a larger role in planning, and the TWGs began as subgroups of LECTAC.
OCR for page 113
Strengthening the National Institute of Justice As their titles indicate, some focus on specific technologies (e.g., body armor), some on criminal justice areas (e.g., community corrections), and others on crime problems (e.g., school safety). Most, if not all, TWG members are criminal justice practitioners appointed by the OST deputy director and approved by the NIJ director. TWGs do not have formal policies or procedures regarding their size, management, or activities. There are no term limits on membership; members simply have to be actively employed. The TWGs meet twice a year to develop, rank, and finalize a list of priority technology needs in their respective areas. Each TWG generates about 50 priority needs, which results in a long list of roughly 900 technology needs. OST estimates that it is able to address one-sixth of them through competitive R&D and another one-sixth through operational testing and demonstration.12 The needs that are addressed depend on budget and directives as well as gap analyses conducted by OST staff to determine that no other federal agency is investing in similar technologies. The OST process is well regarded by the OST staff and by many practitioners who are familiar with it through their association on the TWGs. However, the current process reflects numerous deficiencies. Planning by OST does not appear to include systematic or formal outreach to the research or academic communities. Researchers are seldom included as members of either the technical working groups that develop the initial list of “technology priority needs” or on LECTAC, which selects the overall priorities. Although one might argue that technology priority needs are not meant to be the same thing as research priorities, the technology priority needs are the basis for the operational requirements that NIJ uses to provide direction for its research and development activities. According to NIJ, its solicitations for research and development of new tools and technologies for criminal justice practice are advised by TWGs and LECTAC recommendations (National Institute of Justice, 2009a). Although the TWGs play an important role in identifying practitioner needs, and through those needs identify priorities, they do not operate like formal scientific advisory groups. With the exception of the DNA Forensics TWG, their membership does not include scientists who can use their expertise to assess the state of the art or the methodology required to move the technology forward. The absence of scientific or technical advice means that needs are developed in a vacuum, without input as to what is possible. As a result, opportunities may be missed. TWG meetings are not open, nor are their deliberations made public. The Homeland Security Act of 2002 specifically exempts the TWGs from the Federal Advisory Committee Act (FACA) rules which require notification of meetings and access by the 12 E-mail from George Tillery, OST deputy director, May 16, 2008.
OCR for page 114
Strengthening the National Institute of Justice public to documents and meetings. As a result, they do not seek or elicit guidance from the broader scientific and practitioner community. The current TWG structure is defined around the OST R&D portfolio areas and is not intended to be static. The TWGs are expected to adapt as priorities in the field change or as new technological capabilities emerge (National Institute of Justice, 2009a). However, the committee noted at least one example of a TWG (Corrections TWG) continuing to meet despite the absence of funded research awards in their respective portfolios over a number of years. OST’s investment portfolios are likely to change first, because of limits on available funds or office reorganization, before the TWGs are restructured. As such, the advice coming from the TWGs often falls into a void, and it is unclear to what extent their advice and recommendations can and do result in solicitations and subsequent awards. According to OST’s research, development, testing, and evaluation process (described in Chapter 3), if practitioner needs are not addressed in research and development programs because potential solutions already exist (or perhaps development is already sponsored by another agency), these needs are then addressed in Phase 4 of the process, in which NIJ considers supporting demonstrations and operational evaluations of potential solutions, or in Phase 5, in which NIJ considers publishing resource guides or standards documents that may be useful to the practitioner community. The elaborate advisory structure is used more often to inform the broader federal research community of the technology needs of criminal justice practitioners and to define NIJ’s dissemination and outreach practices regarding technological capabilities than to define its technology research and development agenda. Thus, while the OST process is standardized and more formalized than the ORE process, the OST process shares some major deficiencies. It is unclear what guidance and advice either office receives from the research community. In the case of ORE, there is some attempt to document researcher input by posting agendas and summary minutes of meetings and workshops on specific research topics on the NIJ website. But it is very unclear how final decisions about priorities are made or how these decisions in turn influence which solicitations are released. It is clear that both offices rely almost exclusively on staff judgments as to what the state of the art is, what is needed, what is feasible, and how a particular research technology or question should be approached. The committee has doubts as to whether the scientific and technical qualifications of the staff are up to the task. Consequently, the committee thinks that the advice and guidance of researchers need to be made a meaningful part of a sustained agenda-setting process.
OCR for page 115
Strengthening the National Institute of Justice Solicitations Directed and Field-Initiated Solicitations The committee heard brief descriptions of NIJ’s solicitations in presentations by staff and was told that, to understand NIJ’s research priorities, we should look at their solicitations. NIJ maintains an archive of solicitations from 1996 to the present.13 From this archive, the committee reviewed the types of solicitations over the years, changes in topics, and changes in deadlines. In addition, we received from NIJ a list of titles for applications received and subsequent awards by solicitation for the period 2004 to 2007. ORE and OST issue different kinds of solicitations to the field. ORE funds solicitations that are “directed” and describe the scope of the research or evaluation and the nature of the proposed research activity. It also issues an annual “open” or “field-initiated” solicitation. This kind of solicitation allows the researcher creative license to propose research studies on a broad array of topics. Typically $500,000 to $1.5 million is set aside for each directed solicitation and from $2.1 to $2.8 million for the annual field-initiated solicitation. However, in FY 2008, $5.4 million was awarded under the field-initiated Crime and Justice Research solicitation. OST issues solicitations dealing with a specific technology area (e.g., communications, less lethal technologies, biometrics) that call for research studies and other kinds of activities to solve problems related to adoption and use of new technologies by criminal justice agencies. The solicitations are usually structured in two phases: submission of a concept paper, followed by a full proposal if the concept paper is approved. Concept papers under OST solicitations, which were introduced in 2005, differ from full proposals in that they are much shorter and do not contain detailed budgets. All concept papers and subsequent proposals are subject to review by a panel comprised of expert practitioners and technologists14 (National Institute of Justice, 2006b). An average of $1 million is designated for grants funded under each solicitation for targeted technology development. For some solicitations such as Personal Protective Equipment and Corrections Technology, $250,000 is set aside while up to $3 million is set aside for the Forensic DNA Research solicitation. OST also issues a second kind of solicitation, which calls for capacity-building activities. These are issued for such programs as the Fo- 13 See http://www.ojp.usdoj.gov/nij/funding/expired-96-07.htm and http://www.ojp.usdoj.gov/nij/funding/expired.htm [accessed November 24, 2009]. 14 NIJ uses the term “technologist” to refer to scientists and engineers; it can include academicians. Technologists are often program managers from one of NIJ’s federal partner agencies.
OCR for page 134
Strengthening the National Institute of Justice by NIJ, and 3 were assigned to the Bureau of Justice Assistance. NIJ continues to make use of SETA contractors to provide scientific and analytic services; they are used to create an expanded work capacity and quick turnaround to meet immediate program requirements. They also free up program managers, who can then respond quickly and proactively to new developments. SETA contractors are subject to certain restrictions. Continued reliance on contractors may weaken NIJ in the short and long term. Contractors are not treated like federal employees. They cannot speak for NIJ at meetings or conferences and are not allowed to access certain types of information (e.g., the electronic grant management information system). This restriction was noted by NIJ grantees, who complained that the SETA contractors with whom they consulted were limited in their ability to handle certain grant monitoring functions. As a result, the needs of researchers may not be getting met. Also when contracts expire and are not renewed, the resulting turnover may be detrimental to NIJ’s stability. In FY 2008, 75 percent of the contractors supported the Forensic Science Division, the one that handles approximately 75 percent of the NIJ budget.31 Staff Turnover and Vacancies As indicated above, the past decade has been one of staff growth and decline, with very high turnover occurring at all levels of the organization. Looking at the period 1998-2008—only 21 of the original 111 NIJ staff from 1998 remained in 2008.32 Of these 21, 15 individuals continued to have responsibility for research-related activities. Comparing two points in time—2002 and 2008—63 individuals (out of 93) who appeared on the 2002 rolls were gone by 2008, and 41 new staff were hired or transferred to NIJ in the interim and on board in 2008. According to a different source, information on NIJ staffing reflects that, during a 5-year period from January 1, 2002, to November 24, 2007, approximately 76 persons separated or transferred out of NIJ, many of whom held management positions.33 The 76 included 12 division director positions distributed among 9 different divisions, 2 deputy director positions (OST and ORE), an NIJ director, and an NIJ deputy director.34 Turnover is to be expected during changes of administration. But personal loyalties do not explain such a great exodus by career staff, many of 31 E-mail from George Tillery, acting director, OST (June 27, 2008). 32 This figure is based on data from the NFC on NIJ staff for the years 1994-2008. 33 The list of OJP losses by separation or transfer was provided by the OJP Office of Audit, Assessment, and Management. 34 During our own brief tenure, the committee witnessed the resignation or transfer of one NIJ director, one deputy director, one ORE director, and one OST director. Several division chiefs have also been shifted within NIJ to vacated management positions.
OCR for page 135
Strengthening the National Institute of Justice whom took lateral positions at other agencies or research organizations. Of the 15 former staff whom the committee interviewed, all had left NIJ during the period 1998-2006. Among the reasons they cited for leaving were lack of leadership, lack of opportunity to use their methodological expertise, and lack of a culture in which substantive expertise was highly valued and could be nurtured (see the discussion on intramural research later in this chapter). A presidential initiative intended to restructure the federal workforce and streamline federal agency operations may also have affected staff turnover at NIJ. Known as the A-76 strategy, after the OMB Circular A-76 for which it was named (U.S. Office of Management and Budget, 1983), it involved classifying positions as “inherently governmental” or “commercial” and opening the latter to competition. The process essentially requires that private companies and in-house personnel engaged in winner-take-all competitions for the tasks in hand. Despite numerous arguments being made by OJP offices as to certain positions being intimately related to the public interest and as such must be performed by government employees, by February 2003, 79 of 98 positions at NIJ were deemed to be “commercial” and therefore open to competition. The NIJ positions included 61 of 79 research grant-monitoring positions, as well as administrative support and information technology jobs (Consortium of Social Science Associations, 2003). In September 2004, the procurement process was initiated, and, for the next 2 years, many staff believed their jobs were in jeopardy pending the outcome of the competition. In September 2006, the Department of Justice cancelled the OJP procurement, after the congressional appropriators directed that any A-76 action related to OJP needed to be brought to them in advance and special justification submitted for any program changes that reduced the personnel of an agency by 10 percent or more (U.S. House of Representatives, 2005). But by then many senior staff had departed. It is obviously difficult to project whether or not such a high rate of turnover will continue and what the impact might be. However, one possible indicator of future turnover among senior staff is the number of staff eligible for retirement. As of April 30, 2008, NIJ had 12 employees eligible for retirement, 7 at the grade service (GS)-14/15 level and 1 at the executive-service level. Contrasted with these numbers is NIJ’s authorized staffing allocation of 87. On February 14, 2008, 21 NIJ positions remained vacant, almost one-fourth of its full-time equivalent positions.35 These vacancies included seven 35 The committee obtained OJP-approved NIJ organization charts dated September 28, 2006; November 21, 2007; and February 14, 2008. Each showed an approved allocation of 87. Information for earlier years was unavailable. Only the organization chart dated February 14, 2008, showed vacancies.
OCR for page 136
Strengthening the National Institute of Justice supervisory positions (either a chief or deputy chief of every ORE division, chief of the Planning, Budget, Management, and Administration Division, and chief of one OST division). In addition, there were two senior-level social science analyst positions at the GS-14/15 level and a physical scientist at the GS-14/15 level. Prior to filling a vacancy, the NIJ director must seek approval from the assistant attorney general, OJP, through the Human Resources Division, Office of Administration. NIJ managers informed us that they were not allowed to move ahead with vacancies. It is possible that some vacancies were advertised but not filled, and we know of one position in the Office of the Director for which this was true. The committee surmises that the sheer number of vacancies reflects NIJ’s inability to obtain approval to move ahead and do what is necessary to fill them. There was consensus among the former and current staff who spoke to us that NIJ is understaffed. While many factors affect how many staff a specific program will require, the committee is unaware of any formal standards by which to judge the staffing levels of federal agencies. But agencies do use some kind of yardstick to determine approved staffing allocations; given budget constraints, one can assume that these allocations will be based on conservative estimates of what is needed. Certainly for the past three years, based on the approved allocation and the number of positions that remain unfilled, NIJ appears to be severely understaffed. Staff Qualifications Education Levels Given that NIJ staff are responsible for developing and managing a research portfolio, the committee was interested in learning about the staff’s educational training and whether it reflects substantive knowledge of relevant science as well as training in research methodologies. We were also interested in understanding the responses to our survey of researchers and practitioners, in which only 57 percent of the respondents were satisfied with the qualifications of staff and researchers were less satisfied than practitioners (53 versus 64 percent). (See Appendix B for a discussion of the survey and its results.) Information on individuals’ educational attainment proved almost impossible to obtain. The committee was informed that educational degrees are considered private information. The committee notes that this is not the case at selective NIH institutes that provide full staff listings including bios and degrees on their web pages.36 The committee was provided with 36 See http://www.nida.nih.gov/about/organization/despr/OffofDirHome.html [accessed April 19, 2010].
OCR for page 137
Strengthening the National Institute of Justice a breakdown of educational attainment by division, which was compiled by NIJ from information provided by NFC.37 Although questions remain about the data, one can conclude that the proportionate number of programmatic staff with advanced degrees (doctorates and postdoctorates) has declined since 1995 in both ORE and OST, most dramatically in OST. In 2008, only 3 persons out of 20 with programmatic responsibilities in OST had doctorate degrees or postdoctoral training. In ORE 9 persons out of 23 had advanced degrees, a lower percentage than in 1995 or 2001. The rise in the number with advanced degrees in ORE in 2001 dovetails with the large numbers of staff recruited during the period 1995-2000. Given the high staff attrition after 2001, it is not surprising that the number of staff with advanced science training also declined. The committee recognizes that educational attainment provides a very limited view of staff qualifications, particularly since we were unable to obtain information on the fields in which degrees were granted. Nevertheless, a research agency needs to be staffed by persons with strong academic and research credentials. Experience and freshness of that experience are also important. One specific program that several committee members reviewed and discussed with NIJ staff was the centers of excellence. On the basis of discussions with NIJ staff, review of the applicants’ proposals, and meetings with center staff, the committee questions whether OST staff have the technical experience and expertise to evaluate the applicants’ qualifications or the work of the centers. Not all jobs or all responsibilities will require the kind of academic training that a doctorate or postdoctorate signifies. However, a critical mass is needed of staff with advanced training in the sciences that can be called on to provide the kind of scientific advice and guidance required and to signal to the research community that NIJ staff fully understand the nature of the research enterprise. One strategy for attracting and retaining this kind of expertise is to offer opportunities to conduct intramural research. The committee notes that other science agencies, including NIH and the National Institute of Standards and Technology, have staff actively engaged in conducting research. In the 1970s and 1980s, NIJ had a division specifically devoted to intramural research. But the function was eliminated at some point during an NIJ reorganization. In 1994, an 37 NIJ worked from NFC lists of staff that divided staff education levels into 22 codes. NIJ staff worked with OJP’s Office of the General Counsel to group those codes into broader categories because of concern that if the cells were too small, individual staff might be identified. NIJ also eliminated staff that were at the GS-7 level or lower and staff at the GS-9 or GS-10 level, as well as anyone who was a “technician” of some sort. People who were budget analysts or program analysts in high grades remained. No further clarification from NIJ was forthcoming regarding the specific codes and the kinds of degrees included in them. Since we were not allowed to see the raw data, we were unable to verify the information.
OCR for page 138
Strengthening the National Institute of Justice intramural research program was resurrected, and staff members with the interest and qualifications to conduct intramural research were specifically recruited. By 1996, 16 NIJ staff members were identified as authors of papers and reports emanating from the intramural research program (National Institute of Justice, 1996). Since then, some staff members have continued to engage in intramural research, but this activity is not offered as a selling point to attract qualified staff, nor does there appear to be any incentives for it. NIJ should consider the role of a formal intramural research program as a way to attract talented criminologists who can bring new energy and expertise to its organization. Such a program can also serve as a strong incentive to retain staff who are well qualified to manage the extramural research. Positions and Grade Levels Most civil service employees hold a position that is part of a job series and a grade level. Exceptions to the broad generalizations here include among others the military, commissioned officers of the Public Health Service, Bureau of Veteran Affairs medical personnel, “special government employees” who serve on committees and in other ways, persons hired through the Intergovernmental Personnel Act. Each job series contains one or more position descriptions, and each position carries with it a set of requisite skills. The grade level determines the rate of pay and amount of authority. The lowest level is GS-1 (an abbreviation for grade service-1) All jobs at GS-7 or above require some form of specialized training, by either government or the private sector. Those entering the job market who have master’s degrees may qualify for GS-9 and those with a doctorate may qualify for a GS-11 position. The majority of staff in ORE fall under the 101 job series that includes social science analysts or social science program specialists. Supervisors and persons with lead responsibility have the title of “supervisory social science analyst” or “lead social science analyst.” This was the prevailing category in 1998 as well as in 2008. Typically NIJ staff are recruited for these positions at the GS-12 or GS-13 level and compete for a GS-14 level position when one is advertised, but this occurs infrequently and is in fact a source of great frustration to the career staff. OST, however, has undergone a major change in the various job series of its staff. In 1998, OST had six staff members who held GS-14 and GS-15 positions.38 Of these, four were classified as operational research analysts and fell under the 1515 series, which emphasizes having “competence in rigorous methods of scientific inquiry and analysis” and 38 Supervisory positions can be at either at the GS-14 or the GS-15 level.
OCR for page 139
Strengthening the National Institute of Justice is geared toward those with an operations research education.39 In 2008, eight OST staff persons were classified as GS-14 and GS-15. Only one of these is classified as a 1515 operational research analyst position; four are in the 1301 physical scientists series, one is in the 801 general engineering series, and two fall into the administrative and management 340 series. The physical scientist 1301 series is a catchall series and includes work in the physical sciences that is either not classified elsewhere or that combines physical science fields, with no single one predominant.40 It does not include a research competency qualification. Although it is difficult to say with certainty, without knowing more about people’s backgrounds and training, it does appear that there has been a shift in the staffing of OST from persons with greater research expertise to persons with more generalized science and technology knowledge or administrative experience. It is difficult to draw exact comparisons with other federal research agencies. Both NSF and NIH have the authority under United States Code Title 42 to appoint scientific positions without regard to the provisions of Title 5, which governs federal appointments. This allows the directors considerable discretion in setting requirements for scientific positions and salaries. Typically, the health scientist administrator position at NIH, a position that most closely approximates the responsibilities of the social science analyst position at NIJ, is targeted at the GS-13 and GS-14 level and specifies a Ph.D. education level; the director has the ability to hire at the GS-14 level. The net result is that NSF and NIH have much more flexibility in recruiting specifically for persons with advanced degrees and scientific skills and in setting pay levels higher than what could be typically done by NIJ. Organizational Culture Staff resources serve as the foundation of any organization and can be quantified to provide a rough picture of how well equipped an organization is to carry out its mission. There are also many other nonquantifiable elements that can affect the quality of the workplace and the work that is performed, and organizational culture is certainly one of them. Although it is difficult to document the psychology, experiences, and values that comprise organizational culture, a fairly consistent picture emerged from our interviews with former and current NIJ staff as well as from a summary of findings from an in-house survey of NIJ’s staff conducted in September 39 See http://www.opm.gov/qualifications/standards/IORs/gs1500/1515.htm [accessed March 9, 2010]. 40 See http://www.opm.gov/qualifications/standards/IORs/gs1300/1301.htm [accessed March 9, 2010].
OCR for page 140
Strengthening the National Institute of Justice 2007. Taken together, these reflect an organization in which there is considerable confusion and disagreement as to the roles and responsibilities of the top leadership, the mid-level managers, and line staff. With few exceptions, the staff faults the failure of top leadership to provide direction and sufficient responsibility to the staff. Mid-level managers are viewed as undeserving of their salaries when others are doing the “real” work of the agency. Finally, the program or line staff most heavily represented in the interviews and surveys expressed very different opinions as to what aspects of their work are or should be most important. According to the NIJ in-house survey summary of 35 current employees, respondents are described as falling into two groups: (1) those who believe program administration is at the heart of their work and most important and (2) those who believe that the substantive work of knowledge development, dissemination, and other service to the criminal justice field is most important. In contrast, former and current NIJ staff persons who were interviewed did not express views that would identify themselves as persons strongly committed to either knowledge development or to program administration. These interviewees assigned great importance to the need for strong substantive and communication skills and the need to staff the agency with persons with strong subject-matter expertise and R&D (particularly on the technology side) experience. One plausible explanation for this difference may be that, of the 26 interviewees, 15 were former staff and 11 were current staff, and the median length of service for both groups was fairly long at 9 years for former staff and 7 for current staff. None mentioned a similar kind of conflict, nor did anyone describe day-to-day activities in the same way. One area in which those interviewed and surveyed appeared to be in agreement was that attention to the mundane operations exceeded that spent on substantive issues. Examples of such unrelated research tasks include resolving issues of language with the NIJ website, estimating quarterly travel expenses, responding to short turnaround times on reviews of OJP documents, and resolving personnel issues. In any assessment of an agency that involves interviews with former and current staff, one expects to hear criticism, but the extent and frequency of the critical comments went well beyond what would be considered normal complaints about a bureaucracy. Taken together, these views reflect an organization that is urgently in need of strong leadership that can resolve many staffing issues that prevent it from having a more positive work environment.
OCR for page 141
Strengthening the National Institute of Justice CONCLUSIONS Three conclusions emerge from the committee’s analysis of NIJ research operations and staffing resources. Researchers currently play a limited role in NIJ’s research operations. In reviewing NIJ’s research operations, the committee was struck by the limited role that researchers currently play in the planning, peer-review, and report review processes. NIJ has never used an advisory infrastructure to help with planning or to provide feedback in a focused and consistent way. The lack of researcher involvement in current planning processes is particularly evident in the technology research as well in the social science research areas. Even when the advice of researchers is solicited, as it is with peer review of grant applications and report reviews, the scientific judgments of researchers do not take precedence over the judgments of practitioners, who are also being asked to assess scientific quality and to numerically rank applications using the same criteria. In the case of peer-reviewed reports, it is not clear whether scientific merit trumps other criteria, such as relevance and utility, and, unlike peer review of applications, practitioners and researchers are not brought together to discuss their concerns and to arrive at a consensus. The committee is not aware of any science agency in which the research community is less involved in providing guidance and feedback or in which practitioners play a coequal role in assessing research proposals and research reports. One possible reason for this is the perceived need, on the part of NIJ or of OJP, to equalize the contributions made by practitioners and researchers in order to ensure that NIJ research is policy relevant. Whatever the reason, NIJ appears to have devalued the contribution of researchers and distanced itself from the research community, even to the extent that the term “researcher” is not used to describe the participants in the peer-review process.41 This distancing is also reflected in NIJ’s organizational culture, in which contact with researchers other than grantees appears very limited or avoided, on the grounds that such contact may create a competitive advantage or encourage an “old boy” network. 41 “Proposals received under a solicitation are reviewed by independent peer panels, comprised of technologists (emphasis added) from academia, industry, and government organizations, along with practitioners from Federal, State and local agencies. Once reviewers have completed evaluations, NIJ Program Managers recommend individual proposals to the NIJ Director, who makes final award decisions” (from the NIJ website at http://www.ojp.usdoj.gov/nij/funding/proposal-review.htm [last updated November 15, 2007] [accessed December 10, 2009]).
OCR for page 142
Strengthening the National Institute of Justice There is a lack of transparency and consistency regarding NIJ’s research operations. Nothing affects the stature of a research institute more than a lack of transparency as to how funding priorities are arrived at and funding decisions are made. Because the grant award process is a highly competitive one and researchers must invest considerable time and effort in preparing applications, researchers will not seek support from an institution they believe is arbitrary in its decision making. The committee heard firsthand from the NIJ leadership that the grant award process was altered from one in which individual applications on a particular topic competed against each other to one in which all applications for numerous solicitations were grouped together and considered for the same pool of funds. The change in the grant award process was not announced to applicants prior to their submitting an application, nor were NIJ staff aware that this might occur. Not being clear on how decisions are made or, worse, changing the rules in the middle of the game, has serious implications for researchers’ perceptions of the fairness and consistency of NIJ’s practices. Once these perceptions are formed, they are difficult to change. The committee also encountered considerable difficulty in obtaining documentation for NIJ’s research operations as well as other kinds of administrative information. While we note that most programmatic as well as administrative information is maintained by OJP, there are steps NIJ can take to document its research processes, improve its record-keeping, and become a more transparent operation. One of these is to improve access to data through the NIJ website. Despite numerous improvements that have been made during the course of our study, the committee found it difficult to determine whether research grants had resulted in available final reports or to trace titles of existing reports or publications back to grants. Many websites of research agencies42 have a searchable online database of research awards that allows users to search for awards by topic, principal investigator, or grantee institution. In addition, these agencies provide funding amounts, names of principal investigators, and abstracts for each award. NIJ does have an archive list of awards dating back to 1995 online that are sorted by topic and, in recent years, by solicitation as well; funding amounts and grantee institution are provided for each award listed. However, names of principal investigators ceased to be included after 2003, there are no abstracts or descriptions available for 42 See, for example, the Institute for Education Sciences at http://ies.ed.gov/funding/grantsearch/index.asp and NSF at http://www.nsf.gov/awardsearch/ [accessed July 28, 2009].
OCR for page 143
Strengthening the National Institute of Justice the awards, and there is no mechanism to distinguish research grants from other types of capacity-building or technology assistance grants. Recognizing the importance of transparency and conducting a thorough assessment of its research operations, financial, and administrative records to determine how it might be achieved are important steps for NIJ to take. OJP’s oversight of various aspects of NIJ’s research operations and administrative functions is incompatible with NIJ’s role as an independent research organization. The committee identified at least two research operations in which OJP plays a heavy hand: the approval of solicitations and the peer-review process. In the case of the solicitation review process, OJP appears to have created a lengthy and convoluted process in which NIJ solicitations undergo extensive review by numerous OJP offices, and the focus of the review appears to go beyond what are strictly legal or financial issues. The centralization of peer review has also resulted in several requirements that may not be best suited for a research organization. All solicitations are not equal, and NIJ needs the authority to assess the peer review process in light of its own needs and to make changes it deems necessary. Under the current system, this is difficult to do. NIJ’s staffing resources have also been greatly impacted by OJP’s oversight and control. The analysis of NIJ’s staffing resources paints a picture of an organization that has experienced great variations in staffing size and composition caused by high turnover during the period 1994-2008. For many of these years, NIJ has been badly understaffed because it was prevented by the OJP leadership from recruiting against its approved staffing allocation. Management positions have been particularly affected, and with two or three exceptions there is almost no one currently in a management position who held that same position at NIJ as recently as two years ago. There also appears to be a shift from NIJ staff with strong science backgrounds to those with a more generalized knowledge of technology and a continued reliance on contractors to compensate for this lack of research experience and technology expertise. While it is difficult to show cause and effect, the committee thinks that these changes have taken a great toll on NIJ’s current staff, as reflected by its current organizational culture. NIJ staff interviews also reflect frustration with the lack of administrative support, and the nonresearch-related demands growing out of OJP oversight. While remaining very understaffed, NIJ has a cadre of experienced people who expressed a strong desire to produce high-quality research and to be of service to the criminal justice field. For this to occur, NIJ needs control over its own operations, needs to undertake a thorough self-examination
OCR for page 144
Strengthening the National Institute of Justice in an effort to improve its operations and become more transparent, and needs the authority and capacity to make the changes. Key among these is for NIJ to ensure the proper role of science in its decisions and to involve the research community in a more meaningful way in accomplishing its mission.