National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

Achieving Effective Acquisition of Information Technology in the Department of Defense

Committee on Improving Processes and Policies for the Acquisition and Test of Information Technologies in the Department of Defense

Computer Science and Telecommunications Board

Division on Engineering and Physical Sciences

NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W.
Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by Contract No. W911NF-07-C-0115 between the National Academy of Sciences and the Defense Information Systems Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.

International Standard Book Number-13: 978-0-309-14828-3

International Standard Book Number-10: 0-309-14828-6

Copies of this report are available from the

National Academies Press,

500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.

Copyright 2010 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine


The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.


The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.


The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.


The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.


www.national-academies.org

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

COMMITTEE ON IMPROVING PROCESSES AND POLICIES FOR THE ACQUISITION AND TEST OF INFORMATION TECHNOLOGIES IN THE DEPARTMENT OF DEFENSE

WILLIAM H. CAMPBELL,

BAE Systems, Inc.,

Co-Chair

DAWN C. MEYERRIECKS,1

Dawn Meyerriecks, LLC,

Co-Chair

ROBERT F. BEHLER,

MITRE Corporation

PHILIP E. COYLE III,

World Security Institute

RENATO A. DiPENTIMA,

SRA International (retired)

JOHN M. GILLIGAN,

Gilligan Group, Inc.

JOHN GOODENOUGH,

Carnegie Mellon University

PAUL J. KERN (NAE),2 

The Cohen Group

H. STEVEN KIMMEL,

Alion Science and Technology

DEIDRE A. LEE,

Professional Services Council

JOSHUA S. LEVINE,

ESP Technologies Corporation

NACHIAPPAN NAGAPPAN,

Microsoft Research

FRANK A. PERRY,

Science Applications International Corporation

VAHO REBASSOO,

The Boeing Company

DANIEL C. STURMAN,

Google, Inc.

Staff

JON EISENBERG, Director,

Computer Science and Telecommunications Board

KEVIN LEWIS, Senior Program Officer

LYNETTE I. MILLETT, Senior Program Officer

RENEE HAWKINS, Financial and Administrative Manager

VIRGINIA BACON TALATI, Program Associate

MORGAN MOTTO, Program Associate (through April 2009)

1

Dawn Meyerriecks resigned from the committee in September 2009 upon her appointment as Deputy Director of National Intelligence for Acquisition and Technology.

2

National Academy of Engineering.

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD

ROBERT F. SPROULL,

Sun Microsystems, Inc.,

Chair

PRITHVIRAJ BANERJEE,

Hewlett Packard Company

WILLIAM J. DALLY,

NVIDIA Corporation and Stanford University

DEBORAH ESTRIN,

University of California, Los Angeles

KEVIN C. KAHN,

Intel Corporation

JAMES KAJIYA,

Microsoft Corporation

JOHN E. KELLY III,

IBM Research

JON M. KLEINBERG,

Cornell University

WILLIAM H. PRESS,

University of Texas, Austin

PRABHAKAR RAGHAVAN,

Yahoo! Research

DAVID E. SHAW,

Columbia University

ALFRED Z. SPECTOR,

Google, Inc.

PETER SZOLOVITS,

Massachusetts Institute of Technology

PETER J. WEINBERGER,

Google, Inc.

JON EISENBERG, Director

VIRGINIA BACON TALATI, Program Associate

SHENAE BRADLEY, Senior Program Assistant

RENEE HAWKINS, Financial and Administrative Manager

HERBERT S. LIN, Chief Scientist

LYNETTE I. MILLETT, Senior Program Officer

ERIC WHITAKER, Senior Program Assistant

ENITA A. WILLIAMS, Associate Program Officer

For more information on CSTB, see its website at http://www.cstb.org, write to CSTB, National Research Council, 500 Fifth Street, N.W., Washington, DC 20001, call (202) 334-2605, or e-mail the CSTB at cstb@nas.edu.

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

Preface

The information technology (IT) revolution of the past several decades has dramatically changed the world. The Internet, Web 2.0 technologies, social networking tools, online search engines, text messaging, video teleconferencing, and multimedia-enabled smart-phones with embedded cameras are but a sample of IT-based capabilities that have altered the ways in which people communicate and work.

In the military, IT has enabled profound advances in weapons systems and the management and operation of the defense enterprise. A significant portion of the Department of Defense (DOD) budget is spent on capabilities acquired as commercial IT commodities, developmental IT systems that support a broad range of warfighting and functional applications, and IT components embedded in weapons systems. The ability of the DOD and its industrial partners to harness and apply IT for warfighting, command and control and communications, logistics, and transportation has contributed enormously to fielding the world’s best defense force.

But despite the DOD’s decades of success in leveraging IT across the defense enterprise, the acquisition of IT systems continues to be burdened with serious problems. Accordingly, the Defense Information Systems Agency (DISA) asked the National Research Council (NRC) to assess the efficacy of the DOD’s acquisition and test and evaluation (T&E) processes as applied to IT. In response, the NRC formed the Committee on Improving Processes and Policies for the Acquisition and Test of Information Technologies in the Department of Defense—a group of IT sys-

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

BOX P.1

Statement of Task

This study will bring together defense and defense industry experts in acquisition and test and evaluation (T&E); commercial software developers; and software engineers, computer scientists, and other academic researchers to assess the efficacy of the DOD acquisition and T&E processes as specifically applied to information technology. Through briefings, site visits, and committee deliberations, the study committee will:

  1. Evaluate legislative requirements for acquisition and T&E and the current DOD acquisition process (as defined in the “DOD 5000 series”) to determine whether the law and the defined processes permit enough flexibility to rapidly bring capabilities to users;

  2. Examine the processes and capabilities of the commercial IT sector to determine whether industry best practices can be adopted by DOD to improve the acquisition, systems engineering, and T&E process;

  3. Examine the Department’s various concepts for systems engineering and testing in virtual environments, and make recommendations for how to integrate them into a cohesive, efficient, and robust capability;

  4. Examine the DOD acquisition environment, including its institutional and cultural dimensions, for barriers that inhibit program managers/acquisition executives from taking advantage of existing flexibility in law and defined processes and recommend solutions; and

  5. Make recommendations to responsible agency, executive branch, and legislative officials about how to improve the acquisition, systems engineering, and T&E processes to achieve the Department’s net-centric goals.

tems acquisition and T&E experts, commercial software developers; and software engineers, computer scientists, and other academic researchers. The committee was tasked with the following: (1) an evaluation of applicable legislative requirements, (2) an examination of the processes and capabilities of the commercial IT sector, (3) an examination of the DOD’s concepts for systems engineering and testing in virtual environments, (4) an examination of the DOD acquisition environment, and (5) the formulation of recommendations on how to improve the acquisition, systems engineering, and T&E processes to achieve the DOD’s network-centric goals. (The full statement of task appears in Box P.1.) The tasks were completed in November 2009. This report provides the committee’s findings and recommendations, which are based on document reviews, briefings from commercial and military experts in IT systems acquisition, internal deliberations, and the committee members’ personal expertise.

Briefings to the committee from staff of the Office of the Secretary

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

of Defense showed that the acquisition of major automated information systems (MAIS) is especially troublesome. This problem has been broadly recognized for years, and there have been many attempts at reform. Nonetheless, today’s processes for the acquisition and testing of DOD IT systems often last 5 or more years before delivering solutions to the end users. Given the rapid pace of change in the IT world, it is no wonder that solutions ultimately delivered by DOD IT programs are often considered by end users to be inadequate. Much the same could be said about the historical adoption of IT in the commercial sector, where there have been extraordinary successes and colossal failures. Fortunately, the commercial sector has enjoyed some great successes in recent years by employing agile IT acquisition approaches that can also be leveraged by the DOD.

In examining the current DOD processes for acquiring IT systems and comparing them with the processes adopted by leading-edge firms in the commercial sector, the committee found stark differences. The DOD is hampered by a culture and acquisition-related practices that favor large programs, high-level oversight, and a very deliberate, serial approach to development and testing (the waterfall model). Programs that are expected to deliver complete, nearly perfect solutions and that take years to develop are the norm in the DOD. In contrast, leading-edge commercial firms have adopted agile approaches that focus on delivering smaller increments rapidly and aggregating them over time to meet capability objectives. Moreover, the DOD’s process-bound, high-level oversight seems to make demands that cause developers to focus more on process than on product, and end-user participation often is too little and too late. These approaches run counter to agile acquisition practices in which the product is the primary focus, end users are engaged early and often, the oversight of incremental product development is delegated to the lowest practical level, and the program management team has the flexibility to adjust the content of the increments in order to meet delivery schedules.

The committee concluded that the key to resolving the chronic problems with the DOD acquisition of IT systems is for the DOD to adopt a fundamentally different process—one based on the lessons learned in the employment of agile management techniques in the commercial sector. Agile approaches have allowed their adopters to outstrip established industrial giants that were beset with ponderous, process-bound, industrial-age management structures. Agile approaches have succeeded because their adopters recognized the issues that contribute to risks in an IT program and changed their management structures and processes to mitigate the risks. There are clear parallels in the DOD that support making this process change the centerpiece of improving IT acquisition.

For the DOD to succeed in adopting new approaches to IT acquisition, the first step is to acknowledge that simply tailoring the existing

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

processes is not sufficient. DOD acquisition regulations do permit tailoring, but the committee found few examples of the successful application of the current acquisition regulations to IT programs, and those that were successful required herculean efforts or unique circumstances. Changes broader than tailoring are necessary; they must encompass changes to culture, redefinition of the categories of IT systems, and restructured procurement, development, and testing processes as identified in this report. In the aggregate, these changes must realign processes that today are dominated by deliberate approaches designed for the development of large, complex, hardware-dominated weapons systems to processes adapted to the very different world of software-dominated IT systems.

The specific, actionable recommendations made by the committee address the four dimensions of its task discussed above. The body of the report and the appendixes include detailed discussions, rationale, and two proposed new process models for acquiring IT within the DOD. One model is structured for programs focused on the development of new software to provide new functionality or to integrate commercial off-the-shelf (COTS) components (e.g., MAIS programs). The second model is designed for the acquisition of COTS IT hardware, software, or services. Both have parallels in the commercial sector and are especially relevant for acquiring systems that support DOD information enterprise requirements and operate using the DOD IT infrastructure. The changes are not recommended for adoption in acquiring IT components embedded in weapons systems at this time, but the committee believes that as these changes are refined and institutionalized, many will be applicable to IT components of weapons systems as well.

The committee believes that there is an imperative for change, and it strongly urges the DOD to adopt the recommendations offered in this report. Strong support from the highest levels of the DOD will be required to implement changes of the magnitude recommended.

The committee extends its thanks to the individuals listed in Appendix E who briefed the committee. It also thanks Steven Hutchison, DISA Test and Evaluation Executive, for helping to make this study possible, and Dr. Hutchison and Judith Hill for their assistance throughout the course of the study. Finally, the committee extends its thanks and appreciation to Jon Eisenberg, Kevin Lewis, Lynette Millett, and Virginia Bacon Talati of the NRC’s Computer Science and Telecommunications Board whose dedicated support made this report possible.


William H. Campbell, Co-Chair

Committee on Improving Processes and Policies for the Acquisition and Test of Information Technologies in the Department of Defense

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

Acknowledgment of Reviewers

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:

Eddie Bair, E. Bair Associates, LLC,

Calvin Carerra, The Carrera Group, Inc.,

Felix Dupré, The Durango Group, LLC,

Bruce A. Finlayson, University of Washington,

Jacques S. Gansler, University of Maryland,

Michael F. Goodchild, University of California, Santa Barbara,

Richard F. Hilliard II, Independent Consultant, Bar Harbor, Maine,

Steven B. Lipner, Microsoft Corporation,

Charles E. McQueary, Independent Consultant, Arlington, Virginia,

Frank Ostroff, Ostroff Consultants Group, LLC,

Stuart H. Starr, National Defense University,

John P. Stenbit, TRW, Inc. (retired),

Kevin J. Sullivan, University of Virginia,

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

Anthony M. Valletta, SRA International,

George Wauer, Independent Consultant, Centreville, Virginia, and

Peter J. Weinberger, Google, Inc.

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Butler W. Lampson, Microsoft Corporation. Appointed by the National Research Council, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×

3

 

SYSTEMS AND SOFTWARE ENGINEERING IN DEFENSE INFORMATION TECHNOLOGY ACQUISITION PROGRAMS

 

47

   

 The Evolution of Department of Defense Policy and Practice for Software Development,

 

47

   

 Iterative, Incremental Development,

 

51

   

 Platforms and Virtualization: Key Underpinnings for Information Technology Systems,

 

60

   

 A Recommended Acquisition Management Approach for Information Technology Programs,

 

63

   

 Proposed Acquisition Management for SDCI Programs,

 

66

   

 Proposed Acquisition Management for CHSS Programs,

 

74

4

 

ACCEPTANCE AND TESTING

 

79

   

 Introduction,

 

79

   

 Shortcomings of Present Defense Test and Evaluation,

 

80

   

 “Big-R” Requirements and “Small-r” Requirements,

 

85

   

 Incorporating the Voice of the User,

 

86

   

 Toward Continuous Operational Assessment,

 

86

   

 Acceptance Teams,

 

88

   

 Evaluation Through Operational Use Metrics,

 

89

   

 Incorporating Common Services Definitions,

 

90

   

 Virtual Information Technology Test Environments,

 

92

 

 

BIBLIOGRAPHY

 

97

 

 

APPENDIXES

 

 

   

 A  Brief Overview of the Defense Acquisition System for Information Technology

 

103

   

 B  Program Phases and Decision Milestones for SDCI Programs

 

116

   

 C  Program Phases and Decision Milestones for CHSS Programs

 

123

   

 D  Programs That Succeeded with Nontraditional Oversight

 

127

   

 E  Briefings to the Committee

 

131

   

 F  Biosketches of Committee and Staff

 

134

   

 G  Acronyms

 

147

Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R13
Suggested Citation:"Front Matter." National Research Council. 2010. Achieving Effective Acquisition of Information Technology in the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/12823.
×
Page R14
Next: Summary and Recommendations »
Achieving Effective Acquisition of Information Technology in the Department of Defense Get This Book
×
Buy Paperback | $52.00 Buy Ebook | $41.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In the military, information technology (IT) has enabled profound advances in weapons systems and the management and operation of the defense enterprise. A significant portion of the Department of Defense (DOD) budget is spent on capabilities acquired as commercial IT commodities, developmental IT systems that support a broad range of warfighting and functional applications, and IT components embedded in weapons systems. The ability of the DOD and its industrial partners to harness and apply IT for warfighting, command and control and communications, logistics, and transportation has contributed enormously to fielding the world's best defense force.

However, despite the DOD's decades of success in leveraging IT across the defense enterprise, the acquisition of IT systems continues to be burdened with serious problems. To address these issues, the National Research Council assembled a group of IT systems acquisition and T&E experts, commercial software developers, software engineers, computer scientists and other academic researchers. The group evaluated applicable legislative requirements, examined the processes and capabilities of the commercial IT sector, analyzed DOD's concepts for systems engineering and testing in virtual environments, and examined the DOD acquisition environment. The present volume summarizes this analysis and also includes recommendations on how to improve the acquisition, systems engineering, and T&E processes to achieve the DOD's network-centric goals.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!