Click for next page ( 8


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 7
2 General Assessment of the Information Technology Laboratory LABORATORY MISSION AND PROGRAMS The Information Technology Laboratory supports the NIST mission through its own mission “to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology through research and development in information technology, mathematics, and statistics.”9 In support of this mission, the ITL has posed two strategic goals:  “Accelerate, through standards, tests, and metrics, the development, deployment and use of secure, usable, interoperable and reliable information systems that make American businesses more innovative and more competitive.  “Enable world-class measurement and testing through research innovations in the areas of computer science and systems engineering, mathematics and statistics.”10 In support of its mission and strategic goals, the ITL has formed a very strong scientific and technical team with core competencies in technology development in information technology (IT) measurement and testing, mathematical and statistical analyses for measurement science, modeling and simulation for measurement science, and information technology standards development and deployment. Further, the ITL has in recent years focused its R&D agenda on eight broad program areas: complex systems; cyber and network security; the enabling of scientific discovery; identity- management systems; information discovery, use, and sharing; pervasive information technologies; trustworthy information systems; and virtual measurement systems. The ITL now has a number of programs in these broad areas.11 The ITL program portfolio contains the following:  Complex Systems Program: This program examines systems that are composed of large, interrelated, and interacting entities that, taken together, show a macroscopic behavior that is not predictable through an examination of the individual entities. This program pursues an understanding of the fundamental science of complex systems and the development of rigorous 9 Cita M. Furlani, ITL Director, “The Information Technology Laboratory,” presentation to the panel, Gaithersburg, Maryland, March 21, 2011, p. 3. 10 Ibid., p. 9. 11 The program descriptions that follow were drawn from the descriptions provided to the panel by the ITL staff. 7

OCR for page 7
descriptions (analytic, statistical, or semantic) that enable the prediction and control of the behavior of such systems. The program is initially focused on the Internet and grid computing, and it will facilitate predictability and reliability in these areas and in other complex systems (e.g., biotechnology, nanotechnology, semiconductors, and complex engineering).  Cloud Computing Program: The purpose of this program is to accelerate the federal government’s secure adoption of cloud computing by building a U.S. government cloud computing standards roadmap12 that focuses on the highest- priority U.S. government cloud computing security, interoperability, and portability requirements; and by leading efforts to develop standards and guidelines in close consultation and collaboration with standards bodies, the private sector, and other stakeholders.  National Initiative for Cybersecurity Education: This initiative was established to “build a comprehensive framework that promulgates the availability of education, training, and awareness resources, designed to improve the cybersecurity knowledge, skills, and behavior of every segment of the population.”13  Quantum Information Program: In order to develop a measurement and standards infrastructure for information systems based on the principles of quantum physics, this program pursues the following objectives: to understand the potential for quantum information to revolutionize information science; to develop theory, methods, architectures, and algorithms to enable the engineering and testing of quantum computing components and systems; and to demonstrate and to test secure, commercial-grade communication components, systems, and protocols for the quantum era.  Identity Management Systems Program: The purpose of this program is to advance the development and adoption of fingerprint, face, and iris identification and surveillance technologies through the designing of appropriate performance metrics, evaluation methodologies, test suites and test data, prototypes and testbeds, workshops, and standards and guidelines.  Health Information Technology Program (Health IT Program): This program was established to support the accelerated development and harmonization of standards for health technologies, to create a health IT testing infrastructure, to consult on certification processes, to expand R&D and the deployment of security protocols, to support the usability of health technologies, and to address health care development beyond traditional physical locations, such as telemedicine and pervasive health care.  Pervasive Information Technology Program: This program studies the trend toward increasingly ubiquitous connected computing sensors, devices, and networks that monitor and respond transparently to human needs. The program promotes the development of standards and measurement methods 12 The first edition of the NIST Cloud Computing Standards Roadmap, NIST SP 500-291, July 2011, is available at http://collaborate.nist.gov/twiki-cloud- computing/pub/CloudComputing/StandardsRoadmap/NIST_SP_500-291_Jul5A.pdf. Accessed August 12, 2011. 13 Contained in the program descriptions provided to the panel by the ITL staff. 8

OCR for page 7
for reliable, interoperable, and ubiquitous communication and networking of personal and medical devices by facilitating the creation of standards for sensor communication, networking interoperability, and sensor information security enabling the use of pervasive information technologies to enhance personal and professional productivity and quality of life.  National Strategy for Trusted Identities in Cyberspace (NSTIC): This is a White House initiative for improving the privacy, security, and convenience of sensitive online transactions. This work is to be done collaboratively with the private sector, advocacy groups, public-sector agencies, and other organizations. The goals of the NSTIC are “to protect individuals, businesses, and public agencies from the high costs of cyber crimes such as identity theft and fraud, while simultaneously helping to ensure that the Internet continues to support innovation and a thriving marketplace of products and ideas.”14  Virtual Measurement Systems Program: This program was established to investigate uncertainties produced primarily by computer simulations or by computer-assisted measurements. The program introduces metrological constructs (i.e., standard references, uncertainty characterization and propagation, and traceability) into scientific computation and computer- assisted measurement technologies. A “virtual measurement” is information related to a physical model or system, but gleaned from analysis and measurement of a computer model or a computer simulation together with uncertainties in the computed quantities. Examples might include computational models of physical systems and visualizations of the results. As with physical measurement systems, the development of a virtual metrology infrastructure will yield predictive computing with quantified reliability, resulting in better-informed decision making when the results of computer simulations are used. Five of the above programs (Cloud Computing, Health IT, Pervasive IT, NSTIC, and Virtual Measurement Systems) are led from the ITL Headquarters Office. In addition to these programs in the focused R&D areas, the ITL conducts the following program:  Voting Standards Program: This program responds to the mandates in the Help America Vote Act of 2002 (HAVA; Public Law 107-252) and the Military and Overseas Voter Empowerment (MOVE) Act of 2009 (Public Law 111-84) by developing new standards and test methods; this program also conducts research that supports innovative technologies. The ITL works on programs supporting national priorities and on other programs deemed to be strategic to the ITL. The Quantum Information, Health IT, and Voting Standards Programs are examples of programs addressing national priorities. Strategic programs include the Complex Systems, Pervasive IT, and Virtual Measurement Systems Programs. 14 National Institute of Standards and Technology, Information Technology Laboratory, June 2011, p. 5. See http://www.nist.gov/itl/upload/ITLbrochure2011.pdf. Accessed June 11, 2011. 9

OCR for page 7
The ITL’s approach to program management is to work either within a division or to work in a collaborative, crosscutting fashion across divisions. Examples of crosscutting programs addressing national priorities are the Quantum Information, Health IT, and Voting Standards Programs. The ITL continues to produce products of national and international import. Some examples include the following:  The Digital Library of Mathematical Functions (DLMF): This work provides carefully selected, edited, and validated mathematical reference information covering a broad area of applicable mathematics; it is a unique and enduring accomplishment without peer in the broader community. Ongoing work on the DLMF includes maintenance, graphics, infrastructure for Math-on-the- Web, tables on demand, and the Painleve Project (addressing Painleve transcendents, a new class of functions represented in the DLMF).  Performance metrics, evaluation methodologies, test suites and test data for fingerprint, face, and iris identification and multibiometrics.  Standards (American National Standards Institute [ANSI]-NIST/ITL): These standards are for biometric data-exchange formats, biometric sample quality, biometric acquisition and processing protocols, and conformance testing methodologies.  NIST Special Publication 800* series: These publications are renowned for providing technically sound, unbiased, relevant guidelines that are frequently adopted voluntarily in private-sector procurements and practices and often mandated by the Office of Management and Budget (OMB) for use by the federal government.  Cryptographic standards and guidelines: These include the Advanced Encryption Standard (FIPS-197), Recommendation for Random Number Generation Using Deterministic Random Bit Generators (SP800-90), and Recommendation for Block Cipher Modes of Operation (SP800-38 series). OBSERVATIONS AND RECOMMENDATIONS Following are observations and recommendations of the panel resulting from its 2011 assessment of the Information Technology Laboratory. Observations 1 through 3 pertain directly to how the ITL is performing with respect to the three assessment criteria from the Director of NIST. Observations 4 through 6 address changes that have taken place since the 2009 assessment by the NRC panel appointed for that assessment.15 Observations 7 through 10 focus on areas of concern. Observations 1. The programs of the Information Technology Laboratory are focused on research and development that advance measurement science, standards, and 15 National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2009. Washington, D.C.: The National Academies Press, 2009. 10

OCR for page 7
technology. As an example, the Virtual Measurement Systems Program has identified the importance of understanding virtual measurements and uncertainties in advancing industry’s increasing reliance on software modeling and simulation in, for example, the design of new, advanced products. Similarly, the Cloud Computing Program is working on a U.S. government cloud computing technology roadmap that is focused on the highest-priority national cloud computing interoperability, portability, and security requirements. The Health Information Technology Program is working to improve standards for health technologies. These programs and the others reviewed are all making substantial progress toward meeting their objectives and are well aligned with the ITL mission and responsibilities. 2. The technical merits and scientific caliber of the current ITL programs are very high relative to comparable programs worldwide as measured by publications and especially by outstanding products such as the Digital Library of Mathematical Functions (DLMF) and the NIST Special Publication 800* series. The DLMF is without peer in the broader community, and the NIST Special Publication 800* series is renowned for providing technically sound, unbiased, relevant guidelines that are frequently adopted voluntarily in private-sector procurements and practices and often mandated by the Office of Management and Budget for use by the federal government. 3. The ITL R&D efforts appear to be carefully aligned with the mission-critical deliverables for which the ITL is responsible. Programs in cloud computing, health information technology, identity management, cybersecurity education, trusted identities, and voting standards are all addressing national priorities in information technology. National priorities with critical information technology aspects are being addressed by projects in biosciences and bioimaging, cyber physical systems, forensics, greenhouse gas measurement, optical medical imaging, public safety communications, quantum information, smart grid, and trusted networking (Internet Protocol Version 6 [IPv6], Domain Name System Security Extensions [DNSSEC]). 4. The Software and Systems Division (SSD) has made great strides since the previous assessment panel registered concerns in its 2009 report.16 The most prominent concern was “the lack of strong scientific and administrative leadership within the SSD and also, in some cases, at the programmatic level.”17 Today those concerns are being aggressively addressed, and the SSD has become more focused and better able to respond to its current challenges. 5. The ITL leadership has done an excellent job in filling two critical management positions: division chief for the Computer Security Division (CSD) and division chief for the Software and Systems Division. The ITL management is still faced with finding a permanent chief for the Advanced Network Technologies Division (ANTD). 6. The ITL has struggled with how crosscutting programs—those that involve work in a collaborative fashion across divisions—would be managed, since 16 Ibid. 17 Ibid., p. 15. 11

OCR for page 7
they do not fit neatly into the divisional structure. The ITL answer has been to use a matrix management structure. In 2007,18 less so in 2009, the panel was aware of considerable angst on the part of management and staff as to how that would work. This year there were no signs of that distress. It appears that the ITL has done an excellent job of working out the kinks and implementing matrix management. 7. The Statistical Engineering Division (SED) is continuing on an even keel with strong leadership and technical expertise. However, as observed in the 2009 assessment report, the division workload is growing but the division is not. The SED is seriously understaffed, and this problem needs to be addressed with some urgency.19 8. The Computer Security Division is also understaffed, although neither performance nor morale has as yet been affected. 9. The work of the Applied and Computational Mathematics Division (ACMD) continues to be excellent. However, the scientific culture of the division may not be sufficiently focused on collaboration to address the problems of multiscale and multiphysics involving complex geometries that are emerging as national priorities. 10. The Advanced Network Technologies Division is doing an excellent job in responding to several national priorities in both the short and long term, including its continued outstanding activities in Internet infrastructure protection and its newer efforts in smart grids and public safety communications. The division has also improved the quality of its internal and external collaborations, as well as the quality of its publications. The ANTD is understaffed for the portfolio of activities that it is undertaking. The various teams handling projects with short deadlines do not have as much time to dig into the subjects as they would like or would be useful. Another consequence of the understaffing is that basic research activities are perhaps below levels that are healthy. ANTD management has not yet articulated a long-term, strategic view of networking. Recommendations 1. At least two ITL divisions, the Statistical Engineering Division and the Computer Security Division, are feeling the constraints of increasing workloads and insufficient staffing (the SED more so than the CSD). If the ITL is to maintain its prominence in these areas, it should consider plans to address the growth that will be needed to support the expanding workload of each of these divisions. 2. Because the trend toward simulations of increasing model fidelity and numerical accuracy is expected to continue, the Applied and Computational 18 National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2007. Washington, D.C.: The National Academies Press, 2007. 19 National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2009. Washington, D.C.: The National Academies Press, 2009, pp. 2, 9, 14. 12

OCR for page 7
Mathematics Division will be called on to play an increasing role in addressing problems that are multidisciplinary. To ensure that the ITL is ready to support this work, the ACMD should devise a strategy to change the scientific culture of the division to meet those increased challenges. 3. The ITL should fill the position of chief of the Advanced Network Technologies Division with a permanent chief. ANTD management should address the understaffing issue in the division, and in particular it should ensure that there are adequate resources to handle both the short- and long- term needs of the division. ANTD management should create a strategic roadmap for the technical work of the division. The roadmap should be useful in managing the division’s resources and portfolio of activities. 4. The ITL should devote attention now to strategic, long-term technical needs in cloud computing that NIST may be called on to address in the future, including questions surrounding the scale of cloud computing and how such a scale could be accommodated in a laboratory or simulation environment. 5. The ITL should consider creating a collaborative effort between the Computer Security Division and the Software and Systems Division that would be responsible for the creation of standards and guidelines on secure software development for application by government, industry, and academia. 6. The ITL and the Software and Systems Division should reconsider the SSD mission statement, given the fresh focus of the new leadership, and after the SSD strategic planning process is complete. 7. The ITL and the Software and Systems Division should hire additional formally trained individuals in the SSD’s core foundational areas. 8. The Information Access Division (IAD) supports the development of technologies and their transition into the commercial marketplace as well as government applications. The division currently relies on substantial and sustained amounts of other agency (OA) funding (approximately 60 percent of IAD funding). Most of the OA funding is security-related (from the Department of Homeland Security, the Department of Defense, the Federal Bureau of Investigation, and the Intelligence Advanced Research Projects Activity). The reports, standards, and evaluation studies of the IAD are closely followed by academia and industry. In light of increasing foreign dominance of the biometric industry, IAD’s reliance on OA funding, and IAD’s work in support of biometrics technology development, it is important that the IAD and the ITL remain mindful of the NIST mission to promote U.S. innovation and industrial competitiveness, and so IAD efforts should continue to place highest priority on the needs of the nation’s commerce even while pursuing activities involving international sponsors. 9. The ITL should review the approval process of the Institutional Review Board20 to maximize the efficiency of the process and minimize unnecessary latency. 20 See http://www.hhs.gov/ohrp/humansubjects/commonrule/. Accessed July 11, 2011. The Office for Human Research Protections at the Department of Health and Human Services provides oversight for the protection of human subjects in research through the regulations that are spelled out for Institutional Review Boards in the so-called Common Rule (45 C.F.R. 46). 13

OCR for page 7
LABORATORY RESPONSES TO THE 2009 PANEL REPORT In the 2009 report of the NRC review panel,21 seven recommendations were made to the ITL. The panel recommended as follows in the 2009 assessments: 1. ITL staff, perhaps led by the program managers, should look for linkages with external organizations such as research universities and laboratories. The recent addition of temporary funding associated with the economic recovery can help build these connections. 2. The ITL should make efforts to raise its profile through outreach (connections with major research universities and laboratories, hosting faculty, postdoctoral researchers, and other short-term visitors; and staff participation in professional service) and publication (in highly respected journals and conferences). 3. Program managers who are capable of providing technical leadership and also devote effort to promoting the interests of their programs should be regarded by the staff as positive contributors, even if they are no longer writing code or doing other technical tasks associated with individual projects. 4. There is a need for additional senior technical leadership.  The Software and Systems Division (SSD) needs to hire a strong health informatics leader.  NIST should appoint a full-time chief for the SSD, which currently has an acting chief who divides time between leading the division and working in the Office of the ITL Director.  The panel found multiple cases of the SSD’s suffering from a lack of sufficient focused leadership at a time when the SSD is being asked to be the lead in several important efforts, such as health care. 5. SSD leadership should encourage its staff toward greater innovation and redirection in keeping with developments in the broader research and scientific community. 6. Apart from the current chief, there has been no perceptible growth in the permanent staff of the Statistical Engineering Division for years. The division is short-staffed, and such growth should be pursued with urgency before the next review. 7. The ITL needs a process for sunsetting programs and encouraging the bottom- up development of new programs. Overall, the ITL provided to the current panel adequate responses to the seven recommendations in the 2009 report. Several observations need to be made, however, regarding the ITL responses: 1. The combined responses to the first two recommendations in the list above were appropriate. Prestigious publications and professional activities should certainly raise awareness of NIST in the scientific community. However, missing from the ITL response was any discussion about how effective these 21 National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2009. Washington, D.C.: The National Academies Press, 2009, pp. 1-2. 14

OCR for page 7
activities have been in the R&D area. Such a discussion would have been enlightening and should be included in all future responses to panel recommendations. 2. The sixth recommendation was not completely satisfied. Two new employees were hired, but their impact was lessened by the loss of two staff members. The ITL’s action maintained the status quo, and once again it is recommended that there be increased staffing for the SED. 3. The ITL’s response to the seventh recommendation was a description of how programs were evaluated, changed, or moved during 2010, but there was no mention of sunsetting or of a process for sunsetting, so the response was not adequate. The ITL still needs to address the issue of a sunsetting process. 15