8
Information Technology Laboratory



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 8 Information Technology Laboratory

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 PANEL MEMBERS Louise H.Trevillyan, IBM T.J.Watson Research Center, Chair Tony Scott, General Motors Corporation, Vice Chair Michael Angelo, Compaq Computer Corporation Bishnu S.Atal, AT&T Laboratories Research Matthew Bishop, University of California at Davis Linda Branagan, Secondlook Consulting Jaime Carbonell, Carnegie Mellon University Aninda DasGupta, Philips Consumer Electronics Albert M.Erisman, Institute for Business, Technology, and Ethics John R.Gilbert, Xerox Palo Alto Research Center Roscoe C.Giles, Boston University Stephen T.Kent, BBN Technologies Jon R.Kettenring, Telcordia Technologies Catherine C.Lasser, IBM T.J.Watson Research Center John W.McCredie, University of California at Berkeley Vijayan N.Nair, University of Michigan Lawrence O’Gorman, Veridicom, Inc. David R.Oran, Cisco Systems Thomas Parenty, Consultant, Oakland, California Craig Partridge, BBN Technologies K.K.Ramakrishnan, TeraOptic Networks, Inc. William Smith, Sun Microsystems Stephanie M.White, Long Island University Submitted for the panel by its Chair, Louise H.Trevillyan, and its Vice Chair, Tony Scott, this assessment of the fiscal year 2001 activities of the Information Technology Laboratory is based on a site visit by the panel on February 27–28, 2001, in Gaithersburg, Md., and documents provided by the laboratory.1 1   U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Information Technology Laboratory Technical Accomplishments 2000, NISTIR 6558, National Institute of Standards and Technology, Gaithersburg, Md., October 2000; U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Report to the NRC Assessment Panel for Information Technology, National Institute of Standards and Technology, Gaithersburg, Md., February 2001; U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Information Technology Laboratory Publications 2000, National Institute of Standards and Technology, Gaithersburg, Md., January 2001.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 LABORATORY-LEVEL REVIEW Technical Merit According to laboratory documentation, the mission of the Information Technology Laboratory (ITL) is to strengthen the U.S. economy and improve the quality of life by developing and applying technology, measurements, and standards for information technology. According to the laboratory’s strategic plan, the laboratory has two primary functions: (1) to provide the U.S. information technology industry and key information technology users with the world’s best technical infrastructure and to increase the quality of software in industry, returning the best possible value to the economy and society, and (2) to provide the best in information technology services for all of NIST, thus enabling its staff to use information technology to improve their service delivery and efficiency and therefore provide the best value to their customers. The mission of the Information Technology Laboratory is very broad, and the programs under way no only encompass technical and standards-related activities but also provide consulting services in mathematical and statistical techniques and computing support throughout NIST. To manage this wide array of projects, the laboratory is organized into eight divisions (see Figure 8.1). The Mathematical and Computational Sciences Division is responsible for developing and disseminating analytical and computational methods for solving scientific and engineering problems. The Advanced Networking Technologies Division works with industry to address the technical challenges of an increasingly connected world; areas of emphasis include wireless communications and pervasive, optical, and multimedia networks. The Computer Security Division addresses needs in information technology security by developing standards, metrics, tests, and validation programs for security in systems and services and by acting as advisors on issues related to information technology security risks and vulnerabilities. The Information Access Division is responsible for technologies that facilitate interactions with computing devices and focuses on relevant measurement methods and standards in areas like speech and image recognition, information retrieval, and usability technologies. The Convergent Information Systems Division’s goal is to investigate the exchange, storage, and manifestation of digital content and its feasibility and scalability for integrated systems, applications, and architectures. The Information Services and Computing Division provides support for information technology capabilities throughout NIST; its responsibilities include maintaining a high-performance network, providing Web, personal computer, and help-desk facilities, and supporting central administrative applications (such as those used in the accounting, procurement, and inventory offices). The Software Diagnostics and Conformance Testing Division focuses on emerging languages and products and provides tools like conformance test suites, reference implementations, and advanced testing methodologies for standards development and compliance. The Statistical Engineering Division collaborates with industry and staff across NIST and provides guidance on experimental design, statistical modeling, and data analysis. Overall, the panel found that the programs under way in all of these divisions are appropriate and well-aligned with the laboratory and divisional missions. Examples of excellent ongoing activities and more detailed discussion of specific issues observed by the panel are discussed in detail in the divisional reports later in this chapter. A major reorganization of two divisions occurred this past year. The Distributed Computing and Information Services Division and the High-Performance Systems and Services Division were discontinued. The many information technology support functions of the laboratory, which had been spread between those two divisions, are now consolidated in the new Information Services and Computing Division. The High-Performance Systems and Services Division group that provided visualization

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 FIGURE 8.1 Organizational structure of the Information Technology Laboratory. Listed under each division are the division’s groups. expertise to NIST scientists has joined the Mathematical and Computational Sciences Division and is strengthening the collaborative efforts that are the focus of that division. Finally, the research functions, like work on standards for electronic books, of the High-Performance Systems and Services Division formed the core of the new Convergent Information Systems Division. The new organization provides the laboratory with a more coherent structure in which the divisions can have a tighter focus on key functions and programs can be managed more effectively with opportunities for synergies clearly highlighted. Although the full consequences of the new structure cannot yet be evaluated, the panel is very

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 enthusiastic about this reorganization, particularly about its potential for improving information technology support services at NIST. The Information Services and Computing Division is clearly starting off in the right direction by working closely with the Information Technology Services Planning Team, and the panel believes that the effort to develop an information technology architecture plan, i.e., a blueprint for information technology support, will help the division to define its roles and responsibilities. Overall, the panel is extremely pleased with the progress made in the Information Technology Laboratory since the last assessment. Under the guidance of the (relatively) new laboratory director, the management team as a whole has become significantly stronger. The “acting” titles are gone and two new division chiefs and a new deputy laboratory director have been hired. The new strategic plan2 more clearly lays out the laboratory’s goals and responsibilities and is organized so as to tie each division very specifically to the laboratory and NIST missions. Throughout the laboratory, the focus is sharpening, and the panel observed many instances of mature projects being appropriately concluded, existing activities being merged to produce programs that are more cohesive and better aligned with the laboratory’s mission, and new projects being started in areas that are important to industry and suitable for NIST. While a potential concern with an organization the size of the Information Technology Laboratory is its ability to respond to change in a timely manner, the panel has been pleased with the flexibility shown by the divisions. The Advanced Networking Technology Division and the Software Diagnostics and Conformance Testing Division are to be particularly commended on their processes for discontinuing mature activities, identifying important areas where NIST’s contributions are needed, and redeploying staff with the appropriate skills to fill the voids. The NIST director’s office does provide some competence funding to assist in the development of new programs and build new expertise in the laboratories, and the panel encourages Information Technology Laboratory staff to submit high-quality proposals in order to take advantage of this funding source as the laboratory programs evolve. In the past, the panel has commented on the importance of interdivisional collaboration on programs within the Information Technology Laboratory. This year, there were several examples of staff working together across divisions to make good progress on interdisciplinary projects. However, the panel feels that some additional interactions might be necessary to ensure that work in the various divisions is consistent and nonoverlapping, to increase awareness of areas in which there could be collaborations and synergies, and to cross-train staff in areas where key problems need to be understood from a variety of technical perspectives. Areas in which the panel felt increased communication and coordination between divisions would be helpful include activities relevant to biometrics in the Computer Security, Convergent Information Systems, and Information Access Divisions and the streaming media efforts in the Convergent Information Systems and Advanced Networking Technology Divisions. The panel also continues to emphasize the importance of integrating the laboratory-wide efforts in pervasive computing and suggests that connections between this work and the projects related to interactive television in the Convergent Information Systems and Software Diagnostics and Conformance Testing Divisions could be strengthened. In some cases, active collaboration across divisional lines will be needed, but informal cooperation is also very valuable. For example, often an individual project is relevant to a variety of technical communities but the terminologies used by these communities can be quite different or even inconsistent. If project staff can learn this unfamiliar technology from their colleagues in other divisions, they will be a better position to communicate effectively with their potential customers and increase their credibility both inside and outside NIST. 2   U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Information Technology Laboratory Strategic Plan, National Institute of Standards and Technology, Gaithersburg, Md., April 2000.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 Program Relevance and Effectiveness The Information Technology Laboratory has a wide array of projects that effectively support the U.S. economy and enable the federal government to meet its responsibilities to the American people. The divisional reports discuss in detail the various mechanisms used in the divisions to ensure that NIST’s work is relevant and affects the laboratory’s customers, but a few projects are highlighted here to exemplify the breadth and effectiveness of the laboratory’s programs. In the Statistical Engineering Division, the work on characterization of high-speed optoelectronic devices is a good example of a project that solves an important practical problem, develops cutting-edge scientific methods, and demonstrates the value of an effective collaboration across NIST laboratory boundaries. Working with staff from the Electronics and Electrical Engineering Laboratory, division researchers investigated issues related to accurately measuring the performance of high-speed optoelectronic devices such as photodiodes and sampling oscilloscopes. The result was the development of statistical signal processing techniques that are being used to correct for the effects of time-base distortion, timing jitter, and impedance mismatch in measurements using high-speed optoelectronic detectors. The approaches worked out have been incorporated into a new measurement service that will support the telecommunications, cable television, and fiber channel networks industries. The selection process for the Advanced Encryption Standard was completed this year. The Rijndael algorithm was chosen as the new standard, and the Computer Security Division’s management of the process is a good example of how NIST can capitalize on its reputation as an unbiased, technically expert organization to achieve international impact in an important area. NIST organized an open international competition in which candidate algorithms were made publicly available so that they could be judged by a large number of expert cryptographers; this was an impeccable approach for selecting a high-quality new standard that had the support of the international security community. Security is critical to government and industry, and the NIST-led selection process provided the encryption user community with a strong, stable, generally accepted standard to meet current and future security needs. This activity received a great deal of scrutiny and public attention, so the Computer Security Division’s successful management of it adds luster to NIST’s reputation. Another example of how the Information Technology Laboratory provides key support in standards is the work on Fortran 90 graphics in the Mathematical and Computational Sciences Division. The problem was that graphics software developed in Fortran was not portable across systems because of incompatible interfaces. To resolve the issue, the division developed a standardized Fortran graphics interface, which was endorsed by industry’s OpenGL Architecture Review Board. NIST made the new standard available on the Internet and provided implementation guidance to support and encourage use of the standard. The standard has already been incorporated into the product lines of major providers of Fortran compilers, including Compaq Computer Corporation and Lahey Computer Systems, Inc. Other projects worthy of mention are the Braille reader produced by the Convergent Information Systems Division and the information technology architecture initiative in the Information Services and Computing Division. Work on the Braille reader prototype is consistent with the division’s mission of integrating diverse technologies, and the division’s efforts in this area resulted in a technical breakthrough that could reduce the cost of such devices by thousands of dollars. The information technology architecture plan is in its formative stages, but the panel believes that it is a solid approach for determining what platforms should be supported at NIST and for communicating to staff and management throughout NIST what services can be expected from this new division. Once the division’s responsibilities have been defined, appropriate resources should be allocated to support their efforts, and techniques to measure the quality of the services provided should be put into place.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 All of the divisions in the Information Technology Laboratory are actively involved in standardsrelated activities. Examples include standards for Java Numerics, broadband wireless access, advanced encryption, fingerprints, digital television, and XML. NIST contributions include management of standards development processes and coordination of the relevant parties, analysis and testing of proposed standards, and development of tools needed to test conformance to standards. The key factors in NIST’s success in information technology standards-related work are the laboratory staff’s reputation for providing unbiased, high-quality technical advice, data, and tools, and the timeliness of the laboratory’s efforts (i.e., involvement early in the standards development process). The panel commends the laboratory for its continuing commitment to standards-related work. The industries that use the many standards that have been improved by NIST contributions are appreciative of the value the Information Technology Laboratory provides, and the laboratory staff’s work on standards does help build an awareness within information technology communities of the important role NIST plays. Traditionally, industry used open approaches to standards development (e.g., work within organizations like the IEEE). In recent years, however, companies have begun to supplement these approaches with standards development in consortia and other private groups, where membership is conditional upon payment of a fee or signing of an agreement limiting disclosure or use of intellectual property. The value of these nonopen groups is that they can often be faster and more efficient than the traditional public approaches. The potential downside to these groups is that companies have been known to exclude their competitors from a group and manipulate the process to develop standards that give their products a competitive advantage. However, to meet NIST’s mandate to support U.S. industry, the divisions of the Information Technology Laboratory in some cases must consider joining private organizations and consortia. While the panel, and NIST staff as well, recognize that it would be inappropriate for NIST to join organizations with especially restrictive policies on membership or prohibitive constraints enforced by intellectual property or nondisclosure agreements, not all nonopen standards processes are unfairly exclusionary. The laboratory should therefore establish a policy to help divisions decide when participation in closed consortia is appropriate and should consider how NIST can encourage industry to utilize open, or at least inclusionary, approaches to standards development. The panel notes that when the laboratory does decide to participate in a nonopen standards group, appropriate and timely support from the Department of Commerce legal services will be needed so that staff may play their crucial roles in the standards development process and industry may benefit from the measurement and evaluation tools developed at NIST. A key element in the effectiveness of Information Technology Laboratory programs is the relationships NIST staff have built with companies, universities, and other government agencies. These interactions help staff gather input on current and future activities and disseminate results to relevant communities. However, for the laboratory to strengthen current relationships and establish new partnerships, the panel believes that NIST should increase the visibility of its efforts in information technology. In some areas, such as networking, this means careful consideration of the quality and reputation of various journals and conferences so as to ensure that NIST publishing and speaking efforts produce the maximum impact. In other areas, such as mathematics and statistics, collaborative articles in scientific publications should be supplemented by papers in mathematics or statistics journals. In general, the goal is to have NIST’s value recognized not only by technical researchers within industry, government, and academia but also by their senior management. Information Technology Laboratory managers have indicated that they plan to create a position with responsibility for enhancing the laboratory’s reputation outside NIST, and the panel encourages this step. Overall, the panel is impressed with the programs under way and believes that the Information Technology Laboratory is well-positioned to have an even greater impact than in the past. It is clear that

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 the broad topics that the laboratory has chosen to work on are the right ones, and the laboratory’s efforts to address relevant issues within these areas can be expected to have positive effects nationally and internationally. However, the panel is somewhat concerned about the selection of individual projects within the broad topics. While the bottom-up approach to identifying activities continues to produce good and effective projects, it is important for the laboratory not to neglect the larger view, which should ensure that the most important questions within the laboratory’s purview are being addressed, that the diverse expertise available across the laboratory is fully exploited, and that appropriate balance is maintained among various types of laboratory activities (standards-related work, basic research, technology development, paper writing and conference attendance, and so on). The panel noticed a proliferation of small (often one-person) projects, and these may be a result of the laboratory’s not considering how its collection of individually appropriate projects can be fit together to meet larger goals. Laboratory Resources Funding sources for the Information Technology Laboratory are shown in Table 8.1. As of March 2001, staffing for the Information Technology Laboratory included 368 full-time permanent positions, of which 302 were for technical professionals. There were also 92 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers. Last year, the panel raised concerns about the fact that the explosive growth in information technology in industry was not being matched by growth in the funding or staffing levels in the Information Technology Laboratory. The need for new standards and protocols, for analysis of issues related to scalability and feasibility, and for support of new software and hardware continues to affect every division; laboratory staff simply cannot work on all of the problems important to their customers. While increased resources have been provided for work in the computer security area, other divisions continue to have flat funding, and staffing is actually decreasing. The panel fears that owing to resource constraints the laboratory is being forced to neglect important issues, and it again recommends that NIST compile a list of areas that the Information Technology Laboratory cannot tackle with its current staff and funding levels and describe the potential negative impact on industry. The panel is still concerned about the continued housing of the Mathematical and Computational Sciences Division and the Statistical Engineering Division in NIST North. The work of these divisions is highly collaborative, and staff feel that their separation from their research partners on the main NIST campus significantly decreases their effectiveness and productivity. It is not clear to the panel that all possible approaches to bringing these divisions back to the main campus have been explored. Creative solutions should be investigated by laboratory and NIST management, and—most importantly—staff from these divisions should be kept up to date on the efforts and results. If solutions cannot be found, the reasons need to be clearly communicated to personnel at NIST North. This issue needs to be resolved. In general, morale at the laboratory appears to be extremely high. Personnel are generally happy with the changes in management and structure that have occurred over the past year, and the panel commends laboratory management as a whole for the overall improvement in the working environment. Staff retention continues to be good, especially in light of the highly competitive job market for people with information technology expertise. During its visits to the Information Technology Laboratory and its individual divisions, the panel spoke with staff members without management present (skip-level meetings). Several issues arose in these conversations. One is an apparent need for more career counseling and development and for clearer criteria, or communication of criteria, on how staff performance is measured. Some staff wanted career development advice because they sensed that there were

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 TABLE 8.1 Sources of Funding for the Information Technology Laboratory (in millions of dollars), FY 1998 to FY 2001 Source of Funding Fiscal Year 1998 (actual) Fiscal Year 1999 (actual) Fiscal Year 2000 (actual) Fiscal Year 2001 (estimated)a NIST-STRS, excluding Competence 31.6 31.6 31.9 38.3 Competence 0.9 1.5 1.6 1.3 STRS-supercomputing 11.8 12.1 12.0 12.4 ATP 1.8 1.8 2.4 1.6 Measurement Services (SRM production) 0.0 0.0 0.0 0.4 OA/NFG/CRADA 10.6 8.4 9.9 13.7 Other Reimbursable 1.5 0.5 1.6 1.6 Agency Overhead 12.0 14.4 16.4 18.9 Total 70.2 70.3 75.8 88.2 Full-time permanent staff (total)b 362 381 381 368 NOTE: Funding for the NIST Measurement and Standards Laboratories comes from a variety of sources. The laboratories receive appropriations from Congress, known as Scientific and Technical Research and Services (STRS) funding. Competence funding also comes from NIST’s congressional appropriations but is allocated by the NIST director’s office in multiyear grants for projects that advance NIST’s capabilities in new and emerging areas of measurement science. Advanced Technology Program (ATP) funding reflects support from NIST’s ATP for work done at the NIST laboratories in collaboration with or in support of ATP projects. Funding to support production of Standard Reference Materials (SRMs) is tied to the use of such products and is classified as Measurement Services. NIST laboratories also receive funding through grants or contracts from other government agencies (OA), from nonfederal government (NFG) agencies, and from industry in the form of Cooperative Research and Development Agreements (CRADAs). All other laboratory funding, including that for Calibration Services, is grouped under “Other Reimbursable.” aDue to the reorganization of the ITL that began in November of 2000 and became official in February 2001, the budget estimates and staff numbers for FY 2001 are as of March 2001. bThe number of full-time permanent staff is as of January of that fiscal year, except in FY 2001, when it is as of March. few formal avenues for them to upgrade their skills or develop new expertise. Others, because they were uncertain about the mission and priorities of the laboratory, how their projects fit in, and whether they were valued by management, wanted to know how their performance is measured. Such insecurity is a normal byproduct of all the changes that the Information Technology Laboratory has experienced over the past several years, but the panel encourages management to be aware of the concerns and try to resolve them. The skip-level sessions caused the panel to become somewhat uneasy about single-person projects. These small projects are of concern for two reasons: (1) without a critical mass of relevant expertise, staff can grow technically isolated, which affects the quality of their work and their morale, and (2) isolation also may contribute to staff’s uncertainty about the value of their contribution. Last year’s report mentioned that NIST’s network connectivity to the outside world was significantly poorer than that available to many universities and industrial research organizations via the Internet2 Project (12 Mbps for NIST versus 155 Mbps or faster on Internet2). This issue is discussed in detail in the section on the Information Services and Computing Division. The panel notes that this outdated level of technology limits the quality of the staff’s connections to their customers and NIST’s

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 ability to efficiently and effectively distribute its results to industry. It is important for the Information Technology Laboratory to seek NIST-wide solutions for this problem so that individual groups do not begin to try work-around approaches on their own, which could lead to duplicated efforts or divergent solutions. DIVISIONAL REVIEWS Mathematical and Computational Sciences Division Technical Merit According to division documentation, the mission of the Mathematical and Computational Sciences Division is to provide technical leadership within NIST in modern analytical and computational methods for solving scientific problems of interest to U.S. industry. The panel is very impressed with the current status of the division. The division is in excellent shape, with technically strong and effective projects and capable and well-respected leadership. Good people are working enthusiastically on tough, important problems, and the level of collaboration with staff throughout the NIST laboratories is high, which advances the division’s ability to make an impact. The Mathematical and Computational Sciences Division currently contains four groups: Mathematical Modeling, Mathematical Software, Optimization and Computational Geometry, and Scientific Applications and Visualization. This last group was transferred to the division late in 2000 from the former High Performance Systems and Services Division and is a strong and appropriate addition to the Mathematical and Computational Sciences Division. Both the new and old groups in the division appear to have welcomed the move, and integration of the new team with the established staff and programs in the division is going well. Overall, the technical work done in the division continues to be excellent, with many projects well focused on important objectives. There are several examples that particularly impressed the panel. One was the project on blind deconvolution of images, in which the division is investigating ways to deblur images without knowing the cause of the blur. NIST is advancing the state of the art in analysis of such images, in part to support microscopy work ongoing within NIST, but the division’s methods have also garnered a great deal of attention in the external scientific community. The work on time-domain algorithms for computational electromagnetics is already strongly influencing the relevant technical community, and the division’s products have the potential to supply NIST staff and their customers with useful tools. The project on parallel adaptive refinement and multigrid methods, through its software package PHAML, has already had a positive impact both inside and outside NIST; now staff are investigating the intriguing possibility that it could be applied to modeling the behavior of the elements of a quantum computer. Finally, the Matrix Market, a standard collection of sparse matrices with search tools, statistics, and visualizations, is an excellent example of mathematical reference data and a good fit for NIST’s mission. Although the division currently has limited resources devoted to this project, the collection continues to grow and is the second busiest Web site in all of the Information Technology Laboratory. The work on the Digital Library of Mathematical Functions (DLMF) is an excellent fit with the division and NIST missions. This project has ambitious and important goals, and the panel was pleased to see that progress on assembling this reference site is on schedule; chapter contracts are in place and writing is moving ahead well. A major accomplishment of the past year was securing support for this project from the National Science Foundation. In the coming year, the key challenge will be bringing

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 the completed chapters online on the Web.3 The panel believes that there are two issues associated with this task. One is whether the Web version, as it becomes fully available, will be able to incorporate lessons from user experience into its design. The second question is how and if users will be able to obtain function values. The Handbook of Mathematical Functions,4 the hard copy predecessor to DLMF, included tables that constituted reference values from NIST. The data were useful for readers and also provided some ability to validate implementations. It is not clear what the corresponding capability should be in the Web-based version. One option is for it to somehow use the reportedly very stable International Mathematical Subroutine Library or the Numerical Algorithms Group special functions codes. Another option is to try to take advantage of other projects in the division and investigate whether working with the Java Numerics project to produce downloadable Java software is feasible. This question of how users could obtain function values is a fundamental issue, and the panel believes that it will require some original thinking to find the right solution. In addition to the work on established and traditional projects, the Mathematical and Computational Sciences Division is exploring a number of important new directions, such as combinatorial methods, quantum computing, bioinformatics, and mathematical metrology. The challenge in these new areas will be to carve a focused agenda out of a huge realm of possibilities. Initiatives like the work on bioinformatics and the new DARPA proposal on quantum computing seem quite promising, but some overall strategic vision will be needed to define a coherent program in which NIST can be most effective. Complementing the division’s exploration of new directions is its appropriate conclusion of some existing work, which will free up resources to invest in new efforts. The project on appearance of coated objects and the work for the Army in terrain modeling have achieved their main goals and are coming to an orderly close. Both projects have had an impact, and the division is correct to treat them as completed and replace them with new priorities. Program Relevance and Effectiveness The Mathematical and Computational Sciences Division performs highly effective work in mathematical modeling, software, and visualization. Demand for the results of these activities and for participation by staff in collaborative projects exceeds the time staff have available, and customers are very appreciative of the division’s efforts. The division has received glowing testimonials on the impact of its work from Ford, Hewlett-Packard, and other companies, from university research groups, and from staff in the other NIST laboratories, including materials scientists, physicists, and building and fire researchers. The Java Numerics project is an example of an activity that demonstrates the way in which this division maximizes its impact and how NIST leadership plays a vital role in the chaotic and fast-moving world of information technology standards. Three important steps by the division ensured that this project would make a difference. First, staff recognized that Java would become an important tool for numerical computing but that its design contained significant problems. Then the division brought together a technical community to address the problems and propose solutions. Finally, NIST staff 3   The introductory page for the DLMF is available online at <http://dlmf.nist.gov/>, and a mockup of it, including links to draft outlines and chapters, is available online at <http://dlmf.nist.gov/Contents/>. 4   M.Abramowitz and I.A.Stegun, eds., Handbook of Mathematical Functions, with Formulas, Graphs, and Mathematical Tables, National Bureau of Standards, Gaithersburg, Md., 1964.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 current effort is a promising step in this direction. Initial results from the performance metrics team are expected in 2001, and the panel is very interested in seeing the outputs. Division Resources Funding sources for the Information Services and Computing Division are shown in Table 8.7. As of March 2001, staffing for the Information Services and Computing Division included 131 full-time permanent positions, of which 106 were for technical professionals. There were also 13 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers. Recruiting and retention programs are working well. Over the past year, the division experienced approximately 5 percent turnover in its staff, a low rate for an IT support organization in a major metropolitan area. Although the division has lost a few key people, its initiatives in hiring and its embrace of flexible approaches such as telecommuting, when appropriate, seem to be effective, and morale appears to be high. The panel did observe that management at the group and division level was entirely male and encourages division and laboratory management to look for ways to increase diversity. Female leaders do exist at the team level. Recently, the director of the Information Technology Laboratory was named acting NIST Chief Information Officer (CIO). It may be difficult for one individual to carry out the responsibilities of both positions. However, the value of having the IT service group integrated into the Information Technology Laboratory has been observed in the past, and the panel understands that the organizational location of the IT support unit was a primary factor in determining the CIO for NIST. The role of CIO does not TABLE 8.7 Sources of Funding for the Information Services and Computing Division (in millions of dollars), FY 1998 to FY 2001 Source of Funding Fiscal Year 1998 (actual) Fiscal Year 1999 (actual) Fiscal Year 2000 (actual) Fiscal Year 2001 (estimated)a NIST-STRS, excluding Competence 0.6 0.6 0.9 0.3 STRS-supercomputing 0.9 0.9 0.9 7.7 Measurement Services (SRM production) 0.0 0.0 0.0 0.1 OA/NFG/CRADA 0.1   0.0 0.7 Other Reimbursable 0.6 0.4 0.6 1.5 Agency Overhead 6.3 7.1 8.2 17.3 Total 8.5 9.0 10.6 27.6 Full-time permanent staff (total)b 62 72 77 131 NOTE: Sources of funding are as described in the note accompanying Table 8.1. aThe difference between the FY 2000 and FY 2001 funding and staff levels reflects the reorganization of ITL in which the visualization group was moved out of the Convergent Information Systems Division and into this division. bThe number of full-time permanent staff is as of January of that fiscal year, except in FY 2001, when it is as of March.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 appear to be clearly articulated, and the panel expects the responsibilities to evolve over time as the Department of Commerce clarifies what is expected of this person and as NIST progresses in defining the appropriate level of IT services to be provided by a central organization. As noted in previous assessments, NIST does not appear to participate in the academic and industrial research community’s collaborative efforts to develop and utilize high-performance networks. Several laboratory research groups have stated that their dissemination and communication efforts would be significantly improved if they were able to connect to the world outside NIST at a higher bandwidth (e.g., it would be easier for external groups to access large reference data files such as those produced in the Information Access Division’s speech recognition work). The panel recommends that the division explore relationships with organizations such as the University Corporation for Advanced Internet Development, the university-led nonprofit consortium that coordinates Internet2 activities among research and academic institutions around the world. Involvement with such an organization would allow the division to determine if and how NIST staff could be provided with broader bandwidth. The division is already studying NIST researchers’ needs in high-performance scientific computing, and advanced networking technologies can be an important element in the delivery of high-performance computing capabilities. Software Diagnostics and Conformance Testing Division Technical Merit According to division documentation, the mission of the Software Diagnostics and Conformance Testing Division is to develop software testing tools and methods that improve quality, Conformance to standards, and correctness; to participate with industry in the development of forward-looking standards; and to lead efforts for Conformance testing, even at the early development stage of standards. The division’s work in designing Conformance and diagnostic tests and developing reference implementations for standards bodies clearly fulfills its mission and is consistent with the goals of both the laboratory and NIST missions. The division’s philosophy is to concentrate on key areas at the forefront of technology (that is, to get involved early), to partner with industry (that is, to fill voids where companies cannot or will not work), to transfer relevant technology developed at NIST, and to move on to new projects. The panel finds this focus on timely involvement and close collaboration with industry appropriate and effective. The division is organized into three groups: Standards and Conformance Testing, Software Quality, and Interoperability. The Standards and Conformance Testing Group develops Conformance tests and reference implementations, performs research into better ways to do Conformance testing, and develops standards jointly with industry. Some of the work is focused on standards for electronic commerce, such as the efforts on extensible markup language (XML) technologies and the XML Registry and Repository. In conjunction with an XML standards organization, staff are establishing the means by which software and systems will communicate over the Internet. This group is also responsible for the division’s component of the Information Technology Laboratory-wide program on pervasive computing. Initially, the focus was on demonstrating the feasibility of these embedded computing systems, but now the division has moved on to issues related to ensuring software quality for these complex systems. A current project is the development of a three-dimensional graphical simulation tool that will enable cost-effective measurement and testing of pervasive computing systems. The staff are also leveraging expertise and knowledge from their previous research on architecture definition languages (ADLs) in their work on rigorously specifying service discovery protocols. For example, simulation of the ADL

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 implementation of a standard protocol for pervasive computing has already led to the led to discovery of an inconsistency in the Jini protocol. In addition to interdivisional work on pervasive computing, the Software Diagnostics and Conformance Testing Division also has a strong collaboration with the Statistical Engineering Division on software testing using statistical methods. This consists of a competence project funded by the NIST director’s office as well as a companion project within the Software Quality Group. The division’s work in this area is concentrating on component testing. This focus is appropriate, because the development of high-quality, reusable components would produce significant cost reductions in the software industry. The Software Quality Group develops software diagnostic tools, reference data, and methods to automate software testing and also performs research in formal methods. The group’s testing research is making significant contributions to this field. In their work on the generation of automatic tests for software, staff have developed an innovative method that uses mutations or counterexamples to generate comprehensive tests from formally specified system requirements and are currently working on a metric to quantify the amount of coverage these tests provide. They are also developing systems that can translate between the different formal models used in industry, academia, and government; the hope is for these translators to make formal methods efforts much more interoperable and hence more widely used and effective. Overall, the Software Diagnostics and Conformance Testing Division is leading the way for industry and government in the usage of formal methods to analyze and improve standards specifications and develop conformance tests. Already the division is applying the formal methods developed during the automatic test generation project to the work in XML conformance testing and to the object-oriented portion of the competence project on software testing by statistical methods. The Interoperability Group focuses on federal agency needs in interoperability. Its activities include providing technical support to voluntary standards committees and helping government and industry achieve interoperability through the application of products developed at NIST. Appropriate and useful projects under way include the National Software Reference Library (NSRL) and the work on verification of computer forensics tools. For the NSRL, the group is developing reference data sets that contain standard examples of common computer programs and files. The file signatures from these reference sets can then be compared to computer files that have been seized in the course of an investigation in order to identify unaltered common programs. By using the NSRL data set to eliminate a significant number (40 to 95 percent) of nonpertinent files automatically, investigators can concentrate on searching potentially relevant computer content, thus increasing their efficiency and saving hundreds of staff-hours. In a related project, division staff are planning to hold focus groups on defining quality requirements and aim to develop rigorous testing procedures for computer forensics tools. These specifications and tests will be used to ensure that the tools yield objective, repeatable, reproducible results that hold up in court. In other projects, staff from this group work through the federal Chief Information Officers Council to ensure that federal agency requirements are taken into account by voluntary standards committees and support the NIST paperless office effort utilizing digital signatures. The panel continues to be impressed by the timely and effective way in which projects are phased out throughout the division. The software industry tends to change technologies and shift priorities rapidly, and the division has demonstrated that it is able to react both quickly and appropriately. This year, the projects on instructional management systems, real-time Java, and the error, fault, and failure repository were all concluded and the freed resources allocated to other projects.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 Program Relevance and Effectiveness The Software Diagnostics and Conformance Testing Division provides necessary technical leadership to industry through its work on standards, reference implementations, and conformance testing. In the standards area, the division’s efforts contribute significantly to software quality and interoperability, and many organizations and areas could benefit from the expertise available in this division. In fact, based on their reputation for helping to create high-quality standard specifications, division staff are invited to participate in numerous standards bodies and must constantly set priorities based on where the division can have the greatest impact. The panel continues to be impressed by the decisions the division makes in this area and by the effectiveness of its efforts. The success of the division in the standards arena is due to two factors: (1) its early involvement in efforts to define standards and (2) its work on technical tools to support those efforts. Whenever possible, NIST staff develop reference implementations and conformance test suites, the tools needed to facilitate industry’s adoption of and compliance with standards, in parallel with the definition of the standards specification and well before industry implementations have begun. Taking advantage of NIST’s reputation for unbiased, high-quality technical advice, the division works within the community as a neutral party to resolve errors in the specifications and provide feedback on implementation issues before final standards are crystallized. Currently, the Software Diagnostics and Conformance Testing Division is emphasizing activities related to standards efforts relevant to electronic commerce, including XML and interactive TV. Electronic commerce applications are growing exponentially, changing the way business is conducted and how consumers behave. The division is appropriately participating in standards committees and developing test suites in this area at a very early stage, and the value placed on its efforts is clear from testimonials offered by personnel from the organizations trying to establish standards in these emerging areas. The executive director of the Organization for the Advancement of Structured Information Standards (OASIS) writes: “The OASIS XML conformance work is greatly enhanced by the leadership of NIST. NIST’s mission is standards and testing. Their many years of experience in providing quality, comprehensive conformance test suites are a tremendous asset to OASIS.” The chair of the Society of Motion Picture and Television Engineers’ Declarative Data Essence Group is equally supportive of the NIST plan to develop an interactive television testbed: “PLEASE DO IT! Everyone I’ve talked to is excited about your proposal….” Given the ubiquity of the Internet and the many potential applications of electronic commerce technologies, it is not unreasonable to expect that the division’s work in this area will have an impact worldwide. In all of the Software Diagnostics and Conformance Testing Division programs, the focus on emerging technologies and on maintaining close partnerships with industry ensures that NIST efforts will have a significant impact. By building strong connections with companies and industry organizations, the division can involve them in the collaborative development of tests and implementations so that technology transfer is not a hurdle tackled at the end of an activity but an ongoing effort, smoothly integrated throughout the life of the project. This approach allows the division to stay in close contact with industry participants and get continuous feedback on work performed and results delivered to companies and standards bodies. An example of how the division uses this model effectively is the work with Ford Research Laboratories on automatic test generation from formal specifications. The division’s work in this area will simplify the development of conformance tests and allow these tests to cover a wider range of potential faults, thus decreasing the number of errors and increasing software quality. Already a number of corporations, including Ford Motor Company Powertrain Division and Argus, a developer of secure operating systems, are using the reference implementation developed at

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 NIST to automatically generate tests from a formal description of their product requirements. Ford is now commercializing this product, and the division has successfully ensured the transfer of NIST results and broad dissemination of a useful tool. Division Resources Funding sources for the Software Diagnostics and Conformance Testing Division are shown in Table 8.8. As of March 2001, staffing for the Software Diagnostics and Conformance Testing Division included 35 full-time permanent positions, of which 32 were for technical professionals. There were also 15 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers. Retention of staff in the Software Diagnostics and Conformance Testing Division has been reasonably good for a number of reasons. One is that the division provides personnel with a chance to work in interesting areas and allows them to participate fully in their projects. Management effectively shares information about the “big picture” with everyone in the division and includes even the most junior staff in all strategy meetings and interactions with NIST customers. (The fact that the division provides funding and encourages the travel required for building customer relationships also has a positive effect on morale.) Pay increases have been reasonable, and flexible working hours and granting of leave add to the sense of a supportive working environment. NIST provides the staff with a high degree of both autonomy and stability, and the division appears to be an attractive place to work. Some turnover has occurred in the past year, but morale remains high as those who departed did so to do different types of work (usually still within NIST) or retire, while the new people all are very excited to be at NIST. The technology labor market underwent some significant changes in the year 2000, including widespread decreases in compensation based on the dropping value of stock options. The division may benefit from these changes, as the stability of NIST may make it an attractive alternative for many qualified potential employees. TABLE 8.8 Sources of Funding for the Software Diagnostics and Conformance Testing Division (in millions of dollars), FY 1998 to FY 2001 Source of Funding Fiscal Year 1998 (actual) Fiscal Year 1999 (actual) Fiscal Year 2000 (actual) Fiscal Year 2001 (estimated) NIST-STRS, excluding Competence 4.6 4.8 4.8 4.7 Competence 0.5 0.6 0.5 0.4 ATP 0.4 0.4 0.6 0.2 OA/NFG/CRADA 2.2 0.6 1.0 1.8 Other Reimbursable 0.2 0.0 0.0 0.0 Total 7.9 6.4 6.9 7.1 Full-time permanent staff (total)a 41 39 37 35 NOTE: Sources of funding are as described in the note accompanying Table 8.1. aThe number of full-time permanent staff is as of January of that fiscal year, except in FY 2001, when it is as of March.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 The Software Diagnostics and Conformance Testing Division continues to be hampered by the slowness of some NIST and Department of Commerce support functions. While there do not appear to be any immediate difficulties concerning the amount of capital resources, staff have remarked that equipment can be very difficult to procure, even when authorization has been obtained and funds are available. Long delays in procuring equipment can seriously hamper the division’s ability to support rapidly changing technologies, particularly in the hardware-dependent pervasive computing effort. Also, the poor responsiveness of the Department of Commerce legal department remains an ongoing problem. Although the 3-year struggle to officially join the World Wide Web Consortium appears to finally have been resolved, there is no sign of any improvement in the division’s ability to enter into a legal agreement in a timely manner. The division needs this capability because NIST’s impact in software standards is dependent on working with industry (often via consortia arrangements) early in the standards development process. Statistical Engineering Division Technical Merit According to division documentation, the mission of the Statistical Engineering Division is to advance measurement science and technology by collaborating on NIST multidisciplinary research and by formulating, developing, and applying statistical methodology for the collection and analysis of data critical to NIST scientists and engineers. The division continues to be involved in a broad range of activities including collaborative research with scientists at NIST, promulgation of measurement standards, experiment planning and inference, and development of new statistical methodology. While division staff are making important contributions to projects in the Information Technology Laboratory, a large portion of the division’s current efforts (about 80 percent) involves work with people outside the laboratory. This far-reaching array of collaborations represents a continuing shift of the division’s focus back to the tradition of working on important problems across all of NIST. As in past years, the Statistical Engineering Division is successfully promoting the use of state-of-the-art statistical methods in metrology, experimentation, and data analysis across NIST. The staff are also involved in collaborative research that has led to the development of new methodologies. The projects described below show the value and diversity of the division’s contributions over the past year. The work on characterization of high-speed optoelectronic devices is an excellent example of a research project that is making fundamental contributions to an important scientific problem while also developing cutting-edge statistical methodology. In the Optoelectronics and Radio Frequency Technology Divisions of the NIST Electronics and Electrical Engineering Laboratory, staff are working on techniques to accurately measure the performance of high-speed optoelectronic devices, such as photodiodes and sampling oscilloscopes. These techniques are critical in the design of high-performance systems that take advantage of the potential bandwidth of optical fiber. Statistical Engineering Division staff have contributed to the NIST efforts by developing statistical signal processing techniques to correct for the effects of time-base distortion, timing jitter, and impedance mismatch in measurements using high-speed optoelectronic detectors. Recent accomplishments include new results on jitter estimation and uncertainty analysis of time-base distortion. The productivity of this effort is demonstrated by the seven publications (three in refereed journals and four in conference proceedings or technical digests) that NIST staff produced in this area in 1999 and 2000. Another project with EEEL is the work on new measurement methods for characterizing the permit-

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 tivity and loss tangent properties of dielectric materials. Statistical Engineering Division staff are contributing by developing statistical methods for the design and analysis of experiments, optimal estimation of dielectric property parameters, and quantification of uncertainty. The division’s work can have international implications. Staff are working with Building Environment Division personnel from the NIST Building and Fire Research Laboratory on a multiyear assessment of the comparability, equivalence, and traceability of thermal conductivity hot-plate measurements at the national laboratories of five countries. NIST staff have designed experiments and analyzed data for these international intercomparisons, and the results have demonstrated that one of the laboratories consistently produced outlying values and that laboratory-material interactions exist. The ultimate goal of this project is to help establish worldwide equivalence for hardware and protocols used in hot-plate measurements as well as to define a globally accepted international reference material in thermal conductivity. The work on comparing cigarette ignition properties is also in collaboration with the Building and Fire Research Laboratory. Prompted in part by the Cigarette Safety Act of 1984, past NIST work empirically established the feasibility of developing fire-safe cigarettes and led to the development of standardized test methods for assessing ignition propensity. Over the past year, at the request of the Federal Trade Commission, NIST staff have been studying how experimental cigarettes and test equipment have changed over time and evaluating the ignition properties of commercial cigarettes. Division staff are not neglecting good opportunities for appropriate collaborations within the Information Technology Laboratory. One example is the work on ranking algorithms for face recognition with staff from the Information Access Division. The goal of this project is to develop a test suite that will allow researchers to compare the performance of various algorithms for identifying still images of humans without detailed knowledge of the algorithms. While this effort is still in its early stages, division staff are contributing to progress on the project by providing advice on database development and empirical evaluation of the algorithms and systems, investigating various relevant techniques, and developing new methods for comparing and ranking the algorithms. The collaborative nature of the division’s work is not limited to cooperative efforts within NIST. One of the key activities currently under way in the division is the NIST/SEMATECH Engineering Statistics Internet Handbook. This 5-year project is producing a Web-based book intended to help practitioners design experiments and analyze data using good statistical practices, even when they cannot work closely with an experienced statistician. The book covers applications in a variety of areas, including measurement process characterization, process monitoring and improvement, and reliability. While this reference will certainly be helpful to many of the division’s partners at NIST, the intended audience is much broader, and preliminary versions8 of the handbook have been received enthusiastically by both industrial practitioners and the academic community. Current plans call for the final version of the Handbook to be released in May 2001 and to be featured in a session at that month’s Quality and Productivity Research Conference. The above projects are just a few examples of the many diverse activities under way in the Statistical Engineering Division. Division scientists have also been involved in research on statistical methodology and inference, investigating basic questions stimulated by the applications at NIST. They continue 8   The NIST/SEMATECH Engineering Statistics Handbook is available online at <http://www.itl.nist.gov/div898/handbook/index.html>.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 to provide statistical consulting on the many measurement services provided by NIST, and they also reach out to other parts of the U.S. government to supply statistical advice needed in areas such as prediction of retirements at NIST and impartial methodologies for draft lottery selections. An emerging effort for the division is increasing NIST’s role in international metrology and standards. As noted above, staff are already working with other NIST laboratories on international intercomparisons. Two division staff members are serving on standards committees, and some collaborations with foreign standards organizations are being developed. The division is also represented on the NIST-wide task group on voting standards, which is investigating how NIST might productively play a role in the national discussion on improving election systems. Program Relevance and Effectiveness The efforts of the Statistical Engineering Division have a broad impact on the work of the NIST scientists and engineers with whom they collaborate and also on industry practices in general. Looking again at the examples described above, the approaches developed during the project on characterization of high-speed optoelectronic devices have been incorporated into a new measurement service that will benefit industries involved in optical fiber and wireless communications as well as in ethernet and fiber channel networks. The work on characterizing dielectric materials will facilitate the design of new devices and may yield significant savings in industry’s research and development costs. The work on fire-safe cigarettes is not industry-focused but deals with an important public safety concern and is consistent with the NIST mission to improve the quality of life. The NIST/SEMATECH Engineering Statistics Handbook will certainly increase the productivity and accuracy of the scientists and engineers who use it, and it also has potential value as an educational tool. The division’s primary mode of operation is collaborative, and through these collaborations, the division’s results are immediately communicated to and used by relevant parties. However, as mentioned above, the methodologies and outcomes of the division’s work are often relevant to a wider audience, making publications and presentations a necessary element of dissemination efforts. Over the past year, division staff published about 30 articles in refereed journals and 36 papers in conference proceedings or other outlets, and 25 more articles have been submitted. This level of output is very good, and the publications are complemented by the staff’s involvement in professional activities, including talks at conferences, memberships on committees, and service on editorial boards. The Statistical Engineering Division is certainly producing quality work that appropriately supports the NIST mission. However, the panel believes that the impact of the division could be greatly enhanced. The long-term challenge for NIST is to reestablish the division’s stature as a premiere statistical research and consulting organization. A concerted effort is needed to bolster the division’s profile and visibility among professional statisticians in order to restore the division’s standing, which has slowly declined over the past 30 years. Reclaiming a reputation for excellence and a role as a national resource is certainly possible. Few other groups in the United States have access to the range of interesting and challenging statistical problems accessible to the Statistical Engineering Division. The division’s recently revised long-term plans include placing a higher priority on publishing papers in statistical journals, participating in professional activities, hosting statistical symposia and workshops, and developing internship programs with academia, and the panel endorses these tactics. In fact, the panel firmly believes that the revitalized Statistical Engineering Division can play a pivotal role for U.S. industry by promoting industrial statistics and by helping to link key statistical groups in academia and industry and at U.S. national laboratories.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 TABLE 8.9 Sources of Funding for the Statistical Engineering Division (in millions of dollars), FY 1998 to FY 2001 Source of Funding Fiscal Year 1998 (actual) Fiscal Year 1999 (actual) Fiscal Year 2000 (actual) Fiscal Year 2001 (estimated) NIST-STRS, excluding Competence 2.8 2.9 3.0 3.1 Competence 0.3 0.5 0.6 0.6 STRS-supercomputing 0.5 0.1 0.0 0.0 Measurement Services (SRM production) 0.0 0.0 0.0 0.3 OA/NFG/CRADA 0.0 0.1 0.1 0.3 Total 3.6 3.6 3.7 4.3 Full-time permanent staff (total)a 21 23 19 17 NOTE: Sources of funding are as described in the note accompanying Table 8.1. aThe number of full-time permanent staff is as of January of that fiscal year, except in FY 2001, when it is as of March. Division Resources Funding sources for the Statistical Engineering Division are shown in Table 8.9. As of March 2001, staffing for the Statistical Engineering Division included 17 full-time permanent positions, of which 15 were for technical professionals. There were also 6 nonpermanent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers. The panel is delighted to report that, after 3 years of searching, the Statistical Engineering Division now has a permanent chief. The new chief has been at NIST for less than a year but has already had a major positive impact on the atmosphere, morale, and functioning of the division. She has involved division members in discussions of how to do long-term planning, organize individual project efforts, set standards, and establish balance among division programs. The division is also developing a structured performance appraisal process and an organized approach to recognizing staff contributions, both of which should facilitate priority setting, address staff concerns about being overloaded, and ensure that personnel receive full and timely credit for their efforts. The panel commends all of these programs and notes that they are consistent with recommendations made in past assessments. The new division chief is providing exactly the right kind of direction needed to move this division forward. The biggest and most immediate challenge facing the division is the need to rebuild the staff by recruiting high-quality people with the right mix of skills. Areas that are especially important include statistical computing, large databases, and Bayesian inference. A major obstacle to the division’s recruiting efforts is the fact that NIST’s starting salaries for new Ph.D.s are not competitive with what they could obtain in industrial research laboratories. This gap, which is substantial, is particularly severe for researchers working at the interface between computer science and statistics. The division needs sufficient salary flexibility to attract the best candidates at the entry level. In addition, resources and slots should be made available to allow the division to recruit at a senior level, where several key people left NIST in the past few years. While the panel encourages the division, with the support of

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 Information Technology Laboratory management, to actively and aggressively seek to hire people, it cautions that it will not be possible, or even wise, to fill all of the open slots in 1 or 2 years. Restoring the division to full strength will take at least 3 to 5 years, and management at all levels should be committed to sustaining the recruiting efforts over a long period. The panel continues to be concerned about the relative isolation of the division at NIST North. For the staff to be physically separated from their partners in the NIST laboratories negatively affects their productivity and the effectiveness of their collaborations. It may also impede recruiting. Given the interdisciplinary nature of the division’s work and the value of its contributions to projects throughout NIST, the panel once again urges NIST management to thoroughly investigate all possible approaches to relocating the Statistical Engineering Division to appropriate quarters on the main campus. MAJOR OBSERVATIONS The panel presents the following major observations: The panel is extremely pleased with the progress made in the Information Technology Laboratory since the last assessment and appreciates the responsiveness and openness of laboratory management and staff to the panel’s requests and suggestions. Increasing the visibility of the work done in the Information Technology Laboratory is an important goal for management. Enhancing the laboratory’s reputation outside NIST will improve and expand customer relationships. Publications in respected journals and presentations at quality conferences are key elements in this outreach effort. Industry is increasingly using closed consortia instead of open processes to develop standards. In some cases, these closed groups are fairly inclusive and can be the most effective forum for NIST staff to impact industry standards. The Information Technology Laboratory should consider developing a policy on when participation in closed consortia is appropriate as well as on how NIST can encourage industry to utilize open, or at least inclusive, approaches to standards development. The unification of the information technology support functions in the new Information Services and Computing Division was an excellent move, and the panel expects the new organization to provide a better structure for delivering more coherent services to NIST. However, questions remain about what exactly the responsibilities of this division will be and what levels of staffing and resources will be necessary for the division to meet its obligations. The panel urges the division and the Information Technology Services Planning Team to conclude work on the new Information Technology Architecture Plan in a timely manner so that the level and type of NIST-wide services can be defined and communicated to users and division staff alike. The Mathematical and Computational Sciences Division and the Statistical Engineering Division are still located in NIST North, despite reasonable concerns that this separation negatively affects the ability of staff in these divisions to collaborate with scientists on the main campus. The panel strongly urges laboratory and NIST management to thoroughly explore creative solutions to this problem and to communicate these efforts to division personnel. This issue needs to be resolved.

OCR for page 229
An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2001 This page in the original is blank.