10

Computer Systems Laboratory

PANEL MEMBERS

John F. Sheeran, The Boeing Company, Chair

Herbert D. Benington, Paramax Systems Corporation

Dorothy E. Denning, Georgetown University

Clarence G. Feldmann, SofTech, Inc.

James George, Mesa Graphics

Robert E. Kahn, Corporation for National Research Initiatives

Sandra M. Lambert, Citibank

Roger R. A. Morton, Eastman Kodak Company

O. R. Pardo, Bechtel/Parsons Brinckerhoff

Lawrence R. Rabiner, AT&T Bell Laboratories

Michael B. Spring, University of Pittsburgh

Raymond T. Yeh, International Software Systems, Inc.

Submitted for the panel by its Chair, John F. Sheeran, this assessment of the fiscal year 1993 activities of the Computer Systems Laboratory is based on site visits by individual panel members, a formal meeting of the panel on June 13-15, 1993, in the Gaithersburg, Maryland, facilities of the National Institute of Standards and Technology (NIST), and on the annual report of the laboratory.

LABORATORY OVERVIEW
Mission

The Computer Systems Laboratory (CSL) at the National Institute of Standards and Technology helps government and U.S. industry to increase their productive use of computer and related telecommunications systems and to improve the security management and technology of these systems. CSL works closely with the users and producers of computer and telecommunications systems to improve the competitive posture of the U.S. computer-related industry. CSL's programs are mandated by the Federal Property and Administrative Services Act of 1949 as amended by the Computer Security Act of 1987 (Public Law 100-235).

Strategy

The CSL, which includes five divisions (Figure 10.1), improves the productive and reliable use of computers by developing standards and providing advice on the planning, deployment, and use of new information technologies; improves security management and technology by developing timely and innovative solutions to security problems; and contributes to



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 10 Computer Systems Laboratory PANEL MEMBERS John F. Sheeran, The Boeing Company, Chair Herbert D. Benington, Paramax Systems Corporation Dorothy E. Denning, Georgetown University Clarence G. Feldmann, SofTech, Inc. James George, Mesa Graphics Robert E. Kahn, Corporation for National Research Initiatives Sandra M. Lambert, Citibank Roger R. A. Morton, Eastman Kodak Company O. R. Pardo, Bechtel/Parsons Brinckerhoff Lawrence R. Rabiner, AT&T Bell Laboratories Michael B. Spring, University of Pittsburgh Raymond T. Yeh, International Software Systems, Inc. Submitted for the panel by its Chair, John F. Sheeran, this assessment of the fiscal year 1993 activities of the Computer Systems Laboratory is based on site visits by individual panel members, a formal meeting of the panel on June 13-15, 1993, in the Gaithersburg, Maryland, facilities of the National Institute of Standards and Technology (NIST), and on the annual report of the laboratory. LABORATORY OVERVIEW Mission The Computer Systems Laboratory (CSL) at the National Institute of Standards and Technology helps government and U.S. industry to increase their productive use of computer and related telecommunications systems and to improve the security management and technology of these systems. CSL works closely with the users and producers of computer and telecommunications systems to improve the competitive posture of the U.S. computer-related industry. CSL's programs are mandated by the Federal Property and Administrative Services Act of 1949 as amended by the Computer Security Act of 1987 (Public Law 100-235). Strategy The CSL, which includes five divisions (Figure 10.1), improves the productive and reliable use of computers by developing standards and providing advice on the planning, deployment, and use of new information technologies; improves security management and technology by developing timely and innovative solutions to security problems; and contributes to

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 FIGURE 10.1 Structure and organization of the Computer Systems Laboratory.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 U.S. industrial competitiveness by developing generic methods and techniques that facilitate technology improvement and by transferring technology to government and industry organizations. Activities The CSL's array of activities includes: Developing standards, guidelines, profiles, tests and test methods, measurement techniques, reference materials, and prototypes; Developing technical, management, physical, and administrative standards and guidelines for the cost-effective protection of security and the privacy of sensitive information in federal computer systems; Providing technical advice and assistance to federal agencies in implementing standards and guidelines; Collaborating with industrial users in setting standards and implementation protocol; Conducting research to support technical activities; and Participating in planning within NIST and across government agencies for new technology-related initiatives. Resources The CSL's resources include $11.2 million from congressionally appropriated funds, $18.9 million from reimbursements for services provided for other agencies, $1 million from competence programs, and $1.2 million from Information Technology Services, for a total of $31.3 million. The division's staff consists of 237 full-time permanent employees, 17 part-time or intermittent employees, and 24 guest scientists and research associates. CSL-WIDE FINDINGS AND RECOMMENDATIONS--FISCAL YEAR 1993 Presented below are the panel's fiscal year 1993 findings and recommendations on CSL's (1) strategic planning, (2) role in standards setting, (3) role in federal initiatives, (4) facilities, and (5) equipment. The panel notes that CSL followed up on the panel's fiscal year 1992 recommendations regarding higher-powered computers (p. 251), improved standards-setting (p. 252), integrated services digital networks testing (p. 266), and the framework for open systems (p. 257).

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Strategic Planning Findings Some of CSL's divisions have begun to focus in their plans on the demands of external “drivers” for their work. Also, collaboration between CSL divisions is increasing, and there is enthusiasm for initiating a CSL program in electronic commerce, a relatively new technology for the use of digital technology in business transactions. The CSL's 1991 strategic plan does not appear to have been updated, despite the panel's fiscal year 1992 recommendation. The criteria used by most divisions for selecting projects are not clearly based on computer and telecommunications technological opportunities. The panel continues to be concerned about CSL's overdependence on other-agency funding, i.e., about CSL's lack of core funding for development of standards and technology in direct response to CSL's congressional assignments. Other-agency funding provides a market test of the value of CSL's efforts to government and a window on real-world applications but cannot be relied on for long-term planning. Recommendations on Strategic Planning The Computer Systems Laboratory should adopt the strategic planning framework outlined in VCAT Annual Report 1993 (Figure 2, p. 5; Visiting Committee on Advanced Technology, National Institute of Standards and Technology, Gaithersburg, Maryland, January 1994). As a prelude to strategic planning, the Computer Systems Laboratory should categorize current and planned projects as either fundamental research or applied research to determine the balance between these two categories, whatever their source of funding. If the desired ratio of fundamental to applied research is proper, the sources of funding are a secondary consideration. CSL's Role in Setting Standards Findings The CSL's role in setting standards for the government's use of computers and in providing standards and leadership in computing technology for the commercial sector is increasingly constrained by external factors: A few major vendors no longer dominate computing technology and standards. Participants in the joint national and international process for setting standards for computing increasingly regard

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 the process as “broken”; i.e., there are no schedules, commitments, accountability, or overall priority setting or planning for setting computing standards. Vendors and active users have formed consortia over the past several years to accelerate the development of standards, partially because of the inadequacy of the existing formal process. Such consortia in turn are often bypassed by single-topic groups, which seek to define a standard and adopt a position quickly (often to protect the group's market position). The technology involved is moving rapidly, and many leading-edge (and some trailing-edge) issues need resolution. The CSL's role in setting national and international standards for computing is as a participant in “conventions” in which participants are expected to abide by agreed-upon standards. This role is much different from NIST' s role in setting national and international standards for the measurement of physical properties in terms of fundamental constants and basic units of measure. The CSL sets standards for use by the federal government for computing primarily by “rule making,” whereas industry sets computing standards by consensus in formal and consortia forums or by de facto adoption in the marketplace. Despite efforts by industry and CSL to provide for open systems (interoperability) through development and adoption of joint national and international standards (open specifications openly arrived at), de facto standards still dominate the marketplace. Government is now, as distinct from the 1960s and 1970s, too small a portion of the market to dictate standards for computing products and services. In fact, government-dictated standards now run the risk of being bypassed by innovations in the much larger commercial market. It is relevant to note that the government clients for which CSL sets standards do not participate in evaluating CSL's performance. The above constraints limit CSL's impact on setting standards and dictate that CSL should change its approach to influencing the setting of the computing standards government and industrial use. Recommendations on Standards Setting The Computer Systems Laboratory should determine specific needs of U.S. industry, civilian agencies, and Department of Defense agencies for its services. The market for CSL's products and services can be determined by conducting surveys, public workshops, and management interviews; otherwise, setting priorities and developing performance metrics will continue to be difficult.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 To minimize internal inefficiencies and maximize external value, CSL could emulate industry's increasingly customer-driven approach by inviting its government clients to participate in annual assessments of CSL's performance. The Computer Systems Laboratory should arrange for increased collaboration among the major users and promulgators of computer-related standards in prioritizing, setting, and adopting standards. Such collaboration offers the best hope for achieving standards that will be in the best interest of users and will lead to a better focus on the concrete problems affecting interoperability of CSL's commercial and government clients. The Computer Systems Laboratory's goal for standards setting should be interoperability among distributed, heterogeneous systems, rather than the current, more limited vision of open systems. The former goal better addresses the current mix of proprietary, de facto, and formal standards. It also offers incremental value, rather than the postponed perfection of open systems, and thus will be more readily supported by CSL's constituents. This goal has already been adopted by a few of CSL's divisions. The Computer Systems Laboratory should increase its collaboration with other NIST activities, particularly with NIST's Manufacturing Technology Centers. Such collaboration would provide real-world testbeds of CSL's work and channels for disseminating working standards. The Computer Systems Laboratory should develop, in collaboration with its government constituencies, measures for its success and/or failure in setting standards--e.g., vendor and user adoption of such standards. CSL's Role in Federal Initiatives The CSL proposes a mix of technical leadership in communications infrastructure and parallel processing, and in second-level issues such as electronic commerce and distributed multimedia in the federal National Information Infrastructure (NII) initiative. Other federal efforts that involve CSL are the High Performance Computing and Communications (HPCC) initiative and the National Research and Education Network (NREN) program. Even in the context of uncertainty of funding, the CSL's plans lack the comprehensive assessment and aggressive tactics necessary to address factors that will surely limit the Clinton administration 's national initiatives, such as: Standards for connectivity. The administration's initiatives involve “networks of networks.” The standards and rules for connectivity will have major technical, economic, and policy impacts. Security across networks. Standards, procedures, and policies will be needed to provide confidentiality and integrity

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 of data, to control access, to authenticate sources, and to ensure auditability and traceability. Network and system management, including user and system directories. The CSL provided leadership with its government standard for network management, but the enormous complexity of the systems and networks to be interconnected will demand major improvements in techniques, standards, and software for system and network management. At this time, the CSL's plans and resources are not of the scale to address adequately the above limiting factors. Conclusions on Role in Federal Initiatives The panel endorses findings reported in the section “Information Technology,” pp. 19-20, VCAT Annual Report 1992 (National Institute of Standards and Technology, Gaithersburg, Maryland, January 1993). . . . We believe that networking standards and hardware/software interoperability, largely the responsibility of the Computer Systems Laboratory at NIST, are crucial to the implementation of the great “information highways” of the future. . . . However, a clear picture of how the many disparate activities fit together is lacking. In short, we see a crucial leadership vacuum in information technology. . . . NIST has an obvious leadership function, particularly in the areas of networking standards, equipment and software interoperability and performance standards, and government computer systems specifications. . . . Unfortunately, complete fulfillment of these tasks demands considerably more effort than the Computer Systems Laboratory can muster at its current size. Consequently, we do not think NIST can credibly represent the U.S. government' s position and interests in information technology standards and networking under the current circumstances. NIST's authority and resources are simply not on a par with its natural responsibilities. We believe that this situation is one aspect of our national policy in information technology that requires attention by the Administration and the Congress. Facilities Current constraints on both office and laboratory space limit the CSL's plans for acquisition of staff and will inhibit the laboratory 's recruitment (funding permitting). Staff acquisition and retention are central to the laboratory's ability to accomplish its mission.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Recommendation on Facilities The Computer Systems Laboratory should consider alternatives to its current facilities, e.g., temporary office space on campus or leased commercial space nearby, if additional space cannot be made available within NIST facilities. Equipment CSL's continuing lack of cutting-edge equipment and leading-edge applications is cited in several of the divisions' reviews. Equipment problems were addressed in the panel's fiscal year 1992 report (p. 251): “CSL does not have the high-computing-power, graphics-intensive personal computers and workstations that are expected to be commonplace in a 2- to 5-year time frame.” Recommendation on Equipment The Computer Systems Laboratory should seek to acquire cutting-edge equipment that mirrors the scale and heterogeneity of government and commercial computing. ASSESSMENT OF DIVISION PROGRAMS Information Systems Engineering Division Mission and Resources The Information Systems Engineering Division develops standards for information systems and provides technical assistance to government and industry in data administration, data management technology, computer graphics, and the validation of software standards. In fiscal year 1993, the Information Systems Engineering Division had 38 full-time, 3 part-time permanent, and 8 intermittent employees. Funding for Scientific and Technical Research and Services (STRS) from NIST was $1.3 million, reimbursement for services to other agencies was $5.6 million, and NIST's director provided $204,000 for building technical competence. Industry provided $205,000 worth of hardware and software. The division's broad collection of hardware consists predominantly of desktop systems. The division has a significant shortage of modern, powerful workstations, i.e., the leading-edge hardware necessary to be a leader in applying information systems technology.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Panel's Findings--Fiscal Year 1993 Personnel in the Information Systems Engineering Division were dedicated, highly motivated, and discussed their activities with candor. Their short-term tactical planning is commendable; however, some of the staff are not aware of the division's strategic plans. Researchers are self-motivated, pursue professional interests, gain support for their ideas from the division management or other agencies, and achieve worthwhile results that they publish. Despite a nationwide preponderance of desktop computing equipment, the division ignores the needs of desktop computing users, choosing rather to address needs related to the few major government purchases. For example, the division develops standards and provides technical assistance for Sun, SGI, and VAX but develops few if any services for personal computers. Desktop systems are growing in power (if not in cost) and will become the laboratory and networked computers of tomorrow. With the division's current focus, the division-generated Federal Information Processing Standards could become irrelevant to a majority of future government computing equipment. About 80 percent of the division's funding comes from other agencies. As a result, the division focuses on other-agency conformance testing, which consumes valuable staff time that, if core funding were available, could be assigned to CSL 's primary statutory mission and NIST's long-term goals. The division has proposed that alternative sources be used to conduct conformance testing; however, the division would still be needed as a proactive catalyst for promoting the development and practice of conformance testing. The panel commends the Information Systems Engineering Division on its excellent support of the next-generation Information Resource Dictionary System (IRDS2). The next stage in the expansion of data administration and management involves object-oriented technology, which directly affects both data administration and management and software engineering. Panel's Recommendations--Fiscal Year 1993 In the next iteration of its strategic planning, the Information Systems Engineering Division should methodically identify its markets and customers, prioritize services to be provided, appraise the division's capacity to meet customer needs, schedule delivery dates for services, and plan for the dissemination and delivery of services. Services that have decreasing or marginal utility should be identified for possible elimination or reduction. Also, the division should explore the gains from planning for joint programs with CSL's Systems and Software Technology Division. The Information Systems Engineering Division should influence the entire spectrum of computing in government,

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 including desktop computing, rather than only the computing of a few large machines purchased by government. The division should become more involved in developing standards and practices for desktop computers. The Information Systems Engineering Division should (1) improve the timeliness of its delivery of conformance tests (conformance tests have more impact during the ramping-up phase than at the end of the product cycle), (2) evaluate the utility of the rewritten Graphical Kernal Standard conformance test as a function of timeliness in the product cycle, (3) apply formal description languages to assist in the development of conformance tests, (4) host industry leaders in a product area in need of testing, to discuss industrial needs for conformance testing and arrange for a collaboration in developing the conformance testing methodology and tools, and (5) search for alternative sources for the development, validation, and performance of conformance tests. The Information Systems Engineering Division should reevaluate its apparent strategy of seeking support for ongoing rather than new activities. The Information Systems Engineering Division should develop an IRDS2 testbed. Systems and Software Technology Division Mission and Strategy The Systems and Software Technology Division provides technical assistance to users of information systems and software and to the corresponding industry through promotion and development of concepts and technology for open systems. This mission and the division's seven specific tactical objectives provide much better guidance than was evident at the time of the fiscal year 1992 assessment. The division proposes (1) to align its efforts with the national HPCC, NREN, NII, and Information Infrastructure and Technology Applications initiatives; (2) to respond to U.S. user requirements in government and industry; and (3) to provide technical support to technology development and standards making in support of U.S. industry. The divisions's seven tactical objectives are to Identify and nurture emerging information technologies; Promote the convergence of standards that underpin information systems and software; Build consensus among developers, vendors, and users; Act as a catalyst and accelerate the development and use of information technology; Maintain national and international perspectives and participation in selecting and implementing projects;

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Serve as an unbiased (third-party) technical authority in the research and user communities and in the marketplace; and Seize opportunities to make a difference through the development of technology and setting of standards leading to open systems. Resources The division's total funding for fiscal year 1993 was $4.8 million, of which $1.1 million was directly appropriated by Congress, $205,000 was from the NIST's director's competence building fund, $3.3 million (70 percent) was from other federal agencies for services rendered, and $100,000 was from the Department of Defense as compensation for conformance tests. In fiscal year 1993, 38 of the staff were full-time permanent employees (a net increase of 7 over fiscal year 1992). Five university faculty members, five students, one guest worker, and two research associates supplemented the staff's laboratory research. Responses to Fiscal Year 1992 Recommendations The Systems and Software Technology Division responded well to the panel's fiscal year 1992 recommendations. The panel believes that the division is now much better organized and coordinated, although the division considered that the panel's criticism regarding “lack of coordination with other NIST groups” reflected a misunderstanding rather than a true picture. In any case, the division's interaction and coordination with other groups within NIST now include regular biweekly technical interchanges as well as cooperative technical efforts. The panel endorses the division's strategy of using specific tactical objectives and aligning its efforts with the four national initiatives mentioned above. The emphasis on joint proposals with other CSL divisions is also commendable. However, the panel was concerned that the division did not seem to follow up on its good strategy with a process for selecting which standards and technology to work on. The needs and interests of the user community, not the talents of division staff, should be paramount in selecting the division's projects. Existing staff should be reassigned and recruited to mirror users' needs. Long-range technology forecasting seemed to be lacking in the division, which has, in general, a short-term focus. The impression is that the division considers potential areas of investigation “bottom-up ” rather than “top-down.” That is, individual technical topics are considered and selected without evidence that the staff has given due consideration to the broad picture of the technical area. For example, the areas of multimedia and virtual reality do not seem to be as critical as

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 The staff collaborates extensively with people in other agencies and the private sector to obtain leverage from other work; to develop prototypes, standards, and guidelines that are responsive to government needs; and to affect the development of national and international standards that satisfy U.S. government requirements. Collaboration with government users in the development of prototypes, standards, and guidelines ensures that the division's results are relevant. Resources The Computer Security Division is severely understaffed and underfunded given its statutory security responsibilities, the growing national recognition of the need to protect unclassified but sensitive information, and the unique role the division can play in fostering security in commercial architectures, hardware, and software. The division's computing hardware appears to be adequate for existing projects. However, limited space would make it difficult to hire new staff if funding were to be increased. Panel's Findings--Fiscal Year 1993 The CSD has prime responsibility for NIST's computer security activities; however, there is also a small program managed by the CSL's associate director for computer security as well as security-related activities in other CSL divisions. The Computer Security Division makes full use of work of others, establishes cost-sharing partnerships, transfers routine work externally, and influences others to respond to the division's objectives. Noteworthy achievements of the division in cryptographic-based security technology since the fiscal year 1992 assessment include the following: The Digital Signature Standard will be released as a Federal Information Processing Standard (FIPS) pending a patent resolution; The Secure Hash Algorithm was published as a Federal Information Processing Standard (FIPS 180); Security requirements for cryptographic modules (FIPS 140-1) were drafted; Security guidelines for local area networks were drafted and will soon be released for public comment; Prototype “smart” cards for authentication and digital signatures were developed; An advanced smart card access control system was developed under sponsorship of the Advanced Research Projects Agency (ARPA); and

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 The division sponsored extensive participation in national and international standards activities. The smart cards are an advanced technology and are in demand by government agencies. The Computer Security Division received four awards for development of these cards. Another noteworthy achievement is completion of draft federal criteria for computer security. This was followed by initiation of a major collaborative effort with other countries to develop common criteria. The common criteria will pull together and supersede the federal criteria, European international criteria, and Canadian criteria and will give vendors the advantage of having to meet only one set of security criteria. The federal criteria, together with comments received on them, will be a major source of input for developing the common criteria. Although the impact of the federal and common criteria on bringing secure products to market has yet to be determined, NIST's contributions to developing the criteria have received considerable recognition. Many participants are optimistic that the new criteria will allow for a shorter assessment cycle, especially for products requiring lower levels of assurance, than do the existing federal criteria. The division participated, as secretariat, in the Forum of Incident Response and Security Teams, an effort aimed at helping government and nongovernment organizations to deal effectively with computer break-ins, viruses, and other threats. Considerable progress was made toward publishing a computer security handbook and framework that will enable agencies to identify, assess, and control secure computer applications and communications. Significant progress was made in support of developing security services and interfaces in all areas of open system environments. NIST makes its work products available electronically through a dial-up bulletin board and the Internet. These products provide a valuable service to the public as well as to other federal agencies. Future Directions, with Panel Comments In the cryptography area, the Computer Security Division would like to develop standard services and interfaces for authentication, single log-on, distributed certification authority systems, and standards and guidelines for validating cryptographic systems and environments. All of these areas build on current work and are appropriate for the division's to pursue. In the area of federal criteria and common criteria, the division wants to develop (1) a “trust technology” assessment program for evaluating computer security and products, (2) an

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 evaluation methodology for composite or integrated systems, and (3) infrastructure and guidelines for in situ accrediting systems. The panel endorses a research project being considered by the division to develop guidance for securely handling data and software in distributed systems. The research would tie in with that of CSL's Systems and Software Technology Division and the Information Systems Engineering Division. The CSD also proposes research on using audit analysis for assessing intrusion detection systems. The Computer Security Division is currently counseling the White House on security for medical information as part of the White House 's development of a national health care program. Panel's Recommendations--Fiscal Year 1993 The Computer Security Division should develop criteria for determining how much of a networked system must be trusted. Much of this work would tie into the security needs of the National Information Infrastructure initiative. Since several research projects on detection of intrusion are well under way outside NIST, the division should rely on external research rather than start an independent project. The Computer Security Division should continue to consult, develop guidelines, and provide training programs, especially in computer security risk management and analysis, and should complete or upgrade its computer security handbook. The Computer Security Division should consider expanding its activities in ensuring the security of medical information. The division should take a much more active role in planning for research on ensuring overall National Information Infrastructure security requirements. Systems and Network Architecture Division Mission, Strategy, and Resources, with Panel Comments The Systems and Network Architecture Division develops Open Systems Interconnection (OSI) technology and standards, develops and applies automated protocol methods, and develops technology for integrated, interoperable network management. The division views its mission as bounded by protocols for open systems interconnections. Even though the division's comprehensive draft strategic plan for October 1993 through 1998 was well structured, was informative, and provided an excellent basis for operation, the plan is being outdated by NIST's expanding mission. The division's emphasis on “getting closer to the customer” was a positive step in fiscal year 1993 that should be

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 institutionalized, especially in the development of interoperability across heterogeneous networks. During fiscal year 1993, the division received $1.1 million in directly appropriated funds from Congress, $3.0 million in reimbursements from other agencies, and $170,105 in equipment loans from industry. The staff consists of 46 employees. The division has a competent staff and--because of its involvement in testing for interoperability--has, for the most part, state-of-the-art equipment. The division works primarily (73 percent) for other agencies. The division could benefit from more varied and flexible arrangements for cooperation, e.g., cooperative arrangements such as those used in the Technology Reinvestment Program of the U.S. Department of Defense, in addition to Cooperative Research and Development Agreements. The panel endorses the division's plan for inviting guest researchers. Response to Panel's Fiscal Year 1992 Recommendations The panel's fiscal year 1992 report (pp. 256-257) recommended by inference that (1) the CSL (i.e., the Systems and Network and Architecture Division) should evaluate the relative merits of industry's de facto standards and protocol for independent interfaces as well as network interoperability, thereby providing vendor-neutral input needed to best pursue coexistence and transition (i.e., issues related to OSI, TCP/IP, systems network architecture, and other major networking architectures); (2) the division should plan for defining standards for the new NREN, for a major upgrade in national network capability, and for developing new mechanisms for collaboration with other major participants in the NREN; and (3) the division should provide needed leadership for OSI network management in the United States through increasing the resources of its Common Management Information Protocol Network Management Laboratory. Recommendation (1) was partially addressed through participation in the Internet Society's development of standards and specifications and in the Open Management Roadmap partnership for developing network management specifications (Open Management Network Interoperability Points) to satisfy user priorities for products that vendors agree to supply. The protocol, standards, and specifications need to be developed at a faster pace. With reference to recommendation (2), the definition of standards for the NREN was deferred until fiscal year 1994, pending the allocation of funds. With reference to recommendation (3), the division decided that the current resources of its Common Management Information Protocol Network Management Laboratory were adequate given the division's involvement in workshops, consortia, and the Network Management Forum.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Panel's Findings--Fiscal Year 1993 The Systems and Network Architecture Division has good national and international credibility. Its customers are correctly identified as users and vendors of networking technologies, and the division is active in both the OSI and Internet arenas. Within the scope of nonproprietary networks with open systems interconnections, the division has correctly identified areas for future activity. The division maintains contact with its customers, helps develop consensus among users and vendors, aids the standards-setting process, defines missing elements in the development and use of the technology, and provides the test definition and support called for in its mission. Highlights of the Systems and Network Architecture Division's achievements during fiscal year 1993 were as follows: The division completed the draft Industry/Government Open Systems Specification, which harmonizes the profile specifications of the Government Open Systems Interconnection Profile (GOSIP), Manufacturing Automation Protocol and Technical and Office Protocol, Unified Communications Architecture, and Canadian GOSIP. Division staff joined vendors and users in an Open Management Roadmap partnership to develop network management specifications called Open Management Network Interoperability Points (OMNI Points) that freeze specifications as prioritized by the users and according to which vendors agree to build products. Such OMNI Points tend to minimize the risks to vendors and users. Two major network management protocols--the Systems Network Management Protocol and the Common Management Information Protocol--have grown out of the Open Management Roadmap partnership. The division sponsored well-attended industrywide laboratory tests for OSI dynamic routing and frame relay. The division provided staff support for the electronic mail and directory services interface standards of the Institute for Electrical and Electronics Engineers. Other notable achievements reported by the division for fiscal year 1992 included the continued publication of guidelines for use of OSI applications, support of the GOSIP testing program, work on helping the Internet Society to resolve the TCP/IP addressing problem, and the use of ESTELLE (a formal specification language) to evaluate the Network Security Protocol Specification. The division plans to emphasize developing a model for open communications protocols that draws on both the TCP/IP and OSI standards; assisting federal agencies in the use of electronic document interchange and promoting its integration into open systems; promoting standards for standardizing electronic transactions; developing models for electronic commerce--starting

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 with electronic data interchange; analyzing the necessary infrastructure standards, services, and products; and working with other agencies to develop model applications. Based on its experience in GOSIP certification and other forms of testing against standards, the division is considering initiating a program in automated testing that uses ESTELLE as a tool for generating code for implementing protocols from formal definitions of requirements. Currently, much money and time are spent developing manual test suites and manually running them. Other opportunities for the division include working with federal agencies to deploy large X.400 and X.500 (electronic data transmission protocols) applications; collaborating with industry in defining the barriers to and interface requirements for interoperations involving de facto standards and proprietary networks; and defining the protocol requirements for NREN and NII networks in the near term, medium term (5 to 10 years), and long term (10 to 20 years). Also, the division is well suited to become a demonstration site of OSI functionality among its own workstations. In its close associations with industry, the division is a full partner, as is illustrated by the division's linkage with industry and users through the Network Management Forum. Panel's Recommendations--Fiscal Year 1993 A division-wide skills assessment is in order if the Systems and Network Architecture Division is to address a broader network mission. In the area of electronic commerce, the Systems and Network Architecture Division should focus on only a few, well-defined pilot projects, because of the large scope of electronic commerce. The division should develop a program that focuses on automated standards testing, explores the use of formal definitions, and supports the development of tools such as ESTELLE and commercial automated test generators. The Systems and Network Architecture Division's exemplary strategic plan should be revised to keep pace with NIST's expanding mission and requirements related to its role in developing the National Research and Education Network. The Systems and Network Architecture Division should move beyond viewing its mission as bounded by open (i.e., nonproprietary) protocols. The division should retain its current internal Open Systems Interconnection (OSI) focus but should also pioneer in the promotion of OSI for use in the United States. The division should provide broad support for its customers in defining, operating, and evolving heterogeneous network systems. This approach should include refinement of open system network concepts and continued development of associated supporting concepts, such as distributed systems networking, specific frameworks for addressing user requirements, and

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 standards to ensure the interoperability of heterogeneous network systems. The Systems and Network Architecture Division should address problems that users experience with proprietary (or partially proprietary) systems and networks. Many of the major government systems are legacy systems developed on proprietary communications architectures. The Systems and Network Architecture Division should help ensure that tomorrow's open networks provide access to these major systems from workstations that span networks using open and de facto standards. These same problems confront U.S. industry and academia, and the solutions will dramatically affect overall national productivity. The division should form a joint task force with counterparts in the Manufacturing Engineering Laboratory in order to define their separate and joint responsibilities for computer systems and networks--especially where real-time processes are involved. Advanced Systems Division Mission, Strategy, and Resources, with Panel Comments The Advanced Systems Division conducts research and provides technical assistance to federal agencies and industry organizations in advanced communications such as integrated services digital networks, distributed systems, automated recognition, data storage technologies, and parallel processing. The division's goals are (1) to accelerate the commercialization of otherwise unfocused academic or industrial research, (2) to broker essential cooperation among industrial segments, and (3) to provide measurement and instrumentation methods needed for commercialization. The Advanced Systems Division's plans anticipate major participation in the multiagency HPCC program as described in the High-Performance Computing Act of 1991 and the Information Infrastructure and Technology Act of 1992 introduced by then-U.S. Senator Albert Gore. Given that the division's three general goals are appropriate and clearly within the scope of CSL's mission, the division's performance should be appraised in the context of those goals. For example, significant efforts were made to develop handwriting recognition technology through the development of a corpora for testing. As a first major test of the corpora, the work of one participating vendor, much of which was not made public because of its proprietary nature, showed a recognition success rate of 98.5 percent of corpora input. No other systems successfully recognized more than 97 percent, and NIST systems scored no higher than 95 percent. In view of documented technical achievements prior to the division's initiative, the panel 's concern is whether the Advanced Systems

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Division's effort adequately addresses the division's goal of accelerating the commercialization of otherwise unfocused academic or industrial research. Even so, development of a corpora for testing handwriting technology provides reliable, baseline information. The division's stated strategy is to focus its contributions on the convergence of computation, communication, and display technology. However, it is not clear to the panel what this focus contributes to selecting particular projects. The division's research in handwriting and speech recognition can easily be related to human-computer interfaces; however, if this is the purpose, it is less clear why gesture input, “eyephones,” and a myriad of other human-computer input and output technologies are not also examined. Also, CSL's virtual reality research is housed in the Systems and Software Technology Division, but there is much overlap with the mission of the Advanced Systems Division. The panel had difficulty in assessing the relevance and relatedness of the Advanced Systems Division's various projects as described in the final draft of Implementation Plan for High Performance Computing and Communications/Information Infrastructure Budget Initiative for Fiscal Year 1994 (National Institute of Standards and Technology, April 16, 1993). In contrast, the division's projects as outlined in a 1991 joint planning document of the Advanced Systems Division and the Computing and Applied Mathematics Laboratory were coherent and relevant. The division's strategic planning process is plagued by often delayed or uncertain funding and a budget that is more than 60 percent dependent on other-agency funding. During fiscal year 1993, the Advanced Systems Division received $2 million in STRS funding, $3.7 million from other agencies, $639,000 from the NIST's director's competence building fund, and reimbursements of $1 million for technical services to support a staff of 74. Resources available to the division in terms of equipment and personnel are appropriate. The division has a reasonable array of equipment, which may be characterized as being on the cutting edge. More importantly, state-of-the-art work is being carried out on the equipment. The division has benefited over the last several years from a demonstration personnel project authorized by Congress that allows NIST to more easily recruit and retain well-qualified personnel. Response to Fiscal Year 1992 Recommendations In its fiscal year 1992 report, the panel made no specific recommendations regarding the Advanced Systems Division.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Panel's Findings--Fiscal Year 1993 The Advanced Systems Division is having a positive impact on the marketplace by encouraging new technologies as well as by developing and disseminating test metrics and methodology. The division's scholarly reports provide significant impetus to its counterpart research communities. The panel reviewed the following: (1) parallel processing and performance measurement, (2) corpora for speech and handwriting, (3) corpora for text retrieval, (4) optical disk work, and (5) work on integrated services digital networks and distributed systems. The parallel processing and performance measurement project has been focused over the last few years on developing advanced measurement devices (e.g., MULTIKRON). While MULTIKRON began as a means for measuring the performance of parallel processing systems, it appears to be applicable for unobtrusive in situ measurements for industrial process control. MULTIKRON's design is simple, elegant, easily extended, and suited for multiple applications. The potential scope of MULTIKRON 'S application may far exceed the applications proposed or imagined by the division staff. The Advanced Systems Division provides a clearinghouse for speech recognition databases obtained either through contracts with industry or from ARPA programs. These databases are distributed on CD-ROMs, for which the division contracts production. Currently, only ARPA contractors and a newly formed private corporation, the Linguistic Data Consortium, have access to the division's databases. Unfortunately, since the division's Speech Group is funded primarily by ARPA, there is little incentive to satisfy or examine the needs of speech researchers outside ARPA. The division's corpora and competitions in text retrieval address a mature technology. While the development of testbeds for a 40-year-old technology would not be easily justified, there is evidence that new retrieval techniques are being developed. Other retrieval issues beg for attention; therefore, testbed development requires a strong justification. For example, image retrieval and intelligent electronic text conversion are not being addressed by the division, are as important and difficult as current projects within the division, and have equally challenging technical problems. Data preservation is of concern to every community working in information technology. The division's efforts in optical technology, which has been heralded as a significant step in developing stable, long-term storage media, are of strategic importance to data preservation; however, the division's efforts to quantify and measure the stability of other emerging storage media, e.g., magnetic and electrostatic media, are of equal importance.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 Deployment of integrated services digital networks (ISDNs) is relatively recent; however, the related standard has existed in one form or another for many years. The Advanced Systems Division is correctly shifting its efforts toward higher-bandwidth and more robust protocols (e.g., broadband-ISDN, asynchronous transfer mode). Its examination of wireless communication needs more planning. Exploration of the integration of the base technologies and their impact on available bandwidth is encompassed by the charge to the Advanced Systems Division. Similarly, understanding distributed systems is critical to the success of the HPCC initiative. The Advanced Systems Division is only one of the CSL divisions addressing the HPCC initiative. Panel's Recommendations--Fiscal Year 1993 The Advanced Systems Division should not overemphasize its involvement in the High Performance Computing and Communications initiative but rather should concentrate on its mission. The Advanced Systems Division should collaborate with other divisions within the Computer Systems Laboratory in developing program priorities. The Advanced Systems Division should collect and maintain standard databases for developing and evaluating speech recognition systems. The Advanced Systems Division should sponsor open competition in the development of speech and handwriting recognition technology, and the division should make better use of its special strengths and expertise to advance speech recognition research in general rather than serve only a small part of the research community. The division should expand its efforts to inform others about the MULTIKRON chip through publications, reports, and other mechanisms, e.g., professional contacts. The division should explore additional uses of the chip. The division's classical optical disk metrology should be further promoted. The panel's biggest concern about the Advanced Systems Division's efforts is what it cannot do rather than what it is doing. The division reports annually to the panel on what it can do given its expertise, funding sources, and time limitations. The division should also specify what it is not working on and why. The division management began to address these questions for fiscal year 1993.

OCR for page 213
An Assessment of the NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY PROGRAMS: Fiscal Year 1993 This page in the original is blank.