Executive Summary

INTRODUCTION

Nuclear power plants rely on instrumentation and control (I&C) systems for monitoring, control, and protection. During their extensive service history, analog I&C systems have performed their intended monitoring and control functions satisfactorily. Although there have been some design problems, such as inaccurate design specifications and susceptibility to certain environmental conditions, the primary concern with the extended use of analog systems is effects of aging, e.g., mechanical failures, environmental degradation, and obsolescence.

The industrial base has largely moved to digital-based systems1 and vendors are gradually discontinuing support and stocking of needed analog spare parts. The reason for the transition to digital I&C systems lies in their important advantages over existing analog systems. Digital electronics are essentially free of the drift that afflicts analog electronics, so they maintain their calibration better.2 They have improved system performance in terms of accuracy and computational capabilities. They have higher data handling and storage capacities, so operating conditions can be more fully measured and displayed. Properly designed, they can be easier to use and more flexible in application. Indeed, digital systems have the potential for improved capabilities (e.g., fault tolerance, self-testing, signal validation, process system diagnostics) that could form the basis for entirely new approaches to achieve the required reliabilities. Because of such potential advantages, and because of the general shift to digital systems and waning vendor support for analog systems, the U.S. nuclear power industry expects substantial replacement of existing, aging analog systems with digital I&C technology. For the same reasons, designs for new, advanced nuclear power plants rely exclusively on digital I&C systems.

Challenges to Successful Introduction of Digital Instrumentation and Control Systems

Successful introduction of digital I&C systems into U.S. nuclear power plants faces several challenges:

  • uncertainty inherent in introduction of new technology

  • shift of existing technology base from analog experience

  • technical problems identified from some applications of digital I&C in nuclear power plants

  • difficult, time-consuming, and customized licensing approach

  • lack of consensus (between the U.S. Nuclear Regulatory Commission [USNRC] and the regulated industry) on issues underlying evaluation and adoption of digital I&C technology and means to obtain a satisfactory resolution

In essence, the problem is to develop a systematic regulatory review and approval methodology for digital I&C systems that allows obtaining the safety and reliability benefits available from this technology while avoiding the introduction of offsetting safety problems.

The transition from analog to digital I&C systems in nuclear power plants is not straightforward; one must carefully account for the ways in which digital I&C implementations are different and frame regulations that reflect those differences.

Response of the U.S. Nuclear Regulatory Commission to the Challenges

The USNRC has reviewed a number of analog-to-digital ''retrofits" in nuclear power plant I&C systems and is in the

1  

The committee intentionally avoided partitioning digital systems between hardware and software; rather the committee believes that digital systems are better treated in an integrated manner. Nevertheless, some of the specific topics addressed in the report merited discussion as "hardware" or "software" items.

2  

The reader should note, however, that since most sensors will remain analog-based, drift will not be eliminated, though it will likely be improved, especially if the digital I&C component contains software specifically designed to offset expected sensor drift.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Executive Summary INTRODUCTION Nuclear power plants rely on instrumentation and control (I&C) systems for monitoring, control, and protection. During their extensive service history, analog I&C systems have performed their intended monitoring and control functions satisfactorily. Although there have been some design problems, such as inaccurate design specifications and susceptibility to certain environmental conditions, the primary concern with the extended use of analog systems is effects of aging, e.g., mechanical failures, environmental degradation, and obsolescence. The industrial base has largely moved to digital-based systems1 and vendors are gradually discontinuing support and stocking of needed analog spare parts. The reason for the transition to digital I&C systems lies in their important advantages over existing analog systems. Digital electronics are essentially free of the drift that afflicts analog electronics, so they maintain their calibration better.2 They have improved system performance in terms of accuracy and computational capabilities. They have higher data handling and storage capacities, so operating conditions can be more fully measured and displayed. Properly designed, they can be easier to use and more flexible in application. Indeed, digital systems have the potential for improved capabilities (e.g., fault tolerance, self-testing, signal validation, process system diagnostics) that could form the basis for entirely new approaches to achieve the required reliabilities. Because of such potential advantages, and because of the general shift to digital systems and waning vendor support for analog systems, the U.S. nuclear power industry expects substantial replacement of existing, aging analog systems with digital I&C technology. For the same reasons, designs for new, advanced nuclear power plants rely exclusively on digital I&C systems. Challenges to Successful Introduction of Digital Instrumentation and Control Systems Successful introduction of digital I&C systems into U.S. nuclear power plants faces several challenges: uncertainty inherent in introduction of new technology shift of existing technology base from analog experience technical problems identified from some applications of digital I&C in nuclear power plants difficult, time-consuming, and customized licensing approach lack of consensus (between the U.S. Nuclear Regulatory Commission [USNRC] and the regulated industry) on issues underlying evaluation and adoption of digital I&C technology and means to obtain a satisfactory resolution In essence, the problem is to develop a systematic regulatory review and approval methodology for digital I&C systems that allows obtaining the safety and reliability benefits available from this technology while avoiding the introduction of offsetting safety problems. The transition from analog to digital I&C systems in nuclear power plants is not straightforward; one must carefully account for the ways in which digital I&C implementations are different and frame regulations that reflect those differences. Response of the U.S. Nuclear Regulatory Commission to the Challenges The USNRC has reviewed a number of analog-to-digital ''retrofits" in nuclear power plant I&C systems and is in the 1   The committee intentionally avoided partitioning digital systems between hardware and software; rather the committee believes that digital systems are better treated in an integrated manner. Nevertheless, some of the specific topics addressed in the report merited discussion as "hardware" or "software" items. 2   The reader should note, however, that since most sensors will remain analog-based, drift will not be eliminated, though it will likely be improved, especially if the digital I&C component contains software specifically designed to offset expected sensor drift.

OCR for page 1
process of reviewing designs of advanced plants. However, the review process has largely been customized for each application because of the lack of agreed-upon applicable criteria.3 In addition, advisory committees, including the Advisory Committee on Reactor Safeguards (ACRS) and the Nuclear Safety Research Review Committee (NSRRC), have expressed concern that the USNRC may be lagging behind in its understanding of digital I&C systems and have urged the development of a framework to guide the regulation of digital I&C technology. To address technical concerns, and in hopes of developing a wide consensus across the USNRC and the nuclear industry for a regulatory program, the USNRC held a workshop in September 1993. While a useful forum, the workshop did not lead to a consensus, and the USNRC requested the assistance of the National Research Council. THIS STUDY Committee's Task The National Research Council was asked by the USNRC to conduct a study (including a workshop) on application of digital I&C technology to commercial nuclear power plant operations. The National Research Council accordingly appointed a committee (hereafter the committee) to carry out the study, which was conducted in two phases. In Phase 1, the committee was charged to define the important safety and reliability issues that arise from the introduction of digital I&C technology in nuclear power plant operations, including operations under steady-state, transient, and accident operating conditions. In response to this charge, the committee identified eight key issues associated with the use of digital I&C systems in existing and advanced nuclear power plants. In Phase 2 of the study, the committee was charged to identify criteria for review and acceptance of digital I&C technology in both retrofitted reactors and new reactors of advanced design; to characterize and evaluate alternative approaches to the certification or licensing of this technology; and where sufficient scientific basis exists, recommend guidelines on the basis of which the USNRC can regulate and certify (or license) digital I&C technology, including means for identifying and addressing new issues that may result from future development of this technology. In areas lacking sufficient scientific basis to make such recommendations, the committee was to suggest ways in which the USNRC could acquire the required information. In carrying out its Phase 2 charge, the committee limited its work to those issues identified in Phase 1. The issues were chosen because they were difficult and controversial. Further, the committee recognized that by law, the responsibility for setting licensing criteria and guidelines for digital I&C applications in nuclear plants rests with the USNRC. Thus, the reader should not form too literal an expectation that the committee has provided a cogent set of principles, design guidelines, and specific requirements for ready use by the USNRC to assess, test, license, and/or certify proposed systems or upgrades. Rather, the results of the study are presented in the form of conclusions and recommendations related to each issue and primarily addressed to the USNRC for their consideration and use. In the committee's view, there is substantial further work to be accomplished. The committee expects the USNRC and the nuclear industry to extend the work of criteria development beyond where this Phase 2 report leaves it. To guide further work, the committee's report offers findings and recommendations in four broad categories: (a) current practice that is essentially satisfactory or requires some fine tuning, (b) points of weakness in the USNRC's approach, (c) issues that merit further inquiry and research before satisfactory regulatory criteria can be developed, and (d) criteria and guidelines that are unreasonable to expect in the near future. KEY ISSUES Digital instrumentation and control systems for nuclear power plants have technological characteristics—equipment, response time, input and output range, and accuracy—very similar to those of digital instrumentation and control systems for other safety-critical applications such as chemical plants and aircraft. What distinguishes digital I&C applications in nuclear power plants from other digital I&C applications is the need to establish very high levels of reliability and safety under a wide range of conditions. Because of the potentially far greater consequences of accidents in nuclear power plants, the I&C systems must be relied upon to reduce the likelihood of even low-probability events. The USNRC has developed a regulatory process with the goal of achieving these high levels of reliability and thus assuring public safety. This process is subject to public scrutiny. Developing the Key Issues (Phase 1) In Phase 1 of the study, the committee identified eight key issues associated with the use of digital I&C systems in existing and advanced nuclear power plants. In the committee's view, these issues need to be addressed and a working consensus needs to be established regarding these issues among designers, operators and those responsible for 3   Licensing of any systems for use in a nuclear power plant is governed by formal, documented criteria that the USNRC and the regulated industry use to implement changes to a nuclear power plant. General criteria, applicable to either digital or analog I&C systems in nuclear plants, are contained in the Code of Federal Regulations, Part 50, Appendix A. This very general guidance is supplemented by more specific guidance in various forms such as "regulatory guides" that endorse industry standards or interpret USNRC regulations. To date, the more specific regulatory criteria for digital I&C have largely been determined on a case-by-case basis rather than as generally applicable criteria.

OCR for page 1
maintenance of such systems, and regulators in the nuclear industry. The process the committee followed to identify these issues is discussed in the Phase 1 report and is only briefly summarized here. In essence, the committee considered the impact of digital I&C systems against a set of standard regulatory approaches to assessing and ensuring safety (defense-in-depth, safety margins, environmental qualification, quality assurance, and failure invulnerability). From this analysis, the committee identified a number of questions and issues. After extensive deliberations, the committee selected eight key issues. The eight issues can be separated into six technical issues and two strategic issues. The six technical issues are systems aspects of digital I&C technology, software quality assurance, common-mode software failure potential, safety and reliability assessment methods, human factors and human-machine interfaces, and dedication of commercial off-the-shelf hardware and software. The two strategic issues are the case-by-case licensing process and the adequacy of technical infrastructure (i.e., training, staffing, research plan). The committee recognizes that these are not the only issues and topics of concern and debate in this area. Nevertheless, the committee reaffirms its judgment, initially formed during Phase 1, that developing a consensus on these eight issues will be a major step forward and accelerate the appropriate use and licensing of digital I&C systems in nuclear power plants. Analyzing the Key Issues (Phase 2) In conducting Phase 2 of its study the committee employed a systematic process, which is reflected in the structure of most of the chapters in this report. The committee reviewed a large number of documents made available by the USNRC and variety of other sources. The committee also interviewed selected personnel from the USNRC, from the two advisory committees discussed above (ACRS, NSRRC), from the nuclear industry,4 and from other industries5 using digital systems in safety-critical applications. The committee also sought the view of individuals from academia and research organizations. In addition, the committee visited control room simulators, a nuclear plant, and a fossil-fueled power plant with extensive digital I&C systems. The committee also had frequent and detailed internal discussions, both face-to-face and via paper and electronic communications. The committee also brought to bear a wide range of experience in and knowledge of the field. Carrying Out the Charge The committee took seriously the charge that it identify criteria for review and acceptance of digital I&C technology and that it recommend guidelines for regulation and certification. In carrying out its charge, the committee recognized that: In order to develop useful guidance, only a limited number of issues could be dealt with in the relatively brief duration of the study. General, high level criteria would not be particularly useful. The final criteria are legally the USNRC's responsibility. Further, since the nuclear power industry is heavily regulated in the public interest, the licensing criteria should be forged in a detailed interaction among the regulators, the industry, and the public. The committee has a wide range of expertise and experience in digital systems and nuclear power plants but it is not a surrogate for this interaction among the stakeholders. Hence, the committee could serve by clearly delineating and defining issues and providing guidance for resolving these issues rather than developing specific licensing criteria. Accordingly, the committee selected eight issues for study and worked on those issues. These eight issues address the two major intertwined themes associated with the use of digital instrumentation and control in nuclear power plants. These are: Dealing with the specific characteristics of digital I&C technology as applied to nuclear power plants. Dealing with a technology that is more advanced than the one widely in use in the existing nuclear power plants. This technology is rapidly advancing at a rate and in directions largely uncontrolled by the nuclear industry but at the same time likely to have a significant impact on the operation and regulation of the nuclear industry. The technical issues the committee focuses on first in this report are primarily related to digital technology itself (Theme 1), while the strategic issues that follow are primarily related to the process of adopting advanced technology (Theme 2). The committee concentrated on reviewing the current approaches being taken by the nuclear industry and its regulators toward dealing with the selected key issues. The committee also tried to learn from the experience of the international nuclear industry as well as gather and evaluate information about how other safety-critical industries and their regulators dealt with these issues. Also, through the technical expertise and knowledge of its various members, the committee explored work done by the digital systems community at large, including both research activities and academic work. 4   These individuals were from the U.S. domestic industry and also from Japan, Canada, and the United Kingdom. The committee also reviewed literature on the French nuclear program. 5   These individuals were from the railroad, aerospace, defense, and medical products industry.

OCR for page 1
As the committee worked through the issues it discovered there is a major impediment to progress. This is the communication barriers that exist among the key technical communities and individuals involved. The basic reason for the communication difficulty is apparent. Work is simultaneously going on in many areas, each with its own technology, research focus, and agenda. Unfortunately, although many of these areas use common terms, these terms often have different meanings to different groups, resulting in either a lack of communication or very difficult communication. This is particularly troublesome for the nuclear power industry and its regulators, who are not dominant in this technology and must try to synthesize information and experience from a variety of sources and apply it in power plants where safety hazards must be dealt with in a rigorous way, under public scrutiny. In Chapter 11 the committee discusses this communication problem in more detail and provides suggestions for a way forward. Making substantial progress in this area should have a multiplicative effect as it eases the resolution of many specific technical and strategic issues. Overall, while there are important steps that remain to be taken by the USNRC and industry as addressed in this report, the committee found no insurmountable barriers to the use of digital instrumentation and control technology to nuclear power plants. The committee also believes that a forward-looking regulatory process with good and continuing regulations and industry communication and interaction will help. All participants must recognize that crisp, hard-edged criteria are particularly difficult to come by in this rapidly moving area and good practices and engineering judgment will continue to be needed and relied upon. For the key technical issues (systems aspects of digital I&C technology; software quality assurance; common-mode software failure potential; safety and reliability assessment methods; human factors and human-machine interfaces; and dedication of commercial off-the-shelf hardware and software) the committee provides specific recommendations and conclusions which include a number of specific criteria. These are listed in each chapter (see Chapters 3 through 8). But recognizing the difficulty of defining specific criteria, and the need for the nuclear technology stakeholders, particularly the USNRC, to make the final decisions, the committee focused on (a) providing process guidance both in developing guidelines and in the short-term acceptance of the new technology; (b) identifying promising approaches to developing criteria and suggestions for avoiding dead-ends; and (c) mechanics for improving communication and strengthening technical infrastructure. For the key strategic issues (the case-by-case licensing procedure and adequacy of the technical infrastructure) the committee: Emphasizes guidance to implement a generically applicable framework for regulation that follows current USNRC practice and draws a distinction between major and minor safety modifications. The committee also provides guidance for the evaluation and updating of this regulatory framework (see Chapter 9). Identifies a need to upgrade the current USNRC technical infrastructure and suggests specific research activities that will support the needed regulatory program and USNRC's research needs. The committee also suggests several improvements to the technical infrastructure to improve and maintain technical capabilities in this rapidly moving, technically challenging area. The results of this process are set forth below, where the committee introduces each of the key issues—first the technical, then the strategic—with an "issue statement" developed during Phase 1 of the study. Following each issue statement are the conclusions and recommendations formulated by the committee during Phase 2 of the study. TECHNICAL ISSUES Systems Aspects of Digital Instrumentation and Control Technology Issue Statement. Along with important benefits, digital I&C systems introduce potential new failure modes that can affect operations and margins of safety. Therefore, digital I&C systems require rigorous treatment of the systems aspects of their design and implementation. What methods are needed to address this concern? How can the experience and best practices of the various technical communities involved in applying digital I&C technologies be best integrated and applied to nuclear power plants? What procedures can be put in place to update the methods and the experience base as new digital I&C technologies and equipment are introduced in the future? Conclusion 1. Continued effort is warranted by the USNRC and the nuclear industry to deal with the systems aspects of digital I&C in nuclear power plants. Conclusion 2. The lack of actual design and implementation of large I&C systems for U.S. nuclear power plants makes it difficult to use learning from experience as a basis for improving how the nuclear industry and the USNRC deal with systems aspects. Conclusion 3. The USNRC's intent to upgrade their regulatory guidance in the systems aspects of digital I&C applications in nuclear power plants is entirely supported by the committee's observations about systems aspects. Conclusion 4. Existing regulatory guidance lacks the specificity needed to be effective, and the revision should address this shortcoming. Recommendation 1. The USNRC should make a trial application of the proposed regulatory guidance documents on systems aspects to foreign nuclear plant digital systems, both

OCR for page 1
existing and in progress. In particular, this review should focus on assessing whether or not the revised guidance documents have the necessary level of specificity to adequately address the systems aspects of nuclear plant digital I&C implementations. Recommendation 2. The USNRC should identify and review systems aspects guidance documents provided in other industries, such as chemical processing and aerospace, where large-scale digital I&C systems are used. The focus of this review would be to compare these other guidance documents with those being developed by the USNRC, paying due attention to common problems and application-specific differences. Recommendation 3. To obtain practical experience, the USNRC should loan staff personnel, perhaps on a reciprocal basis, to other agencies involved in regulating or overseeing large safety-critical digital I&C systems. Recommendation 4. The USNRC should require continuing professional training for appropriate staff in technologies particularly germane to systems aspects, such as fault-tolerant, distributed systems. Software Quality Assurance Issue Statement. The use of software is a principal difference between digital and analog I&C systems. Quality of software is measured in terms of its ability to perform its intended functions. This, in turn, is traced to software specifications and compliance with these specifications. Neither of the classic approaches of (a) controlling the software development process or (b) verifying the end-product appears to be fully satisfactory in assuring adequate quality of software, particularly for use with safety-critical systems. How can the USNRC and the nuclear industry define a generally accepted, technically sound solution to specifying, producing, and controlling software needed in digital I&C systems? Conclusion 1. Software quality assurance procedures typically monitor process compliance rather than product quality. In particular, there are no generally accepted evaluation criteria for safety-related software; rather, standards and guidelines help to repeat best practices. Because most software qualities related to system safety, e.g., maintainability, correctness, and security, cannot be measured directly, it must be assumed that a relationship exists between measurable variables and the qualities to be ensured. To deal with this limitation, care must be taken to validate such models, e.g., using past development activities, and to assure that the measurements being made are appropriate and accurate in assessing the desired software qualities. Conclusion 2. Prior operating experience with particular software does not necessarily ensure reliability or safety properties in a new application. Additional reviews, analysis, or testing by a utility or third-party dedicator may be necessary to reach an adequate level of assurance. Conclusion 3. Testing must not be the sole quality assurance technique. In general, it is not feasible to assure software correctness through exhaustive testing for most real, practical I&C systems. Conclusion 4. USNRC staff reviews of the verification and validation process used during software development seem quite thorough. Conclusion 5. Exposing software flaws, demonstrating reliable behavior of software, and finding unintended functionality and flaws in requirements are different concepts and should be assessed by a combination of techniques including: Systematic inspections of software and planned testing with representative inputs from different parts of the systems domain can help determine if flaws exist in the software. Functional tests can be chosen to expose errors in normal and boundary cases, and measures of test coverage can be reported for them. Testing based on large numbers of inputs randomly selected from the operational profiles of a program can be used to assess the likelihood that software will fail under specific operating conditions. Requirements inspections can be an effective method for detecting software defects, provided requirements are studied by several experienced people who did not participate in their construction. The effectiveness of these reviews also depends on the quality of the requirements. A system-level hazard analysis can identify states that, combined with environmental conditions, can lead to accidents. The analysis should extend into software components to ensure that software does not contribute to system hazards. Conclusion 6. The USNRC research programs related to software quality assurance appear to be skewed toward investigating code-level issues, e.g., coding in different languages to achieve diversity and program slicing to identify threads containing common code. Conclusion 7. Rigorous configuration management must be used to assure that changes are correctly designed and implemented and that relationships between different software artifacts are maintained. Conclusion 8. Software is not more testable simply because the design has been implemented on a chip. Use of any technology requiring equivalent design effort to software requires commensurate quality assurance. For example, this conclusion applies to ASIC (application-specific integrated circuit), PLC (programmable logic controllers), and FPGA

OCR for page 1
(field programmable gate arrays). However, the committee notes that these technologies may be useful in addressing some configuration management problems. Recommendation 1. Currently, the USNRC's path is to develop regulatory guides to endorse (with possible exceptions) a variety of industry standards. The USNRC should develop its own guidelines for software quality assurance that focus on acceptance criteria rather than prescriptive solutions. The draft regulatory guide, Software in Protection and Control Systems, by Canada's Atomic Energy Control Board is an example of this type of approach. The USNRC guidelines should be subjected to a broad-based, external peer review process including (a) the nuclear industry, (b) other safety-critical industries, and (c) both the commercial and academic software communities. Recommendation 2. Systems requirements should be written in a language with a precise meaning so that general properties like consistency and completeness, as well as application-specific properties, can be analyzed. Cognizant personnel such as plant engineers, regulators, system architects, and software developers should be able to understand the language. Recommendation 3. USNRC research in the software quality assurance area should be balanced in emphasis between early phases of the software life cycle and code-level issues. Experience shows that the early phases contribute more frequently to the generation of software errors. Recommendation 4. The USNRC should require a commensurate quality assurance process for ASICs, PLCs, and other similar technologies. Common-Mode Software Failure Potential Issue Statement. Digital technology introduces a possibility that common-mode software failures may cause redundant safety systems to fail in such a way that there is a loss of safety function. Various procedures have been developed and evolved for evaluating common-mode failure potential in analog devices. Do these same procedures apply to computers and software or are different approaches to ensuring reliability needed? What does software diversity mean? Can it be achieved and assessed and, if so, how? Do techniques exist for assessing common-cause failure and common-mode failure when computers are involved? What are the implications of common-mode software failure for the licensing process and the use of component diversity? Are redundancy and diversity the most effective way to achieve reliability for digital systems? Conclusion 1. The USNRC position of assuming that common-mode software failure could occur is credible, conforms to engineering practice, and should be retained. Conclusion 2. The USNRC position with respect to diversity, as stated in the draft branch technical position, Digital Instrumentation and Control Systems in Advanced Plants, and its counterpart for existing plants, is appropriate. Conclusion 3. The USNRC guidelines on assessing whether adequate diversity exists need to be reconsidered. With regard to these guidelines: (a) The committee agrees that providing digital systems (components) that perform different functions is a potentially effective means of achieving diversity. Analysis of software functional diversity showing that independence is maintained at the system level and no new failure modes have been introduced by the use of digital technology is no different from that for upgrades or designs that include analog instrumentation. (b) The committee considers that the use of different hardware or real-time operating systems is potentially effective in achieving diversity provided functional diversity has been demonstrated. With regard to real-time operating systems, this applies only to operating systems developed by different companies or shown to be functionally diverse. (c) The committee does not agree that use of different programming languages, different design approaches meeting the same functional requirements, different design teams, or different vendors' equipment used to perform the same function is likely to be effective in achieving diversity. That is, none of these methods is a proof of independence of failures. Conversely, neither is the presence of these proof of dependence of failures. Conclusion 4. There appears to be no generally applicable, effective way to evaluate diversity between two pieces of software performing the same function. Superficial or surface (syntactic) differences do not imply failure independence, nor does the use of different algorithms to achieve the same functions. Therefore, funding research to try to evaluate design diversity does not appear to be a reasonable use of USNRC research funds. Conclusion 5. Although many in the software community believe that there are more cost-effective techniques for achieving high software reliability than redundancy and diversity, there is no agreement as to what these alternatives may be. The most promising of these appear to be the extension of standard safety analysis and design techniques to software and the use of formal (mathematical) analysis. Conclusion 6. The use of self-checking to detect hardware failures and some simple software errors is effective and should be incorporated. However, care must be taken to assure that the self-checking features themselves do not introduce errors. Recommendation 1. The USNRC should retain its position of assuming that common-mode software failure is credible. Recommendation 2. The USNRC should maintain its basic position regarding the need for diversity in digital I&C systems as stated in the draft branch technical position, Digital

OCR for page 1
Instrumentation and Control Systems in Advanced Plants (see Chapter 5), and its counterpart for existing plants. Recommendation 3. The USNRC should revisit its guidelines on assessing whether adequate diversity exists. The USNRC should not place reliance on different programming languages, different design approaches meeting the same functional requirements, different design teams, or using different vendors' equipment ("nameplate" diversity). Rather, the USNRC should emphasize potentially more robust techniques such as the use of functional diversity, different hardware, and different real-time operating systems. Recommendation 4. The USNRC should reconsider the use of research funding to try to establish diversity between two pieces of software performing the same function. This does not appear to be possible. Specifically, it appears the USNRC funding of the Unravel tool is based on the use of this tool for this purpose and, as such, is unlikely to be useful. Safety and Reliability Assessment Methods Issue Statement. Effective, efficient methods are needed to assess the safety and reliability of digital I&C systems in nuclear power plants. These methods are needed to help avoid potentially unsafe or unreliable applications and aid in identifying and accepting safety-enhancing and reliability-enhancing applications. What methods should be used for making these safety and reliability assessments of digital I&C systems? Conclusion 1. Deterministic assessment methodologies, including design basis accident analysis, hazard analysis, and other formal analysis procedures, are applicable to digital systems. Conclusion 2. There is controversy within the software engineering community as to whether an accurate failure probability can be assessed for software or even whether software fails randomly (see Chapter 6). However, the committee agreed that a software failure probability can be used for the purposes of performing probabilistic risk assessment (PRA) in order to determine the relative influence of digital system failure on the overall system. Explicitly including system failure on the overall system. Explicitly including software failures in a PRA for a nuclear power plant is preferable to the alternative of ignoring software failures. Conclusion 3. The assignment of probabilities of failure for software (and more generally for digital systems) is not substantially different from the handling of many of the probabilities for rare events. A good software quality assurance methodology is a prerequisite to providing a basis for the generation of bounded estimates for software failure probability. Within the PRA, uncertainty and sensitivity analysis can help the analyst assure that the results are not unduly dependent on parameters that are uncertain. As in other PRA computations, bounded estimates for software failure probabilities can be obtained by processes that include valid random testing and expert judgment.6 Conclusion 4. Probabilistic analysis is theoretically applicable in the same manner to commercial off-the-shelf (COTS) equipment, but the practical application may be difficult. The difficulty arises when attempting to use field experience to assess a failure probability, in that the experience may or may not be equivalent. For programmable devices, the software failure probability may be unique for each application. However, a set of rigorous tests may still be applicable to bounding the failure probability, as with custom systems. A long history of successful field experience may be useful in eliciting expert judgment. Recommendation 1. The USNRC should require that the relative influence of software failure on system reliability be included in PRAs for systems that include digital components. Recommendation 2. The USNRC should strive to develop methods for estimating the failure probabilities of digital systems, including COTS, for use in probabilistic risk assessment. These methods should include acceptance criteria, guidelines and limitations for use, and any needed rationale and justification. Recommendation 3. The USNRC and industry should evaluate their capabilities and develop a sufficient level of expertise to understand the requirements for gaining confidence in digital implementations of system functions and the limitations of quantitative assessment. Recommendation 4. The USNRC should consider support of programs that are aimed at developing advanced techniques for analysis of digital systems that might be used to increase confidence and reduce uncertainty in quantitative assessments. Human Factors and Human-Machine Interfaces Issue Statement. At this time, there does not seem to be an agreed-upon, effective methodology for designers, owner-operators, maintainers, and regulators to assess the overall impact of computer-based, human-machine interfaces on human performance in nuclear power plants. What methodology and approach should be used to assure proper consideration of human factors and human-machine interfaces? Conclusion 1. Digital technology offers the potential to enhance the human-machine interface and thus overall operator performance. Human factors and human-machine interfaces are well enough understood that they do not represent a major barrier to the use of digital I&C systems in nuclear power plants. 6   Committee member Nancy Leveson did not concur with this conclusion.

OCR for page 1
Conclusion 2. The methodology and approach adopted by the USNRC for reviewing human factors and human-machine interfaces provides an initial and acceptable first step in a review. Existing USNRC procedures, for both the design product and process, are consistent with those of other industries. The guidelines are based on many already available in the literature or developed by specific industries. The methodology for reviewing the design process is based on sound system engineering principles consistent with the validation and verification of effective human factors. Conclusion 3. Adequate design must go beyond guidelines. The discussion in NUREG-0711 on advanced technology and human performance and the design principles set out in Appendix A of NUREG-0700 Rev. 1 provide a framework within which the nuclear industry can specify, prototype, and empirically evaluate a proposed design. Demonstration that a design adheres to general principles of good human-system integration and takes into account known characteristics of human performance provides a viable framework in which implementation of somewhat intangible, but important, concepts can be assessed. Conclusion 4. There is a wide range in the type and magnitude of the digital upgrades that can be made to safety and safety-related systems. It is important for the magnitude of the human factors review and evaluation to be commensurate with the magnitude of the change. Any change, however, that affects what information the operator sees or the system's response to a control input must be empirically evaluated to ensure that the new design does not compromise human-system interaction effectiveness. Conclusion 5. The USNRC is not sufficiently active in the public human factors forum. For example, proposed human factors procedures and policies or sponsored research, such as NUREG-0700 Rev. 1, are not regularly presented and reviewed by the more general national and international human factors communities, including such organizations as the U.S. Human Factors and Ergonomics Society, Institute of Electrical and Electronics Engineers (IEEE), Society on Systems, Man, and Cybernetics, and the Association of Computing Machinery Special Interest Group on Computer-Human Interaction. European nuclear human factors researchers have used nuclear power plant human factors research to further a better understanding of human performance issues in both nuclear power plants and other safety-critical industries. Other safety-critical U.S. industries, such as space, aviation, and defense, participate actively, benefiting from the review and experience of others. Recommendation 1. The USNRC should continue to use, where appropriate, review guidelines for both the design product and process. Care should be taken to update these guidelines as knowledge and conventional wisdom evolve—in both nuclear and nonnuclear applications. Recommendation 2. The USNRC should assure that its reviews are not limited to guidelines or checklists. Designs should be assessed with respect to (a) the operator models that underlie the them, (b) ways in which the designs address classic human-system interaction design problems, and (c) performance-based evaluations. Moreover, evaluations must use representative tasks, actual system dynamics, and real operators. Recommendation 3. The USNRC should expand its review criteria to include a catalog or listing of classic human-machine interaction deficiencies that recur in many safety-critical applications. Understanding the problems and proposed solutions in other industries is a cost-effective way to avoid repeating the mistakes of others as digital technology is introduced into safety and safety-related nuclear systems. Recommendation 4. Complementing Recommendation 2, although human factors reviews should be undertaken seriously, e.g., in a performance-based manner with realistic conditions and operators, the magnitude and range of the review should be commensurate with the nature and magnitude of the digital change. Recommendation 5. The USNRC and the nuclear industry at large should regularly participate in the public forum. As noted in NUREG-0711, advanced human interface technologies potentially introduce many new, and as yet unresolved, human factors issues. It is crucial that the USNRC stay abreast of current research and best practices in other industries, and contribute findings from its own applications to the research and practitioner communities at large—for both review and education. (See also Technical Infrastructure chapter for additional discussion.) Recommendation 6. The USNRC should encourage researchers with the Halden Reactor Project to actively participate in the international research forum to both share their results and learn from the efforts of others. Recommendation 7. As funds are available, the USNRC's Office of Nuclear Regulatory Research should support research exploring higher-level issues of human-system integration, control, and automation. Such research should include exploration, specifically for nuclear power plant applications, of design methods, such as operator models, for more effectively specifying a design. Moreover, extensive field studies should be conducted to identify nuclear-specific technology problems and to compare and contrast the experiences in nuclear application with those of other safety-critical industries. Such research will add to the catalog of recurring deficiencies and potentially link them to proposed solutions. Recommendation 8. Complementing its own research projects, the USNRC should consider coordinating a facility, perhaps with the U.S. Department of Energy, in which U.S. nuclear industries can prototype and empirically evaluate proposed designs. Inexpensive workstation technologies permit the development of high-fidelity workstation-based

OCR for page 1
simulators of significant portions of control rooms. Other industries make extensive use of workstation-based part-task simulators (e.g., aviation); results are found to scale quite well to the systems as a whole. Dedication of Commercial Off-the-Shelf Hardware and Software Issue Statement. What methods should be agreed upon by the regulators and the licensees to evaluate and accept the use of commercial off-the-shelf digital I&C systems in safety applications in nuclear power plants? Conclusion 1. Use of COTS hardware and software is an attractive possibility for the nuclear industry to pursue, provided that a technically adequate dedication process can be formulated and that this process does not negate the cost advantages of COTS. Conclusion 2. The recently developed draft guideline of the Electric Power Research Institute (EPRI) working group, Guideline on Evaluation and Acceptance of Commercial Grade Digital Equipment for Nuclear Safety Applications, appears to have potential as the basis for reaching industry and USNRC consensus on the COTS issue. In view of this possibility, the committee notes that the guideline and the follow-on (second-tier) guidance should assure that the necessary and sufficient attributes of digital I&C application are defined for both hardware and software. Once these attributes are well-defined, various acceptable methods of assessing the validity of the attributes can be more readily ascertained and used and the requisite experience gained. As an example of the type of approach the committee considers appropriate, the EPRI working group and the USNRC staff should consider the FAA's DO-178B guideline for digital avionics, Software Considerations in Airborne Systems and Equipment Certification, which includes guidance on COTS. Conclusion 3. Software quality assurance and safety and reliability assessment methods are strongly related to COTS. The committee's conclusions in Chapters 4 and 6, respectively, should therefore also be considered. Dedication processes for COTS should also prove relevant in cases where standardized software is reused among similar nuclear applications. Conclusion 4. The USNRC involvement in the EPRI, Nuclear Utilities Software Management Group (NUSMG), IEEE, and International Society for Measurement and Control (ISA) working groups is very useful and should aid the USNRC in developing specific guidance to address the COTS issue. Conclusion 5. The approach to COTS must apply criteria and verification activities commensurate with the safety significance and complexity of a specific application. For example, the level of verification activities applied to small-scale replacements of recorders and indicators would not be the same as that applied to large-scale replacements of reactor protection systems. Recommendation 1. The USNRC staff should assure that their involvement in the ERPI, NUSMG, IEEE, and ISA working groups means that USNRC concerns and positions are being addressed so that any standards or guidelines developed by these groups can be quickly accepted and endorsed by the USNRC. Recommendation 2. The USNRC should establish what research is needed to support USNRC acceptance of COTS in safety applications in nuclear plants. This research should then be incorporated into the overall research plan. Recommendation 3. The USNRC regulatory guidance on the use of COTS should recognize and be based on the principle that criteria and verification activities are to be commensurate with the safety significance and complexity of the specific application. STRATEGIC ISSUES Case-by-Case Licensing Process Issue Statement. What changes should be considered in the regulatory process to provide more efficient and effective regulation of digital I&C systems in nuclear power plants? How can sufficient flexibility be incorporated to address the rapidly changing nature of the digital I&C technology and better match the time response of the regulatory process to the technology it controls? How can the regulatory process be made more efficient while maintaining its technical integrity? Conclusion 1. As a general observation, the role of the regulator in overseeing the implementation of digital upgrades can be a valuable and important one. Particularly in an area such as digital I&C systems, where the state of the art evolves rapidly and where first-of-a-kind nuclear applications are contemplated, the oversight role of the regulator can bring valuable insights to the implementation of such upgrades. Indeed, the committee found several specific examples of this happening. Conclusion 2. Nevertheless, the committee found that the regulatory response to the development and implementation of digital I&C upgrades in nuclear plants has proceeded in a manner that resulted in some degree of confusion and uncertainty within the licensee community with regard to the applicable regulatory requirements and the procedural framework for implementing such upgrades. This uncertainty and the resultant incremental cost has been a major contributor to the reluctance on the part of utilities in proceeding with digital upgrades. Conclusion 3. The lack of generically applicable regulatory requirements for digital upgrades has resulted in a case-by-case approach that has contributed to the confusion and uncertainty. This approach to reviews may have been necessary in the early phase of the transition to digital systems.

OCR for page 1
But the USNRC now has a sufficient body of experience with safety-related digital upgrades, gained over recent years and supplemented by the extensive experience of other countries and other industries, to enable the agency to establish a generically applicable regulatory regime that would govern the review and approval of such upgrades. Conclusion 4. The process established in 10 CFR 50.59, wherein the agency has defined those circumstances where a licensee may make a modification without prior USNRC review and approval, is fundamentally sound, necessary, and consistent with the USNRC's responsibility to protect the public health and safety. In particular, it recognizes the practical necessity for licensees to make facility modifications consistent with their facility licensing basis, without the need for prior USNRC review and approval. Moreover, the process appropriately reflects the gradation of significance in changes that might be made in a nuclear plant and the USNRC's attendant role based upon these gradations. In this regard, the committee strongly believes that it is important for the USNRC to distinguish between digital upgrades that are significant (i.e., pose unreviewed safety questions) and those that are not, and tailor the scope and depth of the regulatory review in a manner that is commensurate with this gradation. Conclusion 5. The committee believes that defining all safety-related digital upgrades as resulting in an unreviewed safety question, as stated in the USNRC's draft generic letter of August 1992, is contrary to both the letter and spirit of 10 CFR 50.59. Conclusion 6. The agency has no formal process for cataloguing determinations made under 10 CFR 50.59 with regard to digital upgrades and the bases for these determinations. Such information would assist both the USNRC and the utilities in determining whether particular upgrades pose unreviewed safety questions. Conclusion 7. Early interaction between a utility applicant and the USNRC can be extremely helpful in identifying and fleshing out important issues. Where this proactive interaction has occurred, the committee found that the subsequent regulatory review was more efficient and focused, minimizing resources that would otherwise be required on the part of both the utility and the USNRC. Recommendation 1. The USNRC should place a high priority on its effort to develop a generically applicable framework for the review and evaluation of digital I&C upgrades for operating reactors. Recommendation 2. In view of the rapid evolution of digital technology, a process should be established to ensure that the regulatory framework is updated to stay abreast of new developments. To ensure that this framework takes into account the best practices in other safety-critical industries, external and public review is highly desirable. Recommendation 3. The USNRC should consider additional ways in which the guideline development process can be accelerated and streamlined. For example, consideration could be given to establishing chartered task groups involving representatives from the USNRC, the industry, and academia. These groups would be tasked and managed on a project basis to investigate and resolve unreviewed matters of possible safety significance that arise in the development and use of digital systems. Recommendation 4. In developing its regulatory requirements, the USNRC should ensure that where issues arise that are unique to digital systems, they are treated appropriately. On the other hand, where issues arise with regard to digital upgrades that are no different from issues posed for analog systems, such issues should be treated consistently. The opportunity (or obligation) for the USNRC to review and approve digital upgrades should not be seen as an opportunity to impose new requirements on individual licensees unless the issue is unique to the application proposed. Recommendation 5. In view of the substantial benefits of early interaction with individual utilities considering digital upgrades, as well as the benefit of working closely with industry groups and other interested members of the public in the development of standards and guidelines, the USNRC should undertake proactive efforts to interact early and frequently with individual utilities and with industry groups and other interested members of the public. In addition, it would be of benefit for the USNRC to be familiar with the broader evolving applications of digital I&C systems in both nuclear and nonnuclear applications. This, in turn, will provide a foundation for a cooperative working relationship. Recommendation 6. The USNRC should revisit the "systems level" issue addressed in Generic Letter 95-02 and EPRI Report TR-102348 to ensure that this position is consistent with the historical interpretation of 10 CFR 50.59. The committee strongly endorses maintaining and formalizing the distinction between major and minor safety system upgrades containing digital technology. Recommendation 7. The USNRC should establish a process for cataloguing 50.59 evaluations of digital upgrades in some centralized fashion, so that individual utilities considering such upgrades can review and consider past 50.59 determinations regarding when a particular modification has been found to result in an unreviewed safety question. Adequacy of Technical Infrastructure Issue Statement. Does the USNRC need to make changes in its staffing, training, and research program to support its regulation of digital I&C technology in nuclear power plants? If so, what is the appropriate program for the USNRC? How should this program be structured so that it

OCR for page 1
maintains its effectiveness in the face of rapidly moving and developing technology and generally declining budgets? Conclusion 1. The USNRC should make changes in its staffing, training, and research program to support its regulation of digital I&C technology in nuclear power plants. Specific recommendations are provided below. Conclusion 2. The issue of adequate technical infrastructure is applicable not only to the USNRC but also to the nuclear industry as a whole. Many of the committee's recommendations for the USNRC have parallel applications to the nuclear industry. Conclusion 3. The USNRC must anticipate that the regulatory technical infrastructure will continue to be challenged by advancing digital I&C technology. The focus of the near-term licensing effort will be on digital upgrades and certification of the advanced plants. The USNRC will have to continue to expand its technical infrastructure as use of digital technology expands and its sophistication increases. Conclusion 4. There are problems inherent in the historical process for developing standards and industry guidelines, particularly those applied to the rapidly advancing digital technology. Pending development of alternate approaches, early involvement by the USNRC in developing standards and industry guidelines will foster more timely availability of regulatory guidance and acceptance criteria. Conclusion 5. A strategic plan is needed for the USNRC research program on digital I&C applications. The current research program is a disjointed collection of studies lacking an underlying strategy and in some specific cases pursuing topics of questionable worth. The staff structure of the USNRC, which separates the staff of the staff of the Office of Nuclear Reactor Regulation (NRR) from the staff of the Office of Nuclear Regulatory Research (RES) and mandates that the RES staff respond to NRR ''user needs," may be an obstacle to development of a coherent plan that balances near-term regulatory decision making and long-term research into problems on the horizon. Periodic outside review of the USNRC research program could help assure that the right issues are being addressed and could also lead to areas of collaborative research. The committee is aware of and notes favorably the impact of the existing Nuclear Safety Research Review Committee. However, a more formal, outside review would be useful. Perhaps this could be done on an exchange basis with other agencies to reduce resource demands. Recommendation 1. Despite difficulties posed by declining budget and staffing levels in the face of rapidly moving technology and a stagnating nuclear industry, the USNRC must explore ways to improve the efficiency of the review process with existing staff and resources. Recommendation 2. The USNRC should define a set of minimal and continuing training needs for existing and recruited staff. Particular attention should be paid to software quality assurance expertise. Once defined, the USNRC training program should be subjected to appropriate external review. Certification of USNRC expertise levels is one possibility the USNRC may wish to consider. Recommendation 3. Consistent with Conclusion 5 above, the USNRC should develop a strategic plan for the research program conducted by the RES and NRR offices. The plan should emphasize balancing short-term regulatory needs and long-term, anticipatory research needs and should incorporate means of leveraging available resources to accomplish both sets of research objectives. It should also reach out more effectively to relevant technical communities (e.g., by the establishment of research simulators for human factors research), to the Electric Power Research Institute, to the Department of Energy, to foreign nuclear organizations, and to other safety-critical industries dealing with digital I&C issues. In making this recommendation, the committee recognizes the Halden Reactor Project provides an example of such cooperative research; but much of the Halden work cannot be published widely and therefore lacks the benefit of rigorous peer scrutiny. Recommendation 4. Because research in the digital I&C area may require a longer time frame than that of single fiscal years, the USNRC should give consideration to planning and arranging funding on a multiyear basis. Recommendation 5. Consistent with Conclusion 4 above, the USNRC should consider ways to accelerate preparation and updating of needed standards and guidance documents. In particular, the USNRC should consider using chartered task groups (see Recommendation 3 pertaining to the case-by-case licensing process). CONCLUDING STATEMENT The committee has presented what it believes to be a pragmatic approach for meeting the challenge. One key obstacle is overcoming impediments to communication. There are a number of ways to address the communication difficulty. Some are already being pursued, some need to be initiated. The committee particularly emphasizes five areas of need: the need for better, clearer, crisper statements of the regulatory concern and the appropriate acceptance criteria that are valid at any point in time the need for the nuclear power industry and the USNRC to be more proactive in the relevant technical communities the need for the nuclear power industry and its regulator to strengthen its technical infrastructure in digital systems the need to formally address the communication problem in a systematic way

OCR for page 1
the need to tune up the regulatory mechanisms that are employed when an advanced technology, like digital I&C, has temporarily outpaced the regulations Turning to high-level issues more specifically related to digital technology, the committee emphasizes the following: The use of digital I&C technology does not obviate the standard methods for safety assessments of nuclear power plants. Digital I&C systems (and digital systems in general) should not be addressed only in terms of hardware or software. Most practical digital I&C systems cannot be exhaustively tested and therefore cannot be shown to be free from any and all errors. In summary, the committee notes that digital instrumentation and control is state-of-the-art technology and is widely used both inside and outside the nuclear industry. Digital I&C systems offer powerful capabilities that can, however, affect nuclear power plant safety; therefore, digital systems should be treated carefully, particularly in safety-critical applications. It appears the USNRC and the nuclear power industry are moving forward with procedures, processes, and technical infrastructure needed to assure continued safe operation of the plants. The committee has suggested several improvements.