National Academies Press: OpenBook
« Previous: Appendix E Glossary and Acronyms
Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

References

American Telephone & Telegraph 1994 Using ISO 9000 to Improve Business Processes. Murray Hill, N.J.: AT&T Corporate Quality Office.

Ascher. H., and H. Feingold 1984 Repairable Systems Reliability: Modeling, Inference, Misconceptions and Their Causes. New York: Marcel Dekker.

Atkinson. A.C. 1994 Fast very robust methods for the detection of multiple outliers. Journal of the American Statistical Association 89(428):1329-1339.


Bain, L.J., and M. Engelhardt 1991 Statistical Analysis of Reliability and Life Testing Models, Theory and Methods. Second edition. New York: Dekker.

Barlow, R.E., and F. Proschan 1975 Statistical Theory of Reliability and Life Testing: Probability Models. New York: Holt. Rinehart and Winston, Inc.

Blue Ribbon Defense Panel 1970 Report to the President and the Secretary of Defense on the Department of Defense: Appendix F-Staff Report on OT&E. Washington, D.C.: U.S. Department of Defense.

Box. G.E.P., and J.S. Hunter 1961 The 2K-p fractional factorial designs, Part 1. Technometrics 3(3):311-351.

Breitenberg. M. 1993 More Questions and Answers on the ISO 9000 Standard Series and Related Issues. Standards Code and Information Program, Office of Standards Services. Report #NISTIR 5122. Gaithersburg, MD: National Institute of Standards and Technology.

Bridgman, M.S.. and D.V. Glass 1992 Better Assessment of Operational Suitability. Volume 1: Final Report. McLean, VA: Logistics Management Institute.


Carlin, B.P., and T.A. Louis 1996 Bayes and Empirical Bayes Methods for Data Analysis. New York: Chapman and Hall.

Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

Chatfield, C. 1995 Model uncertainty, data mining and statistical inference. Journal of the Royal Statistical Society 158(3):419-444.

Citro, C.F., and E.A. Hanushek, eds. 1991 Improving Information for Social Policy Decisions: The Uses of Microsimulation Modeling. Volume I: Review and Recommendations, Volume II: The Uses of Microsimulation Modeling. Panel to Evaluate Microsimulation Models for Social Welfare Programs, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press.

Clemen, R.T. 1991 Making Hard Decisions: An Introduction to Decision Analysis. Boston: PWS Kent.

Cochran, W.G. 1977 Sampling Techniques. 3rd edition. New York: Wiley.

Cohen, M.L., J.E. Rolph, D.L. Steffey, eds. 1998 Statistics, Testing, and Defense Acquisition: Background Papers. Panel on Statistical Methods for Testing and Evaluating Defense Systems, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press.

Coleman, D.E., and D.C. Montgomery 1993 A systematic approach to planning for a designed industrial experiment. Technometrics 35(1):1-12.

Computer Sciences Corporation 1994 A Comparative Analysis of Tri-Service Accreditation Policies and Practices. Volume I of the Accreditation Requirements Study Report. Report JTCG/AS-93-SM-20. Camarillo, Calif.

Cook, R.D, 1994 Using dimension-reduction subspaces to identify important inputs in models of physical systems. American Statistical Association Proceedings of the Section on Physical and Engineering Sciences. Washington, D.C.: American Statistical Association.

Cox, D.R., and P.A.W. Lewis 1972 The Statistical Analysis of Series of Events. London: Chapman and Hall.

Currin, C., T. Mitchell, M. Morris, and D. Ylvisaker 1991 Bayesian prediction of deterministic functions, with applicability to use design and analysis of computer experiments. Journal of the American Statistical Association86:953-963.

Dalal, S.R., E.B. Fowlkes, and B. Hoadley 1989 Risk analysis of the space shuttle: Pre-Challenger prediction of failure. Journal of the American Statistical Association 84(408):945-957.

Daniel, C. 1976 Applications of Statistics to Industrial Experimentation. New York: Wiley.

Deming, W.E. 1986 Out of the Crisis. Cambridge, Mass.: Center for Advanced Engineering Study.

Draper, D. 1995 Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society Series B, 57(1):45-97.


Escobar, L., and W. Meeker 1994 Fisher information matrix for the extreme-value, normal and logistic distributions and censored data. Applied Statistics 43:533-540.


Friedman, J. 1991 Multivariate adaptive regression splines (with discussion). Annals of Statistics 19:1-141.

Fries, A. 1994a Design of experiments in operational test and evaluation: Where should we go next? The ITEA Journal of Test and Evaluation 14(4):20-33.

Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

1994b Recent Demonstrations of the Inherent Value of Operational Test and Evaluation (OT&E). Unpublished white paper prepared for the Director, Operational Test and Evaluation. Alexandria, Va.: The Institute for Defense Analyses.

Gaier, E.M., and R.C. Marshall 1998 Strategic information generation and transmission: The evolution of institutions in DoD operational testing. In M.L. Cohen, J.E. Rolph, and D.L. Steffey, eds., Statistics, Testing, and Defense Acquisition: Background Papers. Panel on Statistical Methods for Testing and Evaluating Defense Systems, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press.

Galway, L.A., and C.H. Hanks 1996 Data Quality Problems in Army Logistics: Classification, Examples, and Solutions. Santa Monica, Calif.: RAND.

Gaver, D.P., and I.G. O'Muircheartaigh 1987 Robust empirical Bayes analyses of event rates. Technometrics 29:1-15.

Gelman, A., J.B. Carlin, H.S. Stern, and D.B. Rubin 1995 Bayesian Data Analysis. New York: Chapman and Hall.

Giadrosich, D.L. 1990 Modeling and simulation in operational test and evaluation: A dinosaur's viewpoint. The ITEA Journal of Test and Evaluation XI(1):45-52.


Hahn, G.J. 1977 Some things engineers should know about experimental design. Journal of Quality Technology 9(1):13-20.

1984 Experimental design in the complex world. Technometrics 26(1):19-31.

Hall, M.J., L. Weidell, S.S. Nair, and M.O. Gonzales 1994 Test and Evaluation Plan for the Initial Operational Test and Evaluation of the Longbow Apache (AH-64D). Alexandria, Va.: United States Army.

Hampel. F., E.M. Ronchetti, P.J. Rousseuw, and W.A. Stabel 1986 Robust Statistics: The Approach Based on Influence Functions. New York: Wiley.

Hill, H.M. 1960 Experimental designs to adjust for time trends. Technometrics 2:67-82.

Hoadley, B. 1981 Quality management plan (QMP). Bell System Technical Journal 60:215-273.

Huber, P.J. 1981 Robust Statistics. New York: Wiley.


Iman, R.L., and W.J. Conover 1980 Small sample sensitivity analysis techniques for computer models, with an application to risk assessment. Communications in Statistics-Theory and Methods 9(17):1749-1842.

Ishikawa, K. 1985 What is Total Quality Control? The Japanese Way. Englewood Cliffs, NJ: Prentice Hall.


Kaminski, P.G. 1995 Reinventing DoD Test & Evaluation. Address to the International Test and Evaluation Association Symposium, Huntsville, Alabama, October.


Laird, M. 1972 Cost Estimating for Major Defense Systems. Memorandum from the Secretary of Defense for the Secretaries of the Military Departments .

Lall, P., M. Pecht, and M.J. Cushing 1994 A physics-of-failure (POF) approach to addressing device reliability in accelerated testing. Proceedings of the 5th European Symposium on Reliability of Electronic Devices. Failure Physics and Analysis. Glasgow, UK, October 4-7.

Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

Lawless, J.F. 1982 Statistical Models and Methods for Lifetime Data. New York: John Wiley and Sons.

Margolis, M.A. 1975 The CAIG: In pursuit of improved cost estimates. Defense Management Journal 11:2024.

McKay, M.D. 1995 Evaluating Prediction Uncertainty. Technical report (NUREG/CR-6311) prepared for the Division of Systems Technology of the Nuclear Regulatory Research, U.S. Nuclear Regulatory Commission. Los Alamos, NM: Los Alamos National Laboratory.

McKay. M.D. 1997 Non-parametric variance-based methods of assessing uncertainty importance. Reliability Engineering and System Safety 57:267-279.

McKay. M.D., W.J. Conover, and R.J. Beckman 1979 A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21:239-245.

Meeker. W.Q., and Escobar, L.A. 1993 A review of recent research and current issues in accelerated testing. International Statistical Review 6:147-161.

Meeker. W.Q., L.A. Escobar, and D.A. Hill 1992 Sample sizes for estimating the Weibull hazard function from censored samples. IEEE Transactions on Reliability 41(1):133-138.

Meeker. W.Q., and W. Nelson 1976 Weibull percentile estimates and confidence limits from singly censored data by maximum likelihood. IEEE Transactions on Reliability R-25(1):20-24.

1977 Weibull variances and confidence limits by maximum likelihood for singly censored data. Technometrics 19(4):473-476.

Mizuno, S.. ed. 1988 Management for Quality Improvement: The Seven New QC Tools. Portland, Ore.: Productivity Press.

Morgan, M.G., and M. Henrion 1990 Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge. Mass.: Cambridge University Press.

Morris, C.N. 1983 Journal of the American Statistical Association 78(381):47-65.

Morris, M.D. 1991 Factorial sampling plans for preliminary computational experiments. Technometrics 33(2):161-174.

Morris, M., T. Mitchell, and D. Ylvisaker 1993 Bayesian design and analysis of computer experiments. Technometrics 35(3):243-255.


Nair, V.N., D.A. James. W.K. Ehrhich, and J. Zevallos 1998 Statistical issues in assessing software testing strategies. Statistica Sinica 8(1):165-184.

National Research Council 1995 Statistical Methods for Testing and Evaluating Defense Systems: Interim Report. Panel on Statistical Methods for Testing and Evaluating Defense Systems, Committee on National Statistics. Washington, D.C.: National Academy Press.

1996 Statistical Software Engineering. Panel on Statistical Methods in Software Engineering, Committee on Applied and Theoretical Statistics. Washington. D.C.: National Academy Press.

Nelson, W. 1990 Accelerated Testing: Statistical Models, Test Plans and Data Analyses. New York: John Wiley and Sons.

Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

Nelson, W., and W.Q. Meeker 1978 Theory for optimum accelerated censored life tests for Weibull and extreme value distributions. Technometrics 20(2):171-177.

Oshana, R. 1997 Software testing with statistical usage based models. Embedded Systems Programming 101:40-55.

Owen, A.B. 1992 Randomized orthogonal arrays for computer experiments, integration, and visualization. Statistica Sinica 2:439-457.


Packard, D. 1971 Use of Parametric Cost Estimates. Memorandum from the Deputy Secretary of Defense for the Secretaries of Military Departments. December 7.

Pollock, S.M. 1997 A Bayesian Approach to "Dubin's Challenge": Optimally Selecting a Small Number of Operational Test Environments. Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor.

Poore, J.H., and C.J. Trammell 1996 Cleanroom Software Engineering: A Reader. Cambrdige, Mass.: Blackwell Publishers.

Proschan, F. 1963 Theoretical explanation of observed decreasing failure rate. Technometrics 5:375-383.


Rolph, J.E., and D.L. Steffey, eds. 1994 Statistical Issues in Defense Analysis and Testing: Summary of a Workshop. Committee on National Statistics and Committee on Applied and Theoretical Statistics, National Research Council. Washington, D.C.: National Academy Press.

Ross, S.M. 1996 Simulation: Statistical Modeling and Decision Science. San Diego, Calif.: Academic Press.


Sacks, J., W.J. Welch, T.J. Mitchell, and H.P. Wynn 1989 Design and analysis of computer experiments. Statistical Science 4(4):409-435.

Samaniego, F.J., and Y.S. Chong 1998 On the performance of Weibull life tests based on exponential life testing designs. In M.L. Cohen, J.E. Rolph, and D.L. Steffey, eds., Statistics, Testing, and Defense Acquisition: Background Papers. Panel on Statistical Methods for Testing and Evaluating Defense Systems, Committee on National Statistics, National Research Council. Washington, D.C.: National Academy Press.

Seglie, E. 1992 How Much Testing is Enough? Background paper prepared for The Workshop on Statistical Issues in Defense Analysis and Testing, Committee on National Statistics. National Research Council. September 24-25.

Sheedy, J., and M. Stolle 1996 Mathematical Modeling to Support Operational Test & Evaluation. Prepared for the Second Test & Evaluation International Aerospace Forum, The Royal Aeronautical Society, London, UK.

Steinberg, D., and W. Hunter 1984 Experimental design: Review and comment. Technometrics 26(2):71-130.


Taub, A.E., and M.A. Thomas 1983 Confidence Intervals for CEP When the Errors are Elliptical Normal. Dahlgren, Va.: Naval Surface Weapons Center.


U.S. Department of Defense 1960 Handbook H108: Sampling Procedures and Tables for Life and Reliability Testing (Based on the Exponential Distribution). New York: John Wiley and Sons.

Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×

1973 OSD Cost Analysis Improvement Group. DoD Directive No. 5000.4. June 13.

1981 Military Handbook 189, Reliability Growth Management. MIL-HDBK-189. Philadelphia: Naval Publications and Form Center.

1982 Test and Evaluation of System Reliability, Availability, and Maintainability: A Primer. Third edition, Report No. DoD 3235.1-H.

1992 COMPOTEV FOR Test Plant for Project 1376-OT-11C1.

1993 Longbow Apache Test and Evaluation Master Plan - Annex D: Mission Profiles and Operational Mode Summary for AH-64 Apache Longbow. Prepared by Scenarios and Simulations Branch.

1994a DoD Modeling and Simulation (M&S) Management. Directive No. 5000.59. Undersecretary of Defense for Acquisition.

1994b Reliability. Availability. and Maintainability (RAM) Rationale Report for the Longbow (AH-64D). Fort Rucker, Ala.: U.S. Army Aviation Center and Fort Rucker.

1995a Family of Medium Tactical Vehicles (FMTV). Office of the Director, Operational Test and Evaluation, The Pentagon, Washington, DC, 18 August. (unclassified version), [Online]. Available: http://www.dote.osd.mil/.

1995b Test and Evaluation Master Plan for the ATACMS/BAT. U.S. Army Operational Test and Evaluation Command.

1996 DoD 5000 Series. Washington, D.C.: U.S. Department of Defense.

1997 Director, Operational Test and Evaluation FY 1996 Annual Report. (unclassified version), [Online]. Available: http://www.dote.osd.mil/.

1998a Army Performance Improvement Criteria. Strategic Management and Innovations Division, Management Directorate. Headquarters, Department of the Army.

1998b Director, Operational Test and Evaluation FY 1997 Annual Report. (unclassified version), [Online]. Available: http://www.dote.osd.mil/reports/FY97/97tocmain.html

U.S. General Accounting Office 1986 Weapon Performance: Operational Test and Evaluation Can Contribute More to Decision making. Washington, D.C.: U.S. Government Printing Office.

1987a DoD Simulations: Improved Assessment Procedures Would Increase the Credibility of Results. Washington, D.C.: U.S. Government Printing Office.

1987b Testing Oversight. Washington, D.C.: U.S. Government Printing Office.

1996 Best Practices: Commercial Quality Assurance Practices Offer Improvements for DoD. Washington, D.C.: U.S. Government Printing Office.

1997 Test and Evaluation: Impact of DoD's Office of the Director of Operational Test and Evaluation. Washington, D.C.: U.S. Government Printing Office.

Vardeman, S. 1992 Response to How Much Testing is Enough? a background paper prepared for The Workshop on Statistical Issues in Defense Analysis and Testing, Committee on National Statistics, National Research Council. September 24-25.

von Winterfeldt, and W. Edwards 1986 Decision Analysis and Behavioral Research. Cambridge: Cambridge University Press.


Wiesenhahn, D.F., and R.D. Dighton 1993 A Framework for Using Advanced Distributed Simulation in Operational Test. Alexandria, Va.: The Institute for Defense Analyses.

Wright, S.J. 1993 Assessment of the Utility of Simulation Environments in Support of Operational Test & Evaluation. Briefing for Dr. William J. Perry, Deputy Secretary of Defense, September 30. U.S. Operational Test and Evaluation Command .


Zelen, M., and M. Dannemiller 1961 The robustness of life testing procedures derived from the exponential distribution. Technometrics 3:29-49.

Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 215
Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 216
Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 217
Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 218
Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 219
Suggested Citation:"References." National Research Council. 1998. Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements. Washington, DC: The National Academies Press. doi: 10.17226/6037.
×
Page 220
Next: Biographical Sketches of Panel Members and Staff »
Statistics, Testing, and Defense Acquisition: New Approaches and Methodological Improvements Get This Book
×
Buy Paperback | $57.00 Buy Ebook | $45.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign.

This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!