Review of the U.S. Department of Defense Air, Space, and Supporting Information Systems Science and Technology Program (2001)

This chapter addresses DoD’s efforts to develop information systems technology (IST) to support air and space operations, as well as other military operations. Information systems are becoming pervasive throughout the battlespace. Although some IST development projects are uniquely applicable to air and space operations, many overlap, perform the same functions, or rely increasingly on IST being developed for ground and sea operations. For example, advanced visualization capabilities for “seeing” the battlespace will be as important to the commander of air or space operations as to the commander of ground or sea operations. The same capability can be used across the whole IST spectrum.

The complex relationship of air, sea, and ground information systems made it difficult for the committee to identify unique air and space S&T on IST in the Army and Navy. The committee assumed that all of the Air Force’s IST S&T projects would support air and space operations and that IST development at DARPA aligned with the Air Force would primarily support air and space operations.

Visions for air and space operations are becoming increasingly dependent on information systems, yet Air Force IST S&T budgets continue to decline each year. Although projected Air Force IST S&T funding through FY05 appears to be increasing, the annual trend has been to disregard the investment strategy and reduce the “then-year” funding while proposing an increase in future funding (see Figure 4–1).

A common assumption is that commercial IST developments can reduce the impact of these reductions. This belief is based on industry’s huge investment in commercial information technology and the many hardware and software breakthroughs that have changed the way we do business, indeed, the way we live (Etter, 2000). However, leveraging these developments is not as easy as it appears. For example, commercial hardware is normally not rugged enough for military use, and software is not protected from determined adversaries or viruses (e.g., “Melissa” in 1999 and the “I Love You” virus in 2000). Commercial technologies can satisfy only some DoD needs, provide only near-term solutions, do not have stable investments in research and development (and have very little investment in long-range basic research), and may not be able to provide long-term support. Although DoD should leverage commercial technology where appropriate, it also needs to maintain a capability advantage over commercial technologies to counter the spread of commercial information technologies to all nations and interest groups. DARPA’s S&T efforts are intended to support the joint needs of the services, but inadequate service funding and differences between DARPA’s investment planning and execution processes and the services’ investment planning and execution processes have made it difficult to transition DARPA successes to the services.

Service visions are not as well coordinated as they could be. For example, they do not address the issue of interoperability with, and the leveraging of, air and space IST developments by NASA, the National Reconnaissance Office, and the Ballistic Missile Defense Organization. Interoperability is a huge issue that can-

FIGURE 4–1 Decline in Air Force Research Laboratory Information Systems Directorate S&T budget, FY96 to FY00, compared with President’s budget (PB) request. SOURCE: DSTAG, 2000a.

not be fully addressed with current levels of funding. Finally, each service would benefit from a high-level advocate who has an understanding of the state of the art of information systems and can formulate service S&T budgets, participate in corporate policy and decision-making activities, and work closely with other DoD and non-DoD organizations.

The DoD IST vision should be joint, should emphasize interoperability, and should include programs from NASA, the National Reconnaissance Office and BMDO. Additionally, there should be a high-level advocate in each service; increased, stable funding; a stronger basic research program; an applied research and technology development program that takes advantage of DARPA successes but does not depend solely on those efforts; and an S&T program supported with a very flexible budget to leverage the best commercial technology yet maintain a strong technological capability advantage over commercial capabilities to ensure military superiority.

For purposes of this study, IST is defined differently from commercial information technology. IST is one of the most important enabling technologies for DoD’s command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) military capability. IST is defined as technologies that support the collection of data; the processing of data (transforming data to information to knowledge); and the dissemination, protection, and presentation of data (tailoring information to meet user needs) to support battlefield information superiority, mission planning and rehearsal, training, and system acquisition. Technology areas (Box 4–1) include computing and software, seamless communications, decision making, information assurance, and modeling and simulation (DDR&E, 1996).

Information superiority is described by the Joint Chiefs of Staff as the “degree of dominance in the in-

formation domain that permits the conduct of operations without effective opposition” (DDR&E, 2000). Information superiority is a combination of command, control, communications, and computers (C4); intelligence, surveillance, and reconnaissance (ISR); and information operations (IO). Command and control (C2) is the “exercise of authority and direction by a properly designated commander over assigned forces in the accomplishment of the mission. Command and control functions are performed through an arrangement of personnel, equipment, communications, facilities, and procedures employed by a commander in planning, directing, coordinating, and controlling forces and operations in the accomplishment of the mission” (DDR&E, 2000). The ISR component of information superiority should provide near-real-time awareness of the location and activity of friendly, hostile, and neutral forces throughout the battlespace.

Because battlespace information systems are pervasive in all of the services and DoD agencies, visions for DoD information systems in support of air and space operations should be joint visions or describe how they would support joint efforts. Therefore, the investment strategies for IST developments should be joint strategies. In a recent DoD technology area review and assessment (TARA) of IST projects, the service and agency vision statements defined both joint and individual service needs, yet the development and execution of these visions did not appear to have joint cooperation. The following discussion describes IST needs based on various air and space visions (DSTAG, 2000a, 2000b, 2000c).

Future Air Force concepts are driven by information and information systems, which are becoming the

“force multiplier” for the Air Force of the future. The increased emphasis on information and associated trends are reflected in Air Force warfighting concepts. These concepts include (DSTAG, 2000a):

• dynamic aerospace command

• joint battlespace infosphere

• information operations

• integrated aerospace operations

• the Expeditionary Aerospace Force

• effects-based operations

More detailed listings, definitions, and associated IST drivers can be found in Appendix E.

The Navy’s visions for information systems are similar to those of the Air Force. The evolutionary concept of network-centric warfare, the most well defined vision, depends increasingly on network nodes (users) that can communicate with each other. Most naval nodes, including aircraft, are currently not capable of participating in network-centric warfare because of inadequate communications capability. This concept, as well as the Navy initiative called Information Technology for the 21^st Century, depends heavily on satellite communications that are not currently available according to the Military Satellite Communications Emerging Requirements Database, the military strategic and tactical relay (MILSTAR) satellite program operational requirements document (ORD), the “Tactical Ballistic Missile Defense ORD,” and other Navy documents. The needs of naval aviators are similar to those of Air Force aviators; both depend on intelligence obtained through air and space assets. The Navy’s visions for other information systems are also similar to those of the Air Force. These include decision-making tools, information assurance technologies, seamless communications, computing and software, modeling and simulation, and sensors (DSTAG, 2000c).

The Army’s visions also depend increasingly on information systems. The “digitized force” is based on advancements in IST. Recent visions of lighter, air-transportable, strategic forces (the Future Combat Systems concept) promote the dependency on “information, not iron” as a means of protecting the force. The strategic use of forces will require long-range satellite communications to support dispersed command groups and provide “reach-back capability” for decision-support tools and logistics. The Army’s air and space IST requirements for its aviation and intelligence components are similar to those of the Air Force and the Navy (DSTAG, 2000b).

An analysis of joint-service and individual-service visions by the DoD Project Reliance IST technology area panel derived a list of key requirements that IST S&T must address (see Box 4–2).

The purpose of DoD’s current information systems program for air and space operations is to provide global awareness, intelligence, targeting information, communications, weather information (both space and atmospheric), navigation support, and decision-support tools. In addition, information systems are being developed to support training, analysis, and system acquisition. Major elements and associated goals are listed below (DSTAG, 2000a):

1. Global awareness—to provide consistent, integrated battlespace information on demand and tailored to the needs of individual warfighters.

   • information obtained through fusion of data from high-resolution, all-weather sensors

   • automated information-exploitation tools with visualization of uncertainty

   • fusion of information into a single, consistent operating picture that provides situational awareness

   • an affordable, integrated, global information base that supports real-time exploitation and fusion

2. Dynamic planning and execution—to enable commanders to shape, visualize, and control the pace and phasing of engagements by exploiting global awareness and global information-exchange capabilities.

   • capability for predictive planning, integrated force management, and mission execution

   • capability for real-time sensor-to-shooter operations

   • collaborative, distributed, real-time mission planning, training, and rehearsal (battlespace simulation)

3. Global information exchange—to ensure that information is available anywhere, anytime for a mission through adaptable, scalable, fault-tolerant information systems.

   • seamless collaborative workspaces

   • 1,000-fold increase in capability for global communications to aircraft capability

   • continuous 24-hour per day, 7-day per week, in-transit visibility

   • worldwide information on demand

   • information warfare capability to protect, detect, and react

   • reliable, survivable networking

Commercial investments in non-DoD-related information technology in the United States amount to about a trillion dollars per year. The DoD IST budget is significantly less, about a billion dollars per year. DoD should leverage this large commercial resource, where appropriate, by adopting, adapting, and/or reengineering new technologies and determine the shortcomings of commercial technologies that warrant investing in DoD-unique S&T.

Leveraging commercial information technologies is difficult, however, because industry rapidly changes direction to meet rapidly changing customer demands and because the time to market must be as short as possible. Fierce competition dictates limited, short research and development cycles and near-term investment strategies. Very little funding is being invested in basic research, which is usually outsourced to academia (NRC, 1999). Industry’s short-term needs cannot support the longer-range visions of the services. Although commercial technologies show promise in providing significant near-term capabilities, leveraging them could require much effort (and significant resources) to adopt, adapt, or reengineer them. The services need to continue to be intelligent users that can recognize and exploit the military worth of commercial technologies.

Another caveat about using commercial information systems is that they are becoming available to all nations and interest groups. If the services depend on commercial technologies for advancing the state of the art in their information systems, potential enemies may come close to achieving parity (or even asymmetrical superiority) with U.S. forces. The services can try to maintain an advantage with better systems engineering, but that advantage may be temporary. Therefore, the services, and the rest of DoD, should protect and maintain a strong systems-engineering capability that can transform the commercial advances available to many nations into very powerful C4ISR systems.

Because of the global availability of advanced commercial technologies, DoD needs to expand its basic research to explore the frontiers of science in search of new technological approaches for maintaining military superiority. Figure 4–2 shows that, for a particular technology, the combination of commercial development that approaches DoD’s capabilities and the flattening of the capability S-curve as perceived technical or physical limits are reached will significantly limit DoD’s advantage. In this example, continued DoD investment would probably not increase its warfighting capability advantage. Therefore, DoD needs to either raise the “perceived” technical or physical limit or move to a totally new technology that provides a much greater warfighting capability. Either (or both) would require high-risk, large-payoff, basic research. The committee believes that DoD should continue to explore the frontiers of science and that basic research has never been more important to DoD.

Identifying gaps in commercial technology development is often difficult because most commercial

FIGURE 4–2 Notional S-curve depicting shrinking military warfighting advantage as technology matures and commercial development catches up to DoD development. SOURCE: Lupo, 2000.

work is proprietary. However, trends drawn from professional journals, trade shows, market analyses, and similar sources can provide good indicators. For discussion of commercial advances in communications, information assurance, computing and software, decision-making tools, and modeling and simulation and a description of commercial shortcomings in satisfying DoD needs, see Appendix F.

In summary, many advances in the commercial sector could be leveraged by DoD by making more rugged hardware, building top-layer applications for software, and adapting human-computer interfaces. However, many shortcomings of commercial technologies challenge the assumption that commercial information technology development can satisfy DoD needs. Many DoD-unique needs cannot be satisfied by the commercial sector.

The IST that will support the air and space systems of the 2010 to 2020 time frame will depend on basic research (6.1), applied research (6.2), and advanced technology development (6.3), in that order. Most of the basic research will come to fruition in the 2010 to 2020 time frame, applied research will support information systems in the 2005 to 2015 time frame, and the results of technology development will probably be supportive of information systems in the 2000 to 2010 time frame.

DoD’s basic research program has been very successful in supporting warfighter needs. Products have become enablers across a wide range of systems. The objective of basic research supported by DoD is to maximize the value of its investment by producing enabling technologies for realizing the operational concepts and mission goals of Joint Vision 2010 and Joint Vision 2020 (JCS, 1999; JCS, 2000). The technologies that result from basic research will be the cornerstones for satisfying air, space, and supporting IST needs in the 2020 time frame.

DoD’s timely investments in past basic research have resulted in many value-added capabilities and systems, including the Internet, night-vision technology, precision guidance for missiles, airborne lasers, the Global Positioning System, and mine countermeasures (DDR&E, 1999a). Some DoD basic research projects are conducted in DoD laboratories and research centers or in industry laboratories, but the major portion are conducted by academia.

Each service emphasizes basic research areas that support its long-term needs. For example, the Army emphasizes information technologies (mathematics), computer science (intelligent systems, software, and architecture), electronics for digitizing the battlefield, materials science for armor and for soldier protection, optical sciences for target recognition, chemistry and biological sciences for chemical and biological agent defense, and geosciences for terrain-related knowledge relevant to battlefield mobility. The Navy has a full-spectrum program that emphasizes a wide range of ocean-science activities, including predicting weather and currents, mapping the ocean floor, using acoustics to detect objects in the ocean, and conducting biotechnological research, such as understanding and mimicking communications between mammals. Air Force research is concentrated on aerospace sciences, materials, physics, electronics, chemistry, life sciences, and mathematics for application to air vehicles, space systems, and communications, command, control, computers, and intelligence (C4I) (DDR&E, 1999a). The service laboratories also conduct and manage basic research for DARPA, the Ballistic Missile Defense Organization, and other defense research agencies.

Considering the Air Force’s requirement for IST to support future air and space operations, it should be emphasizing information technologies. Yet the Defense Basic Research Plan reveals many weaknesses. For example, the Air Force has very limited efforts in the area of information electronics research to develop technologies for wireless communications, decision-making tools, advanced modeling and simulation, and image/target analysis and recognition. Similarly, Air Force research does not focus on atmospheric and space atmospheric sciences, meteorology, or remote sensing (DDR&E, 1999a). Unless the Air Force expands its basic research program in support of IST, it may not be able to satisfy the requirements of its future air and space operations.

A review of the DoD IST TARA shows that overall DoD support for IST in applied research (6.2) and technology development (6.3) is commensurate with service visions. However, realizing DoD visions will depend on funding and, in many cases, DARPA initiatives. This dependence on DARPA may be risky because DARPA’s approach to S&T is different from the approach of the services. DARPA frequently uses the briefing chart shown at Figure 4–3 to highlight what it believes are the differences between its approach to S&T and the services’ approaches. According to

FIGURE 4–3 Defense Advanced Research Projects Agency (DARPA) briefing chart contrasting DARPA’s approach to R&D with its view of the approach taken by the services. SOURCE: Fernandez, 2000.

DARPA, it focuses on radical innovations, while the services support requirements-based R&D.¹

Based on the differences between DARPA’s approach and the services’ approaches, DoD could develop an S&T program that provides both market pull (requirements-based innovation) and technology push (radical innovation). However, reductions in service funding, particularly Air Force funding, in IST have made the services more and more dependent on DARPA. A combined market-pull/technology-push approach would require that DARPA closely coordinate its investment strategy with the services and continue working in technology areas long after “radical innovations” had been developed to support technology development and transition to the services. However, DARPA has not traditionally supported projects past the radical innovation phase through the transition into actual systems, which are much more costly to develop.

To exploit DARPA’s successes, the services need to have the proper people and resources ready to take over a DARPA technology and continue its development until it is ready for transition to a service or joint information system. This approach appears to be far more acceptable than changing DARPA’s focus. However, service funding is often insufficient for the services to accept a DARPA technology, develop it, and transition it to an information system. The services do not have the flexibility to shift discretionary funding (or any other funding) to take advantage of a DARPA (or even commercial) radical innovation. Under the present system, service funding has to be programmed years in advance.

The services need to deliberately place a priority on their IST S&T investments. The services uniformly recognize IST as being central to their visions; however, the details of the required IST S&T programs are complex and arcane. This can lead to IST S&T investment effectively receiving a lower priority than high-visibility, more easily understood investments in sys-

tems such as faster, longer-range air and space platforms that can carry heavier payloads. Advanced systems are still needed; however, IST investments are just as necessary. In after-action reports on Kosovo, it is often mentioned that warplanes could not complete bombing missions (hit the appropriate targets) because of poor weather conditions and inadequate information (DoD, 2000; Graniero, 2000). Pilots were unable to “see” through thick clouds because of the lack of high-resolution, all-weather sensors and information systems (Lupo, 2000). Other problems were inadequate tracking or loss of targets from the time they were detected, which indicates a need for smarter, global, time-independent information systems. As the services make their investment decisions, trade-off analyses based on simulation-based experiments and technical features of the air, space, and supporting information systems can help provide insight into the value of their investment in IST.

Air Force funding for IST basic research (6.1), applied research (6.2), and technology development (6.3) does not appear to be adequate to support future air and space needs. Recent trends indicate that future funding may be even less adequate. Because the Air Force is focused primarily on air and space operations, Air Force funding is used as a benchmark in this section.

DoD funding for basic research in information systems has increased slightly since FY97 (see Figure 4–4). As discussed in the previous section, there are weaknesses in the Air Force IST-related basic research program, and industry has very little interest in basic research. Therefore, funding in basic research may not be adequate to support air- and space-related IST needs for the 2010 to 2020 time frame. The committee believes that the Air Force basic research program is too small to match its IST visions and compensate for the limited availability of commercial research.

In addition, as Figure 4–5 shows, there is proportionally little in-house funding for basic research in AFRL’s Information Directorate (AFRL/IF) budget (only $2.5 million of the total S&T budget of $555.4 million), not nearly enough to maintain in-house IST expertise. As shown in Figure 4–4, the Air Force invests a total of about $30 million per year in IST-related basic research. About $27 million of that supports outsourced research in academia and industry.

FIGURE 4–4 Service funding for information systems technology-related basic research, FY95 to FY01. SOURCE: DSTAG, 2000d.

FIGURE 4–5 FY00 budget for Air Force Research Laboratory Information Systems Directorate (AFRL/IF). SOURCE: DSTAG, 2000a.

Figure 4–5 shows that the amount of FY00 Air Force funding in IST applied research and technology development ($77.4 million) is only about 5 percent of the total amount of AFRL S&T funding ($1.391 billion), much less than the committee believes is required to meet Air Force needs. Figure 4–5 also shows that the Air Force is using DARPA and other funding to offset the shortfall. However, Air Force core funding for 6.2 and 6.3 ($77.4 million) is not enough to maintain a service capability in IST and still continue the development and transition of technologies shown to be promising by DARPA and other DoD sources (DSTAG, 2000a). Air Force 6.2 funding can barely pay the operating costs of the AFRL/IF laboratory including inhouse salaries. Unless salaries can be covered by non-Air Force customer funding, thereby freeing up some IF 6.2 funding, no money will be left for external research and development (e.g., by industry and universities) (personal communication, Dr. Northrup Fowler, AFRL/IF, February 7, 2001).

As Figure 4–1 shows, IST funding has not been steady. Each year since FY96, the Air Force has established a funding program for the AFRL/IF that appeared to be stable or to increase for the next five years. And in each subsequent year, the predicted level of funding did not materialize. Even if the FY01 budget is realized, the amount will still be inadequate. Even the very optimistic funding for the out years in the President’s budget request falls short of FY96 and FY97 levels. These decreases have occurred at the same time that Air Force operational visions have become more dependent on information systems and supporting non-DoD commercial efforts (especially long-term basic research) have continued to fall short of expectations.

As would the other areas of Air Force S&T, the Air Force IST S&T program would benefit from S&T-focused representation and advocacy at the corporate policy and decision-making level of the Air Force. A trusted, authoritative Air Force champion at that level

could help ensure that the Air Force IST investment better matches its warfighting visions.

Conclusion 4–1. The Air Force visions for air and space operations have become increasingly dependent on information systems, yet the Air Force has reduced its IST S&T budgets each year. Although projections for Air Force IST S&T funding show increases through FY05, the trend has been to disregard the investment plan, reduce funding, and then propose increased funding for the future. The committee believes that budgets for 6.1, 6.2, and 6.3 S&T are much too small to support Air Force IST visions.

Conclusion 4–2. Service visions are not truly joint visions. They do not address the issues of interoperability with, and leveraging of, air and space technology developments by NASA, the National Reconnaissance Office, the Ballistic Missile Defense Organization, and other government organizations.

Conclusion 4–3. Although about $1 trillion is invested in non-DoD commercial IST development in the United States every year, leveraging the results has been difficult because commercial technologies are not focused on military needs, they satisfy only some needs for IST, they provide mostly near-term solutions, they do not benefit from stable investments in research and development (and there is very little investment in long-range basic research), and they may not be able to provide long-term support. DoD technologies should leverage commercial technology where appropriate and use advanced systems-engineering techniques to provide a differential capability. For DoD to invest its limited resources wisely to maintain its capability advantage, budgeting processes need to be flexible enough to take advantage of commercial advances as they become available.

Conclusion 4–4. The Air Force is becoming increasingly dependent on DARPA to support its S&T needs. However, inadequate Air Force funding, differences in DARPA’s and the Air Force’s S&T roles, and differences in investment planning and execution processes have made it difficult for the Air Force to exploit and transition DARPA successes.

Conclusion 4–5. The Air Force (and other services) would benefit greatly from a high-level advocate for S&T programs during internal budgetary decision making.

Recommendation 4–1. The committee believes that the Air Force should increase its science and technology (S&T) budget for information systems technology (IST). The basic research (6.1) program should support long-term air and space IST needs, surpass previous-year levels, support a strong in-house program (with appropriate researchers), and compensate for limited long-term commercial investment.

Recommendation 4–2. The U.S. Department of Defense (DoD) should develop a better DoD-wide joint vision for information systems technology (IST) science and technology (S&T) that takes into consideration work by the National Aeronautics and Space Administration, the National Reconnaissance Office, the Ballistic Missile Defense Organization, and other government agencies. The services should develop closer working relationships with other DoD and non-DoD organizations.

Recommendation 4–3. The U.S. Department of Defense (DoD) should make as much use of non-DoD commercial developments in information systems technology (IST) as possible to reduce or offset the impact of recent reductions in IST science and technology (S&T) programs. S&T programs and flexible budgeting processes would enable the services to leverage the best commercial technologies and still maintain a strong military advantage.

Recommendation 4–4. The Air Force (and other services) should develop an investment strategy for applied research (6.2) and technology development (6.3) that takes advantage of Defense Advanced Research Projects Agency successes, but is not dependent on them.

Recommendation 4–5. Each service should designate a high-level advocate who has an understanding of the state of the art of information systems and can formulate service S&T budgets, participate in corporate policy and decision-making activities, and work closely with other DoD and non-DoD organizations.

DDR&E (Director, Defense Research and Engineering). 1996. Advanced Battlespace Information System (ABIS), Task Force Final Report, May 1996. Washington, D.C.: U.S. Department of Defense.

DDR&E. 1999a. Basic Research Plan, February 1999. Washington, D.C.: U.S. Department of Defense.

DDR&E. 1999b. Defense Technology Area Plan, February 1999. Washington, D.C.: U.S. Department of Defense.

DDR&E. 2000. Joint Warfighting Science and Technology Plan, February 2000. Washington, D.C.: U.S. Department of Defense.

DoD (Department of Defense). 2000. Kosovo/Operation Allied Force After-Action Report. Report to Congress, January 31, 2000. Washington, D.C.: U.S. Department of Defense.

DSTAG (Defense Science and Technology Advisory Group). 2000a. Air Force Strategy in Information , briefing by John A.Graniero at the Information Systems Technology (IST) Technology Area Review and Assessment (TARA), Air Force Research Laboratory, Rome, New York, March 13–17, 2000.

DSTAG. 2000b. Transforming to Meet the 21st Century Requirements— The Army Vision, briefing by James J.Barbarello at the Information Systems Technology (IST) Technology Area Review and Assessment (TARA), Air Force Research Laboratory, Rome, New York, March 13– 17, 2000.

DSTAG. 2000c. Overview of IST Plans in the Navy, briefing by Andre van Tilborg at the Information Systems Technology (IST) Technology Area Review and Assessment (TARA), Air Force Research Laboratory, Rome, New York, March 13–17, 2000.

DSTAG. 2000d. Information Science and Technology, briefing by James Gantt at the Information Systems Technology (IST) Technology Area Review and Assessment (TARA), Air Force Research Laboratory, Rome, New York, March 13–17, 2000.

Etter, D.M. 2000. Defense Science and Technology, presentation by D.M. Etter, Deputy Under Secretary of Defense for Science and Technology, to the Committee on Review of the Department of Defense Air and Space Systems Science and Technology Program, National Research Council, Washington, D.C., January 24, 2000.

Fernandez, F.L. 2000. Presentation to the Land Warfare Conference 2000, Melbourne, Australia, October 16, 2000.

Graniero, J. 2000. Information Systems Technology: Briefing to National Research Council, presentation to the Committee on Review of the Department of Defense Air and Space Systems Science and Technology Program, National Research Council, Washington, D.C., January 24, 2000.

JCS (Joint Chiefs of Staff). 1999. Joint Vision 2010. Joint Chiefs of Staff. Washington, D.C.

JCS. 2000. Joint Vision 2020. Joint Chiefs of Staff. Washington, D.C.: U.S. Government Printing Office.

Lupo, J. 2000. Interview with Jasper Lupo, director, Defense Sensor Systems, Office of the Deputy Under Secretary of Defense for Science and Technology, March 24, 2000.

NRC (National Research Council). 1999. Trust in Cyberspace. Committee on Information Systems Trustworthiness, Computer Science and Telecommunications Board. Washington, D.C.: National Academy Press.

NSTC (National Science and Technology Council). 1995. National Security Science and Technology Strategy. Available online at <http://www.ostp.gov/html/nssts/html/nssts.html>.

USD (A&T) (Under Secretary of Defense for Acquisition and Technology). 1995. Modeling and Simulation (M&S) Master Plan, dated October 1995. Department of Defense. Washington, D.C.: Defense Modeling and Simulation Organization.