Satellite Observations of the Earth's Environment: Accelerating the Transition of Research to Operations (2003)

As discussed in Chapters 2 and 5, effective transitioning of research and technology to operations requires a supportive organizational structure and culture; effective planning, communication, and coordination; a strong scientific and technological foundation; a balance between research “push” and operational “pull”; and adequate infrastructure and financial and human resources. Case studies show that complete transitions of environmental measurements have often taken a decade or more, although there is considerable variability (see Chapter 5, the section entitled “Case Studies and Lessons Learned,” and Appendix B). Partial transitions—typically involving measurements acquired by research missions that are then incorporated into operations (e.g., scatterometry)—have occurred frequently. There are also examples in which instruments have been flown routinely on operational satellites prior to completion of the data reduction algorithms and the development of techniques for assimilating the measurements into operational prediction models.

The committee recognizes that there are many different ways to accomplish transitions and that valuable transitions would continue without any changes to the present system. However, to speed up the rate of transitions and thereby increase the return on the research investment, there is need for a more organized and focused mechanism for transitioning NASA research into NOAA operations. Specifically, the transition process could be dramatically improved by a mechanism that is systematic and transparent (i.e., has well-understood processes and structures), robust (i.e., can influence and drive the transition), and universal (i.e., may be applied to a diverse set of scientific opportunities, technological capabilities, and operational requirements rather than being case-specific).

The elements and building blocks for successful transition architectures described in Chapter 5, in the section entitled “Transition Pathways and Processes,” constitute the criteria against which a candidate transition architecture can be evaluated (see Figure 6.1). To be valuable, any proposed architecture must provide a comprehensive solution to the problems imposed by the valleys of death and lost opportunities, in effect establishing a reliable bridge over which the transition from research-to-operations can proceed in an efficient and coordinated manner. Furthermore, the architecture must address the relevant transition communities from end to end, including the research community, the sensor systems community, the data-analysis and data-assimilation community, and the users. (See Box 6.1 for a descriptive analogy of research-to-operations transition pathways.)

The committee considered several approaches to improving the end-to-end transitioning process. After evaluating the strengths and weaknesses of each approach, the committee found that one approach best addresses the objective of establishing a mechanism to develop effective transition architectures without introducing new and equally challenging issues. The committee has identified the need for the creation of an Interagency Transition Office that will both support the research and operational efforts in NASA and NOAA and strengthen the transitioning process.

The committee recommends the formation of an Interagency Transition Office (ITO) that would have responsibility for strategic planning and coordination, but not implementation, of all transition activities. This office is intended to support and simplify transitions by augmenting, enabling, and leveraging the existing infrastructure within NASA and NOAA rather than by introducing duplicative capability or bureaucracy.

The ITO should be chartered through a formal agreement, such as a Memorandum of Agreement, between the agencies, and signed at least one level above the highest level at which transition-related activities occur. The responsibilities and authorities of the ITO should encompass transitions required by NPOESS, GOES, and all NOAA operational spaceborne systems and should explicitly reflect the needs and capabilities of all parts of both agencies that are responsible for the transition of research to user applications. As envisioned (see the description below), the ITO would strengthen the partnership between NASA and NOAA and support the missions and activities of both agencies.

The ITO should be a small but permanent, full-time office (the committee anticipates approximately 10 people)—staffed half by NOAA and half by NASA. It would include representatives from the NOAA operational entities (such as the National Weather Service and the National Ocean Service) that use the research products and the NASA entities that carry out the research. It should receive input from the broad scientific and user communities through a high-level advisory council with membership drawn from the relevant research and operational communities outside the participating agencies. Funding should come equally from both agencies. The lead ITO person from each agency should report directly to the person who has the highest authority for transition-related activities at NOAA and at NASA headquarters, respectively. An executive board, envisioned by the committee

as including the NASA and NOAA administrators and the President’s Science Advisor at a minimum, should provide high-level oversight and review of the ITO. NASA and NOAA should consider including as executive board members representatives at an equivalent level from DOD (for example, the undersecretary of defense for acquisition and technology) and from other agencies when appropriate to the mission of the ITO.

In its role as a strategic planning and coordination office, the ITO would provide the systems engineering guidance and oversight which ensure that all aspects of each transition are identified, properly incorporated into the transition plan, and effectively implemented. In particular, the ITO should be a champion for transition elements such as data-archiving and information-extraction systems that traditionally “fall through the cracks.” The ITO should have the following responsibilities and authorities associated with the architecture (or design) and oversight of the entire transition pathway, including the necessary building blocks for successful transition. Specifically, the ITO should carry out these tasks:

• Review long-term (0 to 20-year) NASA and NOAA plans in order to identify requirements, mission/instrument needs and capabilities, and related activities that require transition support or are candidates for transition. The ITO should formally evaluate all NASA missions for their potential for operational applicability. On an annual basis, the ITO should produce a NASA-NOAA Transition Planning and Status Report, summarizing the plans, status, and prioritization of the agreed-upon transition projects and associated budgets, with signature approval by authorities at the highest levels at which transition-related activities are budgeted and implemented at NOAA and NASA, respectively.

• For each identified transition project, establish a NASA-NOAA Project Transition Plan for accomplishing the transition, with signature approval by authorities at the highest levels at which transition-related activities occur at NOAA and NASA headquarters, respectively. Each plan should define measures of transition effectiveness and should describe the activities associated with the transition, including the operational requirements to be satisfied, scientific validation of the measurement technique, technology development requirements, any need for pre-operational test beds or flight validation, algorithm and assimilation needs, a data-management plan, and other, related activities. The plan should describe (in detail appropriate to the time frame of the activity) how each activity is to be accomplished; it should include the following:

  • A schedule of transition activities, to include the training and education necessary for full operational capability;

  • The responsibilities of NASA and NOAA and other participating agencies or partners for each transition activity;

• • A description of the financial and human resources required for each transition activity, as well as an estimate of expected resources required for operations and maintenance after the acceptance of initial operational capability; and

  • A description of the necessary interfaces with other NASA, NOAA, and participating agencies’ or partners’ activities.

• Identify the infrastructure and in-house capability required within NASA and NOAA and their government partners to ensure that transitions can be accomplished effectively, and recommend needed changes within the NASA-NOAA Transition Planning and Status Report.

• Define measures of transition effectiveness and systematically monitor the progress of NASA and NOAA projects in implementing the agreed-upon transitions. On an annual basis, the ITO should describe the progress of each identified transition project in the NASA-NOAA Transition Planning and Status Report.

• Provide a forum for identifying transition-related weaknesses in the NOAA requirements and in the NASA research plans, describe transition issues, and present candidate solutions for resolution at the highest level of authority required at NOAA and NASA headquarters, respectively.

The annual NASA-NOAA Transition Planning and Status Report should be recognized as a supporting document in the budget requests of each agency. Implementation—that is, project and program management—of the activities described by the plan should be carried out not by the ITO but rather by each agency, as agreed to in the plan and as budgeted by the agency. It is anticipated that the ITO will not replace the existing transition approaches and processes, but rather employ these when beneficial and strengthen them where necessary and feasible. On a regular basis (e.g., every 5 years), the ITO should be reviewed through an independent process.

A significant advantage of the ITO approach is that it provides a neutral honest broker in the transition planning and coordination process because the ITO has dent external advisory council. This approach has the following additional benefits equal high-level representation from both NASA and NOAA, as well as an indepenwith regard to the NASA-NOAA transition process:

• It is well integrated in the leadership structure of both NASA and NOAA, supporting, rather than competing with, NASA and NOAA priorities.

• It provides jointly developed specific guidance (budgetary and programmatic) to assist the agencies in producing well-coordinated plans.

• It replaces the current ad hoc process with a formal process that facilitates all transitions (both push- and pull-driven) and can be readily reviewed and evaluated for effectiveness.

• It provides a clear point of contact for research-to-operations transitions.

• It is straightforward to establish. It does not require a presidential directive or an act of Congress to create a new organization. It does not create a new bureaucracy that takes authority from NASA and NOAA.

• Since the annual NASA-NOAA Transition Planning and Status Report is approved at high levels in the agencies, it will be accompanied by the appropriate implementation resources allocated within each organization. All involved in the transition activities will understand their responsibilities.

• The people who best know the research (NASA) and operational needs (NOAA) are the ones providing input to the transition plan.

• It provides an open and fair process, with input from the broad external scientific and user communities, for prioritizing, developing, and monitoring transition-oriented efforts.

• It can incorporate and expand on the existing transition approaches rather than disruptively replacing them.

• It is consistent with the existing NASA research peer-review process. The peer-review process introduces uncertainty into transition planning but is central to the integrity and quality of NASA research. An appropriate balance between these needs could be achieved by encouraging the pursuit of ITO-generated priorities in proposals, but evaluating the proposals using normal peer-review processes.

• It invents a proactive forum for advertising and marketing emergent research push and operational pull.

• It has the additional benefit of being sufficiently flexible to support the future inclusion of other agencies (e.g., DOD) that could be interested in becoming the operational beneficiaries of NASA research. Representatives of the new member agency would simply be added to the ITO, that agency would produce its own planning documents, and the ITO would produce an additional document concerning agreed-upon transitions.

The approach has potential disadvantages, including these:

• Redirecting existing resources to establish and staff a new office; establishing a new bureaucracy, if the ITO is not implemented properly;

• The need for two agencies to reach agreement before a particular transition project can be implemented; and

• Giving the appearance that transition responsibilities are now exclusively the responsibility of a dedicated office.

This recommended approach is consistent with and builds upon the recent NRC report From Research to Operations in Weather Satellites and Numerical Weather

Prediction: Crossing the Valley of Death (NRC, 2000a). In particular, that report recommends creating a replacement for the Operational Satellite Improvement Program (OSIP), which was terminated in 1982. The functions of the ITO include those of OSIP but go beyond them by considering the entire end-to-end process for transitions. This end-to-end process supports many of the recommendations in the NRC (2000a) report. The approach and recommendations presented here are also consistent with a recent study of the socioeconomic benefits of Earth science research, especially the recommendations that NASA and other government agencies should expend sustained resources on better understanding and on improving the flow of information from science to applications (Hertzfeld and Williamson, 2002). The Hertzfeld and Williamson study recommended that “a detailed analysis of the research to applications should be conducted in each applications area in order to achieve the best return on investment in Earth science research,” which is what the NASA-NOAA Transition Planning and Status Report will accomplish. A discussion of how the ITO would work in practice is provided in Box 6.2.

The committee considered four alternatives to its recommended approach. It found that all had greater difficulties in achieving one or more of the necessary critical elements for managing the research-to-operations transition in a continuous, robust, and influential manner. The alternative approaches are outlined in the following subsections.

The first alternative approach considered by the committee is that of establishing a new transitioning agency, with full responsibility for research in support of environmental remote sensing operations and the concomitant transition of capabilities. The responsibilities of the new agency would include research and planning, and it would have programming and budgeting authority, as well as responsibility for implementing the transitions. Such responsibility translates into having the resources to conduct research, development, testing, and evaluation and demonstration, as well as to procure, operate, and maintain the space and ground segments used for environmental remote sensing. This agency would function in an independent capacity, with accountability at the highest level of government, similar to the way in which NASA, the NSF, and other independent agencies function.

There are advantages to establishing such an agency:

• Those inefficiencies that result from different agencies having separate research and operational missions could be overcome. Under the current structure, separate plans, progress reports, and budgets must be submitted through different bureaucracies to support the same ultimate operational capability.

• The new agency would facilitate centralized reporting of the national status and plans for civil applications of satellite-based environmental observations. This “one-stop shopping” would minimize the chances of ambiguity in conveying our nation’s environmental remote sensing program to all interested parties in government, industry, academia, and internationally.

The establishment of a new agency under this approach also has distinct disadvantages:

• It would be extremely difficult in the present budgetary and political climate to establish such a new agency.

• Focusing the responsibility for transitioning research to operations would release mission agencies from the responsibility for transition and would also remove

the influence and control that these agencies enjoy under the current organization. That is, there is no mechanism under this proposed structure to ensure that NOAA’s mission needs (notably in the National Weather Service and the National Ocean Service) would receive adequate attention from the new agency.

• Many current efficiencies in the use of research resources would be lost. Assuming that there are no new resources to support this new agency, resources for it would have to be allocated from existing sources. As a result, there would undoubtedly be losses of efficiencies and duplication of investment. This would be especially pronounced within the research component, where current resources (i.e., laboratory and computational facilities) are being used to support a multitude of efforts, only some of which are oriented toward environmental remote sensing.

Creating a new agency as described here is very similar to the approach of placing all of the research and operational activities, including the applications, within one existing agency (such as NOAA). The primary difference is that one of the existing agencies could not be an “honest broker” (the importance of which is described above). If the responsibilities were given to NOAA, for example, NASA would not be an equal partner, likely leading to poor communications and coordination, budget inconsistencies, prioritization disagreements, and inadequate joint planning.

The benefits defined above are small, and they would be attainable through means other than the difficult processes associated with establishing a new agency or placing all transition activities within one existing agency. Similarly, the advantages described above could be realized without a major bureaucratic reorganization.

The second alternative approach considered by the committee is that of retaining the status quo of case-by-case transitions (described in Chapter 5) and instituting improvements where desirable and feasible. The committee recognizes that some existing transition activities have been effective and efficient. The existence of successful transitions (as shown by the case studies discussed in Chapter 5 and Appendix B) illustrates that transitions can be accomplished in certain cases, when properly planned and coordinated by the mutual agreement of NOAA and NASA along with the strong individual involvement of those wielding resource authority.

However, the case-by-case approach on the whole is inefficient, inconsistent, and fiscally unpredictable; it leaves critical elements unaddressed; and it is not a robust, long-term solution. Moreover, as described in Chapter 4, scientific and technological opportunities are increasing rapidly, and the complexity of research-

to-operations transitions is also expected to increase significantly over the next two decades, with issues such as multiple national and international partners. Therefore, even an improved ad hoc approach will become increasingly unable to systematically assess and ensure the existence of all the critical elements necessary for successful operational transition.

The third alternative approach considered by the committee is that of expanding the role of the NPOESS Integrated Program Office (IPO) to include all of the transition-related activities that occur between NASA and NOAA. It is reasonable to consider broadening the IPO charter to explicitly include overall responsibility for the planning and implementation of NASA-NOAA transition activities. This approach has the objective of establishing a single entity with centralized responsibility for the planning and implementation of transitions. The IPO is an appropriate candidate to lead this activity, given its charter for operational environmental systems. However, several significant issues arise with this approach.

The committee believes that the ITO would perform a function fundamentally different from that of the IPO, and that consolidation of NASA-NOAA transition activities with the IPO would adversely impact the nature and effectiveness of NASA and NOAA. Figure 6.2 illustrates the differences. In this figure, the vertical axis

FIGURE 6.2 Differences between the existing NPOESS Integrated Program Office (IPO) and the Interagency Transition Office (ITO). (See discussion in text.)

represents the progression with time from research, or R&D, to operations. In the IPO case (left), both NOAA and DOD perform R&D and operations within their respective charters. In most cases, these R&D and operational activities do not overlap (e.g., NOAA does not develop weapons). An important exception is the development and operation of the polar satellites, where there is a clear overlap (the vertical gray area). The primary objective in forming the NPOESS IPO was to address the duplication problem caused by largely similar NPOESS and DOD operational polar environmental satellite programs—not to address the broad research-to-operations transition problem that is the subject of this study.

The NASA-NOAA transition problem in general (right side of Figure 6.2) is orthogonal to the IPO situation. NASA primarily carries out research, while NOAA mostly conducts operations. The horizontal “gray area” covers a wide range of missions and capabilities, and occurs at the handoff between research and operations.

The IPO charter included a broadly worded role for NASA to provide research and development activities in support of NPOESS,¹ and NASA has responded to this role through development programs such as the Advanced Technology Microwave Sounder (ATMS) and the NPOESS Preparatory Project (NPP). Despite these examples, the IPO charter does not specify that NASA, the IPO, or any joint office has ultimate responsibility for overall planning and implementation of transition activities. As a result, even apparently successful transitions such as the ATMS and NPP were planned and accomplished on an ad hoc basis, through case-by-case negotiations between NASA and the IPO. With limited exceptions, future IPO requirements (such as the list of unsatisfied EDRs² or plans for Pre-Planned Product Improvement) are not coordinated with a NASA research plan in order to investigate and develop operational solutions. Similarly, NASA research capabilities are not formally evaluated for their operational potential to NPOESS. Furthermore, as the current IPO charter responsibility formally ends with the provision of EDRs, the IPO does not itself represent the needs and requirements of the user community, but rather must seek them elsewhere within NOAA and DOD.

Specific disadvantages with the assignment of the entire transition role to the IPO include these:

• Without strong IPO influence over NASA research activities, the ability of the IPO to identify candidate research and to plan associated transitions is no more effective than the current system, unless transition authority is given to the NASA IPO representative or unless a joint NASA-IPO office with authorities similar to those envisioned for the ITO is established.

• In order to centralize NASA-NOAA transition authority, the IPO charter would have to be expanded to include GOES and other NASA-NOAA transition activities.

• An IPO-led transition entity would not be perceived as an “honest broker” of the transition process, placing transition authority on the operations (“pull”) side of the research-to-operations balance.

• Unless the IPO role is further expanded beyond that designated by the current charter, the IPO is not an entity with responsibility for the end-to-end transition process, particularly with regard to representing the end user. An IPO-led transition entity would require additional interfaces with other planning entities.

• While the IPO has extensive experience in developing and procuring an operational environmental satellite system, it does not have experience in performing research. It would either have to develop this experience or limit its transition role to that of planning but not implementation, similar to the role proposed for the ITO.

The fourth alternative approach considered by the committee is that of using the existing Office of the Federal Coordinator for Meteorological Services and Supporting Research (OFCM) to facilitate transitions. As discussed in Chapter 5, the OFCM has an existing federal infrastructure, relevant mission responsibility, and a past record of successfully coordinating multiagency projects. A NASA-NOAA Transition Program Council could be established, with high-level representation from NASA and NOAA (and probably DOD),³ that would provide the means to develop transition plans and obtain the budget commitment needed, through the agency representatives as the agencies directed. Other federal agencies could be added as appropriate, since they would have fundamental representation within the overall OFCM infrastructure, although not initially on the NASA-NOAA Transition Program Council.

However, as an existing staff organization, the necessary direct line organizational and budgetary authority links needed to achieve the funding commitments and focus required over the longer term may not be achievable. Dedicated agency staffing, with full-time, high-level agency representatives, may be required to provide the necessary greater focus on transition planning and implementation.

The committee’s findings point to the necessity of creating a joint research-to-operations transition office. The benefits of this approach, as described above, include these:

1. A dedicated, organized, and focused mechanism for transitioning research to operations;

2. A balanced and neutral office, reporting to high-level authorities in NASA and NOAA, charged with planning and coordinating the transitional process from research to operations, thereby strengthening the partnership between NASA and NOAA and supporting the missions of each;

3. A continuous, long-term planning mechanism connected to the budgetary planning processes in NASA and NOAA;

4. On-going review responsibility of all transitions;

5. Valuable input provided by an independent, external advisory council made up of individuals knowledgeable about both the research and operations; and

6. An equally balanced, high-level staff reporting to equally high-level authorities (within their respective agencies), charged with planning and coordinating the transitional process from research to operations.

While the committee believes that creation of the Interagency Transition Office would be a major step toward making transitions from research to operations faster and more effective, it realizes that the ITO by itself will not guarantee success. A strong and sustained commitment by NASA and NOAA leadership to the ITO in particular and to transitioning in general is required, and the ITO must be provided sufficient resources and authority to do its job. In addition, cultures in both agencies that favor technology transfer must be fostered, and appropriate reward systems need to be implemented to attract the high-quality people needed to make the ITO successful.