4
The Vital Role of Program/Project Management and Systems Engineering at NASA
Given the size, complexity, and number of parallel system developments required in both the near and the long term, achieving the goals of the Vision for Space Exploration (VSE) would be extremely challenging even under ideal circumstances. However, requirements for personnel with specific skills and expertise, such as experienced program/project managers and systems engineers, in NASA’s workforce will make the VSE an even more ambitious undertaking.
NASA PROGRAM MANAGEMENT AND SYSTEMS ENGINEERING
NASA’s SEITT report stated that the agency’s development of a space exploration architecture raises several major concerns, among them “the availability of sufficient expertise in the management of large space development programs.”1 More specifically, the agency’s April 2006 Workforce Strategy discussed agency workforce competency trends. Section 3C of that assessment identified an increased need through fiscal year 2009 in five skill areas: program/project management, systems engineering and integration engineering, mission operations competencies, systems analysis and mission planning, and quality/safety/performance.2 NASA identified a requirement for the following number of full-time employees with the competencies indicated:
-
150-200 with program/project management competency;
-
100-150 with systems engineering and integration engineering competency;
-
200-240 with mission operations competencies;
-
25-40 with systems analysis and mission planning competency; and
-
50-75 with quality/safety/performance competency.
Thus, the agency requires 525 to 705 personnel in these areas, a requirement driven primarily by the establishment of the Constellation program. Several of these skill areas are particularly challenging to fill because they
1 |
NASA, Office of Program Analysis and Evaluation, Systems Engineering and Institutional Transitions Study, Final Report, April 5, 2006, p. 13. |
2 |
NASA, National Aeronautics and Space Administration Workforce Strategy, April 2006, pp. 15-16, available at http://nasapeople.nasa.gov/HCM/WorkforceStrategy.pdf. |
are heavily experience-based, with competence developed as a result of involvement over many years in many projects.
Although the mission operations competencies category is the largest, the committee notes that this competency is relatively easier to fill from NASA’s existing workforce because the agency already conducts mission operations in both the Space Shuttle and the International Space Station programs. In contrast, in the competency areas of program/project management and systems engineering and integration engineering (with a requirement for 250-350 full-time employees), the agency is at a greater disadvantage because it has not conducted significant development of human spaceflight systems in-house in nearly three decades.
Not since the design of the Space Shuttle has there been such a demand or opportunity for the development of major human spaceflight systems. NASA’s most recent human spaceflight development project, the International Space Station, engaged a primary contractor as the systems engineering and integration lead and provided only a limited number of systems engineering and program management learning opportunities for NASA’s workforce, which concentrated on the operational aspects of ISS such as pre-launch integration testing, on-orbit assembly, and flight operations.
To fulfill requirements for expertise in program/project management and systems engineering and integration engineering, NASA will thus be forced either to recruit highly skilled personnel from industry or to use personnel who have less experience than it might desire. The committee heard evidence that the agency has taken both of these approaches.
The committee believes that highly skilled personnel in these categories are key to the successful conduct of projects. As noted in a 2003 Department of Defense report on the acquisition of national security space programs, inadequate systems engineering in the early design and definition stages of a project has historically been the cause of major program technical, cost, and schedule problems (see Box 4.1).3
Other government agencies with missions similar to NASA’s, such as the U.S. Air Force and the National Reconnaissance Office, have also recognized the importance of highly skilled systems engineers as well as the difficulties of developing and maintaining them.4 Based on previous studies conducted for those organizations as well as comments made to the committee during its meetings, the committee concluded that skilled program managers and highly skilled systems engineers are a vital resource.5
The industry-wide requirement for highly skilled program managers and systems engineers is not new; on the contrary, it has been recognized as a problem within the aerospace industry for at least 20 years. Rather than an acute crisis, the requirement thus remains an ongoing challenge. In NASA’s case and for human spaceflight in particular, the challenge is more formidable because of the lack of recent large human spaceflight technology development programs, constrained hiring as a result of the downsizing in the 1990s, and the prospective workforce’s skepticism about whether NASA can offer a long-term commitment to providing challenging work at competitive salaries that will survive a given presidential administration.
The committee notes that identification by NASA, industry, and the national security space establishment of the requirement for skilled program/project management and systems engineers not only underscores the importance of these skill areas, but also highlights the difficulty that NASA faces in obtaining them—the agency must compete with industry and DOD for employees that those sectors also value highly.
3 |
Department of Defense (DOD), Report of the Defense Science board/Air Force Scientific Advisory Board Joint Task Force on Acquisition of National Security Space Programs, Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, DOD, Washington, D.C., 2003. |
4 |
During presentations before the Air Force Studies Board in January 2007, several Air Force and NRO officials spoke about the difficulties of conducting systems engineering for national security space programs. The presenters included Roberta M. Ewart, “The Counterspace Architecting Process: Briefing to the National Research Council,” January 9, 2007; Colonel James Horejsi, “Applying Systems Engineering to Pre-Milestone A Activities,” January 8, 2007; and Major General Mark Shackelford, “Thoughts on Systems Engineering,” January 8, 2007. The Air Force in particular faces difficulties in maintaining these skills in-house and has lost many of them to industry. |
5 |
Gen. Thomas S. Moorman, Jr., U.S. Air Force (retired), testimony before the House Subcommittee on Space and Aeronautics, May 15, 2001; John Williams, Booz Allen Hamilton, presentation to NRC Workshop on Meeting the Workforce Needs for the National Vision for Space Exploration, January 23, 2006; Report of the Defense Science Board/Air Force Scientific Advisory Board Joint Task Force on Acquisition of National Security Space Programs, May 2003, Office of the Undersecretary of Defense for Acquisition, Technology, and Logistics, available at http://www.acq.osd.mil/dsb/reports/space.pdf. |
BOX 4.1 Common Factors Contributing to the Success or Failure of NASA Programs Factors contributing to program failures or significant cost growth (from investigations)
Contributing factors associated with successful programs (from organizational literature)
SOURCE: Adapted from NASA, Office of Program Analysis and Evaluation, Systems Engineering and Institutional Transitions Study, Final Report, April 5, 2006. |
In considering whether NASA also faces a requirement for highly skilled systems engineering and program/ project management personnel in its robotic spaceflight program, the committee found that a steady succession of robotic programs has provided opportunities for sustaining a base of expertise in NASA and industry in robotic spaceflight. Nevertheless, given the concerns it heard expressed within NASA, academia, and industry about the amount of experience within the program/project management and systems engineering base in the robotic spaceflight program, the committee concluded that these concerns apply to both the human spacecraft and the robotic spaceflight programs at NASA.6
The committee struggled with the question of whether the requirement for 250-350 highly skilled program/ project managers and systems and integration engineers in the short term constituted a shortfall in NASA’s workforce. The problem for the committee was that the numbers presented by NASA in its Workforce Strategy did not indicate the amount of experience desired by the agency for each of the positions.
INDUSTRY PERSPECTIVE
To gain a perspective on this issue, the committee sought input from an industry consortium. A sampling of representatives of 25 member companies of the Aerospace Industries Association (AIA) expressed specific concerns regarding NASA’s systems engineering and program management experience, the fact that many NASA engineers
TABLE 4.1 Short- and Long-term Challenges in Attracting Properly Skilled Aerospace Workers, as Identified by AIA Industry Representatives
|
Challenges |
|
NASA |
Industry |
|
Short term |
|
|
Long term |
|
|
do not have program start-up experience, and the risk that delays in the Orion program would likely increase attrition if its workers chose to leave NASA to work on more active projects in industry. For the longer term, i.e., after Orion development, there was concern about a lack of appropriately experienced engineers at NASA and NASA’s ability to compete with compensation levels in industry at the more senior levels.
Although there was a sense among the industry representatives sampled that the needed workforce was currently available in industry, there was nevertheless concern that industry also probably lacks depth in some key skill areas. Industry was seen as having an advantage in being able to move workers between the national security, civil government, and commercial sectors to meet varying needs. For the longer term, industry representatives indicated that meeting workforce needs will be influenced by whether the Vision for Space Exploration is compelling and its funding stable enough to attract young workers and minimize attrition rates.
The AIA group of aerospace industry representatives who provided input to the committee identified the challenges for NASA and industry outlined in Table 4.1.
NASA RECRUITING CHALLENGES
In the near term, NASA is pursuing the hiring of relatively senior aerospace industry personnel—and in some cases DOD personnel—to address its shortfall of highly skilled large-program managers and systems engineers. Several challenges face NASA in its recruitment of senior personnel, including competition with industry for the same skills; NASA’s inability to pay competitive salaries at the senior levels; uncertainty over continued political and funding support for the Vision for Space Exploration; and conflict-of-interest limitations on flexibility in assignments for senior managers recruited from the aerospace industry.
Although NASA’s pay scale is competitive with industry’s at the entry and middle levels, industry senior-level compensation packages, including recruitment bonuses, far exceed federal government pay scales. Congress responded by approving the NASA Flexibility Act of 2004, which provided for expansion of NASA’s authority to offer employee recruitment and retention bonuses. The committee believes that the act has helped but that NASA has not yet taken full advantage of it, and the committee encourages NASA to continue to exploit the act to its fullest limits. The committee believes, however, that the act cannot completely alleviate many of the difficulties apparent when all senior government positions are compared with equivalent positions in industry, such as stock and retirement packages for senior executives in industry that can range into the tens of millions of dollars.
Historically, NASA’s exciting mission has provided the best competitive edge for NASA’s ability to attract senior personnel. Given the pressures on NASA’s budget, the perceived threat to the sustainability of the VSE programs at meaningful levels through the next presidential election, and the attendant risk to an individual employee of leaving a secure position in an aerospace company with a broader variety of challenging programs, participation in NASA missions is not the attraction it once was, especially for senior-level personnel.
Shifts over the past 25 years in pension programs, from defined-benefit plans that encourage and reward long-term service to 401k pension programs that are portable, tend to tie workers, including federal government workers, less to a single employer. Moreover, in industry the provision of matching stock in 401k programs and employee bonuses in the form of stock options to senior personnel who possess 20-25 years of experience has meant that senior aerospace company program managers and systems engineers who are recruited into the federal government are limited in terms of allowable contractual and technical decision making and oversight roles for projects involving their former company. Given that there are now only three major human spaceflight contractors, this limitation affects NASA’s flexibility in using senior-level industry employees.
NASA’S IN-HOUSE PROJECT MANAGEMENT AND SYSTEMS ENGINEERING TRAINING PROGRAM
Neither NASA nor U.S. industry has in recent decades undertaken a human space system development program of the magnitude and breadth required for implementing the Vision for Space Exploration. Earlier large programs—Mercury, Gemini, Apollo, the Space Shuttle—accomplished more than each program’s immediate task by providing vital training and experience for NASA’s workforce. Although a more recent program, the ISS posed fewer high-risk technology challenges than those faced by earlier large programs, and it therefore depended more heavily on the contractor workforce for systems development, with the NASA workforce concentrating on the operational aspects of the program and thus having fewer hands-on development training opportunities. Having focused on different programs since the development of the shuttle was completed, NASA must now rely on coursework and virtual training programs to give personnel critical exposure to systems engineering and program management for human spaceflight systems.
NASA’s principal in-house training programs for project management and systems engineering are managed by the agency’s Academy of Program/Project and Engineering Leadership (APPEL) in the Office of the Chief Engineer.7,8 APPEL supports curriculum development and classroom training, providing a variety of opportunities that include retreat-style workshops lasting 1 or 2 days, in-house courses that last several weeks, and residential coursework at universities; knowledge sharing through agency publications and workshops utilizing highly skilled NASA practitioners and outside experts; independent assessments and consultations for projects, teams, and individuals; and applied research to develop advanced project management concepts relevant to NASA’s needs. APPEL also coordinates the annual Project Management Challenge event, which in 2006 attracted nearly 1,000 participants from NASA, industry, and academia. The conference is supported by the online magazine PM Perspectives.9 However, according to the experience of committee members, the program has been criticized for the lack of direct incentives for participation.
APPEL’s coursework strongly emphasizes case studies, including online video, audio, and interactive demonstrations.10 However, this set of case studies lacks any large human spaceflight systems examples. Although
7 |
|
8 |
In the late 1980s, NASA created the Program and Project Management Initiative (PPMI), a small office charged with providing coursework meant to promote project management excellence. The effort remained small for much of its history, often operating with only one full-time employee. At the time, NASA considered program management a skill adequately taught by NASA’s slate of active missions, and it placed little emphasis on the program: “The initial PPMI curriculum efforts were limited in scope to traditional training approaches, reflecting the status of Adult Learning theory and technology at the time…. In this era of a few very large programs, with an abundance of project expertise cultivated through the challenges of Apollo and Shuttle, such a strategy was both logical and desirable.” See http://appel.nasa.gov/node/12. PPMI evolved into the APPEL program in the 1990s. |
9 |
|
10 |
See, for example, http://appel.nasa.gov/case_studies/case_study_near/near_case_study.html. |
valuable lessons can be learned from case studies involving notable science missions such as Viking and NEAR, or from experimental piloted aviation programs, they cannot fully address what is needed for the multiple, parallel human spaceflight systems development programs required by the Vision for Space Exploration.
The various APPEL programs draw instructors from current and former senior NASA experts, academic professionals, and expert consultants. APPEL is in the process of developing a new, four-level, core curriculum for technical professionals in NASA by which an engineer or a manager can progress through a combination of multi-week courses and on-the-job experiences.11 According to NASA officials, the program, which will stress an integrated approach to project management and systems engineering, has been slated for initiation in FY 2007.
APPEL also has developed a systems engineering “competency framework” that arrays lists of knowledge, skills, and behaviors that systems engineers should possess under 10 major competency areas—e.g., systems design, technical management, and project management and control—and defines four levels of proficiency that specify the degree of knowledge or performance that is expected of systems engineers for different levels of responsibility.12 A matrix of competencies and proficiency levels is intended to be used to help determine specific training needs for individuals and organizations and to help guide planning for professional development for systems engineers. NASA has a similar matrix to facilitate the professional development of project managers.
The committee is concerned that at the same time NASA is seeking to acquire and develop the in-house technical capability that NASA’s administrator, and the committee, believe is vital to successfully implementing the Vision for Space Exploration, the agency is simultaneously reducing its primary in-house training programs for project management and systems engineering. The committee believes that this trend must be reversed.
IMPROVING THE SUPPLY OF HIGHLY SKILLED PERSONNEL: THE ROLE OF THE EXPLORER PROGRAM, BALLOONS, AIRCRAFT, AND SOUNDING ROCKETS
The committee was informed by various experts that when it comes to developing systems engineering and project management skills, there is ultimately no substitute for hands-on training. Coursework is vital, but just as it is impossible to become skilled at baseball in a classroom, so also is it impossible to learn the necessary skills for managing and integrating complex spacecraft without actually working on them.
The importance of hands-on experience has been recognized often throughout NASA’s history. In 1991, the National Academy of Public Administration completed a study of the distribution of NASA science and engineering work between NASA and contractors and described how that distribution might affect NASA’s in-house capabilities.13 Among the questions that the study considered were whether NASA had contracted out too much of its technical work to remain a “smart buyer,” whether NASA’s in-house technical capability had eroded over time, whether hands-on work was important to the development of a competent science and engineering workforce, and whether there were enough hands-on opportunities for NASA’s workforce.
The NAPA report indicated that the balance of expertise had shifted away from NASA to contractors in a number of discipline specialty areas, including systems engineering, and consequently that NASA risked losing its ability to be a smart buyer and to properly manage technical work. The report cited evidence that fewer NASA scientists and engineers were working on hands-on R&D tasks and that there had been a shift to assigning more of the in-house staff to project management and operations support. This situation is remarkably similar to the situation that NASA officials have said they face today.
Although human and robotic systems are distinct, they do share many management and engineering processes as well as system technical characteristics. Given that the bulk of the development activities over the past 10 years have been in robotic spacecraft, NASA needs to leverage the robotic spacecraft workforce skill development opportunities to meet some of the human spaceflight program development skill needs. The committee believes
that systems engineering methodology and technical skills acquired from complex robotic spacecraft development can serve as an important base for the transition to systems engineering of human spacecraft.
High-quality systems engineers and program or project managers cannot be trained entirely by universities, or by NASA’s APPEL program; the skills required to become truly expert in those positions can only be acquired through on-the-job experience. However, NASA can take steps to help both university students and entry-level employees start along the road toward becoming a program manager or a systems engineer by increasing the number of development programs.
In addition to developing small spacecraft, NASA conducts several activities that are useful for providing hands-on experience to younger personnel. One of these activities is the agency’s Explorer program. NASA’s Explorer program consists of independent, robotic space missions designed to target a wide range of fields, including atmospheric, solar, and cosmic research. These missions are managed by the Explorer Program Office at Goddard Space Flight Center and are characterized by their moderate cost and shorter development time relative to the larger space observatories. Since inception of the program in 1958, 79 of 83 Explorer missions have been successful, and many have resulted in significant discoveries. However, the FY 2007 budget reduces the Explorer program by 20 percent, with no launches between 2009 and 2012.14
NASA has long supported several flight research programs that utilize specially equipped aircraft, high-altitude balloons, and suborbital sounding rockets to carry research instruments aloft for periods of time ranging from minutes to hours to days. The programs are notable for their relatively low costs compared with those for spaceflight missions, the relatively short turnaround time from payload design to flight (or reflight), and their relatively high risk tolerance compared with spaceflight missions. All of these attributes make suborbital projects especially valuable methods for giving junior-level scientists and engineers firsthand experience with mission systems design, development, testing, and operations (see Figure 4.1).
The current high-altitude balloon program provides about 20 research flights per year to altitudes above 30 km for flight durations of hours to days to as much as several weeks. The balloon program, which can carry research payloads of more than 1,000 kg, is utilized by researchers in atmospheric science, astronomy, high-energy astrophysics, and solar and space physics. According to NASA officials, the current balloon program involves about 40 institutions, 200 scientists and engineers, 25 graduate students, and 50 undergraduates. NASA balloon flight opportunities have decreased over the years, although individual flight durations have increased and the payloads have become more sophisticated, essentially “mini-satellites” in themselves, complete with solar panels, stabilization and communications systems, and instruments. However, the lower number of flights means fewer opportunities to fly payloads and gain experience (see Figure 4.2).15
The sounding rocket program provides opportunities to carry payloads in the range of several hundred kilograms to altitudes of hundreds of kilometers for flight durations of a few to more than 20 minutes (see Figure 4.3). Sounding rockets can carry stabilized platforms to facilitate the pointing of astronomical telescopes, and they also provide important opportunities for other types of research measurements in aeronomy, space plasma physics, and astrophysics. The program currently supports about 10 principal investigators per year at an annual flight rate of 10 to 20 flights per year.
The suborbital programs are particularly suitable means to provide less highly skilled engineers and scientists with direct hands-on experience with the full range of space mission tasks, including mission definition, design, development, flight operations, and data analysis. Thus these types of programs constitute an ideal medium for gaining skills that are important for systems engineers and project managers on larger spacecraft. The suitability of suborbital projects as training opportunities is enhanced by the fact that they can be conducted in relatively short periods of time (typically 1 to 3 years) and at relatively low cost and with a greater degree of tolerance for mission risk.
The number of suborbital flight opportunities has fallen dramatically over the past two decades. Figure 4.4 shows the annual number of sounding rocket flights from 1959 through 2005. The sounding rocket launch rate peaked at above 150 per year in the late 1960s, and it has subsequently fallen to less than 1/6 that launch rate now.
(Similarly, balloon launches occurred at a rate of 70-90 per year in the 1970s but have fallen to fewer than 20 per year.) Several factors have contributed to the drop in the number of suborbital programs, including increased complexity and capability (and attendant costs) of vehicles and payloads, increased operating costs, and reductions in program budgets.
The committee believes that the Explorer program and its Earth Probe and Mars Scout counterparts, and balloon, aircraft, and sounding rocket flights should not be viewed by the agency simply as competed science projects, but also as internal and external training programs to develop expertise that is specifically needed by NASA. The agency should reinvigorate these programs. The committee believes that this can be accomplished with a relatively small amount of money that can be reallocated within the agency’s current budget. Giving frequent flight opportunities to younger members in the NASA workforce—both in the civil service and in academia—will improve the skills that will be useful to the agency in future years.
Finding 4: There is a longstanding, widely recognized requirement for more highly skilled program/project managers and systems engineers who have acquired substantial experience in space systems development. Although the need exists across all of NASA and the aerospace industry, it seems particularly acute for human spaceflight systems because of the long periods between initiation of new programs (i.e., the Space Shuttle program in the 1970s and the Constellation program 30 years later). NASA training programs are addressing some of the agency’s
requirements in this experience base, but the current requirement for a strong base of highly skilled program/project management and systems engineering personnel, and limited opportunities for junior specialists to gain hands-on space project experience, remain impediments to NASA’s ability to successfully carry out VSE programs and projects.
Recommendation 4: Provide hands-on training opportunities for NASA workers.
The committee recommends that NASA place a high priority on recruiting, training, and retaining skilled program/project managers and systems engineers and that it provide the hands-on training and development opportunities for younger and junior personnel required to establish and maintain the necessary capabilities in these disciplines. Specific and immediate actions to be taken by NASA and other parts of the federal government include the following:
-
In establishing its strategy for meeting VSE systems engineering needs, NASA should determine the right balance between in-house and out-of-house work and contractor roles and responsibilities, including the use of support service contractors.
-
NASA should continue and also expand its current employee training programs such as those being conducted by the Academy of Program/Project and Engineering Leadership (APPEL). To facilitate the development of key systems engineering and project management skills, NASA should increase the number of opportunities for entry-level employees to be involved in hands-on flight and end-to-end development programs. A variety of programs—including those involving balloons, sounding rockets, aircraft-based research, small satellites, and so on—can be used to give these employees critical experience relatively early in their careers and allow them to contribute as systems engineers and program managers more quickly.