|
Items in cart [4] |
Questions? Call 888-624-8373 |
PAPERBACK + PDF PAPERBACK PDF BOOK PDF CHAPTERS Free Resources |
||||||||||||||||
|
||||||||||||||||
|
|
|||||||||||||||
| ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 130
5
Organization and Management
of Large-Scale Biomedical
Research Projects
' n the fields of biology and biomedical research, formal management
of projects and staff traditionally has not been a major topic of con-
~ cern, nor has it been widely studied. Training in management prac-
tices has been quite rare for Ph.D. candidates, and direct assessment of an
investigator's managerial skills has played little or no role in promotion
decisions or in the review of grant applications. The traditional structure
of academic research laboratories, consisting of a single, independent
principal investigator who oversees a small number of trainees (graduate
students and postdoctoral fellows) and technicians, has been thought to
present little need for hierarchical or formalized management methods.
Furthermore, managerial oversight of investigator-initiated research by
funding organizations has been minimal or nonexistent. With the advent
of larger-scale projects that involve more scientists and larger budgets,
however, effective management, both scientific and administrative, has
become more important. This is especially true when multiple principal
investigators and multiple institutions join forces to pursue a common
mission or goal. In such collaborative efforts, it can be quite challenging
to ensure that all the components of a project fit together and work effec-
tively toward a collective goal. Project leaders must strive for a common
vision and cultural integration among the various participants, who may
include scientists and managers from different disciplines or different
sectors, such as academia, industry, and government agencies.
Unfortunately, there is little information to guide the establishment of
good managerial practices in such cases. This is due in part to the variabil-
130
OCR for page 131
ORGANIZATION AND MANAGEMENT
131
ity of research programs and goals, which makes it difficult to set uniform
guidelines. In addition, the management of science, even in large-scale
projects, has not been widely studied or assessed. Indeed, even in such
fields as high-energy physics, in which large-scale, multi-institutional col-
laborations have been the norm for decades, the issue of research man-
agement has garnered little attention from scholars and remains a con-
cern for scientists (National Research Council, 2001b). According to the
American Institute of Physics (1992) "without a dedicated effort to under-
stand [these complex] collaborations, policy makers and administrators
will continue to have only hearsay and their own memories to guide their
management. . . " (page 3~.
The issue of research management is now coming to the fore because
of increased demand by the government to account for the way federal
funds are being used (see Chapter 4~. The current Bush Administration is
adamant about applying performance standards to assess the manage-
ment and productivity of both large and small research endeavors. Ac-
cording to John Marburger, Director of the White House Office of Science
and Technology Policy, performance measurement is "an inevitable as
well as an essential aspect" of the relationship between the government
and scientific research. In particular, he notes that "individuals depen-
dent on large facilities bear the heavy responsibility of making judicious
choices, ensuring prudent management and optimizing the quotient of
discovery versus dollars" (Hafner, 2002: page 1~.
EXAMPLES OF MANAGEMENT ASSESSMENT FOR
LARGE-SCALE PROJECTS
Assessment of Federally Funded Laboratories
A recent study of federally funded U.S. research and development
(R&D) laboratories examined their structure, management, and output
through surveys and case studies, and revealed a great variety of organiza-
tional designs (Crow and Bozeman, 2001~. Although a large portion of the
laboratories examined were not performing biomedical research, there may
be some applicable lessons to be learned from the study results. The au-
thors of the study found that a proliferation of large research centers after
the 1980s had resulted in new institutional and organizational designs. For
example, there are now more collaborative research facilities and multi-
sector centers, such as university-industry partnerships. In addition, there
are more core user facilities and equipment- or process-driven centers. Tech-
nology development and technology transfer are also more common.
These newer types of research facilities and collaborations required a
concomitant change in the approach to management. Before the emer-
OCR for page 132
32
LARGE-SCALE BIOMEDICAL SCIENCE
gence of research centers, most federally funded research was undertaken
on a smaller scale by individual investigators conducting discipline-
oriented projects. As a result, scientists who became center directors often
lacked management experience, and some large-scale organizational
needs suffered as a result. Furthermore, there appeared to be few mecha-
nisms for diffusing managerial knowledge, and almost no incentive to do
so. Rather, the authors of the study concluded that successful manage-
ment approaches emerged from blind variation and selective retention
(Crow and Bozeman, 2001~. The study found that the partition of center
management and scientific leadership (analogous to the chief scientist in
industry) was quite effective. It also revealed that selecting research direc-
tions through a combination of traditional peer review and a nontradi-
tional emphasis on building research capacity worked well.
With regard to funding, the authors concluded that the stability of
funding is often more important than the actual amount of funding, as it
provides a core for long-term planning and the development of support
systems. Stable funding also facilitates "capacity evaluation" rather than
"output evaluation." The distinction is important because government
funding managers generally need to think in terms of projects and grants,
but research managers often think in terms of resources and work activ-
ity. This divergence leads to a conundrum of trying to maintain the effec-
tiveness of competitive peer review without stifling the productivity of
research centers. Furthermore, determining a program's value within the
environment of the Government Performance and Results Act (GPRA) is
difficult when there are so many diverse contributions.
Evaluation of the National Science Foundation's Science and
Technology Centers Program
Lessons may also be learned from an evaluation of the National Sci-
ence Foundation's (NSF) Science and Technology Centers (STC) program.
As described in Chapter 3, the STC program funds large-scale collabora-
tive research that often is multidisciplinary and has broad, long-term
goals. An extensive evaluation, including an assessment of organization
and management, was conducted about 10 years after the program was
initiated (National Academy of Public Administration, 1995, ABT Associ-
ates, 1996; National Research Council, 1996~. A study panel appointed by
the National Academies concluded that the success of the Centers is highly
dependent on both their scientific and administrative management.] Be-
cause the Centers vary widely in their scope, objectives, research foci,
1 The evaluation was based on site-visit reports, a survey of and interviews with the Center
directors, and a previous report from the National Academy of Public Administration.
OCR for page 133
ORGANIZATION AND MANAGEMENT
133
appropriate institutional linkages, and other characteristics, their man-
agement and organizational structures are also quite varied. However,
effective oversight of the research programs present several common chal-
lenges for both Center directors and NSF program managers.
The National Academies panel found that a major challenge for the
Center directors is to ensure that their Centers embody real collaboration
and are not just groups of independent scientists working in a related
area. A challenge for program leaders is to maintain focus over time. In a
rapidly evolving field, for example, it can be difficult even for a successful
Center that is meeting its initial goals to shift its focus in response to the
field's natural evolution (e.g., from a basic to a more applied orientation
or from one scientific emphasis to another). Moreover, any given Center
may not be well constituted to make such large changes and remain suc-
cessful. One of the greatest difficulties for NSF managers is ensuring that
review and monitoring processes are effective. The panel concluded that
site review by committees that include expert peers is very important,
particularly in the first few years of a new program. The periodic site-
review process was deemed very helpful in several cases when manage-
ment problems occurred, as it assisted program leaders in identifying the
problems and developing solutions (National Research Council, 1996~.
SPECIAL CONSIDERATIONS FOR THE MANAGEMENT OF
LARGE-SCALE BIOMEDICAL RESEARCH PROJECTS
Large-scale science clearly requires good management schemes and
good managers. But what makes for a good manager, and what defines
good management? Of course, there is no single response to this question,
as the answer will vary depending on the project goal and the methods
used to achieve that goal, both of which can be highly diverse. For ex-
ample, the managerial needs of a large-scale project designed purely for
the purpose of collecting data and creating a database to be used as a
research resource may be quite different from those of a large-scale col-
laborative project addressing a complex research question. In general,
project management entails four basic components:
· Setting goals and objectives
· Establishing a timeframe
· Planning, orchestrating, and coordinating activities to achieve the
goals within that timeframe
· Evaluating progress toward the stated goals
However, the size, cost, complexity, and visibility of large-scale proj-
ects generate unique or heightened concerns and therefore demand
greater stewardship and accountability than are characteristic of tradi-
OCR for page 134
34
LARGE-SCALE BIOMEDICAL SCIENCE
tional small-scale projects.2 Potential management problems tend to be
proportional to the size of the project, and even apparently minor deci-
sions could have major, precedent-setting implications for a large-scale
project. As a result, planning and oversight become more labor- and
time-intensive for the grant recipient as well as the funding agency, and
thus the skills and commitment of both scientific and administrative
managers for such projects are critical. For example, the funding agency
must develop succinct and unambiguous terms for the award. Other-
wise, it could be difficult to suspend or terminate a large-scale project
once it has been launched, because of the visibility, politics, and sheer
complexity of the undertaking. Agency staff must also define clearly the
plans for monitoring and evaluating progress toward short-term mile-
stones and long-term objectives, including potential actions to take when
adequate progress is not being made, while still allowing enough flex-
ibility to adapt to change as the work progresses. Oversight of many of
the models described in Chapter 3 involves steering committees and
advisory groups that include scientists who are well-respected peers in
the field but are not directly involved in the projects. The extra respon-
sibilities of the grant applicants include developing detailed, long-range
plans to justify the large budget and the commitment of the funding
agency. In fact, large-scale projects may require planning beyond a 5-
year timeframe. Such long-range planning is extremely difficult in rap-
idly changing fields, and such timeframes are essentially unheard of
even in the corporate world, where strategic planning is commonly un-
dertaken (National Research Council, 1998~.
When a large-scale project is carried out at multiple institutions or is
funded by multiple sources, the complexities and difficulties associated
with planning, coordination, monitoring, and assessment are exacerbated.
Federal, industrial, academic, and nonprofit participants may each have
their own priorities and ideas for how best to achieve their goals. Each
funding source may also have different requirements for oversight or
different stipulations for how to handle data release and intellectual prop-
erty issues. Even when funding comes from multiple federal agencies, or
perhaps even multiple Institutes within NIH, there can be disagreements
over the roles and contributions of the various funders. This was certainly
the case in the early efforts to launch the Human Genome Project, when
the Department of Energy (DOE) and NIH were competing for funds and
control of the project (reviewed by Davies, 2001; Cook-Deegan, 1994;
Kevles and Hood, 1992~. The leaders of a project must be able to commu-
2 From STEP Administrative Strategies Forum: Big Science: Big Challenges, March 1,
2002, Bethesda, MD, National Institutes of Health.
OCR for page 135
ORGANIZATION AND MANAGEMENT
135
nicate its vision in order to foster teamwork within the project and accep-
tance by the field as a whole.
Encouraging and maintaining open and effective communication
among a project's various team members is also inherently challenging
for large-scale, multi-institutional projects, especially when more than
one discipline is involved. Participants must be able to speak the same
language and need to trust each other enough to discuss ideas and work
in progress. Several of the models described in Chapter 3 include regu-
lar meetings of the scientists working within collaborative projects, and
this approach appears to be quite useful for facilitating good communi-
cation.3 Such forums might be useful for collective decision making
within large-scale projects as well. Advances in information technology
(the Semantic Web, for example) may also facilitate communication
within collaborative or multidisciplinary projects (Hendler, 2003~.
Because the time commitment for principal investigators leading large-
scale initiatives is likely to be much greater than is the case for more tradi-
tional projects (Mervis, 2002; Sulston and Ferry, 2002), it is often necessary
to hire managers to oversee the day-to-day work of such a project. In
academia, there appears to be a preference for managers with strong re-
search credentials rather than strong management experience. However,
there is no correlation between a person's abilities as a scientist and as a
manager. Scientists are rarely trained to be managers, but it can also be
argued that traditional business management training programs are not
very applicable to the management of science because they are not adaptive
enough (Austin, 2002~. Conventional project management methods work
best when the chances are good that a project will progress as expected. In
contrast, science projects entail discovery and thus are more likely to re-
quire cyclical or iterative planning.
Effective scientist managers must have both technical and concep-
tual knowledge of the science involved in a project, in some cases in
multiple disciplines, as well as good people skills, good judgment, and
flexibility. The same is true of program managers within funding agen-
cies. However, finding qualified individuals to take such positions can
be difficult for a variety of reasons. Within academia, credit for a suc-
cessful project may be given primarily to the principal investigator, even
if the project manager has assumed significant responsibility. Further-
more, project managers, both in government agencies and in academia,
do not necessarily have a sense of ownership of the data or other prod-
ucts of a project. Thus, taking on such a position could be a risky career
3 Carol Dahl, former director of NCI's Unconventional Innovations Program, in a presen-
tation to the National Cancer Policy Board, July 16, 2002.
OCR for page 136
136
LARGE-SCALE BIOMEDICAL SCIENCE
move. Similarly, project management staff within federal agencies may
not be viewed as scientific peers and may not be willing to assume the
risks associated with managing a large-scale endeavor. Indeed, people
with the right skills and qualifications for management positions are
likely to find industry more appealing because of the career structure,
incentives, and rewards for successful completion of a projec* (see the
section below for more detail). To address these issues, the National
Laboratories use an alternative model encompassing dual career lad-
ders that recognize and reward the achievements of managers who may
not have the scientific credentials of top-tier researchers (Crow and
Bozeman, 2001, see previous section). The National Laboratories also
have a long history of managing large-scale projects for academic inves-
tigators and for rewarding scientists for their participation in team-
oriented research.
THE INDUSTRY MODEL OF PROlECT MANAGEMENT:
COMPARISON WITH ACADEMIA
The details of management approaches vary greatly depending on
many factors, such as the environment in which the work is being done
and the nature of the desired outcome. Historically, the approaches used
most commonly in industry and academic settings are quite different.
In fact, the quintessential academic research project involves rela-
tively little formal management beyond the individuals doing the work.
The laboratory head or principal investigator is responsible for obtaining
research funding through proposals that outline the objectives, methods,
and expected timeframes of the project. He or she then oversees the work
of one or a few graduate students, postdoctoral scientists, or technicians
who perform the experiments. There is little or no oversight of the project
by department or university officials or by officials of the funding agency.
The overall work and productivity of the individual principal investiga-
tor are reviewed by university or department officials infrequently, such
as when decisions regarding tenure or promotion are made. There is great
variation across institutions in how the work of graduate students is
evaluated; in the case of postdoctoral scientists, a recent survey indicates
that most academic institutions do not require written performance evalu-
ations or progress reviews (National Resource Council, 2000~.
In contrast, most research undertaken in an industry setting involves
4 Carol Dahl, former director of the NCI's Unconventional Innovations Program, in a
presentation to the National Cancer Policy Board, July 16, 2002.
OCR for page 137
ORGANIZATION AND MANAGEMENT
137
a team effort in which many investigators share similar levels of responsi-
bility for bringing a single project to successful completion. There are
many more layers of oversight and supervision, and everyone must work
together toward a common goal. Progress is measured against written
goals, a practice that promotes good planning and keeps everyone in-
formed about what is expected of them. Generally, all members of the
team are formally reviewed on an annual basis using a numerical ranking
system that determines pay scales and advancement and is designed to
elicit improvements from staff.5 Such review may entail traditional top-
down assessment of employees by their immediate supervisors. More
recently companies have also been using another form of staff review
known as "360 review" (see Box 5-1) a method for assessing teamwork
in which an employee's performance is evaluated by everyone in the
circle that surrounds him or her, including peers, supervisors, and those
who work for the employee (Edwards, 1996~.
A significant obstacle to undertaking large-scale, collaborative proj-
ects within academia may be the inability of the current academic system
to assess the work and productivity of individual team members and to
reward those who make a significant contribution to a large-scale effort.
For such work to be valued and respected, the criteria used for tenure,
promotion, and hiring within academia would need to be changed or
expanded to include a wider range of scientific achievements. A shift in
emphasis away from measuring a scientist's success in obtaining tradi-
tional ROl-type grants and toward an evaluation of research output and
research capacity in the form of collaboration networks could facilitate
such change. Perhaps the greatest obstacle to implementing this concept
would be changing the mind-set of the reviewers who make decisions
about promotions and tenure. However, there is at least some precedent
for this approach in academia in the review of program grants (e.g., POls),
where the total research effort is expected to be greater than the sum of its
individual components (see Box 4-7~. Nonetheless, industry is still likely
to have many more options at its disposal for recognizing and rewarding
the work and contributions of team scientists through bonuses, pay in-
creases, and opportunities for advancement within the company. Such
career issues are discussed in greater detail in the following chapter.
5 In some cases, a minimum or maximum number of employees must be assigned to the
highest and lowest rankings to ensure that the ranking system is meaningful. For example,
if a scale of 1-5 is used, with 1 being low and 5 being high, a group may be limited to no
more than 15 percent of staff ranked as 5s while being expected to have a minimum of 5
percent of staff designated as Is or 2s.
OCR for page 138
138
LARGE-SCALE BIOMEDICAL SCIENCE
SUMMARY
The capacity of large-scale biomedical research projects to make
innovative and novel contributions to the field depends on their organi-
zational structure and oversight. Because of their organizational complex-
ity, cost, and visibility, large-scale projects have greater needs for man-
agement and oversight than is the case for more traditional biomedical
research projects. Effective administrative management and scientific
leadership are crucial for meeting expected milestones on schedule and
within budget; thus the success of a large-scale project is greatly depen-
dent upon the skills and knowledge of the scientists and administrators
managing the project. Scientific managers must be well versed in the
technical and conceptual aspects of the project, which may be multidiscipl-
inary, and must also have exceptional organizational and communication
skills to facilitate collaboration. However, it may be quite difficult to re-
cruit scientists with the needed skill set into managerial positions because
of the unusual status of such positions within the scientific career struc-
ture, and because scientists rarely undergo formal training in manage-
ment. Furthermore, there is little information available on how to struc-
ture such management and oversight, and there are few precedents to
follow in biomedical research.
OCR for page 139
ORGANIZATION AND MANAGEMENT
139
To pursue large-scale endeavors in biomedical research effectively
and efficiently, then, both universities and government agencies will need
to develop incentives to encourage qualified scientists to take on the risks
and responsibilities of managerial positions. Doing so could entail new
approaches for assessing teamwork and management, as well as novel
ways of recognizing and rewarding accomplishment in such positions.
Both industry and the National Laboratories may serve as instructive
models in achieving these goals, as they have a history of rewarding
scientists for their participation in team-oriented research. Universities
may need to define new faculty and staff categories that are consistent
with this type of research, along with appropriate criteria for performance
evaluation and promotion.
One attempt on the part of NIH to facilitate interdisciplinary team-
work is being undertaken by the Bioengineering Consortium (BECON),
one of the few organizational units at NIH that crosses all Institutes and
Centers. BECON is currently organizing a symposium called "Catalyzing
Team Science," aimed at producing a set of guidelines for NIH on how to
stimulate, facilitate, and reward collaborative efforts. The workshop will
also include a discussion of academic institutions' assessment and reward
procedures. It would also be expedient for NIH to formally assess the
organization and management of its ongoing large-scale projects, as the
National Science Foundation has done in the past. Such an exercise could
perhaps lead to the formulation of guidelines for organizing and manag-
ing future large-scale projects more effectively or for assessing the man-
agement structure of proposed projects.
Representative terms from entire chapter:
center directors
