The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
4
Characteristics and Metrics for a World-Class Army Research, Development, and Engineering Organization
Working from the discussion of pillars in Chapter 3, the committee developed characteristics and descriptions of the metrics associated with them.
CHARACTERISTICS
For the purposes of this study, characteristics are the distinguishing qualities, properties, or features of the pillars. The previous discussion of the five pillars ended with Figure 3-1, which designated 25 important features. These features characterize the pillars, and the committee believes they represent the distinguishing qualities of each pillar. Accordingly, the committee judged these 25 features to be the characteristics most relevant to an Army RD&E organization. A discussion of these characteristics and how they can be assessed appears below. The discussion follows a pillar-by-pillar format, with particular emphasis on determining metrics for evaluating the characteristics of each pillar.
Customer Focus Pillar
Customer focus is directed toward internal customers (e.g., product development teams) and external customers (e.g., soldiers). Both groups of customers can be surveyed to ascertain their satisfaction with the technological solutions and products delivered, the timeliness of delivery, and the quality of technical capabilities and support provided. Customer involvement in setting program objectives and following program progress can also be evaluated.
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole. Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 31
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
4
Characteristics and Metrics for a World-Class Army Research, Development, and Engineering Organization
Working from the discussion of pillars in Chapter 3, the committee developed characteristics and descriptions of the metrics associated with them.
CHARACTERISTICS
For the purposes of this study, characteristics are the distinguishing qualities, properties, or features of the pillars. The previous discussion of the five pillars ended with Figure 3-1, which designated 25 important features. These features characterize the pillars, and the committee believes they represent the distinguishing qualities of each pillar. Accordingly, the committee judged these 25 features to be the characteristics most relevant to an Army RD&E organization. A discussion of these characteristics and how they can be assessed appears below. The discussion follows a pillar-by-pillar format, with particular emphasis on determining metrics for evaluating the characteristics of each pillar.
Customer Focus Pillar
Customer focus is directed toward internal customers (e.g., product development teams) and external customers (e.g., soldiers). Both groups of customers can be surveyed to ascertain their satisfaction with the technological solutions and products delivered, the timeliness of delivery, and the quality of technical capabilities and support provided. Customer involvement in setting program objectives and following program progress can also be evaluated.
OCR for page 32
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Although an RDEC must focus on the primary markets it serves, the committee believes that some market diversification (i.e., looking to related markets for technology or products) is proper for any RD&E organization. Indeed, in the private sector world-class RD&E organizations seek to exploit fully the results of their research and product development. Market diversification has merit especially for an Army RD&E organization that has DoD-wide or even broader responsibilities. The extent of market diversification can be determined by examining such things as (1) the penetration and extension of markets for products and technologies and (2) whether these markets lead to expanded or entirely new lines of business. However, the committee notes that market diversification must be considered carefully in the case of Army RDECs because they exist primarily to support their Army missions. Also, RDECs rely on government funding, which is usually authorized to satisfy specific government needs rather than to diversify markets served by individual organizations.
Measuring customer satisfaction and customer involvement in research, development, and engineering, and the nature and appropriateness of market diversification, indicates how well the RDEC is connected with and focused on the long-term and short-term needs of the various customers the RDEC serves.1
Resources and Capabilities Pillar
Resources and capabilities can be evaluated in terms of personnel (i.e., human resources); facilities and infrastructure (i.e., physical resources); budget (i.e., financial resources); RD&E capabilities, skills, and talents (i.e., intellectual resources); the use of external resources; important technologies for each directorate; applied information technology; and organizational climate. Internal (e.g., staff and management) and external (e.g., peer, higher headquarters, and customer) reviews can be conducted periodically to assess resources and capabilities. These reviews may include analyses of the core technical programs, evaluations of employee morale and the research climate,
1
Other indicators can be used to assess customer focus (e.g., an organization 's success at anticipating unstated customer needs or how well an organization “hears” the voice of the customer in determining overall direction). However, the committee decided that, for an Army RDEC, it is preferable to concentrate on satisfaction, involvement, and diversification.
OCR for page 33
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
and assessments of the ability and effectiveness of program managers to acquire technology from outside vendors (i.e., reach “make versus buy” decisions). Measuring the quality and quantity of the human, physical, and financial resources and the core capabilities of the RDEC gives an indication of the ability and power of the RDEC to achieve world-class results. A positive organizational climate is most often correlated with high productivity (Miller et al., 1996).
Strategic Vision Pillar
The strategic vision must be shared with (i.e., communicated to and discussed with) and understood by staff and stakeholders alike. The vision must then be translated into action. How well this is done depends on the quality of the leadership of the RDEC. Reviews to assess strategic vision should include assessing RDEC leadership. Internal and external (e.g., peer) reviews can determine if the strategic vision and the mission are aligned. They can also determine if anticipatory strategic planning is sufficient to develop future Army and joint service products rapidly. To assess stakeholder buy-in, important classes of stakeholders must first be identified; then the extent to which buy-in by particular stakeholders is critical and the degree to which buy-in is obtained can be assessed. Measuring the quality of the strategic vision of the RDEC (i.e., the extent to which the vision and mission are aligned, to which planning is anticipatory, and to which stakeholders “live” the vision) will give a reading of the enduring capability of the RDEC to plan and achieve world-class results.
Value Creation Pillar
Value creation is often a perception. Perceived values are based on comparisons of the benefits (or lack thereof) of previous products with current products or of the properties of products and costs. Another perception is whether the right products are being delivered to the right place at the right time. Reviews of the breadth of RD &E (i.e., the portfolio of programs), the performance of products and the benefits of services, cycle time and responsiveness, and the value of work in progress are all important to assessing value creation (Miller et al., 1996). Reviews can be both internal and external evaluations of programs in progress, products developed, or services delivered. Measuring the
OCR for page 34
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
extent to which the RD&E organization produces outstanding, meaningful results (i.e., creates value for customers) yields an understanding of the present and potential impact of the organization.
Quality Focus Pillar
Several important characteristics are associated with quality focus. First is a capacity for scientific, technological, and engineering breakthroughs. This characteristic can be assessed, in part, by reviewing past performance (e.g., how many breakthroughs have already occurred). Next is the ability to improve continuously, which can be assessed by reviewing specific efforts to improve processes and products. Commitment to quality products and services must be assessed at all levels, from topmost management to the lowest working level. To improve the quality of work, work processes must first be understood, defined, and (to a degree) structured. Structured processes (i.e., working in a disciplined and organized fashion) can be assessed by examining processes and results. Reviews can determine the ability of staff members and the organization as a whole to learn, acquire knowledge (and capabilities), and use this knowledge to achieve outstanding results. Finally, the quality of research can be assessed by expert reviews on several levels (e.g., to determine whether high standards of technical excellence are being maintained). Measurements of all these characteristics can give an overall assessment of the focus on quality in an Army RD&E organization.
METRICS
For the purposes of this study, metrics are defined as standards for measuring the characteristics of each pillar. Before applying metrics in a particular situation, it is necessary to understand some aspects of metrics in general. This understanding is revealed by the answers to several questions.
Who, What, Why, When, and How
Metrics can be used in many ways for evaluating an RD&E operation (IRI, 1996). Long lists of metrics have been developed, but selecting
OCR for page 35
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
the most pertinent metrics is important. Critical questions can be asked about using metrics, the answers to which are applicable to Army RDECs.
Who develops the metrics?
Metrics can be established by the involved group (e.g., an RDEC or its directorates) or by others interested in the performance of the group. The metrics must be understood by those making assessments and by those being assessed.
What specific metrics should be developed?
Different sets of metrics are meaningful to different groups. Metrics must be useful for the organization; specific metrics will drive the behavior and actions of people within the organization. For an RD&E organization, metrics should foster improvement and be related to the vision and mission of the organization.
When selecting metrics one must keep in mind that RD&E efforts are part of a system (Brown and Svensen, 1988). Figure 4-1 shows input into the system, which comes from various sources (e.g., personnel, dollars, equipment, scientific knowledge) and moves through RD&E processes to produce output (e.g., reports, patents, concepts). Product output is converted by receivers into outcome (e.g., new products, services, cost savings, benefits for soldiers). 2
Metrics can be developed to measure (1) the quality of input, (2) the operation of RD&E processes, (3) the quality of output, and (4) the value of outcome. Also, metrics can be developed that relate these four items to the past, present, and future. For example, variations on one measure of an RD&E organization's output might be the number of patents used in the last five years (past), number of patents filed (present), and the number of patent disclosures anticipated during the next five years (future).
2
The portrayal of RD&E as a system is not meant to exclude the myriad of interactions with external stakeholders, especially customers. The diagram is useful for isolating several key RD&E processes. However, from a larger perspective (e.g., an entire business), the RD&E system is only a subsystem.
OCR for page 36
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
FIGURE 4-1 Research, development, and engineering as a system.
Why use metrics?
Metrics can help an organization assess and determine business or technical objectives, encourage changes, and serve as mechanisms for planning, screening (e.g., setting priorities for improvement), and managing RD&E programs. Metrics can also measure contributions from management, research scientists and engineers, and support staff toward developing, producing, and delivering products.
When should metrics not be used?
The wrong metrics can be harmful. Some metrics may actually limit performance or lead to inappropriate behavior, actions, or results (e.g., metrics based on false cause-effect relationships or wrong work-process models).
When should metrics be developed?
Ideally, metrics should be developed as part of the process of setting objectives or part of the evaluation of how well objectives have been met.
OCR for page 37
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
How are metrics developed?
Metrics can be developed collectively or individually, and they can be tailored to match the unique environment of the group or organization being assessed.
Referring to metrics described by the Industrial Research Institute (IRI, 1996) and others (Roussel et al., 1991; Davidson and Prudent, 1996), the committee developed a set of metrics to assess Army RD &E organizations like the Natick RDEC. Four metrics were developed for each of the 25 characteristics identified at the beginning of this chapter.
Of paramount concern during the development of the metrics was the committee's desire to emphasize the importance of demonstrated commitment by the organization's senior leadership (and, where applicable, the next level of command) to attaining and maintaining world-class performance throughout the RD&E organization. Without this commitment, attaining world-class performance will be impossible.
Applying Metrics
The 100 metrics described below can be used to monitor improvements and assess the Natick RDEC in terms of world-class performance. The metrics can also be used by Natick RDEC or other RDEC personnel for self-evaluation or by higher-level Army commands for evaluating other RD&E organizations.
In other situations metrics are used to provide numerical measurements, such as the number of patents written per year or the number of Ph.D.s on staff. Although these measures clearly are relevant to the performance of an RDEC, the committee does not believe that numbers, by themselves, accurately describe the many facets of RDEC performance.3 Accordingly, the metrics chosen for this assessment are based on qualitative descriptors for four levels of performance (Adler
3
“Numerical indexes are easily understood and readily compared.... However, there are also some serious drawbacks associated with quantitative measures that counterbalance their virtues....They don't work well in professional groups, such as R&D organizations, where much of the work is characterized by uncertainty and variability…” (Brown and Gobeli, 1992).
OCR for page 38
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
et al., 1992).4 These levels are poor, adequate, good, and excellent.5 The committee believes for an organization to be deemed world-class, performance must be predominantly excellent.
The committee also considered the category of best-in-class, which is a level of performance beyond excellent. Best-in-class is not included in the following metrics because descriptors for this category apply to unique situations (e.g., an organization or process that is, indeed, the very best) and might not be applicable in a situation where there are no comparisons.
Table 4-1, Table 4-5 summarize, pillar by pillar, the committee's metrics and the four levels of performance for each characteristic. These tables are not necessarily all inclusive; additional tailored characteristics and metrics may be necessary for some evaluations. The descriptors should be looked upon as conceptual guides that can be modified for specific situations.
The results of assessments can be recorded as illustrated in Table 4-6. Overall assessments for each pillar would be arrived at on the basis of judgments of the assessments for each characteristic in that pillar. The committee believes that good or excellent performance in each characteristic, and excellent overall performance in all five pillars, are necessary for an organization to be judged world-class.
The results might also be presented in graph form in typical “spider diagrams” (see Figure 4-2). Spider diagrams may have several uses. For example, Figure 4-2 could be a convenient summary showing if an RDEC approaches or achieves world-class performance on the basis of the pillars alone. A more complicated diagram could include radials for each of the 25 characteristics. Spider diagrams for each directorate could provide visual comparisons. In the case of multiple assessments, a spider diagram for the Natick RDEC could be compared with diagrams for other RDECs.
4
Many recent metrics (e.g., the Industrial Research Institute's Technology Value Program) that began with the Adler article now use the four-stage model; hence the committee adopted this model to be more consistent with the current literature and the practice of several companies.
5
The committee also considered using the terms Stage 1, etc. to designate levels of performance. However, the committee was concerned that using numbered stages might lead to a “numbers game” and an inflexible scoring system; hence the committee decided to use words. The words do have some negative and positive connotations, which are intended.
OCR for page 39
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
FIGURE 4-2 Spider diagram.
It should be noted that the committee has implicitly given equal weight to all five pillars. Under some circumstances, it may be appropriate to assign greater weight to one pillar or another.
Finally, the committee observes that application of the metrics in this report need not be limited to organizations striving to attain world-class performance. For example, if the metrics are used for self-assessment or self-understanding, they should be used in the context of an organization's role, responsibilities, and goals. If the organization's goals do not include world-class performance, then the organization could choose to use the metrics and approach the assessment in a different way. In some cases an organization could decide to use the metrics as a guide for self-assessment and develop its own definitions of poor, adequate, good, or excellent. Also, an organization might wish to focus on some characteristics or pillars more than others. This adaptation process in itself can be important for self-improvement.
OCR for page 40
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
REFERENCES
Adler, Paul, William McDonald, and Fred MacDonald. 1992. Strategic Management of Technical Functions. Sloan Management Review. vol. 33 (2): 19–37.
Brown, Warren and David Gobeli. 1992. Observations on the Measurements of R&D Productivity: A Case Study. IEEE Transactions on Engineering Management. 39 (4): 325–331.
Brown, Mark and Raynold Svenson. 1988. Measuring R&D Productivity. Research-Technology Management. 31 (4): 67–71.
Davidson, Jeffrey and Ann Lorette Prudent. 1996. Quality Deployment in R&D Organizations. Research-Technology Management. 39 (1): 49–55.
IRI. (Industrial Research Institute). 1996. Measuring the Effectiveness of R&D: A set of 50+ metrics available on a searchable disk. Available from IRI, Suite 1100, 1550 M Street, N.W., Washington, D.C., 20005-1708. (Phone: 202-296-8811).
Miller, Joseph, Parry Norling, and John Collette. 1996. Research/Technology Management: Leading, Managing, and Getting Results . Kirk-Othmer Encyclopedia ofChemical Technology. Volume R.In press.
Roussel, Philip, Kamal Saad, and Tamara Erickson. 1991. Third Generation R&D. Cambridge, Mass.: Harvard Business School Press.
OCR for page 41
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Tables 4-1 through 4-6
OCR for page 56
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
TABLE 4-4 Metrics of the Value Creation Pillar
Characteristics
Performance Level
Metrics
Proper Portfolio
Poor
Products are developed that do not meet customer needs; products have poor customer acceptance; customers perceive that commercial alternatives are cheaper, perform better, and are more durable.
Adequate
An analytical process to examine the product portfolio is used to design and field products that have greater value and soldier acceptance; results of the analytical process lead to modifications in product design; major changes may be made after fielding the initial product.
Good
Portfolio analyses of a program are an integral part of the strategic planning process; there is broad and active customer involvement in the portfolio analysis; programs yield products that have significant customer acceptance, meet or exceed customer requirements, and demonstrate increased value compared to current products or commercial alternatives; minor changes in product design occur after initial fielding.
Excellent
Portfolio analyses result in RD&E processes that yield products and services with excellent value, performance, and customer acceptance.
OCR for page 57
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Product Performance
Poor
Products do not meet customer requirements (e.g., in terms of weight, volume, function, durability, or maintainability); customers complain that product performance does not meet the developer's claims; products are not suitable for use in certain locations or environmental extremes.
Adequate
Products meet customer requirements, needs, and expectations.
Good
Products fully meet or exceed customer requirements; products are perceived as better than the ones they replace.
Excellent
Products not only exceed customer expectations, but product performance includes some pleasant, unexpected surprises (e.g., reduced maintenance requirements, longer shelf life, longer mean time to failure, resource savings).
OCR for page 58
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Cycle Time and Responsiveness
Poor
Cycle time for project completion is longer than anticipated; milestones are routinely missed; program delays result in increased end-item cost; research programs do not anticipate customer needs; management and staff are not flexible to modifications of product requirements.
Adequate
Elapsed time from project initiation to project completion is measured and can be reliably forecasted; research programs are described as being on-time and on-budget.
Good
RD&E programs are initiated and completed significantly faster than similar government or commercial programs; research staff is responsive to “quick fixes” for troops, and numerous examples are readily available for major products; senior management ensures that adequate resources are reprogrammed to fulfill quick-fix requests.
Excellent
RD&E programs are initiated and completed substantially (e.g., one third) quicker than similar government or commercial programs; innovative processes and technical solutions reduce typical quick-fix response times by nearly half; the staff monitors foreign and domestic industrial and academic research for solutions to new and unanticipated technical problems; commanders directly and indirectly express gratitude for responsive quick fixes.
OCR for page 59
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Value of Work in Progress
Poor
No evaluations of historical RD&E programs are available for comparison to current programs; no methodology is in place to assess current RD&E programs; customer perception of prior RD&E programs is predominantly critical and negative, and little or no value is placed upon the current programs by the customers.
Adequate
A database on select historical RD&E programs and all current programs is available; current RD&E programs are vividly described, and these descriptions are used during peer-review discussions to justify programs and prioritize personnel and budget requests; customer perception of prior RD& E programs is generally positive; customer perception of current RD&E programs is positive (i.e., the products and services will generally meet user requirements and be delivered on time and on budget).
Good
A database is maintained on all past major projects (e.g., for the last decade) and their primary and secondary impacts; the database is used for comparison with current RD&E programs; leadership creates a scale to rate continuously the potential value of current programs compared with previous programs and show improvements; customers rate RD&E programs as very good (i.e., products are expected to fully meet or exceed customer requirements; products are perceived as likely to be better than the ones they replace).
Excellent
A complete historical database and evaluation methodology are used to demonstrate the value of the organization's products and services; data are used to justify and defend program expenditures; customers rate products and services as excellent (e.g., product performance exceeds customer expectations); product performance exceeds anything projected to be available from domestic and foreign sources for at least several years.
OCR for page 60
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
TABLE 4-5 Metrics of the Quality Focus Pillar
Characteristics
Performance Level
Metrics
Capacity for Breakthroughs
Poor
RD&E programs are routine and unimaginative; there is no evidence of imaginative or innovative solutions being applied to RD&E tasks; resources are directed to meeting specific customer requirements only.
Adequate
RD&E programs are characterized by steady but incremental improvement; several innovative solutions can be pointed out; minimal funding is available for programs that anticipate future military requirements.
Good
Although most programs are characterized by incremental improvements in technology, the organization has demonstrated several leap-ahead improvements; the organization encourages and funds opportunities to seek truly innovative, moderate-risk solutions.
Excellent
Unexpected innovation based on breakthroughs in technology occur fairly regularly among internal and external (cooperative) RD&E programs; moderate- and high-risk research that offers high return receives stable funding; numerous examples of breakthrough research are cited from the previous five to ten years.
OCR for page 61
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Continuous Improvement
Poor
There is no tangible evidence of senior management commitment to continuous improvement; the need and ability to focus on continuous improvement are recognized, but not funded; products and services show incremental changes; innovations are not rewarded; solutions from industry and academia are discounted as “not invented here.”
Adequate
Quality of the work is discussed and several measures of quality are used routinely; innovative solutions are encouraged; staff members frequently make suggestions for improvement; several changes are made (and documented) each month for improving the work and the output of the organization.
Good
The organization takes steps to improve work processes and RD&E results significantly; quality audits are performed periodically by internal and external review groups; numerous improvements can be pointed out; productivity is an important topic of discussion; report cards are issued annually by senior leadership; senior managers have the resources to enact recommendations.
Excellent
Greater productivity, enhanced research and product quality, improved customer involvement and satisfaction, and continuing education of the work force are areas of primary interest to senior management; the concepts of continuous improvement and excellent product value are embedded in the goals of each RD&E and support function; there is a systematic analysis of research and support processes to eliminate non-value-added activities; research personnel are renowned for finding innovative solutions to technically difficult problems.
OCR for page 62
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Commitment to Quality
Poor
Management espouses a commitment to quality, but no formal process to review and evaluate quality is in place; some quality-related results are managed by exception; the quality of products and services varies between RD&E units in the organization.
Adequate
Management is investing resources for total quality training and implementation; the variability of products and services is being measured and tracked; personnel are aware of the importance of quality.
Good
Total quality implementation is a major goal in the organization's strategic plans; a framework and methodology for measuring and assessing total quality is in place; measurable objectives for work-process improvement are established; there are methods (e.g., statistical process controls) to improve effectiveness and product quality with existing resources.
Excellent
The commitment to total quality is inherent and pervasive throughout the organization; the focus of all measurements is on optimizing the RD&E processes to deliver value; frameworks, such as ISO 9000/2 (international quality standards), Baldrige criteria, or locally developed systems, are used for assessment; recommendations to improve quality are immediately funded and implemented.
OCR for page 63
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Structured Processes
Poor
Work processes and procedures are understood and milestones are established, but there is no system of internal or external review; project management results in products or services that are delivered late and over budget; delays result in termination of projects; disciplined approaches to defining problems and the scientific method are rarely used.
Adequate
Work processes and procedures are monitored; project costs and milestones are closely tracked; processes are established to improve quality incrementally, contain or reduce RD&E cost, and reduce product cycle time; disciplined approaches and the scientific method are used most of the time.
Good
Program managers are flexible and adaptive; senior leadership and staff are receptive to innovative ideas for improving work processes and procedures; product quality and customer focus mean continuous improvement; disciplined approaches and the scientific method are used consistently.
Excellent
The senior leadership strives to identify and incorporate best business practices into the organization; processes are considered flexible and not overly restrictive, prescriptive, or bureaucratic; management is focused on achieving superior performance and product quality; emphasis on cross-project management ensures timeliness and the proper allocation of resources; disciplined approaches to problem solving include an extensive network linked to Army technological resources worldwide; the scientific method is strictly followed.
OCR for page 64
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Learning Environment
Poor
Senior leadership is characterized as reactive; little if any learning takes place on an organizational basis; some managers and staff learn from mistakes.
Adequate
Senior leadership recognizes and communicates the importance of organizational learning; management and staff learn from mistakes and from others; personnel are well networked both inside and outside the organization; teams on one project teach teams assigned to other projects; new skills and techniques are acquired through new hires and continuing professional education.
Good
Organizational learning is characterized as adaptive; the organizational climate is conducive to learning; personnel are rewarded and encouraged for taking risks and entrepreneurial initiatives despite occasional mistakes; personnel learn from others and by doing; management experiments with new organizational concepts to discover new ways of doing things.
Excellent
Organizational learning is adaptive and anticipatory; research and technical capabilities continually expand, and management anticipates change; traditional and innovative methodologies are used to measure and evaluate organizational learning.
OCR for page 65
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
Quality of Research
Poor
Research and technology programs are not generally aligned with customer requirements and needs; records of research methodology and results are poor; although recorded in technical reports, data are not published in peer-reviewed journals or cited by other scientists in academia or industry; research results cannot be replicated by scientists and engineers outside the organization.
Adequate
Research and technology programs are aligned with customer requirements and needs; research methodology and results are peer-reviewed and published as both technical reports and journal articles; the research staff is invited to participate in scientific meetings and workshops; research results are easily replicated by other laboratories.
Good
The research and technology programs are recognized by peers as being of very high caliber; several programs are among the best in the federal government and are described as innovative and original; some patents are awarded.
Excellent
The quality of the research and technology programs is considered to be among the best in the world; basic research not only fulfills customer needs, but also anticipates future requirements, thus reducing cycle time for new products; research and technology programs are innovative and state-of-the-art; new procedures, processes, and materials are developed by personnel; numerous patents are issued for RD& E innovations.
OCR for page 66
WORLD-CLASS RESEARCH AND DEVELOPMENT: Characteristics for an Army Research, Development, and Engineering Organization
TABLE 4-6 Organizational Assessment
Assessment
Component
Poor
Adequate
Good
Excellent
Customer Focus Pillar
Customer Satisfaction
____
____
____
____
Customer Involvement
____
____
____
____
Market Diversification
____
____
____
____
Pillar Assessment:
____
____
____
____
Resources and Capabilities Pillar
Personnel Quality
____
____
____
____
Budget
____
____
____
____
RD&E Capabilities, Skills, Talents
____
____
____
____
Use of External Resources
____
____
____
____
Important Technologies
____
____
____
____
Organizational Climate
____
____
____
____
Information Technology
____
____
____
____
Facilities and Infrastructure
____
____
____
____
Pillar Assessment:
____
____
____
____
Strategic Vision Pillar
Alignment of Vision and Mission
____
____
____
____
Anticipatory Strategic Planning
____
____
____
____
Stakeholder Buy-In
____
____
____
____
Leadership
____
____
____
____
Pillar Assessment:
____
____
____
____
Value Creation Pillar
Proper Portfolio
____
____
____
____
Product Performance
____
____
____
____
Cycle Time and Responsiveness
____
____
____
____
Value of Work in Progress
____
____
____
____
Pillar Assessment:
____
____
____
____
Quality Focus Pillar
Capacity for Breakthroughs
____
____
____
____
Continuous Improvement
____
____
____
____
Commitment to Quality
____
____
____
____
Structured Processes
____
____
____
____
Learning Environment
____
____
____
____
Quality of Research
____
____
____
____
Pillar Assessment:
____
____
____
____
Items in cart [0]
