Strengthening American Manufacturing The Role of the Manufacturing Extension Partnership: Summary of a Symposium (2013) / Chapter Skim
Currently Skimming:
Panel IV: Measuring Success: Assessment and the Demands of the New Strategy
Panel IV: Measuring Success: Assessment and the Demands of the New Strategy
Pages 99-111
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 99... ...
We need to know more, she said, both about how this has been measured in the past, and how it could better be measured in the future "to understand where we are and how to move forward." Relevant and accurate measurement is critical during a time of transition, she said, "to make sure that we are measuring the right kinds of things." There are many different kinds of measurements, including outcome metrics and process metrics, which are quantitative, as well as qualitative metrics. "I think it's going to be important as we move forward to really understand how and what we should measure so that assessments are aligned with the new strategy MEP is laying out." She said that the speakers would discuss the current state of assessment, proposals for some future measures, and some of the challenges of measurement, "which are true for any organization I've ever worked with." THE MEP ASSESSMENT MECHANISMS Gary Yakimov Manufacturing Extension Partnership National Institute of Standards and Technology Mr.
|
From page 100... ...
. OMB indicated that the program was well-managed, with regular reviews to assess performance, while NAPA found that the metrics used to evaluate programmatic performance and outcomes are "extensive," and highlighted an SRI report which noted that "the significance of these efforts is not in the methods used or the results generated, but in the integration of evaluation into a longer-term, strategic framework." In addition, he said, "we are constantly invited to talk to other federal agencies about our approach to data, which I think is a good signal." A Culture of Accountability He said that one of the reasons why MEP is praised for its evaluations is that it "has a culture of measurement and accountability." At the beginning of each year, each center files an operating plan, and the MEP uses many tools to "hold the center accountable to us and to our investment." They report quarterly on their clients, and the MEP then surveys those clients.
|
From page 101... ...
The top responses were (1) ongoing continuous improvement / cost reduction strategies, (2)
|
From page 102... ...
We need to move beyond minimally acceptable impact metrics that allow us to distinguish our really strong performers." He added that there would always be centers that struggle to meet minimal performance, and the MEP would work with them. A New Scorecard Mr.
|
From page 103... ...
To avoid multiple naming conventions, he said, the qualitative metrics would be organized in the same categories as the panel reviews: Does a center understand and help lead its market, do they have a sustainable business model, what are their strategic partnerships, are they financially sound, and are they aligned with MEP strategies.
|
From page 104... ...
I actually think that we could be informing manufacturing strategy better than anyone else if we can figure out a way to better mine our own data and produce custom reports on a regular basis." EVALUATING MEP EVALUATION Daniel Luria Michigan Manufacturing Technology Center Dr. Luria, vice president for research at the Michigan Manufacturing Technology Center, an MEP, opened with the thought that "if we keep doing what we've always been doing, we're probably going to get the same kinds of results." He said that he was an economist whose major field of graduate study, done at the University of Michigan's Institute for Social Research, was economic surveys.
|
From page 105... ...
Luria said that three rigorous client control studies had been done for MEP: one for the 1987-1992 period, by Jarmin & Jensen, which found no sales effect and a 5 percent gain in productivity (value added per employee) ; another, for the 1992-1997 period, focusing on the seven MEP centers in Pennsylvania, by Eric Oldsman of Nexus Associates, which found the same results; and a third for 1997-2002 by an SRI/Georgia Tech, which found neither a sales nor a productivity effect.
|
From page 106... ...
Luria cautioned. The first is that "it is highly unlikely that MEP or the centers can get most clients to ascribe new sales to MEP services." It is easy to see the source of a change that eliminates bottlenecks in the production system if the MEP brings in industrial 27 He offered the following example of this survey problem: An MEP center helped a client achieve compliance to the ISO 9000 standard required by customers accounting for 80 percent of its sales.
|
From page 107... ...
"I think what it's really about is that when companies grow, their managers feel like they're geniuses, and when companies don't grow, the managers feel that their customers are bad people. There's not a lot of room for them to say, I am growing somewhat, and a significant part of that is because I learned something from my interaction with MEP." More serious, he said, is that all sales impacts must be presumed to be zero-sum or very nearly so for U.S.
|
From page 108... ...
For example, he said, his center had reviewed which kinds of MEP interventions produced the largest reported increases in new sales. The results were that quality-based projects produced the largest increases, "lean" projects the next largest, and growth projects the least.
|
From page 109... ...
How to Reach Companies Ready to Advance In guidance for the centers, Dr. Gaster said that detailed differentiation of the interventions is important, as is differentiation of populations: For example, a company may want to adopt a green manufacturing strategy, but no one at the meeting had talked about what percentage of companies could realistically be expected to adopt a green strategy.
|
From page 110... ...
Dr. Yakimov said that the centers currently report clients to the MEP at the end of the client interaction, and the client is surveyed six months after that.
|
From page 111... ...
Also, we want to know if they just use the good ideas that are already on the market, or are they creating those new ideas." Finally, another item is the quality of the CEO, "which is another strong indicator." The Long-term Value of MEP Services Diane Palmintera said she was concerned about capturing the long-term value of MEP services. She mentioned the example of Georgia Tech, where the MEP program gives a small amount of money to the entrepreneurial venture lab to help with business development.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.