Panel V: Extending Assessment—Challenges and Opportunities

Introduction

Iain Cockburn

University of British Columbia

Dr. Iain Cockburn set the stage for the panel by commenting on the tension between a publicly funded program, such as the Advanced Technology Program (ATP), and granting proprietary rights to technology developed by recipients. One issue is that, because technologies funded by the ATP are new, the patents issued on them are likely to be very broad. Consequently, there is a legitimate concern about the potential for restricted access to platform technologies. Dr. Cockburn said that people involved in public research and development (R&D) programs are aware of these issues, but it would be worth discussing them in the context of the ATP. Addressing these intellectual property issues is important, and this is why it has come up so many times in the day's discussions.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 98
--> Panel V: Extending Assessment—Challenges and Opportunities Introduction Iain Cockburn University of British Columbia Dr. Iain Cockburn set the stage for the panel by commenting on the tension between a publicly funded program, such as the Advanced Technology Program (ATP), and granting proprietary rights to technology developed by recipients. One issue is that, because technologies funded by the ATP are new, the patents issued on them are likely to be very broad. Consequently, there is a legitimate concern about the potential for restricted access to platform technologies. Dr. Cockburn said that people involved in public research and development (R&D) programs are aware of these issues, but it would be worth discussing them in the context of the ATP. Addressing these intellectual property issues is important, and this is why it has come up so many times in the day's discussions.

OCR for page 98
--> Information Needs for Measuring Spillovers from Public-Private R&D Partnering Maryellen Kelley Carnegie Mellon University and National Institute of Standards and Technology Overview The Advanced Technology Program (ATP) is unique among federal agencies supporting research and development activities in private industry. To merit funding, an industry-initiated project proposed to the ATP has to be focused on an enabling, high-risk technology that has the potential for broad economic benefits. These selection criteria reflect the ultimate long-term goal of the program to achieve greater productivity growth in the economy as a whole through technical advances that become incorporated in industrial processes, and new products and services. In the long-run, we expect the economic benefits to consumers and other firms using these innovations to be substantially greater than the profit that the ATP-funded firm makes from developing the technology. Economists characterize these benefits as spillovers, that is, positive externalities from innovative activity that are not captured fully by the innovating firm. For example, the commercial viability of a technology developed by one company may depend on complementary technologies developed by other firms. A case in point is the ATP-supported technology developed by X-Ray Optical Systems, Inc. In his remarks earlier, David Gibson eloquently described how his company's ATP-sponsored advances in lens technology are enabling complementary innovations by other companies. Through informal alliances with these companies, X-Ray Optical's technology eventually will be incorporated in a variety of applications in a broad set of industries. An ATP-funded project typically takes three to five years to complete. Because these projects are by design high-risk and precompetitive, some will not succeed. Yet even when a project is highly successful, as Rosalie Ruegg, Director of the ATP's Economic Assessment Office (EAO), indicated in her remarks, the broad-based economic benefits from the innovation may not be apparent for a number of years after the project has ended. For some technologies, a decade or more may pass before these economic benefits are realized. By necessity, evaluation of the economic impacts of the ATP portfolio of R&D projects relies on near-term and intermediate indicators of the long-run economic benefits that go beyond those that are captured by the firm undertaking the initial R&D. In selected cases, where the technology is especially promising, we expect our analyses to include projections of future benefits. On a routine basis, the EAO collects information on participating organizations for the duration of the ATP project and for six years after the project has

OCR for page 98
--> ended. For each organization, the Business Reporting System (BRS) involves an initial baseline report, annual reports submitted during the life of the project, and a closeout report collected at the end of the project. For six years after the project has concluded, the EAO plans to conduct a series of follow-up interviews with the technical and business leads of the project in each ATP-funded organization and with their counterparts in other organizations that did not receive ATP funds but are now using the ATP-funded technology, pursuing specific applications of the technology, or building upon the ATP-funded technology in a related area. From time to time, the EAO also conducts special studies that usually are carried out by independent researchers on a contract basis. These studies include statistical analyses involving comparisons with ATP awardees to enterprises that have not received any ATP funding, investigations of the factors that promote (or inhibit) spillovers in specific industries or technical areas, and in-depth case studies of specific projects or a group of related projects. Pathways to Broad-Based Economic Benefits In general, an ATP-funded project can achieve broad economic benefits by the direct introduction of a new product or service into the market, or by indirect means, through the take-up by other organizations of the knowledge and techniques that were developed during the project. Some ATP projects will proceed along the direct path, yielding benefits to consumers that are directly traceable to the project results. However, because the ATP selects only enabling, high-risk R&D projects for funding, it is expected that the indirect route will be important as well. At the EAO, information is collected on the efforts of ATP-funded organizations to bring to market new products or services based directly on the ATP-supported technology. Information is also collected on the indirect pathways through which ATP-supported technologies achieve economic benefits. Another area of focus are the EAO's efforts to address the information requirements involved in tracking the progress of ATP-funded technologies that proceed to market via indirect pathways. The results of an ATP-funded technology project take an indirect route to market when knowledge is passed from one firm to another, and in a series of such exchanges over time, the ATP-funded technology is altered, improved upon, or integrated with other technologies. Frequently, companies other than those supported by the ATP are involved after the project has ended. These companies may be involved in developing complementary technologies, carrying out additional R&D on the ATP-funded technology, or undertaking the final product development and marketing activities necessary to bring a commercially viable innovation to market. When (and if) the indirect route eventually yields such an innovation, the contribution of the ATP-funded R&D project may be only a distant memory to those finally responsible for bringing this innovation to market.

OCR for page 98
--> Without a research effort to follow these developments as they unfold, it will be difficult to determine the contribution that the ATP project has made. Hence, the EAO is planning to ''follow'' the technology in the post-project period, wherever it goes. R&D Partnering and Information Sharing The ATP supports innovative activity in a number of different technical fields and in a variety of industry contexts. How a firm organizes its partnering relationships, how it learns about relevant technical advances by other R&D organizations, and the ways in which it shares information (and how much it shares) with other companies vary by field and by industry. To evaluate the spillover benefits from ATP-funded R&D activities, information is needed about the variety of formal and informal cooperative organizational arrangements that are formed in ATP-funded projects and how these arrangements facilitate the innovation process. Since 1993, the EAO has collected information through the BRS on both the formal and informal cooperative activities of organizations engaged in ATP-supported R&D projects. To date, the BRS data show that the vast majority of ATP-funded projects involve some degree of formal collaboration by a for-profit company with one or more of the following: another company, a university, a government laboratory, or other nonprofit organizations. Knowledge spillovers and complementary technology developments are facilitated by interorganizational cooperation in R&D and by information sharing among companies with complementary technical capabilities. However, U.S. companies have had considerable difficulty in forming and sustaining cooperative R&D activities with other enterprises. Outside of ATP joint venture projects (and other government-supported consortia), most cooperative R&D ventures formed by companies are short-lived. Their short lives reflect the problems that U.S. companies experience in establishing the terms for sharing information among member firms, resolving disagreements over the division of property rights among participating firms, and in sustaining commitment of key members to the joint effort. Preexisting consortia already have resolved these cooperation and coordination problems, and hence, ATP support is not viewed as critical to their formation. For funding decisions, the ATP views a preexisting consortium as a single entity. For example, in the 2mm joint venture, a project of auto suppliers, producers, and universities designed to reduce dimensional variations, the Auto Body Consortium (ABC) was the lead organization. ABC counts as only one member of that joint venture. ATP support for joint venture activity is limited to the formation of new cooperating teams, made up of at least two for-profit organizations, for long-term (up to five years) R&D projects. The initiation of a new

OCR for page 98
--> R&D project by an existing group may be supported if the proposed joint venture also contains at least two separate for-profit organizations. The ATP as a Catalyst to Cooperation ATP funding has been instrumental in the formation of new cooperative arrangements in a number of technical and industry application areas. ATP-funded joint ventures vary in size (i.e., the number of separate organizations) and also in composition (i.e., the different types of organizations). In purpose, some joint ventures are system integration efforts, involving customer firms and their technology suppliers, as in the case of the various automobile manufacturing technology projects. Others, such as the project on printed-wiring-board technology, include rival producers focusing on a problem that is common to all of them. A third type is the venture in which separate companies combine their complementary technical expertise for the purpose of developing a new product or service. This is illustrated in the Spoken-Language Forms Translator for Information Transactions project, which was funded in 1998. Two small companies, Language System, Inc., and Eloquent Technology, Inc., are working together on a joint effort, combining one firm's expertise in speech recognition and speech synthesis with the other's specialization in language understanding and translation programs. All three types of ATP-supported joint ventures are institutions designed to internalize at least part of the spillover benefits among the participant member organizations. In cases in which the membership includes lead users and technology developers, the take-up of the technology by the user organizations is expected to be faster than would occur in the absence of the joint venture. When the venture brings together complementary expertise in different fields, we expect that new products or process innovations based on these combined capabilities will be introduced sooner than if these firms had proceeded independently of one another. Moreover, to the extent that ATP support enhances a company's capability to collaborate with other organizations on R&D projects, we expect participating organizations to continue to form new collaborative ventures and to sustain the cooperative activities with their joint venture partners after the ATP project has ended. Formal R&D partnering also may occur through subcontracting ties to other organizations. Subcontracting is a frequent form of R&D partnering employed by companies having a single-company award from the ATP. Most projects involve subcontracting relationships. The EAO collects information on the division of responsibilities between principals and subcontractors on ATP-funded projects. We also plan to collect additional information on these relationships in future studies. Informal partnering and information-sharing arrangements are difficult to measure. However, the EAO attempts to do so by collecting information on a

OCR for page 98
--> number of indicators of informal cooperation. An ATP-funded company may form one or more alliances with other companies that are not direct participants in the project. These alliances may involve potential customers, or involve companies with complementary technical or business expertise. In addition, all ATP award recipients report on the papers published by scientists and engineers, and provide data on patent applications and awards related to the ATP project. When the papers involve coauthoring by scientists and engineers with different organizational affiliations, we consider this to be an indicator of interorganizational cooperation in R&D activities. Joint patenting activity that includes an ATP-funded company and another organization is another indicator of cooperation that can be measured independently of the formal partnering arrangements reported as a subcontract, a joint venture, or an informal alliance. Similarly, conference presentations are a form of information sharing. The BRS also includes data on the conference presentations given by scientists who are working on ATP-supported R&D projects. Post-project: Handoffs, Acquisitions, and Spinoffs An ATP-funded technology that travels an indirect pathway to market frequently involves post-project advances in other, complementary technological developments that may be undertaken by the company initiating the project or by organizations other than the company responsible for carrying out the initial R&D project. In either case, the complementary technologies are not funded by the ATP, but are important to the technology that was developed with ATP support. In the post-project data collection effort, the EAO plans to collect information on these complementary technologies and the organizations responsible for their development. In the BRS, we currently collect information on the awardee's plans, tracking the number and types of specific applications. Our post-project data collection effort is designed to follow the advances in specific short-term and long-term application areas that the management of the innovating firm considers important. In the post-project data collection effort, we also plan to collect information on the number of new applications and to identify those applications that no longer are being pursued by the company or any other organization. With respect to the take-up of ATP-funded technologies by other firms, the EAO staff and its contractors have identified a number of important mechanisms by which technology transfer occurs across organizations. Some companies in ATP projects focus only on R&D and depend on the product development and commercialization activities of other enterprises. The commercialization strategy of such an R&D contractor organization is usually to license the technology to another enterprise. This type of innovating company captures only a small share of the economic value of the technology in the revenues that it receives from the sale of licenses and other property rights that it holds on the technology.

OCR for page 98
--> The post-project data collection effort is designed to collect information on the revenues that these companies earn from these sources. With this handoff of the technology from one company to another, the realization of broader economic benefits depends on the success of the partnering firm in commercializing the technology. The post-project information-gathering effort also includes a systematic follow-up of the commercialization activities of these partners. Another way in which technology is transferred from one organization to another is through an acquisition. Owners of small R&D companies may view the acquisition of their firm by another company as a desirable exit strategy. The contribution of the ATP-funded technology development project to the value of the firm being acquired is a post-project indicator of a short-term economic benefit. Over the longer term, the achievement of broad-based economic benefits from the ATP-funded technology depends on what the acquiring company does with the technology. Hence, the post-project data collection effort of the EAO is designed to track developments by the acquiring firm in using or developing technologies that are related to the technology funded by ATP. Acquisitions are also the result of business failures. From the point of view of the ATP, the failure of the initial innovating firm does not necessarily signal the end of the technology's development path. In the EAO-sponsored study on the first 38 completed ATP-funded projects, a few small companies went out of business after the project ended. However, in at least one case, another enterprise acquired the business and the rights to the technology developed with ATP support. If the acquiring firm develops and brings to market technologies related to the ATP project, then the social benefits from the ATP's investment may be positive, even though the funded company was unable to reap much, if any, of that benefit. Hence, the post-project data collection effort is designed to distinguish between cases in which the technology "lives" but the firm "dies" and cases in which both the technology and the firm ''die" together. Last, a technology also can change hands when a larger company spins off a unit for the purpose of developing a new product or market. When this occurs, the EAO plans to follow the progress of the spinoff unit, rather than the parent organization. Concluding Remarks The goal of the ATP is to support the development of technologies with the potential for broad-based economic benefits. If the ATP is making appropriate project selection, the value that is captured by the innovating firm is only a small part of the potential economic benefit from the technology. In the extreme case, an ATP-funded company may cease to exist and derive little or no direct benefit from its R&D activities. Yet, the ATP-supported technology may live on through the activities of other firms (e.g., a customer, competitor, or firm that has acquired rights to the technology for use in another industry). Even though the firm

OCR for page 98
--> responsible for carrying out the initial R&D project may benefit very little, the project still may have a large economic impact through some spillover mechanism. The EAO post-project evaluation effort is designed to follow the technology in order to trace the economic benefits in the medium term that are associated with the post-ATP project development of the technology, whether or not the awardee is the organization pursuing it. Wherever possible, the post-project data collection effort of the EAO also will track developments in related technologies that are complementary to, or otherwise build upon, the ATP-funded technology. In its evaluation studies, the ATP is committed to protecting the confidentiality of the proprietary information that companies provide us through the BRS and other special studies. This protection is compatible with our use of these data in statistical analyses to assess the economic impact of the ATP portfolio. Moreover, as a result of its evaluation research efforts, the EAO is in the process of compiling a unique database on private sector research, technology development, and commercialization activities. For large and small companies alike across a broad spectrum of technical fields and industries, the information we are collecting tracks R&D projects from their inception to their conclusion and beyond. The resulting database will surpass all other existing databases in its level of detail on each project, the breadth of private sector R&D activities covered, its inclusion of both successful and unsuccessful cases, and its panel quality (i.e., multiple measures of the activities of the same organizations and specific technologies that are taken repeatedly over an 8-to 11-year period). In the long-run, analyses of these data should increase our understanding of the factors affecting the ATP's success. In addition, future policy makers will benefit from the research that we are conducting now on the ways in which public-private partnering arrangements facilitate the development of technologies and how the resulting R&D activities bring about broad-based economic benefits. Finally, our research also should shed light on the nation's innovation system, especially in providing a more detailed map of the pathways through which technological change contributes to economic growth and to the productivity increases necessary to sustain and improve the standard of living of Americans in the next century. Assessing Productivity Impacts in Health Care Information William Lehr Columbia University Dr. Lehr said that the research he would discuss was conducted jointly with his colleague at Columbia, Dr. Frank Lichtenburg. The research has examined several projects in ATP's focused program on the use of information technology

OCR for page 98
--> in the health care sector. The motivation for the research was the so-called productivity paradox, in which investments in information technology that seem intuitively to have productivity-enhancing potential do not appear to pay off in terms of higher productivity. Dr. Lehr said the paradox was summarized in Robert Solow's quip that "we see computers everywhere except in the productivity statistics." With productivity growth on the rise again, the paradox has received less attention, even if its roots remain imperfectly understood. Dr. Lehr said that both he and Dr. Lichtenburg believe that information technology was improving productivity in health care, but that better data needed to be assembled in order to make the case. Drs. Lehr and Lichtenburg also believed that integrating case studies into empirical research would be a useful strategy to addressing the productivity issue. For today's session, Dr. Lehr said that he would discuss one case study from the ATP and preliminary results from their empirical work. Dr. Lehr described two categories in which investments in information technology could pay off in health care: Improved quality of care: Doctors may have better information, make faster decisions, and prescribe drugs more appropriate to the diagnosis. Lower costs: Health care delivery has very high administrative costs and, at least in principle, information technology can help lower these costs. The latter category is easier to measure than the former, but both are challenging, Dr. Lehr said. A criterion for selecting case studies for the research was to look for projects on which Lehr and Lichtenburg could obtain the necessary information. Empirical Analysis For the empirical work, the basic idea was to find or develop a measure of the quality of health care, and that this is depends on a variety of things. In past econometric work, economists used tax rates, existence of various kinds of health insurance plans (e.g., prospective payment plans) to measure quality. Lehr and Lichtenburg have tried to include investment in information technology as an explanatory variable. Lehr and Lichtenburg combined a data set on hospitals that has measures of the quality of hospitals with data from a hospital accreditation body on best practices in use of information technology. Although the data on hospital quality cover 15 to 20 states, only Iowa was used for their first cut at estimation; thus the data set contained only 24 observations. From this, they were able to estimate the impact of information technology on health care quality while controlling for other factors that influence health care quality. Even with the very small sample, meaning that the results should only be

OCR for page 98
--> considered suggestive, Lehr and Lichtenburg found that investment in information technology contributed to improvements in health care quality. Case Studies Lehr and Lichtenburg believe that case studies can yield a richer sense of the private and social returns to information technology investments in health care. Dr. Lehr noted that the information from the case studies can only be prospective, that is, the potential benefits are being examined early in investment cycle; one cannot expect benefits to show up until several years have passed. The kinds of questions that Lehr and Lichtenburg asked were whether ATP grants were used for purposes consistent with the ATP's goals, and how ATP's selection process could be improved. Lehr and Lichtenburg looked at two companies: Instream, a Massachusetts company developing electronic commerce solutions for hospitals; and Sunquest, a company developing alert systems, in which pagers transmit information to doctors about patients' conditions. The Instream Case Dr. Lehr described Instream as a company developing electronic commerce products for behavioral health care companies (e.g., psychiatric services) and their provider networks. In the behavioral health care field, as throughout the entire industry, it is becoming more challenging to administer health care delivery among a network of dispersed providers. Much of this has to do with information burdens, which are increasing because of things such as liability. To address this, Instream proposed to develop "smart forms" to automate the communication between health care providers and administrators of insurance plans. Smart forms would be built using an open architecture so that the wide variety of electronic forms used by various health care providers could be readable on any computer system. This was largely an integrative project, and used no new technology. Indeed, it would be easy to argue that Instream was doing something that some company would have done soon anyway. In funding Instream, the ATP was accelerating the deployment of a certain technology in a particular health care sector, that is, behavioral health. The benefits were clear, in that the electronic commerce approach supplanted a system by which faxes and phone calls carried out the communication tasks, while reducing the opportunity for errors in, for example, keying in patient identification numbers. Lehr and Lichtenburg then estimated the savings from the Instream technology that users could expect. With respect to administrative costs, the electronic commerce solution saved 22 percent on administrative expenses. For the administrator of the health care plan, operating margins were estimated to increase by

OCR for page 98
--> 140 percent. For Instream's customers, cost savings were estimated to be $10 million per year; Instream received $1.37 million from ATP to develop this technology. The expected internal rate of return, concluded Dr. Lehr, was approximately 95 percent. Ironically, Instream failed in 1998, showing how risky these projects can be. The company expanded into on-line content, which proved more costly than anticipated, and eventually resulted in a cash-flow squeeze that proved too severe. One clear long-term benefit, even though Instream failed, is learning spillovers. There are now approximately 150 companies in this sector, but Instream was among the first. The later entrants learned a great deal about the market from Instream's experience and, moreover, Instream's technology is still "on the radar screen" in the industry. In terms of broader lessons learned, Dr. Lehr said: ATP funding leverages private capital. Instream received venture capital and ATP funding helped to attract it. Technical industry expertise was acquired. Instream gained expertise, but the Instream grant was part of a broader ATP-focused program. This provided some scale among other actors in the industry, and thus an incentive for them to invest in these technologies. Because the ATP held conferences around the focused program, there were opportunities for industry players to network. As a technology integration project, as opposed to development, there was no technology waiting to be picked up by others when Instream failed. However, there were spillover benefits because Instream proved that the technology could work, and left a network of users that followers could, and did, exploit. Comparing ATP and Non-ATP Research Joint Ventures Nicholas Vonortas George Washington University Dr. Vonortas described the research project that he conducted for the ATP, which investigated research joint ventures at the ATP. The notion behind his work was to look at the performance of ATP-funded joint ventures from 1990 to 1995 as compared to joint ventures not funded by ATP in the same period. Dr. Vonortas and his colleagues at George Washington developed a database of research joint ventures using filings from the Department of Justice (DoJ) and other data describing the financial performance of firms in joint ventures. The database also included joint ventures not from DoJ because not all joint ventures

OCR for page 98
--> register with DoJ. In total, Dr. Vonortas had information on 102 ATP-funded joint ventures and 510 joint ventures that did not receive ATP funding. The objective of his research was to go beyond the traditional economic literature on joint ventures, which Dr. Vonortas does not find fully satisfactory. The literature argues that firms—typically large ones—enter into joint ventures to "diversify virtually." In addition to econometric analysis, Dr. Vonortas and his colleagues conducted case studies on ATP joint ventures, including one on Genometrix. Comparing Performance of Joint Ventures with Non-Joint Ventures Turning to his data and results, Dr. Vonortas pointed out that the number of ATP grants to joint ventures has grown noticeably since 1994 and 1995; this increase coincides with the start of focused programs at the ATP. In terms of technologies, there is no difference in the distribution of technology areas pursued by ATP joint ventures versus non-ATP joint ventures. This, said Dr. Vonortas, should be considered a good thing. For his analysis, Dr. Vonortas used a control group of firms that had never participated in any joint venture. This creates three groups: ATP joint ventures, non-ATP joint ventures, and non-joint venture participants. He found that there are significant differences among firms that have never participated in a joint venture and firms that have been in joint ventures (irrespective of whether it was an ATP or a non-ATP joint venture). In focusing on firm growth, non-joint venture participants seem driven strongly by the history of profitability, something that is not the case for joint venture participants. Turning to evaluation, Dr. Vonortas emphasized that the case studies are important supplements to empirical work. Interviewing company officials for his case studies opened up new avenues for research for Dr. Vonortas and, had he not done case studies, he would have concluded that the ATP joint venture project had failed. In summing up, Dr. Vonortas urged researchers to use case studies as a way to gain deeper understanding of the ATP and the individual projects. Questions from the Audience Egon Wolff of Caterpillar, Inc., observed that a benefit from the ATP—indeed a spillover—is that it has taught U.S. companies how to collaborate. Prior to the ATP, U.S. companies had some experience with collaboration in large programs such as SEMATECH, but the ATP encouraged collaboration more broadly. With respect to spillovers from R&D investment, approximately 70 percent of Caterpillar's R&D investment ends up in small companies and universities. The dissemination of knowledge is ongoing for Caterpillar's ATP projects when R&D funding is channeled through other firms. Reflecting on R&D project approval at Caterpillar, Mr. Wolff said that it is

OCR for page 98
--> probably as difficult to gain approval for a R&D project at Caterpillar as it is to obtain venture capital financing. He also said that he felt that the benefit of outside review of R&D proposals, as done in ATP, is enormous. Caterpillar would benefit, Mr. Wolff believed, from having roughly 30 percent of its R&D budget subject to external review. Steve Isaac from IBC Advanced Technologies made an observation about intellectual property protection. From the perspective of a company that has received ATP grants as a single applicant, he said that strong intellectual property protection is an important incentive to participation in the program. Even with strong intellectual property rights, an ATP award is a "sparkplug to get his company motivated" to invest its own resources in a particular project. To apply, his company must have a clear commercial use in sight when undertaking a grant, which eventually places the technology in the commercial arena. If a company does not commercialize, Mr. Isaac continued, there are a number of actions that can be taken that will move the technology eventually into the public domain. At one end of the spectrum, the government can exercise "march-in rights" on publicly funded technology that is not used by a company, and essentially seize it from the company. This is an extreme measure, which is rarely, if ever, used. Alternatively, if a technology is not commercialized because the company fails, that technology is bound to be sold in the process of dismantling the firm. Mr. Isaac also noted that the market will recognize a good technology, and it is likely to find it if it has commercial potential. In response to Mr. Isaac's observations, Dr. Kelley said that a study that the ATP has under way looks at how firms across different sectors protect intellectual property. Patents may be important in some sectors, less so in others. Similarly, cooperation may promote spillovers in some sectors more so than in others. The ATP is working to understand these differences across sectors. In summarizing, Dr. Cockburn said that there are circumstances under which publicly funded R&D is licensed on a nonexclusive basis, and that there is no shortage of venture capital firms seeking to license good ideas funded by public R&D programs. With respect to cooperation, Dr. Cockburn said that, at least today in the United States, there seems to be plenty of momentum to cooperate. And there is a well-developed infrastructure of lawyers and business consultants to put joint ventures together. The question of which joint ventures work and which do not remains open. On patent protection, Dr. Cockburn said that many companies have a "use it or lose it" provision in their patent laws; if, after five or seven years of patent protection, a company does not use the idea, it loses its patent.