National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×

Summary

CONTEXT

In The Art of War, written in the 6th century B.C., Sun Tzu described surprise:

In conflict, direct confrontation will lead to engagement and surprise will lead to victory. Those who are skilled in producing surprises will win. Such tacticians are as versatile as the changes in heaven and earth.1

Novel technologies are one of the principal means of surprising enemies or competitors and of disrupting established ways of doing things. Military examples of surprise include the English longbow, the Japanese long lance torpedo, the American atomic bomb, stealth technologies, and the Global Positioning System (GPS). Commercial examples include the telephone (Bell), business computers (UNIVAC and IBM), mobile phones (Motorola), recombinant DNA technologies (Genentech), PageRank (Google), and the iPod (Apple).

Until the 1970s, technological innovation tended to come from a limited number of well-established “techno clusters” and national and corporate laboratories.2 Today, the number of techno clusters and laboratories is growing rapidly everywhere. Policy makers are concerned with the emergence of high-impact technologies that could trigger sudden, unexpected changes in national economies or in the security and quality of life they enjoy and that might affect the regional, national, or global balance of power. As such, policy makers and strategic planners use technology forecasts in their planning.

The value of technology forecasting lies not in its ability to accurately predict the future but rather in its potential to minimize surprises. It does this by various means:

  • Defining and looking for key enablers and inhibitors of new disruptive technologies,

  • Assessing the impact of potential disruption,

1

Available at http://www.mailsbroadcast.com/the.artofwar.5.htm. Last accessed March 3, 2009.

2

A techno cluster refers to a science- and high-tech-oriented Porter’s cluster or business cluster (available at http://www.economicexpert.com/a/Techno:cluster:fi.htm; last accessed May 6, 2009). A business cluster is a geographic concentration of interconnected businesses, suppliers, and associated institutions in a particular field. Clusters are considered to increase the productivity with which companies can compete, nationally and globally. The term “industry cluster,” also known as a business cluster, a competitive cluster, or a Porterian cluster, was introduced, and the term “cluster” was popularized by Michael Porter in The Competitive Advantage of Nations (1990). Available at http://en.wikipedia.org/wiki/Business_cluster. Last accessed March 3, 2009.

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
  • Postulating potential alternative futures, and

  • Supporting decision making by increasing the lead time for awareness.

The Office of the Director of Defense Research and Engineering (DDR&E) and the Defense Intelligence Agency (DIA) Defense Warning Office (DWO) asked the National Research Council (NRC) to set up a committee on forecasting future disruptive technologies to provide guidance on and insight into the development of a system that could forecast disruptive technology. The sponsor recognizes that many of the enabling disruptive technologies employed by an enemy could potentially come out of nonmilitary applications. Understanding this problem, the sponsor asked the committee to pay particular attention to ways of forecasting technical innovations that are driven by market demand and opportunities. It was agreed that the study should be unclassified and that participation in it not require security clearances. The sponsor and the committee strongly believe that if a forecasting system were to be produced that was useful in identifying technologies driven by market demand, especially global demand, then it would probably have significant value to a broad range of users beyond the Department of Defense and outside the United States. The sponsor and the committee also believe that the creation of an unclassified system is crucial to their goal of eliciting ongoing global participation. The sponsor asked the committee to consider the attributes of “persistent” forecasting systems—that is, systems that can be continually improved as new data and methodologies become available. See Box S-1 for the committee’s statement of task.

This report is the first of two requested by the sponsors. In this first report, the committee discusses how technology forecasts are made, assesses several existing forecasting systems, and identifies the attributes of a persistent disruptive forecasting system. The second report will develop forecasting options specifically tailored to needs of the sponsors.

It is important to note that the sponsor has not asked the committee to build and design a forecasting system at this time. Instead, the intent of this report is to look at existing forecasting methodologies, to discuss important attributes and metrics of a persistent system for forecasting disruptive technologies, and to examine and comment on selected existing systems for forecasting disruptive technologies.

In 2007, the sponsor contracted the development of a persistent forecasting system called X2 (the name was later changed to Signtific).3 At the time of this writing, not enough data had been generated from this system to provide a meaningful analysis of potentially disruptive technology sectors. The characteristics of X2 are analyzed in depth in Chapter 6.

CHALLENGE OF SUCH FORECASTS

All forecasting methodologies depend to some degree on the inspection of historical data. However, exclusive reliance on historical data inevitably leads to an overemphasis on evolutionary innovation and leaves the user vulnerable to surprise from rapid or nonlinear developments. In this report, a disruptive technology is an innovative technology that triggers sudden and unexpected effects. A methodology that can forecast disruptive technologies must overcome the evolutionary bias and be capable of identifying unprecedented change. A disruptive event often arrives abruptly and infrequently and is therefore particularly hard to predict using an evolutionary approach. The technology that precipitates the event may have existed for many years before it has its effect, and the effect may be cascading, nonlinear, and difficult to anticipate.

New forecasting methods must be developed if disruptive technology forecasts are to be effective. Promising areas include applications from chaos theory; artificial neural networks; influence diagrams and decision networks; advanced simulations; prediction markets; online social networks; and alternate reality games.

3

Signtific, originally known as the X2 project, is a forecasting system that aims to provide an innovative medium for discussing the future of science and technology. It is designed to identify the most important trends and disruptions in science and technology and their impacts on the larger society over the next 20 years. Signtific is built and run by the Institute for the Future (http://www.iftf.org/node/939).

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×

BOX S-1

Statement of Task

The NRC will establish an ad hoc committee that will provide technology analyses to assist in the development of timelines, methodologies, and strategies for the identification of global technology trends. The analyses performed by the NRC committee will not only identify future technologies of interest and their application but will also assess technology forecasting methodologies of use both in the government and in other venues in an effort to identify those most useful and productive. The duration of the project is twenty-four months; two reports will be provided.

Specifically, the committee will in its first report:

  • Compare and contrast attributes of technology forecasting methodologies developed to meet similar needs in other venues.

  • Identify the necessary attributes and metrics of a persistent worldwide technology forecasting platform.*

  • Identify data sets, sources, and collection techniques for forecasting technologies of potential value.

  • Comment on the technology forecasting approach set forth by the sponsor.

    • Comment on the Delta Scan data sets and/or other data sets provided by the sponsor.

  • Describe effective “dashboard” techniques for forecasting scenarios.

  • From real-time data provided by the sponsor:

    • Select and comment on emerging technology sectors.

    • Advise the sponsor on where and how emerging and persistent technologies trends might become disruptive.

    • Provide rationale for selections and indicate what key aspects will influence the rate of development in each.

The first report will be provided 16 months from contract award. The committee’s second report will be delivered during the second year, and will expand and refine report one in light of subsequent information provided by the more complete technology analyses anticipated. The statement of task of the final report will be developed in the course of meetings of the NRC staff and sponsor and will be brought back to the NRC for approval.

  

*After discussion, the committee chose to use the word “system” instead of “platform” throughout the report, due to the fact that the term platform has developed different connotations over time. This change to the Statement of Task was agreeable to the sponsor.

OVERVIEW OF FORECASTING TECHNIQUES

The field of technology forecasting is relatively new, dating back to work from the RAND Corporation during the years immediately following World War II (WWII). One of the earliest methods employed was the Delphi method, a structured process for eliciting collective expert opinions on technological trends and their impacts (Dalkey, 1967). Gaming and scenario planning also emerged as important technology forecasting methods in the 1950s and dramatically increased in popularity during the 1970s. All of these methods, as well as other more quantitative methods, are in use today.

In general, current forecasting methods can be broken into four categories: judgmental or intuitive methods; extrapolation and trend analysis; models; and scenarios and simulation. The advent of ever more powerful computation platforms and the growing availability of electronic data have led to a steady increase in the use of quantita-

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×

tive methods as part of the technology forecasting process. New Internet-based forecasting tools and methods are leveraging the power of open source applications, social networks, expert sourcing (using prescreened experts to make technology forecasts), and crowd sourcing (allowing public participation with no prerequisites).

The committee believes that there is no single perfect method for forecasting disruptive technologies. Each has its strengths and weaknesses. Before choosing one or more methodologies to employ, a forecaster should consider the resources that can be applied to the forecast (financial, technology, forecasting infrastructure, and human capital), the nature and category of the technology being forecasted, the availability of experts and willingness of the crowd to participate in a forecast, the time frame that the forecast must address, and how the stakeholders intend to use the forecast.

Several pioneering systems already exist that attempt to forecast technology trends, including TechCast, Delta Scan, and X2.4 The committee chose to examine these platforms because they incorporate many of the committee-defined attributes of a well-designed disruptive technology forecasting system. Also, all three platforms are currently used by researchers and governments to aid in the forecasting of disruptive technologies—TechCast and X2 are used by the U.S. government and Delta Scan was developed for the government of the United Kingdom. The committee was briefed by the teams responsible for the systems. Analysis of these systems offers important insights into the creation of persistent forecasts:

  • TechCast (1998). Voluntary self-selecting of people who examine technology advances on an ad hoc basis. The system’s strengths include persistence, quantification of forecasts, and ease of use.

  • Delta Scan (2005). Part of the United Kingdom’s Horizon Scanning Centre, organized with the goal of becoming a persistent system.

  • X2 (2007). Persistent system with a novel architecture, qualitative assessment, and integration of multiple forecasting techniques.

These existing systems demonstrate that ambitious and sophisticated systems can help anticipate new technologies and applications and their potential impact.

Forecasting systems such as X2/Signtific use a combination of techniques such as the Delphi method, alternative reality gaming, and expert sourcing to produce a forecast. Others such as TechCast5 employ expert sourcing in a Web environment. Popular Science’s Prediction Exchange (PPX)6 combined crowd sourcing and predictive markets to develop technology forecasts.

ATTRIBUTES OF AN EFFECTIVE SYSTEM

The following are viewed by the committee as important attributes of a well-designed system for forecasting disruptive technologies. Most are covered more thoroughly in Chapter 5. Proactive bias mitigation is discussed in detail in Chapter 4.

  • Openness. An open approach allows the use of crowd resources to identify potentially disruptive technologies and to help understand their possible impact. Online repositories such as Wikipedia and SourceForge.net have shown the power of public-sourced, high-quality content. Openness can also facilitate an understanding of the consumer and commercial drivers of technology and what disruptions they might produce. In a phenomenon that New York Times’ reporter John Markoff has dubbed “inversion,” many advanced

4

In 2009, the name “X2” was changed to “Signtific: Forecasting Future Disruptions in Science and Technology.”

5

TechCast is a technology think tank pooling the collective knowledge of technology experts around the world to produce authoritative technology forecasts for strategic business decisions. TechCast offers online technology forecasts and publishes articles on emerging technologies. It has been online since 1998. TechCast was developed by William E. Halal and his associates at George Washington University. Available at http://www.techcast.org/.

6

Popular Science’s Prediction Exchange (PPX) is an online virtual prediction market run as part of the magazine’s Web sites, where users trade virtual currency, known as POP$, based on the likelihood of a certain event being realized by a given date. The prediction market ran from June 2007 until May 2009. At its peak, PPX had over 37,000 users. Available at http://en.wikipedia.org/wiki/PPX.

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×

technologies are now arriving first in the hands of the ordinary consumers, who are the largest market segment. These technologies then slowly penetrate smaller and more elite markets, such as large business or the military (Markoff, 1996). Openness in a forecasting process does not mean that all information should be open and shared. Information that affects national security or violates the proprietary rights or trade secrets of an individual, organization, or company is justifiably classified and has special data-handling requirements. Forecasters need to consider these special requirements as they design and implement a forecasting system.

  • Persistence. In today’s environment, planning cycles are highly dynamic, and cycle times can be measured in days instead of years. For this reason it is important to have a forecasting system that monitors, tracks, and reformulates predictions based on new inputs and collected data. A well-designed persistent system should encourage the continuous improvement of forecasting methodologies and should preserve historical predictions, forecasts, signals, and data. In doing so, forecasts and methodologies can be easily compared and measured for effectiveness and accuracy. Openness and persistence are synergistic: Open and persistent systems promote the sharing of new ideas, encourage new research, and promote interdisciplinary approaches to problem solving and technology assessment.

  • Transparency. The contributors and users of the system need to trust that the system operators will not exploit personal or other contributed information for purposes other than those intended. The system should publish and adhere to policies on how it uses, stores, and tracks information.

  • Structural flexibility. This should be sufficient to respond to complexity, uncertainty, and changes in technology and methodology.

  • Easy access. The system should be easy to use and broadly available to all users.

  • Proactive bias mitigation. The main kinds of bias are cultural, linguistic, regional, generational, and experiential. A forecasting system should therefore be implemented to encourage the participation of individuals from a wide variety of cultural, geographic, and linguistic backgrounds to ensure a balance of viewpoints. In many fields, technology is innovated by young researchers, technologists, and entrepreneurs. Unfortunately, this demographic is overlooked by the many forecasters who seek out seasoned and established experts. It is important that an open system include input from the generation most likely to be the source of disruptive technologies and be most affected by them.

  • Incentives to participate.

  • Reliable data construction and maintenance.

  • Tools to detect anomalies and sift for weak signals. A weak signal is an early warning of change that typically becomes stronger when combined with other signals.

  • Strong visualization tools and a graphical user interface.

  • Controlled vocabulary. The vocabulary of a forecast should include an agreed-upon set of terms that are easy for both operators and users to understand.

BENCHMARKING A PERSISTENT FORECASTING SYSTEM

After much discussion, the committee agreed on several characteristics of an ideal forecast that could be used to benchmark a persistent forecasting system. The following considerations were identified as important for designing a persistent forecasting system:

  • Data sources. Data must come from a diverse group of individuals and collection methods and should consist of both quantitative and qualitative data.

  • Multiple forecasting method. The system should combine existing and novel forecasting methodologies that use both quantitative and qualitative techniques.

  • Forecasting team. A well-managed forecasting team is necessary to ensure expert diversity, encourage public participation, and help with ongoing recruitment.

  • Forecast output. Both quantitative and qualitative forecast data should be presented in a readily available, intuitive format.

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
  • Processing tools. The system should incorporate tools that assess impact, threshold levels, and scalability; detect outlier and weak signals; and aid with visualization.

  • System attributes. The system should be global, persistent, open, scalable and flexible, with consistent and simple terminology; it should also support multiple languages, include incentives for participation, and be easy to use.

  • Environmental considerations. Financial support, data protection, infrastructure support, and auditing and review processes must also be considered.

HOW TO BUILD A PERSISTENT FORECASTING SYSTEM

Building a persistent forecasting system can be a complex and daunting task. Such a system is a collection of technologies, people, and processes. The system being described is not a software-only system. It is important to understand both the power and the limits of current computer science and not try to force the computer to perform tasks that humans can perform better. Computers are great tools for raw data mining, automated data gathering (“spidering”), statistical computation, data management, quantitative analysis, and visualization. Humans are best at pattern recognition, natural language interpretation and processing, intuition, and qualitative analysis. A well-designed system leverages the best attributes of both human and machine processes.

The committee recommends that a persistent forecasting system be built in phases and over a number of years. Successful Web-based systems, for example, usually use a spiral development approach to gradually add complexity to a program until it reaches completion.

The committee outlined eight important steps for performing an effective persistent forecast for disruptive technologies. These steps include:

  • Define the goals of the mission by understanding key stakeholders’ objectives.

  • Determine the scope of the mission by ascertaining which people and resources are required to successfully put the system together, and meet mission objectives.

  • Select appropriate forecasting methodologies to meet the mission objectives given the requirements and the availability of data and resources. Develop and use methods to recognize key precursors to disruptions, identifying as many potential disruptive events as possible.

  • Gather information from key experts and information sources using ongoing information-gathering processes such as assigning metadata, assessing data sources, gathering historical reference data, assessing and mitigating biases, prioritizing signals, and applying processing and monitoring tools.

  • Prioritize forecast technologies by estimating their potential impact and proximity in order to determine which signals to track, necessary threshold levels, and optimal resource allocation methods.

  • Optimize the tools used to process, monitor, and report outliers, potential sources of surprise, weak signals, signposts, and changes in historical relationships, often in noisy information environments.

  • Develop resource allocation and decision-support tools that allow decision makers to track and optimize their reactions as the probabilities of potential disruptions change.

  • Assess, audit, provide feedback, and improve forecasts and forecasting methodologies.

CONCLUSION

This is the first of two reports on disruptive technology forecasting. Its goal is to help the reader understand current forecasting methodologies, the nature of disruptive technologies, and the characteristics of a persistent forecasting system for disruptive technology. In the second report, the committee plans to summarize the results of a workshop which will assemble leading experts on forecasting, system architecture, and visualization, and ask them to envision a system that meets the sponsor requirements while incorporating the desired attributes listed in this report.

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×

REFERENCES

Dalkey, Norman C. 1967. DELPHI. Santa Monica, Calif.: RAND Corporation.

Markoff, John. 1996. I.B.M. disk is said to break billion-bit barrier. New York Times. April 15.

Sun, Tzu. 599-500 B.C. The Art of War. Edited and translated by Thomas Cleary, 1991. Boston: Shambhala Publications.

Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 1
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 2
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 3
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 4
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 5
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 6
Suggested Citation:"Summary." National Research Council. 2010. Persistent Forecasting of Disruptive Technologies. Washington, DC: The National Academies Press. doi: 10.17226/12557.
×
Page 7
Next: 1 Need for Persistent Long-Term Forecasting of Disruptive Technologies »
Persistent Forecasting of Disruptive Technologies Get This Book
×
Buy Paperback | $46.00 Buy Ebook | $36.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Technological innovations are key causal agents of surprise and disruption. In the recent past, the United States military has encountered unexpected challenges in the battlefield due in part to the adversary's incorporation of technologies not traditionally associated with weaponry. Recognizing the need to broaden the scope of current technology forecasting efforts, the Office of the Director, Defense Research and Engineering (DDR&E) and the Defense Intelligence Agency (DIA) tasked the Committee for Forecasting Future Disruptive Technologies with providing guidance and insight on how to build a persistent forecasting system to predict, analyze, and reduce the impact of the most dramatically disruptive technologies. The first of two reports, this volume analyzes existing forecasting methods and processes. It then outlines the necessary characteristics of a comprehensive forecasting system that integrates data from diverse sources to identify potentially game-changing technological innovations and facilitates informed decision making by policymakers.

The committee's goal was to help the reader understand current forecasting methodologies, the nature of disruptive technologies and the characteristics of a persistent forecasting system for disruptive technology. Persistent Forecasting of Disruptive Technologies is a useful text for the Department of Defense, Homeland Security, the Intelligence community and other defense agencies across the nation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!