Summary
CONTEXT
In The Art of War, written in the 6th century B.C., Sun Tzu described surprise:
In conflict, direct confrontation will lead to engagement and surprise will lead to victory. Those who are skilled in producing surprises will win. Such tacticians are as versatile as the changes in heaven and earth.1
Novel technologies are one of the principal means of surprising enemies or competitors and of disrupting established ways of doing things. Military examples of surprise include the English longbow, the Japanese long lance torpedo, the American atomic bomb, stealth technologies, and the Global Positioning System (GPS). Commercial examples include the telephone (Bell), business computers (UNIVAC and IBM), mobile phones (Motorola), recombinant DNA technologies (Genentech), PageRank (Google), and the iPod (Apple).
Until the 1970s, technological innovation tended to come from a limited number of well-established “techno clusters” and national and corporate laboratories.2 Today, the number of techno clusters and laboratories is growing rapidly everywhere. Policy makers are concerned with the emergence of high-impact technologies that could trigger sudden, unexpected changes in national economies or in the security and quality of life they enjoy and that might affect the regional, national, or global balance of power. As such, policy makers and strategic planners use technology forecasts in their planning.
The value of technology forecasting lies not in its ability to accurately predict the future but rather in its potential to minimize surprises. It does this by various means:
-
Defining and looking for key enablers and inhibitors of new disruptive technologies,
-
Assessing the impact of potential disruption,
1 |
Available at http://www.mailsbroadcast.com/the.artofwar.5.htm. Last accessed March 3, 2009. |
2 |
A techno cluster refers to a science- and high-tech-oriented Porter’s cluster or business cluster (available at http://www.economicexpert.com/a/Techno:cluster:fi.htm; last accessed May 6, 2009). A business cluster is a geographic concentration of interconnected businesses, suppliers, and associated institutions in a particular field. Clusters are considered to increase the productivity with which companies can compete, nationally and globally. The term “industry cluster,” also known as a business cluster, a competitive cluster, or a Porterian cluster, was introduced, and the term “cluster” was popularized by Michael Porter in The Competitive Advantage of Nations (1990). Available at http://en.wikipedia.org/wiki/Business_cluster. Last accessed March 3, 2009. |
-
Postulating potential alternative futures, and
-
Supporting decision making by increasing the lead time for awareness.
The Office of the Director of Defense Research and Engineering (DDR&E) and the Defense Intelligence Agency (DIA) Defense Warning Office (DWO) asked the National Research Council (NRC) to set up a committee on forecasting future disruptive technologies to provide guidance on and insight into the development of a system that could forecast disruptive technology. The sponsor recognizes that many of the enabling disruptive technologies employed by an enemy could potentially come out of nonmilitary applications. Understanding this problem, the sponsor asked the committee to pay particular attention to ways of forecasting technical innovations that are driven by market demand and opportunities. It was agreed that the study should be unclassified and that participation in it not require security clearances. The sponsor and the committee strongly believe that if a forecasting system were to be produced that was useful in identifying technologies driven by market demand, especially global demand, then it would probably have significant value to a broad range of users beyond the Department of Defense and outside the United States. The sponsor and the committee also believe that the creation of an unclassified system is crucial to their goal of eliciting ongoing global participation. The sponsor asked the committee to consider the attributes of “persistent” forecasting systems—that is, systems that can be continually improved as new data and methodologies become available. See Box S-1 for the committee’s statement of task.
This report is the first of two requested by the sponsors. In this first report, the committee discusses how technology forecasts are made, assesses several existing forecasting systems, and identifies the attributes of a persistent disruptive forecasting system. The second report will develop forecasting options specifically tailored to needs of the sponsors.
It is important to note that the sponsor has not asked the committee to build and design a forecasting system at this time. Instead, the intent of this report is to look at existing forecasting methodologies, to discuss important attributes and metrics of a persistent system for forecasting disruptive technologies, and to examine and comment on selected existing systems for forecasting disruptive technologies.
In 2007, the sponsor contracted the development of a persistent forecasting system called X2 (the name was later changed to Signtific).3 At the time of this writing, not enough data had been generated from this system to provide a meaningful analysis of potentially disruptive technology sectors. The characteristics of X2 are analyzed in depth in Chapter 6.
CHALLENGE OF SUCH FORECASTS
All forecasting methodologies depend to some degree on the inspection of historical data. However, exclusive reliance on historical data inevitably leads to an overemphasis on evolutionary innovation and leaves the user vulnerable to surprise from rapid or nonlinear developments. In this report, a disruptive technology is an innovative technology that triggers sudden and unexpected effects. A methodology that can forecast disruptive technologies must overcome the evolutionary bias and be capable of identifying unprecedented change. A disruptive event often arrives abruptly and infrequently and is therefore particularly hard to predict using an evolutionary approach. The technology that precipitates the event may have existed for many years before it has its effect, and the effect may be cascading, nonlinear, and difficult to anticipate.
New forecasting methods must be developed if disruptive technology forecasts are to be effective. Promising areas include applications from chaos theory; artificial neural networks; influence diagrams and decision networks; advanced simulations; prediction markets; online social networks; and alternate reality games.
3 |
Signtific, originally known as the X2 project, is a forecasting system that aims to provide an innovative medium for discussing the future of science and technology. It is designed to identify the most important trends and disruptions in science and technology and their impacts on the larger society over the next 20 years. Signtific is built and run by the Institute for the Future (http://www.iftf.org/node/939). |
BOX S-1 Statement of Task The NRC will establish an ad hoc committee that will provide technology analyses to assist in the development of timelines, methodologies, and strategies for the identification of global technology trends. The analyses performed by the NRC committee will not only identify future technologies of interest and their application but will also assess technology forecasting methodologies of use both in the government and in other venues in an effort to identify those most useful and productive. The duration of the project is twenty-four months; two reports will be provided. Specifically, the committee will in its first report:
The first report will be provided 16 months from contract award. The committee’s second report will be delivered during the second year, and will expand and refine report one in light of subsequent information provided by the more complete technology analyses anticipated. The statement of task of the final report will be developed in the course of meetings of the NRC staff and sponsor and will be brought back to the NRC for approval. |
OVERVIEW OF FORECASTING TECHNIQUES
The field of technology forecasting is relatively new, dating back to work from the RAND Corporation during the years immediately following World War II (WWII). One of the earliest methods employed was the Delphi method, a structured process for eliciting collective expert opinions on technological trends and their impacts (Dalkey, 1967). Gaming and scenario planning also emerged as important technology forecasting methods in the 1950s and dramatically increased in popularity during the 1970s. All of these methods, as well as other more quantitative methods, are in use today.
In general, current forecasting methods can be broken into four categories: judgmental or intuitive methods; extrapolation and trend analysis; models; and scenarios and simulation. The advent of ever more powerful computation platforms and the growing availability of electronic data have led to a steady increase in the use of quantita-
tive methods as part of the technology forecasting process. New Internet-based forecasting tools and methods are leveraging the power of open source applications, social networks, expert sourcing (using prescreened experts to make technology forecasts), and crowd sourcing (allowing public participation with no prerequisites).
The committee believes that there is no single perfect method for forecasting disruptive technologies. Each has its strengths and weaknesses. Before choosing one or more methodologies to employ, a forecaster should consider the resources that can be applied to the forecast (financial, technology, forecasting infrastructure, and human capital), the nature and category of the technology being forecasted, the availability of experts and willingness of the crowd to participate in a forecast, the time frame that the forecast must address, and how the stakeholders intend to use the forecast.
Several pioneering systems already exist that attempt to forecast technology trends, including TechCast, Delta Scan, and X2.4 The committee chose to examine these platforms because they incorporate many of the committee-defined attributes of a well-designed disruptive technology forecasting system. Also, all three platforms are currently used by researchers and governments to aid in the forecasting of disruptive technologies—TechCast and X2 are used by the U.S. government and Delta Scan was developed for the government of the United Kingdom. The committee was briefed by the teams responsible for the systems. Analysis of these systems offers important insights into the creation of persistent forecasts:
-
TechCast (1998). Voluntary self-selecting of people who examine technology advances on an ad hoc basis. The system’s strengths include persistence, quantification of forecasts, and ease of use.
-
Delta Scan (2005). Part of the United Kingdom’s Horizon Scanning Centre, organized with the goal of becoming a persistent system.
-
X2 (2007). Persistent system with a novel architecture, qualitative assessment, and integration of multiple forecasting techniques.
These existing systems demonstrate that ambitious and sophisticated systems can help anticipate new technologies and applications and their potential impact.
Forecasting systems such as X2/Signtific use a combination of techniques such as the Delphi method, alternative reality gaming, and expert sourcing to produce a forecast. Others such as TechCast5 employ expert sourcing in a Web environment. Popular Science’s Prediction Exchange (PPX)6 combined crowd sourcing and predictive markets to develop technology forecasts.
ATTRIBUTES OF AN EFFECTIVE SYSTEM
The following are viewed by the committee as important attributes of a well-designed system for forecasting disruptive technologies. Most are covered more thoroughly in Chapter 5. Proactive bias mitigation is discussed in detail in Chapter 4.
-
Openness. An open approach allows the use of crowd resources to identify potentially disruptive technologies and to help understand their possible impact. Online repositories such as Wikipedia and SourceForge.net have shown the power of public-sourced, high-quality content. Openness can also facilitate an understanding of the consumer and commercial drivers of technology and what disruptions they might produce. In a phenomenon that New York Times’ reporter John Markoff has dubbed “inversion,” many advanced
4 |
In 2009, the name “X2” was changed to “Signtific: Forecasting Future Disruptions in Science and Technology.” |
5 |
TechCast is a technology think tank pooling the collective knowledge of technology experts around the world to produce authoritative technology forecasts for strategic business decisions. TechCast offers online technology forecasts and publishes articles on emerging technologies. It has been online since 1998. TechCast was developed by William E. Halal and his associates at George Washington University. Available at http://www.techcast.org/. |
6 |
Popular Science’s Prediction Exchange (PPX) is an online virtual prediction market run as part of the magazine’s Web sites, where users trade virtual currency, known as POP$, based on the likelihood of a certain event being realized by a given date. The prediction market ran from June 2007 until May 2009. At its peak, PPX had over 37,000 users. Available at http://en.wikipedia.org/wiki/PPX. |
-
technologies are now arriving first in the hands of the ordinary consumers, who are the largest market segment. These technologies then slowly penetrate smaller and more elite markets, such as large business or the military (Markoff, 1996). Openness in a forecasting process does not mean that all information should be open and shared. Information that affects national security or violates the proprietary rights or trade secrets of an individual, organization, or company is justifiably classified and has special data-handling requirements. Forecasters need to consider these special requirements as they design and implement a forecasting system.
-
Persistence. In today’s environment, planning cycles are highly dynamic, and cycle times can be measured in days instead of years. For this reason it is important to have a forecasting system that monitors, tracks, and reformulates predictions based on new inputs and collected data. A well-designed persistent system should encourage the continuous improvement of forecasting methodologies and should preserve historical predictions, forecasts, signals, and data. In doing so, forecasts and methodologies can be easily compared and measured for effectiveness and accuracy. Openness and persistence are synergistic: Open and persistent systems promote the sharing of new ideas, encourage new research, and promote interdisciplinary approaches to problem solving and technology assessment.
-
Transparency. The contributors and users of the system need to trust that the system operators will not exploit personal or other contributed information for purposes other than those intended. The system should publish and adhere to policies on how it uses, stores, and tracks information.
-
Structural flexibility. This should be sufficient to respond to complexity, uncertainty, and changes in technology and methodology.
-
Easy access. The system should be easy to use and broadly available to all users.
-
Proactive bias mitigation. The main kinds of bias are cultural, linguistic, regional, generational, and experiential. A forecasting system should therefore be implemented to encourage the participation of individuals from a wide variety of cultural, geographic, and linguistic backgrounds to ensure a balance of viewpoints. In many fields, technology is innovated by young researchers, technologists, and entrepreneurs. Unfortunately, this demographic is overlooked by the many forecasters who seek out seasoned and established experts. It is important that an open system include input from the generation most likely to be the source of disruptive technologies and be most affected by them.
-
Incentives to participate.
-
Reliable data construction and maintenance.
-
Tools to detect anomalies and sift for weak signals. A weak signal is an early warning of change that typically becomes stronger when combined with other signals.
-
Strong visualization tools and a graphical user interface.
-
Controlled vocabulary. The vocabulary of a forecast should include an agreed-upon set of terms that are easy for both operators and users to understand.
BENCHMARKING A PERSISTENT FORECASTING SYSTEM
After much discussion, the committee agreed on several characteristics of an ideal forecast that could be used to benchmark a persistent forecasting system. The following considerations were identified as important for designing a persistent forecasting system:
-
Data sources. Data must come from a diverse group of individuals and collection methods and should consist of both quantitative and qualitative data.
-
Multiple forecasting method. The system should combine existing and novel forecasting methodologies that use both quantitative and qualitative techniques.
-
Forecasting team. A well-managed forecasting team is necessary to ensure expert diversity, encourage public participation, and help with ongoing recruitment.
-
Forecast output. Both quantitative and qualitative forecast data should be presented in a readily available, intuitive format.
-
Processing tools. The system should incorporate tools that assess impact, threshold levels, and scalability; detect outlier and weak signals; and aid with visualization.
-
System attributes. The system should be global, persistent, open, scalable and flexible, with consistent and simple terminology; it should also support multiple languages, include incentives for participation, and be easy to use.
-
Environmental considerations. Financial support, data protection, infrastructure support, and auditing and review processes must also be considered.
HOW TO BUILD A PERSISTENT FORECASTING SYSTEM
Building a persistent forecasting system can be a complex and daunting task. Such a system is a collection of technologies, people, and processes. The system being described is not a software-only system. It is important to understand both the power and the limits of current computer science and not try to force the computer to perform tasks that humans can perform better. Computers are great tools for raw data mining, automated data gathering (“spidering”), statistical computation, data management, quantitative analysis, and visualization. Humans are best at pattern recognition, natural language interpretation and processing, intuition, and qualitative analysis. A well-designed system leverages the best attributes of both human and machine processes.
The committee recommends that a persistent forecasting system be built in phases and over a number of years. Successful Web-based systems, for example, usually use a spiral development approach to gradually add complexity to a program until it reaches completion.
The committee outlined eight important steps for performing an effective persistent forecast for disruptive technologies. These steps include:
-
Define the goals of the mission by understanding key stakeholders’ objectives.
-
Determine the scope of the mission by ascertaining which people and resources are required to successfully put the system together, and meet mission objectives.
-
Select appropriate forecasting methodologies to meet the mission objectives given the requirements and the availability of data and resources. Develop and use methods to recognize key precursors to disruptions, identifying as many potential disruptive events as possible.
-
Gather information from key experts and information sources using ongoing information-gathering processes such as assigning metadata, assessing data sources, gathering historical reference data, assessing and mitigating biases, prioritizing signals, and applying processing and monitoring tools.
-
Prioritize forecast technologies by estimating their potential impact and proximity in order to determine which signals to track, necessary threshold levels, and optimal resource allocation methods.
-
Optimize the tools used to process, monitor, and report outliers, potential sources of surprise, weak signals, signposts, and changes in historical relationships, often in noisy information environments.
-
Develop resource allocation and decision-support tools that allow decision makers to track and optimize their reactions as the probabilities of potential disruptions change.
-
Assess, audit, provide feedback, and improve forecasts and forecasting methodologies.
CONCLUSION
This is the first of two reports on disruptive technology forecasting. Its goal is to help the reader understand current forecasting methodologies, the nature of disruptive technologies, and the characteristics of a persistent forecasting system for disruptive technology. In the second report, the committee plans to summarize the results of a workshop which will assemble leading experts on forecasting, system architecture, and visualization, and ask them to envision a system that meets the sponsor requirements while incorporating the desired attributes listed in this report.