6
Evaluating Existing Persistent Forecasting Systems

INTRODUCTION

The committee selected three existing forecasting systems with which to compare the committee’s ideal system characteristics. Three criteria were considered for their selection. First, each of the systems attempted to forecast and track multiple technology arenas of interest to society at large (information technology, biotechnology, energy, and so on). Second, each system solicited the viewpoints of multiple experts, though the degree of openness to nonexperts varied. Third, the committee was able to access each system and meet at least once with a senior member of the staff. The Institute for the Future’s (IFTF’s) X2 system was chosen at the sponsor’s request. The remaining two systems, Delta Scan and TechCast, were chosen because they facilitated evaluation of different approaches to forecasting, presentation of the data, persistence, and openness to the public.

Each of the three selected systems is described next, along with its strengths and weaknesses in comparison to an ideal system. The chapter concludes with a side-by-side analysis of all three systems according to six key characteristics.

DELTA SCAN

Delta Scan was developed as a tool for the Horizon Scanning Centre as part of the United Kingdom’s national persistent forecasting effort. It is the sister system to Sigma Scan, which focuses on public policy and social issues. In contrast, Delta Scan focuses on technology. Initially guided by IFTF, the system first collected science and technology perspectives from over 250 experts in government, business, academia, and communications through workshops, interviews, and wikis. Then a team from the Horizon Scanning Centre reviewed projects and programs to determine strategic priorities, with the goal of augmenting overall technological capabilities. According to Horizon Scanning Center staff member Harry Woodroof, in his presentation to the committee on October 15, 2007, the goal of the Center was to perform

the systematic examination of potential threats, opportunities, and likely developments, including but not restricted to those at the margins of current thinking and planning. Horizon scanning may explore novel and unexpected issues as well as persistent problems or trends (Woodroof, 2007).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 92
6 Evaluating Existing Persistent Forecasting Systems INTRODuCTION The committee selected three existing forecasting systems with which to compare the committee’s ideal system characteristics. Three criteria were considered for their selection. First, each of the systems attempted to forecast and track multiple technology arenas of interest to society at large (information technology, biotechnology, energy, and so on). Second, each system solicited the viewpoints of multiple experts, though the degree of openness to nonexperts varied. Third, the committee was able to access each system and meet at least once with a senior member of the staff. The Institute for the Future’s (IFTF’s) X2 system was chosen at the sponsor’s request. The remaining two systems, Delta Scan and TechCast, were chosen because they facilitated evaluation of different approaches to forecasting, presentation of the data, persistence, and openness to the public. Each of the three selected systems is described next, along with its strengths and weaknesses in comparison to an ideal system. The chapter concludes with a side-by-side analysis of all three systems according to six key characteristics. DELTA SCAN Delta Scan was developed as a tool for the Horizon Scanning Centre as part of the United Kingdom’s national persistent forecasting effort. It is the sister system to Sigma Scan, which focuses on public policy and social issues. In contrast, Delta Scan focuses on technology. Initially guided by IFTF, the system first collected science and technology perspectives from over 250 experts in government, business, academia, and communications through workshops, interviews, and wikis. Then a team from the Horizon Scanning Centre reviewed projects and pro - grams to determine strategic priorities, with the goal of augmenting overall technological capabilities. According to Horizon Scanning Center staff member Harry Woodroof, in his presentation to the committee on October 15, 2007, the goal of the Center was to perform the systematic examination of potential threats, opportunities, and likely developments, including but not restricted to those at the margins of current thinking and planning. Horizon scanning may explore novel and unexpected issues as well as persistent problems or trends (Woodroof, 2007). 9

OCR for page 92
9 EVALUATING EXISTING PERSISTENT FORECASTING SYSTEMS The resulting technology perspectives were collected and synthesized into 100 papers by the Scanning Centre and the IFTF as they looked at potential developments in the field of science and technology over the next 50 years. The forecast comprised a short “outlook” and an Accompanying “snippet”; a set of overarching themes; and a map of the future. One example of a single-sentence outlook and its “snippet”—here, radio frequency identification—is as follows: Tracking objects is made easy by RFID Radio Frequency Identification Devices (RFID) tagging systems will probably be widely used by government, in - dustry, retailers and consumers to identify and track physical objects by 2015. This will vastly improve corporate inventory management, increase the efficiency of logistics, reduce loss (including through theft) at all stages between manufacturer and end-user, facilitate scientific research in hospitals by making it easier to track patients and lab samples, and make mislaid objects a thing of the past as individuals will be able to “google” and hence locate objects. IFTF identified six science and technology (S&T) themes from the forecast: • Small world, • Intentional biology, • Extended self, • Mathematical world, • Sensory transformation, and • Lightweight infrastructure. It also identified three meta themes: democratized innovation, transdisciplinarity, and emergence. The map of the future in Figure 6-1 was created by IFTF. The results of the forecast were published both in a report and on the Horizon Scanning Centre’s Web site. They were then were used to test strategic assumptions and to identify contingency trigger points to monitor. An impact audit was performed that weighed the identified risks against existing projects and priorities. The results were used to generate new questions and create fresh strategic priorities that could be fed back into the forecasting system. See Figure 6-2 for a diagram of the Delta Scan forecasting process. Strengths and Weaknesses A strength of this forecasting platform is that its goals, process, and approach were well defined from inception. It was designed to ensure that the architecture of the underlying data store could support and produce a forecast. The forecasting process was straightforward, practical, and used established forecasting methods such as interviews with experts and professionally led workshops. The system was developed with a modest level of resources and called on professional forecasters (IFTF staff) for help. Participants were drawn from areas of expertise that cor - responded to stakeholders’ priorities. The output of the forecast was clear and concise and helped to drive decision making, program direction, and resource allocation. In addition, the resulting time line and wiki are useful to future planners and forecasters. The system, though not persistent, was designed to be iterative. The system’s potential weaknesses include its lack of support for languages other than English, its emphasis on data of local rather than global origin, the exclusive use of expert views, and the single-dimensionality of the resulting forecast, which failed to offer alternatives to its vision of the future. The system was designed as an itera - tive platform for forecasting, but the time between forecasts is relatively long, and new signals do not immediately impact the forecast. The system requires the discrete linear processing of each step (goal setting, process design, interviews and scans, data normalization and preparation, workshops, synthesis, and audit of impact), and within each forecasting cycle all steps must be completed before a new forecast is produced. The system is therefore not designed to be persistent. While the resulting forecast was insightful, it was not particularly surprising. Topic areas

OCR for page 92
9 PERSISTENT FORECASTING OF DISRUPTIVE TECHNOLOGIES FIGURE 6-1 Science and technology outlook, 2005-2055. SOURCE: Woodroof, 2007. Crown Copyright, 2005. Produced for the U.K. Foresight Horizon Scanning Centre by the Institute for the Future. Used with permission from the U.K. Foresight Horizon Scanning Centre. • Builds Commitment & Impact Question Owner? Outcome? • Stakeholder Management Process design • Rapid Delivery of Diverse Contex t Interviews / Scans + • Cranking up endorsement • Cognitive Challenge / Priming Prep material • Cauldron of Interactions Workshop (professional) • Challenge, Insight Econ / Sci / Social / Other • Co -creation Public / Private • Managing authority / credibility Synthesis & QA of outputs • Evolve & promote model Audit of Impacts FIGURE 6-2 Process diagram from the Foresight Horizon Scanning Centre. SOURCE: Woodroof, 2007. Produced by the United Kingdom’s Foresight Horizon Scanning Centre and used with the center’s permission. 6-2.eps

OCR for page 92
95 EVALUATING EXISTING PERSISTENT FORECASTING SYSTEMS tended to be limited to well-understood technologies, and not enough attention was paid to possible wild cards or second-order effects. The committee would like to have seen more detailed regional analysis performed in the report and a greater emphasis on identification of possible signposts and tipping points. TECHCAST TechCast is an online research system which acts as a robust tool for pooling knowledge and forecasting tech - nology. This system is designed to be highly flexible and can be used to gather expert data from an enterprise, a specific target group, a government, or the general public. TechCast relies on the observations of volunteer experts to forecast technological advances. The experts, who are either invited or self-nominated, are regularly surveyed on topics in their area of expertise. The resulting information is then used to create reports on an ad hoc basis. TechCast was launched in 1998 by William E. Halal at George Washington University. The administrator of TechCast first scans a variety of sources to find technology that would be interesting to forecast. A wiki is produced that lists key events, data points, and trends. One hundred experts from around the world are then asked to make a prediction and comment on the predictions of other participants using a computational wiki system. Specifically, the experts are asked to predict when a new technology will be adopted, the scale of its market impact, and the expert’s confidence in his or her own forecast. It also encourages experts to explain their prediction. Figure 6-3 is a diagram of the TechCast process (Halal, 2009). This forecasting system is persistent: Each input into it automatically updates the forecast. TechCast.org has produced forecasts in 70 technology areas. Users can scan the prediction results using a wiki. Figure 6-4 summa - rizes the forecasts produced by the system. It describes when experts believe a new technology will reach adoption, the scale of the market, and experts’ confidence in the prediction. The computational wiki is automatically updated and its contents are added to a dashboard displaying the expert survey results, as seen in Figure 6-5. Figure 6-6 summarizes a set of TechCast.org forecasts by time, market size, and average expert confidence, while Figure 6-7 summarizes the data by sector and the expected year of mainstream adoption. Strengths and Weaknesses TechCast is a very flexible platform and can support a wide range of prediction needs. Its strength lies in its simplicity. The system is entirely wiki based and does not require physical proximity of the forecasters. The four- step process is straightforward and simple to administer and can be maintained with few resources. It is easy to use and is persistent. Forecasters get instant results, and the site is as current as the last prediction. Output is clear and easy to understand. TechCast relies on a geographically diverse set of experts for its input; approximately half the contributors are foreign. Experts are prescreened and can comment and predict across disciplines, allowing the cross-fertilization of ideas and expertise. Participation from the expert pool is good and the administrator keeps experts engaged by frequently asking for new input to new questions generated within the system. The TechCast.org Web site has some drawbacks. First of all, it is dependent on the administrators to select topics for expert commentary. The first two steps in TechCast’s process are performed by administrator, and the methodologies employed to screen and to analyze topics are not disclosed. The system’s output is also highly influenced by the composition of the expert pool, and it is unclear what the criteria are for inclusion. The TechCast. org Web site is currently English-only and does not support regional analysis. The system of communication is relatively basic, depending solely on wikis for interactions between experts. Finally, while the system analyzes background data, it does not publish the specific signals, signposts, or tipping points necessary for continued tracking of forecasts.

OCR for page 92
9 PERSISTENT FORECASTING OF DISRUPTIVE TECHNOLOGIES Status Quo Defined as 30 - 40 % Decrease 30 - 40 % Decrease Remaining Uncertainty about 30 % 10 0 % Uncer tainty In Uncer tainty In Uncertainty +/ – 3 years/ $100 B / 5 % confid. Expert Scanning Results Analysis Survey • Internet • Event/ • Forecasts/ • Expert • Media Issues Tracking / Knowledge Arrivals • Science • Data • Data • Interviews • Integration Points Analysis • Conferences • Trends • Estimates • Comments Iterations FIGURE 6-3 TechCast online research system. SOURCE: Halal, 2009. 6-3.eps FIGURE 6-4 Expert survey. SOURCE: Halal, 2009.

OCR for page 92
97 EVALUATING EXISTING PERSISTENT FORECASTING SYSTEMS FIGURE 6-5 Result of expert survey results. SOURCE: Halal, 2009. FIGURE 6-6 Longitudinal summary of forecasts. SOURCE: Halal, 2009.

OCR for page 92
98 PERSISTENT FORECASTING OF DISRUPTIVE TECHNOLOGIES Contact 2035 Deep Space Nuclear Fusion of f scale Life Span = 10 0 Hydrogen 2030 Maglev Economy Trains Men On Child Traits Mars Hypersonic Most Likely Year to Enter Mainstream Planes Grown Genetic 2025 Biocomputing (Usually defined as 30 % adoption level) Automate d Organs Therapy Moon Base Desalinization Thought Highways Smar t Power AI Robots Cancer Cure Distributed Small Aircraft Grid Quantum Micro Computing Ar tificial Organs Machines Alternative GMO Energy 2020 Telework Power Personalized Electric Storage Treatment Vir tual NanoTech Cars Eco- Organic Optical Recycling Realit y Bikes Farming Body Computers Monitoring Global Modular Homes Pervasive Vir tual Access Aquaculture Networks Fuel Cell Education Synthetic Life Designed Cars 2015 n Farming Precisio On -Line Materials Grid Intelligent Publishing TeleMedicine Computing Interface Space Mass Intelligent Climate Tourism E-Gov’t. Customization Cars Control Green Convergence Business Cloud Web 3.0 B2B Hybrid Cars Smar t Computing E-Tailing 2010 Sensors Broadband Web 2 .0 Biometrics Smar t Entertainment on -D emand Phone Online Wireless Finance 20 05 Information E- Commerce Manufacturing Medicine & Transpor tation Energy & Space Technology & Robotics Biogenetics Environment FIGURE 6-7 Summary of forecast results. SOURCE: Halal, 2009. 6-7.eps x2 (SIgNTIFIC) The X2 project, under development by IFTF, is a re-envisioning of the Victorian era X-Club, designed to foster collaboration among diverse groups by a combination of social networking (including Facebook and Digg), futur- ism, and forecasting. It is a novel system with a greater degree of software sophistication than the other systems described. IFTF recently gave X2 a new name—Signtific—to serve as a global collaborative research platform created to identify and facilitate discussions around future disruptions, opportunities, and trends in science and technology.1 The X2 system combines workshops, an online, wiki-based platform, and ARGs to produce a forecast. It has three main activities: signal development, forecast generation, and “big story” creation. Big story creation is the X2 creators’ version of alternative future generation. Figure 6-8 shows their methodology. The first component of X2 is expert workshops. IFTF held seven workshops around the world, with each workshop having at least 15 and sometimes more than 30 expert participants. All workshops were conducted in English. Workshops were cohosted by other organizations and employed new visualization technologies such as ZuiPrezi. A typical workshop agenda included the following: headline (theme), futures of science, geographies of science, signal entry, and breakout. Figure 6-9 is the output of an X2 workshop. 1Available at http://www.iftf.org/node/939. Last accessed May 6, 2009.

OCR for page 92
99 EVALUATING EXISTING PERSISTENT FORECASTING SYSTEMS Scenarios, maps, perspectives, reports Big e us Stories gamers , IFTF, outside authors, others etc. to sy ea Trends and emerging patterns n, Forecasts joi platform, workshops to ple sim Interesting or unusual things Signals platform, workshops FIGURE 6-8 X2 content and methodology. SOURCE: Castro and Crawford, 2008. Courtesy of the Institute for the Future. 6-8.eps FIGURE 6-9 X2 Workshop output. SOURCE: Castro and Crawford, 2008. Courtesy of the Institute for the Future.

OCR for page 92
00 PERSISTENT FORECASTING OF DISRUPTIVE TECHNOLOGIES The second component of X2 was an interactive online platform. The platform allowed experts to post signals of interest to specific subject groups. As of November 2008, 235 experts had been invited by IFTF to participate in the platform. The 28 subject groups collectively produced 697 signals, 262 forecasts, and 11 perspectives. Signals were inputted by experts (Figure 6-10), who suggested questions to be answered. Topics were tagged, and abstracts could be added by experts. Experts could comment and rate the likelihood and potential impact of any signal created. They would then use these signals to generate sample forecasts such as the following: “Growing infrastructures for ‘citizen science’ will help shape twenty-first century science.” —Hyungsub Choi, Pennsylvania “Soft technology will be a protagonist for innovations in the twenty-first century.” —Zhouying Jin, China “Bayesian networks utilized in creating more mobile social networks.” —Emma Terama, Austria From predictions captured by the X2 platform, IFTF then proceeded to generate “perspectives.” Sample perspectives included these: • The Indian Ocean as a new nexus for science, • Islam’s coming scientific revolution, • Designing the next revolution in science, • Will green chemistry be the central science of the twenty-first century? • The interdisciplinary future of energy research, • The transformation of science parks, and • Linking innovation to manufacturing. The third component of the X2 system is the use of “thought experiments” conducted in the form of games. IFTF employed ARGs to flush out an alternative future using crowd sourcing. Players accept the premise of the alternative future and then role-play. The game is guided by “game masters” who monitor the progress of the game and generate events within the scenario. The philosophy behind the creation of the ARG is to develop engaging scenarios that reflect perspectives generated by game players and that could serve as a foundation for discussion using Web-based social media tools. IFTF felt it was equally important to develop an appropriate community architecture focused on attracting participants and game masters and to clearly communicate the promise of the game to the players. Box 6-1 shows a sample scenario, and Figure 6-11 shows a sample game screen. As mentioned previously, the games are crowd sourced. The ARG is promoted on the Internet and is open for players from around the world to play. The games are used to gain a better understanding of the impact of various perspectives and future scenarios. Strengths and Weaknesses X2 is an interesting mix of forecasting methodologies. It combines traditional Delphi approaches with inno - vative methods such as ARGs and expert wikis. The system is managed by experienced professional forecasters, draws on experts to flush out signals and to forecast, and then calls on the crowd (through the use of gaming) to understand impact. The workshops, which are held around the world, are attended by a diverse set of experts. The X2 platform is a persistent system that allows experts to participate at their convenience; the workshops and games are organized events. It does an excellent job of allowing experts to create categories, input signals, and discuss potentially disruptive technologies. Finally, ARG is an innovative and effective way to flush out the potential impact of an alternative future. It does a good job of attracting players from around the world to participate in an engaging role-playing exercise.

OCR for page 92
0 EVALUATING EXISTING PERSISTENT FORECASTING SYSTEMS FIGURE 6-10 Expert signal framework of X2. SOURCE: Castro and Crawford, 2008. Courtesy of the Institute for the Future. BOX 6-1 Sample X2 Game Scenario: Wall Street Becomes the Mechanism for Setting the Scientific Agenda and Provides the Financing It’s the year 2020, and wealthy individual entrepreneurs are at the forefront of scientific and technologi- cal innovation. . . . Innovation moves from institutional to individual. • Who are you in 2020? How does this new model of innovation affect you? • If you are a future Xoogler, what research would you fund? • If you are not wealthy, does the growing influence of individuals and less government oversight concern you? What would you do? • How might this change the relationship between amateur and professional scientists? SOURCE: Castro and Crawford, 2008. Courtesy of the Institute for the Future.

OCR for page 92
0 PERSISTENT FORECASTING OF DISRUPTIVE TECHNOLOGIES FIGURE 6-11 Sample game screen of an ARG game used by IFTF. SOURCE: Castro and Crawford, 2008. Courtesy of the Institute for the Future. In spite of X2’s many innovations, the system had some shortcomings. Each component of the X2 process required participants to speak English, and none of the workshops were held in a language other than English. The committee was also concerned that not enough attention was paid to making sure that the participants in the workshops were diverse enough to represent the science and technology community of the region and that there was only limited participation from young scientists, researchers, technologists, and entrepreneurs. More thought could have also been given to the selection of locations for the workshops. The committee felt strongly that X2 workshops should seek out places with developing knowledge and techno clusters that had not already been sur- veyed to gain technology forecasts. There were also concerns that the X2 platform was not engaging enough to encourage continuous participa - tion by the experts and that over 50 percent of the signals were generated from U.S. experts. The participation in each forecast is too sparse to allow the wiki to collect and compute statistically meaningful likelihood and impact projections. It would also be useful to specifically ask the experts to predict a realization date for each forecast as a part of gathering data for the computational wiki. EvALuATION OF FORECASTINg PLATFORMS The committee’s evaluations of the three forecasting systems, summarized in Table 6-1, are seen as preliminary and subject to additional discussion in later reports. While none of the systems met all the requirements of an ideal persistent forecasting system laid out by the committee, all had elements that were valuable. The committee has already seen great potential in existing forecasting systems such as Delta Scan, TechCast, and X2 (Signtific). These projects demonstrate many of the concepts presented in this report and give the com - mittee confidence that an open, persistent forecasting platform for disruptive technologies can be created. Any of these existing forecasting systems could serve as the foundation for a more robust system. Significant insight can be gained even with limited or partial implementation.

OCR for page 92
0 EVALUATING EXISTING PERSISTENT FORECASTING SYSTEMS TABLE 6-1 Initial Evaluation of Existing Forecasting Systems Tech Cast Signtific (X2) Delta Scan Data sources Partially meets. Largely Partially meets. Open system. Partially meets. Open to access from selected panel experts. Multiple data sources (stewards, but not to contribute. Data Other sourcing methods and workshops, games) and baseline gathered from more than 250 geographic sourcing not data. Geographical diversity experts with a wide array of apparent. Open to invited improving (workshops, Web backgrounds through workshops members only. Novel data site) but still is English only. and interviews. English only. sources, collection techniques, Participant’s profiles captured. Novel data sources, collection and data audit methods not Some applications of novel data techniques, and data audit apparent. English only. sourcing and collection techniques. methods not apparent. Data audit methods not apparent. Forecast methods Partially meets. Self-selected Partially meets. Forecasts and Partially meets. Principally expert opinion with some signals derived from stewards, expert opinion. Forecast supplied predictive elements. Part experts, and public via workshops by the Institute for the Future. qualitative and quantitative. and games. Signals and forecasts No apparent way for the public No apparent barriers to largely qualitative. Quantitative to contribute or collaborate. recruiting experts. methods not apparent. Forecast team Partially meets. Self-selected Partially meets. Team consists of Partially meets. Forecasts experts and small team. employees, experts, and public. supplied by the Institute for the English only. No apparent Public participation strong and Future. Appears to be English barriers to recruiting experts. growing in some areas—7,000 only. No apparent public Methods to assess expert participants in the last AGR. More participation. Methods to assess diversity, country or quals limited public participation in diversity of experts by country, not published. No public signal generation and forecasting. culture, discipline, etc. not participation. Evidence of third-party community apparent. development, collaboration, and initiative becoming apparent. English only. Data output Partially meets. Quantitative Partially meets. Principally Largely meets. Qualitative with and qualitative. Each forecast qualitative. Quantitative some quantitative. 1-5 scale quantifies estimated time representation is limited or not assessment of impact, likelihood, of realization, confidence apparent. Third-party access (and and controversy. Qualitative levels, market size, and range export capabilities) to the data on assessment of geographical of dispersion. Qualitative player behavior is not apparent. impact. Signals, enablers, assessments of forecast inhibitors, Centers of excellence, strengths and weaknesses. data sources, analogies, word Visualization limited to static tags and links identified. graphs. Unclear if data are Visualization and navigation exportable. could be strengthened. Processing tools Limited. Some enablers and Limited. Some enablers and Limited. Signals, enablers, inhibitors identified in forecast inhibitors identified by the inhibitors identified but no narrative. Processing done community. Diversity of the apparent way to automatically by the expert community. community processing the data measure progress toward Diversity of experts appears to be improving. Other thresholds. Diversity of unclear. Other processing processing tools (dashboards, community processing data tools (dashboards, data data visualization, signal and link unclear. Other processing tools visualization, signal and link processing) are not apparent. (dashboards, data visualization, processing) are not apparent. signal and link processing) are not apparent. continues

OCR for page 92
0 PERSISTENT FORECASTING OF DISRUPTIVE TECHNOLOGIES TABLE 6-1 Continued Tech Cast Signtific (X2) Delta Scan System attributes Partially meets. System is Partially meets. System is persistent Limited. Open to access but persistent but not open. Bias and open. Bias mitigation processes not to participate. System not mitigation processes not not apparent. Degree of scalability persistent (last updated in 2006). apparent. Degree of scalability unclear. English only. Additional Bias mitigation processes not unclear. English only. communication tools could improve apparent. Degree of scalability Intuitive system with limited usability. unclear. English only. Additional communication tools. communication tools could improve usability. Summary Partially meets. TechCast Partially meets. X2 partially meets Partially meets. Delta Scan partially meets the attributes the attributes of the persistent partially meets the attributes of the persistent forecasting forecasting system, with strength of the ideal forecasting system system with strengths in ease in openness, qualitative data with particular strength in the of use and in quantifying the capture, multiple data sources, robustness of the data forecast probability of occurrence, and multiple forecasting methods. output. The system could be impact, and forecast timing. Could improve by adding multiple strengthened by being persistent Could improve by broadening language support, strengthening and including additional data methods of data sourcing, quantitative forecasting methods, sources, forecast methods, public utilizing more forecasting processing tools, and visualization participation, native language techniques, incorporating techniques. The system lacks support, and better processing multiple languages, consistent participation from users tools. diversifying the forecast team and forecasters. Forecast suffers (including the public), and from inadequate preparation of the strengthening data output and initial problem set and definition processing tools. The forecast of the scope of the forecasting produced is a single view of mission. the future synthesized by the system operator. REFERENCES Castro, Cesar, and Mathias Crawford. 2008. X2: Threats, opportunities, and advances in science & technology, Presentation to the committee on November 6 by the Research Director for the Institute for the Future. Halal, William E. 2009. Forecasting the technology revolution, Presentation to the committee on August 3 by the President of TechCast, LLC. Woodroof, Harry. 2007. Delta Scan, Uses in U.K. government, Presentation to the committee on October 15 by the Leader, Delta (S&T) Scan, Horizon Scanning Centre, Government Office for Science, United Kingdom.