Summary

The National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation (NSF) communicates its science and engineering information to data users in a very fluid environment that is undergoing modernization at a pace at which data producer dissemination practices, protocols, and technologies, on one hand, and user demands and capabilities, on the other, are changing faster than the agency has been able to accommodate.

NCSES asked the Committee on National Statistics and the Computer Science and Telecommunications Board of the National Research Council to form a panel to review the NCSES communication and dissemination program that is concerned with the collection and distribution of information on science and engineering and to recommend future directions for the program according to its statement of task (see Box S-1).

The Panel on Communicating National Science Foundation Science and Engineering Information to Data Users reviewed NCSES’s existing approaches to communicating and disseminating statistical information, including the division’s information products, website, and database systems; examined existing NCSES data on websites, information gathered by and from NCSES staff, volunteered comments of users, and input solicited by the panel from key user groups; assessed the varied needs of different types of users in the NCSES user community; considered the impact that current federal and NSF website guidance and policies have on the design and management of the NCSES online (Internet) communication and dissemination program; considered current research and practice in collecting, storing, and utilizing metadata, with particular focus on specifications for



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary T he National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation (NSF) communicates its science and engineering information to data users in a very fluid environ- ment that is undergoing modernization at a pace at which data producer dissemination practices, protocols, and technologies, on one hand, and user demands and capabilities, on the other, are changing faster than the agency has been able to accommodate. NCSES asked the Committee on National Statistics and the Computer Science and Telecommunications Board of the National Research Council to form a panel to review the NCSES communication and dissemination program that is concerned with the collection and distribution of informa- tion on science and engineering and to recommend future directions for the program according to its statement of task (see Box S-1). The Panel on Communicating National Science Foundation Science and Engineering Information to Data Users reviewed NCSES’s existing approaches to communicating and disseminating statistical information, including the division’s information products, website, and database sys- tems; examined existing NCSES data on websites, information gathered by and from NCSES staff, volunteered comments of users, and input solicited by the panel from key user groups; assessed the varied needs of different types of users in the NCSES user community; considered the impact that current federal and NSF website guidance and policies have on the design and management of the NCSES online (Internet) communication and dis- semination program; considered current research and practice in collecting, storing, and utilizing metadata, with particular focus on specifications for 1

OCR for page 1
2 COMMUNICATING SCIENCE AND ENGINEERING DATA BOX S-1 Statement of Task An ad hoc panel will review the communication and dissemination program of the National Science Foundation (NSF) National Center for Science and Engineering Statistics (NCSES) that is concerned with the collection and distribution of information on science and engineering and recommend future directions for the program. Specifically, the panel will 1. Review NCSES’s existing approaches to communicating and dis- seminating statistical information, including the division’s informa- tion products, website, and database systems. [This review will be conducted in the context of both current “best practices” and new and emerging techniques and approaches.] 2. Examine existing NCSES data on websites, information gathered by and from NCSES staff, volunteered comments of users, and input solicited by the panel from key user groups, and assess the varied needs of different types of users within NCSES’s user community. 3. Consider the impact current federal and NSF website guidance and policies have on the design and management of the NCSES’s online (Internet) communication and dissemination program. 4. Consider current research and practice in collecting, storing, and utilizing metadata, with particular focus on specifications for social science metadata developed under the Data Documentation Initia- tive (DDI). 5. Consider the impact of government-wide activities and initiatives (such as FedStats, Data.gov) and the emerging user capability for online retrieval of government statistics. The panel will facilitate its review by conducting a 2-day public work- shop that will feature invited presentations and discussions. The panel will subsequently prepare an interim letter report that will focus on is- sues regarding transition from current approaches and a final report with specific recommendations, including a discussion of related technical, staffing, and funding issues. social science metadata; and considered the impact of government-wide activities and initiatives (such as FedStats, Data.gov) and the emerging user capability for online retrieval of government statistics. In accomplishing this review, the panel conducted two workshops to gather information from data users and experts on various aspects of data

OCR for page 1
3 SUMMARY storage, retrieval, dissemination, and archiving. An interim report issued early in 2011 addressed data content and presentation, meeting changing storage and retrieval standards, understanding data users and their emerg- ing needs, and data accessibility. The analysis and recommendations from the interim report are carried into this final report, along with the results of subsequent analysis by the panel. These are exciting and challenging times for federal government statisti- cal agencies responsible for disseminating their data products to their user communities, and the times are especially challenging for NCSES, which is finding the importance of its data magnified many fold by the growing recognition of the role that science and engineering investment is playing as a source of economic growth. The vision of a data dissemination program for NCSES is also in a time of flux. The agency is confronting new roles and missions, as directed in the America COMPETES Reauthorization Act of 2010, which changed more than its name. Technology is also opening the door to significant leaps in the ability of NCSES to communicate data and analytical products to data users. The promise of such services as Data. gov and the emergence of such private-sector solutions as the Google Public Data Explorer are just becoming recognized. The semantic web (Web 3.0) holds promise of communicating data to users in entirely new ways, much to the advantage of users and the federal agencies themselves (Berners-Lee and Hendler, 2001). These technological advances open the way to new opportunities, but they are also problematic in that they are rapidly pro- mulgated and, many times, they rapidly become obsolete. The panel sug- gests that NCSES adopt an approach to modernization that stresses the basics of data provision (common formats with appropriate metadata) and partnerships with the private sector as opportunities become available, so that NCSES will avoid the issue of rapid obsolescence associated with rapid change in the particular tools and systems offered by the private sector. In the face of these environmental and technological forces, we make a number of recommendations to the National Center for Science and Engineering Statistics to improve its dissemination program. The first set of recommendations has to do with how the survey-based data are received and input into the NCSES database, managed once there, and preserved for posterity. (The recommendations are numbered as they appear in the body of this report.) Recommendation 3-1. The National Center for Science and Engineer- ing Statistics should incorporate provisions in contracts with data providers for the receipt of versioned microdata, at the level of detail originally collected, in open machine-actionable formats.

OCR for page 1
4 COMMUNICATING SCIENCE AND ENGINEERING DATA Recommendation 3-2. The National Center for Science and Engineer- ing Statistics should transition to a dissemination framework that emphasizes database management rather than data presentation and strive to use auditable machine-actionable means, such as version con- trol, to ensure integrity of the data and make the provenance of the data used in publications verifiable and transparent. Recommendation 3-3. The National Center for Science and Engineer- ing Statistics (NCSES) should require that data received from contrac- tors be accompanied by machine-actionable metadata so as to allow for automated production of NCSES publications, comparability with previous analysis, and efficient access for third-party visualization, integration, and analysis tools. Recommendation 3-4. The National Center for Science and Engineer- ing Statistics should proceed to make its data available through open interfaces and in open formats compatible with efficient access for third-party visualization, integration, and analysis tools. Recommendation 3-5. The National Center for Science and Engineer- ing Statistics should develop a plan for redesign of its retrieval tools utilizing the emerging, sustainable capabilities of other government and private-sector resources. Recommendation 3-6. The National Center for Science and Engineer- ing Statistics (NCSES) should work with the National Archives and Records Administration (NARA) to ensure long-term access and preser- vation of all of its publications and all data necessary to replicate these publications. As a necessary step, NCSES should review and update the request for disposition authority that is filed with NARA to ensure prompt and complete disposition of records and should regularly review the status of compliance with the records retention directive. Engaging with its data users is an essential activity for NCSES. There is much that can be done to make that engagement more productive. Recommendation 4-1. The National Center for Science and Engineering Statistics (NCSES) should analyze the results of its initial online con- sumer survey and refine it over time. Using input from other sources, such as regular structured user focus groups and panel-based periodic user surveys, NCSES should regularly and systematically collect and analyze patterns of data use by web users in order to develop a typol- ogy of data users and to identify usability issues.

OCR for page 1
5 SUMMARY Recommendation 4-2. The National Center for Science and Engineer- ing Statistics should educate users about the data and learn about the needs of users in a structured way by reinstating the program of user workshops and instituting user webinars. Recommendation 4-3. The National Center for Science and Engineer- ing Statistics should employ user-focused design and user analysis, starting with an initial heuristic evaluation and continuing as a regular and systematic part of its website and tool development. Recommendation 4-4. The National Science Foundation should spon- sor research and development on accessible data visualization tools and approaches and potential other means for browsing and explor- ing tabular data that can be offered via web, mobile, and tablet-based applications, or browser-based ones. The implementation of this report’s recommendations should be undertaken within an overall framework that accords priority to the basic quality of the data and the fundamentals of dissemination, then to signifi- cant enhancements that are achievable in the short term, while laying the groundwork for other long-term improvements. The framework could be organized along the following lines (highest priority first): 1. Focus on collecting the right data (by contractor or otherwise); using appropriate change management and version control to estab- lish data provenance, flag data errors and correct them; annotating those data with sufficient machine-actionable metadata to establish a process for interpreting the data, enabling efficient access to third-party data and to automated NCSES publications; and pub- lishing the data in formats with web-accessible open interfaces for all to use. 2. Publish methods for combining old data and new data that have been collected under different assumptions or categories or that are disseminated in ways that make them difficult to reintegrate—this is especially necessary for the data from the old and new industry research and development expenditure surveys that will populate the Industrial Research and Development Information System. 3. Provide the essential data reductions and visualizations that NSF’s mission requires, for example, when Congress asks for authorita- tive data on a certain topic, a trusted group must be able to use the data and derived publications to calculate answers. 4. Provide a growing array of visualizations and printed products tailored for the many different uses and users.

OCR for page 1
6 COMMUNICATING SCIENCE AND ENGINEERING DATA Not every recommendation made in this report can or should be imple- mented immediately. Some recommendations must build on the implemen- tation of others; for example, development of a database structure that can support accessibility through the semantic web requires that NCSES obtain data from its contractors in different formats than are now received and that it define metadata to accompany the data elements. We therefore suggest a time-phased approach to improving data dissemination, focusing on five major initiatives: 1. Change the means and content of the data received from contrac- tors and actively participate in the development and implementa- tion of the Data.gov compatible metadata standard now being explored by W3C and the SCOPE project. 2. Gain a better understanding of the needs of users of the data— those primary, secondary, and tertiary blocks of users—and then use the information to engage them in an effort to educate them and otherwise meet their needs. 3. Conduct a continuous usability evaluation program, much akin to a program of continuous improvement that is part and parcel of any total quality management program. 4. Provide data in retrievable formats and encourage private-sector providers and individual users to import the data into their visual- ization tools. 5. Ensure full short- and long-term access to the data by updating its retrieval tools and ensuring proper archiving of its publications and database.