14
Review of Surveillance, Training, and Intervention Effectiveness Research
Key Findings and Recommendations for Surveillance, Training, and Intervention Effectiveness Research
|
STRATEGIC GOALS AND OBJECTIVES
Surveillance is an essential component for the identification in health and safety areas that require attention and for the evaluation of program effectiveness. The strategic goal dealing with surveillance, training, and intervention effectiveness in the Mining Program is to “determine the impact of changing mining conditions, new and emerging technologies, and the changing patterns of work on worker health and safety.” Fortunately, the Mine Safety and Health Administration (MSHA) provides the best available industry-wide database on injuries, illnesses, and exposures—a tremendous asset for the National Institute for Occupational Safety and Health (NIOSH) Mining Program.
Table 14-1 lists the intermediate goals falling within this strategic goal. Intermediate goal 1 is unlikely to achieve the implied goal, because the goal stated is actually a task. The implied goal is to acquire demographic information about miners in order to make better use of the MSHA data set on fatal and non-fatal injuries. One
TABLE 14-1 Intermediate Goals and Performance Measures of Surveillance, Training, and Intervention Effectiveness Research and Committee Comments
Intermediate Goala |
Performance Measurea |
Committee Comments |
|
1. |
Conduct a demographics survey of mine workers |
This goal will be achieved by completing and publishing a comprehensive demographic survey of the mining industry by 2009 |
Such a survey will be invaluable, not only for denominator data, but for guiding research and training activities that are necessary to prevent a sharp increase in occupational health and safety problems in the evolving workforce. The intermediate goal is stated as a task. The implied goal is to acquire demographic information about miners in order to make better use of MSHA data. As stated, this intermediate goal can be achieved by conducting a survey, but the information may not be adequate |
2. |
Document organization of work changes in the domestic and global mining industry and assess their impact on worker health and safety |
This goal will be achieved if the impacts of organization of work changes have been identified and recommendations for mitigating adverse impacts are issued over the next 5 years |
The proposed surveillance research will permit evaluation of the effects of new regulations and work practices. The most effective of these can then be considered for adoption in the United States and other countries |
3. |
Examine emerging technologies for potential health and safety benefits and risks |
NIOSH sponsored two recent major studies of emerging technologies by RAND Corp. in 2001 (Peterson et al., 2001) and by the National Research Council in 2002. The identified emerging technologies will be monitored, and efforts will continue to track new developments and their impact on mining safety and health |
This is a low-level effort given the relatively slow infusion of new technologies in mining and will not require a separate project effort in the foreseeable future |
could conduct the survey, and thus achieve the goal, but still acquire inadequate information. Intermediate goal 4 should be used to support the remaining goals by providing a consistent means to determine intervention effectiveness. While intermediate goals 2, 3, 5, and 6 are important, an ideal mining program would take the approach of identifying hazardous work processes using a systematic approach, and then evaluating the effects of these specific exposures and changes in mining conditions, technologies, and work patterns on injuries or illnesses (see Chapter 3). Exposure monitoring, including but not limited to industrial hygiene sampling and heat stress evaluation, should be used as part of the surveillance system to identify potential health risks.At present, there are no intermediate goals related to training, which should be a component of all strategic goals. The emphasis of the Mining Program research in this area should be on evaluation of training program effectiveness and the identification of the most effective methods of training workers prior to hazard exposure. The intermediate goals would also benefit from the addition of specific criteria by which completion could be judged.
The description of intermediate goals related to the surveillance system is also inadequate. Development of key research questions is required and should include means of identifying hazardous work processes and exposures. In determining future research, the Mining Program should continue to work with industry, organized labor, MSHA, academe, and international partners. Both internal and external peer review as described by the Mining Program in its procedures documentation will be useful for selecting projects. While not a focus of research, the Mining Program should partner with universities to develop training materials for mining engineering students and occupational or environmental health students.
Surveillance, training, and intervention effectiveness are cross-cutting issues essential to achieving other strategic goals. Projects listed by the Mining Program under “other safety research” are intramural applied studies concerning the communication of mine safety information and education and the collection of information about mine safety hazards. The former dissemination activities are logically related to the latter surveillance activities, since what is reported back and prioritized as hazardous might then presumably be the subject for subsequent attention through publications, web sites, and training. Organizational and individual behavioral responses might then result, which could contribute to attainment of the stated strategic goals and objectives.
REVIEW OF INPUTS
Surveillance, training, and intervention effectiveness research is a relatively new area in the Mining Program; therefore allocation of total funds to the component projects has not been substantial until the recent past. The total budget for this
research area in 2005 was $3,646,400, having increased from $1,581,200 in 1998. Combining the Pittsburgh Research Laboratory (PRL) and Spokane Research Laboratory (SRL) employees, there were 19 individuals (and two vacant positions, at the time of this review) working on surveillance and statistical support and 7 individuals working on extramural coordination and information dissemination, although 30 full-time employees were listed as working in this area in materials supplied to the committee. Resources for the projects listed are, in all cases, modest. This is particularly true for the Internet dissemination and health communication work.
REVIEW OF ACTIVITIES
The need for research in this area was clearly indicated during stakeholder presentations to the committee. Mining remains a dangerous occupation, requiring continued surveillance. Table 14-2 lists surveillance, training, and intervention effectiveness research projects conducted by the Mining Program.
The Mining Program did not describe to the committee the quality assurance process followed by individual researchers. The research projects, as they have been presented to the committee, are consistent with the overall strategic goal but do not address all the intermediate goals. Intermediate goal 3 (examination of emerging technologies for potential health and safety benefit or risks) apparently does not have projects associated with it. Two of the projects presented do not fall easily within any of the intermediate goals. Since the surveillance research has not been carried out to determine which exposures or work processes in the mining industry have the greatest potential to lead to adverse health effects (other than dust and noise exposure), it is not possible to ascertain that the chosen projects are the most relevant. As the surveillance program develops further, this information will become more accessible. The intermediate goals, however, are certainly relevant and important.
The committee considered other activities that could be included in this area of research, given appropriate resources:
-
Disease in retired miners should be investigated, since the effects of many exposures may not be seen until after miners have left the workforce.
-
Substance abuse in the mining workplace is an important subject identified by MSHA as an area requiring focus.
-
There should be continued study of the need for training programs in languages other than English.
-
As the Education and Training for an Evolving Mining Work Force project progresses, further research should be conducted into worker-centered peer train-
TABLE 14-2 Surveillance, Training, and Intervention Effectiveness Research Projects and Committee Assessment of Relevance and Impact
Project Titlea |
Intermediate Goal |
Purposea |
Relevance |
Impact |
National Survey of Mining Population |
1 |
Conduct a survey of mines and their employees for each of the five major mining sectors (coal, metal, nonmetal, stone, and sand and gravel) |
Highly relevant—would provide denominator data for the numerator data in the MSHA injury database, enabling calculation of occupation, age, and experience-specific injury rates by cause and other risk factors |
The project is unlikely to acquire sufficient information with the current methods given that this is a one-time survey that will become obsolete rapidly. It would be better to create a sustainable annual data collection process |
Surveillance of Mine Safety Hazards |
1 |
Provide surveillance services, data management, and outcome evaluation guidance to SRL researchers to help ensure that research decisions and directions are evidence based and in agreement with NIOSH goals and stakeholder priorities |
Relevant—but should be combined into a larger surveillance program with the focus of improving the usefulness of the MSHA database in a sustainable fashion and identifying hazardous exposures and injury, illness, and disease clusters |
Should support the intervention effectiveness evaluations of other research programs and demonstrate responsiveness to stakeholder questions |
Education and Training for an Evolving Mining Workforce |
2 |
Conduct training needs assessment studies to identify and improve good educational measures for miner training |
Highly relevant given the current and ongoing turnover in the mining workforce. Correctly identifies the training needs of a demographically changing mining workforce as a critical problem area requiring intensive research. Involved clearly defined stakeholder input. Could benefit by staff review of new research on the issues of worker empowerment, worker control or lack of control in education and training, and technological changes and new forms of work organization that are significantly changing working conditions |
Too preliminary for the reporting of research outputs. A significant number of training needs assessment studies have been conducted and are under way. As of February 2005, a number of informational materials had been produced, seminars have been held for safety and health professionals and on-the-job trainers, and an on-the-job training program has been created to improve transfer of information from experienced miners to new employees |
Disseminating Safety and Health Interventions via the Internet |
4 |
Develop methods for indexing content on the Mining Program web site to improve customer access to that information |
Highly relevant—the Internet is an efficient channel for the dissemination of information; information design principles can improve the appearance, readability, search efficiency, and information-seeking satisfaction of web site users. Strongly supported by industry representatives on the review committee |
Too preliminary to report research outputs |
Project Titlea |
Intermediate Goal |
Purposea |
Relevance |
Impact |
Evaluation of Heat Stress and Interventions in Surface and Underground Mines |
5 |
Determine if a relationship exists between overexposure to heat during mining and related activities and increased risk of injury |
Highly relevant—addresses the strategic goal of enhancement and safety of emergency response |
Extremely productive within a limited time. Has already resulted in adoption by a significant number of rescue teams in the United States and Canada |
Hazard Evaluation and Technical Assistance |
5 |
Provide flexibility in technical assistance within a framework that allows researchers to identify latent or emerging hazards |
Relevent—helps to answer specific exposure, exposure control, and illness or disease questions posed by specific companies and helps provide NIOSH access to mine sites that might not otherwise be available. Would benefit from a system for choosing which problems to undertake. Should not provide a routine service more appropriately provided by private industrial hygiene consultants |
Provides many benefits to the Mining Program and the industry. A mechanism should be identified to share findings with other companies |
-
ing (e.g., Labor Institute and Paper, Allied-Industrial, Chemical and Energy Workers International Union, 2005) using training models focused on the elimination and reduction of hazards in a systems of safety approach (e.g., the Small Group Activity Method, Merrill, 1995). Research should include the use of worker peer trainers and train-the-trainers models; worker participation in curriculum development, including lessons learned from systems failures; worker conduct of program evaluations (e.g., Participatory Action Research, Green et al., 1995; McQuiston, 2000); and worker participation in health and safety program administration. The workplace consists of a network of safety systems, and nearly all incidents result from failures of those systems, thus health and safety interventions have to focus on improving those systems (Roland and Moriarty, 1983; Meshkati, 1995; Perrow, 1999). It is further recommended that NIOSH project staff take up these issues with workers (and their representatives, when the worksite is unionized).
-
The work organization program should focus on other end points beyond stress, including cardiovascular disease and possibly depression.
External partners have participated in some projects, including the Twentymile Coal Company, Pennsylvania Services Corporation, J. H. Fletcher and Company, Morton Salt, Rurher’s Quarry, the State of Pennsylvania Department of Environmental Protection, and MSHA. In some projects, the role of stakeholder input is less clear.
REVIEW OF RESEARCH OUTPUTS
Since 1995, there has been significant output by those working on surveillance, training, and intervention effectiveness research within the Mining Program. The program has reported developing more than 80 publications, training modules videos, workshops, and software products with help from universities, mining companies, MSHA, and other providers of mining training. Outputs focus on high-priority areas and, where appropriate, target vulnerable groups such as new miners. Outputs reflect the productivity of many current activities, although a number of these outputs are associated with activities included under other strategic goals of the Mining Program. Many activities have not yet generated significant output, but future outputs are expected to increase in quality and relevance. Given data provided by the Mining Program, it was not possible to determine the full extent of cross-agency, cross-institute, or internal-external collaboration. Very few of the publications on surveillance and intervention research were in peer-reviewed journals, but publication in formats such as technical journals may be more broadly distributed.
A few specific project outputs should be highlighted:
-
The Health Communications Program takes advantage of the use of humor and storytelling techniques in videos. This is novel and may increase training effectiveness markedly. Gathering information on training effectiveness should certainly remain a research goal. While this program does not identify itself as serving small business, the toolbox training modules have been requested by MSHA for inclusion in its Small Mines manuals.
-
The Education and Training for an Evolving Workforce project had produced, as of February 2005, 15 documents and 33 presentations addressing the project’s four tasks. More interesting and effective information material may lead to greater attention from miners than would otherwise be the case. An on-the-job training program was created to improve sharing information from experienced miners to new employees. An across-the-board preference for active learning (hands-on practice, simulation, etc.) was found at mine sites. Two seminars were developed to provide professional development opportunities for safety and health professionals and on-the-job trainers. Both translated adult education theory into practical strategies that can be used to train miners of any age. A computer-based training intervention to train new miners in map reading skills is under development.
While the overall research output is appropriate given the newness of the projects and the modest level of funding, these conditions also make the determination of impacts quite preliminary. Ideally, an improved overall surveillance program would be used to monitor changes in injury or illness rates in the particular area focused on by each output, although due to budget limitations this may only be possible for selected outputs. At the very least, surveys could be carried out with industry partners to determine how many were aware of specific outputs and if they had implemented any of the recommended changes.
REVIEW OF TRANSFER ACTIVITIES
Members of the committee agree that the Mining Program has stimulated major changes in the way mine safety and health training has been practiced since 1977, including greater emphasis on collaborative and active problem-solving learning, greater realism in training scenarios, greater fidelity of visual illustrations, and greater use of authenticated and field-tested training materials. The Mining Program is also engaged in finding better training processes and methods, including computer simulations, virtual reality models, interactive problem-solving stories, degraded stereoscopic (three-dimensional) images of hazardous conditions, and videotaped interviews with miners. Understanding which of these various modalities are most effective for communicating health and safety information
for all mining-related workers and supervisors should continue as an objective of the Mining Program.
Transfer activities occurring within this strategic goal are valued by external program stakeholders and are being used in at least a modest number of workplaces, though how many is not documented. New knowledge has been documented, indicating positive outcomes resulting from them. Although these transfer activities can claim some success, there is no description of how they fit together into a larger education-technology transfer program. Ideally, shared expertise across these programs could lead to greater effectiveness.
REVIEW OF INTERMEDIATE OUTCOMES AND CAUSAL IMPACT
Research and transfer activities need to target specific intermediate outcomes for the overall surveillance program. This should include improving MSHA surveillance through collection of denominator data, development of specific surveillance goals in concert with strategic partnerships, collection of the data necessary to determine the effectiveness of each intervention, and collaboration with international partners to better harmonize data collection. The current intermediate goals all focus on surveillance, yet the five potential intermediate outcomes pertinent to this strategic goal, as described by the Mining Program to the committee, relate predominantly to training:
-
Improving miner safety by developing toolbox training sets,
-
Improving miner safety by developing interactive computer simulation training,
-
Improving the accuracy of determining worker exposure to airborne silver,
-
New miner training, and
-
Improved technology transfer via the web.
The Mining Program is recognized as a center of excellence for training activities, but has not yet made extensive use of its surveillance program to produce intermediate outcomes. Given the recent organization of some of these projects not enough time has passed for them to have impact on the workplace.
Other intermediate outcomes include the following:
-
U.S. and Canadian mine rescue teams have adopted temperature pre-screening and heart rate tools to avoid adverse effects from heat stress.
-
The toolbox training demonstrated a significant increase in safety knowledge. As reported by NIOSH, participatory storytelling was found to be more
-
effective for younger miners, while a more traditional lecture-based format was more effective with older miners (presentation by Dr. Elaine Cullen, NIOSH, to committee, January 13, 2005). Although no statistical tests were reported, two of the most recent training videos Aggregate Training for the Safety Impaired and The Sky Is Falling reported an increase in safety knowledge of trainees. In future evaluations by NIOSH, the inclusion of statistical analysis of these changes will be needed.
-
In a survey of 52 responding organizations that purchased the Interactive Problem Solving Stories: A New Approach to Preventing Miners’ Occupational Injuries and Illnesses exercises from the National Mine Health and Safety Academy, 60 percent of the respondents rated the exercises as more useful than traditional instructional materials, 40 percent as equally useful, and 0 percent as less useful. Most trainers (79 percent) thought that the exercises helped them to make better use of workers’ knowledge and experience during training (NIOSH, 2005a).
REVIEW OF END OUTCOMES
Several surveillance, training, and intervention effectiveness research activities in the Mining Program are too new to have measurable end outcomes at this time. Development of an improved surveillance system should assist greatly in measuring end outcomes for all activities of the Mining Program. The committee has heard directly from numerous stakeholders that they have positively utilized several of the tools developed by the program for specific training purposes. One operator, for example, described work done with researchers at PRL and MSHA analyzing construction, maintenance, and repair (CMR) accidents from the MSHA database. After developing training tools and teaching safety and health personnel how to use them, CMR accidents went from 66 percent to 55 percent of all incidents (presentation by Kelly Bailey, Vulcan Materials Company, to committee, February 21, 2005).
REVIEW OF OTHER OUTCOMES
Other than a focus on older workers, new recruits, and some attention to safety issues associated with Spanish-speaking workers (NRC, 2003), none of the projects have a specific focus on vulnerable populations, and vulnerable populations have not been clearly defined in the mining industry. More studies on non-English-speaking workers in the U.S. mining industry could help define whether this population is truly more vulnerable. Most of the Mining Program products will be useful for small business, but each project should explicitly describe how these worker populations will be served. There is also a great opportunity to improve
mining health and safety internationally by providing training materials to other countries.
The extent to which NIOSH should work to directly assist developing countries in evaluating exposures should be determined as part of the strategic planning process, and a program could then be developed to prioritize requests if this area is funded. Examples of past efforts in this area include the 2001 assistance to Mexico’s Instituto Nacional de Salud Pública to assess mercury exposure at reprocessing plants, the 1999 assistance to Ecuador to assess occupational exposure of gold miners, and a similar project in Venezuela in 1998.
ASSESSMENT OF RELEVANCE AND IMPACT
The Mining Program is engaged in transfer activities leading to the adoption of recommendations and technologies by stakeholders. Surveillance and determination of intervention effectiveness are essential highest-priority components of a mining research program. Although the Mining Program is also engaged in transfer activities within this specific strategic goal, additional work is needed to create a better surveillance system. The activities in this specific goal have made a moderate contribution to impact on workers on the basis of intermediate outcomes and transfer activities. The relevance and impact of all projects depends on the types of projects that industry is willing to partner, execute successfully, and use.
The committee feels that the present research program activities and outputs are likely to produce improvements in worker health and safety. Research being done under this strategic goal is in high-priority subject areas and is likely to result in improvements in workplace protection.
PROGRESS IN TARGETING NEW RESEARCH
Surveillance, training, and intervention effectiveness research is a new and not well-developed research area within the Mining Program. The committee does not expect it to have a well-developed process for targeting new research at this time, but acknowledges it should develop a plan for the future. New projects should be chosen based on the results of a NIOSH surveillance program, requests from industry, evaluation of international developments such as new mining techniques and regulations, and addressing stakeholder needs (including MSHA), which this program is currently doing. The Mining Program should consider the potential for interaction with other NIOSH research programs and take advantage of surveillance conducted by these programs relevant to the mining industry. Research choices should be informed by surveillance data that indicate increased exposure and risk of injury and illness, as well as the results of well-designed epidemiologic
studies. Research to improve the data collection process and to make it more useful for the identification of the root causes of problems could ultimately result in the reduction of injury and illness in the mining industry. The identification of disease and injury incidence related to workplace exposures would help to indicate where application of resources could achieve the best results. Research into the effectiveness of Mining Program training and educational programs is also necessary.
Signature Accomplishments: Communications Working in partnership and with funds or in-kind services provided by industry, MSHA, the International Society of Mine Safety Professionals (ISMSP), mining associations, and state agencies, NIOSH developed a list of critical needs topics and created 10 mine safety and health training videos from 1999 through 2004. Approximately 12,000 videos have been shipped to mine safety and health trainers in more than 36 countries. Thirteen articles and papers have been published by SRL on this work since 2000; 35 presentations have been given at national and international conferences; and 2 reports have been prepared by outside researchers under contract to evaluate the effectiveness of the videos as training tools. Industries as diverse as insurance, tunnel building and construction, the military, and even university occupational safety and health programs have started to use the training videos. NIOSH videos have been recognized for their outstanding contribution to safety and health training and have earned several national awards. These include the NIOSH top honors for Educational Materials (Alice Hamilton Awards) in 2000 and 2001, the Centers for Disease Control and Prevention Communicators’ Roundtable Award for Electronic Media in 2002, a Guiding Light award from ISMSP in 2002, the ISMSP Highest Degree of Safety Award in 2000, and a Telly award in 2003. |