|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. 3
|
||||||||||||||||
 |
 |
Page 46 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 46
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
3
Overall Program Assessment
INTRODUCTION
Agriculture, forestry, and fishing are three distinct sectors with disparate workforce populations, relevant stakeholders, and occupational concerns. The committee initially attempted to conduct a separate evaluation of the National Institute for Occupational Safety and Health (NIOSH) research program in each of those sectors. After considering information provided by the NIOSH Agriculture, Forestry, and Fishing Research Program (the AFF Program) and discovering that agricultural safety and health represented nearly 99% of the program’s resources, the committee decided instead to evaluate the AFF research program as a whole. The committee focused its evaluation on research in agricultural safety and health but also assessed safety and health efforts in forestry and fishing to the extent that information was provided by NIOSH.
The committee provides an overall evaluation of the AFF Program’s strategic goals, including “external” factors that, although beyond the program’s control, affect its activities and performance. It then identifies other important activities that the AFF Program ought to pursue and is pursuing: identifying and engaging stakeholders, conducting health services research and training, carrying out program evaluation initiatives, and providing input to public policy. Finally, the committee highlights an element that in many ways has the characteristics of the ideal AFF Program described in Chapter 2.
OCR for page 47
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
AGRICULTURE, FORESTRY, AND FISHING RESEARCH PROGRAM GOALS
The AFF Program identified five major goals consistent with its congressional mandate: surveillance, special populations at risk, health effects of exposure to agricultural agents, control systems, and outreach (see Box 3-1). The committee evaluated each major goal and compared programmatic components of the existing program with those of the ideal AFF Program described in Chapter 2. The committee chose to rename the major goals so that their names would better encompass and depict the subjects that the AFF Program had identified: surveillance (Chapter 7), populations at risk (Chapter 8), health effects research (Chapter 9), intervention research (Chapter 10), and knowledge diffusion and technology transfer (Chapter 11).
Following the guidance of the Framework Document, the committee carried out its evaluation by using the terminology and organization of a logic model
BOX 3-1
Major Goals of the AFF Program
Goal 1: Surveillance—Reduce injuries and illnesses in the agriculture, forestry, and fishing workforce by understanding the characteristics of those injuries and illnesses so as to target research and interventions that reduce hazardous exposures.
Goal 2: Priority Populations at Risk—Reduce injuries and illnesses of special populations of workers in these sectors by determining their significant risk factors and identifying and recommending interventions.
Goal 3: Health Effects of Agricultural Agent Exposures—Reduce injuries and illnesses by understanding the long-term, chronic effects of exposures from agriculture-related chemical or physical agents to farmers, their families, and applicators so as to implement controls that prevent harmful exposures.
Goal 4: Hazard-Control Systems—Reduce injuries and illnesses resulting from work-related exposures by developing, demonstrating, and making available control systems that eliminate, guard against, or warn of the hazard.
Goal 5: Outreach—Reduce injuries and illnesses by informing and educating employers and employees in AFF about occupational safety and health hazards and control systems.
SOURCE: NIOSH, 2006a.
OCR for page 48
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
adopted by NIOSH to characterize the steps in the committee’s work. An examination of goals, inputs, activities, and outputs was used to assess the relevance of the program’s research. Intermediate outcomes and end outcomes were examined to evaluate the impact of the program’s research. Illustrative examples of each of those terms, as defined by the Framework Document and used in this report, are provided in Box 3-2.
Evaluation of Strategic Goals and Objectives
The AFF Program’s stated goals are general and appear appropriately aligned with congressional mandate, but they do not include specific measurable objectives that would guide the subprograms and allow for assessment of progress. Furthermore, it does not appear that strategic goals have been developed specifically for each sector, that is, agriculture, forestry, and fishing. The committee requested additional information from NIOSH regarding the strategic plan for the AFF subprograms and questioned whether there were separate plans for each sector. The NIOSH response indicated that although a series of informal planning meetings focusing on safety and health research in agriculture was organized in the early 1990s, management of the resulting projects followed the established process and structure: projects were primarily investigator-initiated and proposed to meet needs described in general plans but with very little institute-wide tactical planning or programmatic management. Each NIOSH division then made annual continuation and funding decisions on the basis of perceived need on a project-by-project basis (NIOSH Response to Committee Questions 1-3, see Appendix C).
In 1996, on the basis of input from more than 500 stakeholders, NIOSH presented the National Occupational Research Agenda (NORA) that identified 21 priority research areas in the broad categories of Disease and Injury, Work Environment and Workforce, and Research Tools and Approaches. Priority research included traumatic injuries, special populations at risk, control technology and personal protective equipment, intervention effectiveness research, and surveillance research methods. It appears that NORA was used in lieu of a more formalized sector-specific strategic planning process for the first decade (1996-2005).
NORA repeatedly raised the question of sector-specific research (NORA, 2000). NIOSH decided that the most effective way to integrate consideration of research in specific sectors was to apply a “matrix approach” of coordinated research in some or all of the 21 priority areas for each sector. As evidence of the success of that approach, NIOSH presented data showing roughly half the NIOSH NORA research funds for FY 2000 going to agriculture ($8.3 million), construction ($9.3 million), mining ($12 million), and healthcare ($5.1 million). However, it is not clear whether funds spent in those sectors were directed toward projects that would
OCR for page 49
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
BOX 3-2
Logic Model Terms and Examples
Planning Inputs: Stakeholder input, surveillance, and intervention data, and risk assessments (e.g., input from Federal Advisory Committee Act panels or the National Occupational Research Agenda research partners, intramural surveillance information, Health Hazard Evaluations [HHEs]).
Production Inputs: Intramural and extramural funding, staffing, management structure, and physical facilities.
Activities: Efforts and work of the program, staff, grantees, and contractors (e.g., surveillance, health effects research, intervention research, health services research, information dissemination, training, and technical assistance).
Outputs: A direct product of a NIOSH research program that is logically related to the achievement of desirable and intended outcomes (e.g., publications in peer-reviewed journals, recommendations, reports, website content, workshops and presentations, databases, educational materials, scales and methods, new technologies, patents, and technical assistance).
Intermediate Outcomes: Related to the program’s association with behaviors and changes at individual, group, and organizational levels in the workplace. An assessment of the worth of NIOSH research and its products by outside stakeholders (e.g., production of standards or regulations based in whole or in part on NIOSH research; attendance in training and education programs sponsored by other organizations; use of publications, technologies, methods, or recommendations by workers, industry, and occupational safety and health professionals in the field; and citations of NIOSH research by industry and academic scientists).
End Outcomes: Improvements in safety and health in the workplace. Defined by measures of health and safety and of impact on processes and programs (e.g., changes related to health, including decreases in injuries, illnesses, or deaths and decreases in exposures due to research in a specific program or subprogram).
External Factors: Actions or forces beyond NIOSH’s control (e.g., by industry, labor, regulators, and other entities) with important bearing on the incorporation in the workplace of NIOSH’s outputs to enhance safety and health.
SOURCE: Framework Document (see Appendix A).
OCR for page 50
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
be identified by stakeholders as having the highest priority. And it is unclear how the manager or coordinator of a sector-specific research program (such as the AFF Program or the Mining Program), even if armed with a well-defined, clearly described strategic plan with detailed measurable objectives, could influence allocation or direction of resources to achieve sector-specific goals.
NIOSH is currently restructuring its research portfolio (NORA 2). The research programs will be subdivided into eight NORA sector programs based on industry-sector groups; 15 NIOSH cross-sector programs based on occupational health outcomes, statutory programs, and global health efforts; and seven NIOSH-coordinated emphasis areas, which are cross-cutting programs to be integrated into the sector and cross-sector programs (FY07 Project Planning Guidance in NIOSH, 2006a). Each of the 30 research programs will have a manager and coordinator.
AFF is one of the eight sector programs and is developing strategic goals. The Office of Management and Budget (OMB) requires that the strategic goals be organized by outcomes, such as reductions in injuries and illnesses, instead of more general topics, such as reducing all occupational mortality in AFF industries. That approach would yield specific targeted outcomes, performance measures for evaluating progress toward meeting outcome goals, and intermediate objectives necessary to meet goals. NIOSH identified the following challenges to setting such goals (available at http://www.cdc.gov/niosh/programs/agff/goals.html):
The focus is on a subgroup of issues where NIOSH can have an impact. A long list would spread resources too thin; not all worthwhile topics can be included.
It is difficult to develop performance measures. Injury statistics have limitations, and exposure and health outcome measures are often unavailable.
NIOSH is a research agency and so it does not often directly influence outcomes. It is ambitious to set goals to achieve such outcomes as reductions in a national mortality rate; NIOSH would need to form effective partnerships and influence other groups to show results.
On the basis of that information, it appears that the AFF Program will be moving toward an improved strategic planning process that will entail development of well-defined, clearly described goals with measurable objectives. However, it remains to be seen whether the complex nature of this program portfolio matrix will facilitate or hinder research planning, implementation, communication, and assessment. It was noted that intervention effectiveness research no longer has specific emphasis in NORA, although it would probably fall in the AFF sector or appropriate cross-sector programs or coordinated emphasis areas, such as personal protective technology or engineering controls.
OCR for page 51
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
Inputs
Planning Inputs
Planning inputs include surveillance data, stakeholder needs, partner aims, information emanating from symposia and conferences, program evaluations, and program goals. Complete and accurate surveillance data with complete detail are critical for the success of the AFF Program and are required to define the current health status of the population at risk, identify health risks within the population, track changes in the population health risk, and identify the need for and evaluate the effectiveness of interventions. Most available data on occupational illness, injury, and hazard surveillance have relied heavily on the Bureau of Labor Statistics Survey of Occupational Injuries and Illnesses (SOII). However, the SOII has shortcomings that result in incomplete information on the AFF Program population at risk: it does not cover farming establishments with 10 or fewer full-time workers (which make up more than 95 percent of U.S. farms) and does not track injuries on farms that have no hired workers (74 percent of U.S. farms) (NIOSH, 2006a). NIOSH has attempted to fill the surveillance gaps through various means, including intramural and extramural programs in illness surveillance, injury surveillance, and traumatic death surveillance. Despite those efforts, the surveillance input remains inadequate, and the size and characteristics of the populations at risk, the health risks, and changes remain uncertain. The uncertainties hamper the ability of AFF Program personnel to plan and to evaluate the effectiveness of previously implemented programs.
Shortcomings in the program goals themselves have been described previously: the stated goals of the AFF Program are general and do not include specific measurable objectives that would guide the subprograms and allow for assessment of progress. The strategic goals do not appear to have been developed specifically for each AFF sector. Thus, progress toward measurable objectives cannot be used as a planning input to direct future activities.
The lack of more specific program goals and objectives also limits the effectiveness of program evaluations. The AFF Program has undergone several evaluations over the last decade to examine various components of the overall program (NIOSH, 2006a). The evaluations included review of the extramural cooperative agreement programs (Kennedy, 1995), review of intramural research by the subcommittee for agricultural review of the NIOSH Board of Scientific Counselors (NIOSH, 2000), and an annual self-evaluation of the NIOSH Centers for Agricultural Disease and Injury Research, Education, and Prevention (Ag Centers) orchestrated by the High Plains Intermountain Center for Agricultural Health and Safety (HICAHS). NIOSH has responded to recommendations made in the 1995 and 2000 evaluations, but the ability of those review groups to evaluate the
OCR for page 52
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
AFF subprograms was probably hindered by a lack of formal program goals with measurable objectives.
Planning inputs in the form of stakeholder needs, partner aims, and information on emerging issues and the effectiveness of practices and programs seem to come largely from interaction and networking at conferences and symposia. In addition, on the recommendation of the 1995 program review, NIOSH established an internal NIOSH Agricultural Steering Committee (NASC) that included representatives of all divisions engaged in the agricultural research program. The group met annually and established research priorities; however, NASC did not manage existing projects or approve new ones, so participation waned; there have been no meetings since 2003. The history of the steering committee is cause for concern because it is unclear whether the approach outlined for NORA 2 addresses the supposed causes of the lack of effectiveness, that is, will the sector-based approach give the director of the AFF Program adequate authority and resources to direct the intramural and extramural research subprograms effectively?
Production Inputs
Production inputs include budget, staff, facilities, management structure, extramural entities, and partners. Over the period 1997-2006, the annual agriculture budget averaged about $24 million (a total of $237,750,550 for 1997-2006). The funds were distributed among Ag Centers (31 percent), intramural programs (44 percent), and other extramural programs (25 percent). Of the total research budget for the 10-year period, about 57 percent ($136 million) was administered through the Office of Extramural Programs (OEP) and directed toward goals in surveillance, priority populations, health effects, control systems, health promotion, support, and the Ag Centers. The remaining 44 percent was distributed among the various divisions, laboratories, and offices performing agricultural research, including the Division of Applied Research and Technology (DART), the Division of Respiratory Disease Studies (DRDS), the Division of Surveillance, Hazard Evaluations, and Field Studies (DSHEFS), the Division of Safety Research (DSR), the Education and Information Division (EID), the Health Effects Laboratory Division (HELD), the National Personal Protective Technology Laboratory (NPPTL), the Pittsburgh Research Laboratory (PRL), and the Spokane Research Laboratory (SRL).
It is difficult to determine the total amount of funding directed specifically to intervention research in the AFF Program because detailed funding information on the Ag Centers was not provided. The lack of that information limits the effectiveness of a review of the subprograms and hinders efforts to evaluate progress toward goals. Apart from intervention research that occurs in the Ag Centers, cumulative total funding for AFF intervention research appears to be around $16 million,
OCR for page 53
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
which represents about 7 percent of the AFF Program budget ($238 million) for 1997-2006. That percentage, which would increase with the inclusion of Ag Center intervention research, demonstrates a substantial commitment by the AFF Program.
The annual number of NIOSH full-time equivalents (FTEs) working on agriculture has ranged from the current low of about 60 (in 2006) to slightly more than 90 in 1999. A breakdown of cumulative FTEs for the 10-year period by program goals shows about 270 FTEs (43 percent) devoted to health effects, about 140 (22 percent) to surveillance, about 110 (17.5 percent) to control systems, about 70 (11.1 percent) to health promotion, and about 40 (6.4 percent) to priority populations. That distribution reflects only NIOSH FTEs and does not show the distribution of Ag Center personnel by AFF Program goals. It is difficult to evaluate the inputs for the Ag Centers because the evidence package presents the overall funding and does not provide more detailed information on how funding was distributed among centers and toward which goals the funds were allocated.
It is difficult to assess the adequacy of AFF Program budget and staff allocations without some type of benchmark or reference research program for comparison. The AFF Program could accomplish more with additional funding, but there is no way to assess the performance of the program without additional information or metrics. Regarding staff inputs, a simple comparison of the ratio of FTEs to intramural research dollars shows that the AFF Program has a higher result (62 FTEs/$9 million = 6.9 in 2006) than other research agency or institute intramural programs, such as that of the National Cancer Institute [1766 FTEs/$687 million = 2.6 in 2006 (NCI, 2006)] and that of the National Institute of Environmental Health Sciences [468 FTEs/$167 million = 2.8 in 2006 (NIEHS, 2007)]. The ratio of FTEs to budget dollars is 5.5 FTEs/$1 million for NIOSH as a whole in 2006 compared with 0.6 and 0.9 for NCI and NIEHS, respectively. That metric may not be appropriate for comparing intramural staffing for research programs of such widely different scope and size, but the result suggests that NIOSH staffing levels are at least as high as those of other organizations when standardized according to funding levels.
Despite the fact that the total number of FTEs associated with the AFF Program seems to be adequate, the management structure and its influence on the effective use of production inputs would also need to be considered. The management structure of NIOSH is based on the divisions, laboratories, and other administrative organizational components, whereas the research programs are organized in groupings that cut across the divisions. As a result, a tabulation of FTEs for the AFF Program includes scientists and engineers in the divisions and laboratories who do not report to the director of the AFF Program. That most probably limits the ability of the AFF director to manage and allocate resources, and this limitation could
OCR for page 54
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
lead to the same set of circumstances that limited the effectiveness of the NASC, that is, the inability to manage existing projects or approve new ones.
NIOSH has division, laboratory, and office facilities in Pittsburgh, Pennsylvania; Morgantown, West Virginia; Spokane, Washington; Cincinnati, Ohio; Washington, DC; and Atlanta, Georgia. No information was provided regarding the condition of those facilities, so it is not possible to assess their adequacy as production inputs.
No specific production input information on extramural entities and partners was provided.
Intramural and Extramural Research
Faced with a relatively small annual budget and what was characterized by Congress as a national crisis concerning the health of farmers and other agricultural workers, NIOSH made two pivotal and far-reaching decisions. The first was to conduct intramural research in fields of science that appeared less well developed. For example, NIOSH conducted ground-breaking work in endotoxin analysis and assisted university-based scientists in describing respiratory effects of particulate exposure in intensive animal production just as that industry was undergoing marked production changes from small, family, largely outdoor operations with little human exposure to large indoor facilities with the emergence of the intensively exposed “8-hour/day” worker.
The second was to establish the Ag Centers: university-based, regionally distributed centers for research, training, and prevention (see Box 3-3). NIOSH was able to encourage regional capacity development across the nation and to take advantage of the extensive support that is a characteristic of university-based research and development. When the 1985 NIOSH-supported International Symposium on Health and Safety in Agriculture was held in Canada, the majority of the little science available was descriptive; there was almost no analytical research and virtually no advanced training or prevention programs. That NIOSH was able, in a period of less than 20 years, to stimulate the development of a legitimate scientific field with new students being trained and good science being conducted across the country has to be regarded as one of the major successes in linking government, universities, and the private sector. Notwithstanding those accomplishments, the committee concludes that a more tightly led cohesive program might have resulted in more gains for AFF workers and their families.
The move to support regionally distributed, university-based programs as a key aspect of the AFF Program has produced far-reaching effects on the quality of research and training and on the diversity of subjects studied. In establishing the university-based Ag Centers, NIOSH appears to have successfully predicted that,
OCR for page 55
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
BOX 3-3
NIOSH Centers for Agricultural Disease and Injury Research, Education, and Prevention (Ag Centers)
The Ag Centers were established as part of the Centers for Disease Control and Prevention (CDC) NIOSH Agricultural Health and Safety Initiative in 1990. The enabling legislation (P.L. 101-517) called for the Ag Centers to “develop model programs for the prevention of illness among agricultural workers and their families … and [to] educat[e] on agricultural safety and health.” The centers were established as cooperative agreements that are geographically distributed to be responsive to agricultural health and safety issues peculiar to different regions of the country. At the time of this report, 10 centers were listed on the NIOSH Web site. The Ag Centers represent well over half the AFF Program investment in extramural research and constitute one of the most important contributions to the AFF Program. The centers are described briefly below.
Great Lakes Center for Agricultural Safety and Health, Columbus, Ohio
The goal of the Great Lakes Center for Agricultural Safety and Health (GLCASH) is to promote agricultural safety and health for farm, forestry, and fishery employers, workers, families, and their communities in the Great Lakes region. GLCASH serves Illinois, Indiana, Kentucky, Michigan, Minnesota, Ohio, Pennsylvania, Wisconsin, and West Virginia. Other Ag Centers overlap in some of those states; it is expected that activities in these states will be communicated and coordinated among centers. The states in the GLCASH working circle share many attributes, such as crops, farming practices, farm sizes, migrant streams, poverty in the Appalachian areas, and strong commercial fishing and timber industries.
Great Plains Center for Agricultural Health, Iowa City, Iowa
Serving Iowa, Kansas, Missouri, and Nebraska, the Great Plains Center for Agricultural Health is dedicated to one central goal: finding ways to protect and promote the health and safety of farmers, farm workers, their families, and their neighbors.
High Plains Intermountain Center for Agricultural Health and Safety, Fort Collins, Colorado
The High Plains Intermountain Center for Agricultural Health and Safety (HICAHS) is dedicated to the improvement of the occupational health, safety, and well-being of the residents of Colorado, the High Plains, and the Rocky Mountain region. HICAHS has served the agricultural population of Public Health Service Region VIII (Colorado, Utah, Wyoming, Montana, North Dakota, and South Dakota) for more than 10 years. The overarching goals of HICAHS are to reduce agricultural injury and illness through focused research, education, and intervention.
OCR for page 56
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
National Children’s Center for Rural and Agricultural Health and Safety, Marshfield, Wisconsin
The National Children’s Center for Rural and Agricultural Health and Safety provides guidance for childhood injury prevention programs in the private and public sectors; guides and supports efforts of major agriculture-related organizations in identifying potential interventions to protect children from agricultural hazards; convenes consensus development sessions to address complex or controversial issues in childhood injury prevention; addresses rural recreational safety, including horses and all-terrain vehicles; provides technical assistance and training for professionals on youth-safety issues; collaborates with major agricultural organizations, health and safety professionals, and youth-serving groups; and enhances communication linkages among child-safety advocates in the public and private sectors.
Northeast Center for Agricultural and Occupational Health, Cooperstown, New York
The New York Center for Agricultural Medicine and Health (NYCAMH) was established by the New York state legislature in 1988. Recognizing the unacceptably high rates of occupational injury and illness in New York’s largest industry, the legislature charged NYCAMH to provide research into the causes and prevention of agricultural injury and illness, education and prevention activities in the farm community, education of professionals serving the farm community, and clinical help for farm-related health problems. In addition to its state mandate, NYCAMH has been designated by NIOSH as the Northeast Center for Agricultural Safety and Health (NEC). Serving a 13-state region from Maine through Virginia, NEC promotes farm health and safety research, education, and prevention activities.
Pacific Northwest Agricultural Safety and Health Center, Seattle, Washington
The Pacific Northwest Agricultural Safety and Health (PNASH) Center works with the farming, forestry, and fishing industries to improve workplace safety and health in Washington, Oregon, Idaho, and Alaska. The main research focus is on developing interventions to reduce injuries and illnesses. The PNASH Center extends the knowledge gained to employers and workers through outreach and professional education.
Southeast Center for Agricultural Health and Injury Prevention, Lexington, Kentucky
The mission of the Southeast Center for Agricultural Health and Injury Prevention is to develop and sustain an innovative program of research, education, and health advocacy to prevent work-related illness and injury and to improve the safety and health of agricultural workers and their families in the southeastern United States. The center serves stakeholders in Kentucky, Tennessee, Alabama, Mississippi, North Carolina, South Carolina, Florida, Georgia, West Virginia, and Virginia.
OCR for page 58
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
committee believes that there has been too much reliance on the latter for national direction on AFF research issues. National priorities would stem from AFF Program leadership to guide extramural research efforts; cooperative agreements provide structure for specific responses to AFF issues, while the research grants maximize investigator interests and competencies.
Stakeholder Input
The AFF Program organized or participated in numerous conferences, symposia, working meetings, and so on, devoted to engaging stakeholders in discussions of vulnerable working populations and small operators. Some activities are found in Table 3-1. The Iowa Center for Agricultural Safety and Health has sponsored regional and national symposia structured around specific AFF issues, such as the national agricultural research agenda (1992 Surgeon General’s Conference on Agricultural Health and Safety), design of occupational and environmental medical
TABLE 3-1 Conferences, Symposia, and Working Meetings to Engage Stakeholders
Conference
Year
CHILDHOOD AGRICULTURAL INJURY
Surgeon General’s Conference on Agricultural Safety and Health
1991
Childhood Agricultural Injury Prevention Symposium
1992
Child and Adolescent Rural Injury Control
1995
4th International Symposium: Rural Health and Safety in a Changing World
1998
Agricultural Safety and Health in a New Century
2000
National Occupational Injury Research Symposium
2000
Federal Interagency Working Group on Preventing Childhood Agricultural Injuries
2001
National Institute for Farm Safety—Annual Meeting
2001
2001 Summit on Childhood Agricultural Injury Prevention
2001
National Injury Prevention and Control Conference
2005
MINORITY POPULATIONS
Expert Panel on Hired Farmworker Occupational Health and Safety
1995
LOGGING
Three workshops devoted to helicopter logging standards in Alaska
1993-1997
FISHERMEN
Fishing Industry Safety and Health (FISH) Workshop
1992
FISH II Workshop
1997
1st International Fishing Industry Safety and Health (IFISH) Workshop
2000
IFISH II
2003
IFISH III
2006
SOURCE: NIOSH, 2006a.
OCR for page 59
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
capacity within Education and Research Centers and the Ag Centers (1988 Institute of Agricultural Medicine and Occupational Health), conducted annual training since 1996 of healthcare professionals who prevent and treat occupational illnesses and injuries occurring on agricultural premises, studied violence prevention within rural areas (1992 Conference on Handgun Injuries: A Public Health Approach), and responded to the agricultural tractor-related death and injury crisis in the United States (1997 Tractor Risk Abatement and Control: The Policy Conference) (Donham et al., 1998). Both the High Plains Intermountain Center for Agricultural Health and Safety and the Southwest Center for Agricultural Health and Safety cosponsored a national conference examining worker health issues associated with agro-terrorism in 2004. Additionally, the Southeast Center for Agricultural Health and Injury Prevention led the effort from 2005-2007 to assess farmer and rancher attitudes towards retrofitting older agricultural tractors as a part of the National Agricultural Tractor Safety Initiative. The Western Center for Agricultural Safety and Health assembled a multi-disciplinary team from the United States and Canada in 2006 to assess the ergonomic and anthropometric parameters of the North American Guidelines for Children’s Agricultural Tasks (NAGCAT) pertaining to tractor operations.
In January 2007, the High Plains Intermountain Center for Agricultural Health and Safety prepared a report evaluating the 10 Ag Centers, and found that the centers collectively produced over 800 products and reached about 4.2 million people.
Responsiveness to Stakeholder Input
Much of the interaction between AFF Program personnel and stakeholders occurs at formal meetings, conferences, and symposia and through NORA town hall gatherings. It is apparent that there is also extensive interaction among investigators and research staff and the stakeholders who partner or participate in research activities. There are numerous examples in the project descriptions of activities that undoubtedly provide opportunities for stakeholders to offer input. However, despite those opportunities and NIOSH’s response to stakeholder input, only a small fraction of all stakeholders participated in the process and felt that they were actively engaged in AFF Program efforts.
There is also a conceptual gap between seeking stakeholder involvement and actually engaging stakeholders, given the large numbers of AFF workers. Convening a conference or a public gathering, in the manner generally used by federal agencies, normally brings “the usual suspects” to the forefront. Advocacy groups claim to speak on behalf of various AFF workers and are often readily able to participate in the types of gatherings described above, whereas a paucity of AFF workers speak on their own behalf. This gap is not entirely the fault of the agency; it is difficult to
OCR for page 60
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
engage important segments of the AFF workforce, especially non-English speaking workers, or the large numbers who lack authorization for U.S. employment.
Special efforts are clearly needed to reach and engage workers in settings that they feel comfortable in, and the AFF Program staff in general has yet to show inclination to initiate direct worker involvement. The notion that simply “hearing” what stakeholders and their representatives have to say about workforce needs is sufficient for input is wrong. A more meaningful way to engage stakeholders would be to directly include them in ongoing discussions of designing and implementing AFF research.
Consistency of Production Inputs with Program Goals and Objectives
Considerable resources are being directed toward achieving the goals of the AFF Program. It is difficult to assess the adequacy of production inputs given the lack of detail regarding the distribution of Ag Center funding and how it is directed toward program goals. The management structure of NIOSH seems to limit the role of AFF Program personnel in directing resources, managing projects, and evaluating progress toward measurable objectives. The lack of a formal strategic planning process with stated aims and measurable objectives exacerbates the problem of management structure and hampers program evaluation. If the role of the AFF Program management is only to compile and manage information, there are substantial shortcomings in the current process for compiling, categorizing, coding, and sorting through the large quantity of information related to the activities, outputs, and outcomes of the various projects.
Activities
Promotion of Activities: Quantity and Quality
The committee found no evidence that inputs were squandered or devoted to activities unrelated to the defined program goals. A more difficult and more important question is whether the limited resources were allocated in a way that would maximize the impact on the highest-priority subjects for the program. The general nature of the program goals and the lack of a more formal strategic planning process with measurable objectives make it difficult to determine how well the inputs were used.
Adequate surveillance data are important for achieving an effective AFF Program, and there are recognized problems with the available sources of surveillance data for AFF populations; the inputs might not be adequate. On the basis of budget information provided in the evidence package (Figures 2-9 and 2-10 in NIOSH,
OCR for page 61
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
2006a), about 15 percent of AFF funding has been allocated to surveillance (Ag Center allocation to surveillance unknown). Additional surveillance efforts occur with respect to high-priority populations and would increase surveillance funding, but the amount cannot be determined from the data provided. As mentioned previously, the distribution of FTEs by AFF Program goals (Figure 2-14 in NIOSH, 2006a) shows a disproportionate number devoted to health effects (roughly twice that to surveillance). There may be valid logistical reasons for program goals to require different levels of personnel, but it is unclear why a goal that is fundamental to the success of the AFF Program (surveillance) would require so many fewer FTEs. Even if high-priority population FTEs (about 40) are combined with surveillance FTEs (about 140), the total (180) is still much lower than the number for health effects (about 270).
The distribution of intramural and extramural funding across the AFF Program appears to be roughly comparable (about $30-40 million) for surveillance and health effects over the period 1997-2006 (Figure 2-9 in NIOSH, 2006a). Again, given the critical nature of surveillance, an equal distribution of resources across the goals may not be the most effective strategy. AFF Program leaders, stakeholders, partners, and so on would need to first determine what resources would be required to establish a comprehensive surveillance program for AFF populations, implement the strategy, and then use the resulting data to allocate remaining resources and direct research in the other subjects.
Planned Transfer Activities
The most recent request for application (RFA) for Ag Centers (PAR-06-057) states specific goals requiring elements of transfer activities. These include the development, implementation, and evaluation of education projects; providing consultation or training to researchers, safety and health professionals, and agricultural extension agents; development, implementation, and evaluation of model programs for prevention of illness and injury; and development of linkages and communication with other government and non-government bodies involved in agricultural health and safety with emphasis on communication with other agricultural safety and health programs. Those requirements ensure that Ag Centers engage in some minimal amount of transfer activity. That is not to suggest that the centers are not now engaged in transfer activities; centers undertake extensive education and outreach activities as described in the comprehensive reports in the evidence package.
Although the AFF Program does not have a planned set of transfer activities, it has funded the development and maintenance of the National Agriculture Safety Database (NASD). That project established a national central repository of
OCR for page 62
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
agricultural health, safety, and injury prevention materials for the agricultural community and agricultural safety specialists. The goals of the NASD are to provide a national information resource for the dissemination of information; disseminate information to agricultural workers on prevention of occupational hazards associated with injury, death, and illness; promote the consideration of safety and health issues in the management of agricultural operations; and provide a convenient way for the agricultural safety and health community to share educational and research materials.
Peer-Review Process
The peer-review process for intramural and extramural research programs is a strength of the AFF Program. Ag Center activities are supported under an RFA process specifically requiring centers to include prevention or intervention programs that address agricultural safety and health. The most recent RFA (PAR-06-057) says that Ag Centers will
Conduct research studies related to the prevention of occupational disease and injury in agricultural producers, workers, and their families.
Develop, implement, and evaluate education projects for promoting health and safety for production agriculture, forestry, and fishing, including farmers, workers, and their families. This would include providing consultation or training to researchers, health and safety professionals, graduate and professional students, agricultural extension agents, and others in a position to improve the health and safety of agricultural workers.
Develop, implement, and evaluate model programs for the prevention of illness and injury in agricultural producers, workers, and their families.
Develop linkages and communication with other government and non-government bodies involved in agricultural health and safety with emphasis on communication with other agricultural health and safety programs.
The RFA and the peer-review process that leads to an Ag Center award ensure that there is a commitment to intervention research and specifically requires that centers include process and outcome measures of intervention research to the extent possible. The process measures are to be detailed enough to allow for replication in other fields. The outcome measures include exposure to injury hazards, knowledge of safety and health hazards, documentation of safety and health behavior change, and changes in the incidence of disease, injury, and death.
Given the management structure of the AFF Program, the peer-review process and the Ag Center RFAs are critical for ensuring that resources are directed
OCR for page 63
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
toward high-priority topics and that activities are consistent with achieving the program goals. The peer-review process is similarly critical for other extramural and intramural research projects. Because AFF Program personnel do not directly manage projects, the only mechanism to ensure that research funding can support projects related to AFF goals is the inclusion of the priorities in NORA and in the initial peer-review process that directs funding to the most relevant projects. An important disadvantage of this approach to project funding is that it assumes an adequate pool of intramural and extramural investigators who will submit high-quality proposals that target the priorities of the AFF Program. If for some reason there are not enough proposals directed toward AFF Program priorities, the funding is likely to go elsewhere. It seems likely that the NORA 2 sector-based approach will strongly encourage proposals targeting AFF priorities; under NORA 1, there were no AFF-specific priorities.
Quality-Assurance Procedures
The AFF Program lacks a program-wide approach to monitoring quality assurance. Quality-assurance controls occur mainly at the beginning of a project through the peer review of the proposal and the comments of reviewers and toward the later stages of a project when outputs are typically presented as peer-reviewed manuscripts or at professional conferences or meetings where peers have the opportunity to review and comment. Although opportunities to assess quality through review of progress reports are needed, there is usually little feedback provided to investigators through this mechanism. In general, investigators assume the responsibility for quality assurance and for oversight of a research project that will lead to credible data, analyses, and conclusions.
Outputs
In the early years of the AFF Program, conferences were the principal mechanism for evaluating progress. The conferences brought together intramural and extramural investigators to facilitate coordination of activities and promote collaboration. The conferences have included the following:
Surgeon General’s Conference on Agricultural Safety and Health (1991)
Farm Flood Response Workshop—Implications for Agricultural Safety and Health (1993)
First National Conference for NIOSH-Sponsored Centers (1994)
Second NIOSH Agricultural Health and Safety Conference (1995)
Third NIOSH Agricultural Health and Safety Conference (1996)
OCR for page 64
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
National Action Plan: Childhood Agricultural Injury Prevention (1996)
Second National Fishing Industry Safety and Health Workshop (1997)
TRAC: The Policy Conference (1997)
Construction-Agriculture-Mining Partnership (CAMP) Workshop (1999)
International Fishing Industry Safety and Health Conference (2000)
ASH-NET Agricultural Safety and Health Conference (2001)
Summit on Childhood Agricultural Injury Prevention (2001)
Second International Fishing Industry Safety and Health Conference (2003)
National Symposium on Agricultural Health and Safety (2004)
Third International Fishing Industry Safety and Health Conference (2006)
It is not possible to characterize how well the AFF Program outputs are disseminated as a whole. Because there is no unified, program-wide approach to transfer activities, the program relies on individual centers and investigators to undertake the process of research-to-practice. In general, it is more difficult to accomplish transfer activities on a larger scale, so the projects that are most successful are probably the ones that have a manageable number of stakeholders or that have partners and existing infrastructure to facilitate widespread dissemination of outputs. The success of transfer activities also depends on the targeted audience. If the primary “consumer” of an output is the community of agricultural health and safety researchers and professionals, presentation at a professional conference or research symposium or in the peer-reviewed literature can be a highly effective means of transfer. Because of the diverse nature of the numerous intervention research projects undertaken and differences in the targeted consumers and partners, some projects are more successful than others; there is no standardized objective means of evaluating the success of transfer activities.
Intermediate Outcomes
The committee lists here a number of the contributions of AFF Program research to inform public policy and regulatory action.
State-level policies in which AFF Program research informed decisionmaking:
Morbidity and Mortality Weekly Report (MMWR) article in August 2004 about chloropicrin drift exposure in California (CDC, 2004) provided justification of new legislation in California requiring growers to reimburse medical expenses incurred by persons injured by pesticide drift.
OCR for page 65
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
Wisconsin Act 455, passed in 1996, prohibits people younger than 16 years old from driving farm tractors on public roads until they complete a tractor and machinery certification course; it was based on Agricultural Health Promotion System research in Wisconsin.
MMWR article in November 1999 about illnesses associated with pesticides used to control Medfly infestations (CDC, 1999a) led the U.S. Department of Agriculture (USDA) and the Florida Department of Agriculture to adopt non-pesticide methods for Medfly control.
Washington State Supreme Court mandated in 2000 that the Washington State Department of Labor and Industries develop a cholinesterase (ChE) monitoring program for workers handling acutely toxic pesticides. Research supported by the AFF Program in California discovered substantial limitations on the accuracy of ChE field testing and led to a new approach to optimize clinical determinations. In 1995, a Technical Advisory Group formed by the Washington department found that a ChE monitoring program was technically feasible and necessary to protect worker health. The rule was implemented by the department in February 2004.
Federal policies in which AFF Program research informed decisionmaking:
The Occupational Safety and Health Administration (OSHA) used a 1976 NIOSH document for a proposed logging standard as a basis of the 1994 OSHA logging standard (29 CFR 1910.266). From July 1989 to October 1990, NIOSH provided three sets of comments to OSHA supporting a proposed logging rule. OSHA’s final logging standard incorporated most of the comments.
In 1999, the U.S. Coast Guard (USCG) initiated a Dockside Enforcement Program to identify and correct safety hazards identified by NIOSH in crab fishing. The program has been “institutionalized” by USCG.
The U.S. Environmental Protection Agency (EPA) banned chlorpyrifos for residential use in 2000 partly on the basis of AFF Program neurological effect studies.
On October 25, 1994, the AFF Program submitted comments to the U.S. Department of Labor (DOL) in response to an advance notice of proposed rule-making regarding child labor. In its comments, the AFF Program provided recommendations for new hazardous orders for both nonagricultural and agricultural industries.
The Youth Worker Protection Act introduced in both 2003 and 2005 by Representative Tom Lantos intended to revise child-labor law was based largely on NIOSH hazardous order recommendations released in 2002. In 2005, the CARE Act (HR 3482) introduced by Representative Lucille Roybal-Allard proposed changes in child-labor laws in agriculture and identified youth farm injury data collected
OCR for page 66
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
by the AFF Child Agricultural Injury Survey as a source of data to be used in an annual report on occupational injuries in youths working on farms in the United States.
End Outcomes
Aside from data provided by the Alaska Field Station and the childhood agricultural initiative, few data were available to link AFF Program research conclusively with reductions in injuries and illnesses in AFF populations. A more thorough examination of end outcomes by research goals is found in Chapters 4-8.
External Factors
The AFF Program operates in an environment shaped by many factors that it cannot control. Some are so fundamental to the nature of the program that the committee found it essential to keep them in mind for all aspects of its review.
The AFF Program is limited in its ability to effect change in the workplace because of the various stakeholders involved (Box 3-4). As a program in a research agency, the AFF Program is in a position to produce knowledge about workplace hazards that could cause injuries and illnesses and to promote the application of this knowledge in the workplace, but it is not responsible for minimizing hazardous workplace environments or ensuring worker compliance; this responsibility falls ultimately on employers, who respond to economic and regulatory imperatives. Some employers may resist implementing recommended measures in the workplace for economic reasons. Authority to establish and enforce workplace regulations lies with DOL’s OSHA. NIOSH is expected to make recommendations to OSHA, but OSHA needs to consider the views of other interested parties that may have concerns that differ from those of NIOSH. Where statutory and labor exemptions apply, NIOSH faces federal regulatory constraints that make it difficult to affect worker safety and health.
NIOSH is a part of a federal agency whose agenda and budget are subject to congressional directive; this makes it difficult for the AFF Program to develop subprograms and dedicate funding for long-term research activities. As a government entity, NIOSH complies with rules and standards in seeking and submitting information. The external approval process has yet to be streamlined by other government agencies, so long delays are common and often hold up research projects and information dissemination. Furthermore, national policy developments can affect employment and harvest patterns in agriculture, forestry, and fishing. The Food Security Act (the Farm Bill), management of fisheries, energy policies, immigration policies, and trade policies are examples of policies that dictate the types
OCR for page 67
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
BOX 3-4
External Factors That Affect the AFF Program (as identified by NIOSH)
The extent to which research activities lead to reductions in injury, illness, or exposure is affected by stakeholder activities and inputs. Actions of industry, labor, and other entities are beyond NIOSH control and have a substantial bearing on the adoption of AFF Program outputs in the workplace. NIOSH has identified three general categories of external factors that may influence AFF Program outcomes: social conditions, economic conditions, and the current regulatory environment.
Social Conditions: NIOSH identified difficulties in identifying and working with the most effective partners as an external factor in the category of social conditions. It stated that early attempts to use the USDA extension service model to distribute research findings and outputs was abandoned to move toward application of the community nurse model in rural settings combined with an active surveillance component. No additional information on the change in tactics was provided. NIOSH also reported some difficulty in partnering with the medical community to deliver cancer-prevention messages directly to patients: It was found that physician cooperation was partial or uneven. As an example of favorable but unpredictable external factors, NIOSH noted the activities of several stakeholder organizations. The National Safety Council adopted a “Farm Safety and Health Week” and established the National Education Center for Agricultural Safety in Iowa. The American Society of Agricultural Engineers (now the American Society of Agricultural and Biological Engineers) established priorities in safety and health and initiated the Journal of Agricultural Safety and Health. The Farm Foundation established an agricultural safety workgroup, and a number of tractor manufacturers launched at-cost rollover-protective-structure retrofit programs. Finally, the Kellogg Foundation funded several grants that targeted sustainable interventions for special populations.
Economic Conditions: NIOSH noted a continuing decrease in the number of farmers and a corresponding increase of concentration in agricultural production. Although the number of farms has declined, the demand for agricultural products has increased. The demand has been met through the use of large-scale mechanization, improvements in crop varieties, and the use of commercial fertilizers and pesticides. The need for human labor has decreased, as indicated by an increase in labor efficiency from 27.5 acres/worker in 1890 to 740 acres/worker in 1990. Another trend in farms is the aging of self-employed principal farm operators. This “graying” of the farm population raises concerns about the long-term health of family farms. It was noted that during the period 1989-2003, farm size shifted toward the smallest and largest sales categories, and production shifted sharply to very large family farms and non-family farms. It was predicted that shifts in production away from farms in the $10,000-249,999 sales class would continue. Small farms tend to specialize in raising beef cattle and various crops, whereas large farms tend to raise hogs and higher-value crops. Some 22% of farms produced more than two commodities, and 65 percent produced one or two commodities. Medium and large farms were more likely to
OCR for page 68
Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health
produce more than one commodity. NIOSH also observed that although equipment dealers have generally resisted safety legislation, some have helped with interventions by selling safety equipment and accessories.
Current Regulatory Environment: AFF sector regulation is spread across several federal agencies, and there are large gaps in the coverage and enforcement of regulation. Agricultural operations are addressed in specific OSHA standards for agriculture and general industry; however, restrictions set through the appropriations process limit enforcement of regulations to operations that employ more than 10 workers. Similar limitations apply to regulation of commercial fishing. Regulation at the state level has varied: It has been somewhat effective in states such as California and Washington with a long history of labor organizing or workforce activism, and less effective in states where self-employed labor is dominant. With respect to the latter, a notable exception is rules requiring the use of slow-moving vehicle signs on farm vehicles traveling slower than 25 mph now in place in 49 states. Most AFF workers are not covered by workers’ compensation programs or do not work for entities that are required to report injuries and illnesses, so few data are available to estimate injury and illness rates and economic costs. Several federal agencies have regulatory responsibilities for portions of AFF operations: EPA regulates pesticide applicators, DOL enforces the Migrant and Seasonal Agricultural Worker Protection Act, USCG enforces standards under the Commercial Fishing Vessel Safety Act, and the Department of Homeland Security, the Federal Aviation Administration, and EPA oversee aerial applications of pesticides and fertilizers.
SOURCE: NIOSH, 2006a.
of products that a region can produce, the amount harvested, and the number of workers employed. The AFF Program can and needs to anticipate policy changes, but it is difficult to predict how the changes will affect worker safety and health.
Over the course of its information gathering, the committee came to understand the degree to which the AFF Program is undergoing change as part of NIOSH’s reorganization effort in conjunction with NORA 2 and by virtue of its self-scrutiny in preparation for this committee’s evaluation. The program identified new research goals and named new leadership in 2006 as it prepared for this evaluation. The program intends to develop a strategic plan through its newly formed NORA 2 AFF Sector Council, but it has deferred results from that activity until the conclusion of the present committee’s evaluation.