Agriculture, Forestry, and Fishing Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Overall Program Assessment INTRODUCTION Agriculture, forestry, and fishing are three distinct sectors with disparate work- force populations, relevant stakeholders, and occupational concerns. The commit- tee initially attempted to conduct a separate evaluation of the National Institute for Occupational Safety and Health (NIOSH) research program in each of those sectors. After considering information provided by the NIOSH Agriculture, For- estry, and Fishing Research Program (the AFF Program) and discovering that agricultural safety and health represented nearly 99% of the program’s resources, the committee decided instead to evaluate the AFF research program as a whole. The committee focused its evaluation on research in agricultural safety and health but also assessed safety and health efforts in forestry and fishing to the extent that information was provided by NIOSH. The committee provides an overall evaluation of the AFF Program’s strategic goals, including “external” factors that, although beyond the program’s control, affect its activities and performance. It then identifies other important activities that the AFF Program ought to pursue and is pursuing: identifying and engag- ing stakeholders, conducting health services research and training, carrying out program evaluation initiatives, and providing input to public policy. Finally, the committee highlights an element that in many ways has the characteristics of the ideal AFF Program described in Chapter 2. 46 

Overall Program Assessment 47 AGRICULTURE, FORESTRY, AND FISHING RESEARCH PROGRAM GOALS The AFF Program identified five major goals consistent with its congressional mandate: surveillance, special populations at risk, health effects of exposure to agricultural agents, control systems, and outreach (see Box 3-1). The commit- tee evaluated each major goal and compared programmatic components of the existing program with those of the ideal AFF Program described in Chapter 2. The committee chose to rename the major goals so that their names would better encompass and depict the subjects that the AFF Program had identified: surveil- lance (Chapter 7), populations at risk (Chapter 8), health effects research (Chapter 9), intervention research (Chapter 10), and knowledge diffusion and technology transfer (Chapter 11). Following the guidance of the Framework Document, the committee carried out its evaluation by using the terminology and organization of a logic model BOX 3-1 Major Goals of the AFF Program Goal 1:  Surveillance—Reduce injuries and illnesses in the agriculture, forestry, and fishing workforce by understanding the characteristics of those injuries and illnesses so as to target research and interventions that reduce hazardous exposures. Goal 2:  Priority Populations at Risk—Reduce injuries and illnesses of special populations of workers in these sectors by determining their significant risk factors and identifying and recommending interventions. Goal 3:  Health Effects of Agricultural Agent Exposures—Reduce injuries and illnesses by understanding the long-term, chronic effects of exposures from agriculture-related chemical or physical agents to farmers, their families, and applicators so as to implement controls that prevent harmful exposures. Goal 4:  Hazard-Control Systems—Reduce injuries and illnesses resulting from work- related exposures by developing, demonstrating, and making available control systems that eliminate, guard against, or warn of the hazard. Goal 5:  Outreach—Reduce injuries and illnesses by informing and educating employers and employees in AFF about occupational safety and health hazards and control systems. SOURCE: NIOSH, 2006a. 

48 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH adopted by NIOSH to characterize the steps in the committee’s work. An exami- nation of goals, inputs, activities, and outputs was used to assess the relevance of the program’s research. Intermediate outcomes and end outcomes were examined to evaluate the impact of the program’s research. Illustrative examples of each of those terms, as defined by the Framework Document and used in this report, are provided in Box 3-2. Evaluation of Strategic Goals and Objectives The AFF Program’s stated goals are general and appear appropriately aligned with congressional mandate, but they do not include specific measurable objectives that would guide the subprograms and allow for assessment of progress. Further- more, it does not appear that strategic goals have been developed specifically for each sector, that is, agriculture, forestry, and fishing. The committee requested additional information from NIOSH regarding the strategic plan for the AFF sub- programs and questioned whether there were separate plans for each sector. The NIOSH response indicated that although a series of informal planning meetings focusing on safety and health research in agriculture was organized in the early 1990s, management of the resulting projects followed the established process and structure: projects were primarily investigator-initiated and proposed to meet needs described in general plans but with very little institute-wide tactical planning or programmatic management. Each NIOSH division then made annual continua- tion and funding decisions on the basis of perceived need on a project-by-project basis (NIOSH Response to Committee Questions 1-3, see Appendix C). In 1996, on the basis of input from more than 500 stakeholders, NIOSH presented the National Occupational Research Agenda (NORA) that identified 21 priority research areas in the broad categories of Disease and Injury, Work En- vironment and Workforce, and Research Tools and Approaches. Priority research included traumatic injuries, special populations at risk, control technology and personal protective equipment, intervention effectiveness research, and surveillance research methods. It appears that NORA was used in lieu of a more formalized sector-specific strategic planning process for the first decade (1996-2005). NORA repeatedly raised the question of sector-specific research (NORA, 2000). NIOSH decided that the most effective way to integrate consideration of research in specific sectors was to apply a “matrix approach” of coordinated research in some or all of the 21 priority areas for each sector. As evidence of the success of that approach, NIOSH presented data showing roughly half the NIOSH NORA research funds for FY 2000 going to agriculture ($8.3 million), construction ($9.3 million), mining ($12 million), and healthcare ($5.1 million). However, it is not clear whether funds spent in those sectors were directed toward projects that would 

Overall Program Assessment 49 BOX 3-2 Logic Model Terms and Examples Planning Inputs:  Stakeholder input, surveillance, and intervention data, and risk assess- ments (e.g., input from Federal Advisory Committee Act panels or the National Occupational Research Agenda research partners, intramural surveillance information, Health Hazard Evaluations [HHEs]). Production Inputs:  Intramural and extramural funding, staffing, management structure, and physical facilities. Activities:  Efforts and work of the program, staff, grantees, and contractors (e.g., surveil- lance, health effects research, intervention research, health services research, information dissemination, training, and technical assistance). Outputs:  A direct product of a NIOSH research program that is logically related to the achievement of desirable and intended outcomes (e.g., publications in peer-reviewed jour- nals, recommendations, reports, website content, workshops and presentations, data- bases, educational materials, scales and methods, new technologies, patents, and technical assistance). Intermediate Outcomes:  Related to the program’s association with behaviors and changes at individual, group, and organizational levels in the workplace. An assessment of the worth of NIOSH research and its products by outside stakeholders (e.g., production of standards or regulations based in whole or in part on NIOSH research; attendance in training and education programs sponsored by other organizations; use of publications, technologies, methods, or recommendations by workers, industry, and occupational safety and health professionals in the field; and citations of NIOSH research by industry and academic scientists). End Outcomes:  Improvements in safety and health in the workplace. Defined by measures of health and safety and of impact on processes and programs (e.g., changes related to health, including decreases in injuries, illnesses, or deaths and decreases in exposures due to research in a specific program or subprogram). External Factors:  Actions or forces beyond NIOSH’s control (e.g., by industry, labor, regu- lators, and other entities) with important bearing on the incorporation in the workplace of NIOSH’s outputs to enhance safety and health. SOURCE: Framework Document (see Appendix A). 

50 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH be identified by stakeholders as having the highest priority. And it is unclear how the manager or coordinator of a sector-specific research program (such as the AFF Program or the Mining Program), even if armed with a well-defined, clearly described strategic plan with detailed measurable objectives, could influence al- location or direction of resources to achieve sector-specific goals. NIOSH is currently restructuring its research portfolio (NORA 2). The re- search programs will be subdivided into eight NORA sector programs based on industry-sector groups; 15 NIOSH cross-sector programs based on occupational health outcomes, statutory programs, and global health efforts; and seven NIOSH- coordinated emphasis areas, which are cross-cutting programs to be integrated into the sector and cross-sector programs (FY07 Project Planning Guidance in NIOSH, 2006a). Each of the 30 research programs will have a manager and coordinator. AFF is one of the eight sector programs and is developing strategic goals. The Office of Management and Budget (OMB) requires that the strategic goals be organized by outcomes, such as reductions in injuries and illnesses, instead of more general topics, such as reducing all occupational mortality in AFF industries. That approach would yield specific targeted outcomes, performance measures for evaluating progress toward meeting outcome goals, and intermediate objectives necessary to meet goals. NIOSH identified the following challenges to setting such goals (available at http://www.cdc.gov/niosh/programs/agff/goals.html): • The focus is on a subgroup of issues where NIOSH can have an impact. A long list would spread resources too thin; not all worthwhile topics can be included. • It is difficult to develop performance measures. Injury statistics have limita- tions, and exposure and health outcome measures are often unavailable. • NIOSH is a research agency and so it does not often directly influence outcomes. It is ambitious to set goals to achieve such outcomes as reductions in a national mortality rate; NIOSH would need to form effective partnerships and influence other groups to show results. On the basis of that information, it appears that the AFF Program will be mov- ing toward an improved strategic planning process that will entail development of well-defined, clearly described goals with measurable objectives. However, it remains to be seen whether the complex nature of this program portfolio matrix will facilitate or hinder research planning, implementation, communication, and assessment. It was noted that intervention effectiveness research no longer has specific emphasis in NORA, although it would probably fall in the AFF sector or appropriate cross-sector programs or coordinated emphasis areas, such as personal protective technology or engineering controls. 

Overall Program Assessment 51 Inputs Planning Inputs Planning inputs include surveillance data, stakeholder needs, partner aims, information emanating from symposia and conferences, program evaluations, and program goals. Complete and accurate surveillance data with complete detail are critical for the success of the AFF Program and are required to define the current health status of the population at risk, identify health risks within the population, track changes in the population health risk, and identify the need for and evalu- ate the effectiveness of interventions. Most available data on occupational illness, injury, and hazard surveillance have relied heavily on the Bureau of Labor Statistics Survey of Occupational Injuries and Illnesses (SOII). However, the SOII has short- comings that result in incomplete information on the AFF Program population at risk: it does not cover farming establishments with 10 or fewer full-time workers (which make up more than 95 percent of U.S. farms) and does not track injuries on farms that have no hired workers (74 percent of U.S. farms) (NIOSH, 2006a). NIOSH has attempted to fill the surveillance gaps through various means, includ- ing intramural and extramural programs in illness surveillance, injury surveillance, and traumatic death surveillance. Despite those efforts, the surveillance input remains inadequate, and the size and characteristics of the populations at risk, the health risks, and changes remain uncertain. The uncertainties hamper the ability of AFF Program personnel to plan and to evaluate the effectiveness of previously implemented programs. Shortcomings in the program goals themselves have been described previously: the stated goals of the AFF Program are general and do not include specific mea- surable objectives that would guide the subprograms and allow for assessment of progress. The strategic goals do not appear to have been developed specifically for each AFF sector. Thus, progress toward measurable objectives cannot be used as a planning input to direct future activities. The lack of more specific program goals and objectives also limits the effec- tiveness of program evaluations. The AFF Program has undergone several evalua- tions over the last decade to examine various components of the overall program (NIOSH, 2006a). The evaluations included review of the extramural cooperative agreement programs (Kennedy, 1995), review of intramural research by the sub- committee for agricultural review of the NIOSH Board of Scientific Counselors (NIOSH, 2000), and an annual self-evaluation of the NIOSH Centers for Agri- cultural Disease and Injury Research, Education, and Prevention (Ag Centers) orchestrated by the High Plains Intermountain Center for Agricultural Health and Safety (HICAHS). NIOSH has responded to recommendations made in the 1995 and 2000 evaluations, but the ability of those review groups to evaluate the 

52 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH AFF subprograms was probably hindered by a lack of formal program goals with measurable objectives. Planning inputs in the form of stakeholder needs, partner aims, and informa- tion on emerging issues and the effectiveness of practices and programs seem to come largely from interaction and networking at conferences and symposia. In addition, on the recommendation of the 1995 program review, NIOSH established an internal NIOSH Agricultural Steering Committee (NASC) that included repre- sentatives of all divisions engaged in the agricultural research program. The group met annually and established research priorities; however, NASC did not manage existing projects or approve new ones, so participation waned; there have been no meetings since 2003. The history of the steering committee is cause for concern because it is unclear whether the approach outlined for NORA 2 addresses the supposed causes of the lack of effectiveness, that is, will the sector-based approach give the director of the AFF Program adequate authority and resources to direct the intramural and extramural research subprograms effectively? Production Inputs Production inputs include budget, staff, facilities, management structure, ex- tramural entities, and partners. Over the period 1997-2006, the annual agriculture budget averaged about $24 million (a total of $237,750,550 for 1997-2006). The funds were distributed among Ag Centers (31 percent), intramural programs (44 percent), and other extramural programs (25 percent). Of the total research budget for the 10-year period, about 57 percent ($136 million) was administered through the Office of Extramural Programs (OEP) and directed toward goals in surveillance, priority populations, health effects, control systems, health promotion, support, and the Ag Centers. The remaining 44 percent was distributed among the various divisions, laboratories, and offices performing agricultural research, including the Division of Applied Research and Technology (DART), the Division of Respiratory Disease Studies (DRDS), the Division of Surveillance, Hazard Evaluations, and Field Studies (DSHEFS), the Division of Safety Research (DSR), the Education and Information Division (EID), the Health Effects Laboratory Division (HELD), the National Personal Protective Technology Laboratory (NPPTL), the Pittsburgh Research Laboratory (PRL), and the Spokane Research Laboratory (SRL). It is difficult to determine the total amount of funding directed specifically to intervention research in the AFF Program because detailed funding information on the Ag Centers was not provided. The lack of that information limits the effective­ ness of a review of the subprograms and hinders efforts to evaluate ­progress toward goals. Apart from intervention research that occurs in the Ag Centers, cumulative total funding for AFF intervention research appears to be around $16 million, 

Overall Program Assessment 53 which represents about 7 percent of the AFF Program budget ($238 million) for 1997-2006. That percentage, which would increase with the inclusion of Ag Center intervention research, demonstrates a substantial commitment by the AFF Program. The annual number of NIOSH full-time equivalents (FTEs) working on agri- culture has ranged from the current low of about 60 (in 2006) to slightly more than 90 in 1999. A breakdown of cumulative FTEs for the 10-year period by program goals shows about 270 FTEs (43 percent) devoted to health effects, about 140 (22 percent) to surveillance, about 110 (17.5 percent) to control systems, about 70 (11.1 percent) to health promotion, and about 40 (6.4 percent) to priority populations. That distribution reflects only NIOSH FTEs and does not show the distribution of Ag Center personnel by AFF Program goals. It is difficult to evaluate the inputs for the Ag Centers because the evidence package presents the overall funding and does not provide more detailed information on how funding was distributed among centers and toward which goals the funds were allocated. It is difficult to assess the adequacy of AFF Program budget and staff allocations without some type of benchmark or reference research program for comparison. The AFF Program could accomplish more with additional funding, but there is no way to assess the performance of the program without additional information or metrics. Regarding staff inputs, a simple comparison of the ratio of FTEs to intramural research dollars shows that the AFF Program has a higher result (62 FTEs/$9 million = 6.9 in 2006) than other research agency or institute intramural programs, such as that of the National Cancer Institute [1766 FTEs/$687 million = 2.6 in 2006 (NCI, 2006)] and that of the National Institute of Environmental Health Sciences [468 FTEs/$167 million = 2.8 in 2006 (NIEHS, 2007)]. The ratio of FTEs to budget dollars is 5.5 FTEs/$1 million for NIOSH as a whole in 2006 compared with 0.6 and 0.9 for NCI and NIEHS, respectively. That metric may not be appropriate for comparing intramural staffing for research programs of such widely different scope and size, but the result suggests that NIOSH staffing levels are at least as high as those of other organizations when standardized according to funding levels. Despite the fact that the total number of FTEs associated with the AFF Program seems to be adequate, the management structure and its influence on the effec- tive use of production inputs would also need to be considered. The management structure of NIOSH is based on the divisions, laboratories, and other administra- tive organizational components, whereas the research programs are organized in groupings that cut across the divisions. As a result, a tabulation of FTEs for the AFF Program includes scientists and engineers in the divisions and laboratories who do not report to the director of the AFF Program. That most probably limits the abil- ity of the AFF director to manage and allocate resources, and this limitation could 

54 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH lead to the same set of circumstances that limited the effectiveness of the NASC, that is, the inability to manage existing projects or approve new ones. NIOSH has division, laboratory, and office facilities in Pittsburgh, Pennsylva- nia; Morgantown, West Virginia; Spokane, Washington; Cincinnati, Ohio; Wash- ington, DC; and Atlanta, Georgia. No information was provided regarding the condition of those facilities, so it is not possible to assess their adequacy as pro- duction inputs. No specific production input information on extramural entities and partners was provided. Intramural and Extramural Research Faced with a relatively small annual budget and what was characterized by Congress as a national crisis concerning the health of farmers and other agricul- tural workers, NIOSH made two pivotal and far-reaching decisions. The first was to conduct intramural research in fields of science that appeared less well developed. For example, NIOSH conducted ground-breaking work in endotoxin analysis and assisted university-based scientists in describing respiratory effects of particulate exposure in intensive animal production just as that industry was undergoing marked production changes from small, family, largely outdoor operations with little human exposure to large indoor facilities with the emergence of the inten- sively exposed “8-hour/day” worker. The second was to establish the Ag Centers: university-based, regionally dis- tributed centers for research, training, and prevention (see Box 3-3). NIOSH was able to encourage regional capacity development across the nation and to take ad- vantage of the extensive support that is a characteristic of university-based research and development. When the 1985 NIOSH-supported International Symposium on Health and Safety in Agriculture was held in Canada, the majority of the little science available was descriptive; there was almost no analytical research and vir- tually no advanced training or prevention programs. That NIOSH was able, in a period of less than 20 years, to stimulate the development of a legitimate scientific field with new students being trained and good science being conducted across the country has to be regarded as one of the major successes in linking government, universities, and the private sector. Notwithstanding those accomplishments, the committee concludes that a more tightly led cohesive program might have resulted in more gains for AFF workers and their families. The move to support regionally distributed, university-based programs as a key aspect of the AFF Program has produced far-reaching effects on the quality of research and training and on the diversity of subjects studied. In establishing the university-based Ag Centers, NIOSH appears to have successfully predicted that, 

Overall Program Assessment 55 BOX 3-3 NIOSH Centers for Agricultural Disease and Injury Research, Education, and Prevention (Ag Centers) The Ag Centers were established as part of the Centers for Disease Control and Preven- tion (CDC) NIOSH Agricultural Health and Safety Initiative in 1990. The enabling legislation (P.L. 101-517) called for the Ag Centers to “develop model programs for the prevention of illness among agricultural workers and their families . . . and [to] educat[e] on agricultural safety and health.” The centers were established as cooperative agreements that are geo- graphically distributed to be responsive to agricultural health and safety issues peculiar to different regions of the country. At the time of this report, 10 centers were listed on the NIOSH Web site. The Ag Centers represent well over half the AFF Program investment in extramural research and constitute one of the most important contributions to the AFF Program. The centers are described briefly below. Great Lakes Center for Agricultural Safety and Health, Columbus, Ohio The goal of the Great Lakes Center for Agricultural Safety and Health (GLCASH) is to promote agricultural safety and health for farm, forestry, and fishery employers, workers, families, and their communities in the Great Lakes region. GLCASH serves Illinois, Indiana, Kentucky, Michigan, Minnesota, Ohio, Pennsylvania, Wisconsin, and West Virginia. Other Ag Centers overlap in some of those states; it is expected that activities in these states will be com- municated and coordinated among centers. The states in the GLCASH working circle share many attributes, such as crops, farming practices, farm sizes, migrant streams, poverty in the Appalachian areas, and strong commercial fishing and timber industries. Great Plains Center for Agricultural Health, Iowa City, Iowa Serving Iowa, Kansas, Missouri, and Nebraska, the Great Plains Center for Agricultural Health is dedicated to one central goal: finding ways to protect and promote the health and safety of farmers, farm workers, their families, and their neighbors. High Plains Intermountain Center for Agricultural Health and Safety, Fort Collins, Colorado The High Plains Intermountain Center for Agricultural Health and Safety (HICAHS) is dedi- cated to the improvement of the occupational health, safety, and well-being of the residents of Colorado, the High Plains, and the Rocky Mountain region. HICAHS has served the agricultural population of Public Health Service Region VIII (Colorado, Utah, Wyoming, Montana, North Dakota, and South Dakota) for more than 10 years. The overarching goals of HICAHS are to reduce agricultural injury and illness through focused research, education, and intervention. continued 

56 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH BOX 3-3 Continued National Children’s Center for Rural and Agricultural Health and Safety, Marshfield, Wisconsin The National Children’s Center for Rural and Agricultural Health and Safety provides guidance for childhood injury prevention programs in the private and public sectors; guides and sup- ports efforts of major agriculture-related organizations in identifying potential interventions to protect children from agricultural hazards; convenes consensus development sessions to address complex or controversial issues in childhood injury prevention; addresses rural recreational safety, including horses and all-terrain vehicles; provides technical assistance and training for professionals on youth-safety issues; collaborates with major agricultural organizations, health and safety professionals, and youth-serving groups; and enhances communication linkages among child-safety advocates in the public and private sectors. Northeast Center for Agricultural and Occupational Health, Cooperstown, New York The New York Center for Agricultural Medicine and Health (NYCAMH) was established by the New York state legislature in 1988. Recognizing the unacceptably high rates of occupa- tional injury and illness in New York’s largest industry, the legislature charged NYCAMH to provide research into the causes and prevention of agricultural injury and illness, education and prevention activities in the farm community, education of professionals serving the farm community, and clinical help for farm-related health problems. In addition to its state mandate, NYCAMH has been designated by NIOSH as the Northeast Center for Agricultural Safety and Health (NEC). Serving a 13-state region from Maine through Virginia, NEC pro- motes farm health and safety research, education, and prevention activities. Pacific Northwest Agricultural Safety and Health Center, Seattle, Washington The Pacific Northwest Agricultural Safety and Health (PNASH) Center works with the farm- ing, forestry, and fishing industries to improve workplace safety and health in Washington, Oregon, Idaho, and Alaska. The main research focus is on developing interventions to reduce injuries and illnesses. The PNASH Center extends the knowledge gained to employers and workers through outreach and professional education. Southeast Center for Agricultural Health and Injury Prevention, Lexington, Kentucky The mission of the Southeast Center for Agricultural Health and Injury Prevention is to develop and sustain an innovative program of research, education, and health advocacy to prevent work-related illness and injury and to improve the safety and health of agricultural workers and their families in the southeastern United States. The center serves stakehold- ers in Kentucky, Tennessee, Alabama, Mississippi, North Carolina, South Carolina, Florida, Georgia, West Virginia, and Virginia. 

Overall Program Assessment 57 BOX 3-3 Continued Southern Coastal Agromedicine Center, Greenville, North Carolina The Southern Coastal Agromedicine Center is an integral component of the North Carolina Agromedicine Institute. The center extends the reach of the institute’s activities through the involvement of partner states in Virginia, North Carolina, South Carolina, Georgia, Alabama, Mississippi, Florida, Puerto Rico, and the Virgin Islands. These states and territories face many common agromedicine issues related to climate, crops, strong timber and fishing industries, a large migrant worker population, and endemic rural poverty. Southwest Center for Agricultural Health, Injury Prevention, and Education, Tyler, Texas The mission of the Southwest Center for Agricultural Health, Injury Prevention, and Educa- tion is to foster, disseminate, and evaluate activities related to health, injury prevention, and education among agricultural interest groups to promote health and safety practices among agricultural workers and their families in New Mexico, Texas, Oklahoma, Arkansas, and Louisiana. Western Center for Agricultural Health and Safety, Davis, California The Western Center for Agricultural Health and Safety is a comprehensive, multidisciplinary program dedicated to the understanding and prevention of illness and injury in western agriculture. The center is affiliated with the University of California, Davis Medical School and the Department of Public Health Sciences. The center conducts extensive surveillance and documentation of health needs. Research focuses on small particulate matter and respiratory health. SOURCE: NIOSH, 2007a. by and large, the centers would be responsive to regional issues. That the budgets for the Ag Centers have been generally modest appears to have reflected NIOSH’s understanding that in an extremely diverse set of industries, the AFF Program could aspire to be only representative, not comprehensive. Given the budget amounts available to NIOSH and the magnitude of the task, the committee believes that that was a prudent approach. Funding and Review  Extramural research is funded by two different mecha- nisms: cooperative agreements and research grants (such as R01 or R21 grants). The Ag Centers are funded through cooperative agreements, and individual research- ers or groups of investigators are funded through traditional research grants. The 

58 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH committee believes that there has been too much reliance on the latter for national direction on AFF research issues. National priorities would stem from AFF Program leadership to guide extramural research efforts; cooperative agreements provide structure for specific responses to AFF issues, while the research grants maximize investigator interests and competencies. Stakeholder Input The AFF Program organized or participated in numerous conferences, sympo- sia, working meetings, and so on, devoted to engaging stakeholders in discussions of vulnerable working populations and small operators. Some activities are found in Table 3-1. The Iowa Center for Agricultural Safety and Health has sponsored regional and national symposia structured around specific AFF issues, such as the national agricultural research agenda (1992 Surgeon General’s Conference on Agri­ cultural Health and Safety), design of occupational and environmental medical TABLE 3-1  Conferences, Symposia, and Working Meetings to Engage Stakeholders Conference Year CHILDHOOD AGRICULTURAL INJURY Surgeon General’s Conference on Agricultural Safety and Health 1991 Childhood Agricultural Injury Prevention Symposium 1992 Child and Adolescent Rural Injury Control 1995 4th International Symposium: Rural Health and Safety in a Changing World 1998 Agricultural Safety and Health in a New Century 2000 National Occupational Injury Research Symposium 2000 Federal Interagency Working Group on Preventing Childhood Agricultural Injuries 2001 National Institute for Farm Safety—Annual Meeting 2001 2001 Summit on Childhood Agricultural Injury Prevention 2001 National Injury Prevention and Control Conference 2005 MINORITY POPULATIONS Expert Panel on Hired Farmworker Occupational Health and Safety 1995 LOGGING Three workshops devoted to helicopter logging standards in Alaska 1993-1997 FISHERMEN Fishing Industry Safety and Health (FISH) Workshop 1992 FISH II Workshop 1997 1st International Fishing Industry Safety and Health (IFISH) Workshop 2000 IFISH II 2003 IFISH III 2006 SOURCE: NIOSH, 2006a. 

Overall Program Assessment 59 capacity within Education and Research Centers and the Ag Centers (1988 Institute of Agricultural Medicine and Occupational Health), conducted annual training since 1996 of healthcare professionals who prevent and treat occupational illnesses and injuries occurring on agricultural premises, studied violence prevention within rural areas (1992 Conference on Handgun Injuries: A Public Health Approach), and responded to the agricultural tractor-related death and injury crisis in the United States (1997 Tractor Risk Abatement and Control: The Policy Conference) (Donham et al., 1998). Both the High Plains Intermountain Center for Agricultural Health and Safety and the Southwest Center for Agricultural Health and Safety co- sponsored a national conference examining worker health issues associated with agro-terrorism in 2004. Additionally, the Southeast Center for Agricultural Health and Injury Prevention led the effort from 2005-2007 to assess farmer and rancher attitudes towards retrofitting older agricultural tractors as a part of the National Agricultural Tractor Safety Initiative. The Western Center for Agricultural Safety and Health assembled a multi-disciplinary team from the United States and Canada in 2006 to assess the ergonomic and anthropometric parameters of the North American Guidelines for Children’s Agricultural Tasks (NAGCAT) pertaining to tractor operations. In January 2007, the High Plains Intermountain Center for Agricultural Health and Safety prepared a report evaluating the 10 Ag Centers, and found that the centers collectively produced over 800 products and reached about 4.2 million people. Responsiveness to Stakeholder Input Much of the interaction between AFF Program personnel and stakeholders oc- curs at formal meetings, conferences, and symposia and through NORA town hall gatherings. It is apparent that there is also extensive interaction among investiga- tors and research staff and the stakeholders who partner or participate in research activities. There are numerous examples in the project descriptions of activities that undoubtedly provide opportunities for stakeholders to offer input. However, despite those opportunities and NIOSH’s response to stakeholder input, only a small fraction of all stakeholders participated in the process and felt that they were actively engaged in AFF Program efforts. There is also a conceptual gap between seeking stakeholder involvement and actually engaging stakeholders, given the large numbers of AFF workers. Convening a conference or a public gathering, in the manner generally used by federal agen- cies, normally brings “the usual suspects” to the forefront. Advocacy groups claim to speak on behalf of various AFF workers and are often readily able to participate in the types of gatherings described above, whereas a paucity of AFF workers speak on their own behalf. This gap is not entirely the fault of the agency; it is difficult to 

60 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH engage important segments of the AFF workforce, especially non-English speaking workers, or the large numbers who lack authorization for U.S. employment. Special efforts are clearly needed to reach and engage workers in settings that they feel comfortable in, and the AFF Program staff in general has yet to show in- clination to initiate direct worker involvement. The notion that simply “hearing” what stakeholders and their representatives have to say about workforce needs is sufficient for input is wrong. A more meaningful way to engage stakeholders would be to directly include them in ongoing discussions of designing and implementing AFF research. Consistency of Production Inputs with Program Goals and Objectives Considerable resources are being directed toward achieving the goals of the AFF Program. It is difficult to assess the adequacy of production inputs given the lack of detail regarding the distribution of Ag Center funding and how it is directed toward program goals. The management structure of NIOSH seems to limit the role of AFF Program personnel in directing resources, managing projects, and evaluating progress toward measurable objectives. The lack of a formal strategic planning process with stated aims and measurable objectives exacerbates the prob- lem of management structure and hampers program evaluation. If the role of the AFF Program management is only to compile and manage information, there are substantial shortcomings in the current process for compiling, categorizing, cod- ing, and sorting through the large quantity of information related to the activities, outputs, and outcomes of the various projects. Activities Promotion of Activities: Quantity and Quality The committee found no evidence that inputs were squandered or devoted to activities unrelated to the defined program goals. A more difficult and more important question is whether the limited resources were allocated in a way that would maximize the impact on the highest-priority subjects for the program. The general nature of the program goals and the lack of a more formal strategic plan- ning process with measurable objectives make it difficult to determine how well the inputs were used. Adequate surveillance data are important for achieving an effective AFF Pro- gram, and there are recognized problems with the available sources of surveillance data for AFF populations; the inputs might not be adequate. On the basis of budget information provided in the evidence package (Figures 2-9 and 2-10 in NIOSH, 

Overall Program Assessment 61 2006a), about 15 percent of AFF funding has been allocated to surveillance (Ag Center allocation to surveillance unknown). Additional surveillance efforts occur with respect to high-priority populations and would increase surveillance fund- ing, but the amount cannot be determined from the data provided. As mentioned previously, the distribution of FTEs by AFF Program goals (Figure 2-14 in NIOSH, 2006a) shows a disproportionate number devoted to health effects (roughly twice that to surveillance). There may be valid logistical reasons for program goals to require different levels of personnel, but it is unclear why a goal that is fundamental to the success of the AFF Program (surveillance) would require so many fewer FTEs. Even if high-priority population FTEs (about 40) are combined with surveillance FTEs (about 140), the total (180) is still much lower than the number for health effects (about 270). The distribution of intramural and extramural funding across the AFF Pro- gram appears to be roughly comparable (about $30-40 million) for surveillance and health effects over the period 1997-2006 (Figure 2-9 in NIOSH, 2006a). Again, given the critical nature of surveillance, an equal distribution of resources across the goals may not be the most effective strategy. AFF Program leaders, stakeholders, partners, and so on would need to first determine what resources would be required to establish a comprehensive surveillance program for AFF populations, implement the strategy, and then use the resulting data to allocate remaining resources and direct research in the other subjects. Planned Transfer Activities The most recent request for application (RFA) for Ag Centers (PAR-06-057) states specific goals requiring elements of transfer activities. These include the development, implementation, and evaluation of education projects; providing consultation or training to researchers, safety and health professionals, and agri- cultural extension agents; development, implementation, and evaluation of model programs for prevention of illness and injury; and development of linkages and communication with other government and non-government bodies involved in agricultural health and safety with emphasis on communication with other agri- cultural safety and health programs. Those requirements ensure that Ag Centers engage in some minimal amount of transfer activity. That is not to suggest that the centers are not now engaged in transfer activities; centers undertake extensive education and outreach activities as described in the comprehensive reports in the evidence package. Although the AFF Program does not have a planned set of transfer activi- ties, it has funded the development and maintenance of the National Agriculture Safety Database (NASD). That project established a national central repository of 

62 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH agricultural health, safety, and injury prevention materials for the agricultural com- munity and agricultural safety specialists. The goals of the NASD are to provide a national information resource for the dissemination of information; disseminate information to agricultural workers on prevention of occupational hazards associ- ated with injury, death, and illness; promote the consideration of safety and health issues in the management of agricultural operations; and provide a convenient way for the agricultural safety and health community to share educational and research materials. Peer-Review Process The peer-review process for intramural and extramural research programs is a strength of the AFF Program. Ag Center activities are supported under an RFA pro- cess specifically requiring centers to include prevention or intervention programs that address agricultural safety and health. The most recent RFA (PAR-06-057) says that Ag Centers will 1. Conduct research studies related to the prevention of occupational disease and injury in agricultural producers, workers, and their families. 2. Develop, implement, and evaluate education projects for promoting health and safety for production agriculture, forestry, and fishing, including farmers, workers, and their families. This would include providing consultation or training to researchers, health and safety professionals, graduate and professional students, agricultural extension agents, and others in a position to improve the health and safety of agricultural workers. 3. Develop, implement, and evaluate model programs for the prevention of illness and injury in agricultural producers, workers, and their families. 4. Develop linkages and communication with other government and non- government bodies involved in agricultural health and safety with emphasis on communication with other agricultural health and safety programs. The RFA and the peer-review process that leads to an Ag Center award ensure that there is a commitment to intervention research and specifically requires that centers include process and outcome measures of intervention research to the extent possible. The process measures are to be detailed enough to allow for repli- cation in other fields. The outcome measures include exposure to injury hazards, knowledge of safety and health hazards, documentation of safety and health be- havior change, and changes in the incidence of disease, injury, and death. Given the management structure of the AFF Program, the peer-review pro- cess and the Ag Center RFAs are critical for ensuring that resources are directed 

Overall Program Assessment 63 toward high-priority topics and that activities are consistent with achieving the program goals. The peer-review process is similarly critical for other extramural and intramural research projects. Because AFF Program personnel do not directly manage projects, the only mechanism to ensure that research funding can support projects related to AFF goals is the inclusion of the priorities in NORA and in the initial peer-review process that directs funding to the most relevant projects. An important disadvantage of this approach to project funding is that it assumes an adequate pool of intramural and extramural investigators who will submit high- quality proposals that target the priorities of the AFF Program. If for some reason there are not enough proposals directed toward AFF Program priorities, the fund- ing is likely to go elsewhere. It seems likely that the NORA 2 sector-based approach will strongly encourage proposals targeting AFF priorities; under NORA 1, there were no AFF-specific priorities. Quality-Assurance Procedures The AFF Program lacks a program-wide approach to monitoring quality assur- ance. Quality-assurance controls occur mainly at the beginning of a project through the peer review of the proposal and the comments of reviewers and toward the later stages of a project when outputs are typically presented as peer-reviewed manu- scripts or at professional conferences or meetings where peers have the opportunity to review and comment. Although opportunities to assess quality through review of progress reports are needed, there is usually little feedback provided to investiga- tors through this mechanism. In general, investigators assume the responsibility for quality assurance and for oversight of a research project that will lead to credible data, analyses, and conclusions. Outputs In the early years of the AFF Program, conferences were the principal mecha- nism for evaluating progress. The conferences brought together intramural and extramural investigators to facilitate coordination of activities and promote col- laboration. The conferences have included the following: • Surgeon General’s Conference on Agricultural Safety and Health (1991) • Farm Flood Response Workshop—Implications for Agricultural Safety and Health (1993) • First National Conference for NIOSH-Sponsored Centers (1994) • Second NIOSH Agricultural Health and Safety Conference (1995) • Third NIOSH Agricultural Health and Safety Conference (1996) 

64 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH • National Action Plan: Childhood Agricultural Injury Prevention (1996) • Second National Fishing Industry Safety and Health Workshop (1997) • TRAC: The Policy Conference (1997) • Construction-Agriculture-Mining Partnership (CAMP) Workshop (1999) • International Fishing Industry Safety and Health Conference (2000) • ASH-NET Agricultural Safety and Health Conference (2001) • Summit on Childhood Agricultural Injury Prevention (2001) • Second International Fishing Industry Safety and Health Conference (2003) • National Symposium on Agricultural Health and Safety (2004) • Third International Fishing Industry Safety and Health Conference (2006) It is not possible to characterize how well the AFF Program outputs are dis- seminated as a whole. Because there is no unified, program-wide approach to transfer activities, the program relies on individual centers and investigators to undertake the process of research-to-practice. In general, it is more difficult to ac- complish transfer activities on a larger scale, so the projects that are most success- ful are probably the ones that have a manageable number of stakeholders or that have partners and existing infrastructure to facilitate widespread dissemination of outputs. The success of transfer activities also depends on the targeted audience. If the primary “consumer” of an output is the community of agricultural health and safety researchers and professionals, presentation at a professional conference or research symposium or in the peer-reviewed literature can be a highly effective means of transfer. Because of the diverse nature of the numerous intervention re- search projects undertaken and differences in the targeted consumers and partners, some projects are more successful than others; there is no standardized objective means of evaluating the success of transfer activities. Intermediate Outcomes The committee lists here a number of the contributions of AFF Program re- search to inform public policy and regulatory action. State-level policies in which AFF Program research informed decision- making: • Morbidity and Mortality Weekly Report (MMWR) article in August 2004 about chloropicrin drift exposure in California (CDC, 2004) provided justification of new legislation in California requiring growers to reimburse medical expenses incurred by persons injured by pesticide drift. 

Overall Program Assessment 65 • Wisconsin Act 455, passed in 1996, prohibits people younger than 16 years old from driving farm tractors on public roads until they complete a tractor and machinery certification course; it was based on Agricultural Health Promotion System research in Wisconsin. • MMWR article in November 1999 about illnesses associated with pesticides used to control Medfly infestations (CDC, 1999a) led the U.S. Department of Agri- culture (USDA) and the Florida Department of Agriculture to adopt non-pesticide methods for Medfly control. • Washington State Supreme Court mandated in 2000 that the Washington State Department of Labor and Industries develop a cholinesterase (ChE) monitor- ing program for workers handling acutely toxic pesticides. Research supported by the AFF Program in California discovered substantial limitations on the accuracy of ChE field testing and led to a new approach to optimize clinical determinations. In 1995, a Technical Advisory Group formed by the Washington department found that a ChE monitoring program was technically feasible and necessary to protect worker health. The rule was implemented by the department in February 2004. Federal policies in which AFF Program research informed decisionmaking: • The Occupational Safety and Health Administration (OSHA) used a 1976 NIOSH document for a proposed logging standard as a basis of the 1994 OSHA logging standard (29 CFR 1910.266). From July 1989 to October 1990, NIOSH provided three sets of comments to OSHA supporting a proposed logging rule. OSHA’s final logging standard incorporated most of the comments. • In 1999, the U.S. Coast Guard (USCG) initiated a Dockside Enforcement Program to identify and correct safety hazards identified by NIOSH in crab fishing. The program has been “institutionalized” by USCG. • The U.S. Environmental Protection Agency (EPA) banned chlorpyrifos for residential use in 2000 partly on the basis of AFF Program neurological effect studies. • On October 25, 1994, the AFF Program submitted comments to the U.S. Department of Labor (DOL) in response to an advance notice of proposed rule- making regarding child labor. In its comments, the AFF Program provided recom- mendations for new hazardous orders for both nonagricultural and agricultural industries. • The Youth Worker Protection Act introduced in both 2003 and 2005 by Representative Tom Lantos intended to revise child-labor law was based largely on NIOSH hazardous order recommendations released in 2002. In 2005, the CARE Act (HR 3482) introduced by Representative Lucille Roybal-Allard proposed changes in child-labor laws in agriculture and identified youth farm injury data collected 

66 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH by the AFF Child Agricultural Injury Survey as a source of data to be used in an annual report on occupational injuries in youths working on farms in the United States. End Outcomes Aside from data provided by the Alaska Field Station and the childhood agri- cultural initiative, few data were available to link AFF Program research conclusively with reductions in injuries and illnesses in AFF populations. A more thorough examination of end outcomes by research goals is found in Chapters 4-8. External Factors The AFF Program operates in an environment shaped by many factors that it cannot control. Some are so fundamental to the nature of the program that the committee found it essential to keep them in mind for all aspects of its review. The AFF Program is limited in its ability to effect change in the workplace because of the various stakeholders involved (Box 3-4). As a program in a research agency, the AFF Program is in a position to produce knowledge about workplace hazards that could cause injuries and illnesses and to promote the application of this knowledge in the workplace, but it is not responsible for minimizing hazard- ous workplace environments or ensuring worker compliance; this responsibility falls ultimately on employers, who respond to economic and regulatory impera- tives. Some employers may resist implementing recommended measures in the workplace for economic reasons. Authority to establish and enforce workplace regulations lies with DOL’s OSHA. NIOSH is expected to make recommendations to OSHA, but OSHA needs to consider the views of other interested parties that may have concerns that differ from those of NIOSH. Where statutory and labor exemptions apply, NIOSH faces federal regulatory constraints that make it difficult to affect worker safety and health. NIOSH is a part of a federal agency whose agenda and budget are subject to congressional directive; this makes it difficult for the AFF Program to develop subprograms and dedicate funding for long-term research activities. As a govern- ment entity, NIOSH complies with rules and standards in seeking and submitting information. The external approval process has yet to be streamlined by other gov- ernment agencies, so long delays are common and often hold up research projects and information dissemination. Furthermore, national policy developments can affect employment and harvest patterns in agriculture, forestry, and fishing. The Food Security Act (the Farm Bill), management of fisheries, energy policies, im- migration policies, and trade policies are examples of policies that dictate the types 

Overall Program Assessment 67 BOX 3-4 External Factors That Affect the AFF Program (as identified by NIOSH) The extent to which research activities lead to reductions in injury, illness, or exposure is affected by stakeholder activities and inputs. Actions of industry, labor, and other entities are beyond NIOSH control and have a substantial bearing on the adoption of AFF Program outputs in the workplace. NIOSH has identified three general categories of external factors that may influence AFF Program outcomes: social conditions, economic conditions, and the current regulatory environment. Social Conditions:  NIOSH identified difficulties in identifying and working with the most effective partners as an external factor in the category of social conditions. It stated that early attempts to use the USDA extension service model to distribute research findings and outputs was abandoned to move toward application of the community nurse model in rural settings combined with an active surveillance component. No additional information on the change in tactics was provided. NIOSH also reported some difficulty in partnering with the medical community to deliver cancer-prevention messages directly to patients: It was found that physician cooperation was partial or uneven. As an example of favorable but unpredictable external factors, NIOSH noted the activities of several stakeholder organiza- tions. The National Safety Council adopted a “Farm Safety and Health Week” and established the National Education Center for Agricultural Safety in Iowa. The American Society of Agricultural Engineers (now the American Society of Agricultural and Biological Engineers) established priorities in safety and health and initiated the Journal of Agricultural Safety and Health. The Farm Foundation established an agricultural safety workgroup, and a number of tractor manufacturers launched at-cost rollover-protective-structure retrofit programs. Finally, the Kellogg Foundation funded several grants that targeted sustainable interventions for special populations. Economic Conditions:  NIOSH noted a continuing decrease in the number of farmers and a corresponding increase of concentration in agricultural production. Although the number of farms has declined, the demand for agricultural products has increased. The demand has been met through the use of large-scale mechanization, improvements in crop varieties, and the use of commercial fertilizers and pesticides. The need for human labor has decreased, as indicated by an increase in labor efficiency from 27.5 acres/worker in 1890 to 740 acres/worker in 1990. Another trend in farms is the aging of self-employed principal farm operators. This “graying” of the farm population raises concerns about the long-term health of family farms. It was noted that during the period 1989-2003, farm size shifted toward the smallest and largest sales categories, and production shifted sharply to very large family farms and non-family farms. It was predicted that shifts in production away from farms in the $10,000-249,999 sales class would continue. Small farms tend to specialize in raising beef cattle and various crops, whereas large farms tend to raise hogs and higher-value crops. Some 22% of farms produced more than two commodities, and 65 percent produced one or two commodities. Medium and large farms were more likely to continued 

68 A g r i c u lt u r e , F o r e s t r y , and Fishing Research at NIOSH BOX 3-4 Continued produce more than one commodity. NIOSH also observed that although equipment dealers have generally resisted safety legislation, some have helped with interventions by selling safety equipment and accessories. Current Regulatory Environment:  AFF sector regulation is spread across several federal agencies, and there are large gaps in the coverage and enforcement of regulation. Agricultural operations are addressed in specific OSHA standards for agriculture and general industry; however, restrictions set through the appropriations process limit enforcement of regulations to operations that employ more than 10 workers. Similar limitations apply to regulation of commercial fishing. Regulation at the state level has varied: It has been somewhat effective in states such as California and Washington with a long history of labor organizing or workforce activism, and less effective in states where self-employed labor is dominant. With respect to the latter, a notable exception is rules requiring the use of slow- moving vehicle signs on farm vehicles traveling slower than 25 mph now in place in 49 states. Most AFF workers are not covered by workers’ compensation programs or do not work for entities that are required to report injuries and illnesses, so few data are available to estimate injury and illness rates and economic costs. Several federal agencies have regu- latory responsibilities for portions of AFF operations: EPA regulates pesticide ­applicators, DOL enforces the Migrant and Seasonal Agricultural Worker Protection Act, USCG enforces standards under the Commercial Fishing Vessel Safety Act, and the Department of Home- land Security, the Federal Aviation Administration, and EPA oversee aerial applications of pesticides and fertilizers. SOURCE: NIOSH, 2006a. of products that a region can produce, the amount harvested, and the number of workers employed. The AFF Program can and needs to anticipate policy changes, but it is difficult to predict how the changes will affect worker safety and health. Over the course of its information gathering, the committee came to un- derstand the degree to which the AFF Program is undergoing change as part of NIOSH’s reorganization effort in conjunction with NORA 2 and by virtue of its self-scrutiny in preparation for this committee’s evaluation. The program identi- fied new research goals and named new leadership in 2006 as it prepared for this evaluation. The program intends to develop a strategic plan through its newly formed NORA 2 AFF Sector Council, but it has deferred results from that activity until the conclusion of the present committee’s evaluation.