Review of the US Navy's Human Health Risk Assessment of the Naval Air Facility at Atsugi, Japan (2001)

Risk communication to the stakeholders should be a continuing process from the beginning of the project. There does not seem to have been a coordinated strategy for risk communication. Such a strategy should have been developed with objectives that are among the overall objectives of the project. The risk-assessment project also seems to lack a fundamental understanding of stakeholders' needs and concerns and a clear process that could be used to update and improve risk communication. Presenting the potential benefits of risk-reduction strategies is an important aspect of risk communication. Improving Risk Communication (NRC 1989), Science and Judgment (NRC 1994), and the Presidential/Congressional Commission on Risk Assessment and Risk Management (1997b) discuss the importance of risk communication in risk assessment and risk management and of how to incorporate risk communication into these processes.

One objective of the NEHC risk assessment was to determine the fraction of health risks at NAF Atsugi attributable to the incinerator complex. The attributable risk has not been adequately evaluated. Pioneer Technologies Corporation listed five ways “to identify the potential impact of emissions from the SIC [Shinkampo Incinerator Complex] ” (Pioneer 2000; p. 82):

1. Comparing the risks due to ambient air when the SIC is ON to when the SIC is OFF (Radian, 2000).

2. Comparing the risks due to ambient air when the SIC is ON, and a site is downwind of the SIC, to when the SIC is OFF (Radian, 2000).

3. Comparing the risks due to ambient air when the SIC is ON, and a site is downwind of the SIC, to when the site is not downwind of the SIC (Radian, 2000).

4. Using the results of the correlation analysis to model concentrations, and subsequently calculate and compare risks, at sites when the site is downwind of the SIC and when the site is not downwind of the SIC.

5. Comparing the risks due to ambient air when the SIC is ON and a site is downwind of the SIC to another site which is upwind of the SIC on the same days (i.e., an “Upwind” versus “Downwind” evaluation).

Pioneer (2000) states that those approaches are discussed in various places in the draft report, but the subcommittee could not find, in any of the documents provided, an adequate evaluation of whether any of the approaches met the objectives of the risk assessment. Of the five, the subcommittee believes that the fourth method is best (or some variation on it) and only it has the potential to assess the contribution of the incinerator to the health risk. The results of the other four could be substantially confounded by pollutants that are not attributable to the incinerator complex. For example, Table 5-11 of the Pioneer draft report (Pioneer 2000; p. 86) shows that the major contributors to the hazard indexes and to the differences in hazard indexes between the two sites examined are pollutants whose concentrations were not correlated with incinerator emissions in any analyses presented.

Two methods—of the Research Triangle Institute (RTI 1999), and Radian (2000d) —that are similar to each other were used to determine the contribution of the incinerator facility by correlating the concentrations of various pollutants measured in the air samples with the fraction of the sampling period during which

the measurement site was downwind of the incinerator (that is, the “percent downwind”). The Research Triangle Institute method (RTI 1999) also used a nonparametric correlation analysis to estimate the impact of the incinerator facility. Neither analysis was interpreted quantitatively, nor were the two methods' results compared. Furthermore, because 24-h sampling was used (Radian 2000a), a site might be downwind for part of a sampling period and upwind or crosswind for the remainder of the period. Continuous or semicontinuous monitoring was previously recommended for correlating meteorologic data and emission-dispersion estimates with ambient concentrations of pollutants (NRC 1998). FTIR monitoring, which is a continuous monitoring method, was conducted in the NEHC risk assessment, but the limit of detection was too high to provide useful data (Radian 2000a). The possibility of using other continuous or semicontinuous methods is not discussed in the NEHC draft summary report. The possibility of “sector-sampling” (switching pumps on and off as the wind direction changes) is discussed but dismissed as impractical and unnecessary for VOCs in the Field Sampling Plan (Radian 1998a, p. 4-13). The subcommittee believes, however, that “sector-sampling ”, or some similar method, might well be necessary to evaluate the incinerator contribution to exposure.

The models used in the correlation analyses are not justified in either the RTI report (1999) or the Radian report (2000a). The correlation analyses consisted principally of fitting a straight line to the relation between percent downwind and the measured concentration or its logarithm, or between rescaled versions of those variables. Such relationships have no physical basis, so it is difficult to interpret the results. As is often the case with environmental measurements, the air-concentration data appear to be approximately lognormally distributed; the distribution should be taken into account in the statistical analysis. Either a least-squares approach to a nonlinear physical model (if the data are log-transformed) or a statistical approach that can account for nonnormal errors should be used.

Pioneer (2000) selected what it called the upwind-versus-downwind approach, but no documentation provides the exact method used, and neither the method nor whether the designations of “upwind” and “downwind” are representative of overall exposures could be determined from the available documentation. The approach uses unverified and possibly unjustified assumptions, and the logic behind the selection of sites for comparison is not clear. The ground-electronics maintenance building (GEMB) is fairly obvious as the site most affected by the incinerator complex, but no rationale is provided for selecting the golf-course as the upwind comparison site. The selection of the golf-course site is particularly surprising in that the “criteria site”—a site southeast of the incinerator facility and considered upwind of the incinerator—was originally selected as the upwind site and designated as the background site in the sampling plan (Radian 1998a; e.g., see Table 4-3, p. 4-24).

The Pioneer document (Pioneer 2000; p. 8) provides the following rationale for ignoring the criteria site:

The criteria site is located southeast of the SIC. No workers, residents, or recreational users are located at this site. Therefore, it was not evaluated in the risk assessment.

That rationale, however, only addresses the use of the criteria site for the evaluation of the total health risk at NAF Atsugi. It does not address the use of the criteria site for evaluating the contribution of the incinerator.

The calculated risk estimates raise the related question of whether the incinerator contributions are important compared with variations among sites that could be caused by sources other than the incinerator. The large difference between “average” and “RME” estimates in Tables 5-2 and 5-10 of the Pioneer draft report (Pioneer 2000) suggests that the available data might not be sufficient to show important differences among sites and that the differences could be due to random variation. The subcommittee recommends that NEHC investigate whether there are statistically significant differences in risk estimates among the various sites.

Radian conducted sophisticated dispersion modeling (Radian 2000a,c). It compared the incinerator's

impact on air quality at the monitoring sites (estimated via modeling) with measured concentrations, correcting for background contaminant concentrations. But the modeling was not used in the risk assessment to determine the contribution of the incinerator to exposures at NAF Atsugi. The subcommittee believes that the dispersion analysis, in conjunction with the correlation analyses, provides the best approach for determining the contribution of the incinerator facility to pollution at NAF Atsugi. Appendix B presents a critique of the dispersion modeling. Although the model has some limitations, the subcommittee recommends that the dispersion modeling and correlation analyses, not an upwind-downwind comparison, be used to determine the risk attributable to the incinerator facility.

In its analyses, NEHC substituted ambient outdoor-air concentrations of pollutants for indoor-air concentrations. The Pioneer risk-assessment draft report (Pioneer 2000) provides four reasons why ambient air concentrations were used as surrogates for indoor air concentrations (see p. 13), but they do not provide an adequate rationale for the substitution in light of the stated objectives for the overall risk assessment (Pioneer 2000; p. 1):

1. Estimate the potential human health risks to U.S. Navy personnel and their families and other individuals living and working on NAF Atsugi, Japan resulting from exposure to constituents of concern (COCs) in soil, ambient air, indoor air, and indoor dust.

2. Estimate the contribution of the risk attributable to emissions from the SIC.

The first reason provided for the substitution of outdoor air measurements for indoor air concentrations is:

1. The objective of collecting the indoor air samples stated in the Sampling and QA/QC Plan to Assess Health Risks Related to Air Quality at NAF Atsugi, Japan (Radian, 1998) was to make a comparison between concentrations found indoors in the United States with concentrations found at NAF Atsugi. The indoor air samples were not intended to determine the contribution of [emissions] from the SIC to concentrations of constituents in indoor air.

The subcommittee, however, is unable to find any indication in the final sampling plan (Radian 1998a) that the objective of the indoor-air sampling was to compare indoor-air concentrations at Atsugi with indoor-air concentrations in the United States. The indoor-air data-quality objectives clearly are based on the requirement to estimate risks associated with indoor inhalation, as is seen in Section 3.2.1 (Radian 1998a):

The question to be addressed by the indoor air sampling effort is:

1. What is the inhalation exposure risk for sensitive receptors in buildings likely to be impacted by the Jinkanpo Incineration Complex?

That that was an objective is reinforced by a response to a comment on the sampling plan (Radian International, unpublished data, January 27, 2000 ²):

The purpose of the indoor air monitoring is to provide exposure estimates for Atsugi residents and dependents that will be used in the risk assessment. These exposure estimates should be representative of a normal or typical total exposure. Although the incinerator emissions are expected to have an adverse effect on indoor air quality, it is not the intent of this project to measure or estimate the differential risk due to the operation of the incinerator. Rather, the goal is to estimate the total risk due to indoor chemical exposures at Atsugi, e.g., is there more risk to live at Atsugi vs. Los Angeles.

The example given in that comment—comparing risks at Atsugi and in Los Angeles—seems to have been interpreted as the entire objective. A sampling plan designed to compare indoor air at Atsugi and the United States, however, would not be designed to collect data suitable for the overall objectives of the project.

In addition, what NEHC means by “normal or typical total exposure” should be explained.

The second stated reason for the substitution of outdoor air measurements for indoor air concentrations is:

Concentrations for the majority of the constituents exceeding RBSCs [risk-based screening concentrations] were higher indoors than outdoors indicating probable indoor air sources (e.g., insulation, carpets, and household chemicals).

Using indoor-air samples that also measure contaminants generated by indoor sources would not affect the use of those samples to estimate the potential health risks to people living at NAF Atsugi, although it might overestimate the impact of the incinerator unless the contributions of sources unrelated to the incinerator can be removed from the exposure estimate.

The third stated reason is:

Passive ventilation systems are used at most locations which make attempts to quantify the contribution of risk attributable to emissions from the SIC highly uncertain.

The use of passive-ventilation systems does not preclude the use of those samples for estimating the overall health risks at NAF Atsugi.

The fourth stated reason is:

Ambient air is the source of constituents in indoor air that are associated with emissions from the SIC.

Although the emissions might be the source of the indoor contaminants, that does not preclude the use of indoor-air measurements to estimate the human health risks at NAF Atsugi.

When comparing the concentrations of contaminants in indoor air at NAF Atsugi with the concentrations in US homes (NEHC 2000; p. 22, and Table 2.5, pp. 25-26), it is not stated whether the status of doors and windows was recorded during indoor sampling. If it was recorded, it should be possible to address the conjecture present on those pages. For instance, indoor and outdoor concentrations could be compared for particular days, and concentrations in buildings with open doors and windows could be compared with those in buildings with closed doors and windows.

Insufficient information is provided on the heating, ventilation, and air- conditioning systems of buildings and their air-exchange rates (for example, see Pioneer 2000; p. 7). More complete information is essential, especially if NEHC is trying to justify the use of outdoor-air concentrations as a surrogate for indoor concentrations in its risk assessment.

To meet the stated objective of evaluating the risks attributable to the incinerator facility, the airmonitoring data collected must be representative. The subcommittee is concerned about potential biases in the monitoring because of the change from the original sampling plan to directed sampling (that is, sampling when NAF Atsugi was predicted to be downwind of the incinerator complex) during the last 2 months. That change was not adequately justified and was not accounted for in the analysis.

The Pioneer risk-assessment draft report (Pioneer 2000; p. 12) states that “in April of 1998 a 14-month ambient air monitoring program was instituted at NAF Atsugi in order to characterize the health effects associated with exposure to ambient air.” Initially, however, a 12-month monitoring program was instituted. Two months of sampling only when the wind was blowing from the incinerator toward NAF Atsugi was added. (i.e., from south or southeasterly directions). Radian (1999b) explained that

These 2 months represent a program extension with a somewhat modified scope of work. These modifications include:

1. The decommissioning of the Golf Course site (following the 10 April 1999 sample).

2. Elimination of sampling during periods when the SIC was not operational.

3. Collection of samples only during periods when winds were projected, based on meteorological forecasts, to directly impact the base (i.e., from south or southeasterly directions).

4. Elimination of sampling for pesticides and SVOCs.

Another change made to the monitoring program during this period was a short-term mercury study using different sampling and analytical methods.

Thus, samples collected during the final 2 months of the study are targeted (except at the golf-course site) and are not equivalent to those of the previous months. For estimating mean exposures at NAF Atsugi, the inclusion of the final 2 months of samples could bias the results by overweighting particular wind directions and probably, if the targeting was successful, overestimating the contribution of the incinerator complex. The targeted sampling does not appear to have been accounted for in any of the later analyses, either in the evaluation of the incinerator-attributable fraction of individual pollutants (on which it probably had a minor effect) or in the risk assessment (in which the effect could be more important).

Even if the monitoring during those months had not been targeted, inclusion of the samples from the additional 2 months would cause overrepresentation of those months of the year if there are substantial month-to-month variations in overall air pollution. Although unbiased annual averages could be estimated, that would not address the issue of targeted sampling. The NEHC draft summary report does not mention potential monthly changes in emissions or exposures, and seasonal variation is noted only briefly in Appendix B. The subcommittee was unable to locate any other analysis of seasonal effects in the documentation, although the sampling plan clearly was devised with that possibility in mind. Because no analysis of month-to-month variations was performed, the subcommittee is unable to determine the magnitude or direction of any bias caused by the extra 2 months of sampling.

The methods used for examining soil-contaminant patterns are described in the Pioneer (2000) risk-assessment draft report. Arsenic, total benzo[a]pyrene (BaP), and total 2,3,7,8-tetrachlorodibenzo-p-dioxin toxicity equivalents (TEQs) were selected for analysis on the basis of their relative toxicity and because they

represent different chemical classes. Pioneer concluded that no spatial trends for arsenic or BaP were found in the Thiessen Polygon (also called Voronoi diagram) analysis. That conclusion for arsenic appears to be based on the apparent randomness of the values in Figure 4. Closer examination of the figure, however, indicates two possible arsenic-contamination areas: one in the southern area close to the incinerator and one in the northeast area. As shown in Figure D-2, zones along the southern border of NAF Atsugi 100 m from the incinerator at their highest points were estimated to have arsenic concentrations in the highest range (6.7–14.7 mg/kg) for the soil layers 0-3 in. deep (0-7.6 cm deep) and 3- 12 in. deep (7.6-30.5 cm deep). Those zones are most frequently downwind of the incinerator. Also, the second-highest concentration range (4-6.7 mg/kg) fans out from a west-northwest direction to the northeast direction. The samples from the surface layer, which appear to have been collected in the Tade River valley north of the incinerator, had arsenic concentrations of 0.43-4 mg/kg. Those concentrations are lower than those of the samples on both sides of the valley. That pattern could be the result of erosion in the river-valley slopes. In addition, the 3-to 12-in. (7.6 to 30.5-cm) soil-layer map shows the highest arsenic concentrations just north of the incinerator. Those results are consistent with the suggestion that arsenic from the incinerator plume has deposited at NAF Atsugi. However, no objective way to confirm or deny those trends is presented. The possibility of deposition of arsenic should be investigated further and the presence or absence of such a trend should be discussed in the conclusions.

It is unclear why no data from the Radian phase II soil-sampling report (Radian 1999d) were used in the risk assessment. Because those data were not used, the discussion of soil trends for metals in the NEHC draft summary report is somewhat misleading. The Radian report concluded that even compared with the undisturbed reference area far from the incinerator facility, few metals in the areas designated by NEHC as areas of concern at NAF Atsugi had concentrations higher than the upper tolerable limit or mean concentrations higher than what was considered safe. The data from the Radian phase II soil-sampling report should be used in the risk assessment before such conclusions are made. It is also unclear why Pioneer's conclusions regarding the Kriging analyses (Pioneer 2000) were not presented in the NEHC draft summary report.

According to the Pioneer draft report (2000; p. 34), toxicity values for 86 of the 246 chemicals of concern were not available from the secondary sources consulted (US EPA's Integrated Risk Information System, IRIS; US EPA's Health Effects Assessment Summary Tables, HEAST; US EPA's National Center for Environmental Assessment, EPA/NCEA; and California EPA). Those 86 chemicals were not evaluated further (see Table 4-4 of Pioneer 2000 for list). It is inaccurate to characterize all those chemicals as having "no available toxicity information". Primary literature and many useful secondary sources should be consulted for toxicity information that could be used in some cases to determine whether exposures to those chemicals at Atsugi might be of concern. For example, there are health-based standards for PM_2.5 and for fluoride in drinking water, and there is a substantial body of toxicity information on ethanol.

As mentioned on page 69 of the Pioneer draft report, failure to consider those 86 chemicals might underestimate risk. A database of cancer potency estimates could be consulted for those 86 chemicals (Gold and Zeiger 1997; see also http://potency.berkeley.edu/cpdb.html for more-current information). Alternatively, a default cancer-slope factor could be used in a sensitivity analysis to assess the impact of including in the risk assessment any of the 86 chemicals rated as potential carcinogens on the basis of weight of evidence. For instance, Caldwell et al. (1998) used the cancer-slope factor of methylene chloride, 7.5 x 10⁻³ (kg-day)/mg as a screening value.