National Institute of Statistical Sciences

Comment on Scientific Input to Decision-Making
on Airborne Particulate Standards

Jerome Sacks, National Institute of Statistical Sciences

Alan F. Karr, National Institute of Statistical Sciences

Richard L. Smith, University of North Carolina at Chapel Hill

Jerry M. Davis, North Carolina State University

The National Institute of Statistical Sciences (NISS) has conducted analyses of data from Cook County, IL and Salt Lake County, UT [8] and in Birmingham, AL [1].

Many studies other than ours (for example, of Birmingham [6], Philadelphia [5,7] and Utah Valley, UT [3]) suggest the existence of a consistent, statistically significant, positive association between particulates and mortality, reproducible across many different communities and types of study. By contrast, our results have produced a bewildering variety of conclusions, which are highly sensitive to the selection of meteorological variables, to the treatment of seasonal and long-term trend effects, and to the definition of the exposure measure.

There is an explanation. Informative study of empirical phenomena depends on

• Presence of relevant science
• Availability of adequate data
• Size of the effects,

If, however, relevant science is lacking, the data are inadequate and effects, if they exist at all, are small, then definitive conclusions are impossible, no matter how delicate, detailed or numerous the statistical analyses. This, we believe, is precisely the situation for fine particulates.

As a consequence, results reported in different studies exhibit strong and seemingly inexplicable dependence on fine points of the analyses, such as

• Which data are collected or assembled;
• Which models are employed, and which variables, especially those representing potential confounding effects, are included;
• What analyses are performed;
• iHow the results are interpreted.

To illustrate,

1. Effects, even when significant, may follow no discernible pattern. Seasonal analyses from Cook County, IL, show a statistically significant PM10 effect for fall and spring, but no effect at all in winter and summer [8]. Those from Philadelphia showed significant particulate effects in the spring, summer and fall, but not the winter [5,7]. Analyses from Salt Lake County, UT, show no significant PM10 effect for any season [8]. (Other studies [3,5] report strongly significant results for nearby Utah Valley.)
2. Inclusion or exclusion of confounding variables can influence results dramatically. One analysis [1] of data from Birmingham demonstrated a statistically significant PM10 effect when no measure of humidity is included, but an insignificant effect when specific humidity in included. (Stressful weather is one hypothesized confounder [2].) Inclusion of sulfur dioxide in analyses of Philadelphia data [5,7] leaves a significant PM10 effect only in the summer. In "combined seasons" analyses of Cook County data, the PM10 effect is significant regardless of whether sulfur dioxide is included.
3. There is high sensitivity to the "exposure" measure. A single model (combining seasons) for Cook County [8] yielded a statistically significant PM10 effect if the exposure measure is taken to be a three-day average of PM10 INCLUDING the current day, but no effect if the average EXCLUDES the current day. However, the results in [1, 6] for Birmingham show a significant effect when the three-day average EXCLUDES the current day, but not when it INCLUDES the current day. In an attempt to reconcile these conclusions, we have examined the effects of individual days' contributions of PM10, producing a statistically significant NEGATIVE effect for the current day, and significant positive effects for one-day and three-day lags. Sensitivity to lags is also noted in [4].

We are, therefore, unable to accept that a scientifically meaningful relationship between particulates and mortality has been established.

[1] Davis, J. M., Sacks, J., Saltzman, N., Smith, R. L. and Styer, P. (1996), Airborne particulate matter and daily mortality in Birmingham, Alabama. Technical Report number 55, National Institute of Statistical Sciences (submitted for publication; currently under revision).

[2] Kalkstein, L. S. and Valimont, K. M. (1986), An evaluation of summer discomfort in the United States using a relative climatological index. Bull. Am. Meteorological Soc. 67 842-848.

[3] Pope, C. A., Schwartz, J. and Ransom, M. R. (1992), Daily mortality and PM10 pollution in Utah Valley. Arch. Environ. Health 47 211-217.

[4] Roth, H. D. and Li, Y. (1996), Analysis of the association between air pollutants with mortality and hospital admissions in Birmingham, Alabama: 1986-1990. Technical report, Roth Associates Inc., Rockville MD.

[5] Samet, J. M., Zeger, S. L. and Berhane, K. (1995), The Association of Mortality and Particulate Air Pollution. In "Particulate Air Pollution and Daily Mortality: Replication and Validation of Selected Studies. The Phase I Report of the Particle Epidemiology Evaluation Project". Health Effects Institute, Cambridge MA, pp. 1-104.

[6] Schwartz, J. (1993), Air pollution and daily mortality in Birmingham, Alabama. Am. J. Epidemiology 137 1136-1147.

[7] Schwartz, J. and Dockery, D. W. (1992), Increased mortality in Philadelphia associated with daily air pollution concentrations. Am. Rev. Respir. Dis. 145 600-604.

[8] Styer, P., McMillan, N., Gao, F., Davis, J. and Sacks, J. (1995), The effect of outdoor airborne particulate matter on daily death counts. Environ. Health Perspectives 103 490-497.