Tobacco Documents Online

AN EPIDEMIOLOGICAL REVIEW OF THE EPA REPORT: HEALTH EFFECTS OF PASSIVE SMOKING: ASSESSMENT OF LUNG CANCER IN ADULTS AND RESPIRATORY DISORDERS IN CHILDREN (EPA/600/6-90/00064 - EXTERNAL REVIEW DRAFT) Prepared by: E. Lee Husting, M.P.H., Ph.D., F.A.C.E. September 1990

Table of Contents Section Topic Page 1.0 BACKGROUND ................................... 1 2.0 OOIvIMEN'IS ON MbTHODS OF META-ANALYSIS .......... 4 2.1 Origins of Meta-Analysis .............................. 6 2.2 Publication Bias .................................... 6 2.3 Effect of Quality of Underlying Studies and Data .............. 9 2.4 Comparability of Individual Studies ........................ 10 2.4.1 Comparable Definition and Measurement of Variables .......... 12 2.4.2 Comparable Freedom from Bias and Confounding ............. 12 2.4.3 Comparability of Timing ............................... 13 2.4.4 Comparability of Subjects Between Studies .................. 14 2.5 Analytical Considerations .............................. 16 25.1 Products of the Meta-Analysis ........................... 16 2.5.2 Strength of Association ............................... 17 2.5.3 Confidence Intervals, Significance ......................... 18 2.5.4 Distorting Effect of Large Studies ........................ 19 2.5.5 Effect of Multiple Comparisons .......................... 19 2.5.6 Paradoxical Results .................................. 20 3.0 EVALUATION OF THE META-ANALYSIS ................ 21 3.1 Inclusion and Quality of.Studies ......................... 21 3.2 Publication Bias .................................... 22 3.3 Comparability of Studies ............................... 25 3.3.1 Method of Data Collection ............................. 26 3.3.2 Matching and Adjustment of Data ........................ 26 3.3.3 Elimination of Bias .................................. 27 3.3.4 Consideration of Confounding ........................... 33 3.3.5 Use of Proxy Data Sources ............................. 35 3.3.6 Characteristics of Subjects ............................. 36 3.3.7 Characterization of ETS Exposure ......................... 36 3.3.8 Responses from Both Male and Female .................... 38 3.4 Influence of the Hirayama Study ......................... 38 3.4.1 Cross-Cultural Comparisons ............................ 38 3.5 Analytical Considerations .............................. 41 3.6 Summary of Remarks on the Meta-analysis .................. 42 4.0 EXAMINATION OF ARG U.MENTS AND CRTMRIA FOR CAUSATION .................................. 43 4.1 Plausibility ........................................ 45 4.2 Consistency ....................................... 46 4.3 Dose-Response ..................................... 48 0o 4.4 Broad-Based Evidence ................................ 48 ~ 4.5 Bias and Confounding ................................ 48 ` W Cn M+

4.6 Strength of Association ............................... 50 4.7 Overview of Causality ................................ 53 5.0 OONCLUSION ..................................... 54 Tablel..................................................... 57 List of References ............................................ 58

1.0 BACKGROUND The Environmental Protection Agency (the "EPA" or the "Agency") released on June 23, 1990, a preliminary draft document titled Health Effects of Passive Smoking: Assessment of Lung Cancer in Adults and Respiratory Disorders in Children, EPA/600/6- 90/006A, May, 1990, Review Draft ("The Report"). I am responding to the Agency's solicitation for comments on the technical accuracy and policy implications of The Report. My remarks are restricted to the validity of the meta-analysis described in The Report and to the Agency's conclusion that environmental tobacco smoke (ETS) exposure is associated causally with increased lung cancer incidence. In The Report, the EPA examined information bearing on the issue of a potential statistical association between ETS exposure and lung cancer incidence. Using epidemiologic data of variable origin and quality, the EPA conducted a meta-analysis across epidemiologic studies of ETS and lung cancer to calculate a summary relative risk or statistical association. The Agency meta-analysis yielded a summary risk estimate of 1.41 which is in the range clearly considered weak by many epidemiologists, that is, below 2.0. The summary estimate of 1.41 has a lower confidence interval of 1.26, and an upper limit of 1.57. (The Report, at 4-38). The lower 95% confidence interval value of 1.26 is only 0.07 from the value of 1.19 which is acknowledged in the Report to be compatible with a true relative risk of 1.0. (The Report at 4-39.)

The possibility that bias and confounding are potentially responsible for the association is considered in Chapter 4 of The Report. Tbe Agency asserts that bias due to misclassification of smoking status is not sufficient to explain the association, in spite of the small margin between the lower confidence limit and the upper value compatible with no association. (The Report at 4-39.) The Agency also asserts that no correlate of ETS has been identified that could explain the observed statistical association. (The Report at 4-1.) From the summary risk of 1.41 (CI = 1.26, 1.57), the EPA concludes that ETS "is associated with an increased risk of lung cancer". However, the Report notes that "the stronger conclusion of a causal association is not warranted from these statistical tests alone. Other factors must be considered as well, including the likelihood that the observed association is attributable to systematic bias or presence of a confounding variable." (The Report at 3- 44.) From the summary risk and a brief discussion of causality, the EPA concludes in The Report that ETS should be designated as a Group A carcinogen (known human carcinogen). The Agency purported to follow the U.S. EPA guidelines for carcinogen risk assessment in reaching this conclusion. The Risk Assessment Guidelines of 1986. EPA/600/8-87/045, August 1987, 51 Fed. Re . 33992-34054, September 24 (the "Risk Guidelines"). aD ~ Ln V N -2- ~

The EPA then uses the calculated summary risk value to estimate the range of lung cancer deaths attributable annually to ETS exposure. This estimation assumes and requires that the meta-analysis is justifiable and valid, that the estimate of statistical association is correct, and that the criteria for causality and for assessing carcinogens are met. The logic applied here may hide a tautological fallary. Both the summary risk and the subsequent application of it to estimate population mortality are dependent upon the validity and accuracy of ascertainment of lung cancer and of exposure to ETS. If either of the latter are invalid or inaccurate, the effects of misclassification and other biases may be compounded. In my comments both the Report and its conclusions on causality will be evaluated in terms of the 1986 Risk Guidelines as well as additional recognized epidemiologic criteria, including the strength of the association. The Agency's conclusion regarding ETS and lung cancer is based on its assertion that the epidemiologic studies establish a strong statistical association that is not due to chance alone (Chapter 3) and which reflects a causal nexus (Chapter 4). If either assertion is not proven, then the classification of ETS as a carcinogen is inappropriate and unjustified. Since the EPA's mortality projections are dependent on the meta-analysis and on the conclusion of causality, the population projections are invalid if the above assertions are not accepted. According to the Risk Guidelines of 1986, in order to infer causation between an exposure and cancer in humans, three criteria must be met: ~ ~ ~ C^, -3- N Gn tn

1. There is no identified bias that could explain the association. 2. The possibility of confounding has been considered and ruled out as explaining the association. 3. The association is unlikely to be due to chance. (The Risk Guidelines at 1-11.) These criteria must be met at three logical levels. At the first level the underlying studies must meet the criteria for inclusion in a meta-analysis, including comparability and control of bias and confounding. At a second level, closely linked to the first, the meta- analysis itself must meet certain criteria, including the criterion for appropriate inclusion of studies to calculate a summary statistic. At a third and final level, if the summary statistic suggests a strong association, the criteria for causation must be met before carcinogenicity can be inferred or population mortality projected. The criteria or questions for evaluating a meta-analysis are discussed in Section I. Then, these criteria are applied in Section II to the meta-analysis described in The Report. In Section III, The Report and its conclusions are evaluated in the light of accepted epidemiologic criteria for drawing a causal inference. 2.0 COMMENTS ON METHODS OF META-ANALYSIS The origins of meta-analyses are important in understanding its strengths and weaknesses. They are described briefly. The usefulness and validity of a meta-analysis are OD ~ ~ tn -4- C^, N ~

necessarily limited by the quality and completeness of the literature base comprising the underlying studies and data. Consequently, publication bias is discussed since it could affect the completeness and representativeness of the analysis. Factors which might limit the comparability of individual studies are then discussed, including control of bias and confounding. Finally, some comments follow on the products of a meta-analysis and their interpretation. A number of criteria or questions must be considered when assessing a meta-analysis. They are parallel to the criteria for assessing a single study. Is the purpose of the analysis clear, and are the independent and dependent variables clearly defined and measurable? Are the inclusion criteria for individual studies specified and if so are they appropriate? Was publication bias avoided and were all the relevant articles identified and included? Were the individual studies reviewed for their control of bias and confounding? Are the individual studies sufficiently comparable to warrant a meta-analysis? Were the correct statistical procedures employed? Was the pooled result strong and significant? Do the confidence intervals and tests of significance indicate that the null hypothesis should be rejected? Is ihere a single large study which may dominate the analysis? Finally, were appropriate criteria for causality applied? aD ~ ~ ~ •S- CGi Gn -1

2.1 Origins of Meta-Analysis Glass, 1976, used the term meta-analysis as a descriptor for statistical analysis of a large collection of results from multiple studies. Meta-analytic methods were especially needed by social scientists who often have access to hundreds of studies on a particular topic. Meta- analysis was not intended to replace narrative reviews and study-tallying approaches, but to provide a means for systematic evaluation and integration of results from sets of studies. Glass et aL, 1981, described potential problems in meta-analyses. These include the use of noncomparable studies, the inclusion of low-quality studies, selection of nonrepresentative studies, nonindependent data, and the combination of studies of differing durations of observation. Parallel discussions of meta-analytic methods and potential problems in their application may be found in texts on meta-analysis by Hunter, 1982, Hedges and Olkin, 1985, Wolf, 1986, and Hunter and Schmidt, 1990. All of the above authors are proponents of properly conducted meta-analyses. Greenland, 1987, provided a thorough discussion of meta-analytic applications in epidemiology. 2.2 Publication Bias A quantitatively accurate meta-analytic summary of a research area must include all, or a representative sample of the relevant studies, including unpublished ones. A classic problem which violates this condition is availability bias, one form of which is publication bias. This problem is the meta-analytic analogue of selection bias in individual studies. Mann, 1990, has indicated that "It is widely believed - though exact proof remains elusive, - that tests with -6-

negative results_are much less likely to be submitted for publication, or even if submitted, to make it into print". Mann, 1990 at 478. The NRC, 1985, discussed the need to ensure inclusion of _relevant studies, which must be representative of the literature. The NRC Committee apparently considered this a difficult task because of the "file-drawer" problem of unpublished studies and the concomitant problem of publication bias against negative or neutral studies. NRC, 1985, at 218. The possible existence of publication bias is crucial to evaluate in a meta-analysis involving epidemiologic studies yielding low risks. Bias can result at the publication stage, at the literature searching stage, or at the stage of selection for inclusion. A bias will result if there are systematic differences related to likelihood of acceptance by various types of publications. Begg and Berlin, 1989, concluded that there is a bias against publishing studies that do not report positive association, and that this bias can have a "serious impact on meta- analyses." Dickersin, 1987, contacted 318 authors of published trials to ask if they had participated in unpublished studies, and "the 156 respondents reported 271 unpublished and 1,041 published trials." Negative results and failure to submit manuscripts, rather than rejection, were mainly responsible for non-publication. The authors concluded that an important publication bias e)osts regarding the publication of statistically significant trials. The editors of the Journal of the American Medical Association considered publication bias sufficiently important to include three articles on it in the issue of March 9, 1990 which was devoted to peer review. Dickersin, 1990, offers clear evidence of publication bias in (Z ~ ~ ~ ~.Z -7- ~ ~ ca