Philip Morris
OSHA Posthearing Submission
Fields
- Author
- Hubert, H.B.
- Type
- REPT, REPORT, OTHER
- ABST, ABSTRACT
- BIBL, BIBLIOGRAPHY
- CHAR, CHART, GRAPH, TABLE, MAPS
- QUES, QUESTIONNAIRE
- ABST, ABSTRACT
- Area
- MCALPIN,LOREEN/OFFICE
- Document File
- 2057837078/2057837447/Cal Epa Appendix III
- Litigation
- Ppla/Produced
- Characteristic
- MARG, MARGINALIA
- Site
- R635
- Named Organization
- American Public Health Assn
- Control Group
- Epa, Environmental Protection Agency
- Exsmoking Groups
- Lung Cancer Group
- Meridian Research
- Multiple Risk Intervention Trial
- Natl Center for Health Statistics
- OSHA, Occupational Safety & Health Administration
- Univ of Aukland
- Univ of Ca
- Who, World Health Org
- Adventist Health Smog Study
- American Cancer Society
- Control Group
- Author (Organization)
- Dept of Medicine + Health Research + Pol
- Stanford Univ Medical Center
- Named Person
- Agresti
- Akiba
- Alderson, M.R.
- Alexander, H.M.
- Andersen, B.
- Barrettconnor, E.
- Breslow
- Brown, K.G.
- Brownson
- Buffler
- Butler
- Butler, T.L.
- Cancer, J.
- Casper, M.
- Chamberlain, J.
- Chan
- Chang, X.L.
- Chee, E.
- Chopra, C.
- Cohort
- Comstock, G.E.
- Criqui, M.H.
- Croft, J.
- Davanzo, B.
- Day
- Dobson, A.J.
- Du, R.Y.
- Epidemiol, A.J.
- Fleiss, J.
- Fong, C.C.
- Fontham
- Franzosi, M.G.
- Fung
- Garfinkel
- Garland, C.
- Gerber, A.
- Gillis, C.R.
- Hames, C.G.
- Hawthorne, V.M.
- He, Y.
- Heller, R.F.
- Helsing, K.J.
- Hirayama, T.
- Hole
- Hole, D.J.
- Huang, J.Y.
- Humble, C.
- Hunt, S.C.
- Jackson, R.
- Janes, D.
- Jia, G.L.
- Kabat
- Kuller, L.H.
- Lam
- Lavecchia, C.
- Layard
- Lee, P.N.
- Levois
- Li, L.S.
- Li, L.X.
- Lloyd, D.M.
- Mantelhaenszel
- Martin
- Martin, M.J.
- Mengersen
- Muscat
- Ockene, J.K.
- Palmer, J.R.
- Qua, Q.L.
- Rosenberg, L.
- Sandler, D.P.
- Shapiro, S.
- Shimizu
- Shore, D.L.
- Sobue
- Stockwell
- Suarez, L.
- Svendsen, K.H.
- Thompson
- Tognoni, G.
- Tunstallpedoe
- Tweedie
- Tyroler, H.A.
- Warburton
- Williams, R.R.
- Wingard, D.L.
- Wynder
- Zheng, J.S.
- Akiba
- Master ID
- 2057837080/7446
Related Documents:- 2057837085 Increased Experimental Atherosclerosis in Cholesterol-Fed Rabbits Exposed to Passive Smoke: Taking Issue with Study Design and Methods of Analysis
- 2057837087-7093 Testimony in Response to OSHA's Identification of Cardiovascular Disease As A Hazard Resulting From Exposure to Environmental Tobacco Smoke in the Workplace
- 2057837107-7108 Comments on the Notice of Proposed Rulemaking Issued by the U.S. Occupational Safety and Health Administration Addressing Indoor Air Quality in Indoor Work Environments
- 2057837109-7152 A Critical Examination of the OSHA Ets Risk Assessment
- 2057837153-7182 An Alternative Explanation for the Apparent Elevated Relative Mortality and Morbidity Risks of Spouses and Other Family Members of Smokers Associated with Exposure to Environmental Tobacco Smoke
- 2057837186-7207 Curriculum Vitae Theodor D.Sterling
- 2057837218-7262 Cardiovascular Effects of Ets Exposure: Comments on Biological Plausibility of Proposed Mechanisms
- 2057837264-7278 Environmental Tobacco Smoke and Coronary Heart Syndromes: Absence of An Association
- 2057837374-7377 Ischemic Heart Disease and Spousal Smoking in the National Mortality Followback Survey
- 2057837379-7386 Publication Bias in the Environmental Tobacco Smoke / Coronary Heart Disease Epidemiologic Literature
- 2057837388-7389 Sidestream Cigarette Smoke and Arteriosclerosis
- 2057837419-7445 Biological Mechanisms Accounting for the Purported Relationship Between Environmental Tobacco Smoke Exposure and Adverse Cardiovascular Effects: A Response to Dr. Glantz
- Date Loaded
- 27 Jan 2000
- UCSF Legacy ID
- byl42d00
Document Images
OSHA POSTHEARING SUBMISSION
t
~ Submitted by:
Helen B. Hubert, Ph.D.
Senior Research Scientist
Department of Medicine and Health Research & Policy
Stanford University Medical Center
Stanford, California
~ August 29, 1995
~
~
I
I
I
~
~
~
I
~
2~9
I p.i'

OSHA POSTHEARING SUBMISSION
Helen B. Hubert. Ph.D.
Senior Research Scientist
Departments of Medicine and Health Research & Policy
Stanford Universiry Medical Center
STanford, California
I have obtained and reviewed a series of five deliverables prepared by Kenneth G. Brown. Ph.D..
Incorporated as a subcontractor to Meridian Research Inc. (Task Order No. 3, contract No. J-9-F-
1-0065), entitled "OSHA's Critical Evaluation of Epidemiological Studies on Cardiovascular
Disease Risk in Nonsmokers Exposed to Passive Tobacco Smoke." These deliverables were
submitted by Dr. E.own over a period of I yeir (November 1993- July 1994) to Ms. Debra Janes
at OSHA and include the following:
First Deliverable: (1) Updated risk analysis on passive tobacco smoke and lung
cancer in the form of revised material from the EPA document,
including a revised version of Chapter 5, additions and revisions to
Appendix A, and additions to the bibliography.
(2) Review and analysis of epidemiologic data on the association
between exposure to passive tobacco smoke in the workplace and
the risk of lung cancer.
Second Deliverable: A report on eptdamiologic studies on passive tobacco smoke and
heart disease prepared by Dr. Brown as a subcontractor to
Meridian Research Inc.
Third Deliverable: Study results on factors that may affect risk of heart disease.
Fourth Deliverable: Tier classification scheme for epidemiologic studies on heart
disease and passive tobacco smoke.
Fifth Deliverable: Tables and figures on heart disease.
Although I recognize these deliverables to be drafts of Dr. Brown's analyses on OSHA's behalf,
I have some major concerns regarding Dr. Brown's assessments of the epidemioiogic data on
exposure to "pagsive" tobacco smoke (PTS, also referred to in this analysis as environmental
tobacco smoke, ETS) and lung cancer and cardiovascular disease risks. In particular, I am
concerned about errors and discrepancies in the report, inappropriate conclusions drawn from the ~
data, and improper use of statistics. ~
Cdl
~
1 (~
~
GO
Io

2057837307
ww m

I
I
I
I
I
I
exposures to other occupational factors that may be associated with lung cancer or heart disease
risks. Furthetmore. because of widely recognized variability in dietary, lifestyle, socioeconomic
conditions, genetics, and other potentially disease-related factors, the relative'irtfluence of
confounders may vary in different countries or regions of the world.
Another weakness of the report on workplace exposure to environmental tobacco smoke and lung
cancer is that none of the published studies adjusted for spousal or other sources of
environmental tobacco smoke to derive the independent contribution of workplace environmental
tobacco smoke on risk. On page 1 I of the first deliverable, Dr. Brown acknowledges that "the
small sample sizes, the low power to detect an effect, the difficulty of assessing exposure in the
workplace, the mix of surrogates used for workplace exposure, and the probable correlation of
exposure to PTS at (work) with exposure in other environments, leave the evidence only
suggestive." The data are obviously of poor quality and Dr. Brown equivocates in his
conclusions on page I 1 by stating, "If PTS is a lung carcinogen, as concluded by EPA, then
exposure to PTS in the workplace adds to the toxic burden from exposure in other environments
to increase the risk of lung cancer." Such a statement is not, in effect, based on any hard, factual
data.
In comparison to Dr. Brown's assessment of the workplace lung cancer data, LeVois and Layard
(1994) performed a meta-analysis of the results of 12 epidemiologio studies that reported risks of
lung cancer and workplace exposure to environmental tobacco smoke. Sixteen relative risks
were provided in the papers identified; 9 of these 16 were above 1.0 and 7 were less than or equal
to 1.0. LeVois and Layard stated that:
"Of the 16 reported relative risks, we combined 15 by computing a weighted average of
their logarithms, me weights being the inverses of the variances of the log relative risks
(the Butler relative risk of 0.0 for males could not be included in the meta-analysis). The
summary relative risk for 12 of the 14 worldwide studies was 1.01, with 95% confidence
interval (0.92, 1.11). The summary relative risk for 7 of the 9 U.S. studies was 0.98 with
95% confidence interval (0.89, 1.09). Although we could not include the Brownson et al.
(1992) and Stockwell et al. (1992) studigs in the meta-analysis, since they did not report
relative risk estimates for workplace exposure, the authors' comments ... indicate that
including them would not change the conclusion that there is no epidemiologic evidence
of an association between workplace ETS exposure and lung cancer." (p. 312)
Whereas one could argue that any approach to combining data from these studies may be flawed,
use of LeVois and Layard's technique does not present any evidence for an effect of workplace
exposure to ETS on lung cancer risk. _
Inappropriate Conclusions Drawn from the Dose-Response Data
The issue of an alleged dose-response demonstrated by studies of spousal or workplace exposure ~
to environmental tobacco smoke and lung cancer or cardiovascular disease is one that has ~

I
I
I
I
I
I
I
I
I
I
I
I
t
I
I
I
bias and confounding inherent in that design; that is. such a[meta-analysis] will likely be
significant [at the conventional 5% level, irrespective of whether a one-sided or two-sided test is
used] simply due to artifact" (p. 3 12).
Statistical Power of Studies
Most of the studies of lung cancer and cardiovascular disease examined by Dr. Brown have
insufficient power to detect relative risks of 1.5 or less as shown in Tables 5-9 (first deliverable
on spousal exposure to environmental tobacco smoke and lung cancer risk) and Table 6 (second
deliverable on spousal exposure to environmental tobacco smoke and cardiovascular disease
risk). Rigorous epidemiologic studies should be designed with sample sizes sufficient to achieve
80-90% power to detect the. risk in question, given the expected disease rate or exposure in the
population (for cohort or case-control studies) (Breslow and Day 1987). Since most of the data
c~ted by Dr. Brov.n were derived from s:udies that were not specifically designed to examine the
role of environmental tobacco smoke on these particular health endpoints, power has been
seriously diminished. The major drawback is that without sufficient power and with such low
risks described, chance cannot be ruled out as an explanation for many of the elevated risks
found. Furthermore, power may actually be more seriously compromised than is reported in the
tables if, in fact, risks for both lung cancer and cardiovascular disease are lower than 1.5 as the
majority of the data suggest.
In Table 5-9 of the first deliverable on spousal exposure to environmental tobacco smoke and
lung cancer, only 3 out of 12 U.S. studies of lung cancer (Brownson et al. 1992, Fontham et al.
1991, and Garflnkel et al. 1985) for which power could be calculated show sufficient (>80%)
power to detect a risk of 1.5. These are considered by Brown to be Tier 1 or Tier 2 studies. Only
one (Fontham et al. 1991) shows significantly elevated risk (OR=1.28), but only using a 90%
(instead of 95%) confidence interval (1.0.3, 1.50). Brownson et al. 1992 shows an OR=0.91 (CI=
0.75, 1.05) and Garfinkel et al. 1985 shows an RR=1.16 (CI=0.89, 1.52). Of the remaining non-
U.S. studies, only Wu-Williams and Samet (1990) has sufficient power, reporting an OR=0.78
(CI=0.63, 0.96).
Greater power is also indicative of greater stability of estimates and reliability, if the study
methods and analyses are appropriate and have ruled out bias and confounding. With regard to
power, on page 5 of the second deliverable on spousal exposure to environmental tobacco smoke
and cardiovascular disease risk, Dr. Brown states "If power is small, the lack of significance may
be due to small sample size instead of due to ,io effect; if the power is large, so that failure to
detect a real effect is unlikely, then lack of significance is more informative." However, Dr.
Brown often fails to acknowledge the importance of nonsignificant findings in instances where
power is large. "
12

3. Important biases addressed'
(al Selection bias
1
1
r
or YES Population-based (geographically defined) study with minimal nonresponse,
and/or drop-out rate.
O or NO Evidence of differential response or drop-out rate in cases versus controls/exposed
versus unexposed or inadequate control group selected.
:1 or BLANK Insufficient information to determine bias.
(b) Information bias
or YES No evidence of bias when validation of smoking and ETS exposure undertaken in
cases and controls, blinding of interviewers to case/control status, and proxy
response rate low and nondifferential in cases and controls.
O or NO Proxy response greater in cases than controls or no validation of smoking and
ETS exposure or unblinded interviewers.
a or BLANK Insufficient information to determine bias.
4. Are the data internally consistent? .
or YES Evidence of crude or significant dose-response, subgroup results that go in the
same direction (e.g., males v. females, blacks v. whites, home v. work), and
exposure preceding outcome. . _
O or NO No evidence of dose-response, subgroup results that go in a different direction, or
unclear temporal relationship.
O or BLANK Insufficient_data to determine dos.e-response, subgroup results, or temporality.
The relative quantities of open circles and blank spaces in Table I suggest that the studies on
cardiovascular disease are not sufficiently reliable to make conclusions with regard to the impact
of environmental tobacco smoke exposure on cardiovascular disease risk. Of greatest
significance is the fact that studies that ruled out chance as an explanation for elevated risks
were, without exception, unable to rule out systematic study bias or confounding as explanations
for the fmdings. My evaluation as to whether the data collected in each study are reliable to
assess the risk of cardiovascular disease associated with exposure to environmental tobacco
smoke exposure is described below.

Additionally, as an expert in the epidemiology of cardiovascular disease, I have performed my
own assessment of all of the available epidemiologic studies on environmental tobacco smoke
exposure and cardiovascular disease, and it is my belief that these studies do not support a
conclusion that the reported associations are real. Specifically, as my analysis shows, the 13
published studies, 2 abstracts, and 1 letter that were available to Dr. Brown at the time of his
report are not reliable because they do not adequately exclude important sources of bias and
confounding nor rule out chance as an explanation for the fmdings. Whereas the copy of the
report I reviewed did not contain Dr. Brown's conclusions on the potential effect of
environmental tobacco smoke exposure on cardiovascular disease risk (my copy of the
deliverable ends on page 7 and the remainder of the document was not available), Dr. Brown's
own critique of the epidemiologic studies similarly points out that the data are inconclusive to
establish an effect and rule out bias and confounding.
Based upon the rndings of my reassessment )f the cardiovascular disease studies and a
recognition that the methodological criticisms apply as well to $TS studies involving other
disease endpoints, I am confident that a similar unbiased reassessment of all the available lung
cancer epidemiology studies would raise similar questions about Dr. Brown's conclusions.
My opinions of Dr. Brown's assessments of lung cancer and cardiovascular disease risks, and
the results of my independent review of the cardiovascular data form the basis for this
posthearing submission to OSHA.
KENNETH BROWN'S APPROACH TO ANALYZING EPIDEMIOLOGIC DATA
The series of deliverables details Dr. Brown's assessment of the potential association between
spousal and workplace exposure to passive tobacco smoke and lung cancer and cardiovascular
disease risks. In general, I have several concerns regarding his approach including reliance on
studies of limited epidemiologic quality, inappropriate conclusions drawn from his analysis of
the available epidemiologic data and of dose-response, limited evaluation of temporality, limited
evaluation of confounding and misclassification, limited evaluation of study heterogeneity,
improper use. of statistics, and use of subjective tier classifications. In addition, I was able to
identify several errors, discrepancies, and misinterpretations in the deliverables that could
compromise the validity of the reports' conclusions.
Reliance on Studies of Limited Epidemiologic Quality
Dr. Brown readily admits to the limited quality of the epidemiologic studies that evaluated
spousal and workplace exposure to environmental tobacco smoke and lung cancer and heart
disease risks. His comments can be found in the Appendices accompanying each deliverable.
For example, among the lung cancer studies reviewed by Dr. Brown to assess workplace.
exposure (first deliverable), he comments that the Akiba study "precludes drawing any firm
conclusion regarding workplace PTS and lung cancer" (App-1); the Buffler study has "little

t
'
l
I
I
I
In Table 2 of the first deliverable, relative risks presented are not comparable since they
do not incorporate uniform definitions of exposure or study population. For example, in
the analysis of workplace exposure and lung cancer, the crude RRs for the Kabat and
Wynder and Lee studies are calculated for both genders combined rather than separately
for males and females, whereas odds ratios, by gender, are in fact presented in the table
footnotes and/or in the appendices. In the Kabat study, these odds ratios by gender show
a statistically significant increased risk for neversmoking males exposed to tobacco
smoke at work (2.57) but not for females (0.68). In the Lee study, the crude ORs for
nonsmoking males and females separately are 1.6 and 0.63, respectively, neither
statistically significant. As another example, in the Akiba study, exposure is defined as
outside the home vs. housewife, whereas other studies define exposure differently.
On p. 4 of the analysis of workplace exposure and lung cancer, Dr. Brown states that
"Most. (9 of 12) [crude] values are above 1.0, two are at or slightly below 1.0, and only a
single value falls well below the null." I believe that this overstates the strength of the
data. According to his Table 2, crude RRs are reported for 13 studies, 4 of the 13 values
fail below 1.0, including CHAN 0.77, GARF 0.93, LEE 0.99, and BUTL 0.6. Six are
below 1.5, including AKIB 1.08, FONT 1.12, KABA 1.11, KOO 1.36, SVEN 1.26, and
WU WI 1.22. Only LAMW is statistically significant.
4. Table 3 of the analysis of workplace exposure and lung cancer, the adjusted risk reported
for BUTL in Table 2, 1.06 does not match that reported for BUTL in Table 3, 1.08.
5. On p. 6 of the analysis of workplace exposure and lung cancer, second paragraph, Brown
does not report whether these odds ratios are crude or adjusted. This is an important
omission since Dr. Brown notes on p. 5-23 of the first deliverable, a revision of
Chapter 5, that'`an adjusted RR is considered preferable to a crude RR unless the study
review .... indicates a problem with the adjustment procedure.... our choice of RR is the
smaller of the crude and adjusted values in ... studies providing both estimates."
6. On p. 8 of the analysis of workplace exposure and lung cancer, fourth line, Dr. Brown
omits BROW from the list of elevated ORs. Again, Dr. Brown does not specify in the
text which of these ORs are crude and which are adjusted.
7. Despite the observation that 90% confidence limits were calculated for the analysis of
lung cancer and passive tobacco smoke exposure, on p. 6 of the second deliverable,
passive tobacco smoke and heart disease, and in the accompanying figures, 95%
confidence limits were reported instead.
8. On p. 6 of the second deliverable, passive tobacco smoke and heart disease, despite Dr.
Brown's observation of the contrast of estimates of RR for males and females exposed to
PTS in the home, he suggests no explanation for this observation.
14

employment status was used as a sutroeate for workplace exposure) are consistent with the
hypothesis that workplace exposure to environmental tobacco smoke increases lung cancer risk.
On page 6, of the first deliverable on workplace exposure and lung cancer risk. Dr. Brown
inappropriately deletes a quite accurate statement, that "workplace exposure to other
carcinogenic substances could arguably give rise to the same observation." Employment status is
not a convincing measure of environmental tobacco smoke exposure since other risk factors may
also be associated with working outside the home.
1
I
I
I
Inappropriate Conclusiops I)rawn Erom the Analyses
I have identified several examples of instances in which Dr. Brown's conclusions are not
reflective of the data he presents. For example, in his analysis of the spousal lung cancer risk by
tier and country, he states on page 5-71, "It is concluded that the association of ETS and lung
cancer observed rrom the analysis of 33 epidemiologic studies in eight different countries is not
due to chance alone and is not attributable to bias or confounding." However, even if one is
prepared to accept the tier ranking approach and discount the Chinese studies, Dr. Brown's
statements are inappropriate since he fails to acknowledge that pooling of first and second best
ranked studies produces no statistically significant effect of environmental tobacco smoke on
lung cancer in the United States or Europe. Only in Greece, Hong Kong, and Japan are results
significant and consistent over tier pooling. The fact that these are not U.S. studies renders them
of limited utility, given different cultural differences and smoking patterns.
In the first deliverable on workplace exposure to environmental tobacco smoke and lung cancei
risk, four studies (Brownson et al. 1987, Chan and Fung 1982, Svendsen et al. 1989, and Lam
1985) collected data on workplace exposure but presented only an index of exposure to all
sources. Therefore; it is incorrect for Dr. Brown to deduce on page 6, "The observation of
elevated ORs in 3 out of 4 studies using a measure of total PTS that incorporates workplace
exposure is consistent with the hypothesis that workplace PTS contributes to risk of lung
cancer ... Lacking analyses of workplace exposure :.. the observed association is implicit rather
than explicit " There is really no way of determining anything about the relationship of
workplace exposure to lung cancer from these data. Workplace exposure could have shown no
elevated risk and the same results could have been achieved given the exposure classifications in _.
these studies (i.e., 4 or more hours per day spent in the presence of a smoker; exposure at home
or at work). In addition, observing that three of four studies reported elevated risks, only one of
which was statistically significant, also does not rule out chance in establishing an association.
This faulty reasoning is apparent throughout the documents on spousal and workplace exposures
to environmental tobacco smoke and lung cancer and cardiovascular disease.
On page 10 of the first deliverable on workplace exposure to environmental tobacco smoke an
lung cancer risk, Dr. Brown points out the inconsistencies of the U.S. data on workplace
exposure compared to Asia and Europe and states: "there is no reason to think that if workplace
exposure is a risk factbr for lung cancer, it would be so only in European and Asian populations: "
Consideration should also be given to regional variations in working conditions and potential
4
I

(1994) appropriately concluded that each smoking study is confounded by uncontrolled
influences and produces a biased estimate of ETS effect suggesting that weak spousal smoking-
lung cancer risk elevations in the individual studies may well be the result of artifact alone.
I
I
I
I
I
I
I
I
I
Use of One-tailed Statistical Tests
Dr. Brown's continued reliance on one-tailed tests to assess the relationship between
environmental tobacco smoke exposure and disease is improper. For example, on page 5-2 of
the f rst deliverable on spousal exposure to environmental tobacco smoke and lung cancer risk,
the revision to Chapter 5 states that:
"Throughout this chapter, one-tailed tests of significance (p=0.05) are used, which
increases the statistical ability (power)'to detect an effect. The 90% confidence intervals
used for the analyses performed are consistent with the use of the one-tailed test. The
justification for this usage is based on the a priori hypothesis (from the plausibility of a
lung cancer effect documented in Chaptets 3 and 4) that a positive association exists
between exposure to ETS and lung cancer."
However, the data contained in Chapters 3 and 4 of the EPA report concern levels of exposure to
environmental tobacco smoke (Chapter 3) and discussions of lung cancer in active smoking and
laboratory animals (Chapter 4), and, therefore, cannot be relied on as the basis of a decision to
utilize a one-tailed test. Second, Dr. Brown's rationale for the use of the one-tailed test (90%
confidence limit) for an overall estimate in a meta-analysis, rather than the conventional 95%
confidence limit, is not supported by statistical or epidemiological theory or methodology in this
instance.
Statistical theory grew out of experimental science, which measures variables or responses. The
typical test of significance for measured responses is a t-test, which tests differences in sample
means. Two-sided (two-tailed) tests of significance are generally employed in the analysis of
simple contrasts in experimental studies. However, one-sided (one-tailed) tests of significance
(in which the investigator only considers deviations from the null hypothesis in one direction,
ignoring deviations in the other direction) may be appropriate when the investigator either knows
in advance that a treatment will affect outcome in only one direction or if the investigator has no
interest in the outcome of a treatment unless it is superior to an existing treatment (such as in
testing the efficacy of a new drug).
In contrast to experimental science, epidemiolo;ical and observational studies are generally
concerned with categorical or discrete data. Standard practice in these studies is to calculate
odds ratios or relative risks. Statistical significance in epidemiologic studies is normally
assessed using Chi-square tests, or equivalent procedures, which generate a confidence interval
around the odds ratio or relative risk. Although a Chi-square test uses only one-tail of the Chi-
square distribution, the test is in fact two sided, since a significant Chi-square can be generated
by deviations from expected values in either direction.
10
