Jump to:

Council for Tobacco Research

Some Statistical Observations on A Cooperative Study of Human Pulmonary Pathology III [When Probability Is Doubtful the Use of Variable Tables to Calculate Results Is the Best Way to Make A Precise and Accurate Answer for Sufficient Range of Pathological Conditions]

Date: 13 Dec 0000 (est.)
Length: 8 pages
CTRMN003457-CTRMN003464
Jump To Images
snapshot_ctr CTRMN003457_3464

Abstract

MAR

Fields

Depository Date
25 Sep 1995
Master ID
Ctrmn00000667-6967

Related Documents:
Author
Burke, M.H., Tirc
Wilson, E.B., Tirc
Request
118
Type
SCIENTIFIC REPORT
BIBLIOGRAPHY
Box
003
UCSF Legacy ID
foq30a00

Document Images

Text Control

Highlight Text:

OCR Text Alignment:

Image Control

Image Rotation:

Image Size:

Page 1: foq30a00
SOME STATISTICkL OBSFRVATIONS ON A COOPERATIVE STUDY- OF HiJMAN PUIMNARY PAZHOLOCY. II I ,/By Edwin B. Wilson and l4ary H. Burke Office of Naval Research, Boston, Hassachuaetts, and Tobacco Industry Research Committee, New York The difference between precision and accuracy could hardly be better stated than it was by Simon ?ievcomb in summing up his determination of the velocity of light. (1) He had given his value of the velocity of light in vacuo as 299,860 km. per second s•nd Michelson's subsecuent value as 299,853, and then stated: "So far as could be determined from the discordance of the separate measures, the mean error of Neuconb's result would be less than 10 km. But making allowance for the various sources of systematic error the actual probable error va.s esti.mated at ± 30 km." Precision is a mathe.aatical concept attaching to a series of observa- tions; accuracy is a scientific concept taking into account the probatie reproduci- bility of the result by other equally coapetent scientists. Precision estimates gu^d against one's sa.~-p-e beir.a too small, but not at all against its being biased. If the sacple is srall eno•a6:., the "sappling error" aay be large enough to cover up any likely bias; if the sa-ple is large, it is very likely to be biased enough, in one way or another, to maite the accuracy less than the precision. This nev:oW'^ believed was the case vith his determination of the velocitq of light, and he was a good enough scientist to desire to claim no more than he believed to be true. (2) The distinction between precision and accuracy, between accidental anfl systematic errors is old enough in the so-called exact sciences to have found a place long ago in the leading dictionaries. (3) It is, however, not eaphasized as much as it should be in the textbooks on statistics. A1 though a11 sciences advance by introducing better definitions s•nd better methods, and thus reduce guesswork, estimate and personal opinion to narrower 1imits, none has succeeded In eliminating CTR. / CONF1DcNUAL: ;,:IKNESOTA TOBACCO LITIGATION 1435 C , ~ R ~`~~ ~`'°~ ~ ~ ~'° ~".~ .z'~
Page 2: foq30a00
- 2 - them, or is likely to, and least of all those whose subjects require the use of statistics in complicated situations in which mep.surement and even claasificatior. is far from precise, where causation is not knovn but sought, and error too often has social implications of right or wrong insteaQ of merely its scientific significance. Nhat sade the exact sciences exact vaa the careful attention to definition, classification, measurement and the reduction of observations, and it is this that is making the inexact sciences less ine.xact. Our first paper discussed a.nd illustratedthe extent to which the readings of the pathologists still differed after some conferences devoted to discussing definitions. It vas clear that the reaaining differences were sufficient to make comparability of their respective findings unsafe unless information was obtained which vould'btanda..-dize" the results. In our second paper the meanB and method of standardization were set forth. (1:) In the present note we return to the problem of the further exan.:nation of the readings of the forty slides using a simple type of variance a.nalysis. (5) Thble 1 gives the data. (6) Clearly the ratings depend in part on the slides read as is seen by the variation in the coluans and in part on the patholog_st doing the reading as is shovn by th• variation in the rows. If Fps represents the rating of the slide s, with s= 1, 2, .... 40, by the pathologist p, with p= 1, 2, .... 12, it is obvious that Fps depends in part on P and in part on s. Ii we average the ratings of the twelve patho=ogists on slide s we get ms, and if we average the ratings of each pathologist on the forty slides we get the mean mp. It is a natural and indeed, the uaual, assumption to consider the mean as the best value we can get for a variable and to consider deviations from the sxan as errors in the statistical sense (not as mistakes). Thus Fps - ms, p=1, 2, .... 12, vill be the 12 errors oi the 12 pathologists on slide s, and Fps - mp, s=1, 2, .... L0, vill be the 1+0 errors (deviations) of the 40 slides according to pathologist p. CTR.t 143G CONFIDENTIAL: MINNESOTA TOBACCO LtTIGATION ~° ,~- r~ ~ l~ 4~ .,° s°C ~ t E-.~
Page 3: foq30a00
3 'Ihble 1 I Line A B C D E F G B I 3 K L S S' V 1 2 3 4 5• 6 7 8 9 10 1 1 0 1 1 1 2 2 2 1 0 0 1 2 1 2 2 2 2 2 0 0 0 1 1 1 2 2 1 1 0 3 1 1 1 1 1 1 3 3 0 0 1 1 2 1 2 2 1 3 _ 1 1 3 1 1 1 1 2 1 1 0 0 0 1 1 1 1 2 1 1 0 0 0 1 1 1 0 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 0 1 1 0 0 0 0 0 1 0 0 0 1 3 6 7 10 12 13 14 15 15 16 3 12 13 12 14 15 22 27 25 30 27 108 107 44 24 11 68 99 75 104 11 12 13 14 15 16 17 18 19 20 1 2 2 1 2 2 2 0 2 2 2 2 2 2 2 2 2 2 2 ' 2 1 3 1 2 1 2 1 2 2 2 3 1 3 2 3 2 3 2 3 2 0 1 3 2 2 2 2 2 2 2 1 1 1 2 2 2 1 1 1 2 2 2 1 .2 2 2 2 2 2 2 0 0 1 0 2 1 2 2 1 2 2-1 0 3 1 1 2 0 1 1 1 2 1 1 2 2 0 2 2 0 1 1 1 2 1 2 1 2 2 2 2 1 0 2 0 0 2 2 2 2 17 17 17 19 19 20 20 21 21 22 33 35 33 37 37 38 38 41 43 44 107 131 107 83 83 56 56 51 75 44 21 22 23 24 25 26 27 26 29 30 1 2 2 2 2 2 2 2 2 2 1 2 3 2 2 2 2 2 2 3 2 2 2 3 1 2 2 2 2 3 3 2 3 1 3 2 2 2 2 3 2 2 3 1 3 2 2 2 2 3 2 1 1 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 3 2 2 1 2 2 1 2 2 2 1 2 2 1 2 1 1 2 2 1 2 2 2 2 2 2 2 2 2 1 3 1 2 2 2 1 2 1 2 2 0 2 2 2 2 2 2 2 2 2 2 22 22 23 23 23 23 23 21 24 25 4-4 42 51 47 49 45 45 48 48 65 U 20 83 35 59 11 11 o 0 155 31 32 33 34 35 36 37 38 39 40 2 2 2 2 2- 1 2 4 4 4 2 2 2 2 2 3 2 4 4 4 2 2 2 2 3 3 3 3 4 4 3 3 2 3 3 3 3 4 3 3 2 3 3 2 3 4 3 4 4 4 2 2 2 2 2 4 3 3 4 4 2 2 2 2 2 1 3 4 4 4 2 3 2 3 2 2 3 4 4 4 2 2 2 1 2 2 2 2 2 3 1 2 3 3 4 3 4 4 4 4 2 2 3 2 1 2 3 3 4 4 2 2 2 2 2 1 3 4 4 4 25 26 26 26 27 27 34 44 47 47 53 60 58 56 65 75 98 1a- 185 185 1-1 41: 20 20 51 171 20 32 11 11 - - S 72 75 92 655 72 74 63 60 bti 65 61 565 2037 9 22 6 s' 158 194 177 244 223 160 170 147 132 150 137 145 2037 V 1136 1036 1455 1.296 1695 1216 1324 1911 1680 1904 1255 2079 17987 1600 times the variance in the column). The data bave been given,for the tvelve readings on the forty slides vitb the cocputation form used for checking, viz: The column (or rov) ms.rked S contains the sum of the ratings in the rovs (or columns), that marked S' contains the sum of the squares of the ratings. The overall sum checks at 865, an~ the sum of squares at 2037. The column (or rov) mdrked V. is computed as 12 S' - S (or 40 S' - S2) so as to remain in whole numbera and is therefore 144 times the variance in the row (or, respectively, 0 r.?,1 1437 CONFIDENTIAL: MINNESOTA ~ TOBACCO LITIGATION M ~~ H(,.~~ ~"~ f'~ ''a'~,,~, te= 9
Page 4: foq30a00
-k- The mean H of all the ps • 480 ratings Fps, T.be mean of the 40 values ms a.•,d the mean of the 12 values r.p all have to be equal. We can "correct" each rating Fps to the means as and mp by vriting: Eps `Fps - H - (me - M) - (mp - M) • Fps - ms - mp + M. That is to say, FPa represents the accidental or residual variation in the rating aSter taking out the systematic parts due to the rating habits of the pathologists ar.d to the pathology of the slides. It is a simple theorem in elementary algebra that the total variance of F(i.e., the standard deviation squared of Fps - M) is ecual to the variance of ms plus the variance of mp plus the variance of the residuals Eps. For the data of Table 1 these variances are: a-2Fps = 0.9962, c-2ms = 0.6023, c-2mp = 0.0595, 0-2F-ps = 0.3345 The sum of the last three is the first, and if ve are villing to use variances to est:rate the relative contributions of each of the last three to the total vhich is the first, ve find that 60.5%, 6.0% and 33.5% respectively of the total variance is d~e to the differences in the sl;des, the differences in the pathologists a_-ui to a:1 other sources. (7) The corresponding standard deviations are: d-Fps = 0.9951, 6's.b = 0.7760, a-Yp = 0.2438, rEpS = 0.5783 wcveti•er, , the standard deviations have not the a.dditive property that the vasiances have ar,d there is no accepted vay of distributiag the first among the last three. To obtain the variaace of the 480 residuslF Eps it is not necessary to find them individually; it is sufficient to find the variances of the original distribution, of the means of the rows, a.nd of the means of the column s, and use the su=ation theoren. However, there Is some interest in looking at the residuals and they are given in Table 2. The variances in the rovs may be taken as measures of disagreement and hence of the difficulty of reading the slides. Although the average of these variances in Table 2 at L8.2 is less by about one- seventh than the 56.7 in Table l) there are about one-third of the slides for I CjR / 1438 ,C0NFIDENTIAL: MINNGSOTA "%TOBACCO LIT/OATION CTR VIN 0034° 60
Page 5: foq30a00
5 Table 2 Line A B C D B F C H I J K L V 1 9 - 6 - 3•9 - 9 - 6.9 9 - 3.6 - 0.3 0.6 - 0.6 11.1 0.3 39.9 2 6 - 9 - 6.9 24 - 9.9 6 - 6.6 - 3.3 - 2.4 - 3.6 8.1 - 2.7 85.1 3 - 7 2 - 7.9 - 1 1.1 29 - 7.6 - k.3 - 3.4 - 4.6 7.1 3.7 94.3 k 2 11 1.1 - 4 - 1.9 2 1.4 4.7 - 6.4 - 7.6 4.1 - 6.7 27.9 5 0 - 3 - 0.9 - 6 8.1 0 - 0.6 2.7 3.6 2.4 2.1 - 8.7 16.2 6 - 1 8 - 1.9 - 7 - 4•9 - i - 1.6 1.7 2.6 1.4 1.1 2.3 13.6 7 10 7 9.1 - 8 6.1 - 2 - 2.6 -11.3 1.6 G.4 0.1 -10.7 49.1 8 9 6 8.1 - 9 5.1 9 8.4 - 0.3 .0.6 -12.6 -12.9 -11.7 75.3 9 9 6 - 3.9 15 - 6.9 - 3 - 3.6 - 0.3 0.6 - 0.6 - 0.9 -11.7 47.1 10 -L 5 4.9 14 16.1 - 4 4~.6_ i._3 - 0.4 -13.6 - 1.9 0.7 62.L 1 13 9 -20 - 5 6 4 -i4 3 4 9 10 6 2 9 10 105 t 1 12 - 5 7 4 - 5•9 18.1 -i.1 . - 8.9 - 5 . 6.4 -14.3 . . -13.4 21.4 - . - 2.9 .3 - 1.7 . 126.C 13 7 4 - 5.9 13 15•1 - 5 - 5.6 - 2.3 - l.li - 2.6 - 2.9 -13.7 63.6 14 - 7 2 4.1 - 1 1.1 5 4.4 -16.3 8.6 -16.6 7.1 8.3 70.9 15 5 2 - 7.9 11 1.1 5 4.4 7.7 - 3.4 - 4.6 - 4.9 -15.7 57.7 16 4 1 3.1 - 2 0.1 4 3•4 - 5.3 - 4.4 6.4 6.1 -16.7 38.6 17 4 1 - 8.9 10 0.1 - 8 3.4 6.7 - 4.4 - 5.6 - 5•9 7•3 38.0 18 -21 0 2.1 - 3 - 0.9 - 9 2.4 5.7 6.6 5.4 5.1 6.3 59.-' 19 3 0. 2.1 9 - 0.9 - 9 2.4 - 6.3 -17.4 5.4 5.1 6.3 51.6 20 2 - 1 1.1 1+ - 1.9 2 1.4 4.7 5.6 -19.6 4.1 5.3 42.c 21 -10 -13 1.1 - 1.9 2 1.4 .7 5.6 4.4 - 7.9 5•3 42.3 22 2 - 1 1.1 - 4 - 1.9 -10 1.4 4.7 - 6.4 4.4 4.1 5.3 21:J 23 1 10 0.1 7 9.1 -u o.4 - 8.3 - 7.4 - 8.6 3.1 L.3 48.= 2L 1 - 2' 12.1 -17 -10.9 1 0.4 3.7 4.6 3.4 3.1 4.3 61.5 25 1 - 2 -11.9 7 9.1 1 0.4 3.7 - 7.4 3.4 - 8.9 4.3 38.1 26 1 - 2 0.1 - 5 - 2.9 1 0.4 - 8.3 4.6 3.4 3.1 4.3 1L.1 27 1 - 2 0.1 - 5 - 2.9 1 0.4 3.7 4.6 3.4 - 8.9 4.3 +,s - 28 a - 3 - 0.9 - 6 - 3.9 0 - o.E 2.7 3.6 2.4 2.1 3.3 ~ 4 29 0 - 3 - 0•9 - 6 - 3.9 0 - o.6 2.7 3.6 2.4 2.1 3.3 8.6 30 -: 8 10.1 5 7.1 -13 1o.L -10.3 - 9.1+ 13.4 -22.9 2.3 11c.E 3; - ~ - ~ - 1.9 5 -.y - 1 - l.b 1.7 2. 1.4 1.1 2.s 7.c 32 - 2 - 5 - 2.9 4 6.1 - 2 - 2.6 12.7 1.6 -11.6 0.1 1.3 33•`. 33 - 2 - 5 - 2.9 - 8 6.1 - 2 - 2.6 0.7 1.6 0.4 12.1 1.3 2; '_ 3L - 2 - 5 - 2.9 4 - 5.9 - 2 - 2.6 12.7 1.6 0.4 0.1 1.3 22.1 35 - 3 - 6 6 1 3 5.1 - 3 - 3.6 - 0 3 0 6 11.4 -12.9 0.3 35 3~ .15 6 . 6.1 3 17.1 21 . -15.6 - 0.3 . -11.4 - 0.6 - 0.9 -11.7 . 131.1. . 17 -10 -13 1.1 - 4 - 1.9 2 1.4 L.7 5.6 4.4 4.1 5.3 3L.= 38 4 1 - 8.9 - 2 0.1 - 8 3.4 6.7 7.6 - 5.6 - 5•9 7.3 33•2 39 ao 1 1 - 2 2 0.1 0.1 -17 -17 - 2.9 - 2.9 1 1 0.4 3.7 0.4 3.7 4.6 3.4 4.6 3.4 3.1 3.1 4.3 4.3 31.5 31 .r v 40.2 30.7 38.2 86.7, 58.2 59.4 •20.6 46.3 37.8 62.5 46.5 50.7 48.2 In the Table are given the 480 residuals ~s, each a~ultiplied by 12 and the product dicinishefl by 1/40 = 0.025 to keep the calculatSona simpler. The rows, therefore, have the aun -0.3 and the coluians -1.0 instead of zeros. The column and the rov marked V give the respective variances in the rows and columns reduced to one decimal place, the mean of each being 48.2, vhich gives 0.3347 on division by 144 and thus agrees as well as can be expected v:th the stated value 0.3345. GTRA 1439 CONF1DflNTlAL: M1NNESOTA TOBACCO LITIOATION - - CTR NH 00"3461
Page 6: foq30a00
6 which the variance in Table 1 is aomewhat less than in Table 2. This means that if one tries to indicate the serial order of diLficulty of reading the slides by the order of the variances, he vill have first to decide vhether he vill use the crude variances or those adjusted to the mean ratings af the diiferent readers. The original ratings were distributed essentially normally in five groups; the adjusted residuals are in nearly 160 groups wahing the distribution more continuous. The standard deviation of the scaled-up ..alues in Table 2 is 6.94. Expreaeed in terms of the standard deviation as a unit the diatribution of the E's is compAred in Table 3 with that of a normal one. It is seen that the E's are not normally distributed. There are too many deviation.a greater than 3 Q"', whether positive or negative, too few in the middle ranges on both sides, and too many small positive ones. Thie departure from bormality is not unusual, but it does"mehe estimates of probability somewhat doubtful; ve shall proceed descriptively. First we remark that were we to use the variance tables, it vould, be necessary to go over from variances to mean se,uares obtained by div:di~g the su= of souares by the degrees of freedom (i^stead of by the numbers irn the sa....^~ples ); these are respectively 479, 11, 39 and 429 f or the original data, the means of the slides, the means of the patholcgists, and the residuals. The mearn squares ;.hs become respectively 0.9983, 0.6177, 0.06k8 and 0.3742; the corresponding squere roots are 0.9991, 0.7859, 0.255 and 0.6117. If we use the last as the presumed best estimate of the standard deviation of the residual "random" variation, a re.ndom sample of 40 vill have its mean subject to a standard deviation of 0.0967 and as the actual standard deviation of the means of the 12 pathologists is 0.255, which is 2.6k'as much, we may infer that the differences between those means is probably decidedly significant. Similarly, the mean of 12 random elements would have a standard deviation of 0.1766, and the deviation of the slides is 4.45 times that amount, vhich merely confirme us in the belief that the set of slides had a quite sufficient range of pathological conditiona CYSA 1440 CONFIDENTIAL: MINNESOTA TOBACCO LITIBATION ,r^^, - R e.,_i[l 00,34162
Page 7: foq30a00
- 7 - Tlnble 3 Scale <-3 _ -3 to -2 -2 to -1 -1 to .5 .5 to -.2 -.2 to 0 E's 3 14 53 5'l 55 34 Normal 0.6 10.3 65 72 54 38 Scale 0 to .2 .2 to . . to 1 1 to 2 2_ to 3 ~12 E'e 63 61 79 51 6 4 Normal 38 54 72 65 10.3 0.6 (1) Encyclopedia Britannica, 11th Ed., Vol. 16, p. 625. (2) Actually Neticomb was not conservative enough. Hichelson'e 1926 value was 299,796 1 1. See Encyclopedia Britannica, 14th Ed., Vol. 23, p. 37. Michelson let Nevcomb's atateaent stand and made no eatiasate of his own accuracy; the ± 1 was his precision. Many of Michelson's collaborators and others in the next fifteen years mnfle determinations which ran around 20 1om/eec less than his, and R. T. Birge, aa undoubted authority in precision s.nnlysis, reviewing all the evidence, in 1941 set the value at 299,776 f k. More recently DuMond and Cohen have set the figure 299,792.9 t 0.8, which is back close to Michelson's of 1926, but statistically inconsistent with Birge's. The long history of the determinations of this import-ant constant gives little ground for belief that precision estimates are safe guides to scient:`ic accuracy. (3) 9. Math." See, for exa...^ple, Webster's Nev International Dictionary under "Error, (4) Tnese Proceedings, 43, 1073-78, 1957, and 45, 389-93, 1959. (5) T'ne type vhich ve sha11 use is only the simplest, the one which is the first step in the analysis of sets of grades assigned by different readers to the answers of a group of students upon examinations, especially when the questions call for the essay type of a.nsver. We do not have here to examine variation in a complicated system where one computes the trend of a presumed effect on a variable, or some of the variables, which represent presumed causes; but it is necessary in that case also to go on to some sort of variance analysis before one can aasign even provisionally the putative a.mounts of causation to the individual variables or the pairs jointly. An'illustration of this can be found in E. L. Thorndike's Your City, Fiarcourt and Brace, 1939, 20+pp• In such cases it is important to find how mm.ich of the total variance of the "effect" is accounted for by the "causal" variables used and how much remnins unaccounted for and to verify that the latter is but a small fraction of the whole. CTR4 1441 CONFIDENTIAL: MINNESOTA TOBACCO LITIGATION C I „ R `f~ ~`~ 0 0 3 4 , '23
Page 8: foq30a00
-6- (6) We follow the classification of our second paper in five groups: 0, norral; 1, hyperplasia; 2, metapl,asia; 3, atypical metapl.asia and carcino=.a- in-situ; I+, cercinos,a. The nunbers in these five cl.e.ases are respectively: 4E, 120, ~23, 57, 32; the mean is 1.80208, the variance is 0.99625, the skewness is 0.21 - 0.11, and the }curtosis is -0.06't 0.22. (7) Among the sources should be counted the differing standard deviations of the different pathologists, for sets of ratings ne,y differ considerably in their scatter about the mean. We understand that Professor C. F. Hosteller of Harvard has a method of alloving for this scatter, but ve sha11 not go into this. Cr R' I 1442 CONFIDENTIAI•: NINt4ESOTIA ~. ~I TOBACCO t.ITN:AT'fnv ~x~"a ~~F~Z 0013464

Text Control

Highlight Text:

OCR Text Alignment:

Image Control

Image Rotation:

Image Size: