Tobacco Documents Online

Second International Congress on Peer Review in Biomedical Publication Sponsored by the American Medical Association

1 The Second Peer Review Congress I Abstracts /Inappropriate and Appropriate Selection of 'Peers' in Grant Review *Stanton A. Glantz Division of Cardiology, Box 0124, University of California. San Francisco. CA 94143-0124 Objective: To assess the California Tobacco Related Disease Research Program (TRDRP) Behaviorial and Public Health Research on Tobacco Study Section members as "peers" to review tobacco policy research. Design: MEDLINE was searched for the years 1989 through 1993 with the use of the key word tobacco for TRDRP reviewers. The National Heart. Lung and Blood Institute (NHLBI) Cardiovascular Study Section, which reviews my NHLBI grant on ventricular function, was searched as a control using the key word heart. Results: Six of 18 TRDRP reviewers (33%) had no citations. The median number of citations was two (interquartile range, zero through four). I had seven publications indexed to "tobacco"; only one member (5%) had more than seven publications indexed on "tobacco." The members' "tobacco" publications concentrated on well-controlled experimental interventions on smoking cessation and prevention strategies. Only one member had primary expertise in tobacco policy research. All 31 (100%) of the NHLBI reviewers had "heart" publications. The median number of citations was nine (interquartile range, seven through 19). I had 11 publications indexed to "heart"; 13 members (42%) had more than 11 publications. Five members had a primary interest in the subject of my NHLBI application. Conclusions: Members' professional interests play a critical role in the level of interest and enthusiasm they will have for a proposal, which directly affects the priority score. The TRDRP's primary focus is experimental intervention; the single member on tobacco control does not overcome the biases intrinsic in this membership. A similar situation exists at the National Cancer Institute, where the Behavioral Medicine Study Section has not funded one of the last 41 applications for tobacco control research. The membership of these review committees has effectively precluded research on tobacco control policy. Statistics and Peer Review Do We Need Structured Reporting of Randomized Controlled Trials? *David Moher, Corrine S. Dulbera, and George A. Wells Clinical Epidemiology Unit, Loeb Medical Research Institute, Ottawa Civic Hospital, 1053 Carling Ave, Ottawa, Ontario, Canada K1Y 4E9 Objective: To examine whether there has been a change in the reporting of statistical power and sample size calculations in published negative randomized controlled trials (RCTs) during the last 15 years. Design: A structured data collection form and guidelines to 12 classify a trial as negative and to determine primary outcomes were developed. All RCTs published in three leading general medical journals published in 1975, 1980, 1985, and 1990 were reviewed. Results: There were 102 negative RCTs published during this 15-year review. Of these. only 33 (32%) reported calculating sample size. Although this number is small, the situation has improved over time, from 0% in 1975 to 43% in 1990. Less than half of the trials reported the control group event rate (45%). Eighteen percent reported whether the alpha was one or two tailed, and no trial reported the statistic on which sample size calculations were based. Only 20 trials made any kind of statement of the clinical sigrtificance of the results with respect to the observed differences. Conclusions: There has been a considerable increase in the number of RCTs published durin= the period of this review. Our results indicate that there is still room for improvement in the quality of reporting of many statistical issues. Structur.J reporting of pertinent information in RCTs mi=ht improve communication in the same way as have structured abstracts. Assessing the Quality of Randomization From Reports of Controlled Trials Published in Journals of Obstetrics and Gynecology Kenneth F. Schulz.' *lain Chalmers.' David A. Grimes.= and Douglas G. Altman' 'UK Cochrane Centre. Summenown Pavilion. Middle Way. Oxford OX2 7LG. England: =Depanment of Obstetrics. Gynecology and Reproductive Sciences. University of California. San Francisco: and 'Medical Statistics Laboratory. Imperial Cancer Research Fund. London. England Objective: To assess the extent to which reports of controlled provide details trials in journals of obstetrics and gynecology of the way in which trial participants were allocated to treatment comparison groups. Design: Survey of all reports of parallel group trials (in which allocation was stated to have been randomized) published in the 1990 and 1991 volumes of four joumals of obstetrics and gynecolog gy. Results: Of 206 reports identified, only 32% gave an acceptable account of the method used to generate random numbers, and only 23% contained information showing that steps had been taken to conceal assignment until the point of treatment allocation. In 125 reports in which hypothesis tests were used to compare baseline characteristics, only 2% were statistically significant at the 5% level. In 96 reports of trials usins unblocked randomization, the differences in sample sizes between treatment and control groups were much smaller than would be expected by chance (P<.0001). Conclusions: Proper randomization is required to generate unbiased comparison groups in clinical trials. Only 10% of the reports of trials published in a'sample of obstetrics and ~ynecoloQy journals described proper methods of randomiza- tion. Additional analyses suggest that there may have been both selective reporting of baseline comparisons and non- random manipulation of comparison groups by investigators.

The Second Peer Review Congress / Abstracts scientific review article were cited significantly less often than other papers by Slutsky. Conclusions: Scientists did not suspect that Slutsky's papers contained fabrications until news of Slutsky's misconduct became public. Afterward, scientists looked to Slutsky's academic institution for guidance as to which articles were tainted. Academic institutions can play a significant role in correcting the scientific literature by thoroughly investigating suspected cases of fraud and disclosing their findings. Duplicate Publication of Original Articles in and From the Nederlands Tijdschrift voor Geneeskunde (Dutch Journal of Medicine) Hans Barnard and *:jnn A. Overbeke Nederlands Tijdschrift voor Geneeskunde. PO Box 75971. 1070 AZ Amsterdam. the Netherlands Objective: To determine the numbers of correct and incorrect duplicate publications of original articles in the Nederlands Tijdschrift voor Geneeskunde (Dutch Journal of Medicine) based on the statement on duplicate publication by the International Committee of Medical Journal Editors (ICMJE). Design: Retrospective bibliometric study followed by interviews. Of the first and second authors of 172 original articles (al195 articles from the first half of 1990 and all 77 articles from the first half of 1992). all biomedical publica- tions havin- the same purport published in 1990 through 1992 were looked up. The authors were subsequently approached by telephone to find out the background of these duplicate publications. Results: For two articles the corresponding articles were not found. Of the remainina 94 articles from the first half of 1990, 11 (12%) proved to have been published again without this fact being mentioned in a footnote (for which omission the journals were responsible in four cases and the authors in seven cases). Of the remaining 76 articles from the first half of 1992, 12 were found to have been published previously, five in accordance with and seven against the IC.'uL1E statement. Duplicate publication against the statement was attributed to incorrect interpretation of the ICMJE statement by the authors. Conclusions: Of the original articles published in the Nederlands Tijdschrift voor Geneeskunde. a minimum of 14% are being or have already been published elsewhere, 11% without this fact beina stated according to the ICMJE statement on duplicate publication. Bias and Peer Review Publication Bias and Public Policy *Lisa Bero and Drummond Rennie Institute for Health Policy Studies, University of California. San Francisco, 1388 Sutter St. I 1 th Floor, San Francisco. CA 94109 Objective: To examine the tobacco industry's claim that publication bias makes risk assessments of environmental tobacco smoke (ETS ) invalid. Public health policies regulating exposure to hazardous compounds, such as ETS, are often based on risk assessments. The tobacco industry has argued, most recently in a lawsuit against the the US Environmental Protection Agency (EPA), that the EPA's risk assessment of ETS, which concluded that ETS is a cause of lung cancer and respiratory disorders, is invalid. The industry claims that the meta-analysis conducted by the EPA was flawed because publication bias against "negative" studies resulted in exclusion of negative studies from the meta-analysis. Design: To determine if negative studies on ETS are published, the numbers of articles that supported ("positive studies") or refuted ("negative studies") the hypothesis that ETS exposure is associated with adverse health effects were assessed. All articles that tested this hypothesis were selected from the peer-reviewed medical literature and tobacco industry-sponsored symposia proceedings. Articles were identified by a computerized search of the medical literature supplemented with material obtained from the tobacco industry. Results: Eighty-two percent (40/49) of the peer-reviewed articles were positive, compared with 37% (24/65) of symposia articles (P= .0001). Ninety-four percent (46/49) of the peer-reviewed articles contained a "Methods" section. compared with 43% (28/65) of symposia articles (P=.0001). Conclusions: These findings raise the policy question of whether negative studies should automatically be included in meta-analyses. We recommend that the quality of the studies, their sponsorship, and the review process to which they have been subjected be carefully evaluated. Effect of Institutional Prestige on Reviewers' Recommendations and Editorial Decisions *Joseph M. Garfunkel. Harvey J. Hamrick, Edward E. Law•son, and Martin H. Ulshen Journal of Pediatrics. CBN 7230. Medical School Wing C. University of North Carolina, Chapel Hill. NC 27599-7230 Objective: To determine whether manuscripts from institu- tions with greater "prestige" are more likely to be recommended for publication by reviewers and to be accepted for publication. »