Tobacco Documents Online

1/9/2 DIALOG(R)File 442:AMA Online Journal (c) 1995 American Medical Assoc. All rts. reserv. 00088598 COPYRIGHT American Medical Association 1992 A Study of Manufacturer-Supported Trials of Nonsteroidal Anti-inflammatory Drugs in the Treatment of ArthritisReporting and Documentation of Efficacy and Toxicity (ARTICLE) ROCHON, PAULA A.; GURWITZ, JERRY H.; SIMMS, ROBERT W.; FORTIN, PAUL R.; FELSON, DAVID T.; MINAKER, KENNETH L.; CHALMERS, THOMAS C. Archives of Internal Medicine tJan 24,, 1994; Original Investigation: p 157 L .I'~i TE COUNT: 00440 0003-9926 Background: To study the relation between reported drug performance in published trials and support of the trials by the manufacturer of the drug under evaluation, we studied a sample of trials of nonsteroidal anti-inflammatory drugs (NSAIDs) used in the treatment of arthritis. Methods: All randomized control trials of NSAIDs published between September 1987 and May 1990 identified by MEDLINE were reviewed. If an article met the following criteria (n=61), it was selected: trialsinvolving adult patients with osteoarthritis or rheumatoid arthritis (n=180), use of nonsalicylate NSAIDs marketed in the United States (n=101), randomized control trial (n=81), duration of the trial 4 or more days (n=78), and use of an efficacy outcome measure (n=61). Reviewers, 'blinded' to manufacturer status, evaluated the narrative interpretation of results and extracted numeric data on efficacy and toxicity. Manufacturer-associated trials were defined as those that acknowledged an association with a pharmaceutical manufacturer. Because of the scarcity of non-manufacturer-associated trials (n=9), we report only on the manufacturer-associated articles.Results: Fifty-two publications (85.2%) representing 56 trials were associatedwith a manufacturer. The manufacturer-associated drug was reported ascomparable with (71.4%) or superior to (28.6%) the comparison drug inall 56 trials. These narrative claims of superiority were usually justified with trial data. Of the trials identifying one drug as lesstoxic (n=22), the manufacturer-associated drug's safety was reported as superior to the comparison drug in 86.4% of cases. Justification for the narrative interpretation of the trial findings regarding lesstoxicity was provided in only 12 (54.5%) of 22 trials. Conclusion: The manufacturer-associated NSAID is almost always reported as being equal or superior in efficacy and toxicity to the comparison drug. These claims of superiority, especially in regard toside effect profiles, are often not supported by trial data. These data raise concerns about selective publication or biased interpretation of results in manufacturer-associated trials. (Arch Intern Med. 1994;154:157-163) Physicians seeking more objective information about the comparative performance of two treatments are likely to turn to the medical literature, specifically to the results of randomized control trials.While it is

assumed that a scientific article would provide much stronger evidence for clinical judgments than information found in advertisements,/1/ this assumption has not been adequately evaluated in regard to drug therapies. There is a long-standing history of collaboration between pharmaceutical manufacturers and medical researchers.J2,3/ but there is little information about the extent of pharmaceutical manufacturerinvolvement in randomized control trials and whether this relationship has an impact on research methods and published findings./4,5/ To investigate the impact of association with a pharmaceutical manufacturer on randomized control trials, we reviewedall efficacy trials of nonsteroidal anti-inflammatory drugs (NSAIDs) used in the treatment of arthritis published between 1987 and 1990. The NSAIDs are among the most frequently prescribed drugs in the United States/6/ and account for as much as 25% of all drug side effects./7/ Because of the challenges of treating many rheumatic conditions, clinicians are eager for more effective and safer therapies. Due to their frequent use and profitability, these drugs are of great interest to pharmaceutical manufacturers. Our objectives were to describe the type of acknowledged pharmaceutical manufacturer involvement and compare the performance of the drug manufactured by the group supporting the study with that of the comparison drug in terms of efficacy and toxicity. RESULTS DESCRIPTION OF STUDY SAMPLE Because of the scarcity of non-manufacturer-associated trials, we report only on manufacturer-associated trials. Of the 56 trials that acknowledged manufacturer association, 31 (55.4%) involved patients with osteoarthritis, 24 (42.9%) involved patients with rheumatoid arthritis, and one (1.8%) involved patients with both conditions. Thenumber of subjects in the trials ranged from 10 to 1328, with a median of 65 (mean<plus or minus>SD, 154.8<plus or minus>219.27). Allbut three (5.3%) of the trials in our sample were comparative drug trials. The other three trials were placebo controlled. An additionaltrial included a comparison with a placebo as well as with a comparative drug. The patients received drug treatment for 14 or moredays in 54 trials (96.4%). In the remaining two trials (3.6%), the drugs were administered for 7 days. Fifty-two articles representing 56 trials reported an association with a pharmaceutical manufacturer (Table 2). The study quality scores were low. Study quality ranged from 0.03 to 0.50 of a possible1.00 (3% to 50% of possible points). The mean (<plus or minus>SD) quality score assigned to the articles was 0.23<plus or minus>0.11. USE OF CLINICALLY RELEVANT EFFICACY OUTCOME MEASURES Estimation of drug efficacy was determined from a global patient-based rating in 11 trials (19.6%), a global observer-based rating in four trials (7.1%), and both patient- and observer-based global ratings in 37 trials (66.1%). Four (7.1%) of the trials did not include a global assessment of patient- or observer-based efficacy measures. All but one (1.8%) of the trials used at least onecategory from our list of clinically relevant efficacy outcome measures. COMPARATIVE DRUG DOSAGE In 23 trials (41.1%), comparable drug doses were used for the manufacturer-associated drug and the comparison drug (Figure 1). In 27 trials (48.2%), the dose of the manufacturer-associated drug was judged to be higher than that of the comparison drug. In two trials (3.6%), the dose of the manufacturer-associated drug was determined to be lower than that of

the comparison drug. In three (5.4%) of the trials the comparison was with placebo, and in one trial (1.8%) both drugs were manufactured by the same company. Because the drugs were both sponsored by the pharmaceutical manufacturer, a comparison drug could not be identified, and therefore this comparison could not be made. EVALUATION OF REPORTED EFFICACY AND TOXICITY Sixteen (28.6%) of the trials reported that one drug was more efficacious than the other in the narrative interpretation of the study findings (Figure 2). In all cases, the superior drug was the manufacturer-associated drug. In 40 trials (71.4%), the drugs were reported to be of comparable efficacy. Fourteen (87.5%) of the 16 trials reporting superior drug efficacy provided at least one test ofstatistical significance in a clinically relevant efficacy outcome measure to justify the narrative interpretation of superiority. In all four trials that included a placebo control group, the comparative drug was found to be more efficacious than the placebo. Twenty-two trials (39.3%) reported that one drug was less toxic than the other in the narrative inter-pretation of the study results (Figure 3). In 19 (86.4%) of these trials, the manufacturer-associated drug was reported as being less toxic than the comparison drug. In three trials (5.4%), the manufacturer-associated drug was reported as being more toxic than the comparison, but in one of these three instances the comparison was with a placebo. In the remaining 34 trials (60.7%), the drugs were reported as being of comparable toxicity in 32 trials, and no data about toxicity were provided in two trials. Ten (45.5%) of the 22 trials reporting one drug as being less toxic provided documentation for this statement by using a test of statistical significance. COMMENT Our results indicate that the manufacturer-associated drug is always reported as being either superior to or comparable with the comparison drug. These claims of superiority, especially with regard to side-effect profiles, are often not supported by trial data. The categories of outcome measures used to evaluate efficacy were broad. All of the trials reporting superior efficacy used at least one of the selected outcome measures. We did not evaluate the appropriateness of the analyses used to evaluate comparative efficacy. For example, several of the studies evaluated multiple efficacy outcomes repeatedly in the trial without correction of the threshold value for statistical significanceJl6/ Similarly, in the 22 trials where one drug was identified as being less toxic, the manufacturer-associated drug was reported as less toxic in the majority of cases. In almost half of these trials, this claim of less toxicity was not supported by a test of statistical significance. Since NSAIDs have fairly similar efficacy profiles,/17/ the choice of an NSAID is often made on the basis of its toxicity profile. Unlike efficacy outcome assessment, for which guidelines are available from the Food and Drug Administration to standardize reporting of efficacy,/15/ we are unaware of any standard informationon drug toxicity that is required to be reported in journal articles. When we evaluated the relative range of dosing of the manufacturer-associated drug and the comparison agents in the trials on the basis of the recommended dosage suggested in standard texts, there was considerable mismatch. In the majority of cases in which the doses were not equivalent, the drug given at the higher dose was that of the supporting manufacturer. Higher doses of NSAIDs may bias the study results on efficacy

911,IIIIIIIIIIUIVVIIIIIVIVIJUll6'IIIIIU~I'Vl9' IIIV.IIpI~l~~ll of standard criteria toreport drug toxicity in clinical trials, evaluation of this problem is difficult. Publication bias is a universal problem that is illustrated by the finding that a positive study is more likely to be published than onewith negative results./18/ Regardless, it is still unlikely that the manufacturer-associated drug should always be of superior or comparable efficacy to the comparison drug, as is reported in the present study. Because pharmaceutical manufacturers may decide whether or not to publish their data/19-21/ or may release only favorable data,/22/ they may be less likely to submit a trial for publication if they have a negative result./4 In addition, pharmaceutical manufacturers may select a comparison agent that is less likely to be superior to their drug./22,23/ We have demonstratedthat doses of drugs are chosen that appear to optimize the efficacy performance of the manufacturer's drug. We chose randomized control trials because they are considered the gold standard for therapeutic evaluation. Because our sample of articles is recent, our findings regarding the frequency and effect of manufacturer support are generalizable to other current trials. Our findings may be explained by reasons other than a direct association with a pharmaceutical manufacturer. For example, the fmdings may be related to the quality of the journals in which thesearticles were published. In general, the quality of these articles asreflected by the quality scores was low. Quality scores vary greatly across the medical literature in various specialties. While there areno published NSAID quality scores, these scores are in contrast to a mean quality score of 0.42<plus or minus>0.16 found in more than 400 clinical trials of miscellaneous treatments./2M In addition, almost a third of the articles in our sample were published in journal supplements, which are unlikely to be peer reviewed./25/ To our knowledge, only one previous study has evaluated the relationship between manufacturer association and its effect on published results. Our results are consistent with those of this previous study by Davidson,/41 in which all clinical trials of pharmacologic and nonpharmacologic therapies published in 1984 in five medical journals were reviewed. A statistically significant association between the source of funding and outcome was found. We originally sought to compare trials that reported an associationwith a pharmaceutical manufacturer with those that did not. Only a few of the trials meeting our inclusion criteria did not acknowledge an association with a pharmaceutical manufacturer. A declaration of manufacturer support may not be required by all journals, leading to a risk of misclassification. Because of the inadequate numbers of trials and the risk of misclassification, we did not compare manufacturer-associated trials with those without such an association. Our definition of manufacturer-association articles is more generous than that of previous studies that have evaluated this issue./4,23/ It can be argued that the provision of study drugs does not imply the same degree of support as acknowledged grant support. Until there are standardized requirements for reporting this information, the true level of support remains difficult to determine. Our findings support the necessity of a clear and full acknowledgment

and description of pharmaceutical manufacturer supportin articles summarizing the results of clinical trials./2J The widelyaccepted uniform requirements for manuscripts submitted to biomedicaljournals state that there should be 'acknowledgment of financial and material support, specifying the nature of the support.'/26/ These guidelines can be improved. In any article it is important to know the potential sources of bias. We propose that information on pharmaceutical manufacturer involvement be clearly stated. Specifically, pharmaceutical manufacturer association should be acknowledged as being present or absent. For all articles where thereis association with a manufacturer, additional information is required on (1) the name of the pharmaceutical manufacturer, (2) the name of the pharmaceutical manufacturer-associated product, and (3) afull description of the type of manufacturer sponsorship. What do these findings mean for the clinician who uses the results of published clinical trials to guide pharmacotherapeutic decision making? Clinicians should be aware of the possibility of publication bias. Our results demonstrate that the manufacturer-associated drug is likely to be reported as being more efficacious and less toxic than the comparison drug. Clinicians should carefully evaluate the trial data to determine if these claims of superior performance are warranted. Our data illustrate that claims of superior efficacy or offewer adverse effects were not always substantiated by tests of statistical significance. These fmdings provide a rationale for structured information to be included at the end of articles to help the reader objectively interpret the trial fmdings to help make appropriate clinical decisions. Accepted for publication May 5, 1993. This study was supported in part by grant RO 1 HS-05936 Agency for Health Care Policy and Research, Public Health Service, Department ofHealth and Human Resources; a research fellowship at the Brockton/West Roxbury Division of the Boston (Mass) area Geriatric Research Education and Clinical Center (Dr Rochon); Clinical Investigators Award K08 AG00510 from the National Institute on Aging,National Institutes of Health, Bethesda, Md (Dr Gurwitz); and Le Fonds de la Recherche en Sante du Quebec grant 910486-103 and Arthritis Society of Canada (Toronto, Ontario) grant 89004 (Dr Fortin). Presented as a poster exhibit at the American Society for Clinical Pharmacology meeting, Honolulu, Hawaii, March 26, 1993. We are indebted to Frederick Mosteller, PhD, for comments on the manuscript, to Mary Abram, MLS, for her assistance with the collection of the reference material, and to Bruce Kupelnick for his ongoing assistance. Reprint requests to Baycrest Centre for Geriatric Care, 3560 Bathurst St, North York, Ontario, Canada M6A 2E1 (Dr Rochon). l. Beary JF. Pharmaceutical ads in journals. Ann Intern Med. 1992;117:616. 2. Relman AS. Dealing with conflicts of interest. N Engl J Med. 1984;310:1182-1183. 3. Rawlins MD. Doctors and the drug makers. Lancet. 1984;2:276-278. 4. Davidson RA. Source of funding and outcome of clinical trials. J Gen Intern Med. 1986;1:155-15 8. 5.

Gotzsche PC. Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Controlled Clin Trials. 1989; 10:31-56. 6. Baum C, Kennedy DL, Forbes JB. Utilization of nonsteroidal anti-inflammatory drugs. Arthritis Rheum. 1985;28:686-692. 7. Committee on Safety of Medicines. Non-steroidal anti-inflammatory drugs and serious gastrointestinal adverse reactions, 1. BMJ. 1986;292:614. 8. Rochon PA, Fortin PR, Dear KBG, Minaker KL, Chalmers TC. Reporting of age data in clinical trials of arthritis: deficiencies and solutions. Arch Intern Med. 1993;153:243-248. 9. Physicians' Desk Reference. 46th ed. Montvale, NJ: Medical Economics Co Inc; 1992;1:2561. 10. Canadian Pharmaceutical Association. Compendium of Pharmaceuticals and Specialties. 27th ed. Ottawa, Ontario: Canadian Pharmaceutical Association and Specialties; 1992; 1. 11. Joint Formulary Committee 1989-1990. British National Formulary. London, England: British Medical Association and Pharmaceutical Press; 1990;19:559. 12. Verzeichnis von Fertigarneimitteln der Mitglieder des Bundesverbandes der Pharmazeutischen Industrie e.V. Rote Liste 1991. Wurt, Germany: Aulendorf Cantor; 1991:1. 13. Farmindustria. Repertorio Farmaceutico Italiano. 4th edition. Milano, Italia; Associazione Nazionale dell'Industria Farmaceutica; 1990;1. 14. Chalmers TC, Smith H Jr, Blackburn B, et al. A method for assessingthe quality of a randomized control trial. Controlled Clin Trials. 1981;2:31-49. 15. Food and Drug Administration. Guidelines for the Clinical Evaluation of Anti-inflammatory and Antirheumatic Drugs (Adults and Children). Washington, DC: US Dept of Health and Human Services, Public Health Service, Food and Drug Administration; 1991. 16. Ingelfmger JA, Mosteller F, Thibodeau LA, Ware JH. Biostatistics in Clinical Medicine. 2nd ed. New York, NY: Macmillan Publishing Co Inc; 1987;1:339. 17. Fries JF, Williams CA, Bloch DA. The relative toxicity of nonsteroidal anti-inflammatory drugs. Arthritis Rheum. 1991;34:1353-1360. 18. Dickerson K. The existence of publication bias and risk factors forits occurrence. JAMA. 1990;263:1385-1389. 19. Levy G. Publication bias: its implications for clinical pharmacology. Clin Pharmacol Ther. 1992;52:115-119. 20.

Lauritsen K, Havelund T, Laursen LS, Rask-Madsen J. Withholding unfavorable results in drug company sponsored clinical trials. Lancet. 1987;1:1091. 21. Mindel JS. Failure of controlled clinical trial data to reach the literature. Clin Pharmacol Ther. 1992;52:4-5. 22. Hillman AL, Eisenberg JM, Pauly MV, et al. Avoiding bias in the conduct and reporting of cost-effectiveness research sponsored by pharmaceutical companies. N Engl J Med. 1991;324:1362-1365. 23. Anderson J, Felson DT, Meenan RF. Secular changes in published clinical trials of second-line agents in rheumatoid arthritis. Arthritis Rheum. 1991;34:1304-1309. 24. Reitman D, Sacks HS, Chalmers TC. Technical quality assessment of randomized control trials (RCTs). Controlled Clin Trials. 1987;8:282. 25. Bero LA, Galbraith BA, Rennie D. The publication of sponsored symposiums in medical journals. N Engl J Med. 1992;327:1135-1140. 26. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Can Med Assoc J. 1992;146:861-868.manufacturer-associated drug is always reported as being either superior to or comparable with the comparison drugthere was considerable mismatchPublication bias is a universal problem Our fmdings support the necessity of a clear and full acknowledgment and description of pharmaceutical manufacturer support From the Geriatric ResearchEducation and Clinical Center, BrocktonfWest Roxbury Veterans Affairs Medical Center, Division on Aging, Harvard Medical School, Boston, Mass (Drs Rochon and Minaker);Program for the Analysis of Clinical Strategies, Gerontology Division, Brigham and Women's Hospital, Harvard Medical School (Dr Gurwitz); Arthritis Center, Boston University School of Medicine (DrsSimms and Felson); Department of Medicine and Division of Clinical Epidemiology, Montreal (Quebec) General Hospital, McGill University (Dr Fortin); and Technology Assessment Group, Harvard School of Public Health (Dr Chalmers). Dr Rochon is now with Baycrest Centre for Geriatric Care and Mount Sinai Hospital, Division of Geriatric Medicine, Department of Medicine, University of Toronto (Ontario). The quality scores assigned were percentages (total score/total possible score). The quality scores could range from 0 for the lowest-scoring articles to a maximum possible score of 1.0. Drug dosage was rated on a scale from 1 to 5 on the basis of the relationship of reported drug dosage used in the trial to the usuallyrecommended dose provided in standard sources/9-13/ or, when the drugs were not marketed, from the standard dose information provided in the articles. The five-point scale for drug dosage was defined as follows: (1) below the usual dosage range, (2) low-dosage range, (3) middosage range, (4) high-dosage range, and (5) above the usual dosage range. The dosage range assigned to each of the drugs is outlined in Table 1. Manufacturer-associated drug dosage was then compared with that of the comparison drug. Clinically relevant efficacy outcome measures were characterized

according to standard criteria outlined by the Food and Drug Administration./15/ For patients with osteoarthritis, four efficacy outcome measures were considered: (1) observer rating of overall drugefficacy, (2) patient rating of overall drug efficacy, (3) pain on activity, and (4) pain at rest. Eight clinically relevant efficacy outcome measures were used for patients with rheumatoid arthritis: (1) observer rating of overall drug efficacy, (2) patient rating of overall drug efficacy, (3) swollen joint count, (4) tender joint count, (5) duration of morning stiffness, (6) grip strength, (7) timeto walk 50 ft, and (8) erythrocyte sedimentation rate. EVALUATION OF STUDY FINDINGS (EFFICACY AND TOXICITY) Efficacy Assessment The information on the efficacy outcome measures and drug toxicity was collected from the trials by three independent reviewers. When all three reviewers could not agree on an evaluation, the assessment of the two agreeing reviewers was used. For each trial, reviewers blinded to pharmaceutical manufacturer status evaluated the narrative interpretation of the study results regarding comparative drug performance as detailed in the abstract and the conclusion of the article. Specifically, each of the reviewers assessed whether one drug was identified as being more efficacious than the comparison drug in the narrative description of the study. If a drug was identified as being more efficacious, the reviewers determined if there was documentation to support this claim. Our definition for this documentation was extremely liberal. We defined adequate documentation as being the reporting of a test of statistical significance (P<less than or equal to>.05) for one or more comparisons in a clinically relevant efficacy outcome measure. Toxicity Assessment The reviewers also evaluated the narrative interpretation of the study results regarding comparative drug toxicity. Specifically, eachreviewer answered the following question: Was one drug identified as being less toxic in the narrative description of the study? If a drug was identified as being less toxic, docu-mentation of this claim was evaluated. Our definition for this documentation was again extremely liberal. We defined adequate documentation as being the reporting of a test of statistical significance (P<less than or equal to>.05) on any component of toxicity. MATERIALS AND METHODS SAMPLE SELECTION All randomized control trials of NSAIDs listed in MEDLINE between September 1987 and May 1990 were identified by means of a previously described MEDLINE search strategy./8/ This search strategy was broad enough to retrieve the abstracts of articles in which NSAIDs were used for the treatment of arthritis and other indications. The MEDLINE search identified 1008 articles published between 1987 and 1990. Sixty-one articles representing 69 individual trials met our inclusion criteria, outlined as follows: trials involving adult patients with osteoarthritis or rheumatoid arthritis (n=180), use of nonsalicylate NSAIDs marketed in the United States in 1992 (n=101), randomized control trial (n=81), duration of the trial 4 or more days(n=78), and use of an efficacy outcome measure (n-61). Additional articles were excluded as follows: duplicate publications of data involving four articles, only two of which were included (n=2), articles comparing two doses of the same drug (n33), gastrointestinalprophylaxis with ranitidine or misoprostol (n=2), and

nonefficacy outcome measure (n=10). The NSAIDs marketed in the United States in 1992 included diclofenac, diflunisal, fenoprofen, flurbiprofen, ibuprofen, indomethacin, ketoprofen, ketorolac, meclofenamate, mefenamic acid, nabumetone, naproxen, phenylbutazone, piroxicam, sulindac, and tolmetin. To ensure that we evaluated only randomized control trials, the word randomized was required in the text of the article. If this information was obscure, the author was contacted for verification ofrandomization status. Of the sixauthors contacted, three confirmed that randomization had been used, and these articles were included inour sample. DEFINITIONS OF TERMS Manufacturer association was defined as any of the following: (1) acknowledged grant support by a pharmaceutical manufacturer, (2) pharmaceutical employee listed as author, (3) drug supplied by manufacturer, and (4) publication in a journal supplement sponsored by a pharmaceutical manufacturer. The type of manufacturer association was assigned to one of these groups in a mutually exclusive manner according to the listed hierarchy. Therefore, an article that both acknowledged grant support from a manufacturer and was published in a sponsored journal supplement would be assigned to the grant support group. The manufacturer-associated drug was defined as the drug of interest to the sponsoring company and was identified on the basis ofinformation provided in the article or from standard references./9-13/ This characterization was performed independent of the evaluation process described below. An assessment of article quality was performed by means of a standard quality scoring system./14/ The quality scores assigned to each article were based on (1) basic study descriptive material, (2) study protocol, and (3) data analysis. To minimize bias in scoring quality, a trained technician extracted the 'Methods' and 'Results' sections from each of these articles, removing identifying information and sources of fmancial support and disguising the results so that the reviewers did not know the name of the drugs under evaluation. Scores were obtained on the quality of the 'Methods' section by one of us (P.A.R.) and on the 'Results' secion by two of us (P.A.R. and P.R.F.) for each of the articles.