Background. Previous analyses of published clinical trials have identified major deficiencies in reporting, design, analysis, and overall quality. The purpose of this study was to determine the strengths and weaknesses of published clinical trials in family practice, and to identify predictors of quality in these trials.

Methods. Randomized controlled clinical trials published in The Journal of Family Practice from 1974 to 1991 were eligible for the study. Two raters independently evaluated the adequacy and appropriateness of reporting, design, and analysis for each clinical trial, using the Chalmers index for assessing clinical trial quality. Multiple linear regression was used to determine the predictors of quality.

The 53 trials included in the study showed deficiencies in reporting, design, and analysis, although fundamental design issues, such as blinding, were a relative strength. On average, the trials scored 35% of the possible points on the scale. Three factors were positively associated with overall quality: year of publication, number of pages of the published report, and the type of intervention. Trials with pharmacologic and nonmedication therapy interventions, such as diet, had higher quality scores than did trials with psychosocial or educational interventions.

Conclusions. The overall quality of these clinical trials was less than optimal but comparable to previously analyzed groups of trials. The improvement in quality over time may be related to improvement in the quality of the trials themselves, or more exacting editorial standards, or a combination of the two.

Clinical trials; randomized controlled trials; meta-analysis; research design; quality of research. (J Fam Pract 1994; 39:225-235) Although many clinical trials are methodologically sound, even a casual review of published clinical trials reveals that many trials have not incorporated fundamental principles of clinical trial research. Analyses of clinical trials published in a wide variety of journals have identified large deficiencies in reporting, design, analysis, and overall quality.

These findings may not be generalizable to clinical trials in the family practice research literature for several reasons. First, to our knowledge, few if any of the previous analyses included trials from the family practice literature. Second, because family practice is a relatively new discipline, few clinical trials have been published in this specialty.Third, given the importance of psychosocial factors in family practice theory, the content of clinical trials in family practice may be different from that of other disciplines.

Several previous analyses of clinical trials also have attempted to identify predictors of overall quality. In a study of breast cancer trials, Liberati and colleagues showed that the quality could be predicted by the year the trial started and biostatistician involvement. In an analysis of clinical trials from a variety of disciplines, Emerson and colleagues showed that quality could be predicted by year of publication and clinical content. However, there are several important factors that may be related to quality that were not considered in either study. The type of intervention (eg, medication vs patient education), research training of the authors (eg, authors with PhD or MPH degrees), affiliation of the authors (university-based vs practice-based), and size of the research team (number of authors) all may be related to quality of the clinical trial. Since adequacy of reporting is one component of the quality of the published report, the number of pages also may be related to quality of the published report.

The purpose of this study was to answer the following three research questions:  What are the strengths and areas for improvement of published clinical trials in the discipline of family practice, as reflected by clinical trials published in The Journal of Family Practice? How do trials published in The Journal of Family Practice compare with those of other disciplines?  What are the predictors of quality?

To answer these questions, we performed a cross-sectional analysis of clinical trials published in The Journal of Family Practice, using a standard instrument, the Chalmers index,to assess trial quality. Although original research in family practice is published in a variety of medical journals, this study is limited to clinical trials in The Journal of Family Practice for two reasons. First, it is the primary journal for original research in the discipline. Faculty who seek or have been nominated for academic promotion(23)(24) are far more likely to publish in The Journal of Family Practice than in any other single journal. As noted in a 1989 review of articles published in family practice, “The Journal of Family Practice remains the principal repository of original work in the field.”(19) Second, if clinical trials from multiple family practice journals had been eligible for inclusion, it would have been impossible to disentangle the effect of year of publication from that of “start-up” difficulties for new journals.