Introduction

In previous chapters in this book, we presented the process and main principles of evidence-based dentistry (EBD);¹ how to search for evidence;² and how to use articles about therapy,³ harm,⁴ diagnosis,⁵ systematic reviews,⁶ clinical practice guidelines,⁷ qualitative studies,⁸ and economic evaluations.⁹ In this chapter of the book, we offer clinicians guidance on how to avoid being misled by biased interpretations of study results.

Academic competition and conflict of interest have fueled misleading presentations of research findings published in peer-reviewed journals. Regardless of whether a researcher works in academia or in the pharmaceutical industry, there is always a personal interest and a rising pressure to succeed and to provide novel and exciting findings; this pressure often results in interpretations of findings that are far more enthusiastic than the data warrant.¹⁰

In the area of psychopharmacology, for example, the investigators of 90% to 98% of industry-funded primary studies comparing two drugs reported results that favored the drug produced by their company, particularly when the active comparator drug was a rival product.¹¹ This situation is not exclusive to primary studies. The investigators of industry-sponsored systematic reviews are less transparent regarding their methods, are less rigorous in their risk of bias assessment, and provide more favorable conclusions toward the study sponsor’s drug than are the investigators of reviews that have not been funded by the investigators’ industry.¹² When companies employ ghostwriters to produce manuscripts under the names of credible and often well-known researchers, the reported results are likely to be overly favorable.13

The involvement of members of a specific industry is not necessary for overenthusiastic interpretations of results. Academic investigators also are subject to the global industry of producing research evidence. The reward system in science involves receiving grants and having research results published, and scientists may believe that overplaying the significance of their work is a requirement for success.¹⁴

Although guidance and tools to help clinicians recognize study results that have a high risk of bias are widely available,^15,16 researchers have made limited efforts to facilitate the identification of distorted interpretations and misleading presentations of the results of clinical studies. We present the following examples not to criticize investigators, but to illustrate the need to increase awareness among clinicians and encourage them to avoid putting excessive trust in investigators’ interpretations of their findings.

Guidance on How to Avoid Being Misled by the Results of Clinical Studies

We present seven criteria that dental professionals can follow to avoid being misled by the results of clinical studies.¹⁷ We illustrate each criterion with a real example from the dental literature, shown in the boxes after each section.

1. Read only the methods and results sections; disregard the inferences.

In not only the discussion but also in the conclusion and the introduction sections of research articles, investigators may provide inferences that differ from those that a less conflicted or involved reader would offer. A number of investigators have addressed the association between funding and the conclusions derived from randomized controlled trials.^18–22 Results have been consistent: researchers are more enthusiastic about new interventions when funding comes from for-profit sources than from not-for-profit sources. In dentistry, investigators have documented that randomized controlled trials in which the authors reported conflicts of interest are more likely to report results supporting the intervention under study than those trials whose authors did not report conflicts of interest (odds ratio [OR] = 2.40; 95% confidence interval [CI], 1.16-5.13).²¹

This situation also affects systematic reviews and meta-analyses of drug interventions. Although industry-sponsored and nonindustry-sponsored reviews (for example, Cochrane systematic reviews) answering the same clinical question report similar treatment effect estimates, the former type of reviews provide more favorable conclusions.¹² In summary, our advice is to read only the methods and the results sections of these articles, skipping the discussion section. However, to apply this guideline, clinicians must be able to assess the rigor of the methods and interpret the results.

2. Read synoptic abstracts published in secondary publications (preappraised resources) for evidence-based dentistry.

Busy clinicians interested in using evidence to inform their clinical practice may not have time to skip the discussion sections of articles and instead critically appraise the evidence, and thus make sense of the results, by themselves. Secondary journals and sources, such as Evidence-Based Dentistry, Journal of Evidence-Based Dental Practice, and the American Dental Association’s Evidence Database, publish synoptic summaries in an abstract format that are accompanied by a brief summary of the original article and a critical appraisal conducted by a team of clinicians and methodologists. These abstracts, developed by independent third parties who have no conflicts of interest, reduce the distortion that the authors of a primary or secondary study may have introduced in the original article. Another objective of this type of synopsis is to educate clinicians about the methodological aspects of different study designs, thereby increasing clinicians’ critical appraisal skills.

3. Beware of large treatment effects presented in trials with few events.

Clinicians often are appropriately skeptical of using evidence from the results of only one study and applying it in clinical practice. One argument is that the first studies that investigators conduct to determine the effects of an intervention usually have a small sample size (for example, fewer than 200 participants) and too few events. A meta-epidemiologic study published in the oral medicine and maxillofacial surgery literature showed that the investigators of small randomized trials (that is, those involving fewer than 200 patients) were more likely to report larger and more beneficial effects compared with the investigators of large randomized trials (that is, those involving at least 200 patients) (OR = 0.92; 95% CI, 0.87-0.98; P =.009).²⁶ Most of the time, our therapeutic interventions target one or two of the many pathologic mechanisms involved in the genesis of a disease.²⁷ This is why, not only in dentistry but also in medicine in general, few interventions are able to demonstrate a large and real treatment effect.

The results of a systematic survey whose investigators analyzed 85,000 meta-analysis results extracted from 3,082 systematic reviews showed that, in 10% of the cases, the results of the first trial showed statistical significance and a large treatment effect, which afterward proved to be much smaller in comparison with the results that the investigators initially reported.²⁸ It is important to notice that, when few events are available, even systematic reviews, including meta-analyses, could have this problem. Readers applying this guideline should beware of treatment effects that look “too large to be real,” because they are likely to be misleading.

Stay updated, free dental videos. Join our Telegram channel

Join