Introduction
Optimal reporting of randomized trials and abstracts enhances transparency and facilitates assessment and identification of trials. The purpose of this study was to investigate the quality of reporting of abstracts of randomized controlled trials published in orthodontic journals.
Methods
Electronic searches with supplementary hand searching to identify randomized controlled trials in the American Journal of Orthodontics and Dentofacial Orthopedics, the Angle Orthodontist , the European Journal of Orthodontics , and the Journal of Orthodontics from 2006 to 2011 were undertaken. The completeness of abstract reporting was evaluated with a modified CONSORT for abstracts statement checklist. The data were analyzed by using descriptive statistics followed by univariate and multivariate examinations of statistical associations ( P = 0.05).
Results
Abstracts of 117 randomized controlled trials were identified and assessed. Most were published in either the American Journal of Orthodontics and Dentofacial Orthopedics (53%) or the Angle Orthodontist (23%); most abstracts (85.5%) were structured. The mean overall reporting quality score was 60.2%. In relation to individual quality items, most abstracts demonstrated clear reporting of interventions (97.4%), objectives (93.2%), and number of participants randomized (95.7%). Insufficient reporting of randomization procedures, allocation concealment, blinding, and failure to report confidence intervals and harms were almost universal. Registrations of randomized controlled trials and sources of funding were not reported in any of the identified abstracts. The highest reporting score was noted in the Journal of Orthodontics (66%; 95% confidence interval, 63.5-68.7).
Conclusions
The quality of reporting of abstracts of randomized controlled trials in orthodontic journals is suboptimal. In view of the primacy of research abstracts, efforts should be made to improve their reporting.
Randomized controlled trials (RCTs) are considered to be the cornerstone of clinical research; high quality RCTs in orthodontics can be used to evaluate clinical interventions stringently with the ultimate aim of reaching valid conclusions about existing and prospective treatment approaches. The reliability of conclusions from individual trials depends greatly on internal validity, which is predicated on the quality of the research design. Research with the appropriate methodology is also more likely to have a significant impact on clinical practice.
It has been reported that lack of controls; absence of randomization, allocation concealment, or blinding; and failure to account for loss to follow-up might introduce bias, rendering research results questionable. Although the RCT is considered to be the design likely to provide the highest quality of evidence, empirical evidence has indicated that the reporting quality of RCTs can be suboptimal. Although it has been shown that low reporting quality is not necessarily synonymous with low study quality, reporting needs to be improved. Better reporting improves transparency, accurate assessment of trials, correct indexing, retrieval, and appropriate inclusion in systematic reviews, which have become instrumental to evidence-based health care.
CONSORT guidelines have been developed in an attempt to standardize and guide researchers directly on reporting and indirectly on the conduct of RCTs. These guidelines comprise 25 items covering key aspects of clinical trials, setting a benchmark on the reporting of design, conduct, and analysis of such studies. The CONSORT guidelines have been embraced in the dental literature and have been publicized and endorsed by editors of orthodontic journals. Abstracts of published articles are considered to be particularly important, since consumers of research often fail to read or do not have access to complete articles; many therefore use the abstract alone to evaluate clinical research. The relative importance of abstract style and content has led to the publication of the CONSORT guidelines for the reporting of abstracts of RCTs ( Fig 1 ).
An analysis of the reporting of abstracts in RCTs in orthodontic journals has not yet been undertaken. The primary objective of this study was therefore to evaluate the quality of reporting of RCT abstracts with reference to the CONSORT guidelines for abstracts in 4 leading orthodontic journals. Secondary aims were to compare the quality of abstracts published in these journals and to identify factors associated with improved reporting in abstracts in orthodontic RCTs.
Material and methods
A literature search was undertaken to identify RCTs in 4 leading orthodontic journals from 2006 to 2011. Electronic searching with supplementary hand searching of the American Journal of Orthodontics and Dentofacial Orthopedics , the Angle Orthodontist , the European Journal of Orthodontics , and the Journal of Orthodontics was undertaken. Eligible studies met the following inclusion criteria: the phrase “randomized controlled trial” or “randomised controlled trial” was in the title or abstract, or it was apparent in the methodology section that the trial was an RCT. Other trials with words in the title or abstract such as “prospective,” “comparative,” or “efficacy,” or when the title indicated that a comparison of treatment groups was undertaken prospectively, were analyzed further to establish whether randomization was implemented. Only studies involving humans were selected; theses, conference abstracts, and laboratory-based randomized trials involving extracted teeth were excluded from the analysis. Two authors (P.S.F. and N.B.) screened potentially relevant articles independently, and any disagreements were resolved by discussion with a third author (N.P.) to reach a consensus.
The reporting of abstracts with reference to the CONSORT for abstracts checklist was initially piloted by 2 authors (P.S.F. and N.B.). The authors were calibrated by assessing the reporting of 10 abstracts together by referring directly to the CONSORT checklist and associated explanations. Modifications were made to the checklist after the initial pilot with additional columns relating to structure, randomization procedures, allocation concealment, and reporting of P values. The checklist included 21 questions related to the CONSORT items. The score per item ranged from 1 to 3, with 1 representing “no description,” 2 corresponding to “inadequate description,” and 3 being “adequate description.” The scores for the 21 items were combined, and a percentage score was calculated for each abstract; a score of 63 corresponded to perfect reporting (100%). Each reporting item on the checklist for individual articles was scored independently by 2 observers, producing an overall CONSORT score for each article. Disagreements were resolved by consulting with a third author (N.P.). Further data from each study included the journal of publication, the number of authors, the country and continent of publication, the significance of the results, and the identification as a single-center or multi-center trial.
Statistical analysis
Descriptive statistics for each reporting item and each RCT were calculated and converted to a percentage scale. The calculated CONSORT scores for abstracts were approximately normally distributed. Inferential testing was carried out with linear regression modeling with univariate analysis to identify characteristics associated with the mean modified CONSORT score; multivariate modeling was used to determine the adjusted effect on the reporting quality score. Significant predictors identified during the multivariable analysis were entered one at a time in the multivariable model. The final model was derived by sequentially comparing candidate models by using the likelihood ratio test. A 2-tailed P value of 0.05 was considered statistically significant with a 95% confidence interval. All analyses were performed with the Stata software (version 12.1; StataCorp, College Station, Tex).
Results
From January 1, 2006, to December 31, 2011, a total of 117 RCTs were identified in the 4 leading orthodontic journals ( Table I ). Most of the RCTs were published in either the American Journal of Orthodontics and Dentofacial Orthopedics (53.0%) or the Angle Orthodontist (23.0%). Most of the research was conducted in Europe (64.1%). The vast majority of RCTs were single-center (86.3%), with slightly more studies reporting significant main outcomes (58.1%) than not. In relation to the quality items, most abstracts (85.5%) were structured with clear reporting of interventions (97.4%), objectives (93.2%), and numbers of participants randomized (95.7%). Conversely, insufficient reporting of the randomization procedures, allocation concealment, and blinding, and failure to report confidence intervals and harms, were almost universal. Registration of RCTs and sources of funding were not reported in any of the identified abstracts ( Tables I and II ).
| Characteristic | Category | n | % |
|---|---|---|---|
| Journals | JO | 11 | 9.4 |
| AJODO | 62 | 53.0 | |
| EJO | 17 | 14.6 | |
| AO | 27 | 23.0 | |
| Continents | Europe | 75 | 64.1 |
| Americas | 27 | 23.1 | |
| Asia | 15 | 12.8 | |
| Authors (n) | <4 | 43 | 36.8 |
| 4-7 | 70 | 59.8 | |
| >7 | 4 | 3.4 | |
| Centers | Single center | 101 | 86.3 |
| Multi-center | 16 | 13.7 | |
| Statistical significance of main finding | No | 49 | 41.9 |
| Yes | 68 | 58.1 |
| Item | No description % | Inadequate % | Adequate % |
|---|---|---|---|
| 1. Title | 45.3 | 39.3 | 15.4 |
| 2. Structured | 14.5 | 0.0 | 85.5 |
| 3. Trial design | 22.2 | 59.8 | 18.0 |
| 4. Participants | 9.4 | 35.9 | 54.7 |
| 5. Interventions | 0.0 | 2.6 | 97.4 |
| 6. Objectives | 1.7 | 5.1 | 93.2 |
| 7. Outcomes | 0.9 | 89.7 | 9.4 |
| 8. Random number generation | 28.2 | 67.5 | 4.3 |
| 9. Randomization restrictions | 97.4 | 0.0 | 2.6 |
| 10. Allocation concealment | 99.1 | 0.0 | 0.9 |
| 11. Blinding | 90.6 | 0.0 | 9.4 |
| 12. Numbers randomized | 4.3 | 0.0 | 95.7 |
| 13. Numbers analyzed | 74.4 | 0.0 | 25.6 |
| 14. Effect estimate | 59.8 | 21.4 | 18.8 |
| 15. Confidence intervals | 94.0 | 0.0 | 6.0 |
| 16. Intention-to-treat analysis or per- protocol analysis | 99.1 | 0.0 | 0.9 |
| 17. Harms | 98.3 | 0.0 | 1.7 |
| 18. Conclusions | 0.9 | 4.3 | 94.8 |
| 19. Registration | 100.0 | 0.0 | 0.0 |
| 20. Funding | 100.0 | 0.0 | 0.0 |
| 21. P value | 36.8 | 25.6 | 37.6 |
The mean overall reporting quality score was 60.2% (95% confidence interval [CI], 59.2-61.2). The highest score was noted in the Journal of Orthodontics (66%). Lower scores were found for the American Journal of Orthodontics and Dentofacial Orthopedics (61.1%), the European Journal of Orthodontics (57.5%), and the Angle Orthodontist (57.4%). Little difference in reporting quality was identified with respect to continent, number of authors, or significant findings ( Table III ). However, reporting was slightly better in the European studies and the multi-center studies.
| Characteristic | Category | Modified CONSORT score | |
|---|---|---|---|
| Mean (%) | 95% CI | ||
| Journals | JO | 66.0 | (63.5-68.7) |
| AJODO | 61.1 | (59.9-62.4) | |
| EJO | 57.5 | (55.0-60.0) | |
| AO | 57.4 | (56.0-58.8) | |
| Continents | Europe | 61.4 | (60.1-62.6) |
| Americas | 58.8 | (57.3-60.4) | |
| Asia | 57.0 | (55.3-58.7) | |
| Authors (n) | <4 | 59.9 | (58.2-61.7) |
| 4-7 | 60.3 | (59.1-61.5) | |
| >7 | 62.3 | (61.5-63.1) | |
| Centers | Single center | 59.7 | (58.7-60.8) |
| Multi-center | 63.4 | (61.7-65.1) | |
| Statistical significance of main finding | No | 60.4 | (58.7-62.1) |
| Yes | 60.1 | (58.9-61.2) | |
| Overall | 60.2 | (59.2-61.2) | |
Comparisons were made between baseline (reference category) and each potential predictor, including journal of publication, number of authors, continent of publication, date of publication (2006-2008 vs 2009-2011), single- or multi-center setting, and significant findings. In the univariate analyses, trials published in the Journal of Orthodontics had significantly higher reporting quality scores compared with the other 3 journals. Additionally, studies published in Europe and multi-center studies ( P <0.05) had significantly higher modified CONSORT scores. There was little difference in reporting quality on an annual basis from 2006 to 2011 ( Fig 2 ), and no statistical difference in the abstract reporting quality since 2009 relative to the earlier period (0.6%; 95% CI, –1.3 to 2.5; P = 0.54). Multivariate analysis demonstrated that reporting quality was significantly better in the Journal of Orthodontics than in the other journals ( Table IV ). In particular, abstract reporting was significantly better in the Journal of Orthodontics than the European Journal of Orthodontics (–8.5%; 95% CI, –11.9 to –5.1; P <0.001) and the Angle Orthodontist (–7.2%; 95% CI, –10.4 to –4.0; P <0.001). Scores in the American Journal of Orthodontics and Dentofacial Orthopedics were 4.2% lower on average than in the Journal of Orthodontics (95% CI, –7.2 to –1.3; P <0.01). The adjusted model indicated that, compared with Europe, abstract quality reporting was significantly lower in studies based in Asia (–3.5%; 95% CI, –6.1 to –0.8; P <0.05). No significant associations were found in the multivariate model for other variables including number of authors, number of centers, or significance of main outcome measures.
