The use of tailored subheadings was successful in enhancing compliance with CONSORT in a dental journal

Abstract

Objectives

Efforts to enhance the reporting of clinical trials have intensified in recent years with automated strategies and editorial involvement showing promise in improving compliance with accepted guidelines. This study aimed to evaluate the effectiveness of a concerted approach to adherence to CONSORT (CONsolidated Standards Of Reporting Trials) guidelines in a dental journal.

Materials and methods

Following the publication of an exemplar clinical trial on the American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO) website and related changes to the author guidelines, trial submissions were required to follow a standard format incorporating subheadings mirroring the CONSORT guidelines. Compliance with CONSORT was assessed in initial submissions over a 30-month period. Reporting was compared to submissions of randomized controlled trials (RCTs) which did not include subheadings over the same period.

Results

Seventy-one RCTs were submitted to the AJO-DO from January 2014 to June 2016, 49 with subheadings and 22 without. Most CONSORT items (e.g. random sequence generation, allocation concealment and blinding) were more frequently adequately reported when RCTs were submitted with inclusion of subheadings. Overall, reporting quality of the submitted RCTs was 15.2% higher with use of the subheadings format (95%CI: 10.5, 20.0; p < 0.001) with a mean overall score of 87.3%.

Conclusion

Enhanced compliance of submitted RCTs was found with use of a bespoke approach to trial presentation utilizing CONSORT item subheadings. The improvement in initial submissions is particularly encouraging as this arose without input either from peer reviewers or journal editors. This simple approach may have wider applicability.

Introduction

The importance of transparent reporting of research studies not least randomized controlled trials (RCTs) is well-established. The CONsolidated Standards of Reporting Trials (CONSORT) guidelines were directed at informing the reporting of RCTs . CONSORT has been endorsed by most leading journals with authors encouraged to adhere to recommendations within their submissions. Moreover, numerous modifications to CONSORT have been made to account for variations in trial design, setting and outcomes .

While CONSORT has become established and accepted, a plethora of research from biomedical specialties suggests that the published literature remains beset by poor reporting . In particular, reporting of trials published in the medical and dental literature have been suboptimal and, while some benefit has accrued from endorsement of reporting guidelines , expose the ineffectiveness of existing passive approaches to CONSORT compliance with more innovative or involved systems required to optimise reporting. This has spawned a range of initiatives including more involved editorial processes and automated means of improving reporting , with increased editorial involvement leading to considerable improvement in reporting of clinical trials within in a dental specialty journal .

The American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO) has been proactive in promoting reporting in accordance with CONSORT with the approach to enhancing CONSORT compliance evolving in recent years. Initially (from June 2011), a systematic process involving the editor-in-chief (EiC), an associate editor (AE) and RCT authors was adopted whereby initial RCT submissions were first assessed by the AE to ensure all CONSORT items were reported completely following transfer from the EiC. The AE replied to the authors listing unreported items and highlighting ways to address incompletely reported items prior to resubmission. Resubmitted manuscripts were again scrutinized by the AE for CONSORT adherence and then sent for standard peer review. This initiative led to near complete reporting of most CONSORT items in published articles but did require significant input from the editorial team . Similar approaches may not, however, be applicable to other journals in view of the increased onus on editorial, time, input and expertise. In order to improve reporting at the submission level, the approach changed in January 2014 with the adoption of a publication template incorporating 20 subheadings corresponding to the 27 CONSORT items. A model clinical trial report and a specialty specific CONSORT explanation paper providing the rationale for reporting of individual items were also published .

As such, the present study examines the impact the use of subheadings based on CONSORT items on the reporting of clinical trials submitted to a dental specialty journal. We also aimed to highlight areas where deficient reporting within initial submissions exists.

Methods

The completeness of reporting was compared between RCTs submitted to AJO-DO with or without subheadings based on the CONSORT items. Both cohorts included all RCTs submitted for publication between the January 2014 and June 2016. The sample was divided based on whether the submission was presented in-line with suggested subheadings ( Table 1 ) . No inclusion restrictions in terms of trial design were applied. Two experienced researchers were involved in the data extraction process. Data from 10 papers were extracted independently and consensus was reached for training and calibration purposes. Consequently, the first author extracted all data from the remaining papers.

Table 1
List of recommended subheadings based on CONSORT diagrams.
1 Title
2 Structured abstract
Introduction
3 Background
4 Specific objectives or hypotheses
Methods
5 Trial design and any changes after trial commencement
6 Participants, eligibility criteria, and settings
7 Interventions
8 Outcomes (primary and secondary) and any changes after trial commencement
9 Sample size calculation
10 Interim analysis and stopping guidelines
11 Randomization (random number generation, allocation concealment, implementation)
12 Blinding
13 Statistical analysis (primary and secondary outcomes, subgroup analysis)
Results
14 Participant flow (include flow diagram, early stopping and time periods)
15 Baseline data (include baseline table)
16 Numbers analyzed for each outcome, estimation and precision, subgroup analyses
17 Harms
Discussion
18 Main findings in the context of the existing evidence, interpretation
19 Limitations
20 Generalizability
Registration
Protocol
Funding

RCT reports were assessed based on whether they reported all items completely. Items were rated as either not reported, incompletely reported, or completely reported. Furthermore, a scoring system was used based on a modified and expanded CONSORT item list for the evaluation of quality of reporting, in line with a previous analysis . Scores for each item ranged from 1 to 3 with 1 indicating no description, a score of 2 representing inadequate description, and 3 indicative of adequate description. The scores for all 37 items were combined and a percentage score was derived. Non-applicable items did not receive any score. For example, the maximum for a trial with adequate description for all items was 111 corresponding to 100%. On the same basis, an RCT with 30 applicable items could receive a maximum score of 90, also equivalent to 100%. Subsequently, each item was converted to a binary variable to enable comparison of RCTs submitted with subheadings to those without subheadings for each reporting item using a dichotomous measure (i.e. adequate reporting vs no reporting/inadequate reporting).

Descriptive statistics and frequency distributions of compliance with CONSORT items were reported for both cohorts. A t -test was conducted to identify differences in reporting scores between submissions with and without subheadings. All analyses were carried out using Stata 14.1 (Stata Corp, College Station, TX, USA), with the level of statistical significance set at p = 0.05.

Methods

The completeness of reporting was compared between RCTs submitted to AJO-DO with or without subheadings based on the CONSORT items. Both cohorts included all RCTs submitted for publication between the January 2014 and June 2016. The sample was divided based on whether the submission was presented in-line with suggested subheadings ( Table 1 ) . No inclusion restrictions in terms of trial design were applied. Two experienced researchers were involved in the data extraction process. Data from 10 papers were extracted independently and consensus was reached for training and calibration purposes. Consequently, the first author extracted all data from the remaining papers.

Table 1
List of recommended subheadings based on CONSORT diagrams.
1 Title
2 Structured abstract
Introduction
3 Background
4 Specific objectives or hypotheses
Methods
5 Trial design and any changes after trial commencement
6 Participants, eligibility criteria, and settings
7 Interventions
8 Outcomes (primary and secondary) and any changes after trial commencement
9 Sample size calculation
10 Interim analysis and stopping guidelines
11 Randomization (random number generation, allocation concealment, implementation)
12 Blinding
13 Statistical analysis (primary and secondary outcomes, subgroup analysis)
Results
14 Participant flow (include flow diagram, early stopping and time periods)
15 Baseline data (include baseline table)
16 Numbers analyzed for each outcome, estimation and precision, subgroup analyses
17 Harms
Discussion
18 Main findings in the context of the existing evidence, interpretation
19 Limitations
20 Generalizability
Registration
Protocol
Funding

RCT reports were assessed based on whether they reported all items completely. Items were rated as either not reported, incompletely reported, or completely reported. Furthermore, a scoring system was used based on a modified and expanded CONSORT item list for the evaluation of quality of reporting, in line with a previous analysis . Scores for each item ranged from 1 to 3 with 1 indicating no description, a score of 2 representing inadequate description, and 3 indicative of adequate description. The scores for all 37 items were combined and a percentage score was derived. Non-applicable items did not receive any score. For example, the maximum for a trial with adequate description for all items was 111 corresponding to 100%. On the same basis, an RCT with 30 applicable items could receive a maximum score of 90, also equivalent to 100%. Subsequently, each item was converted to a binary variable to enable comparison of RCTs submitted with subheadings to those without subheadings for each reporting item using a dichotomous measure (i.e. adequate reporting vs no reporting/inadequate reporting).

Descriptive statistics and frequency distributions of compliance with CONSORT items were reported for both cohorts. A t -test was conducted to identify differences in reporting scores between submissions with and without subheadings. All analyses were carried out using Stata 14.1 (Stata Corp, College Station, TX, USA), with the level of statistical significance set at p = 0.05.

Results

Over the 30-month period (January 2014–June 2016), a total of 71 RCTs were submitted to the AJO-DO and assessed for completeness of reporting. Forty- nine of these initial submissions incorporated subheadings, while 22 did not. The majority of submission were from authors in Asia/Other regions (n = 41; 58%), with 23% (n = 16) from the Americas and 20% (n = 14) from Europe.

A number of pivotal items were more frequently adequately reported when RCTs were submitted with inclusion of subheadings ( Table 2 ). In particular, sample size (subheadings: 25/49, 51%; no subheadings: 6/22, 27%), random sequence generation (subheadings: 33/49, 67% vs no subheadings: 9/22, 41%), allocation concealment (subheadings: 21/49, 43%; no subheadings: 4/22, 18%), blinding (subheadings: 42/49, 86% vs no subheadings: 12/22, 55%), CONSORT flow diagram (subheadings: 40/49, 82% vs no subheadings: 9/22, 41%), baseline information table (subheadings: 37/49, 76%; no subheadings: 11/22, 50%), study limitations (subheadings: 42/49, 86%; no subheadings: 8/22, 36%) and generalizability (subheadings: 38/49, 78% vs no subheadings: 3/22, 14%) were more commonly adequately reported in submissions incorporating subheadings. Notwithstanding this, items such as sample size, allocation concealment or estimates/confidence intervals remain in need of a considerable improvement in reporting ( Table 2 ).

Table 2
Frequency distributions on reporting of items based on the CONSORT checklist and included in the subheadings, according to the structure of the manuscript at initial submission.
Presence of subheadings at submission stage
No Yes Total
No. % No. % No. %
Title
Adequate 6 27 46 94 52 73
Inadequate 16 73 3 6 19 27
Structured Abstract
Adequate 4 18 42 86 46 65
Inadequate 18 82 7 14 25 35
Background
Adequate 19 86 48 98 67 94
Inadequate 3 14 1 2 4 6
Introduction
Adequate 18 82 47 96 65 92
Inadequate 4 18 2 4 6 8
Objective
Adequate 20 91 48 98 68 96
Inadequate 2 9 1 2 3 4
Trial Design
Adequate 11 50 41 84 52 73
Inadequate 8 36 8 16 16 23
No description 3 14 0 0 3 4
Changes after trial commencement
Adequate 0 0 2 4 2 3
Non-applicable 21 95 47 96 68 96
No description 1 5 0 0 1 1
Eligibility criteria
Adequate 22 100 49 100 71 100
Settings
Adequate 18 82 48 98 66 93
Inadequate 2 9 1 2 3 4
No description 2 9 0 0 2 3
Interventions
Adequate 22 100 48 98 70 99
Inadequate 0 0 1 2 1 1
Outcomes
Adequate 21 95 47 96 68 96
Inadequate 1 5 1 2 2 3
No description 0 0 1 2 1 1
Outcome changes
Adequate 0 0 1 2 1 1
Non-applicable 22 100 48 98 70 99
Sample Size
Adequate 6 27 25 51 31 44
Inadequate 14 64 23 47 37 52
No description 2 9 1 2 3 4
Interim analysis
Adequate 0 0 2 4 2 3
Non-applicable 22 100 47 96 69 97
Random number generation
Adequate 9 41 33 67 42 59
Inadequate 10 45 16 33 26 37
No description 3 14 0 0 3 4
Restrictions in randomization
Adequate 6 27 13 27 19 27
Inadequate 0 0 4 8 4 6
No description 16 73 32 65 48 68
Allocation Concealment
Adequate 4 18 21 43 25 35
Inadequate 6 27 15 31 21 30
No description 12 55 13 27 25 35
Implementation of Randomization
Adequate 8 36 27 55 35 49
Inadequate 1 5 14 29 15 21
No description 13 59 8 16 21 30
Blinding
Adequate 12 55 42 86 54 76
Inadequate 2 9 3 6 5 7
Non-applicable 1 5 2 4 3 4
No description 7 32 2 4 9 13
Similarity of Interventions
Adequate 3 14 6 12 9 13
Inadequate 1 5 1 2 2 3
Non-applicable 16 73 40 82 56 79
No description 2 9 2 4 4 6
Statistical Methods
Adequate 13 59 33 67 46 65
Inadequate 9 41 16 33 25 35
Subgroup analysis
Adequate 2 9 2 4 4 6
Inadequate 1 5 2 4 3 4
Non-applicable 19 86 44 90 63 89
No description 0 0 1 2 1 1
Flow Diagram with reason
Adequate 9 41 40 82 49 69
Inadequate 2 9 9 18 11 15
No description 11 50 0 0 11 15
Recruitment Dates
Adequate 7 32 41 84 48 68
No description 15 68 8 16 23 32
Premature trial stop
Adequate 0 0 1 2 1 1
Non-applicable 22 100 48 98 70 99
Baseline Table
Adequate 11 50 37 76 48 68
Inadequate 4 18 8 16 12 17
No description 7 32 4 8 11 15
Intention-to-treat
Adequate 2 9 6 12 8 11
Inadequate 2 9 5 10 7 10
No description 18 82 38 78 56 79
Estimates/Confidence Intervals
Adequate 3 14 10 20 13 18
Inadequate 0 0 8 16 8 11
No description 19 86 31 63 50 70
If Binary Outcome/absolute numbers
Adequate 1 5 6 12 7 10
Non-applicable 21 95 43 88 64 90
Ancillary analysis
Adequate 0 0 1 2 1 1
Inadequate 1 5 1 2 2 3
Non-applicable 21 95 47 96 68 96
Harms
Adequate 4 18 32 65 36 51
Inadequate 3 14 2 4 5 7
Non-applicable 3 14 12 24 15 21
No description 12 55 3 6 15 21
Limitations
Adequate 8 36 42 86 50 70
Inadequate 3 14 1 2 4 6
No description 11 50 6 12 17 24
Generalizability
Adequate 3 14 38 78 41 58
Inadequate 3 14 3 6 6 8
No description 16 73 8 16 24 34
Interpretation
Adequate 10 45 43 88 53 75
Inadequate 11 50 6 12 17 24
No description 1 5 0 0 1 1
Registration
Adequate 5 23 8 16 13 18
Inadequate 0 0 2 4 2 3
Non-applicable 5 23 33 67 38 54
No description 12 55 6 12 18 25
Protocol
Adequate 3 14 5 10 8 11
Inadequate 1 5 2 4 3 4
Non-applicable 6 27 37 76 43 61
No description 12 55 5 10 17 24
Funding
Adequate 8 36 17 35 25 35
Non-applicable 5 23 28 57 33 46
No description 9 41 4 8 13 18
Total 22 100 49 100 71 100
Only gold members can continue reading. Log In or Register to continue

Jun 17, 2018 | Posted by in General Dentistry | Comments Off on The use of tailored subheadings was successful in enhancing compliance with CONSORT in a dental journal
Premium Wordpress Themes by UFO Themes