Abstract
Objectives
This study aimed to compare the reporting quality of randomized controlled trial (RCT) abstracts in prosthodontics before and after the publication of Consolidated Standards of Reporting Trials (CONSORT) guideline for abstracts and identify the characteristics associated with better reporting quality.
Sources
PubMed was searched for RCT abstracts published from 2001 to 2007 (pre-CONSORT period) and from 2010 to 2016 (post-CONSORT period) in six leading prosthodontic journals.
Study selection
After applying the inclusion/exclusion criteria, 131 RCT abstracts were selected. The t test was performed to compare the overall quality between the two periods. Univariable and multivariable linear regressions were used to identify any factors relating to the reporting quality. The level of significance was set at P < 0.05.
Data
The investigators extracted data and scored the abstracts independently based on CONSORT. The mean overall CONSORT score was 5.20 and 6.11 in the pre- and post-CONSORT samples, respectively. Significant changes were observed in reporting for only three items: title, conclusions, and trial registration. Most abstracts adequately reported interventions, objectives, and conclusions (>90%), but failed to report recruitment and outcome in the results section (<3%). Funding was not reported in both periods. The reporting quality was related to a higher impact factor, structured format, and published after CONSORT.
Conclusions
The quality of RCT abstracts in prosthodontics improved over time, but adherence to the CONSORT guideline for abstracts was still suboptimal.
1
Introduction
Prosthodontics is a specific discipline in dental education. In April 2003, the American Dental Association defined it as “the dental specialty pertaining to the diagnosis, treatment planning, rehabilitation and maintenance of the oral function, comfort, appearance and health of patients with clinical conditions associated with missing or deficient teeth or oral and maxillofacial tissues using biocompatible substitutes .” Research in prosthodontics is normally focused on developing new techniques, treatment modalities, and biomaterials with varying physical or chemical properties .
Randomized controlled trials (RCTs) are considered the highest-grade evidence in the hierarchy of research designs and represent the main source of current clinical guidelines . Frequently, RCTs are first presented in an abstract form . Constructed and well-written abstracts help individuals to assess quickly the validity and applicability of the findings and aid the retrieval of reports from electronic databases . Readers often base their assessment of a trial solely on the information in abstracts and then decide whether to seek more details about it . This could be because of the lack of access to full texts, time limitations, or inability to understand the technicalities of clinical trials.
Previous studies showed deficiencies in reporting RCT abstracts . Sometimes, the content of an abstract and the full text of the article were inconsistent . Incomplete reporting of studies in their abstracts can lead to inaccurate interpretation of results, missed identification of potential sources of bias, and missed inclusion in systematic reviews . The Consolidated Standards of Reporting Trials (CONSORT) extension for abstracts was published in 2008 to encourage better informative reporting of abstracts, providing a checklist of items to be included in journal or conference abstracts reporting RCTs . This statement comprised 17 items distributed in 8 sections, endorsed by the World Association of Medical Editors, the International Committee of Medical Journal Editors, and the Council of Science Editors .
The CONSORT guideline was published to aid the authors provide the details and clarity needed by readers. Since its publication, studies examining the adherence to the recommendations in dentistry have been performed , including implant dentistry, periodontology, and orthodontics. However, the quality of abstracts in prosthodontics has not been evaluated. This study assumed some improvement after the publication of dedicated CONSORT statement extension. The objectives of the study were (1) to assess the overall reporting quality of RCT abstracts in prosthodontics from different periods and (2) to identify trial characteristics associated with better reporting quality.
2
Materials and methods
2.1
Study selection
Six leading prosthodontic journals were selected based on the 2015 Journal Citation Report : Journal of Dentistry (JOD; impact factor, 3.109), Journal of Oral Rehabilitation (JOR; 1.926), Journal of Prosthetic Dentistry (JPD; 1.515), The International Journal of Prosthodontics (IJOP; 1.487), Journal of Esthetic and Restorative Dentistry (JERD; 1.231), and Journal of Prosthodontics (JOP; 1.133). A lag period of 2 years was considered appropriate after the release of the CONSORT statements , allowing a sufficient time for authors and editors to incorporate the recommendations into practice. Search results were then limited to abstracts published from January 2001 to December 2007 (pre-CONSORT period), and from January 2010 to December 2016 (post-CONSORT period).
2.2
Search strategy
Two reviewers (CJS and LZ) used an extended version of the Cochrane Highly Sensitive Search Strategy for the retrieval of RCTs using PubMed . A search form was designed to supply the details of the search ( Appendix A ). The RCTs were manually sought from journals published in 2004 and 2013 to confirm that the search was sensitive enough to involve all RCTs published within a given period, and no study was missed. The search protocol is summarized in the PRISMA flowchart shown in Fig. 1 . If the abstract of a potentially relevant article was unclear, the full text was scrutinized to decide whether it should be included. A consensus process was used to resolve disagreements. Ultimately, 131 abstracts meeting the inclusion criteria were included, all of which were collated into the EndNote (version X7; Thomson Reuters). The journal title and author affiliations were removed to allow for blinded quality assessment.
2.3
Pilot study
A pilot study of reviewers’ calibration was performed in a standardized manner to ensure the uniformity and develop an understanding of checklist items. The online randomization software ( http://www . randomizer.org) was used to select abstracts from the list of RCTs in PubMed, which was published before ( n = 10) and after ( n = 10) the CONSORT guideline for abstracts . Then, the 20 abstracts were divided into 2 equal parts randomly and evaluated by 2 reviewers, independently and in duplicate. The Cohen κ statistic was used to calculate the interobserver agreement , which was regarded as excellent with κ > 0.75, fair to good with κ 0.40–0.75, and poor with κ < 0.40 .
2.4
Data extraction and evaluation
Data extraction was performed by two reviewers (CJS and LZ). Among the quality items of the original CONSORT for abstract checklist, one item (authors) was designed specifically for conference abstracts and, therefore, excluded from the assessment. The extraction was done independently and in duplicate. Any uncertainty related to a particular abstract was resolved by discussion with a third reviewer (LCJ). An overall quality score (OQS, range 0–16) was developed by summarizing the individual score (1/0.5/0) across all 16 items. The item was scored 1 if it was well reported, 0 if it was not reported, and 0.5 if it was inadequately reported, just for the items having subtitles (at least one subtitle was adequately reported). Other trial information was extracted as potential predictors of reporting quality: journal name, publication date, region, trial outcome, number of authors, word count, total sample size, structure format, exact P value, and multicenter versus single-center trial ( Appendix B ).
2.5
Statistical analysis
Descriptive statistics were performed. The chi-square test (or Fisher exact test, depending on the number of entries) was applied for each item to test the independence of the two samples. The Student t test was conducted to assess the improvement in OQS in the two periods. Univariable and multivariable linear regression models were used to identify the characteristics associated with a higher OQS. Predictors identified during the univariable analysis were entered one at a time in the multivariable model, and nonsignificant variables were deleted by backward elimination. Tolerance and variance inflation factor (VIF) were used to detect multicollinearity, and any predictor with a tolerance less than 0.1 or a VIF more than 10 was excluded from the final model. All tests of statistical significance were two sided, with the threshold set at P < 0.05 and a 95% confidence interval (CI). Data for descriptive statistics were analyzed using Microsoft Excel 2016 and SPSS (version 19.0; IBM Corporation, NY, USA).
2
Materials and methods
2.1
Study selection
Six leading prosthodontic journals were selected based on the 2015 Journal Citation Report : Journal of Dentistry (JOD; impact factor, 3.109), Journal of Oral Rehabilitation (JOR; 1.926), Journal of Prosthetic Dentistry (JPD; 1.515), The International Journal of Prosthodontics (IJOP; 1.487), Journal of Esthetic and Restorative Dentistry (JERD; 1.231), and Journal of Prosthodontics (JOP; 1.133). A lag period of 2 years was considered appropriate after the release of the CONSORT statements , allowing a sufficient time for authors and editors to incorporate the recommendations into practice. Search results were then limited to abstracts published from January 2001 to December 2007 (pre-CONSORT period), and from January 2010 to December 2016 (post-CONSORT period).
2.2
Search strategy
Two reviewers (CJS and LZ) used an extended version of the Cochrane Highly Sensitive Search Strategy for the retrieval of RCTs using PubMed . A search form was designed to supply the details of the search ( Appendix A ). The RCTs were manually sought from journals published in 2004 and 2013 to confirm that the search was sensitive enough to involve all RCTs published within a given period, and no study was missed. The search protocol is summarized in the PRISMA flowchart shown in Fig. 1 . If the abstract of a potentially relevant article was unclear, the full text was scrutinized to decide whether it should be included. A consensus process was used to resolve disagreements. Ultimately, 131 abstracts meeting the inclusion criteria were included, all of which were collated into the EndNote (version X7; Thomson Reuters). The journal title and author affiliations were removed to allow for blinded quality assessment.
2.3
Pilot study
A pilot study of reviewers’ calibration was performed in a standardized manner to ensure the uniformity and develop an understanding of checklist items. The online randomization software ( http://www . randomizer.org) was used to select abstracts from the list of RCTs in PubMed, which was published before ( n = 10) and after ( n = 10) the CONSORT guideline for abstracts . Then, the 20 abstracts were divided into 2 equal parts randomly and evaluated by 2 reviewers, independently and in duplicate. The Cohen κ statistic was used to calculate the interobserver agreement , which was regarded as excellent with κ > 0.75, fair to good with κ 0.40–0.75, and poor with κ < 0.40 .
2.4
Data extraction and evaluation
Data extraction was performed by two reviewers (CJS and LZ). Among the quality items of the original CONSORT for abstract checklist, one item (authors) was designed specifically for conference abstracts and, therefore, excluded from the assessment. The extraction was done independently and in duplicate. Any uncertainty related to a particular abstract was resolved by discussion with a third reviewer (LCJ). An overall quality score (OQS, range 0–16) was developed by summarizing the individual score (1/0.5/0) across all 16 items. The item was scored 1 if it was well reported, 0 if it was not reported, and 0.5 if it was inadequately reported, just for the items having subtitles (at least one subtitle was adequately reported). Other trial information was extracted as potential predictors of reporting quality: journal name, publication date, region, trial outcome, number of authors, word count, total sample size, structure format, exact P value, and multicenter versus single-center trial ( Appendix B ).
2.5
Statistical analysis
Descriptive statistics were performed. The chi-square test (or Fisher exact test, depending on the number of entries) was applied for each item to test the independence of the two samples. The Student t test was conducted to assess the improvement in OQS in the two periods. Univariable and multivariable linear regression models were used to identify the characteristics associated with a higher OQS. Predictors identified during the univariable analysis were entered one at a time in the multivariable model, and nonsignificant variables were deleted by backward elimination. Tolerance and variance inflation factor (VIF) were used to detect multicollinearity, and any predictor with a tolerance less than 0.1 or a VIF more than 10 was excluded from the final model. All tests of statistical significance were two sided, with the threshold set at P < 0.05 and a 95% confidence interval (CI). Data for descriptive statistics were analyzed using Microsoft Excel 2016 and SPSS (version 19.0; IBM Corporation, NY, USA).
3
Results
3.1
Agreement of reviewers
In the pilot study, the interobserver concordance had a κ value of 0.63 (fair to good), which was 0.81 (95% CI, 0.73 to 0.92) after resolving all disagreements by a discussion with the third reviewer (LCJ), indicating almost perfect agreement.
3.2
Characteristics of included abstracts
A flow diagram of the literature search is depicted in Fig. 1 . After applying the inclusion/exclusion criteria, 131 records were included (pre-CONSORT = 44, post-CONSORT = 87, N = 131). As shown in Table 1 , the number of RCTs published in each journal was as follows: IJOP ( n , n %; 43, 32.82%), JOR (36, 27.48%), JOD (21, 16.03%), JPD (14, 10.69%), JOP (13, 9.92%), and JERD (4, 3.05%). The vast majority of RCTs were performed in a single center (122, 93.13%), with the number of authors between 4 and 7 (85, 64.89%). Slightly more studies were structured (94, 71.76%), and did not report exact P value (87, 66.41%), with a sample size of more than 50 (65, 49.62%). Besides, the research preferred studies with reporting of a positive outcome (75, 57.25%), usually conducted in Europe (60, 45.80%), and with 200–250 words (56, 42.75%).
| Characteristic | Category | Pre-CONSORT(N = 44), n (%) | Post-CONSORT (N = 87), n (%) | All (N = 131), n (%) |
|---|---|---|---|---|
| Journal | JOD | 6 (13.64) | 15 (17.24) | 21 (16.03) |
| JOP | 1 (2.27) | 12 (13.79) | 13 (9.92) | |
| JERD | 2 (4.55) | 2 (2.30) | 4 (3.05) | |
| IJOP | 21 (47.73) | 22 (25.29) | 43 (32.82) | |
| JPD | 4 (9.09) | 10 (11.49) | 14 (10.69) | |
| JOR | 10 (22.73) | 26 (29.89) | 36 (27.48) | |
| Region | Europe | 25 (56.82) | 35 (40.23) | 60 (45.80) |
| Americas | 14 (31.82) | 25 (28.74) | 39 (29.77) | |
| Asia | 4 (9.09) | 21 (24.14) | 25 (19.08) | |
| Africa-Australia | 1 (2.27) | 6 (6.90) | 7 (5.34) | |
| Number of authors | <4 | 15 (34.09) | 17 (19.54) | 32 (24.43) |
| 4–7 | 25 (56.82) | 60 (68.97) | 85 (64.89) | |
| >7 | 4 (9.09) | 10 (11.49) | 14 (10.69) | |
| Center | Single center | 41 (93.18) | 81 (93.10) | 122 (93.13) |
| Multicenter | 3 (6.82) | 6 (6.90) | 9 (6.87) | |
| Word count | <200 | 12 (27.27) | 11 (12.64) | 23 (17.56) |
| 200–250 | 21 (47.73) | 35 (40.23) | 56 (42.75) | |
| 250–300 | 6 (13.64) | 29 (33.33) | 35 (26.72) | |
| >300 | 5 (11.36) | 12 (13.79) | 17 (12.98) | |
| Sample size | <15 | 0 (0.00) | 1 (1.15) | 1 (0.76) |
| 15–30 | 6 (13.64) | 22 (25.29) | 28 (21.37) | |
| 30–50 | 9 (20.45) | 28 (32.18) | 37 (28.24) | |
| >50 | 29 (65.91) | 36 (41.38) | 65 (49.62) | |
| Structure format | Yes | 34 (77.27) | 60 (68.97) | 94 (71.76) |
| No | 10 (22.73) | 27 (31.03) | 37 (28.24) | |
| Exact p value | Yes | 14 (31.82) | 30 (34.48) | 44 (33.59) |
| No | 30 (68.18) | 57 (6o5.52) | 87 (66.41) | |
| Trial outcome | Positive | 25 (56.82) | 50 (57.47) | 75 (57.25) |
| Negative | 19 (43.18) | 37 (42.53) | 56 (42.75) | |
| JOD: Journal of Dentistry | ||||
| JOP: Journal of Prosthodontics | ||||
| JERD: Journal of Esthetic and Restorative Dentistry | ||||
| IJOP: The International Journal of Prosthodontics | ||||
| JPD: The Journal of Prosthetic Dentistry | ||||
| JOR: Journal of Oral Rehabilitation |
3.3
Reporting CONSORT items
Table 2 describes some of the features in the CONSORT checklist. In the general items, conspicuously more trials were identified in the title as “randomized” in the post-CONSORT period ( n = 57, 65.52%; P = 0.002). However, a few abstracts described details regarding the trial design in both periods [pre-CONSORT, n = 5, 11.4% ( n %); post-CONSORT, n = 15, 17.2% ( n %)].
| Items | Description | Pre-CONSORT (N = 44) number of studies (%) | Post-CONSORT (N = 87) number of studies (%) | P value C (Chi-squared) F (Fisher’s exact ) |
|---|---|---|---|---|
| Title | Identification of the study as randomized. | 16 (36.36) | 57 (65.52) | 0.002, C |
| Trial design | Description of the trial design | 5 (11.36) | 15 (17.24) | 0.377, C |
| Methods | ||||
| Participant | Eligibility criteria for participants and the settings where the data were collected a. Eligibility criteria b. Settings |
2 (4.55) | 6 (6.90) | 0.885, C |
| 26 (59.09) | 59 (67.82) | 0.323, C | ||
| 3 (6.82) | 6 (6.90) | 1.00, C | ||
| Interventions | Interventions intended for each group | 43 (97.73) | 87 (100.00) | 0.336, F |
| Objective | Specific objective or hypothesis | 43 (97.73) | 86 (98.95) | 1.00, C |
| Outcome | Clearly defined primary outcome for this report | 1 (2.27) | 6 (6.90) | 0.484, C |
| Randomization | How participants were allocated to interventions | 0 (0.00) | 0 (0.00) | NA |
| a. Random assignment | 42 (95.45) | 83 (95.40) | 1.00, C | |
| b. Sequence generation | 2 (4.55) | 1 (1.15) | 0.543, C | |
| c. Allocation concealment | 1 (2.27) | 1 (1.15) | 1.00, C | |
| Blinding (masking) | Whether or not participants, caregivers, and those assessing the outcomes were blinded to group assignment a. Only described single-blind or double-blind | 3 (6.82) | 1 (1.15) | 0.214, C |
| 2 (4.55) | 12 (13.79) | 0.187, C | ||
| Results | ||||
| Numbers randomized | Number of participants randomly assigned to each group | 23 (52.27) | 56 (64.37) | 0.181, C |
| Recruitment | Trial status | 0 (0.00) | 2 (2.30) | 0.550, F |
| Numbers analyzed | Number of participants analyzed in each group a. Intention-to-treat analysis | 9 (20.45) | 17 (19.54) | 0.901, C |
| 0 (0.00) | 2 (2.30) | 0.550, F | ||
| Outcome | For the primary outcome, a result for each group and the estimated effect size and its precision | 0 (0.00) | 1 (1.15) | 1.00, F |
| a. Primary outcome result for each group | 0 (0.00) | 6 (6.90) | 0.097, F | |
| b. Estimated effect size and its precision | 0 (0.00) | 1 (1.15) | 1.00, F | |
| Harms | Important adverse events or side effects | 3 (6.82) | 15 (17.24) | 0.171, C |
| Conclusions | General interpretation of the results | 41 (93.18) | 87 (100.00) | 0.036, F |
| Trial registration | Registration number and name of trial register | 0 (0.00) | 9 (10.34) | 0.028, F |
| Funding | Source of funding | 0 (0.00) | 0 (0.00) | NA |
Stay updated, free dental videos. Join our Telegram channel
VIDEdental - Online dental courses
