Introduction
In this study, we aimed to investigate whether studies published in orthodontic journals and titled as randomized clinical trials are truly randomized clinical trials. A second objective was to explore the association of journal type and other publication characteristics on correct classification.
Methods
American Journal of Orthodontics and Dentofacial Orthopedics, European Journal of Orthodontics, Angle Orthodontist, Journal of Orthodontics, Orthodontics and Craniofacial Research, World Journal of Orthodontics, Australian Orthodontic Journal, and Journal of Orofacial Orthopedics were hand searched for clinical trials labeled in the title as randomized from 1979 to July 2011. The data were analyzed by using descriptive statistics, and univariable and multivariable examinations of statistical associations via ordinal logistic regression modeling (proportional odds model).
Results
One hundred twelve trials were identified. Of the included trials, 33 (29.5%) were randomized clinical trials, 52 (46.4%) had an unclear status, and 27 (24.1%) were not randomized clinical trials. In the multivariable analysis among the included journal types, year of publication, number of authors, multicenter trial, and involvement of statistician were significant predictors of correctly classifying a study as a randomized clinical trial vs unclear and not a randomized clinical trial.
Conclusions
From 112 clinical trials in the orthodontic literature labeled as randomized clinical trials, only 29.5% were identified as randomized clinical trials based on clear descriptions of appropriate random number generation and allocation concealment. The type of journal, involvement of a statistician, multicenter trials, greater numbers of authors, and publication year were associated with correct clinical trial classification. This study indicates the need of clear and accurate reporting of clinical trials and the need for educating investigators on randomized clinical trial methodology.
A clinical trial is a preplanned experiment on humans that aims to assess the effects or benefits of at least 1 treatment. A randomized clinical trial (RCT) uses randomization and is considered the gold standard for assessing intervention effects. Randomization reduces selection bias by randomly assigning interventions and creating similar treatment groups that are in all respects similar except for the intervention and in which any observed differences between treatment groups are expected to be due to chance.
Randomization in clinical trial methodology has a specific meaning and is implemented by using explicit procedures consisting of the following 3 steps: generation of random allocation, allocation concealment, and implementation of randomization. It should be clearly understood that frequently used terms in the orthodontic literature to describe randomization such as consequent allocation , randomly assigned , and allocation based on some deterministic measure such as date of birth, day of the week, or file number are not considered true randomization approaches in RCT methodology and have been associated with biased results. Random number generation is usually produced by using random tables or software algorithms capable of producing random sequences. Several methods such as simple and permuted block randomization, stratified randomization, and minimization have been described as useful in orthodontic research. Simple randomization resembles a toss of a coin, and, although it is truly random, it is problematic especially in small trials because the probability of unequal treatment arms is high. Permuted blocks ensure balanced treatment groups of so many patients and solve the problem of simple randomization, whereas stratified randomization and minimization ensure balanced treatment groups on important prognostic factors of the outcome.
Allocation concealment ensures the unpredictability of treatment allocation by patients and investigators, and is always feasible, unlike blinding, which is often difficult to implement in orthodontic trials. Methods of allocation concealment can include the use of sequentially numbered sealed opaque envelopes, computerized allocation, or allocation assigned from an external center. The best method to achieve allocation concealment is to use an independent and centralized assignment protocol that does not involve any persons associated with the trial. With this method, the randomization lists are generated and held securely in remote locations, thus reducing the chance of peeking at treatment assignments and eliminating the chance of predicting the next treatment assignment.
Implementation of randomization pertains to who implemented the procedures of randomization, and when, where, and how these procedures (generation of randomization lists, allocation concealment, and treatment assignment) were implemented. In small studies, it is likely that the same investigators will perform all tasks. However, it would be best if different persons performed the various randomization procedures, since this would decrease the chance of biasing the process.
It has been reported that lack of appropriate randomization and allocation concealment might introduce bias and thus render the study results invalid. Among the various study designs, the RCT is considered the one with the potential to provide the highest quality evidence. However, there is evidence in the biomedical literature, including dentistry, that randomization and RCT quality, in general, are suboptimal, and often clinical trials described as RCTs are not really RCTs.
The term randomized clinical or controlled trial has become more common in the titles of articles published in orthodontic journals, but no study has assessed whether studies published as RCTs are truly RCTs. Therefore, the purposes of this study were to investigate the extent to which studies published in orthodontic journals with RCT in the title are mislabeled and to identify probable associations between journal type and other publication characteristics on correct classification.
Material and methods
The orthodontic literature was hand searched to identify original publications of clinical trials that indicated in the title that they were randomized. Studies that might have been RCTs but did not include terms in the title that indicated randomized treatment allocation were not considered. The following 6 orthodontic journals were searched by the first author (D.K.) to locate such articles published from 1979 to July 2011: American Journal of Orthodontics and Dentofacial Orthopedics , European Journal of Orthodontics , Angle Orthodontist , Journal of Orthodontics (formerly British Journal of Orthodontics ), Orthodontics and Craniofacial Research , World Journal of Orthodontics , (recently renamed Orthodontics. The Art and Practice of Dentofacial Enhancement ), Australian Orthodontic Journal , and Journal of Orofacial Orthopedics .
For Orthodontics and Craniofacial Research and World Journal of Orthodontics, the search was limited to 2000 to July 2011, since 2000 was the foundation year for both journals. The appropriateness of the randomization procedure for each included article was evaluated according the following: (1) random allocation sequence generation with details of any restriction such as blocking and block size, and (2) the allocation concealment mechanism for implementation of the random allocation sequence.
The classification was based on reporting without contacting the trial authors. A summary assessment for each included study was produced based on the description of the random allocation sequence generation and the allocation concealment. If the random allocation sequence generation and the allocation concealment were adequately described and appropriate, the trial was marked as an RCT. If the random allocation sequence generation or the allocation concealment was inadequately described, the trial was marked as unclear; if neither was described or was performed inappropriately, the trial was recorded as not an RCT.
The following additional characteristics for each article were also recorded: (1) number of authors, (2) geographic area (continent of the first author), (3) number of years from study publication, (4) involvement of a statistician (collaboration with an epidemiologist or a statistician was determined mainly by the author affiliation information), (5) single or multicenter study (assessed by the affiliation details and any information in the “MATERIAL AND METHODS” section), and (6) all retrieved articles were independently reviewed by 2 authors (D. K. and N. P.), and a consensus was reached on classification after discussion.
Statistical analysis
Frequency distributions of specific study characteristics in the clinical trial sample were examined. The purpose of the analysis was to determine the effect of journal type and other publication characteristics on the categorization of a study as an RCT, not an RCT, or unclear. Because the 3 category levels of the dependent variable were assumed to possess an ordinal element such as that unclear definition implies better randomization methods compared with not RCT and RCT implies better randomization methods compared with unclear definition, the data were analyzed by using ordinal logistic regression modeling (proportional odds model). Univariable and multivariable analyses were used to determine associations between publication characteristics and clinical trial categorizations. The Brant test was used to assess the proportional odds assumption, and comparisons of predicted probabilities with observed results were compared to assess model fit. A 2-tailed P value of 0.05 was considered to be statistically significant with a 95% confidence interval. All analyses were performed with STATA software (version 12.0; StataCorp, College Station, Tex).
Results
The number of published trials indicated in the title as RCTs in the orthodontic literature as a function of publication year are given in the Figure . Further results of this study are summarized in Tables I and II .
Category/unit | RCT | Unclear RCT definition | Not an RCT | Total | |
---|---|---|---|---|---|
n (% ∗ ) or mean (SD) | n (% ∗ ) or mean (SD) | n (% ∗ ) or mean (SD) | n (% ∗ ) or mean (SD) | ||
Journal | JO | 9 (45.00) | 9 (45.00) | 2 (10.00) | 20 (100.00) |
AJODO | 17 (36.96) | 20 (43.48) | 9 (19.57) | 46 (100.00) | |
EJO | 4 (18.18) | 10 (45.45) | 8 (36.36) | 22 (100.00) | |
Angle | 2 (18.18) | 6 (54.55) | 3 (27.27) | 11 (100.00) | |
Other | 1 (7.69) | 7 (53.85) | 5 (38.46) | 13 (100.00) | |
Time since publication | Years | 3.90 (3.22) | 4.17 (3.89) | 6.44 (5.60) | 4.64 (4.28) |
Country of authorship | Europe | 29 (33.72) | 38 (44.19) | 19 (22.09) | 86 (100.00) |
Americas | 4 (21.05) | 8 (42.11) | 7 (36.84) | 19 (100.00) | |
Asia | 0 (0.00) | 6 (85.71) | 1 (14.29) | 7 (100.00) | |
Number of authors | People | 5.93 (4.96) | 4.25 (3.11) | 4.03 (1.55) | 4.69 (3.57) |
Multicenter study | No | 15 (22.06) | 33 (48.53) | 20 (29.41) | 68 (100.00) |
Yes | 18 (40.91) | 19 (43.18) | 7 (15.91) | 44 (100.00) | |
Statistician involvement | No | 19 (21.11) | 45 (50.00) | 26 (28.89) | 90 (100.00) |
Yes | 14 (63.64) | 7 (31.82) | 1 (4.55) | 22 (100.00) | |
Total | 33 (29.46) | 52 (46.43) | 27 (24.11) | 112 (100.00) |
Category/unit | Univariable | Multivariable | |||||
---|---|---|---|---|---|---|---|
Odds ratio | 95% CI | P value | Odds ratio | 95% CI | P value | ||
Journal | JO | Referent | Referent | ||||
AJODO | 0.63 | 0.23-1.72 | 0.37 | 0.18 | 0.05-0.67 | 0.01 | |
EJO | 0.24 | 0.07-0.77 | 0.01 | 0.10 | 0.02-0.40 | <0.01 | |
Angle | 0.31 | 0.07-1.24 | 0.09 | 0.15 | 0.03-0.74 | 0.02 | |
Other | 0.18 | 0.04-0.68 | 0.01 | 0.17 | 0.04-0.73 | 0.01 | |
Time since publication | 1 year | 0.90 | 0.83-0.98 | 0.02 | 0.83 | 0.74-0.92 | <0.01 |
Country of authorship | Europe | Referent | Referent | ||||
Americas | 0.48 | 0.18-1.25 | 0.13 | 0.40 | 0.13-1.26 | 0.12 | |
Asia | 0.55 | 0.15-2.01 | 0.35 | 0.32 | 0.06-1.58 | 0.16 | |
Number of authors | People | 1.12 | 1.00-1.25 | 0.04 | 1.14 | 1.0-1.30 | 0.04 |
Multicenter study | No | Referent | Referent | ||||
Yes | 2.35 | 1.13-4.87 | 0.02 | 2.35 | 0.98-5.67 | 0.05 | |
Statistician involvement | No | Referent | Referent | ||||
Yes | 6.75 | 2.54-17.92 | <0.001 | 15.5 | 4.74-50.9 | <0.001 |