We read with interest the manuscript entitled, “The effect of micro-osteoperforations on the rate of orthodontic tooth movement: a systematic review and meta-analysis” by Sivarajan et al (Sivarajan S, Ringgingon LP, Fayed MMS, Wey MC. The effect of micro-osteoperforations on the rate of orthodontic tooth movement: a systematic review and meta-analysis. Am J Orthod Dentofacial Orthop 2020;157:290-304). After our careful and thorough reading of the work by Sivarajan et al, we are compelled to address significant flaws in their methodology that influence their conclusions. Equally important, we address the value of systematic review (SR) and meta-analysis (MA) to guide clinical decision-making in orthodontics and the need for rigorous peer review and high editorial board standards if SR and MA are to help improve orthodontics treatment.
Of the 1853 records that Sivarajan et al identified through their search strategy, 8 were selected for the SR using the Cochrane flowchart. Of these, 2 met the Preferred Reporting Items for Systematic Reviews and Meta-Analyses homogeneity criteria for the MA. One of the articles selected for the SR, but excluded from the MA, was our study. We found significant misrepresentations of our study that we believe biased its exclusion from the MA, which would alter the conclusion by Sivarajan et al that micro-osteoperforations (MOPs) do not accelerate orthodontic tooth movement.
Why are the misrepresentations of our study (detailed below) a problem? SR and MA should generate evidence-based conclusions regarding optimal treatment options for clinicians who do not have the time to read the overwhelming literature in their specialty, much less resolve conflicting data arising in that literature. The problem is that busy clinicians may not realize that there are authors who refuse to stray from the flowchart- and checklist-driven SR and MA protocols designed by Cochrane and Preferred Reporting Items for Systematic Reviews and Meta-Analyses to standardize these analyses. Mind you, authors are allowed to include a “non-standardized” study in the SR and/or MA, but this requires that they exercise scholarly due diligence and justify their decision to include such a study in the MA. In addition, it is important to note that regardless of the rating given for any risk of bias (ROB) category, the authors must justify the rating. Without this justification, the rating can only be interpreted as the authors’ opinion, thereby negating the evidence-based portion of the evidence-based conclusions for which SR and MA are valued.
Given this, we will detail the misrepresentations of our study in the SR and MR by Sivarajan et al on MOPs efficacy in accelerated orthodontic tooth movement.
In Table III, Sivarajan et al classified our study design as a split-mouth randomized clinical trial (RCT). This classification is not correct. Our study used both a split-mouth and parallel-group RCT design, which meant that we had 2 control groups. All first premolars were extracted in all subjects, and the rate of canine retraction ± MOPs was measured in each group. In the split-mouth group, MOPs were placed on 1 side of the arch but not on the other (our first control group), whereas the parallel-group had no MOPs (our second control group). This study design reduced our ROB by providing 2 controls against which we compared the rate of canine retraction ± MOPs. Because our data from the MOPs side of the split-mouth group differed significantly from both non-MOPs control groups, we justifiably concluded that MOPs significantly accelerates tooth movement. Equally importantly, because ours was the first clinical trial of MOPs, our 2 control groups verified our hypothesis that MOPs induce hyperlocalized inflammation allowing safe and faster movement of specific teeth and that the MOPs effect does not cross the midline.
Given our unusual study design, a truly inquisitive and ethical scientist should question why we included 2 control groups rather than 1. Was this a case of overkill or were we justified in designing our study in such an unusually rigorous way? Apparently, Sivarajan et al thought it was overkill because they did not mention the parallel-group portion of our study, thereby misleading readers to the true rigor of our study. Importantly, Sivarajan et al do not justify this omission.
Indeed, our reason for the dual-control study design can be found in Table V (“Assessment of ROB for the included RCTs”) of Sivarajan et al, in which they classify our “Other bias” as “Unclear.” They justify this rating by stating that our “study had a patent for the MOP device used in” the experiments. Any reasonable reader would immediately realize that it was precisely because our affiliate institution holds the patent and licensing rights to the MOP device (none of the authors hold the patent or license) that we took the extraordinary step of including 2 control groups. We wanted our manuscript reviewers and our readers to understand that we recognized the potential for “High” ROB, so we performed a rigorous double-control study. However, because the Cochrane ROB criteria do not explicitly deal with such an unusual study design, Sivarajan et al took the remarkably irresponsible step of ignoring our dual-control study design and gave us an “Unclear” rating for “Other bias.” We adamantly object to this classification and its justification, and we challenge Sivarajan et al—indeed, any reader—to provide explicit evidence for “Other bias” in our study that would warrant an “Unclear” rating for “Other bias.”
In Table V, Sivarajan et al also indicate that our Random Sequence Generation was “Unclear.” In our manuscript, we state that subjects “…were randomly assigned to one of the study groups. The experimental group received MOPs on either the right or left side. MOPs were randomly assigned to the patients’ left or right sides…” Therefore, there were 2 randomization steps in our study. As with establishing 2 well-designed control groups, our dual randomization protocol was designed to reduce or eliminate ROB and should have earned an unqualified “Low” ROB rating by Sivarajan et al. However, again, they chose to rate this category as “Unclear” because our study did not specifically list one of the randomization protocols from the Cochrane criteria.
Here is another instance in which we claim that Sivarajan et al failed in their scholarly due diligence. The issue of randomization is critical; after all, they don’t call them RCTs for nothing. So, if a reader is “Unclear” whether study subjects were adequately randomized, the first place you look is at the baseline for each group. There is no clearer sign of nonrandomization or failed randomization than having study groups that differ right from the beginning of the study. If Sivarajan et al were “Unclear” about our randomization, all they had to do was look at Table III (“Comparison of the morphological characteristics of the patients in the control and experimental groups”) in our manuscript. Of the 8 characteristics that we measured, not a single one was different between the groups. Did our randomization work? Yes. Therefore, we conclude that our “Random sequence generation” protocol earns a “Low” ROB rating.
A similar argument can be made for the rating of “Unclear” by Sivarajan et al for our “Allocation Concealment” ROB in Table V. We explicitly stated the following in our Materials and Methods section: (1) “Two orthodontic residents (M.R. and E.K.), trained and calibrated by the principal investigator (M.A.), were responsible for examining the subjects, determining their eligibility, and performing the orthodontic treatment under the supervision of a faculty member who was not the principal investigator .”; and (2) “The investigators performing the measurements and data analysis were blinded from the group assignments .” (emphasis added in each case). Once again, we went out of our way to ensure that subjects were assigned to groups without prior determination and that researchers were blinded as to who received what treatment to the best of our ability in such a study. Are we to believe that Sivarajan et al were unable to interpret these statements as worthy of a “Low” “Allocation Concealment” ROB rating? Intriguingly, Dr Sivarajan worked with other researchers to publish their own RCT on orthodontic tooth movement following MOPs. This study was included in the current SR and, like our study, was given an “Allocation Concealment” ROB of “Unclear.” We would like an explanation for how it is possible that Sivarajan et al cannot determine the “Allocation Concealment” ROB from their own study. In addition to our and their studies, Sivarajan et al also hedged their bets in this category on 2 other studies, which were ultimately eliminated from the MA because of “High” ROB in other categories. Intriguingly, all 4 of these studies showed that MOPs accelerate tooth movement.
What is especially disturbing about the “Unclear” “Allocation Concealment” rating given by Sivarajan et al—and is a clear violation of how these ratings are to be used—is that they do not give any justification for this decision for the “Unclear” rating for any of the 4 studies. The authors must justify their ratings; otherwise, their ROB ratings are subjective rather than objective. Unjustified subjectivity raises the ugly specter of confirmation bias. Because these ratings determine which studies will be included in the MA that will guide clinical practice, the lack of justification could warrant a call to have the manuscript retracted because of confirmation bias. Remember, there were only 2 SA studies that were given an “Overall bias” rating of “Unclear”—our study and the RCT by Sivarajan et al. Both of these studies found that MOPs increased the rate of orthodontic tooth movement. Combining these 2 studies with the 2 studies actually used in the MA would negate the conclusion by Sivarajan et al that “MOP does not accelerate tooth movement.” Because we can only question the methodology and conclusions by Sivarajan et al, and not their motivations, we can only caution clinicians to consider the wisdom of ignoring 75% (6 out of 8) of the SR studies demonstrating that MOPs accelerate tooth movement in favor of a poorly conducted MA using only 25% (2 out of 8) of the SR studies that say the opposite.
In conclusion, we also want to caution the American Journal of Orthodontics and Dentofacial Orthopedics’ Editorial Board and Reviewers that SR and MA studies require levels of scientific rigor that must go well beyond the fill-in-the-blank flowchart and checklists that encourage scholarly sloth. This is not the first time we find ourselves dismayed by the editorial decisions to publish anti-MOPs data and reward the authors with a coveted “Original Research” video of the month. Indeed, we took Alkebsi et al to task for concluding that MOPs do not accelerate tooth movement and detailed their sloppy experimental design, data collection, and data analysis that invalidates their conclusions. Mind you, the study by Alkebsi et al was 1 of the 2 studies included in the MA by Sivarajan et al. As with Sivarajan et al, Alkebsi et al were rewarded with an “Original Research” video of the month (June 2018). We recognize the need for, and welcome, dissenting studies regarding the efficacy of MOPs. However, these studies must be based on rigorous scientific and intellectual effort that leaves no room for doubt about the authors’ motives. We say the same to the American Journal of Orthodontics and Dentofacial Orthopedics’ Editorial Board and Reviewers, who risk being suspected of confirmation bias by enabling authors to publish subjective, unjustified opinions as original research.
References
Stay updated, free dental videos. Join our Telegram channel
VIDEdental - Online dental courses
