Since systematic reviews are considered of high didactic value because of their evidence-based potential, we read with interest the article on self-ligating brackets in the June issue (Chen SSH, Greenlee GM, Kim JE, Smith CL, Huang GJ. Systematic review of self-ligating brackets. Am J Orthod Dentofacial Orthop 2010;137:726.e1-18), and we wish to note the following.

Randomized controlled trials (RCTs) are considered indispensable when addressing questions regarding therapeutic efficacy for reasons relating to the risk of bias. The extent and even the direction of bias in nonrandomized studies (NRS) are often impossible to predict. More rigorous studies are more likely to yield results that are closer to the truth. Even if only a few RCTs can be identified, it is strongly recommended that evidence from RCTs and NRSs should not be combined.

Besides randomization, other domains have been shown to be associated with risk of bias, and the use of scales is 1 approach used to summarize it. The authors of this systematic review correctly acknowledged that “All quality ratings have limitations,” when they introduced a relative scale to judge the quality of the studies included. The calculation of a summary score also involves assigning “weights” to different items in the scale, although it is often difficult to justify the weights assigned. Moreover, such approaches for assessing quality or risk of bias are not supported by empirical evidence. Dealing with studies considered to have a high risk of bias could involve the presentation of 1 analysis including all studies and 1 with only those at low risk of bias. Meta-analysis of results from studies of variable risks of bias can result in false positive or negative conclusions.

Taking these remarks into consideration, we use as an example the comparison 2.4. (change of incisor inclination) of Figure 3 to note how results might differ when a different methodology logic is applied. Omission from the analysis of the study by Jiang and Fu, which the authors considered to be of high risk of bias, would produce a still statistically significant result in favor of self-ligating brackets, but now I ² changes to 46%. The fixed-effect analysis assumes that there is no heterogeneity, but now I ² becomes moderately large, and there is some evidence of heterogeneity. Under these circumstances, one might opt for a random-effects analysis that would produce a result that is no more statistically significant in favor of self-ligating brackets contrary to one of the only 2 conclusions that “slightly less incisor proclination” appears to be associated with self-ligating brackets.