I read with interest the article “Prevalence of peg-shaped maxillary permanent lateral incisors: A meta-analysis” (Hua F, He H, Ngan P, Bouzid W. Am J Orthod Dentofacial Orthop 2013;144:97-109); although I congratulate the authors for their effort, I have some objections about the methods used.

The choice between fixed-effect (not “fixed-effects”) and random-effects models is made based on the P value of Cochran’s Q test for heterogeneity. This misconception is often made and is fundamentally wrong: “The choice between a fixed-effect and a random-effects meta-analysis should never be made on the basis of a statistical test for heterogeneity.” The reasons, in short, are (1) this choice should be made in the protocol stage, (2) based on both clinical and statistical reasoning, and, (3) most importantly, the 2 models do not do the same thing! Under the fixed-effect assumption, between-study differences are solely due to random error, whereas under the random-effects, they are due to population or setting differences. The interpretation of random-effects meta-analysis (average intervention effect across studies) differs from that of fixed-effect meta-analysis (best estimate of the intervention effect) and should be accompanied by 95% prediction intervals. Using for some patient groups a fixed-effect model and for others a random-effects model is just absurd.

Subgroup analyses are reported in the text and forest plots, whose methodology is not described, nor is it stated whether they were planned or done post hoc (risk of data dredging). We can assume (Fig 4) that the Partitioning Q method was used (aka, fixed-effect meta-regression), which is very likely to produce seriously misleading results. More appropriate would be a mixed-effects or random-effects approach or Bucher’s method. Therefore, I am skeptical about whether the reported differences between subgroups really exist.

Small-study effects and publication bias were not assessed for all outcomes, but only for the overall prevalence of peg laterals (by principle, a 1-sided measure without comparison). For the sex comparison of peg laterals’ prevalence, the funnel plot (not shown) indicates effect overestimation by small studies and asymmetry; this is confirmed by Egger’s test ( P = 0.028).

To conclude, proper understanding of the methods used is warranted to avoid the risk of producing numbers just for the sake of it and possibly misleading readers. The AJO-DO is the top orthodontic journal, and the amount of work put by the editorial staff into safeguarding the level of published evidence is unique among dental journals. I don’t think this article reflects the high quality of evidence-based articles we are used to seeing in the AJO-DO , and I find it unfortunate that it coincides with the editorial about the Cochrane Collaboration’s 20th anniversary.