The Letter to the Editor to which we are responding refers to two papers published by our research team. In this response, ‘2007 paper’ refers to “Randomized effectiveness study of four therapeutic strategies for TMJ closed lock” (Journal of Dental Research), while ‘2014 paper’ refers to “Effects of four treatment strategies for temporomandibular joint closed lock” (International Journal of Oral and Maxillofacial Surgery).

Outcomes for the 2007 paper consisted of two indexes that provide standardized scores (continuous measures) for temporomandibular disorder (TMD) severity. In contrast, the 2014 paper used post-treatment success criteria recommended at the 1992 International Association of Oral and Maxillofacial Surgeons (IAOMS) conference, described in the Letter to the Editor. These criteria are dichotomous measures determining whether and when a patient achieves the IAOMS-recommended definition of a successful outcome. Non-surgical and surgical outcomes did not differ using either definition of success.

The 2007 report was based on a randomized clinical trial design for which the primary analytical method was a one-way analysis of variance (ANOVA) comparison of the four treatment strategies at each of six follow-up times. With statistical power to detect between-treatment differences much smaller than is considered clinically meaningful, the results in the 2007 paper showed no short-term or long-term differences between treatments for the Craniomandibular Index (CMI; shown in Table 3 of the 2007 paper), or for the Symptom Severity Index (SSI; shown in Table 4 the 2007 paper). These continuous outcome scores allowed additional exploratory analyses. Our areas of disagreement with the author of the Letter to the Editor result from differences in how we interpret these exploratory analyses.

The exploratory analyses included a series of adjusted analyses comparing medical management with rehabilitation, medical management with arthroscopy, and medical management with arthroplasty at each of six follow-up times. These contrasts were performed for both the CMI and the SSI resulting in a total of 36 (3 × 6 × 2) contrasts. Among these 36 analyses, we found and reported a single contrast (the SSI for medical management vs. arthroplasty) at the 6-month follow-up that was statistically significant only if we did not adjust the significance test’s false-positive rate (alpha), to avoid excessive false-positive findings because of the large number of comparisons considered. Given that one out of 20 contrasts will have P < 0.05 by chance alone, we do not agree that this rare occurrence establishes superiority of arthroplasty over medical management.

The Letter to the Editor further claims that a repeated measures analysis in the 2007 paper established superiority of arthroplasty over the other treatment strategies, because that analysis gave P = 0.03 for a difference between groups in time trend. However, this reading misinterprets our report: after we made the plainly unconfirmed assumption that the average outcomes in each treatment group followed a linear trend over time, our data would not confess to anything stronger than this paltry difference. Interpreted properly, this reinforces the conclusion of no treatment difference.

All these exploratory considerations cannot suffice to reject the clear results from the primary analysis that the treatment groups do not differ at any follow-up. The reader will note that the maximum observed difference between treatment groups at any time point was ≤0.09 for the CMI (shown in Table 3 of the 2007 paper) and ≤0.16 for the SSI (shown in Table 4 of the 2007 paper), both 30% less than the minimum clinically meaningful between-group differences of 0.14 (CMI) and 0.24 (SSI) that this study was powered to detect.

We are happy for this opportunity to discuss the findings from our two papers, particularly in so far as we hope to avoid any misunderstanding of the evidence we presented in those papers.